Inversion of the Heat Equation by a Block Based Algorithm Using Spline Wavelet Packets

Abstract

We present a robust algorithm starting from 1D or 2D discrete noised data to approximately invert the heat equation, which is an ill-conditioned problem. Relative contributions of the coherent structure and the noise in different frequency bands of the available data are different. We propose to solve the inversion problem separately in different frequency bands by methods similar to the Tikhonov regularization. This separation is achieved by using spline wavelet packets. The solutions are derived as linear combinations of those wavelet packets.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

The problems are formulated as follows: Let the functions f(x)∈C ²(ℝ¹) and f(x,y)∈C ²(ℝ²) be compactly supported. Denote by $\mathbf{U}^{1}_{t}$ and by $\mathbf{U}^{2}_{t}$ the linear operators such that $\mathbf{U}^{1}_{t}f(x)= g(x,t)$ and $\mathbf{U}^{2}_{t}f(x,y)= g(x,y,t)$ where g(x,t) and g(x,y,t) are the solutions of the heat equations with the initial conditions f(x) and f(x,y), respectively:

$$ \everymath{\displaystyle} \begin{array}{rl@{\qquad }l} \frac{\partial g(x,t)}{\partial t} & = g''_{x}(x,t), & g(x,0)=f(x), \\[12pt] \frac{\partial g(x,y,t)}{\partial t} & = g''_{x}(x,y,t)+ g''_{y}(x,y,t), & g(x,y,0)=f(x,y). \end{array} $$

(12.1)

Problem 12.1

Let t be a fixed time parameter. Given $g(x,t)=\mathbf{U}^{1}_{t}f(x)$, find f(x).

Problem 12.2

Let t be a fixed time parameter. Given $g(x,y,t)=\mathbf{U}^{2}_{t}f(x,y)$, find f(x,y).

For brevity, we concentrate on Problem 12.1. Extension to the 2D case is straightforward. The problem has explicit theoretical solutions [6]. We assume that the initial temperature distribution f(x) is a T-periodic function. Consequently, $g(x,t)=\mathbf{U}^{1}_{t}f(x)$ is T-periodic as well. These functions can be expanded into the following Fourier series:

$$f(x) = \frac{1}{T} \sum_{n\in \mathbb {Z}} f_n e^{2\pi inx/T},\qquad g(x,t) = \frac{1}{T} \sum _{n\in \mathbb {Z}} g_n(t) e^{2\pi inx/T}, \qquad g_n(0)=f_n. $$

If we know the function g(x,t) at some fixed t then

$$ f_n = g_n(t) e^{t(2\pi n/T)^2} \Longrightarrow f(x) = \frac{1}{T} \sum _{n\in \mathbb {Z}} g_n(t) e^{t(2\pi n/T)^{2}} e^{2\pi inx/T}. $$

(12.2)

In real life, the function g(x,t) is known up to some errors, modeled as $\tilde{g}(x,t)={g}(x,t)+{e}(x)$. Generally, there is no reason to assume that the Fourier coefficients of the error tend to zero faster than $e^{-n^{2}t}$ (if they tend to zero at all). Therefore, according to (12.2), application of the direct inversion to the available data $\tilde{g}(x,t)$ results in an unstable solution. However, as the magnitude of the error function |e(x)| becomes smaller, the function $\tilde{f}(x)$ can comprise strong high-frequency components, which do not exist in the original function f(x). Therefore, a regularization, which provides a stability to the solution at the expense of deviation from the available data $\tilde{g}(x,t)$, is needed.

Typically, the data function g(x,t) is sampled on a grid {x[k]} and the samples are corrupted by a random noise, while the sought-after initial temperature distribution f(x) is continuous. Therefore, it is reasonable to design approximated solutions as splines. Splines bridge the gap between the discrete input data and the continuous solution. To take into account different relative shares of the coherent signal and the noise in different frequency components of the available data, we propose to solve the inversion problem separately in different frequency bands. This approach significantly extends the adaptation abilities and the robustness of the method. Practically, this scheme is implemented via the application of the orthonormal spline wavelet packets, which are constructed by using the Spline Harmonic Analysis (SHA) framework. The wavelet packet transform splits the frequency domain of a signal into a set of bands whose overlap is minimal.

2 Elements of SHA

We briefly outline the basics of the SHA techniques. A detailed description is given, for example, in [2, 3].

We assume that N=2^j, j∈ℕ and p=2r>0 is an even integer. The space of N-periodic splines of even order p=2r, which have nodes on the grid {k}, is denoted by . A basis in is formed by translations of the centered periodic B-spline B ^p(x):

The B-spline B ^p(x) belongs to the space C ^p−2. The circular convolution . Thus, the circular convolution of two periodic splines is a spline. Therefore, splines are a proper tool for dealing with convolution-type problems where inversion of the heat equation belongs to.

There exist orthogonal bases in which resemble the Fourier basis. Denote $\omega\stackrel{\mathrm{def}}{=} e^{2\pi i/N}$. The orthogonal basis of the space is constituted by exponential splines

$$\beta^{p}[n](x)\stackrel{\mathrm{def}}{=} \sum _{k=0}^{N} \omega^{-nk} B^{p}(x+k), \quad n=0,\ldots,N-1. $$

The following representation holds:

$$S^{p}(x) = \sum_{k=0}^{N-1}q[k] B^{p}(x-k) = \frac{1}{N} \sum_{n=0}^{N-1} \widehat{q}[n] \beta^{p}[n](x). $$

Here $\widehat{q}[n]=\sum_{k=0}^{N} \omega^{-nk} q[k]$ is the discrete Fourier transform (DFT) of the coefficients {q[k]}. For further use, we single out the sequence

$$ {u}^{p}[n] \stackrel{\mathrm{def}}{=} \beta^{p}[n](0)= \sum_{k=0}^{N-1} \omega^{-nk} B^p(k), $$

(12.3)

which is the DFT of the sampled B-spline. The sequences u ^p[n] are N-periodic and strictly positive. The norms of the exponential splines are $\|\beta^{p}[n] \| =\sqrt{N u^{2p}[n]}$. Thus, the splines,

$$\gamma^{p}[n](x)\stackrel{\mathrm{def}}{=} \frac{\beta^{p}[n](x)}{\|\beta^{p}[n]\|} = \frac{\beta^{p}[n](x)}{\sqrt{N u^{2p}[n]}},\quad n=0,\ldots,N-1, $$

form an orthonormal basis of . The spline is represented by

$$ S^{p}(x) = \sqrt{ \frac{ 1}{N}} \sum_{n=0}^{N-1} \sigma[n] \gamma^{p}[n](x),\quad\sigma[n]=\sqrt{ u[n]^{2p}} \hat{q}[n]. $$

(12.4)

This expansion imposes a specific form of the SHA methodology onto the spline space, where the splines $\{\gamma^{p}[n](x) \}_{n=0}^{N-1}$ act as harmonics and the coordinates {σ[n]}, n=0,…,N−1, which we refer to as to the SHA spectrum of the spline S ^p(x). They act as the Fourier coefficients. Many operations on splines are significantly simplified [2, 3]. Denote by δ ² the second central difference: δ ² f(x)=f(x+1)−2f(x)+f(x−1) and

$$ w[n]\stackrel{\mathrm{def}}{=} 4\sin^{2} \frac{\pi n}{M}, \qquad W[n]\stackrel{\mathrm{def}}{=} \sqrt{ \frac{u^{2(p-2)}[n]}{u^{2p}[n]}}w[n], \qquad V[n]\stackrel{\mathrm{def}}{=} \frac{u^{p}[n]}{\sqrt{u^{2p}[n]}}. $$

(12.5)

Then

(12.6)

(12.7)

It follows from (12.6) that, if a spline S ^p(x) interpolates a sequence y={y[k]} at grid points S ^p(k)=y[k], then its SHA spectrum is

$$ \sigma[n] = \frac{\hat{y}[n]}{V[n] }, \quad\hat{y}[n] = \sum _{k=0}^{N-1}\omega^{-kn} y[k] . $$

(12.8)

3 Global Regularized Spline Solution

We briefly outline the scheme for global solution, which is a realization of the Tikhonov regularization algorithm [7] in the space of periodic splines. A full presentation of the scheme is given in [1].

To immerse Problem 12.1 into the spline setting, V _t denotes the linear operator defined on the spline space such that V _t S(x)=s(x,t), where s(x,t) is the spline solution to the difference approximation of the heat equation

$$ \frac{\partial s(x,t)}{\partial t}=\delta^{2}_{x}s(x,t), \qquad s(x,0)=S(x) $$

(12.9)

from (with respect to x).

Assume the spline S ^p(x) is represented by (12.4). The spline can be represented as

$$s(x,t)={N}^{-1/2}\sum_{n=0}^{N-1} \sigma[n](t) \gamma^{p}[n](x). $$

Using (12.6), we get

$$ \sigma[n](t)=\eta[n](t)\sigma[n], $$

(12.10)

where η[n](t)=e ^−w[n]t. If we know the spline s(x,t) by a fixed t then the spline S(x) becomes

$$S(x)=\sqrt{\frac{1}{N}}\sum_{n=0}^{N-1}e^{w[n]t} \sigma[n](t) \gamma^{p}[n](x). $$

However, typically only the data vector z={z[k]}, k=0,…,N−1, is known, where z[k]=g(k,t)+e _k , e={e _k } are the measurement errors, which we assume to be white noise.

The approximated solution to Problem 12.1 is derived as a spline

where

$$\begin{aligned} I(S) & \stackrel{\mathrm{def}}{=} \bigl\|S''\bigr\|^2 = \frac{1}{N}\sum _{n=0}^{N-1} \bigl|W[n]\sigma[n]\bigr|^2, \\ E(S) & \stackrel{\mathrm{def}}{=} \sum_{k} \bigl( \mathbf {V}_{t}S(k)-z[k] \bigr)^{2}=\frac{1}{N}\sum _{n=0}^{N-1} \bigl \vert V[n] \sigma[n] \eta[n](t)-\hat{z}[n]\bigr \vert ^2 . \end{aligned} $$

A spline solution to the minimization problem is

$$S_{\rho}(x) = \sqrt{ \frac{1}{N}}\sum_{n=0}^{N-1} \sigma[n](\rho) \gamma^{p}[n](x), \quad\sigma[n](\rho) = \frac{\overline{\eta}[n](t)\hat{z}[n] V[n]}{A[n](\rho)}, $$

where $\overline{\eta}[n](t)$ is the complex conjugate of η[n](t), $A[n](\rho)\stackrel{\mathrm{def}}{=} \rho W[n] + (\eta[n](t) V[n] )^{2}$.

Assume we are able to estimate the variance var(e)=ε ² of the error vector. The regularization parameter ρ is derived from the solution of the equation

$$ e(\rho)\stackrel{\mathrm{def}}{=} E(S_{\rho})/N = \frac{1}{N^{2}} \sum_{n=0}^{N-1} \biggl(\frac{\rho W[n]|\hat{z}[n]|}{A[n](\rho)} \biggr)^{2}=\varepsilon^{2}. $$

(12.11)

The function e(ρ) grows strictly monotonically as ρ→∞ and lim _ρ→∞ e(ρ)=N ⁻²∥z∥². If N ⁻¹∥z∥>ε, then (12.11) has a unique solution.

The parameter ρ, which provides a trade-off between approximation and regularization, depends on the relative shares of the coherent signal and the noise in the available data. These shares are different in different frequency components of the data. We propose to solve the problems separately in different frequency bands, while the regularization parameters are to be found according to the signal-to-noise ratio in each band. It is achieved by the application of the orthonormal spline wavelet packet transform, which splits the frequency domain of a signal into a set of bands whose overlap is minimal. The SHA framework provides tools for the design of wavelet packets and for the efficient implementation of the algorithm.

4 Wavelet Packets

Denote by , r∈ℕ, the space of N-periodic splines of even order p on the grid {2^r k}. In the rest of the paper $N_{r}\stackrel{\mathrm{def}}{=} N/2^{r}$, $n_{r}\stackrel{\mathrm{def}}{=} n+{N_{r}}/{2}$. The space is an N _r -dimensional space, where a basis consists of 2^r-sample shifts of the B-splines $B^{p}_{r}$ constructed on the grid {2^r k}. The inclusion relations between the spaces hold. Similarly to the space , the orthogonal and orthonormal bases of are formed by the exponential splines

For the initial scale, we retain the notations $\gamma^{p}[n]\equiv\gamma^{p}_{0,0}[n](x)$, $u^{p}[n]\equiv{u}_{0}^{p}[n]$.

When it will not produce a confusion, we drop the order index ⋅^p.

The two-scale relation between basis splines from adjacent spaces holds to be

$$ \gamma_{r,0}[n](x)= b_{r-1}[n] \gamma_{r-1,0}[n](x)+ b_{r-1} [n_{r}] \gamma_{r-1,0}[n_{r}](x), $$

(12.12)

where

$$b_{r-1}[n] \stackrel{\mathrm{def}}{=} \sqrt{ \frac{u_{r-1}^{2p}[n]}{2u_{r}^{2p}[n]}} \cos^{p} \biggl(\frac{2^{r-1}\pi n}{N} \biggr), \quad n_{r}\stackrel{\mathrm{def}}{=} n+\frac{N_{r}}{2}. $$

Denote by the orthogonal complement to in the space . An orthonormal basis in contains the splines

$$ \gamma_{r,1}[n](x)= \tilde{b}_{r-1}[n] \gamma_{r-1,0}[n](x)+ \omega^{2^{r-1}n}_{r-1} [n_{r}]\gamma_{r-1,0}[n_{r}](x), $$

(12.13)

where $\tilde{b}_{r-1}[n]\stackrel{\mathrm{def}}{=} \omega^{2^{r-1}n}b_{r-1} [n_{r}]$. If r>1, we can apply a similar procedure to the space . As a result, we get the decomposition . By applying the same procedure to all the derived subspaces, we decompose the spline space into a series of orthogonal sums

The orthonormal bases {γ _r,l [n](x)}[n] of the subspaces are derived iteratively by the two-scale relations using the coefficients b _r−1[n] and $\tilde{b}_{r-1}[n]$.

Similarly to the Fourier exponentials, the exponential basis splines γ _r,l [n](x) are complex-valued and are not localized in the space domain. However, their real-valued and well-localized counterparts satisfy

(12.14)

These splines are called the spline wavelet packets. The shifts {ψ _r,l (x−2^r k)}, k=0,…,N _r −1, form an orthonormal basis for the space . Consequently, the union $\biguplus_{l=0}^{2^{r}-1} \{\psi_{r,l} (x-2^{r}k ) \}$ forms an orthonormal basis for the entire space .

At the initial scale, the one-sample shifts of the splines

$$\varphi^{p}(x)\stackrel{\mathrm{def}}{=} \psi^{p}_{0,0}(x) = N^{-1/2} \sum_{n=0}^{N-1} \gamma^{p}[n](x) $$

form an orthonormal basis.

All the spaces belong to , thus, the wavelet packet ψ _r,l (x) forms a subspace and can be expanded over the orthonormal basis {γ ^p[n](x)} of :

$$ \psi_{r,l}(x) = \sqrt{ \frac{1}{N}} \sum_{n=0}^{N-1}{ \nu}_{r,l}[n]\gamma^{p}[n](x). $$

(12.15)

The SHA spectra $\{{\nu}_{r,l}[n] \}_{n=0}^{N-1}$ of the wavelet can be explicitly calculated using the two-scale relations.

Example 12.1

(The first decomposition scale r=1)

The SHA spectra are

Example 12.2

(The second decomposition scale r=2)

The SHA spectra are

Figure 12.1 displays the wavelet packets from the first and the second decomposition scales with their SHA spectra.

Fig. 12.1

Wavelet packets of order 8 from the 1st (top) the 2nd decomposition scales with their SHA spectra (bottom half-band)

The wavelet packets are well localized in space. Their SHA spectra have a near rectangular shape (the higher the spline order is the closer the shape is to rectangular) and produce a sequence of partitions of the frequency band. The SHA spectrum of the wavelet packet ψ _r,l (x) is effectively confined within the band

$$ \varLambda_{r,l}\stackrel{ \mathrm{def}}{=} \biggl[-\frac{(l+1)N}{2^{r+1}}, -\frac{(l)N}{2^{r+1}} \biggr]\cup \biggl[\frac{(l)N}{2^{r+1}}, \frac{(l+1)N}{2^{r+1}} \biggr], \quad l=0,\ldots,2^{r}-1. $$

(12.16)

Since a spline from is the linear combination of the wavelet packets

$$ S_{r,l}(x)= \sum_{k=0}^{N_{r}-1} q_{r,l}[k] \psi^{p}_{r,l} \bigl(x-2^{r}k \bigr), $$

(12.17)

then its SHA spectrum is effectively confined within the band Λ _r,l . This provides opportunities to approximate the heat inversion separately in different frequency bands.

5 Spline Wavelet Packet Transforms

Let a spline be represented by the orthonormal basis splines

$$ S(x)=\sqrt{\frac{1}{N}}\sum _{n=-N/2}^{N/2-1} \sigma[n]\gamma^{p}[n](x). $$

(12.18)

The sequence {σ[n]}, n=−N/2,…,N/2−1, is the SHA spectrum of the spline S(x). The space is the orthogonal sum of the subspaces and whose orthonormal bases are $\{\gamma^{p}_{1,0}[n](x)\}$ and $\{\gamma^{p}_{1,1}[n](x)\}$, respectively, where n=0,…,N ₁−1 and N ₁=N/2. Thus, S(x) can be represented as the sum of its orthogonal projections onto the subspaces , i=0,1: S(x)=S _1,0(x)⊕S _1,1(x), where

$$ S_{1,i}(x) \stackrel{\mathrm{def}}{=}\sqrt{\frac{2}{N}} \sum _{n=0}^{N/2-1} \sigma_{1}[n] \gamma^{p}_{1,i}[n](x), \quad i=0,1. $$

(12.19)

The orthonormality of the spline basis implies

$$ \sigma[n]= \sqrt{N} \bigl\langle S,\gamma^{p}[n] \bigr\rangle, \qquad \sigma_{1,i}[n]= \sqrt{N/2} \bigl\langle S,\gamma^{p}_{1,i}[n] \bigr\rangle,\quad i=0,1. $$

(12.20)

By using the two-scale relations given by (12.12) and (12.13), we derive for n=0,…,N/2−1

(12.21)

(12.22)

We can present (12.21) and (12.22) in a matrix form

$$ \begin{pmatrix} \sigma_{1,0}[n] \\ \sigma_{1,1}[n] \end{pmatrix} = \sqrt{ \frac{1}{2}}\mathbf{A}_{0}[n]\cdot \begin{pmatrix} \sigma[n] \\ \sigma[n_{1}] \end{pmatrix} , \qquad\mathbf{A}_{m}[n]\stackrel{\mathrm{def}}{=} \begin{pmatrix} b_{m}[n] & b_{m}[n_{m+1}] \\ \tilde{b}_{m}[n] & \tilde{b}_{m}[n_{m+1}] \\ \end{pmatrix} . $$

(12.23)

The coordinates of the projections of S(x) onto the subspaces are derived iteratively:

$$ \everymath{\displaystyle} \begin{array}{l@{\quad }l} \begin{pmatrix} \sigma_{m,2l}[n] \\ \sigma_{m,2l+1}[n] \end{pmatrix} = \sqrt{\frac{1}{2}}\mathbf{A}_{m-1}[n] \cdot \begin{pmatrix} \sigma_{m-1,l}[n] \\ \sigma_{m-1,1}[n+N_{r}] \end{pmatrix} & \mbox{if } l \mbox{ is even}, \\ \begin{pmatrix} \sigma_{m,2l+1}[n] \\ \sigma_{m,2l}[n] \end{pmatrix} = \sqrt{\frac{1}{2}} \mathbf{A}_{m-1}[n]\cdot \begin{pmatrix} \sigma_{m-1,l}[n] \\ \sigma_{m-1,1}[n+N_{r}] \end{pmatrix} & \mbox{if } l \mbox{ is odd}. \end{array} $$

(12.24)

6 Wavelet Packet Bases

Assume that the spline is expanded over the orthonormal bases

$$\begin{gathered} S(x) = \frac{1}{\sqrt{N}}\sum_{n=0}^{N-1} \sigma[n] \gamma^{p}[n](x)=\sum_{k=0}^{N-1}q[k] \varphi^{p}(x-k), \\ \sigma[n]=\sum_{k=0}^{N-1} \omega^{-nk} q[k]=\hat{q}[n], \quad q[k] = \frac{1}{N} \sum _{n=0}^{N-1} \omega^{nk} \sigma[n]. \end{gathered} $$

For example, if the samples S(k)=y[k], k=0,…,N−1, are available then (12.8) claims that $\sigma[n]={\hat{y}[n]}/{V[n] }$. Iterative application of the transform given by (12.24) expands the projections splines over the orthonormal bases {γ _r,l [n](x)}. Then, the coordinates {q _r,l [k]} of the alternative expansion (12.17) over the orthonormal wavelet packet bases $\{\psi^{p}_{r,l}(x-2^{r}k)\}$ are derived by the application of the IDFT: $q_{r,l}[k] = N_{r}^{-1} \sum_{n=0}^{N_{r}-1}\omega^{2^{r}nk}\sigma_{r,l}[n]$. The subspace . The spline S _r−1,l (x) can be expanded either over the basis $\{\psi^{p}_{r-1,l}(x-2^{r-1}k)\}_{k=0}^{N_{r-1}-1}$ or over the combined orthonormal basis $\{\psi^{p}_{r,2l}(x-2^{r}k)\} \biguplus\{\psi^{p}_{r,2l}(x-2^{r}k)\}_{k=0}^{N_{r}-1}$. The decision of which basis is preferable is made once a cost function is defined.

Consequently, once the wavelet packet transform of the spline S(x) is implemented, a wide variety of orthonormal wavelet packet bases becomes available. A basis, which is optimal for a given spline with respect to a certain purpose, can be designed by the Best Basis algorithm [4], which compares the cost function of the “parent” spline S _r−1,l (x) with the cost of the “offsprings” S _r,2l (x) and S _r,2l+1(x). Entropy us a typical cost function.

7 Parameterized Spline Solution in the Subspace

7.1 Splines from the Subspaces

The spline S _r,l (x), which is the orthogonal projection of the spline S(x) onto the subspace , is expanded over the orthonormal wavelet packet basis as in (12.17). On the other hand, the spline S _r,l (x) belongs to the initial space and can be expanded over the orthonormal basis of

$$ S_{r,l}(x)=\sqrt{\frac{1}{N}} \sum _{n=0}^{N-1} {\zeta}_{r,l}[n] \gamma^{p}[n](x), \quad\zeta_{r,l}[n] = {\nu}_{r,l}[n] \hat{q}_{r,l}[n]. $$

(12.25)

The coefficients ν _r,l [n] are the SHA spectrum of the wavelet packet ${\psi}^{p}_{r,l}$. We emphasize that the DFT sequence $\hat{q}_{r,l}[n]$ is N _r -periodic, where N _r =N/2^r.

The projection coordinates ζ _r,l [n] can be expressed via the coordinates σ[n] of the spline S(x).

Proposition 12.1

The following representation of the projection coordinates holds:

$$ \zeta_{r,l}[n] = \frac{\nu_{r,l}[n]}{2^{r}} \sum_{\lambda=-2^{r}/2}^{2^{r}/2-1} \sigma[n+\lambda N_{r}] \overline{\nu}_{r,l}[n+\lambda N_{r}] \approx\frac{1}{2^{r}} \sigma[n] \bigl \vert {\nu}_{r,l}[n]\bigr \vert ^{2}. $$

(12.26)

Remark 12.1

The higher the order p is, the closer ζ _r,l [n] is to 2^−r σ[n]|ν _r,l [n]|².

Equation (12.9) implies that the application of the operator V _t to S _r,l (x) results in

$$ S_{r,l}(x,t) = \mathbf {V}_{t}S_{r,l}(x) = \frac{1}{\sqrt{N}} \sum_{n=0}^{N-1} \eta[n](t)\zeta_{r,l}[n] \gamma^{p}[n](x). $$

(12.27)

From (12.6), sampling of the spline S(x,t)=V _t S(x) becomes

$$y[k]\stackrel{\mathrm{def}}{=} S (k,t ) = \frac{1}{N} \sum _{n=0}^{N-1} \eta[n](t) \sigma[n] V[n] \omega^{kn}, \quad V[n]\stackrel{\mathrm{def}}{=} \frac{u^{p}[n]}{\sqrt{u^{2p}[n]}}, $$

while sampling of S _r,l (x,t)=V _t S _r,l (x)

$$ y_{r,l}[k]\stackrel{\mathrm{def}}{=} S_{r,l} (k,t ) = \frac{1}{N} \sum _{n=0}^{N-1} \eta[n](t) \zeta_{r,l}[n] V[n] \omega^{kn}. $$

(12.28)

Equation (12.26) implies the approximated relation

$$ \hat{ y}_{r,l}[n] \approx \frac{1}{2^{r}} \eta[n](t) \sigma[n] V[n] \bigl \vert {\nu}_{r,l}[n] \bigr \vert ^{2} = \frac{1}{2^{r}}\hat{y}[n] \bigl \vert { \nu}_{r,l}[n]\bigr \vert ^{2}. $$

(12.29)

Remark 12.2

Equation (12.29) can be interpreted in a sense that confinement of the operator’s V _t domain from the whole spline space to the subspace effectively results in multiplication of the DFT $\hat{y}[n]$ of the sampled output y[k]=V _t S(k) with the factor 2^−r|ν _r,l [n]|².

7.2 Parameterized Spline Solution

The scheme for a partial solution of Problem 12.1 in the subspace is very similar to the scheme of a global solution presented in Sect. 12.3. By assumption, t is a known time parameter and the vector z={z[k]=g(k,t)+e _k }=g+e is available, where g(x,t)=U _t f(x). Then, a partial approximated inversion of the heat equation (12.1) is derived as a spline

$$ S_{r,l}(x)=\sum _{k=0}^{N_{r}-1}q_{r,l}[k] {\psi}^{p}_{r,l} \bigl(x-2^{r}k\bigr) = \sqrt{\frac{1}{N}} \sum _{n=0}^{N-1}{\zeta}_{r,l}[n] \gamma^{p}[n](x) $$

(12.30)

such that the spline S _r,l (x,t)=V _t S _r,l (x) approximates, in some sense, the available discrete data z. To be specific, Remark 12.2 suggests that the sampled spline S _r,l (k,t) should approximate the “filtered”

$$ {\tilde{z}}_{r,l}[k]\stackrel{ \mathrm{def}}{=}\frac{1}{N}\sum_{n=0}^{N-1} \omega^{kn} \hat{\tilde{z}}_{r,l}[n], \quad\mbox{where } \hat{ \tilde{z}}_{r,l}[n]\stackrel{\mathrm{def}}{=}\hat{z}[n] \frac{\vert {\nu}_{r,l}[n]\vert ^{2}}{2^{r}}, $$

(12.31)

rather than the entire data z. Similarly to Sect. 12.3, we find a spline , which minimizes the functional ρI(S)+E _r,l (S), where

$$ I(S)\stackrel{\mathrm{def}}{=} \bigl\|(S)'' \bigr\|^{2}, \qquad E_{r,l}(S)\stackrel{\mathrm{def}}{=} \sum _{k} \bigl(S(k,t)-\tilde{z}_{r,l}[k] \bigr)^{2}, \qquad S(x,t)\stackrel{\mathrm{def}}{=} \mathbf { V}_{t}S(x), $$

(12.32)

and ρ is a numerical parameter.

Let a spline be represented as in (12.30)

$$s(x)=\sqrt{\frac{1}{N}} \sum_{n=0}^{N-1}{ \zeta}_{r,l}[n]\gamma^{p}[n](x). $$

Then,

$$I(s)= \frac{1}{N}\sum_{n=0}^{N-1} W[n] \bigl \vert \zeta_{r,l}[n]\bigr \vert ^2 , \qquad E_{r,l}(s) = \frac{1}{N} \sum_{n=0}^{N-1} \bigl \vert \eta[n](t) \zeta_{r,l}[n] V[n]-\hat{\tilde{z}}_{r,l}[n] \bigr \vert ^{2}. $$

The sequences W[n] and V[n] are defined in (12.5).

A solution to the minimization problem is the spline from

$$\begin{aligned} S_{r,l}(\rho,x) & = \sqrt{ \frac{1}{N}} \sum_{n=0}^{N-1}{ \zeta}_{r,l}(\rho)[n] \gamma^{p}[n](x), \qquad \zeta_{r,l}(\rho)[n]= \frac{\overline{\eta}[n](t) V[n] \hat{\tilde{z}}_{r,l}(n)}{A[n](\rho)}, \\ A[n](\rho) & \stackrel{\mathrm{def}}{=} \rho W[n]+\bigl \vert \eta[n](t) V[n] \bigr \vert ^{2}. \end{aligned} $$

Its samples on the grid points

$$ {S}_{r,l}(\rho,k) = \frac{1}{N} \sum_{n=0}^{N-1} \omega^{kn} \zeta_{r,l}(\rho)[n] V[n]= \frac{1}{N} \sum _{n=0}^{N-1} \omega^{kn} \frac{\overline{\eta}[n](t) V^{2}[n] \hat{\tilde{z}}_{r,l}(n)}{A[n](\rho)}. $$

(12.33)

7.3 Selection of the Regularization Parameter

Assume that we are able to evaluate the errors vector $\mathbf {e}=\{e_{k}\}_{k=0}^{N-1}$, $e_{k}={N}^{-1}\times\allowbreak \sum_{n=0}^{N-1}\omega^{kn}\hat{e}[n]$, whose variance $\mathrm{var}(\mathbf {e})=\varepsilon^{2} \approx N^{-1}\sum_{k=0}^{N-1} (e_{k})^{2}$. Keeping (12.31) in mind, denote

$$\begin{aligned} {e}_{r,l}[k] & \stackrel{\mathrm{def}}{=} \frac{1}{N} \sum_{n=0}^{N-1} \omega^{kn} \hat{e}_{r,l}[n], \quad\mbox{where } \hat{e}_{r,l}[n] \stackrel{\mathrm{def}}{=} \hat{e}[n] \frac{\vert {\nu }_{r,l}[n]\vert ^{2}}{2^{r}}, \\ (\varepsilon_{r,l} )^{2} & \stackrel{\mathrm{def}}{=} \sum _{k=0}^{N-1} \bigl(e_{r,l}[k] \bigr)^{2} = \frac{1}{N} \sum_{n=0}^{N-1} \bigl \vert \hat{e}_{r,l}[n]\bigr \vert ^{2}. \end{aligned} $$

The function

$$e_{r,l}(\rho) \stackrel{\mathrm{def}}{=} E_{r,l} \bigl({S}_{r,l}(\rho,\cdot) \bigr) = \frac{1}{N} \sum _{n=0}^{N-1} \biggl(\frac{\rho W[n]|\hat{\tilde{z}}_{r,l}[n]|}{A[n](\rho))} \biggr)^{2} $$

grows monotonically from zero to $N^{-1} \sum_{n=0}^{N-1} \vert \hat{\tilde{z}}_{r,l}[n]\vert ^{2} = \sum_{k=0}^{N-1} ({\tilde{z}}_{r,l}[k] )^{2}$ as ρ grows from zero to infinity. Therefore, we derive ρ _r,l from the equation e _r,l (ρ)=(ε _r,l )².

7.4 Modeling the Noise

We assume that the error vector e is a zero mean Gaussian white noise. It is seen from (12.2) that the Fourier coefficients of the function g(x,t)=U _t f(x): $g_{n}(t)=f_{n}e^{-t({2\pi n}/{N})^{2}}$ are fast decaying when n is growing. Thus, the function g(x,t) is efficiently bandlimited. Its significant Fourier coefficients g _n (t) occupy a relatively narrow band around zero, −K(t)<n<K(t), K(t)<N/2, K(t)→0 as t→∞. Hence, the DFT coefficients of the data vector z: $\{\hat{z}[n]\}\approx\{\hat{e}[n]\}$ as n∈[K(t),N/2−1]∪[−N/2,−K(t)]. By relying on the fact that the power spectrum $\{|\hat{e}[n]|^{2}\}$ of the white noise e is close to a constant for all n=−N/2,…,N/2−1, it is possible to evaluate the variance

$$ \sigma^{2} \approx \frac{1}{(N-K(t))^{2}} \sum_{n\in[K(t),N/2-1] \cup[-N/2,-K(t)]}\bigl \vert \hat{z}[n] \bigr \vert ^{2}. $$

(12.34)

Then, the noise vector e is modeled as a zero mean Gaussian random process $\tilde{\mathbf{e}}=\{\tilde{e}_{i}\}_{i=0}^{N-1}$, whose variance is σ ². Let $\{\widehat{\tilde{e}}[n]\}_{n=-N/2}^{N/2-1}$ be the DFT spectrum of the model vector $\tilde{\mathbf{e}}$. Then, the values (ε _r,l )², which are needed for the parameter ρ selection, are estimated as

$$ (\varepsilon_{r,l} )^{2}\approx\frac{1}{2^{r}N} \sum_{n=0}^{N-1} \bigl \vert \bigl(\nu_{r,l}[n]\bigr)^{2} \overline{\widehat{ \tilde{e}}}[n]\bigr \vert ^{2}. $$

(12.35)

Another option for the noise evaluation is to use the scheme in [5].

8 Spline Wavelet Packet Solution to Problem 12.1

The partial spline solution S _m,l (ρ,x) of the inversion problem in the subspace is derived from the filtered data such that the DFT is $\hat{\tilde{z}}_{m,l}[n]\stackrel{\mathrm{def}}{=}\hat{z}[n] {\vert {\nu}_{m,l}[n]\vert ^{2}}{2^{-m}}$.

To determine an optimal set of the subspaces , which reveal the internal structure of the data vector z, we construct the spline $Z(x)=\sum_{n=0}^{N-1}\xi[n]\gamma^{p}[n](x)$, $\xi[n]=\hat{z}[n]/V[n]$, which interpolates the data z. Then, we apply the Best Basis algorithm to obtain the list $PL= \{(\bar{p},\bar{l}) \}$ such that the shifts of the wavelet packets $\psi_{\bar{m},\bar{l}}$ form an optimal basis for the spline Z(x). The list PL determines the subspaces , where the partial solutions for the inversion problem are to be derived. Due to the effective bandlimitedness of the function $g(x,t)=\mathbf{U}^{1}_{t}f(x)$, some subspaces , which correspond to higher frequency bands are “empty” in a sense that they, actually, do not contain a contribution from the initial function f(x). Such subspaces are discarded from the list PL.

A scheme for the approximated inversion of the heat equation

• 1. Calculate the coefficients η[n](t) defined in (12.10).

• 2. Construct the data interpolating spline Z(x).

• 3. Implement the wavelet packet transform of order p of the spline Z(x).

• 4. Apply the Best Basis algorithm to the transform coefficients to collect the list PL of relevant subspaces.

• 5. Reduce the list PL to $\overline{{PL}}$ by discarding the “empty” subspaces.

• 6. Evaluate the error vector to estimate the partial variances (ε _p,l )², $(p,l)\in\overline{{PL}}$, of noise (see (12.35)).

• 7. Determine the optimal values $\rho_{\bar{m},\bar{l}}$ of the regularization parameter for each pair $({\bar{m},\bar{l}})\in \overline{PL}$.

• 8. Find the partial solutions for each pair $(\bar{m},\bar{l})\in\overline{PL}$ (see (12.33)).

The approximated solution to the inversion Problem 12.1 is

Extension of the algorithm to the 2D case is straightforward once the tensor products of the basis splines are utilized:

Figure 12.2 displays the SHA spectra of two wavelet packets of order 10 from the second scale. We observe that the spectra have near-parallelepiped shape. The described algorithm can be utilized for signal and image denoising when the time parameter t=0. In this case, the general scheme remains unchanged.

Fig. 12.2

The SHA spectra of wavelet packets of order 10 from the second resolution scale. Top: $\psi^{10}_{2,2,3}(x,y)$. Bottom: $\psi^{10}_{2,3,1}(x,y)$

9 Numerical Examples

The following are examples, derived from three groups of experiments, on using the block-based methods for 2D images’ restoration:

Denoising::

Restoration of objects corrupted by Gaussian noise (the time parameter t=0).

Pure blurred input::

Restoration of blurred objects when the time parameter t>0 and noise is not known. The advantage of the block based method over the global one materialized in the accurate tuning of the subspaces where the looked for solution is in the effective frequency domain of the blurred image.

Noised blurred input::

Restoration of objects from blurred inputs, which were corrupted by Gaussian noise.

These examples illustrate the difference between the performance of the global Tikhonov algorithm (GTA) presented in Sect. 12.3 and of the Best Basis Algorithm (BBA). Visual perception is compared and the peak-signal-to-noise-ratio (PSNR). Three benchmark images each of which is presented by a 512×512 array of samples are used as the initial temperature distributions. The source images for the experiments are shown in Fig. 12.3.

Fig. 12.3

Left: “Barbara”. Center: “Lena”. Right: Fingerprint

Example 12.3

(Barbara Denoising)

The “Barbara” image was corrupted by Gaussian zero-mean noise with standard deviations STD=25. The time parameter is t=0. Figure 12.4 displays fragments of the noised input image and of the image that was restored by the applications of GTA and BBA. We observe that BBA produces high PSNR values. The noise was suppressed almost completely. The GTA method did not succeed in noise suppression, although the texture is resolved a little bit better in comparison to BBA.

Fig. 12.4

“Barbara”. Left: A noised image, STD=25, PSNR=20.17. Center: An image restored by GTA, PSNR=24.12. Right: An image restored by BBA, PSNR=25.77, spline wavelet packets of the fourth order from 4 levels were used

Example 12.4

(Restoration of a Strongly Blurred Fingerprint)

In this example, the “Fingerprint” image was used as the initial temperature distribution. The input presents the temperature distribution when the time parameter was t=46. The BBA restored the texture of the fingerprint, which was completely smeared in the input. The result produced by GTA was much worse. The results are illustrated in Fig. 12.5.

Fig. 12.5

“Fingerprint”. Left: A blurred image, t=46, PSNR=15.71. Center: An image restored by GTA, PSNR=17.57. Right: An image restored by BBA, PSNR=20.22, spline wavelet packets of the fourth order from 3 levels were used

Example 12.5

(Restoration of Blurred and Noised “Lena”)

The “Lena” image was used as the initial temperature distribution. The input is the distribution when the time parameter was t=2.5 corrupted by Gaussian noise whose STD=10. The BBA-restored image is sharper compared to the GTA and its PSNR is higher. See Fig. 12.6 for the results.

Fig. 12.6

“Lena”. Left: A blurred noised image, t=2.5, noise STD=10, PSNR=24.79. Center: An image restored by GTA, PSNR=28.22. Right: An image restored by BBA, PSNR=28.47, spline wavelet packets of the fourth order from 4 levels were used