Metamorphosis

Abstract

This chapter introduces metamorphoses, which refer to Riemannian metrics based on transformations in which objects can change under the action of diffeomorphisms (or other transformation groups) but also under independent variations, allowing for more flexibility in the evolution.

12.1 Definitions

The Riemannian version of the construction of Sect. 11.1.2 provides a metric based on transformations in which objects can change under the action of diffeomorphisms but also under independent variations. We shall refer to such metrics as metamorphoses [152, 206, 281, 282]. They will result in formulations that enable both object registration and metric comparison.

We start with an abstract description of the construction. We consider the setting in which deformations belong to a Lie group G with Lie algebra denoted V, acting on a Riemannian manifold M. We assume that V is a Hilbert space with norm $\left\| \cdot \right\| _{V}$; the metric on M at a given point $a \in M$ is denoted ${\big \langle {\cdot }\, , \, {\cdot }\big \rangle }_a$ and the corresponding norm $\left| \cdot \right| _a$.

For $\varphi \in G$ and $a\in M$, define

(12.1)

The first two maps are the components of the action, and the third is the right translation on G. It will also be convenient, in the following, to have special notation for derivatives of these maps evaluated at the identity, so we will write $\xi _\varphi = dR_{\varphi }({\mathrm {id}})$, $\xi _a = dR_{a}({\mathrm {id}})$. These maps coincide with the infinitesimal actions, i.e.,

$$ v\cdot \varphi = \xi _\varphi v \text { and } v\cdot a = \xi _a v $$

and we will also use this notation.

If $(\varphi (t), t\in [0,1])$ is a differentiable curve on G, we define its Eulerian velocity v(t) (which is a curve in V) by the relation:

$$\begin{aligned} \partial _t \varphi = \xi _{\varphi (t)} v(t) = v(t)\cdot \varphi (t)\,. \end{aligned}$$

(12.2)

Definition 12.1

A metamorphosis is a pair of curves $(\varphi (t), \alpha (t))$ respectively on G and M, with $\varphi (0)= {\mathrm {id}}$. Its image is the curve a(t) on M defined by $a(t) = \varphi (t)\cdot \alpha (t)$. We will say that $\varphi (t)$ is the deformation part of the metamorphosis, and that $\alpha (t)$ is the residual part. When $\alpha (t)$ is constant, the metamorphosis is a pure deformation.

12.2 A New Metric on M

Metamorphoses, by the evolution of their images, provide a convenient representation of combinations of group actions and of variations on M. Let a metamorphosis $((\varphi (t), \alpha (t)), t\in [0,1])$ be given and $a(t) = \varphi (t)\cdot \alpha (t)$ be its image. Then, we can write

$$\begin{aligned} \partial _t a(t)= & {} dA_{\varphi (t)} (\alpha (t)) \partial _t \alpha (t) + dR_{\alpha (t)} (\varphi (t)) \partial _t\varphi \\= & {} dA_{\varphi (t)} (\alpha (t)) \partial _t \alpha (t) + dR_{\alpha (t)} (\varphi (t)) dR_{\varphi (t)}({\mathrm {id}}) v(t). \end{aligned}$$

Since $R_\alpha \circ R_\varphi = R_{\varphi \alpha }$, we get

$$\begin{aligned} \partial _t a(t) = dA_{\varphi (t)} (\alpha (t)) \partial _t \alpha (t) + dR_{a(t)} ({\mathrm {id}}) v(t). \end{aligned}$$

(12.3)

In particular, when $t=0$:

$$\begin{aligned} \partial _t a(0) = \partial _t \alpha (0) + v(0)\cdot a(0). \end{aligned}$$

(12.4)

This expression provides a decomposition of a generic element $\eta \in T_aM$ in terms of an infinitesimal metamorphosis, represented by an element of $V \times T_aM$. Indeed, for $a\in M$, introduce the map

Then (12.4) can be written as

$$ \partial _t a(0) = \varPhi ^{(a_0)}\left( \partial _t \alpha (0), v(0)\right) . $$

We now introduce the Riemannian metric associated to metamorphoses.

Proposition 12.2

Assume that $v \mapsto v\cdot a$ is continuous on V. With $\sigma ^2 > 0$, the norm

$$\begin{aligned} \left\| \eta \right\| ^2_a = \inf \left\{ \left\| v \right\| ^2_V + \frac{1}{\sigma ^2} \left| \rho \right| _a^2 : \eta =\varPhi ^{(a)}(v,\rho ) \right\} \end{aligned}$$

(12.5)

defines a new Riemannian metric on M. (Note that we are using double bars instead of single ones to distinguish between the new metric on M and the initial one.)

Proof

Because $\varPhi ^{(a)}$ is onto (e.g., $\varPhi ^{(a)}(0, \rho ) = \rho$), $\Vert \eta \Vert _a$ is finite (and bounded by $|\eta |_a/\sigma ^2$). Define $V_a= (\varPhi ^{(a)})^{-1}(0)$. It is a linear subspace of $V \times T_aM$ and $\left\| \eta \right\| ^2_a$ is the norm of the linear projection of $(0, \eta )$ on $V_a$, for the Hilbert structure on $V \times T_aM$ defined by

$$ \left\| (v,\rho ) \right\| _{\mathrm {id}, a}^2 = \left\| v \right\| _V^2 + \frac{1}{\sigma ^2} \left| \rho \right| ^2_a.$$

Thus $\left\| \cdot \right\| _a = \left\| \pi _{V_a}(0, \eta ) \right\| _{\mathrm {id}, a}$ is associated with an inner product. Since it is a projection on a closed subspace ($\varPhi ^{(a)}$ is continuous), the infimum is attained and by definition, cannot vanish unless $\eta =0$. This therefore provides a new Riemannian metric on M.    $\square$

With this metric, the energy of a curve is

$$\begin{aligned} E(a(t))&= \int _0^1 \left\| \partial _t a(t) \right\| _{a(t)}^2 dt \nonumber \\&= \inf _v \left( \int _0^1 \left\| v(t) \right\| _{V}^2 dt + \frac{1}{\sigma ^2} \int _0^1 \left| \partial _t a(t) - v(t)\cdot a(t) \right| ^2_{a(t)} dt\right) , \end{aligned}$$

(12.6)

the infimum being over all curves $t\mapsto v(t)$ on V. It can also be written

$$\begin{aligned} E(a(t)) = \inf _v \left( \int _0^1 \left\| v(t) \right\| _{V}^2 dt + \frac{1}{\sigma ^2} \int _0^1 \left| dA_{\varphi (t)}(\alpha (t))\, \partial _t \alpha (t) \right| ^2_{a(t)} dt\right) \end{aligned}$$

(12.7)

with $\partial _t\varphi = v\circ \varphi$.

The distance between two elements $a_0$ and $a_1$ in M can therefore be computed by minimizing

$$\begin{aligned} U(v, a) = \int _0^1 \left\| v(t) \right\| _{V}^2 dt + \frac{1}{\sigma ^2} \int _0^1 \left| \partial _t a(t) - v(t)\cdot a(t) \right| ^2_{a(t)} dt \end{aligned}$$

(12.8)

over all curves $((v(t), a(t)), t\in [0,1])$ on $V \times M$, with boundary conditions $a(0) = a_0$ and $a(1) = a_1$ (no condition on v). From (12.7), this may also be seen as finding an optimal metamorphosis, by minimizing

$$ {\tilde{U}}(\varphi , \alpha ) = \int _0^1 \left\| (dR_{\varphi (t)}({\mathrm {id}}))^{-1} \partial _t \varphi (t) \right\| _{V}^2 dt + \frac{1}{\sigma ^2} \int _0^1 \left| dA_{\varphi (t)}(\alpha (t)) \partial _t \alpha (t) \right| ^2_{\alpha (t)} dt $$

with boundary conditions $\varphi (0) = {\mathrm {id}}_G$, $\alpha (0) = a_0$, $\varphi (1)\cdot \alpha (1) = a_1$.

This construction can also be interpreted using a Riemannian submersion (Sect. B.6.7). Indeed, the mapping

is a submersion from $G\times M$ to M with $d\pi (\varphi , \alpha )(h, \rho ) = dA_\varphi (\alpha )\rho + dR_\alpha (\varphi ) h$. ($\pi$ is a submersion because $dA_\varphi (\alpha )$ is invertible, with inverse $dA_{\varphi ^{-1}}(\varphi \cdot \alpha )$.) The vertical space at $(\varphi , \alpha )$ is the set of all pairs $(h, \rho )$ satisfying

$$\begin{aligned} \rho = - dA_{\varphi ^{-1}}(\varphi \cdot \alpha ) dR_\alpha (\varphi ) h. \end{aligned}$$

(12.9)

Two pairs $(\varphi , \alpha )$ and $(\tilde{\varphi }, \tilde{\alpha })$ belong to the same fiber if and only if $\varphi \cdot \alpha = \tilde{\varphi }\cdot \tilde{\alpha }$, or, letting $\psi = \varphi ^{-1}\tilde{\varphi }$, $(\tilde{\varphi }, \tilde{\alpha }) = (\varphi \psi , \psi ^{-1}\cdot \alpha )$. This leads us to introduce the right action of G on $G\times M$ defined by

$$ (\varphi , \alpha )\cdot \psi = (\varphi \psi , \psi ^{-1}\cdot \alpha ) =: \tilde{R}_\psi (\varphi , \alpha ), $$

which provides a transitive action on the fiber over $a = \varphi \cdot \alpha$. Assume that a Riemannian metric $\Vert (h, \rho )\Vert _{(\varphi , \alpha )}$ is given on $G\times M$, and that this metric is invariant through this action, so that

$$ d\tilde{R}_\psi (\varphi , \alpha ): (h, \rho ) \mapsto (dR_\psi (\varphi ) h, dA_{\psi ^{-1}}(\alpha ) \rho ) $$

is an isometry between $T_ {(\varphi , \alpha )} (G\times M)$ and $T_ {(\varphi , \alpha )\cdot \psi } (G\times M)$. Notice that this isometry maps vertical spaces onto vertical spaces. Indeed, take $(h, \rho ) \in T_ {(\varphi , \alpha )} G\times M$, so that h and $\rho$ satisfy (12.9). Then

$$\begin{aligned} dA_{\psi ^{-1}}(\alpha ) \rho&= - dA_{\psi ^{-1}}(\alpha ) \, dA_{\varphi ^{-1}}(\varphi \cdot \alpha )\, dR_\alpha (\varphi ) h\\&= - dA_{\psi ^{-1}\varphi ^{-1}}(\varphi \cdot \alpha ) \, dR_\alpha (\varphi ) h\\&= - dA_{\psi ^{-1}\varphi ^{-1}}(\varphi \cdot \alpha )\, dR_\alpha (\varphi )\, dR_{\psi ^{-1}}(\varphi \psi ) \, dR_\psi (\varphi ) h\\&= - dA_{(\varphi \psi )^{-1}}(\varphi \cdot \alpha ) \, dR_{\psi ^{-1}\alpha }(\varphi \psi ) \, dR_\psi (\varphi ) h, \end{aligned}$$

in which we have applied the chain rule to the identities $A_{\psi ^{-1}}A_{\varphi ^{-1}} = A_{\psi ^{-1}\varphi ^{-1}}$, $R_{\psi ^{-1}}R_\psi = {\mathrm {id}}$ and $R_\alpha R_{\psi ^{-1}} = R_{\psi ^{-1}\alpha }$. This shows that $d\tilde{R}_\psi (\varphi , \alpha ) (h, \rho )$ belongs to the vertical space at $(\varphi \psi , \psi ^{-1}\alpha )$.

Because $d\tilde{R}_\psi$ is an isometry that maps vertical spaces to vertical spaces, it also maps horizontal spaces to horizontal spaces. These spaces being isometric shows that $\pi$ is a Riemannian submersion, provided that the norm on M is defined by

$$ \Vert \eta \Vert _a = \min \left\{ \Vert (h, \rho )\Vert _{(\varphi , \alpha )}: (h, \rho )\in T_\varphi G\times T_\alpha M, dA_\varphi (\alpha )\rho + dR_\alpha (\varphi ) h = \eta \right\} $$

and this definition does not depend on $(\varphi , \alpha )$ such that $\varphi \cdot \alpha = a$. One can, in particular, take $(\varphi , \alpha ) = ({\mathrm {id}}, a)$, yielding

$$ \Vert \eta \Vert _a = \min \left\{ \Vert (v, \rho )\Vert _{({\mathrm {id}}, a)}: (v, \rho )\in V\times T_a M, \rho + v\cdot a = \eta \right\} . $$

The construction leading to (12.5) simply corresponds to the special case

$$ \Vert (v, \rho )\Vert _{({\mathrm {id}}, a)}^2 = \Vert v\Vert _v^2 + \frac{1}{\sigma ^2} |\rho |_a^2 $$

(which specifies $\Vert (h, \rho )\Vert _{(\varphi , \alpha )}$ everywhere because right translations are isometries). There is no particular reason for restricting to this special case beside it leading to simpler formulas, and the discussion that follows can easily be extended to the general case of a right-invariant metric on $G\times M$.

12.3 Euler–Lagrange Equations

We provide below optimality conditions in a Lagrangian setting that include metamorphosis in the case when M is a vector space. When M is a general Riemannian manifold, the optimality conditions were worked out in [282] and are expressed as follows (we refer to this reference for a proof of the statement)

$$\begin{aligned} \left\{ \begin{aligned}&\partial _t a - v\cdot a = z\\&\nabla _{\partial _t a} z + \nabla _z^\dagger X^v = 0\\&v = \frac{1}{\sigma ^2} {\mathbb {K}}\xi _a^* v. \end{aligned} \right. \end{aligned}$$

(12.10)

Here, $\nabla$ is the covariant derivative on M and $\nabla _{\partial _t a} = D/Dt$ is the evaluation of this derivative along the curve $t\mapsto a(t)\in M$ (recall that $z(t) \in T_{a(t)}M$); $X^v$ is the vector field $a \mapsto v\cdot a$ on M, and $\nabla _z^\dagger$ is defined by

$$ {\big \langle {Z}\, , \, {\nabla _{Z'}Z''}\big \rangle } = -{\big \langle {\nabla _Z^\dagger Z''}\, , \, {Z'}\big \rangle }, $$

where ${\big \langle {X}\, , \, {Y}\big \rangle }$ is the function $a \mapsto {\big \langle {X(a)}\, , \, {Y(a)}\big \rangle }_a$ for vector fields X, Y on M. Finally, ${\mathbb {K}}$ is, as usual, the inverse duality operator on V.

We now assume that M is a vector space and consider a generalized version of (12.8) minimizing

$$\begin{aligned} U(v, a) = \int _0^1 F\big (v(t), a(t), \partial _t a - v(t)\cdot a(t)\big ) dt \end{aligned}$$

(12.11)

for some function F. Let

$$\begin{aligned} z(t) = \partial _t a - v(t)\cdot a(t). \end{aligned}$$

(12.12)

We will denote by $\partial _v F$, $\partial _a F$ and $\partial _z F$ the partial differentials of F with respect to each of its variables. Computing the variation with respect to v, we get, for all $t\mapsto h(t)\in V$,

$$ \int _0^1 {\Big ( {\partial _v F}\,\Big |\, {h(t)}\Big )} dt - \int _0^1 {\Big ( {\partial _z F}\,\Big |\, {h(t)\cdot a(t)}\Big )} dt = 0. $$

Denoting by $Q_a\partial _z F$ the linear form

$$ {\Big ( {Q_a \partial _z F}\,\Big |\, {\tilde{h}}\Big )}:={\Big ( {\partial _z F}\,\Big |\, {\tilde{h}\cdot a}\Big )}, $$

we obtain the optimality equation for v

$$\begin{aligned} \partial _v F - Q_{a(t)} \partial _z F = 0. \end{aligned}$$

(12.13)

If we now make the variation with respect to a, the result is, for every $t \mapsto \alpha (t) \in M$,

$$ \int _0^1 {\Big ( {\partial _a F}\,\Big |\, {\alpha (t)}\Big )} dt + \int _0^1 {\Big ( {\partial _z F}\,\Big |\, {\partial _t \alpha - v(t)\cdot \alpha (t)}\Big )} dt = 0. $$

Integrating by parts and using the notation

$$ {\Big ( {\tilde{Q}_v \partial _z F}\,\Big |\, {\tilde{\alpha }}\Big )}:={\Big ( {\partial _z F}\,\Big |\, {v\cdot \tilde{\alpha }}\Big )}, $$

we obtain the optimality equation

$$\begin{aligned} - \partial _t \partial _z F - \tilde{Q}_v \partial _z F + \partial _a F = 0. \end{aligned}$$

(12.14)

Equations (12.12)–(12.14) provide the Euler–Lagrange equations for metamorphosis. They also provide the differentials of the energy with respect to v and a and can be used to design minimization algorithms.

12.4 Application to Labeled Point Sets

We consider here diffeomorphisms acting on collections of points

$$ a = (y^{(1)}, \ldots , y^{(N)}), $$

with $y^{(k)} \in {\mathbb {R}}^d$. We therefore have $M = ({\mathbb {R}}^d)^N$ (note that we are not assuming here that points are distinct). We consider the function [51, 189]

$$ F(v, a, z) = \Vert v\Vert ^2_V + \frac{1}{\sigma ^2} \sum _{k=1}^N |z^{(k)}|^2, $$

where V is an admissible RKHS of vector fields. (Note that F does not depend on a.) Here, because $\varphi \cdot a = (\varphi (y^{(1)}), \ldots , \varphi (y^{(N)}))$, we have

$$v\cdot a = (v(y^{(1)}), \ldots , v(y^{(N)})),$$

so that $z^{(k)} = \partial _t y^{(k)} - v(y^{(k)})$. We can therefore write

$$ U(v, a) = \int _0^1 \left\| v(t) \right\| _{V}^2 dt + \frac{1}{\sigma ^2} \sum _{k=1}^N \int _0^1 \left| \partial _t y^{(k)} - v(t, y^{(k)}(t)) \right| ^2 dt. $$

We have $\partial _v F = 2\mathbb Lv$ (where $\mathbb L$ is the duality operator of V) and $\partial _z F = (2/\sigma ^2)(z^{(1)}, \ldots , z^{(N)})$. Moreover,

$$ {\Big ( {Q_a\partial _z{F}}\,\Big |\, {h}\Big )} = \frac{2}{\sigma ^2} \sum _{k=1}^N (z^{(k)})^T h(y^{(k)}), $$

so that the first Euler–Lagrange equation is

$$ \mathbb Lv - \frac{1}{\sigma ^2} \sum _{k=1}^N z^{(k)} \delta _{y^{(k)}} = 0. $$

sFor the second equation, we write

$$ {\left( {\tilde{Q}_v \partial _z {F}}\, \left| {\tilde{Q}_v \partial _z {F}}\, {\alpha }\right. \right) } = \frac{2}{\sigma ^2} \sum _{k=1}^N (z^{(k)})^T dv(y^{(k)}) \alpha _k, $$

yielding

$$ - \partial _t z^{(k)} - dv(y^{(k)})^T z^{(k)} = 0. $$

This provides the system of Euler–Lagrange equations for labeled point-set metamorphosis:

$$\begin{aligned} \left\{ \begin{aligned}&\mathbb Lv = \frac{1}{\sigma ^2} \sum _{k=1}^N z^{(k)} \delta _{y^{(k)}},\\&\partial _t z^{(k)} + dv(y^{(k)})^T z^{(k)} = 0,\\&\partial _t y^{(k)} - v(y^{(k)}) = z^{(k)}. \end{aligned} \right. \end{aligned}$$

(12.15)

Note that, introducing the reproducing kernel of V, the first equation is equivalent to

$$ v(t, x) = \sum _{k=1}^N K(x, y^{(k)}(t))z^{(k)}(t). $$

This implies that minimizing E is equivalent to minimizing

$$\begin{aligned} {\tilde{E}}(a, z) = \sum _{k, l=1}^N \int _0^1&z^{(k)}(t)^T K(y^{(k)}(t), y^{(l)}(t)) z^{(l)}(t)dt \\ +&\frac{1}{\sigma ^2}\sum _{k=1}^N \int _0^1 \left| \partial _t y^{(k)}(t) - \sum _{l=1}^n K(y^{(k)}(t), y^{(l)}(t)) z^{(l)}(t)\right| ^2 dt. \end{aligned}$$

The vectors $z^{(1)}, \ldots , z^{(N)}$ can be computed explicitly given the trajectories $y^{(1)}, \ldots , y^{(N)}$, namely

$$ z = (S(y) + \lambda I)^{-1} \partial _t y. $$

This provides an expression of the energy in terms of $y^{(1)}, \ldots , y^{(N)}$ that can be minimized directly, as proposed in [51]. Most of the methods developed for point sets in Chap. 10 can in fact be adapted to this new framework. Figure 12.1 provides examples of deformations computed with this method.

Fig. 12.1

Point-set matching using metamorphosis, with the same input as in Figs. 9.1 and 10.1, using Gaussian kernels K(x, y) =$\exp (-|x-y|^2/2\sigma ^2)$ with $\sigma = 1, 2, 4$ in grid units

12.5 Application to Images

12.5.1 Formal Analysis

Let M be a set of square integrable and differentiable functions $a:{\mathbb {R}}^d\rightarrow {\mathbb {R}}$. We let $G=\mathrm {Diff}_V$ act on M by $ga = a\circ g^{-1}$. We use the $L^2$ norm as the initial metric on M, and we start with a formal discussion. Since we assume that a is differentiable, we can write $v\cdot a = - \nabla a^T v$. We then define

$$ F(v, a, z) = \left\| v \right\| _{V}^2 + \frac{1}{\sigma ^2} \Vert z\Vert _2^2 , $$

which, here again, does not depend on a.

Equation (12.12) is $z(t) = \partial _t a(t) + \nabla a(t)^T v(t)$. We have

$$ {\Big ( {Q_a\partial _z {F}}\,\Big |\, {h}\Big )} = - \frac{2}{\sigma ^2} \int _{{\mathbb {R}}^d} z\nabla a^T h dx, $$

so that (12.13) is $\mathbb Lv = - z\nabla a dx$. Also,

$$ {\Big ( {\tilde{Q}_v \partial _z{F}}\,\Big |\, {\alpha }\Big )} = - \frac{2}{\sigma ^2} \int _{{\mathbb {R}}^d} z v^T\nabla a dx, $$

so that, using the divergence theorem, $\tilde{Q}_v \partial _z {F} = \mathrm {div}(zv)$ and (12.14) is

$$\begin{aligned} -\partial _t z - \mathrm {div}(zv) = 0. \end{aligned}$$

So the optimality equations for image metamorphosis are

$$\begin{aligned} \left\{ \begin{aligned}&\partial _t a + \nabla a^T v = z,\\&\partial _t z + \mathrm {div}(zv) = 0,\\&\mathbb Lv = -\frac{1}{\sigma ^2} (z\nabla a) dx. \end{aligned} \right. \end{aligned}$$

(12.16)

Figures 12.2, 12.3 and 12.4 provide examples of images matched using the associated energy. In these examples, the first and last images are given as input and two interpolated images are provided. The numerical scheme is described in Sect. 12.5.4.

Fig. 12.2

Image metamorphosis. Estimation of a geodesic between a disc and a square. First row: image evolution in time. Second row: evolution of the diffeomorphism

Fig. 12.3

Creation of new patterns with image metamorphosis. Geodesic between an empty image and a disc. First row: image evolution in time. Second row: evolution of the diffeomorphism

Fig. 12.4

Gray-level image metamorphosis. Geodesic between two gray-level face images. First row: image evolution in time. Second row: evolution of the diffeomorphism

12.5.2 Some Rigorous Results

In this section, we provide a more rigorous treatment of image metamorphosis. We will extend the $L^2$ case considered above to Sobolev spaces, assuming that images (a or $\alpha$) belong to a Hilbert space H, with norm equivalent to the $H^r({\mathbb {R}}^d)$ norm for some integer $r\ge 0$, with notation for the $H^r$ norm

$$ \Vert u\Vert _{r, 2}^2 = \sum _{|k| \le r} \Vert \partial _k u\Vert _2^2, $$

where k denotes a d-dimensional multi-index $(k_1,\ldots , k_d)$, $|k|=k_1+\cdots +k_d$,

$$ \partial _k u = \partial ^{k_1}_1\ldots \partial ^{k_d}_d u $$

and $\Vert \cdot \Vert _2$ is the $L^2$ norm. We will consider metamorphoses with cost

$$ F(v, a, z) = \Vert v\Vert _V^2 + \frac{1}{\sigma ^2} \Vert z\Vert ^2_H $$

and provide a few results describing their properties. The proofs of these results being somewhat technical, we will skip them, referring the reader to [245] for more details.

We first reformulate the problem to make sure that it is also well defined for functions a that are not differentiable. We can write the advection equation $\partial _t a + \nabla a^T v = z$ as $\partial _t \alpha = z\circ \varphi$ with $\alpha = a\circ \varphi$ and $\partial _t\varphi = v\circ \varphi$. We can therefore consider the minimization of

$$\begin{aligned} \int _0^1 \Vert v\Vert _V^2\, dt + \frac{1}{\sigma ^2} \int _0^1 \Vert z(t)\Vert _H^2\, dt \end{aligned}$$

(12.17)

subject to the constraints $\partial _t\varphi = v\circ \varphi$, $\partial _t\alpha = z\circ \varphi$, $\varphi (0) = {\mathrm {id}}$, $\alpha (0) = a_0$ and $\alpha (1) = a_1\circ \varphi (1)$.

We then have the following theorem.

Theorem 12.3

Assume $r \ge 0$ and $p\ge \max (1,r)$. Then the image metamorphosis problem has at least one solution.

The validity of the optimality conditions requires slightly more restrictive assumptions on the boundary conditions $a_0$ and $a_1$. We let ${\mathbb {K}}_V: V^*\rightarrow V$ and ${\mathbb {K}}_H: H^*\rightarrow H$ be the duality operators of the Hilbert spaces V and H and define $\xi _\varphi v = v\circ \varphi$ and $\tilde{\xi }_\varphi \alpha = \alpha \circ \varphi$. Then, the following result holds.

Theorem 12.4

Assume that both $a_0$ and $a_1$ belong to $H^{r+1}({\mathbb {R}}^d)$. Then, if $(v, z, \varphi , \alpha )$ is an optimal solution of the metamorphosis problem, there exist $\rho _\varphi (\cdot )\in C^p_0({\mathbb {R}}^d, {\mathbb {R}}^d)^*$ and $\rho _\alpha \in H^*$ such that the system

$$\begin{aligned} \left\{ \begin{aligned}&\partial _t \varphi = v \circ \varphi \\&\partial _t \alpha = z \circ \varphi \\&\partial _t \rho _\varphi = - \partial _\varphi {\left( {\rho _\varphi }\, \left| {\rho _\varphi }\, {v\circ \varphi }\right. \right) } - \partial _\varphi {\left( {\rho _\alpha }\, \left| {\rho _\alpha }\, {z\circ \varphi }\right. \right) }\\&\partial _t \rho _\alpha = 0 \\&v = {\mathbb {K}}_V \xi _\varphi ^* \rho _\varphi \\&z = \sigma ^2 {\mathbb {K}}_H \tilde{\xi }_\varphi ^* \rho _\alpha \end{aligned} \right. \end{aligned}$$

(12.18)

is satisfied, with boundary conditions $\alpha (0) = a_0$, $\alpha (1) = a_1\circ \varphi (1)$ and

$$ {\left( {\rho _\varphi (1)}\, \left| {\rho _\varphi (1)}\, {\delta \varphi }\right. \right) } + {\left( {\rho _\alpha }\, \left| {\rho _\alpha }\, {\nabla a_1 \circ \varphi (1)^T \delta \varphi }\right. \right) } = 0 $$

for all $\delta \varphi \in C^p_0({\mathbb {R}}^d, {\mathbb {R}}^d)$.

Moreover, the boundary condition propagates, so that

$$\begin{aligned} {\left( {\rho _\varphi (t)}\, \left| {\rho _\varphi (t)}\, {\delta \varphi }\right. \right) } + {\left( {\rho _\alpha }\, \left| {\rho _\alpha }\, {\nabla a(t) \circ \varphi (t)^T \delta \varphi }\right. \right) } = 0 \end{aligned}$$

(12.19)

for all $t\in [0,1]$.

(Equation (12.19) is in fact the horizontality condition associated with geodesics obtained through the Riemannian submersion.)

Finally, the following theorem provides sufficient conditions for the existence of solutions of (12.18) with given initial conditions.

Theorem 12.5

Assume that $p\ge 1+d/2$ and $p\ge r+1$. Then system (12.18) has a unique solution over any bounded interval as soon as $\rho _{\varphi , 0}\in C_0^{p-2}({\mathbb {R}}^d, {\mathbb {R}}^d)^*$ and $\rho _\alpha \in H^{r-1}({\mathbb {R}}^d)^*$.

Note that, with metamorphosis, the boundary condition requires that ${\left( {\rho _{\varphi , 0}}\, \left| {\rho _{\varphi , 0}}\, {w}\right. \right) } = {\left( {\rho _\alpha }\, \left| {\rho _\alpha }\, {\nabla a_0^T w}\right. \right) }$. Assuming that $a_0 \in H^1({\mathbb {R}}^d)$ (which is restrictive only for $r=0$), we see that $\rho _\alpha \in H^{r-1}({\mathbb {R}}^d)^*$ implies that $\rho _{\varphi , 0} \in C_0^{r-1}({\mathbb {R}}^d, {\mathbb {R}}^d)^* \subset C_0^{p-2}({\mathbb {R}}^d, {\mathbb {R}}^d)^*$, since $p\ge r+1$, so that the regularity condition for $\rho _{\varphi , 0}$ is automatically satisfied.

12.5.3 Remarks on the Optimality Conditions

System (12.18) corresponds to Pontryagin’s maximum principle for (12.17) considered as an optimal control problem with state $(\varphi , \alpha )$, control (v, z) and co-state $(\rho _\varphi , \rho _\alpha )$. We first check that, under additional differentiability assumptions on the images, they are equivalent to those found in (12.16) in the $L^2$ case.

When $H = L^2({\mathbb {R}}^d)$, system (12.18) gives $z = \sigma ^2 \tilde{\xi }_\varphi ^*\rho _\alpha$ with $\rho _\alpha \in (L^2({\mathbb {R}}^d))^*=L^2({\mathbb {R}}^d)$ (meaning that we identify $\rho _\alpha dx$ with $\rho _\alpha$). We therefore have$\partial _t {\left( {z}\, \left| {z}\, {\tilde{a}\circ \varphi }\right. \right) } = 0$ for any $\tilde{a}\in {\mathbb {R}}^2$. When z is differentiable, this leads, after differentiating and applying the divergence theorem, to

$$ \partial _t z + {\mathrm {div}}(zv) = 0. $$

We have already seen that, when a is differentiable, the equation $\partial _t\alpha = z\circ \varphi$ implies that $\partial _t a + \nabla a^Tv = z$.

Equation (12.19) applied to $\delta \varphi = w\circ \varphi (t)$ gives ${\left( {\rho _\varphi (t)}\, \left| {\rho _\varphi (t)}\, {w\circ \varphi (t)}\right. \right) } =$ $- {\left( {\rho _\alpha }\, \left| {\rho _\alpha }\, {(\nabla a(t)^T w)\circ \varphi (t)}\right. \right) }$, or

$$ \xi _{\varphi (t)}^* \rho _{\varphi }(t) = - (\tilde{\xi }_{\varphi (t)}^*\rho _\alpha )\nabla a(t) = -\frac{1}{\sigma ^2} z(t)\nabla a(t). $$

This yields $\mathbb L_{V}v(t) = - z(t) \nabla a(t) /\sigma ^2$ at all times (or $- (z(t) \nabla a(t)) dx /\sigma ^2$ if we relax the identification between $L^2$ and its dual), and we retrieve the last equation in (12.16).

We now return to the general case. The second and last equations in (12.18) imply that $\partial _t\alpha = \sigma ^2 \tilde{\xi }_\varphi {\mathbb {K}}_H \tilde{\xi }_\varphi ^* \rho _\alpha$, yielding

$$ \alpha (t) = \alpha (0) + \sigma ^2 \left( \int _0^t \tilde{\xi }_{\varphi (s)} {\mathbb {K}}_H \tilde{\xi }_{\varphi (s)}^*\, ds\right) \rho _\alpha . $$

From the boundary conditions, we therefore get

$$ \rho _\alpha = \left( \int _0^1 \tilde{\xi }_{\varphi (s)} {\mathbb {K}}_H \tilde{\xi }_{\varphi (s)}^*\, ds\right) ^{-1} (a_1\circ \varphi (1) - a_0). $$

We therefore have

$$ \alpha (t) = a_0 + \left( \int _0^t \tilde{\xi }_{\varphi (s)} {\mathbb {K}}_H \tilde{\xi }_{\varphi (s)}^*\, ds\right) \left( \int _0^1 \tilde{\xi }_{\varphi (s)} {\mathbb {K}}_H \tilde{\xi }_{\varphi (s)}^*\, ds\right) ^{-1} (a_1\circ \varphi (1) - a_0), $$

which provides a “closed-form” expression of the template part of the metamorphosis given the diffeomorphism part. When $H = L^2({\mathbb {R}})$, for which ${\mathbb {K}}_H = {\mathrm {Id}}$, we have $\xi _\varphi ^*\rho = \rho \circ \varphi ^{-1}\, |\det (d(\varphi ^{-1}))|$, so that

$$ \xi _\varphi \xi _\varphi ^*\rho = \rho \, |\det (d(\varphi ^{-1}))|\circ \varphi = \rho \, |\det (d\varphi )|^{-1}. $$

It follows that

$$ \alpha (t) = a_0 + \left( \frac{\int _0^t |\det (d\varphi (s))|^{-1} \, ds}{\int _0^1 |\det (d\varphi (s))|^{-1} \, ds}\right) (a_1\circ \varphi (1) - a_0), $$

or

$$ a(t)\circ \varphi (t) = \left( \frac{\int _t^1 |\det (d\varphi (s))|^{-1} \, ds}{\int _0^1 |\det (d\varphi (s))|^{-1} \, ds}\right) a_0 + \left( \frac{\int _0^t |\det (d\varphi (s))|^{-1} \, ds}{\int _0^1 |\det (d\varphi (s))|^{-1} \, ds}\right) a_1\circ \varphi (1), $$

which describes how metamorphosis interpolates between the two images it compares.

When $r > d/2 + 1$, (12.18) for $H^r$ metamorphoses has some interesting singular solutions [152, 245]. In this case, H is a reproducing kernel Hilbert space, and we will use $K_H$ to denote its kernel. We look for solutions of (12.18) in which $\rho _\varphi$ and $\rho _\alpha$ take the form

$$\begin{aligned} \rho _\varphi (t)= & {} \sum _{k=1}^N \beta _k(t) \delta _{x_k^{(0)}},\end{aligned}$$

(12.20)

$$\begin{aligned} \rho _\alpha= & {} \sum _{k=1}^N \gamma _k \delta _{x_k^{(0)}}, \end{aligned}$$

(12.21)

where $x^{(0)} = \{x^{(0)}_k\}_{k=1}^N$ is a collection of points in ${\mathbb {R}}^d$, $\beta (t) = \{\beta _k(t)\}_{k=1}^N$ is a collection of time-dependent vectors in ${\mathbb {R}}^d$, and $\gamma = \{\gamma _k\}_{k=1}^N$ is a time-independent collection of scalars.

Introduce the trajectories $x_k(t) := \varphi (t, x^{(0)}_k)$. Using this notation, we have

$$ {\left( {\xi _{\varphi (t)}^*\rho _\varphi (t)}\, \left| {\xi _{\varphi (t)}^*\rho _\varphi (t)}\, {w}\right. \right) } = {\left( {\rho _\varphi (t)}\, \left| {\rho _\varphi (t)}\, {w\circ \varphi (t)}\right. \right) } = \sum _{k=1}^N \beta _k(t)^T w(x_k(t)), $$

so that

$$ \xi _{\varphi (t)}^*\rho _\varphi (t) = \sum _{k=1}^N \beta _k(t) \delta _{x_k(t)} $$

and (12.18) implies that

$$ v(t, \cdot ) = \sum _{k=1}^N K_V(\cdot , x_k(t)) \beta _k(t). $$

Similarly, one gets

$$ z(t, \cdot ) =\sigma ^2 \sum _{k=1}^N K_H(\cdot , x_k(t)) \gamma _k. $$

The third equation in (12.18) gives, for $w \in C^p_0({\mathbb {R}}^d, {\mathbb {R}}^d)$,

$$\begin{aligned} \sum _{k=1}^N \partial _t \beta _k(t)^T w(x_k^{(0)})&= \\ - \sum _{k=1}^N&\beta _k(t)^T (dv(x_k(t)) w(x_k^{(0)})) - \sum _{k=1}^N \gamma _k \nabla z(x_k(t))^T w(x_k^{(0)}) \end{aligned}$$

from which we get

$$ \partial _t \beta _k(t) = -dv(x_k(t))^T \beta _k(t) - \gamma _k \nabla z(x_k(t)). $$

Using the expansions of v and z and the fact that $\partial _t x_k = v(t, x_k)$, we obtain the fact that (12.20) and (12.21) provide solutions of (12.18) as soon as x, $\alpha$ and z satisfy the coupled dynamical system

$$\begin{aligned} \left\{ \begin{aligned} \partial _t{x}_k(t) =&\sum _{l=1}^{N} K_{V}(x_k(t), x_l(t)) \beta _l(t) \\ \partial _t \alpha _k(t) =&\sum _{l=1}^N K_{H}(x_k(t), x_l(t)) \gamma _l\\ \partial _t {\beta }_k(t) =&- \sum _{l=1}^{N} \nabla _1 K_V(x_k(t), x_l(t)) \beta _l(t)^T\beta _k(t) \\&- \frac{1}{\sigma ^2}\sum _{l=1}^N \nabla _1 K_H(x_k(t), x_l(t)) \gamma _k \gamma _\ell . \end{aligned} \right. \end{aligned}$$

(12.22)

Finally, we note that Eq. (12.19) applied to $\rho _\varphi$ and $\rho _\alpha$ is

$$ \sum _{k=1}^N \beta _k(t)^T \delta \varphi (x_k) = -\sum _{k=1}^N \gamma _k \nabla \alpha (t, x_k^{(0)})^T d\varphi (t, x_k^{(0)})^{-1} \delta \varphi (x_k) $$

for all $\delta \varphi$, yielding

$$ \beta _k(t) = - \gamma _k d\varphi (t, x_k^{(0)})^{-T} \nabla \alpha (x_k^{(0)}) = -\gamma _k \nabla a(t, x_k(t)). $$

These special solutions have been used in [245] to provide approximations of solutions for metamorphoses between smooth images.

12.5.4 Discretization of Image Metamorphosis in the $L^2$ Case

The images in Figs. 12.2 and 12.3 have been obtained after minimization of a discretized version of the image metamorphosis energy,

$$ E(a, v) = \int _0^1 \Vert v(t)\Vert ^2_V dt + \frac{1}{\sigma ^2} \int _0^1 \Vert \partial _t a(t) + \nabla a(t)^Tv(t)\Vert _2^2 dt. $$

The second integrand must be discretized with care. It represents the total derivative of a along the flow. Stable results can be obtained using the following scheme.

Introduce the auxiliary variable $w = \mathbb L^{1/2}v$, so that $v = {\mathbb {K}}^{1/2}w$ where ${\mathbb {K}}= \mathbb L^{-1}$ (the numerical implementation can in fact explicitly specify $\tilde{\mathbb K} = {\mathbb {K}}^{1/2}$ and use ${\mathbb {K}}= \tilde{\mathbb K}^2$). The following discretized energy has been used in the experiments of Figs. 12.2 and 12.3:

$$ E = \sum _{t=1}^T \sum _{x\in \tilde{\varOmega }} \left| w(x) \right| ^2 + \lambda \delta t^{-2} \sum _{t=1}^T \sum _{x\in \tilde{\varOmega }} \left| a(t+1, x + \delta t ({\mathbb {K}}^{1/2}w)(t, x)) - a(t, x) \right| ^2, $$

where x and t now are discrete variables, $\tilde{\varOmega }$ a discrete grid on $\varOmega$ and $\delta t$ the time discretization step (the space discretization is 1). The optimization algorithm alternates a few steps of nonlinear conjugate gradient in v, and a few steps of linear conjugate gradient in a [120, 121].

12.6 Applications to Densities

If one denotes $H^{r}({\mathbb {R}}^d)^*$ by $H^{-r}({\mathbb {R}}^d)$ for $r\ge 0$, one can consider the action of $\mathrm {Diff}_0^p$ on these spaces defined by

$$ {\left( {\varphi \cdot a}\, \left| {\varphi \cdot a}\, {f}\right. \right) } = {\left( {a}\, \left| {a}\, {f\circ \varphi }\right. \right) }, $$

which is well defined on $H^{-r}({\mathbb {R}}^d)$ for $r\le p$. For $r=0$, in particular, this action boils down to the usual action on densities $\varphi \cdot a = a\circ \varphi ^{-1} \, |\det (d(\varphi ^{-1}))|$. Starting with this special case, for which the infinitesimal action is $v\cdot a = -{\mathrm {div}}(av)$, we can consider the metamorphosis problem associated here again with

$$ F(v, a, z) = \left\| v \right\| _{V}^2 + \frac{1}{\sigma ^2} \Vert z\Vert _2^2 , $$

with $z(t) = \partial _t a(t) + {\mathrm {div}}(a(t) v(t))$. The counterpart of system (12.16) is (details being left to the reader)

$$\begin{aligned} \left\{ \begin{aligned}&\partial _t a + {\mathrm {div}}(av) = z,\\&\partial _t z + \nabla z^T v = 0,\\&\mathbb Lv = \frac{1}{\sigma ^2} (a\nabla z) dx. \end{aligned} \right. \end{aligned}$$

(12.23)

More generally, letting $H = H^r({\mathbb {R}}^d)$, we can define metamorphoses in $H^*$ using

$$ F(v, a, z) = \left\| v \right\| _{V}^2 + \frac{1}{\sigma ^2} \Vert z\Vert _{H^*}^2. $$

We can also consider the metamorphosis problem in optimal control form as in (12.17), minimizing

$$\begin{aligned} \int _0^1 \Vert v\Vert _V^2\, dt + \frac{1}{\sigma ^2} \int _0^1 \Vert z(t)\Vert _{H^*}^2\, dt \end{aligned}$$

(12.24)

subject to the constraints $\partial _t\varphi = v\circ \varphi$, $\partial _t\alpha = z\circ \varphi \, |\det d\varphi |$, $\varphi (0) = {\mathrm {id}}$, $\alpha (0) = a_0$ and $\alpha (1) = a_1\circ \varphi (1) |\det d\varphi (1)|$. Defining, in this case $\tilde{\xi }_\varphi z = z\circ \varphi |\det d\varphi |$, the optimality conditions (12.18) become

$$\begin{aligned} \left\{ \begin{aligned}&\partial _t \varphi = v \circ \varphi \\&\partial _t \alpha = z \circ \varphi \, |\det d\varphi |\\&\partial _t \rho _\varphi = - \partial _\varphi {\left( {\rho _\varphi }\, \left| {\rho _\varphi }\, {v\circ \varphi }\right. \right) } - \partial _\varphi {\left( {\rho _\alpha }\, \left| {\rho _\alpha }\, {z\circ \varphi \, |\det d\varphi |}\right. \right) }\\&\partial _t \rho _\alpha = 0 \\&v = {\mathbb {K}}_V \xi _\varphi ^* \rho _\varphi \\&z = \sigma ^2 {\mathbb {K}}_H \tilde{\xi }_\varphi ^* \rho _\alpha \end{aligned} \right. \end{aligned}$$

(12.25)

with boundary conditions $\alpha (0) = a_0$, $\alpha (1) = a_1\circ \varphi (1) |\det d\varphi (1)|$ and horizontality condition

$$ {\left( {\xi _\varphi ^* \rho _\varphi }\, \left| {\xi _\varphi ^* \rho _\varphi }\, {w}\right. \right) } + {\left( {\tilde{\xi }_\varphi ^* \rho _\alpha }\, \left| {\tilde{\xi }_\varphi ^* \rho _\alpha }\, {{\mathrm {div}}(aw)}\right. \right) } = 0 $$

for all $w \in C^p_0({\mathbb {R}}^d, {\mathbb {R}}^d)$.

12.7 Application to Curves

12.7.1 Metamorphosis on Unit Tangents

We now consider the issue of comparing plane curves based on the orientation of their tangents [171, 307, 313]. If m is a plane curve parametrized by arc length, and L is it length, we define the normalized tangent $T^m$ by

The function $T^m$ characterizes m up to translation and scaling, and the pair $(L, T^m)$ characterizes m up to translation. In the following discussion, functions will depend on time $t\in [0,1]$ and normalized arc length s, also in [0, 1]. For clarity, we will let $\varOmega = [0,1]$ for the arc length, i.e., write $t\in [0,1]$, $s\in \varOmega$. We will consider derivatives of functions h of $(t, s)\in [0,1]\times \varOmega$, and we will use the notation $\dot{h}$ or $\partial _t h$ for derivatives with respect to the time variable, t, and dh for derivatives with respect to the space variable, s.

We consider the group of diffeomorphisms $\mathrm {Diff}$ of $\varOmega$, which acts on the set M of measurable functions $a:\varOmega \rightarrow S^{d-1}$ (the unit sphere in ${\mathbb {R}}^d$) by $\varphi \cdot a = a\circ \varphi ^{-1}$. Choosing an RKHS V of vector fields on $\varOmega$ satisfying $v(0) = v(1) = 0$, we can consider the metamorphosis problem associated with

$$\begin{aligned} F(v, a, z) = \Vert v\Vert _V^2 + \frac{1}{\sigma ^2} \Vert z\Vert _2^2. \end{aligned}$$

(12.26)

If V is embedded in $C^1_0(\varOmega , {\mathbb {R}})$, the discussion made in the previous sections applies, with the slight difference that one must take “$z \in T_aM$” (we will not try here to rigorously construct the shape space as a manifold), which means that $z(s)\perp a(s)$ at all times. If, in particular, the boundary conditions $a_0$ and $a_1$ are differentiable, then Eq. (12.10) applies, and leads to the system, for a metamorphosis a(t, s)

$$ \left\{ \begin{aligned}&\partial _t a + v d a = z,\\&\partial _t \zeta + d(v\zeta ) = 0,\\&v = -\frac{1}{\sigma ^2} {\mathbb {K}}_V(da^Tz) \end{aligned} \right. $$

in which $\zeta (, s):\varOmega \rightarrow {\mathbb {R}}$ is the normal coordinate of z(t, s), defined by $z(t,s) = \zeta (t,s) a^\perp (t, s)$, where $a^\perp$ is a rotated by $\pi /2$.

However, the resulting metric takes an interesting form if one considers the Hilbert space V of functions $v:[0,1]\rightarrow {\mathbb {R}}$ such that $v(0) = v(1) = 0$ and

$$\begin{aligned} \Vert v\Vert _V^2 = \int _\varOmega (dv)^2\, ds. \end{aligned}$$

(12.27)

Notice that this Hilbert space is not embedded in $C^1_0(\varOmega , {\mathbb {R}})$ (one needs Sobolev spaces of order larger $d/2+1 = 3/2$ for this, i.e., one would need a second derivative in the norm). Functions $v\in V$ are continuous and satisfy a Hölder condition of order q for any $q<1/2$, but are not necessarily Lipschitz continuous. While the general framework we have considered so far does not apply to this situation, one can directly formulate the metamorphosis problem in terms of time-dependent diffeomorphisms, $(t,s) \mapsto \varphi (t, s)$, of $\varOmega$, letting $v = \dot{\varphi }\circ \varphi ^{-1}$ so that

$$ d v = \frac{d\dot{\varphi }\circ \varphi ^{-1}}{d\varphi \circ \varphi ^{-1}}, $$

which yields, after a change of variables

$$ \int _\varOmega (d v)^2\, ds = \int _\varOmega \frac{ d\dot{\varphi }^2}{d\varphi }\, ds. $$

Writing the second term in template form, i.e., letting $z = \dot{\alpha }\circ \varphi ^{-1}$ and making another change of variable, the metamorphosis objective function in this case becomes

$$\begin{aligned} U_\sigma (\varphi , \alpha ) = \int _0^1\int _\varOmega \frac{( d\dot{\varphi })^2}{d\varphi }\, ds\, dt + \frac{1}{\sigma ^2}\int _0^1\int _\varOmega |\dot{\alpha }|^2 d\varphi \, ds\, dt, \end{aligned}$$

(12.28)

which needs to be minimized over all trajectories $t\mapsto \varphi (t)$ and $t\mapsto \alpha (t)$, such that $\varphi$ is at all times an increasing diffeomorphism of $\varOmega$ and $\alpha$ a function from $\varOmega \rightarrow S^{d-1}$, with boundary conditions $\alpha (0) = a_0$ and $\alpha (1) = a_1\circ \varphi (1)$. We point out (this will be useful later) that this energy can be minimized explicitly with respect to $\alpha$ when the $\varphi$ trajectory is fixed. Indeed, first notice that in order to minimize the second term in U, it suffices to minimize separately the integrals

$$\begin{aligned} \int _0^1 |\dot{\alpha }(t, s)|^2 d\varphi (t, s) dt \end{aligned}$$

(12.29)

for fixed s. Considering such an integral, we write $\alpha (t,s) = \tilde{\alpha }(\lambda (t, s), s)$, where

$$ \lambda (t, s) := \frac{\int _0^t d\varphi (t, s)^{-1}\, dt}{\int _0^1 d\varphi (t, s)^{-1}\, dt} $$

is an increasing function satisfying $\lambda (0,s) = 0$ and $\lambda (1,s) = 1$. We have $\dot{\alpha }= \dot{\lambda }\dot{\tilde{\alpha }}(\lambda , s)$ and

$$ \int _0^1 |\dot{\alpha }(t, s)|^2 d\varphi (t, s) dt \,{=}\, \frac{1}{c(s)}\int _0^1 |\dot{\tilde{\alpha }} (\lambda (t, s), s)|^2 \dot{\lambda }(t, s)\, dt {=} \frac{1}{c(s)} \int _0^1 |\dot{\tilde{\alpha }} (t, s)|^2\, dt $$

with

$$ c(s) = \int _0^1 d\varphi (t, s)^{-1}\, dt. $$

This integral must be minimized subject to $\tilde{\alpha }(0,s) = a_0(s)$, $\tilde{\alpha }(1,s) = a_1\circ \varphi _1(s)$ and $|\tilde{\alpha }(t, s)| = 1$ for all t, and the solution is given by the circular arc between $\tilde{\alpha }(0,s)$ and $\tilde{\alpha }(1,s)$, which can be expressed as

$$ \tilde{\alpha }(t, s) = \frac{\sin ((1-t) \omega (s))}{\sin \omega (s)} a_0(s) + \frac{\sin (t \omega (s))}{\sin \omega (s)} a_1\circ \varphi _1(s), $$

where

$$ \omega (s) = \mathrm {arccos} (a_0(s)^T a_1\circ \varphi _1(s)). $$

The optimal $\alpha$ is therefore given by

$$\begin{aligned} \alpha (t, s) = \frac{\sin ((1-\lambda ^{-1}(t, s)) \omega (s))}{\sin \omega (s)} a_0(s) + \frac{\sin (\lambda ^{-1}(t, s) \omega (s))}{\sin \omega (s)} a_1\circ \varphi _1(s), \end{aligned}$$

(12.30)

where $\lambda ^{-1}(t, s)$ is defined by $\lambda (\lambda ^{-1}(t,s), s) = t$. The optimal cost in (12.29) is then $\omega (s)^2/c(s)$. We note that, because $\omega (s)\in [0, \pi ]$, the coefficients in (12.30) are non-negative. Moreover, $\alpha (t, s)$ is at all times in the plane generated by $a_0(s)$ and $\alpha _1 = a_1\circ \varphi _1(s)$.

We now fix $\varphi _1$ and study optimal metamorphoses with $\varphi (1, \cdot ) = \varphi _1$. Introduce the vector $a_0^\perp$ perpendicular to $a_0$ in the plane generated by $a_0$ and $\alpha _1$, defined by

$$ \alpha _1 = \cos \omega \, a_0 + \sin \omega a_0^\perp . $$

(This is well defined if $\omega \in (0, \pi )$ and we choose $a_0^\perp$ arbitrarily otherwise.) Without loss of generality, we can search for optimal metamorphoses taking the form

$$ \alpha (t,s) = \cos \tau (t, s) a_0(s) + \sin \tau (t, s) a_0^\perp (s)\,. $$

Letting $\xi (t, s) = (\cos \tau (t,s), \sin \tau (t, s))\in S^1$, we can write $U_\sigma (\varphi , \alpha ) = \tilde{U}_\sigma (\varphi , \xi )$, where

$$\begin{aligned} \tilde{U}_\sigma (\varphi , \xi ) = \int _0^1\int _\varOmega \frac{( d\dot{\varphi })^2}{d\varphi }\, ds\, dt + \frac{1}{\sigma ^2}\int _0^1\int _\varOmega |\dot{\xi }|^2 d\varphi \, ds\, dt. \end{aligned}$$

(12.31)

This function now has to be minimized subject to $\varphi (0, \cdot ) = {\mathrm {id}}$, $\varphi (1, \cdot ) = \varphi _1$, $\xi (0, \cdot ) = (1,0)$ and $\xi (1, \cdot ) = (\cos \omega , \sin \omega )$, with $\omega = \mathrm {arccos} (a_0^T a_1\circ \varphi (1, \cdot ))$. In other terms, we have reduced the $S^{d-1}$-valued metamorphosis problem to an $S^1$-valued problem, or, equivalently, our metric on d-dimensional curves to a two-dimensional case.

We now make a second reduction that will simplify the problem. Because $\xi (t, s)$ is differentiable in time, one can define uniquely a differentiable function $\tau (t, s)$ such that $\xi (0,s) = (1,0)$ and $\xi (t,s) = (\cos \tau (t,s), \sin \tau (t, s))$ at all times. Define q(t, s) by

$$ q(t, s) =\sqrt{d \varphi (t, s)} \left( \cos \eta (t,s), \sin \eta (t, s)\right) , $$

with $2\sigma \eta (t, s) = \tau (t, s)$. Then, a straightforward computation yields

$$ 4|\dot{q}(t, s)|^2 = \frac{d\dot{\varphi }^2}{d \varphi } + \frac{1}{\sigma ^2} |\dot{\xi }(t, s)|^2, $$

so that

$$\begin{aligned} \tilde{U}_\sigma (\varphi , \xi ) = 4\int _0^1 \Vert \dot{q}(t, \cdot )\Vert ^2_2\, dt. \end{aligned}$$

(12.32)

We also note that

$$ \Vert q(t, \cdot )\Vert _2^2 = \int _\varOmega d\varphi (t, s)\, ds = 1, $$

so that $q(t, \cdot )$ is a curve on the unit sphere of $L^2(\varOmega , {\mathbb {R}}^2)$. This implies that its energy, $\int _0^1 \Vert \dot{q}(t, \cdot )\Vert ^2_2\, dt$, cannot be larger than that of the minimizing geodesic on this unit sphere, which is the shortest great circle connecting the functions $q_0 = \left( 1, 0\right)$ (which is constant) and $q_1 = \sqrt{d\varphi _1} \left( \cos \left( \eta (1,\cdot )\right) , \sin \left( \eta (1, \cdot \right) \right)$. Letting

$$ \rho = \mathrm {arccos} {\big \langle {q_0}\, , \, {q_1}\big \rangle }_2 = \mathrm {arcos} \int _\varOmega \sqrt{d\varphi _1}\cos \left( \frac{\omega (s)}{2\sigma }\right) \, ds, $$

this geodesic is given by

$$\begin{aligned} \gamma (t, s) = \frac{\sin ((1-t)\rho )}{\sin \rho } q_0(s) + \frac{\sin (t\rho )}{\sin \rho } q_1(s) \end{aligned}$$

(12.33)

with energy equal to $\rho ^2$. We therefore find that

$$\begin{aligned} \tilde{U}_\sigma (\varphi , \xi ) \ge 4\, \mathrm {arcos}^2 \int _\varOmega \sqrt{d\varphi _1(s)}\cos \left( \frac{\omega (s)}{2\sigma }\right) \, ds\,. \end{aligned}$$

(12.34)

This provides a lower-bound for the metamorphosis energy. To prove that this lower-bound is achieved, one needs to show that the trajectory $\gamma$ in (12.33) can be derived from a valid trajectory $(\psi (\cdot , \cdot ), \mu (\cdot , \cdot ))$ that connects $({\mathrm {id}}, 0)$ to $(\varphi _1, \xi _1)$.

We are therefore looking for representations of $\gamma$ in the form

$$ \gamma (t, s) =\sqrt{d \psi (t, s)} \left( \cos \tilde{\eta }(t, s), \sin \tilde{\eta }(t, s)\right) $$

with $\tilde{\eta }(0, s) = 0$, which uniquely defines $\tilde{\eta }(t, s)$ by continuity in t. Notice that we automatically have $d\psi (0, \cdot ) = 1$ and $d\psi (1, \cdot ) = d\varphi _1$ by definition of $q_0$ and $q_1$. For $t\in (0,1)$, we have

$$ d \psi (t, s) = |\gamma (t, s)|^2, $$

so that $\psi (t, \cdot )$ is non-decreasing and satisfies $\psi (t, 0) = 0$, $\psi (t, 1) = 1$. The function $s\mapsto d \psi (t, s)$ is positive if and only if q(t, s) does not vanish, which requires

$$\begin{aligned} \sin ^2((1-t)\rho ) + \sin ^2(t\rho ) d\varphi (1)&\nonumber \\ + 2\sin (&(1-t)\rho ) \sin (t\rho ) \sqrt{d\varphi _1}\cos \left( \frac{\omega }{2\sigma }\right) >0. \end{aligned}$$

(12.35)

A sufficient condition for this to hold for all t is that the cosine term is strictly larger than $-1$, which is equivalent to

$$\begin{aligned} \frac{\omega }{2\sigma } \not \equiv \pi \quad (\mathrm {mod}\, 2\pi ). \end{aligned}$$

(12.36)

Since $\omega \in [0, \pi ]$, this condition will be automatically satisfied if $2\sigma > 1$.

Because $\gamma (1) = q_1$, the other end-point must satisfy

$$\begin{aligned} \tilde{\eta }(1, s) = \omega (s)/2\sigma + 2 k(s) \pi , \end{aligned}$$

(12.37)

where k is an integer-valued function. The right-hand side of (12.33) is a linear combination of $q_0$ and $q_1$ with positive coefficients, which implies that the time-continuous angular representation of q(t) starting at 0 cannot deviate by more than $\pi$ from its initial value, i.e.,

$$\begin{aligned} -\pi \le \tilde{\eta }(1,s) \le \pi , \end{aligned}$$

(12.38)

and we also have

$$\begin{aligned} 0 \le \omega (s) \le \pi . \end{aligned}$$

(12.39)

We now (and, unless otherwise specified, for the rest of the discussion) make the assumption that $2\sigma \ge 1$. Under this assumption, $\tilde{\eta }(1, \cdot ) = \omega /2\sigma$ satisfies both (12.37) and (12.38). Since it is clear that only one value of $\tilde{\eta }(1,s)$ can satisfy the two equations together, we find that, for all $s\in \varOmega$, one has $\tilde{\eta }(1,s) = \omega (s)/\sigma$, and the curve $\gamma$ is associated with a trajectory $(\psi , \beta )$ between $({\mathrm {id}}, 0)$ and $(\varphi _1, \xi _1)$.

We have therefore proved that $\gamma$ in (12.33) provides a valid improved solution to the original $(\varphi , \alpha )$ as soon as (12.36) is satisfied, which is true as soon as $2\sigma > 1$.

If $\sigma =1/2$, then (12.36) may not hold for the curve in (12.33). However, the minimum of $U_\sigma (\varphi , \alpha )$ with given $\varphi (1) = \varphi _1$ is still given by the geodesic energy of this curve. To see this, it suffices to consider a small variation $\tilde{a}_0$ of $a_0$ such that $\tilde{a}_0^T a_1\circ \varphi _1 > -1$, so that (12.36) is satisfied with $\tilde{a}_0$ instead of $a_0$, and the minimum energy when starting from $\tilde{a}_0$ is the geodesic energy of the associated great circle. One can then use the fact that U is a geodesic energy for a Riemannian metric on $\mathrm {Diff}\times M$, and combine this with the triangular inequality for the sequence of geodesics going from $({\mathrm {id}}, a_0)$ to $({\mathrm {id}}, \tilde{a}_0)$ then to $(\varphi (1), a_1)$. Indeed, the energy of the sequence is larger than the minimal energy between $({\mathrm {id}}, a_0)$ and $(\varphi (1), a_1)$, but arbitrarily close to the energy of the minimal geodesic between $({\mathrm {id}}, \tilde{a}_0)$ then to $(\varphi (1), a_1)$, itself arbitrarily close to the lower bound in (12.34).

We summarize this discussion in the following theorem.

Theorem 12.6

Assume that $2\sigma \ge 1$, and let $\varphi _1:\varOmega \rightarrow \varOmega$ satisfy $\varphi _1(0) = 0$, $\varphi _1(1) = 1$ and $\partial _s\varphi _1 >0$. Then

$$\begin{aligned}&\inf \left\{ U_\sigma (\varphi , \alpha ): \varphi (1) = \varphi _1, \alpha (0) = a_0, \alpha (1) = a_1\circ \varphi _1 \right\} \nonumber \\&\qquad \,\,= 4\, \mathrm {arccos}^2 \int _\varOmega \sqrt{d\varphi _1} \cos \left( \frac{\mathrm {arccos} (a_0(s)^Ta_1\circ \varphi _1(s))}{2\sigma }\right) \, ds. \end{aligned}$$

(12.40)

Moreover, if $2\sigma > 1$, the minimum is achieved and can be deduced from a geodesic curve $\gamma$ on the unit sphere of $L^2({\mathbb {R}})$.

This induces a distance on M, given by

$$\begin{aligned} d_\sigma (a_0, a_1) = 2 \inf _{\varphi _1}\left( \mathrm {arccos} \int _\varOmega \sqrt{d\varphi _1}\cos \left( {\mathrm {arccos} (a_0(s)^Ta_1\circ \varphi _1(s))}/{2\sigma }\right) \, ds\right) , \end{aligned}$$

(12.41)

minimized over all strictly increasing diffeomorphisms of $\varOmega$.

This theorem is essentially proved in the discussion that precedes it, in which we have left a few loose ends, mostly regarding measurability and dealing with sets of measure 0, that can be tied up without too much effort by an interested reader.

In the two-dimensional case, one can represent functions $\alpha : \varOmega \rightarrow S^1$ in the form $(\cos \theta , \sin \theta )$ for some angle function $\theta$, which is only defined up to the addition of a multiple of $2\pi$. Given such a representation, one can then consider the transform

$$ \mathcal G: (\varphi , \theta ) \mapsto q = \sqrt{d\varphi } (\cos \theta /2\sigma , \sin \theta /2\sigma ) $$

that defines a mapping from $\mathrm {Diff}\times L^2(\varOmega , {\mathbb {R}})$ to the unit sphere of $L^2(\varOmega , {\mathbb {R}}^2)$. Adding a time dependency, we find, using this transform, that

$$ \int _0^1\int _\varOmega \frac{( d\dot{\varphi })^2}{d\varphi }\, ds\, dt + \frac{1}{\sigma ^2}\int _0^1\int _\varOmega |\dot{\xi }|^2 d\varphi \, ds\, dt = 4\int _0^1 \Vert \dot{q}\Vert _2\, dt, $$

so that minimizers on the left can be associated with geodesics on the unit sphere. One can then compute the metamorphosis distance by minimizing the lengths of great circles between, say, $\mathcal G({\mathrm {id}}, \theta _0)$ and $\mathcal G(\varphi _1, \theta _1\circ \varphi _1)$, for a given $\varphi _1$, and optimizing over all angle representations $\theta _0, \theta _1$ of $a_0$, $a_1$ that satisfy the constraint

$$ -2\sigma \pi \le \theta _1 \circ \varphi _1 - \theta _0 \le 2\sigma \pi $$

because (12.38) still needs to hold for any time-continuous angle representation of q. This provides the same distance $d_\sigma$ as the one obtained in Theorem 12.6. Notice, however, that this construction is special to the two-dimensional case. In dimension $d>2$, our reduction to a unit sphere geodesic depended on the end-points $a_0$ and $a_1$, and could not be deduced from a direct transformation applied to the curves themselves, such as $\mathcal G$. The only exception is the case $\sigma =1/2$, for which $\mathcal G$ is equivalent to $(\varphi ,\alpha ) \mapsto \sqrt{d\varphi } \alpha$. This transform, called the “square root velocity transform”, is clearly applicable to arbitrary dimensions. It has been extensively studied in the literature, and we refer to [267] and references within for additional details and applications.

Returning to the two-dimensional case, alternative expressions of the distance can be derived for simple values of $\sigma$ given angle representations $\theta _0$ and $\theta _1$ for $a_0$ and $a_1$. Indeed, the cosine in (12.41) is given by $\cos (\theta _1\circ \varphi _1 - \theta _0)$ if $\sigma =1/2$, by $|\cos ((\theta _1\circ \varphi _1 - \theta _0)/2)|$ if $\sigma =1$, and by

$$ \max (|\cos ((\theta _1\circ \varphi _1 - \theta _0)/4)|, |\sin ((\theta _1\circ \varphi _1 - \theta _0)/4)|) $$

if $\sigma =2$.

The case $\sigma = 1$ for plane curves was first investigated in [307, 313]. It has interesting additional features, and we will see that it coincides with a geodesic distance that we have discussed previously in this chapter [311]. The optimization of $\varphi _1$ (which is still needed to compute the distance) can be done efficiently by dynamic programming, and the reader is referred to [279, 308] for more details.

To conclude this section, we notice that the metamorphosis metric is invariant under the action of rotations, so that one can optimize for a rotation parameter in all cases considered above. The rotation-invariant version of the distance between plane curves for $\sigma =1$, for example, is

$$\begin{aligned} d_{1, \mathrm {rot}} (a_0, a_1) = 2 \inf _{\varphi _1,c} \left( \mathrm {arccos} \int _\varOmega \sqrt{d\varphi _1} \left| \cos \left( \frac{\theta _1\circ \varphi _1(s) - \theta _0(s) -c}{2}\right) \right| \, ds\right) , \end{aligned}$$

(12.42)

where c is a scalar. In higher dimensions, one needs to optimize (12.41) with $a_0$ replaced by $Ra_0$ when R varies over all rotations of the d-dimensional space.

12.7.2 One-Dimensional Case

In one dimension, the previous representation reduces to functions $a:\varOmega \rightarrow \{-1, 1\}$, which does not leave much room to define time-continuous metamorphoses, and the previous approach cannot be extended to this case. One can however bypass this limitation by considering such functions as flat 2D functions, simply replacing the scalar-valued function a by (a, 0). This defines a very special family of piecewise linear curves, to which the previous distance can nonetheless be applied. In this setting, given the functions $a_0$ and $a_1\circ \varphi _1$, one has $\omega (s) =\pi$ if $a_1\circ \varphi _1(s) \ne a_0(s)$ and 0 otherwise. One therefore gets

$$\begin{aligned} d_\sigma (a_0, a_1) = 2 \inf _{\varphi _1}\bigg (\mathrm {arccos} \int _\varOmega \sqrt{d\varphi _1}&\Big ({\mathbf 1}_{a_0(s) = a_1\circ \varphi _1(s)} \nonumber \\ +&\cos \left( \frac{\pi }{2\sigma }\right) {\mathbf 1}_{a_0(s)\ne a_1\circ \varphi _1(s)}\Big )\, ds\bigg )\,. \end{aligned}$$

(12.43)

Notice that this compares functions modulo reparametrization, i.e., $a_0$ and $a_0\circ \varphi$ are considered as identical for any increasing diffeomorphism of $\varOmega$. If this creates too much invariance, one can use the representation ${\mathcal G}$ for the curves (a, 0) without optimizing for reparametrization, which provides the distance

$$\begin{aligned} \delta _\sigma (a_0, a_1) = 2\mathrm {arccos} \int _\varOmega \sqrt{da_0(s)da_1(s)}&\Big ({\mathbf 1}_{a_0(s) = a_1(s)} \nonumber \\ +&\cos \left( \frac{\pi }{2\sigma }\right) {\mathbf 1}_{a_0(s)\ne a_1(s)}\Big )\, ds\,. \end{aligned}$$

(12.44)

12.7.3 The Smooth Case

In this discussion, we placed few regularity conditions on the functions $a\in M$ beyond their measurability. The resulting class of curves includes, in particular, polygonal curves, for which a is piecewise constant. If one wants to restrict to spaces of smooth curves, then some modifications must be made. In particular, the integrals in (12.29) cannot be minimized independently for each s, because we need to ensure that the solution that one obtains is a continuous value of s. The optimal solution is however still given by (12.30) with, this time, $\omega$ being a continuous lift of $s\mapsto \mathrm {arccos} (a_0(s)^T a_1\circ \varphi _1(s))$. One can therefore still reduce the d-dimensional setting to a two-dimensional one.

Taking the same definition for q, we find that the metamorphosis energy is no larger than four times the geodesic energy of q in the unit sphere of $L^2(\varOmega , {\mathbb {R}}^2)$. When proving that the lower-bound is achieved, one finds that the function k(s) in (12.37) must be continuous, hence constant. Here, we can use the fact that one can take $\omega (0)\in [0, \pi ]$ and use the same argument as in the non-smooth case for $s=0$, yielding $k(0) = 0$ and therefore $k(s) = 0$ for all s since k is constant. However, and regardless of the value of $\sigma$, one cannot ensure that (12.35) is satisfied unless the compared curves are close enough (so that their angles are at distance less than $2\sigma \pi$ after registration). When computed between curves that are too far apart, curves in M deduced from geodesics on the sphere will typically develop singularities (and therefore step out of M if this space is restricted to smooth curves).

The smooth case has also been studied in [29], in which a different transform is proposed, leading to a representation of plane curves in a three-dimensional space. More recently, [174] made a study of the smooth case for planar curves with an approach similar to the one we develop here. As pointed out in this reference, when correcting the distance for rotation invariance (similarly to (12.42)), the constant indetermination $2\pi k$ in (12.37) has no impact and one does not need the assumption that $2\sigma \ge 1$ in that case.

12.7.4 Existence of Optimal Metamorphoses

To complete the computation of the optimal metamorphosis, one must still optimize (12.40) with respect to the final diffeomorphism $\varphi _1$. The resulting variational problem is a special case of those studied in [280], which considered the maximization of functionals taking the form

$$ F(\varphi ) = \int _\varOmega \sqrt{d\varphi } f(s, \varphi (s))\, ds $$

over the set $\text {Hom}^+$ of all strictly increasing functions $\varphi :\varOmega \rightarrow \varOmega$ satisfying $\varphi (0) = 0$ and $\varphi (1) = 1$, where f is a function defined on $\varOmega ^2$. Following the notation there, we let

$$ \varDelta _f = \sup _{\psi \in \mathrm {Hom}^+} \int _\varOmega \sqrt{d\psi } f(\psi (s), s)\, ds $$

and define the diagonal band

$$ \varOmega _c = \left\{ (s, s')\in [0,1]^2\ |\ |s-s'| \le c \right\} \,. $$

We give without proof the following result, which is a consequence of Theorem 3.1 in [280].

Theorem 12.7

Assume that $f\ge 0$ is continuous on $\bar{\varOmega }^2$ except on a set G that can be decomposed as a union of a finite number of horizontal or vertical segments. Assume also that, for some c, with

$$ c > \sqrt{1- \left( \frac{\varDelta _f}{\Vert f\Vert _\infty }\right) ^2}\,, $$

there does not exist any non-empty open vertical or horizontal segment (a, b) such that $(a, b)\subset \varOmega _c$ and $f_l$ vanishes on (a, b), where

$$ f_l(x) = \lim _{\delta \rightarrow 0} \inf _{|y-x|<\delta , y\not \in G} f(y) $$

is the lower semi-continuous relaxation of f.

Then there exists a $\varphi ^*\in \text {Hom}^+$ such that $F(\varphi ^*) = \max \{F(\varphi ),\varphi \in \text {Hom}^+\}$. Moreover, if $\varphi$ is a maximizer of F, one has, for all $s\in \varOmega$, $(\varphi (s), s)\in \varOmega _c$.

Fig. 12.5

Two sets of examples of optimal metamorphoses between curves with $\sigma = 0.5$ (left), 1 (center) and 2 (right). Note that, in the first row, the geodesics with $\sigma =0.5$ and $\sigma =1$ grow the missing finger out of the second finger, while that with $\sigma =2$ grows a new thumb. In each image, the geodesic is computed between the outer curve and the inner curve. Scaling is for visualization only (the computation is scale-invariant)

Fig. 12.6

Optimal correspondences estimated with the three metamorphoses in Fig. 12.5. Background images represent the function $\max (f_\sigma , 0)$ in (12.45), the zeros being indicated in black. The solution attempts to move across higher values of the function, cutting through zeros along vertical or horizontal lines if needed. Each panel corresponds to the same panel in Fig. 12.5

Intuitively, f vanishing over vertical or horizontal segments allows for either very small or very large values of $d\varphi$ at very little cost, resulting in optimal solutions that may have stationary regions or jumps. In (12.41) (with $\sigma = 1/2$), this happens when the tangents of the compared curves are perpendicular. When $\sigma =1$, this happens when $\theta _1$ and $\theta _0$ are oriented in opposite directions, i.e., their difference is equal to an odd multiple of $\pi$. For $\sigma >1$, however, the cosine in (12.41) never vanishes. There is no loss of generality in assuming that $f\ge 0$ in this discussion because if $f < 0$ on some rectangle, it is easy to check that any trajectory $(s, \varphi (s))$ that enters this rectangle can be improved if it is replaced by a trajectory that moves almost horizontally and/or almost vertically within the rectangle, with the new cost converging to 0 over this region. This can be used to show that there is no change in the minimizer if one replaces f by 0 within the rectangle.

One can efficiently maximize F by approximating f by a piecewise constant function taking the form

$$ f(s, \tilde{s}) = \sum _{k=1}^n f_k {\mathbf 1}_{R_k}, $$

where $R_1, \ldots , R_n$ is a family of rectangles that partition the unit square and with $f_{k} \ge 0$, $k=1, \ldots , n$. One can then show that the minimization can be performed over piecewise linear functions $\varphi$, which are furthermore linear whenever they cross the interior of a rectangle. The search for the optimal $\varphi$ can then be organized as a dynamic program, and run very efficiently (see [279, 280] for details). This method is used in the experiments presented in Fig.  12.6 in which the optimal correspondence is drawn over an image representing the function $\max (f_\sigma , 0)$, where

$$\begin{aligned} f_\sigma (s, \tilde{s}) = \cos \left( \frac{{\mathrm {arccos}}\cos (\theta _0(s) - \theta _1(\tilde{s}))}{2\sigma }\right) . \end{aligned}$$

(12.45)

Of course, if the compared curves are polygonal, $f_\sigma$ is already piecewise constant.

12.7.5 The Case of Closed Curves

The previous developments assumed that curves were defined over open intervals, and therefore apply mostly to open curves. Closed curves are defined over $T^1 = [0,1]_*$ (using the notation of Sect. 1.2.2), which is the open unit interval where the extremities are identified. The boundary condition on V, which was $v(0) = v(1) = 0$ for functions defined over $\varOmega$, now only requires $v(0) = v(1)$, offering a new degree of freedom, associated with a change of offset, or initial point of the parametrization, represented by the operation $s \mapsto s+_* \delta$ from $T^1$ to itself (where $+_*$ represents the translation along $T^1$, still using the notation introduced in Sect. 1.2.2). We restrict our discussion to the 2D case, in which we assume that the compared curves $a_0$ and $a_1$ have angle representations $\theta _0$ and $\theta _1$.

One can check easily that the distance in Theorem 12.6 is equivariant through this transformation, so that one can define a distance among closed curves that is invariant under rotations and changes of offset by (taking, for example, $\sigma = 1$)

$$\begin{aligned}&\,\,\bar{d}_{1, \mathrm {rot}}(a_0, a_1) \nonumber \\&= 2\inf _{\varphi , c, \delta } \arccos \int _{T^1} \sqrt{\partial _s \varphi (s)} \left| \cos ((\theta _0(x+\delta ) - \theta _1\circ \varphi (x)-c)/2) \right| dx. \end{aligned}$$

(12.46)

Notice that, even when the resulting distance is still attained at a geodesic (or optimal metamorphosis) on the space of functions $a: T^1\rightarrow S^1$, the corresponding curves at intermediate times, however, are not necessarily closed, because the associated closedness condition requires

$$ \int _{T^1} a(s)\, ds = 0, $$

which is not enforced in this approach. Optimal trajectories are therefore not constrained to consist only of closed curves, and would typically become open for $t\in (0,1)$, even though they start and end with closed curves. This method has been applied to obtain the geodesics shown in Fig. 12.5, to which an extra step has been added in order to close the intermediate curves for nicer visualization. This “closing” operation simply consisted in replacing a by $\tilde{a} = (a-\lambda )/|a-\lambda |$, where $\lambda \in {\mathbb {R}}^2$ was adjusted so that $\int _\varOmega \tilde{a}\, ds = 0$.

To correctly define a geodesic distance on spaces of closed curves, one needs to consider the metric induced on the space $M_c$ of functions $a:T^1 \rightarrow S^1$ such that $\int _{T^1} a\, ds = 0$. This space, however, is not invariant under a change of parameters, so that this induced metric is not associated with a metamorphosis. The expression of this constraint in terms of the q function is simple in the case $\sigma = 1/2$, for which $q = \sqrt{d\varphi }\, \alpha = \sqrt{d \varphi }\, a\circ \varphi$ so that, after a change of variable,

$$ \int _{T^1} a\, ds = \int _{T^1} |q(u)| \, q(u)\, du\,. $$

Even in this case, there exists no closed form for the geodesic energy with fixed final reparametrization, but efficient algorithms have been designed to minimize

$$ 4\int _0^1 \Vert \dot{q}(t, \cdot )\Vert ^2_2\, dt $$

subject to the constraints that $\Vert q\Vert _2^2 = 1$, $\int _{T^1} |q|\,q\, du = 0$, $q(0) = q_0$ and $q(1) = q_1$ (see, for example, [161]).

The metric on closed curves also has a nice interpretation in the case $\sigma = 1$. In this case, let f and g denote the two coordinates of the representation $q=\mathcal G(\varphi , \theta )$ multiplied by $\sqrt{2}$, i.e., $f = \sqrt{2d\varphi } \cos \frac{\theta }{2}$ and $g = \sqrt{2d\varphi }\sin \frac{\theta }{2}$, where $\alpha = a\circ \varphi ^{-1} = (\cos \theta , \sin \theta )$. The closedness constraint, which is

$$ \int _\varOmega \cos \theta \circ \varphi ^{-1} \, ds = \int _\varOmega \sin \theta \circ \varphi ^{-1}\, ds = 0, $$

becomes

$$ \int _\varOmega d\varphi \cos \theta \,ds = \int _\varOmega d\varphi \sin \theta \, ds = 0 $$

after a change of variables. Writing $\cos \theta = \cos ^2\frac{\theta }{2} - \sin ^2\frac{\theta }{2}$ and $\sin \theta = 2\cos \frac{\theta }{2}\sin \frac{\theta }{2}$, this is equivalent to

$$ \Vert f\Vert _2^2 -\Vert g\Vert _2^2 = {\big \langle {f}\, , \, {g}\big \rangle }_2 = 0. $$

Because $\Vert f\Vert _2^2 + \Vert g\Vert _2^2 = 2\int _\varOmega d\varphi = 2$, we find that the constraint is equivalent to $\Vert f\Vert _2^2 = \Vert g\Vert _2^2 = 1$ and ${\big \langle {f}\, , \, {g}\big \rangle }_2 = 0$, i.e., to (f, g) forming an orthonormal 2-frame in $L^2(\varOmega )$, and we have

$$ U_1(\varphi , \alpha ) = 2 \int _0^1 \left( \Vert \partial _t f\Vert ^2 + \Vert \partial _t g\Vert ^2\right) \, ds, $$

where the left-hand side is two times the geodesic energy of the path (f, g) in the Stiefel manifold $St (\infty , 2)$ (see Sect. B.6.7). Repeating the arguments made in Sects. 12.7.1 or 12.7.3 in this setting shows that the optimal metamorphosis with fixed $\varphi _1$ is obtained from the shortest length geodesic in $St (\infty , 2)$ connecting the frames $(\sqrt{2} \cos \frac{\theta ^\delta _0}{2}, \sqrt{2} \sin \frac{\theta ^\delta _0}{2})$ and

$$ \left( \varepsilon \sqrt{2\partial _s\varphi _1}\cos \frac{\theta _1\circ \varphi _1}{2}, \varepsilon \sqrt{2\partial _s\varphi _1}\sin \frac{\theta _1\circ \varphi _1}{2}\right) , $$

where $\theta _0$ and $\theta _1$ are angle representations of $a_0$ and $a_1$ and the optimization is made over all possible measurable functions $\varepsilon : \varOmega \rightarrow \{-1,1\}$, and all possible offsets $\delta$, with the notation $\theta _0^\delta (s) = \theta _0(s +_*\delta )$. (The optimization over $\varepsilon$ results from optimizing over all possible angle representations of the two curves.) There is, however, no closed form expression for the geodesic distance on the Stiefel manifold (although equations for geodesics have been described in [99]), and no simple algorithm to solve this optimization problem. Notice that, if one restricts to smooth curves, the search for an optimal $\varepsilon$ is only over constant functions $\varepsilon = \pm 1$ and optimal geodesics can be obtained using a root-finding algorithm over initial conditions of geodesics in $St (\infty , 2)$.

The rotation-invariant version of the distance also provides an interesting representation, because a rotation acting on curves simply induces a rotation of the frame (f, g), and the space of such frames modulo rotation is now the Grassmann manifold $Gr (\infty , 2)$ of two-dimensional subspaces of $L^2({\mathbb {R}})$. The same analysis carries on, the only difference being that one uses now the geodesic distance on the Grassmannian. This geodesic distance can be computed in quasi closed form [216], and is given by $\sqrt{\mathrm {arccos}^2 \lambda + \mathrm {arccos}^2\mu }$, where $\lambda$ and $\mu$ are the singular values of the matrix

$$ \begin{pmatrix} {\big \langle {f_0}\, , \, {f_1}\big \rangle }_2 &{} {\big \langle {f_0}\, , \, {g_1}\big \rangle }_2\\ {\big \langle {g_0}\, , \, {f_1}\big \rangle }_2 &{} {\big \langle {g_0}\, , \, {g_1}\big \rangle }_2 \end{pmatrix}\,, $$

$(f_0, g_0)$, $(f_1, g_1)$ being orthogonal bases of the two spaces that are compared. This closed form, however, does not lead to a simple version of the distance when one optimizes over changes of sign in $(f_1, g_1)$. More analysis of this framework (in the smooth case), including explicit computations of the geodesic equation and of the scalar curvature, can be found in [311].

12.7.6 Alternative Interpretation ($\sigma =1$)

Returning to the case of open curves, we consider the space of curves $m: \varOmega \rightarrow {\mathbb {R}}^2$, not necessarily parametrized by arc-length. Recall the notation $\partial _s f = d f/|d m|$ for the derivative with respect to the arc length. Consider the parametrization-invariant Sobolev metric on curves given by

$$ \Vert \xi \Vert _m^2 = \frac{1}{L(m)} \int _\varOmega |\partial _s \xi |^2\, ds = \frac{1}{L(m)} \int _\varOmega \frac{|d \xi |^2}{|d m|}\, du, $$

where $L(m) = \int _\varOmega |d m(u')|\, du'$ is the length of the curve m.

Consider a trajectory $(t, u) \mapsto m(t, u)$. Let $\ell (t) = L(m(t))$ and let $\varphi (t, u)$ denote the normalized arc-length parametrization along this curve so that

$$ \varphi (t, u) = \frac{1}{\ell (t)} \int _0^u |d m(t, u')|\, du'. $$

Let $\alpha (t, u) = \partial _s m(t, u)$ be the unit tangent. We want to obtain an alternative expression for

$$ \int _0^1 \Vert \dot{m}(t)\Vert ^2_{m(t)}\, dt = \int _0^1 \int _\varOmega \frac{|d \dot{m}|^2}{|d m|}\, du\, dt $$

in terms of $\varphi$ and $\alpha$. We first notice that

$$ \dot{\alpha }= \partial _t\left( \frac{d m}{|d m|} \right) = \left( \left( \frac{ \dot{m}}{|d m|}\right) ^T \alpha ^\perp \right) \, \alpha ^\perp , $$

where $\alpha ^\perp$ is the unit normal, obtained by rotating $\alpha$ by $\pi /2$. Moreover,

$$ d \dot{\varphi }= \partial _t \left( \frac{|dm|}{\ell }\right) = \frac{1}{\ell }(d\dot{m})^T \alpha - \frac{|d m|}{\ell ^2} \dot{\ell }$$

with

$$ \dot{\ell }= \int _\varOmega (d\dot{m})^T \alpha \, du. $$

A little computation yields

$$ \int _\varOmega \frac{(d\dot{\varphi })^2}{d\varphi }\, du = \frac{1}{\ell } \int _\varOmega \frac{((d\dot{m})^T \alpha )^2}{|d m|} \, du - \frac{\dot{\ell }^2}{\ell ^2}. $$

Given this, we can write

$$\begin{aligned} d\dot{m}&= \left( (d\dot{m})^T \alpha \right) \alpha + \left( (d\dot{m})^T \alpha ^\perp \right) \alpha ^\perp \end{aligned}$$

and

$$\begin{aligned} \Vert \dot{m}(t)\Vert ^2_{m(t)}&= \frac{\dot{\ell }^2}{\ell ^2} + \int _\varOmega \frac{(d\dot{\varphi })^2}{d\varphi } \, du + \int _\varOmega |\dot{\alpha }|^2 d\varphi \, du. \end{aligned}$$

The last two terms integrated over time provide the function $U_1$ obtained in (12.28) with $\sigma =1$. This expression of the metric treats the pair $(\varphi , \alpha )$ and the curve length independently, and the geodesic length can be optimized separately for $\ell$ and $(\varphi , \alpha )$, yielding a geodesic distance given by

$$\begin{aligned}&d(m_0, m_1)^2 = \\&\qquad \left( \log \frac{L(m_1)}{L(m_0)}\right) ^2 + 4\, \mathrm {arcos}^2 \int _\varOmega \sqrt{d\varphi _0\, d\varphi _1}\left| \cos \left( \frac{\theta _1\circ \varphi _1 - \theta _0\circ \varphi _0}{2}\right) \right| \, du, \end{aligned}$$

where $\varphi _0, \varphi _1$ are the arc-length reparametrizations of $m_0$, $m_1$ and $\theta _0$, $\theta _1$ are their angle representations expressed as functions of the arc lengths. Furthermore, the transformation $m \mapsto m/L(m)$ provides a Riemannian submersion onto the space of curves with length 1 (or the space of curves modulo scaling), and the resulting projected distance is exactly the one obtained via the metamorphosis approach.