Fitting Smooth Paths on Riemannian Manifolds: Endometrial Surface Reconstruction and Preoperative MRI-Based Navigation

Abstract

We present a new method to fit smooth paths to a given set of points on Riemannian manifolds using $C^1$ piecewise-Bézier functions. A property of the method is that, when the manifold reduces to a Euclidean space, the control points minimize the mean square acceleration of the path. As an application, we focus on data observations that evolve on certain nonlinear manifolds of importance in medical imaging: the shape manifold for endometrial surface reconstruction; the special orthogonal group SO(3) and the special Euclidean group SE(3) for preoperative MRI-based navigation. Results on real data show that our method succeeds in meeting the clinical goal: combining different modalities to improve the localization of the endometrial lesions.

1 Introduction

Surface reconstruction problem has been widely studied because of its importance in different applications such as medical imaging, computer graphics, mechanical simulations, virtual reality, etc. Particularly, the reconstruction of surfaces from given 3D point clouds is important since they are frequently used in medical imaging and computer graphics [4]. For example, one can use a continuous formulation using PDEs and compute the solution as an implicit surface, which is usually the zero level set of a sufficiently smooth function. Therefore, one can control the resulting surface by adding physics-inspired constraints depending on geometry or external forces [17]. However, when the given data is a set of curves one needs to find an optimal fitting between them by taking into account their parametrization and the non-linearity of their spatial evolution. In this work, we formulate the problem of reconstructing a surface from a given set of curves as a smooth path fitting on the space of curves.

Path fitting on manifolds has been addressed in the literature with different approaches and for various purposes. Generic path fitting methods on manifolds include splines on manifolds [8], rolling procedures [6], subdivision schemes [10], gradient descent [13], and geodesic finite elements [15]. Interpolation of rotations (where the manifold $\mathcal {M}$ is the special orthogonal group $\mathrm {SO}(3)$) is useful in robotics for motion planning of rigid bodies and in computer graphics for the animation of 3D objects [11]. More related to our work, morphing between shapes can be tackled as an interpolation problem on shape space [5].

With an estimated prevalence of 10 %, endometriosis is one of the most common clinical problems affecting women of reproductive age [16]. Various structures can be affected by endometriosis, including the uterosacral ligaments, rectosigmoid colon, vagina, uterus, and bladder [9]. Generally, the absence of an accurate preoperative diagnosis leads to unnecessary surgeries even with possible complications. To reduce the risk of such failure, surgeons need to know exactly the number, the size, the locations and the depth of infiltration of endometrial cysts before the intervention. A preoperative mapping of the lesions is crucial for managing the disease. This mapping can be well defined through medical imaging techniques such as Magnetic Resonance Imaging (MRI) and 2D Transvaginal Ultrasound (TVUS) [3].

When using these two modalities separately, there is an increased risk of false negative and false positive, due to their different advantages and inconvenients [1]. Moreover, they complement each other excellently, as lacking information from one modality can be provided by the other in terms of spatial, contrast or temporal resolution. For instance, lesions hard to detect in MRI are revealed at TVUS acquisition (due to an approximate distance between the probe and the tissue with relatively free movement). That is why registration is helpful here in order to show TVUS with lesions into MRI volume.

After TVUS-MRI registration [14], the position p of the ultrasound probe and its orientation n can be precisely determined within the 3D MRI volume and around the TVUS curve. They form a certain plane $\Pi _{n,p}$. The MRI views corresponding to the intersections of the MRI volume with the registered planes $\Pi _{n,p}$ are used to reproduce the probe movement. As a result, the clinician is able to explore the MRI volume to search for very close and clear views including the pelvic organs. We illustrate this idea by fitting a smooth path $\gamma$ to different key positions of $\Pi _{n,p}$ viewed as points on Riemannian manifolds like SO(3) (rotations) or SE(3) (rotations and translations). For simplicity, we will refer to the resulting path $\gamma$ (in both cases) as a preoperative MRI-based navigation to locate and characterize lesions.

Clinical Context. In practice, TVUS is done on women presenting symptoms corresponding to the presence of endometrial tissues. When 2D TVUS does not provide enough information to confirm the diagnosis, MRI is performed. Given TVUS and MRI, practicians select a set of corresponding landmarks to define surrounding organ boundaries in both images, manually. These anatomical correspondences between structures on MRI and TVUS are then used to measure and locate lesions, separately, which is still a challenging task.

The rest of this paper is organized as follows. Section 2 describes the formulation of our path fitting method. Section 3 presents two applications of this method: (i) MRI surface reconstruction as a path on shape manifold and (ii) navigation in the MRI volume as a path on SE(3). We close this paper with a brief summary in Sect. 4.

2 Problem Formulation

Given a finite set of points $p_0, \ldots , p_n$ at time instants $t_0, \ldots , t_n$ on a Riemannian manifold $\mathcal {M}$, our goal is to construct a smooth path $\gamma$ that interpolates $p_i$ at $t_i$ for $i=0,\dots , n$. When the manifold $\mathcal {M}$ reduces to the Euclidean space $\mathbb {R}^m$, we propose a method that generates a piecewise-Bézier $C^1$ path with minimal mean squared acceleration. This method improves on the technique recently proposed in [5], where the choice of the path velocity direction at the interpolation points was suboptimal even in the Euclidean case. Let the function $t \mapsto \beta _k (t;b_0,\dots ,b_k)$ denote the Bézier curve of order k and $b_0,\dots ,b_k$ the control points. For simplicity, we set time instants $t_i=i$ with straightforward extension to general timestamps. In this work, we only use Bézier curves of degree 2 and 3, expressed in $\mathbb {R}^m$ as:

$$\begin{aligned} \beta _2(t;b_0,b_1, b_2)= & {} b_0(1-t)^2 + 2b_1t(1-t) + b_2t^2 \end{aligned}$$

(1)

$$\begin{aligned} \beta _3(t;b_0, b_1, b_2, b_3)= & {} b_0(1-t)^3 + 3b_1t(1-t)^2 + 3b_2t^2(1-t) + b_3t^3 . \end{aligned}$$

(2)

Using these polynomials, we construct a $C^1$ curve $\gamma : [0,n] \rightarrow \mathbb {R}^m$ (we call it a piecewise-Bézier curve) consisting of Bézier curves of degree 2 for the extremal segments and of degree 3 for the others:

$$\begin{aligned} \gamma (t) = {\left\{ \begin{array}{ll} \beta _2(t;p_0,b_1^-,p_1) &{} \text {if }t\in [0,1]\\ \beta _3(t-(i-1);p_{i-1},b_{i-1}^+,b_i^-,p_i)&{} \text {if }t\in [i-1,i] , \ i=2,\dots ,n-1 \\ \beta _2(t-(n-1);p_{n-1},b_{n-1}^+,p_n)&{} \text {if }t\in [n-1,n], \end{array}\right. } \end{aligned}$$

(3)

where $b_i^+$ and $b_i^-$ are the control points respectively on the right and left hand side of the interpolation point $p_i$. One can observe that this formulation satisfies the interpolation conditions $\gamma (t_i) = p_i$. The differentiability condition of the curve is ensured by imposing velocities to be equal on the left and right of the interpolation points $p_i$s, which allows us to express $b_i^+$ in terms of $b_i^-$:

$$\begin{aligned} \begin{array}{rcl} b_1^+ &{}=&{} \frac{5}{3}p_1 - \frac{2}{3}b_1^-,\\ b_i ^+ &{}=&{} 2p_i - b_i^- \quad i=2,\dots ,n-2,\\ b_{n-1}^+ &{}=&{} \frac{5}{2}p_{n-1} - \frac{3}{2}b_{n-1}^-. \end{array} \end{aligned}$$

(4)

The resulting optimization problem is an unconstrained minimization with $b_i^-$ as variables and the mean square acceleration of the piecewise-Bézier curve as the objective function, defined as follows:

(5)

As the Bézier segments are linear functions of the control points, the objective function is quadratic. The optimal solution is then computed as a critical point of the gradient, which gives rise to a linear system of the form: $AX = CP$ where $X = \begin{bmatrix}b_1^-&\dots&b_{n-1}^-\end{bmatrix}^T \in \mathbb {R}^{n-1\times m}$, $P = \begin{bmatrix}p_0&\dots&p_{n}\end{bmatrix}^T \in \mathbb {R}^{n+1 \times m}$, $A\in \mathbb {R}^{n-1\times n-1}$ and $C\in \mathbb {R}^{n-1\times n+1}$ are tridiagonal matrices with coefficients:

$$\begin{aligned} A\mathtt {\scriptstyle (1,1:2)}= & {} \begin{bmatrix} 64&24\end{bmatrix},\end{aligned}$$

(6)

$$\begin{aligned} A\mathtt {\scriptstyle (2,1:3)}= & {} \begin{bmatrix} 24&144&36 \end{bmatrix},\end{aligned}$$

(7)

$$\begin{aligned} A\mathtt {\scriptstyle (i,i-1:i+1)}= & {} \begin{bmatrix} 36&144&36 \end{bmatrix}, i=3:n-2 \end{aligned}$$

(8)

$$\begin{aligned} A\mathtt {\scriptstyle (n-1,n-2:n-1)}= & {} \begin{bmatrix} 36&144\end{bmatrix} \end{aligned}$$

(9)

and

$$\begin{aligned} C\mathtt {\scriptstyle (1,1:2)}= & {} \begin{bmatrix} 16&72\end{bmatrix},\end{aligned}$$

(10)

$$\begin{aligned} C\mathtt {\scriptstyle (2,2:3)}= & {} \begin{bmatrix} 60&144 \end{bmatrix},\end{aligned}$$

(11)

$$\begin{aligned} C\mathtt {\scriptstyle (i,i:i+1)}= & {} \begin{bmatrix} 72&144 \end{bmatrix}, i=3:n-2 \end{aligned}$$

(12)

$$\begin{aligned} C\mathtt {\scriptstyle (n-1,n-1:n+1)}= & {} \begin{bmatrix} 72&132&-24\end{bmatrix}. \end{aligned}$$

(13)

Since A is invertible, the unique solution is given by:

$$\begin{aligned} X=A^{-1}CP=DP \quad \text {with} \quad \sum _{j=0}^{n}D_{ij}=1, \forall i. \end{aligned}$$

(14)

We generalize this result on a Riemannian submanifold $\mathcal {M}$ embedded in a Euclidean space $\mathcal {E}$. In order to make this possible for a Riemannian manifold $\mathcal {M}$, one needs the tangent space $T_{p}(\mathcal {M})$ of $\mathcal {M}$ at a given point p, the Riemannian exponential map $Exp _{p}$, and its inverse $Log _{p}$ (see [2, 12, Sect. 4] for a formal definition on specific manifolds). Bézier curves (1) and (2) are generalized by means of the Riemannian De Casteljau’s algorithm (see, e.g., [5] for the literature). Conditions (4), of the form $b^+_i = p_i + \alpha (b_i^- - p_i)$ generalize to $b_i = Exp _{p_i}(\alpha Log _{p_i}(b_i^-))$ and ensure that $\gamma$ on $\mathcal {M}$ is $\mathcal {C}^1$. It then remains to generalize (14) for which we propose two approaches:

• 1. method 1: In order to solve the fitting problem on a linear space as for the Euclidean case, we proceed as follows. We initially choose an arbitrary point among the $p_i$s that we will call a root point, e.g., $p_0$. Next we map the rest of the given data points to $T_{p_0}(\mathcal {M})$ as $\tilde{p_i}= Log _{p_0}(p_i)$. Then we solve the linear system $\widetilde{X} = D\widetilde{P}$. Finally, we project the solution $\widetilde{X}$ back to $\mathcal {M}$ as $b_i^{-}= Exp _{p_0}(\tilde{x}_i)$. Numerically, the choice of the root point may affect the quality of the solution.

• 2. method 2: In order to avoid the dependance on the choice of a single root point, an alternative is to choose $p_i$ as the root point for row i of (14). Thus, for each $i=1\ldots n-1$ we map the rest of data points into the tangent space $T_{p_i}(\mathcal {M})$ using the logarithmic map $Log _{p_i}$. The mapped data are then given by $\widetilde{P}=\begin{bmatrix} Log _{p_i}(p_0) \dots Log _{p_i}(p_{n})\end{bmatrix}$ and the solution is given by $\widetilde{x}_i= Log _{p_i}(b_i^-)=\sum _{j=0}^{n}D_{ij} Log _{p_i}(p_j)$. Therefore, each $\widetilde{x}_i$ is mapped back to $\mathcal {M}$ as $b_i^{-} = Exp _{p_i}(\tilde{x}_i)$.

As stated earlier, our method minimizes the mean square acceleration objective when $\mathcal {M}$ is a Euclidean space. This follows from the fact that, when $\mathcal {M}$ is a Euclidean space, we have $Log _{p}(b) = b-p$; this, along with the property $\sum ^n_{j=0}D_{ij}=1$, makes the root point cancel out and recover (14). Even if they seem to provide good solutions, the proposed generalizations of (14) do not guarantee a minimal mean square acceleration when $\mathcal {M}$ is nonlinear. Nevertheless, we will use the second method for the experiments as it was observed to be more efficient than the first one, at least when $\mathcal {M}$ is the unit sphere $S^2$, the special orthogonal group SO(3), or the special Euclidean group SE(3).

3 Experimental Results

In this section, we present two different applications of our framework. On both cases, results are given using real data images obtained from patients with endometriosis characterized by different localizations and depths of infiltration. Figure 1(a,b) shows examples of corresponding landmark curves (uterus, rectum, lesions) in TVUS and MR images. The curves in TVUS have been deformed along during the exam due to the transducer’s pressure as shown in Fig. 2(a).

3.1 Endometrial Surface Reconstruction

As a first application, we performed our path fitting method to reconstruct the endometrial surface $S_{MRI}$ from curves in three steps. First, a radiologist was asked to select different slices (from 4 to 7) and segment curves as boundaries of an interest zone on each slice. Second, we represented each curve as a point on the shape manifold. Note that we aligned and fixed the starting point of each curve (Fig. 1(f)). As given time indexes that have spatial meaning in this case, we used the $z-values$ for each curve from its corresponding slice. Third, we used a modified version of [7] to compute a geodesic path between any two points on shape space. Finally, we applied our method as detailed in Sect. 2 to construct $S_{MRI}$ (Fig. 1(c) and (h)) as a $C^1$-fitting path between curves (see $\Vert \dot{\gamma }(t) \Vert$ in Fig. 1(d)). To give an idea about the quality of the reconstructed surface, we show an example of $S_{MRI}$ constructed from a set of 3D curves (f) using a linear interpolation between them (g) and our method (h).

Fig. 1.

Application 1: from TVUS and MR images (a,b), we reconstruct the MRI surface (c) as a path $\gamma$ interpolating 4 key curves extracted on MRI slices. The velocity of $\gamma$ is continuous (d). From the 6 key curves (e) with fixed starting points (f) we reconstruct (g) as a linear interpolation and (h) with our method.

3.2 MRI-based Navigation

The problem of 2D-3D TVUS-MRI registration was recently addressed by Samir et al. in [14]. The basic idea of their method is a follows. First, they manually segment the cylindrical endometrial tissue surface $S_{MRI}$ from the MRI image and the planar contour from the corresponding TVUS image. This registration provides a one-to-one correspondence of curves between TVUS and MRI. We will refer to the resulting intersecting plane by $\Pi _{n,p}$ (Fig. 2(b)).

Fig. 2.

(a) An illustration of TVUS movement during the exam and (b) is an example of the intersection between MRI volume and a plane $\Pi _{n,p}$ by means of interpolation. (c) and (d) Are two examples of fitting paths on SO(3) and SE(3), respectively.

To look for a very close and more clear views including the pelvic organs in the MRI volume, one has to consider the movement of the probe (rotations only or rotations and translations) to search around $\Pi _{n,p}$. We illustrate this idea by fitting smooth paths of different key positions of $\Pi _{n,p}$ on SO(3) (rotations: Fig. 2(c)) and on SE(3) (rotations and translations: Fig. 2(d)). In both cases, we consider the resulting path $\gamma$ as a preoperative MRI-based navigation. It is clear from Fig. 3 that such navigation has more chances to locate the extent of lesions than TVUS and MRI when used separately. This idea is illustrated in Fig. 3(frames (2, 4, 6, 9)) where new views as points from $\gamma$ provide more accurate characterization of the lesion. In this case, the red coloured areas denotes the region of interest (delineated by the expert) which were not clear enough or non visible on sagittal, coronal, and axial views.

Fig. 3.

Application 2: examples of uniformly sampled frames from MRI navigation obtained as a fitting on SE(3).

4 Summary

In this work, we have proposed a new Riemannian framework for a MR-based navigation system to locate and characterize endometrial tissues in order to improve the preoperative diagnosis. The information is available in the form of landmark curves (extended to surfaces) in the 3D MRI and curves in the 2D TVUS images. Our approach embeds the TVUS intersecting plane into MRI and use a new path fitting method to construct an MRI-based navigation. This way, we have reached very close and more clear views including the pelvic organs in the MRI volume.