Introduction

The treatment of low-risk prostate cancer presents a common clinical dilemma. The standard approach of curative whole gland therapy is associated with significant impact on quality of life, particularly sexual function. The alternative, active surveillance is associated with a low, but real risk of progression and requires a combination of effective communication skills of the physician and a calm, secure and compliant patient. Over the last decade, more accurate localization of cancers within the prostate has generated an interest in focal therapy that targets only cancer rather than the entire prostate, as a less radical approach. Prostate focal therapy has been approved by the National Comprehensive Cancer Network (NCCN) for the treatment of low-risk disease as it may provide the best balance between cancer control and maintenance of quality of life.

Fig. 1
figure 1

(left) Ultrasound guided brachytherapy: trans-perineal insertion of radioactive seeds under ultrasound guidance; (center) TRUS image of the prostate; (right) Corresponding MRI slice. The target lesion (yellow contour) is visible in MRI but not in TRUS

Full size image

Image-guided brachytherapy was developed in the last 15 years as an internal radiotherapy approach to whole gland treatment. Trans-rectal ultrasound (TRUS) is used to generate a 3D volume of prostate slices. Using a dosimetry software, the optimal placement for a set of radioactive seeds is planned to achieve maximal therapeutic effect inside the gland, while attempting to limit dose to critical structures. The prostate, urethra and other anatomical structures are usually manually segmented by contouring on the 2D TRUS slices. Eventually, the seeds are implanted into the prostate gland via the perineum, under real-time TRUS guidance. (Fig. 1, left).

As a minimally invasive approach, brachytherapy is an appealing framework for focal therapy. However, TRUS has a low sensitivity for detection of prostate cancer and the contrast (Fig. 1, center) is generally insufficient to distinguish the target lesion from normal prostate tissue. Multi-parametric MRI (Fig. 1, right) usually offers a much better contrast between the lesion and surrounding tissues as shown in Fig. 1 (right, yellow contour). Brachytherapy is difficult under real-time MRI [1–3], as the required extended lithotomy position is not achievable, thus preventing good access to the prostate due to (Fig. 1, left) pubic arch interference. In addition, the inherent restrictions and overhead imposed by MRI’s strong magnetic field are too demanding in practice.

In the last years, TRUS–MRI registration was proposed as a mean to improve prostate biopsy accuracy. Several registration algorithms were published for the task very recently [4–7]. Considering the very different visual appearance of TRUS and MRI prostate pixels, it is problematic to perform an intensity-based registration between the modalities. Therefore, TRUS–MRI registration has been generally approached as a surface alignment problem in which the prostate external surface must have been previously segmented in both modalities [4, 6, 7]. In [4], the prostate is manually contoured in MRI and aligned to a probabilistic prostate map of the TRUS, first affinely and then by elastic registration. The intraoperative phase of the method is claimed to be free of manual intervention.

In order to cope with incomplete visibility of the prostate contour, a combined Gaussian mixtures–finite elements model (GMM-FEM) framework is proposed in [6] that handles missing data (the incomplete prostate contour) using a GMM estimated by the expectation minimization (EM) algorithm. Biomechanical regularization is provided via finite elements modeling.

In [7], matching between MRI and TRUS prostate contour points is performed using the Bhattacharyya distance on 2D shape-context representations [8]. A 2D diffeomorphic thin-plate-spline (TPS) deformation field (DF) is then inferred from the 2D points correspondences. The method is developed and experimented on a set of manually selected TRUS–MRI 2D slices that are assumed to have the same anatomical position. In [5], a pixel-based approach is proposed in which a modality independent neighborhood descriptor (MIND) is extracted for each MRI and TRUS pixel. Registration is computed by minimizing over the deformation field a point-wise difference measure of the descriptor. Registration requires manual selection of 6 matching landmark points in between the modalities for the initial rigid body transform. The fact that no prostate segmentation is performed requires the calculation of the DF across the whole field of view, including outside the prostate. Consequently, the recovered DF may be sub-optimal for the prostate as it has to accommodate the deformations of surrounding tissues as well.

In [4–7], image-guided trans-rectal biopsy is the goal of TRUS–MRI registration. In this paper, the goal of registration is a focal treatment of prostate cancer by brachytherapy. Providing at least one representative tissue sample into a lesion is usually sufficient for categorization by biopsy. However, in order to treat the same lesion by brachytherapy it is necessary to position many radioactive seeds into the lesion according to the spatial distribution defined by shape-dependent dosimetry calculations to ensure complete lesion destruction. The accuracy requirement is clearly more demanding than for biopsy.

There are very few published works that considered TRUS–MRI fusion for brachytherapy. In [9], rigid followed by elastic registrations are performed between manual contours of the prostate in TRUS and MRI. A Hausdorff distance between point and surface is used in the cost function expression.

Previous methods using deformable registration have not been sufficiently validated to increase the safety of a true focal approach. In previous studies [4–7, 9], the registration accuracy was assessed by measuring residual distance between corresponding anatomical landmarks (e.g., the urethra or calcifications) that were visible in both modalities, after the recovered warping was applied. In practice, due to extreme contrast, SNR and resolution difference, it is very challenging to match accurately between anatomical landmarks in TRUS and MRI, even for a trained expert: the whole process is very subjective. In order to provide a more objective validation of the registration accuracy, we use brachytherapy seeds or fiducial gold seeds implanted during real procedures as non-anatomical landmark, jointly observable in both modalities.

Thus, we propose a deformable registration approach that relies on a 3D extension of the shape context [8], providing a rich descriptor for the neighborhood shape of each point to be matched. To our knowledge, it is the first time this approach is performed for TRUS–MRI registration validation. In the next sections, the proposed methods will be reviewed in details (see “Methods” section) and validated quantitatively (see “Experiments and results” sections). A discussion and conclusions complete the paper (see “Discussion and conclusions” section).

Methods

The main steps of the proposed method are given by the block diagram in Fig. 2. The prostate is first contoured, slice by slice in each MRI slice. The resulting 3D point cloud and the MRI volume are then interpolated to match TRUS volume resolution, which is usually higher than MRIs. The contours are interpolated by fitting a 3D mesh using the ball pivoting algorithm [10] and computing its intersection with the re-slicing planes of the interpolated MRI volume. The resulting contours are projected onto the TRUS volume, to be used as initial TRUS contours. For this purpose, a single mouse click is required near the center of the gland section, in the first TRUS slice where it appears, which corresponds to the prostate’s base. At this point, manual editing may be performed by dragging locally the projected contour points in order to delineate accurately the TRUS prostate shape. MRI contours projection and editing are the only manual steps performed during brachytherapy as the TRUS images are not available beforehand. In order to compensate for pose difference between the modalities, a robust rigid body transform is computed by a RANSAC [11] implementation of iterative closes point (ICP) algorithm [12]. At each ICP iteration, a small subset of the matched closest contour point pairs, \(\alpha \), is repeatedly drawn at random and used to fit a rigid body transform. For each transform, the residual mean squared distance (MSD) is computed between the matched points of the whole dataset after transformation of the MRI points. Eventually, the minimal MSD transform is selected and the process is repeated until convergence of the ICP algorithm. RANSAC is helpful in reducing the influence of deformed areas during the fitting of the rigid body transformation.

Fig. 2
figure 2

Main steps of the proposed method

Full size image

Following rigid registration, it is still necessary to compensate for the local differences in TRUS/MRI prostate shape arising from dissimilarities in diameter, shape and orientation of the endorectal probes and coils used for TRUS and MRI, respectively. In order to capture local shape around each point in the TRUS and MRI contours, we call upon the 3D shape-context descriptor proposed by [13]. For any given point p, a spherical neighborhood is centered on p (Fig. 3). The support region is divided into bins by \(J+1\) equally spaced boundaries in the azimuth dimension \(\left( {\varphi _0 \ldots \varphi _J } \right) \), \(K+1\) in the elevation dimension \(\left( {\theta _0 \ldots \theta _K } \right) \), and \(L+1\) logarithmically spaced boundaries along the radial dimension \((R_0 \ldots R_L )\). A sample bin containing 3 points is highlighted (Fig. 3, red). The logarithmic sampling provides robustness to shape distortion in the radial direction. The boundary radius \(R_l \) for the lth radial division is given by (Eq. 1):

$$\begin{aligned} R_l =\mathrm{exp}\left\{ \ln \left( r_\mathrm{min} \right) + \frac{l}{L}\hbox {ln}\left( \frac{r_\mathrm{max}}{r_\mathrm{min}} \right) \right\} \end{aligned}$$
(1)

where \(r_\mathrm{min} \) and \(r_\mathrm{max} \) are the radius of the first and last radial divisions, respectively. The shape-context descriptor is inherently shift invariant, and rotation invariance may be further obtained. For this purpose, the orientation of the bins in the spherical neighborhood at any point p may be defined with regard to the local estimate of the surface normal.

Fig. 3
figure 3

The 3D shape-context spherical neighborhood centered at point p and divided into bins by \(J+1\) equally spaced boundaries in the azimuth dimension \(\left( {\varphi _0 \ldots \varphi _J } \right) \), \(K+1\) in the elevation dimension \(\left( {\theta _0 \ldots \theta _K } \right) \), and \(L+1\) logarithmically spaced boundaries along the radial dimension \((R_0 \ldots R_L )\). A sample bin containing 3 points is highlighted (red)

Full size image

The shape-context descriptor at any point p is defined by the normalized histogram vectors for the points falling in the bins of the spherical neighborhood defined above.

Once the descriptor has been assigned to each point in both modalities, a cost is computed for the matching of each possible MRI-TRUS point pair. The cost, \(\hbox {C}_{{i,j}} \), of matching between points i and j belonging to MRI and TRUS contours, respectively, is given by [8] (Eq. 2):

$$\begin{aligned} \hbox {C}_{i,j}=\left\{ {{ \begin{array}{ll} \displaystyle \frac{1}{2}\mathop \sum \limits _{\mathrm{k}=1}^{\hbox {K}} \frac{\left[ {\hbox {g}_i \left( \hbox {k} \right) -\hbox {h}_j \left( \hbox {k} \right) } \right] ^{2}}{\hbox {g}_i \left( \hbox {k} \right) +\hbox {h}_j \left( \hbox {k} \right) }&{} \Vert {i\left( {x,y,z} \right) -j\left( {x,y,z} \right) \Vert \le d_\mathrm{max} } \\ {+\infty }&{} \mathrm{else} \\ \end{array} }} \right. \nonumber \\ \end{aligned}$$
(2)

where \(\hbox {g}_i \left( \hbox {k} \right) \) and \(\hbox {h}_j \left( \hbox {k} \right) \) stand for the counts in bin k of the normalized shape-context histogram at points i (MRI volume) and j (TRUS volume), respectively. Since the pose difference has already been resolved in the preceding rigid body step, it is reasonable to assume that corresponding MRI and TRUS points are now located at small distance. In this work, in order to avoid matching excessively distant points that have similar shape-context descriptors, the cost between points is artificially set to \(+\infty \) when their mutual distance exceeds a preset threshold denoted by \(d_\mathrm{max} \).

The optimal assignments that minimize the overall cost of matching between the MRI and TRUS point pairs are computed by the well-known Hungarian algorithm [14]. In Fig. 4 (left), sets of MRI (blue) and TRUS (red) 3D contour points are shown. Line segments connect between point pairs that were matched at minimal cost, given their respective shape-context descriptors, by the Hungarian algorithm.

Using the point pairs matched across the modalities, a smooth and dense nonlinear warping can be computed. As prostate is smoothly deformed under the pressure exercised by endo-rectal probes, B-spline grids provide a convenient representation for a dense and smooth deformation field.

Fig. 4
figure 4

(left) MRI (blue) and TRUS (red) 3D contour point pairs matched by the Hungarian algorithm. The line segments connects between point pairs matched at minimal cost according to their respective shape-context descriptors; (right) the resulting deformation field computed for a sample MRI slice

Full size image

For this purpose, we fit a rectangular 3D mesh of B-spline control points that transforms the MRI points into the matched TRUS points with minimal error. The B-spline deformation field T(x, y, z) at position (x, y, z) is given by [15] (Eq. 3):

$$\begin{aligned} T\left( {x,y,z} \right) =\mathop \sum \limits _{l=0}^3 \mathop \sum \limits _{m=0}^3 \mathop \sum \limits _{n=0}^3 B_l \left( u \right) B_m \left( v \right) B_n \left( w \right) \emptyset _{i+l,j+m,k+n}\nonumber \\ \end{aligned}$$
(3)

where \(B_l \) is the lth basis function of the B-spline,

$$\begin{aligned} B_0 \left( u \right)= & {} \frac{\left( {1-u} \right) ^{3}}{6}\\ B_1 \left( u \right)= & {} \left( {3u^{3}-6u^{2}+4} \right) /6\\ B_2 \left( u \right)= & {} \left( {-3u^{3}+3u^{2}+3u+1} \right) \\ B_3 \left( u \right)= & {} u^{3}/6 \end{aligned}$$

\(\emptyset _{i,j,k} \) is an \(n_x \times n_y \times n_z \) mesh of control points with respective spacing \(\delta _x ,\delta _y ,\delta _z \), and \(i=\mathrm{floor}\left( {\frac{x}{n_x }} \right) -1,\)

$$\begin{aligned} j= & {} \mathrm{floor}\left( {\frac{y}{n_y }} \right) -1, \quad k=\mathrm{floor}\left( {\frac{z}{n_z }} \right) -1, \quad \nonumber \\ u= & {} \frac{x}{n_x }-\mathrm{floor}\left( {\frac{x}{n_x }} \right) , \quad \nonumber \\ v= & {} \frac{y}{n_y }-\mathrm{floor}\left( {\frac{y}{n_y }} \right) , w=\frac{z}{n_z }-\mathrm{floor}\left( {\frac{z}{n_z }} \right) . \end{aligned}$$

The deformation field is shaped by the position of the control points. Large spacing between the control points will constrain the deformation field to be more global. Conversely, tight spacing will allow for stronger local deformations. In order to allow for the efficient recovery of large local deformations, a multi-resolution scheme is adopted. The deformation field estimation is progressively refined by a hierarchy of meshes with control points spacing reduced by half at each level [16]. The final deformation field is given by (Eq. 4):

$$\begin{aligned} T\left( {x,y,z} \right) =\mathop \sum \limits _{s=1}^S T^{s}\left( {x,y,z} \right) \end{aligned}$$
(4)

where S is the number of considered mesh resolutions and \(T^{s}\) is the computed deformation field for the sth mesh resolution. In Fig. 4 (right), the resulting deformation field computed for the set of matched 3D points (left) is shown for a sample MRI slice. Eventually, the recovered deformation field is applied to the MRI prostate voxels in order to project them onto the corresponding TRUS voxels.

Experiments

The algorithms described in previous section were implemented in non-optimized MATLAB code. A purposely designed GUI was developed (Fig. 5). The left and right upper windows show the MRI and TRUS scans, respectively, in axial view. The bottom windows show the corresponding coronal views.

The method was validated on a retrospective set of ten real cases for which TRUS and MRI of the prostate were both available. Axial MRI scans were acquired on a 1.5 T Signa MRI scanner (GE Healthcare, Milwaukee, USA) with endorectal coil and voxel size equal to \(0.27\times 0.27\times 3.6\,\hbox { mm}^{3}\). Three-dimensional TRUS scans were acquired with a transerctal Targetscan (8–12 MHz, transverse mode, Envisioneering medical technologies, St. Louis, USA) ultrasound system with voxel size equal to \(0.12\times 0.12\times 1\,\hbox { mm}^{3}\). The transducer is driven automatically along the hollow tip of the probe, enabling the acquisition of full-gland 3D axial stacks without moving the probe casing with regard to the patient. Out of 10 cases, 4 had previously undergone the implantation of 3 gold seeds fiducials in the prostate in the framework of a programmed external beam radiotherapy (EBRT). MRI and TRUS were both acquired after implantation but before EBRT to avoid imaging the irradiated prostate. The 4 cases are designated as EBRT patients in the following experiments. The remaining 6 cases had previously undergone low-dose brachytherapy. Both TRUS and MRI were acquired at least 6 months after the procedure to skip transient iatrogenic effects such as edema or seed migration. Following brachytherapy, 30–60 radioactive seeds were permanently implanted in the prostate. The 6 cases are designated as brachytherapy patients in the following experiments. Gold fiducials or brachytherapy seeds were utilized as landmark points, visible in both modality, for the quantitative assessment of TRUS–MRI registration accuracy. In Fig. 5, brachytherapy seeds (yellow contours) appear hypo-intense in MRI and hyper-intense in TRUS. In each case, every visible landmark was segmented in 3D by manual contouring in each slice it appeared, in both modalities, and its position was defined by the contours’ centroid. This was done by ZS (Brachytherapy expert), assisted by OP (prostate imaging expert) and GR (urology expert). The prostate was manually contoured by ZS in MRI, assisted by OP, and after projection onto TRUS, the contour was manually adjusted by ZS, assisted by GR. Following TRUS–MRI registration by the algorithm, corresponding landmarks were manually matched across the modalities.

Fig. 5
figure 5

The MATLAB GUI: left and right upper windows show the MRI and TRUS scans, respectively, in axial view. The bottom windows show the corresponding coronal views. Brachytherapy seeds (yellow contours) appear as hypo-intense dots in MRI and hyper-intense dots in TRUS

Full size image
Table 1 Main parameters value
Full size table

Considering all the pairs, M, of matched landmark points \(P_i^\mathrm{MRI} \leftrightarrow Q_i^\mathrm{TRUS} \), \(i=1\ldots M\), the average registration error (\({RE}_{m})\) between a pair of registered TRUS–MRI prostate volumes is defined by (Eq. 5):

$$\begin{aligned} RE_m =\frac{1}{M}\mathop \sum \limits _{i=1}^M \Vert T\left( {P_i^\mathrm{MRI} } \right) -Q_i^\mathrm{TRUS}\Vert _2 \end{aligned}$$
(5)

where \(T\left( {P_i^\mathrm{MRI} } \right) \) is an MRI landmark point transformed by the registration.

The values for the main algorithm parameters used in the validation experiments were optimized using an additional brachytherapy case left out of the validation experiments. The values, summarized in Table 1, were kept constant in all the experiments.

Fig. 6
figure 6

(left) An original MRI slice; (right) the warped MRI slice is shown in overlay with the corresponding TRUS slice after registration. The projected MRI landmarks (blue contours) are shown beside TRUS landmark (red contours). Matched TRUS–MRI landmarks share the same numeric index

Full size image

Results

In Fig. 6, an original MRI slice (left) is shown after warping and overlay with the corresponding TRUS slice (right), following registration. Projected MRI landmarks (blue contours) and TRUS landmark (red contours) are shown together. Among them, matched TRUS–MRI landmarks pairs bear the same numeric index. A box plot of the registration error is given for each brachytherapy patient in Fig. 7. For each box, the central mark (red) is the median, the edges of the box are the 25th and 75th percentiles, respectively, the whiskers extend to the most extreme data points not considered outliers, and outliers are plotted individually (red). The average, minimal and maximal registration errors for each EBRT patient are given in Table 2. A total of 95 and 10 landmark point pairs were used for the brachytherapy (Fig. 7) and EBRT (Table 2) registration error computation, respectively. The quantitative results for all the experiments are summarized in Table 3. The average \((\pm \hbox {std})\) registration errors for EBRT and brachytherapy patients were 2.49 mm \(({\pm }1.32\,\hbox {mm})\) and 3.20 mm (\({\pm }1.13\,\hbox {mm}\)), respectively, thus significantly smaller than the largest voxel dimension in the MRI scans (equal to 3.6 mm). The combined average registration error (\(\pm \hbox {std}\)), accounting for EBRT and brachytherapy patients together, was 2.56 mm (\({\pm }1.32\,\hbox {mm}\)). An average running time of about 1 min was required to compute the fusion between TRUS and MRI after completion of contouring.

Fig. 7
figure 7

Brachytherapy patients: Box plot for the registration error between transformed MRI and TRUS landmark points. For each box, the central mark is the median, the edges of the box are the 25th and 75th percentiles, the whiskers extend to the most extreme data points not considered outliers, and outliers are plotted individually (red cross)

Full size image
Table 2 EBRT patients: average, min and max registration error
Full size table
Table 3 Summary of the quantitative results
Full size table

Discussion and conclusions

We have presented a novel method for TRUS–MRI fusion based on rigid and deformable registration The proposed method represents TRUS and MRI prostate contour points by a 3D shape-context descriptor that embeds local information about neighborhood shape. Matching between TRUS and MRI points is treated as a graph assignment task. A dense deformation field is obtained by fitting a 3D B-splines mesh grid to the set of matched point pairs. The method was validated on ten real cases, six following brachytherapy and four after implantation of gold seed fiducials for EBRT of the prostate.

From a clinical standpoint, the validation performed in this study based on the co-registration of implanted seeds imaged (on the same day) using TRUS and MRI is far more objective than that reported in previous works. To our knowledge, it is the first time that a method for TRUS–MRI fusion is validated using objective landmarks to measure the registration accuracy. Implanted seeds have a well-defined size and shape (4.5 mm long \(\times \) 0.8 mm diameter cylinders) that makes them easier to localize accurately and match across the modalities. Conversely, matching between anatomical landmarks as commonly done in previous works involves an amount of subjectivity that is difficult to evaluate. Furthermore, in the proposed method, the seeds are well-distributed landmarks, providing accuracy measurements throughout the whole gland in contrast to the anatomical structures used for validation in previous works, e.g., the centrally located urethra. The proposed method showed promising results, with an overall average registration error of 2.56 mm (\({\pm }1.32\,\hbox {mm}\)), which is significantly smaller than the largest voxel dimension of the MRI scans (equal to 3.6 mm). Published average registration errors [4–7, 9] were in the 1.5–3.5 mm range. However, considering the lack of uniformity between the datasets, the acquisition protocols and parameters, and even more importantly the validation methodology, a direct numerical comparison of the registration accuracies is problematic.

Assuming a Gaussian distribution of the registration error, a 3-sigma error of 6.52 mm is obtained with the proposed method. In radiotherapy, when target delineation is uncertain, due to suboptimal imaging of the tumor an additional margin of 5–7 mm is commonly applied to define a clinical target volume (CTV). Although more validation is required, in particular by pathology confirmation, this result suggests that the proposed method may lead to clinically applicable CTV margins.

There are some limitations to this work. The MRI slice thickness, 3.6 mm, was relatively high. In ongoing research, slices thinner than 2.5 mm will be available, potentially improving the registration accuracy.

The EBRT dataset was more sensitive to outliers as only three landmark seeds were typically available per case. This may explain the large registration error range (max–min) in EBRT case #1 (3.65 mm) which is much larger than in case #3 (0.19 mm).

Since the proposed registration method is purely geometric (independent from intensity), its accuracy depends on prostate segmentation (contouring) accuracy. The relative deformations between corresponding MRI & TRUS contour segments should reflect real prostate shape changes rather than inter-modality contouring inconsistencies. The proposed projection of MRI contours as initialization for TRUS contours, as opposed to ex novo delineation of the prostate on TRUS, is instrumental in enforcing inter-modality contouring consistency. Resulting TRUS contours will be similar to MRI contours, except at locations where manual corrections are needed as deformations exist between the prostate in both modalities.

Automatic contouring of the MRI prostate was not considered in this work as we focused on the registration step. In ongoing research, an automatic MRI contouring algorithm is being developed to relieve the clinician from this tedious task while allowing for easy-to-perform manual corrections.

The method has been implemented into an intuitive and operational workflow, interfaced to Varian dosimetry software. It will be used in a prospective study (IRB approval pending) to test the clinical feasibility of TRUS–MRI fusion-guided focal therapy. Intraoperative trans-perineal biopsy will be used to confirm the absence of tumor outside the CTV.