Abstract
Image-guided interventions often rely on deformable multi-modal registration to align pre-treatment and intra-operative scans. There are a number of requirements for automated image registration for this task, such as a robust similarity metric for scans of different modalities with different noise distributions and contrast, an efficient optimisation of the cost function to enable fast registration for this time-sensitive application, and an insensitive choice of registration parameters to avoid delays in practical clinical use. In this work, we build upon the concept of structural image representation for multi-modal similarity. Discriminative descriptors are densely extracted for the multi-modal scans based on the “self-similarity context”. An efficient quantised representation is derived that enables very fast computation of point-wise distances between descriptors. A symmetric multi-scale discrete optimisation with diffusion regularisation is used to find smooth transformations. The method is evaluated for the registration of 3D ultrasound and MRI brain scans for neurosurgery and demonstrates a significantly reduced registration error (on average 2.1 mm) compared to commonly used similarity metrics and computation times of less than 30 seconds per 3D registration.
Chapter PDF
Similar content being viewed by others
References
Arbel, T., Morandi, X., Comeau, R.M., Collins, D.L.: Automatic non-linear MRI-ultrasound registration for the correction of intra-operative brain deformations. In: Niessen, W.J., Viergever, M.A. (eds.) MICCAI 2001. LNCS, vol. 2208, pp. 913–922. Springer, Heidelberg (2001)
De Nigris, D., Collins, D., Arbel, T.: Multi-modal image registration based on gradient orientations of minimal uncertainty. IEEE Trans. Med. Imag. (2012)
Felzenszwalb, P.F., Huttenlocher, D.P.: Pictorial structures for object recognition. Int. J. Comp. Vis. 61, 55–79 (2005)
Heinrich, M.P., Jenkinson, M., Bhushan, M., Matin, T., Gleeson, F.V., Brady, S.M., Schnabel, J.A.: MIND: Modality independent neighbourhood descriptor for multi-modal deformable registration. Med. Imag. Anal. 16(7), 1423–1435 (2012)
Heinrich, M.P., Jenkinson, M., Brady, S.M., Schnabel, J.A.: Globally optimal deformable registration on a minimum spanning tree using dense displacement sampling. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part III. LNCS, vol. 7512, pp. 115–122. Springer, Heidelberg (2012)
Heinrich, M., Jenkinson, M., Brady, M., Schnabel, J.: MRF-based deformable registration and ventilation estimation of lung CT. IEEE Trans. Med. Imag. (2013)
Heitz, G., Koller, D.: Learning spatial context: Using stuff to find things. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 30–43. Springer, Heidelberg (2008)
Klein, S., Staring, M., Pluim, J.: Evaluation of optimization methods for nonrigid medical image registration using mutual information and B-Splines. IEEE Trans. Image Proc. 16(12), 2879–2890 (2007)
Lu, C., Chelikani, S., Papademetris, X., Knisely, J.P., Milosevic, M.F., Chen, Z., Jaffray, D.A., Staib, L.H., Duncan, J.S.: An integrated approach to segmentation and nonrigid registration for application in image-guided pelvic radiotherapy. Med. Imag. Anal. 15(5), 772–785 (2011)
Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imag. 16(2), 187–198 (1997)
Maurer, C.R., Hill, D.L.G., Martin, A.J., Liu, H., McCue, M., Rueckert, D., Lloret, D., Hall, W.A., Maxwell, R.E., Hawkes, D.J., Truwit, C.L.: Investigation of intraoperative brain deformation using a 1.5 Tesla interventional MR system: Preliminary results. IEEE Trans. Med. Imaging 17(5), 817–825 (1998)
Mercier, L., Del Maestro, R., Petrecca, K., Araujo, D., Haegelen, C., Collins, D.: Online database of clinical MR and ultrasound images of brain tumors. Med. Phys. 39, 3253 (2012)
Pluim, J., Maintz, J., Viergever, M.: Image registration by maximization of combined mutual information and gradient information. IEEE Trans. Med. Imag. 19(8), 809–814 (2000)
Rivaz, H., Collins, D.L.: Self-similarity weighted mutual information: A new nonrigid image registration metric. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part III. LNCS, vol. 7512, pp. 91–98. Springer, Heidelberg (2012)
Shechtman, E., Irani, M.: Matching local self-similarities across images and videos. In: CVPR 2007, pp. 1–8. IEEE (2007)
Staring, M., van der Heide, U., Klein, S., Viergever, M., Pluim, J.: Registration of cervical MRI using multifeature mutual information. IEEE Trans. Med. Imag. 28(9), 1412–1421 (2009)
Viola, P., Wells III, W.: Alignment by maximization of mutual information. Int. J. Comput. Vision 24(2), 137–154 (1997)
Wachinger, C., Navab, N.: Entropy and laplacian images: Structural representations for multi-modal registration. Med. Imag. Anal. 16(1), 1–17 (2012)
Wein, W., Brunke, S., Khamene, A., Callstrom, M.R., Navab, N.: Automatic CT-ultrasound registration for diagnostic imaging and image-guided intervention. Med. Imag. Anal. 12(5), 577 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Heinrich, M.P., Jenkinson, M., Papież, B.W., Brady, S.M., Schnabel, J.A. (2013). Towards Realtime Multimodal Fusion for Image-Guided Interventions Using Self-similarities. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2013. MICCAI 2013. Lecture Notes in Computer Science, vol 8149. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40811-3_24
Download citation
DOI: https://doi.org/10.1007/978-3-642-40811-3_24
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40810-6
Online ISBN: 978-3-642-40811-3
eBook Packages: Computer ScienceComputer Science (R0)