Introduction

Accurate mapping between images and the patient on the operating table is the cornerstone of any neuronavigation system. The mapping between the pre-operative images and the patient is generally computed after immobilization of the patient’s head, using fiducial markers, anatomical landmarks, surface matching techniques or a combination of these. This image-to-patient registration is rarely completely accurate but enables navigation for planning of surgery and placement of the craniotomy. However, during the procedure the brain will shift and deform, and the spatial relationship with the pre-operative images will be altered even more. Consequently, the navigational accuracy is reduced and the surgeon can no longer fully trust the spatial information given by the navigation system. This is the situation in the commercial systems that do not have intra-operative imaging capabilities. The amount of brain-shift depends on several factors such as loss of cerebro-spinal fluid (CSF), gravity, the size and location of the surgical target, the amount of resection and administration of drugs. A number of groups have evaluated the magnitude and spatial distribution of brain-shift. One of the first studies described the deformation occurring between pre-operative imaging and the start of the surgical resection in 21 patients and found maximum displacements of more than 10 mm in one-half of the patients [1]. In another early study of brain shift, 58 landmarks in 14 patients were tracked and resulted in a mean displacement of 10.9 mm with a maximum of 24.6 mm [2]. More recent studies of brain shift during placement of electrodes for deep brain stimulation have also been performed [3, 4]. In [3] they found maximum displacements of more than 10 mm even in a procedure with a relatively small craniotomy without any resection of tissue. They also found that the displacement of the cortical surface was larger than the displacement of deep structures. In [4] they found displacements up to 4 mm in 28 patients and concluded like [3] that brain shift in this type of surgery is closely related to the amount of CSF loss. A study of brain-shift in patients with brain tumors was presented in [5]. They evaluated the brain shift in 61 patients with supratentorial lesions and found a maximum displacement of 15.2 mm at the end of surgery. They also found significant correlations between brain shift and tumor volume, midline shift and the size of the craniotomy, respectively.

With potential displacements of more than 1 cm, brain-shift represents a major source of error in neuro-navigation systems based on pre-operative images. Due to the heterogeneous nature of cerebral lesions and the non-uniform distribution of brain-shift throughout the brain, pre-operative prediction of the amount and course of brain-shift seems unlikely as pointed out in [5]. Consequently, acquisition of intra-operative images during surgery is an attractive solution. Intra-operative MRI can provide the surgeon with updated information during surgery [6, 7]. The newest 3 Tesla intra-operative MR scanners provide high quality images within a reasonable timeframe. In some centers, this includes the ability to acquire functional MRI (fMRI) [8] and diffusion tensor imaging (DTI) during surgery [9]. However, this solution still presents a number of limitations including high cost resulting from not only the MR scanner itself, but also the need for a large dedicated operating room. In addition, positioning of the patient on the operating table to allow MR imaging is complex and the transfer of the patient to the scanner or vice versa presents challenges for both the surgeons and the anesthesiologists. High field magnets may also pose a safety hazard [10, 11]. In this context, intra-operative ultrasound (US) presents several advantages. Ultrasound imaging is safe, cheap, portable and provides real time imaging even for blood flow. In addition, there is no need for dedicated operating rooms. However, the interpretation of anatomy and pathology in ultrasound images still seem to present an obstacle for more widespread use. This challenge can partly be solved by incorporating ultrasound images into the neuronavigation system and visualize corresponding views of pre-operative MR data and intra-operative US data [12]. While intra-operative ultrasound imaging is useful for acquisition of updated information about anatomy, pathology and the degree of resection, the ultrasound data present some limitations. The ultrasound data cover only a limited part of the brain around the surgical target, and no functional data are available. Even with high quality intra-operative ultrasound imaging, structural MR images are necessary to provide anatomical overview and ease the interpretation of the ultrasound images. Pre-operative fMRI and DTI data might be crucial to identify important functional regions and white matter tracts. Right from the beginning of surgery and throughout the procedure, inaccurate registration between pre-operative images and the patient and subsequent brain-shift will result in potentially large errors in the navigation based on pre-operative MR data. Navigation based on recently acquired ultrasound, however, will remain accurate [13, 14] because the data is not associated with the image-to-patient registration error. Navigation based on recently acquired ultrasound data is associated with an error of less than 1.5 mm, where ultrasound probe calibration is the main contributing factor.

In order to correct the pre-operative MR data to better reflect the intra-operative reality, several image registration techniques have been proposed. These methods register pre-operative T1-weighted MR images with intra-operative B-mode ultrasound images by aligning hyper-echogenic structures [15], gradients [16] or a combination of intensities and gradients [17]. Validation of these methods has been performed using between two and 14 retrospective clinical datasets. No intra-operative validations of registration methods have been presented. Due to the very different image characteristics of MR and ultrasound data, direct registration between T1 MR and B-mode ultrasound is challenging. We have, therefore, previously suggested the use of blood vessels as features for the registration algorithm [18, 19]. The vascular tree in the brain is relatively easy to extract from both ultrasound (power Doppler imaging) data and MR angiography (MRA) and are consequently well suited for image registration. This method has previously been validated using simulated displacements, phantom data and retrospective clinical data. In this paper, we present the full integration of the semi-automatic vessel based registration technique into a neuronavigation system and the results of the application of the method in the operating room during surgery. To our knowledge, this is the first report of an ultrasound-based registration method to correct for brain-shift running in the operating room during surgery. The registration method, the integration into the neuro-navigation system and intra-operative correction of brain-shift in seven clinical cases are presented in the following sections.

Methods

Intra-operative setup and neuronavigation support

The intra-operative setup used in this study is shown in Fig. 1. The complete system consists of two racks: an in-house navigation system called CustusX [20] (SINTEF Medical technology, Trondheim, Norway) and a GE Vingmed E9 ultrasound scanner with a GE 11 L linear array transducer (GE Vingmed Ultrasound, Horten, Norway). The navigation rack assembles the main hardware components including a navigation computer with display, an optical tracking system (Polaris spectra, Northern Digital Inc., Waterloo, Canada) and a frame grabber (Epiphan systems Inc., Ottawa, Canada). This simplifies logistics in the operating room where space is limited. The infrared camera and the 27″navigation screen are both mounted on flexible arms in order to improve the intra-operative setup.

Fig. 1
figure 1

Intra-operative setup with ultrasound scanner and CustusX rack. In the operating room the navigation screen is mounted on a flexible arm

Full size image

The optical tracking system is used to acquire the position and orientation of navigation localizers with reflective spheres relative to a reference localizer mounted on the operating table (Fig. 1). Localizers are attached to the navigation pointer, the ultrasound probes, the ultrasonic surgical aspirator and the biopsy forceps enabling visualization of all these instruments relative to both pre-operative and intra-operative images on the navigation screen. The ultrasound scanner transfers images to the navigation computer via a frame grabber. The position and orientation of the individual 2D frames can be determined by combining the tracking data from the localizer mounted on the ultrasound probe and a pre-calibration of the same probe [21]. Following a freehand scan that covers the region of interest, the tracked 2D ultrasound images are automatically reconstructed into a 3D volume that becomes available for navigation within a few seconds. The navigation screen during an ultrasound acquisition is shown in Fig. 2.

Fig. 2
figure 2

CustusX during ultrasound acquisition (aneurysm-case). The real-time 2D image from the ultrasound scanner (left), 3D visualization with the moving ultrasound probe and the pre-operative MR volume (top right) and extent of 2D ultrasound image in relation to the pre-operative MR volume (bottom right)

Full size image

In this study we have used the navigation platform CustusX for evaluating new functionality and conducting research in the operating room. The system can be used for image-guided surgery in general, but the main focus of the application is intra-operative ultrasound-based navigation. CustusX is designed to be easy to use in the operating room. The most important features are easily accessible at all times, while more advanced features are hidden during normal use. A set of pre-defined configurations is used to quickly switch between the stages of the surgical workflow: Data import and preparation, Registration, Planning, Navigation, and Ultrasound Acquisition. A photo of the CustusX-rack and the ultrasound scanner in the operating room is shown in Fig. 3.

Fig. 3
figure 3

Neuronavigation with CustusX during the aneurysm operation: the navigation screen (right), the ultrasound scanner (middle) and screens with the real time image from the microscope (left). The navigation pointer used by the surgeon to identify the different structures can be seen both on the navigation screen and in the real time images from the microscope

Full size image

Patients and data

Illustrative cases

To verify the accuracy and usefulness of intra-operative brain-shift correction in various neurosurgical procedures, the proposed vessel-based MR-to-ultrasound registration method was evaluated during the following seven clinical cases:

  1. 1.

    Aneurysm: Right middle cerebral artery bifurcation aneurysm.

  2. 2.

    AVM: Arteriovenous malformation (AVM) located in the left frontal lobe.

  3. 3.

    Tumor: High-grade brain tumor in the left occipital/parietal lobe.

  4. 4.

    AVM: AVM located in the left frontal lobe

  5. 5.

    Tumor: Low-grade tumor located in the right insula

  6. 6.

    Tumor: Low-grade tumor located in the left temporal lobe

  7. 7.

    Tumor: Low grade tumor located in the left frontal lobe

The study was approved by the local ethics committee, and the patients gave informed consent prior to the procedure.

Pre-operative MR imaging

Prior to MR imaging, five self-adhesive fiducial markers were glued to the patient’s skin. Pre-operative MR images included in all cases one 3D time-of-flight (TOF) MRA sequence covering the entire head (voxel size: 0.86 × 0.86 × 0.90 mm3), and one TOF MRA sequence with reduced field of view in the z-direction (voxel size: 0.26 × 0.26 × 0.50 mm3) in order to have improved resolution in the region of interest. For the tumor patients, pre-operative MR imaging also included a Gd-enhanced T1-weighted image (voxel size: 0.5 × 0.5 × 1 mm3) and a fluid-attenuated inversion recovery (FLAIR) (voxel size: 1.0 × 1.0 × 1.2 mm3) image. For the AVM and aneurysm patients, the MRA volume covering the entire head was used for fiducial-based image-to-patient registration. For the tumor patient, the Gd-enhanced T1-weighted image was used for image-to-patient registration.

Intra-operative ultrasound imaging

Intra-operative power Doppler-based ultrasound imaging was performed on the intact dura following the opening of the skull. We used an 11 L linear probe attached to the Vivid E9 ultrasound scanner (GE Vingmed Ultrasound, Horten, Norway) for all acquisitions. Prior to image acquisition, the probe was draped and a navigation localizer was attached to the probe handle. In order to optimize the acoustic contact between the probe and the dura, sterile ultrasound gel was applied to the dural surface. We acquired power Doppler-based ultrasound data in all seven cases. For the tumor patients, we also acquired B-mode data in order to visualize the extent of the tumor.

Registration method

The registration method is based on the use of blood vessels segmented from pre-operative MRA and intra-operative power Doppler ultrasound. The segmentation of pre-operative MRA was performed either by simple thresholding directly in the navigation software, or by the region growing segmentation method implemented in ITK-Snap with default settings [22]. The intra-operative ultrasound data were segmented by simple thresholding. Following segmentation of the vascular tree from both modalities, we extracted the vessel centerlines using the algorithm proposed in [23] and implemented in ITK [24]. An example of vessel centerlines before and after registration is shown in Figs. 4. The resulting centerlines from each dataset were then registered using a modified version of the iterative closest point algorithm (ICP) [25] and can be summarized in five steps:

Fig. 4
figure 4

An example of vessel centerlines used in the registration algorithm (AVM-case): MRA in red and ultrasound in green

Full size image
Fig. 5
figure 5

Snapshots from the navigation system during surgery before (top row) and after (bottom row) correction of the MR data. The surgeon is pointing to the tumor border as can be seen from the ultrasound slices (right column) and both MR FLAIR (middle column) and MR angiography (left column, vessels close to the tumor border) seems to be considerable more aligned after the shift correcting registration

Full size image
  1. 1.

    Finding point correspondences: Every point in the moving volume is matched to the closest point in the fixed volume.

  2. 2.

    Sorting and rejecting point pairs: The least trimmed squares (LTS) algorithm [26] is used to reduce the influence of incorrect pairings on the final transformation. The point pairs are sorted based on the distance between the moving and fixed point. The point pairs corresponding to the largest distances are excluded from the estimation of the transformation.

  3. 3.

    Finding the transformation: A rigid transformation is computed based on the remaining point pairs.

  4. 4.

    Transforming points: The resulting transformation is applied to the moving points

  5. 5.

    Error estimation: The previous five steps are iterated until the difference in mean distance between successive iterations is smaller than 0.001 mm or a maximum number of 100 iterations are reached.

CustusX has support for a range of image-to-image registration algorithms. They are all available from the Registration workflow stage. First, the input data are converted to centerlines. For clear-cut cases this conversion can be performed in a few clicks, but each step can be fine-tuned if necessary. Second, the actual registration is performed. The position and orientation of the different datasets can be linked in the software. This is particularly useful when a given dataset (e.g., MR angiography) is used for image registration. The position and orientation of the datasets previously linked to this dataset (e.g., MR T1 and FLAIR) will then be updated automatically as the transformation is applied. The registration can be verified using an undo/redo function with corresponding visualization.

Evaluation of the registration accuracy

The registration accuracy was first evaluated intra-operatively by qualitative assessment of the alignment of the vessels extracted from MRA and power Doppler-based ultrasound in the navigation software. The surgeon then used the tracked navigation pointer and identified landmarks on the vessels. The correspondence between the ultrasound dataset, the corrected MRA dataset and the known location of the pointer in the physical space were then used by the surgeon to evaluate the registration result and decide if the corrected MRA image and other pre-operative MR images linked to the MRA, were accurate enough for navigation. Finally, we identified five landmarks in each dataset (MRA and ultrasound) post-operatively and computed the distances between landmarks before and after registration in order to quantitatively assess the registration accuracy.

Results

Qualitative results

In all seven cases, the surgeon judged the pre-operative MR images to be too inaccurate for navigation before acquisition of ultrasound data. Thus, the MR images could not be trusted in the identification of the feeding arteries and draining veins of the AVMs, in the localization of the aneurysm and last, but not least, in the identification of the tumor boundary for the tumor cases. Following ultrasound imaging and correction of brain-shift, the visual alignment of pre-operative MR data and intra-operative ultrasound data was clearly improved in all cases. Screenshots from the navigation screen before and after correction are shown in Figs. 5 and 6. After correction, the surgeon judged the pre-operative data to be accurate enough for navigation in all seven cases.

Fig. 6
figure 6

MRA in red and ultrasound in grey before (top row) and after (bottom row) registration: Aneurysm patient (left), AVM patient (middle) and tumor patient (right)

Full size image
Fig. 7
figure 7

Example of landmarks placed on the vessels in ultrasound (left) and MR (right)

Full size image

Quantitative results

Post-operatively, we identified five landmarks in both datasets for all seven patients. An example of landmarks placed in corresponding ultrasound and MR volumes is shown in Fig. 7. The distances between corresponding landmarks before and after registration are shown in Table 1. As can be seen in the table, the mean distance has been reduced from 7.39 mm to 2.77 mm. The distances before and after registration were compared using a paired Student’s t-test. The difference is significant with p < 0.001.

Table 1 Mean distances ± std between five landmarks in the MRA and ultrasound volumes before and after brain-shift correction. Maximum distances are given in parentheses
Full size table

Discussion

In this paper, we have shown that correction of brain shift during neurosurgical procedures can be successfully performed by MR-to-ultrasound registration. To our knowledge, this is the first report of a semi-automatic ultrasound-based registration method to correct for brain-shift running in the operating room during surgery, and an example of a successful transfer of an image registration technique to clinical practice.

A large number of image processing algorithms are developed and validated using phantom or retrospective clinical data only, and never transferred to the clinic. Successful clinical validation and application in “real life” cases requires not only a fast and accurate algorithm, but also careful integration into the navigation system and the clinical workflow. Issues such as graphical user interface (GUI), adapted visualization and computation time are crucial factors. The registration method presented here and the navigation system CustusX make it possible for the surgeon to obtain corrected MR data within minutes after the acquisition of the ultrasound images. While several steps in the registration pipeline are fully automatic, the method still requires some limited user interaction. This includes segmentation of the vascular tree from pre-operative MRA, segmentation of ultrasound data, centerline extraction and the actual registration. As explained previously, the design of the navigation software has greatly simplified these steps, and they can now be performed within a few minutes using the same toolbar window in CustusX. The tuning of processing parameters has been reduced to a strict minimum, and default values based on our experience with a large number of datasets have been defined. The development of the user interface through a number of lab experiments using retrospective data and extensive experience with the use of ultrasound and neuronavigation systems in the operating room have been major factors in the successful use of the registration method during surgery. In addition, The CustusX system facilitates qualitative comparison of image datasets by combined 2D and 3D visualization. To date, the biggest uncertainty for successful registrations is the quality of the input data. This applies to the MRA as well as to the power Doppler ultrasound. As the registration algorithm depends on finding corresponding vessels in the two datasets to be aligned, noise, missing data and vessels that are smeared together represent major challenges and potential sources of error. Improved quality and better definition of the vessels in ultrasound imaging is an active field of research in our as well as other groups. With higher and more consistent image quality, the registration method can be made fully automatic. Other possible improvements that are under investigation are 3D reconstruction algorithms especially adapted to ultrasound power Doppler data as well as more advanced segmentation and centerline extraction for MRA and ultrasound data. The requirement still being that the algorithms can run within seconds in the operating room.

Quantitative measurements of registration accuracy during surgery are challenging. The qualitative evaluation by combined 2D and 3D visualization of the datasets in question is straightforward in our system (see Figs. 5 and 6). In addition, the surgeons get an impression of the accuracy by identifying known structures with the navigation pointer and comparing with the datasets on the navigation screen. For this paper, we have also performed quantitative measurements post-operatively. Our error measure is based on the Euclidian distance between landmarks identified on the vessels in the MRA and ultrasound volumes. As the registration algorithm uses the vascular structures to compute the transformation, this error measure is somewhat biased. However, in AVM and aneurysm surgery the vascular structures are of outmost importance and the correct position and orientation of the vessels are, therefore, essential. In tumor surgery, the spatial relationship between the tumor and the vascular tree can also be critical. In tumor surgery, however, the alignment of the tumor boundary in MR and ultrasound is also an important error measure. The distance between vessels before and after registration show that we are able to reduce the distance between the two datasets considerably in all cases. The distances before and after registration were compared using a paired t-test and the result is statistically significant with p < 0.001. Some cases present large amounts of non-linear deformation that will not be captured by the rigid registration. The future integration of non-linear registration will probably improve the results in such cases.

The experienced misalignment between pre-operative MR images and intra- operative ultrasound data during surgery is due partly to errors related to the point based image-to-patient registration and partly to brain-shift. The proposed method corrects for rigid misalignment caused by both sources. Brain-shift is in general highly non-linear and cannot be completely accounted for by a simple rigid registration. However, our previous study [19] shows that rigid registration in some cases can account for a large part of the experienced mismatch. In cases with inaccurate patient registration and possibly in cases with significant shift in the direction of gravity due to CSF leakage, the rigid component of the transformation will be important. In the near future, non-linear vessel based registration will be implemented in the navigation system and validated intra-operatively. This will probably further improve the registration accuracy in a number of cases such as the AVM and tumor cases presented here. We will also investigate the correction of fMRI and DTI data. As the different datasets can be linked in the software, the registration result obtained using the MRA data can be automatically applied to datasets like fMRI and DTI. As these datasets contain information that is not otherwise easily available to the surgeon, this will probably be an important application of the registration method in the future. Correction of fMRI and DTI will make these datasets considerably more accurate and, thus, usable for a longer period of time during surgery. This is particularly important for surgery in eloquent areas.

In this paper, we have presented intra-operative registration between pre-operative MR and the first ultrasound data acquired on the intact dura. Ultrasound data are usually acquired at least three times during surgery, sometimes as many as ten times, when multiple resection controls are needed. This paper describes, thus, the first step in a series of possible corrections for brain-shift during surgery. In the following steps both MR-to-ultrasound (multimodal) and ultrasound-to-ultrasound (monomodal) registrations are possible. These possibilities will be further explored in the near future and evaluated in terms of registration accuracy, robustness and clinical usefulness. We will also explore the use of ultrasound B-mode images alone or in combination with power Doppler data for correction of brain-shift. This might be a valuable addition in cases where the surgical target is located in regions with few or no vessels suitable for vessel based registration. The goal of future developments will still be to capture brain-shift using 3D ultrasound and provide accurate navigation during the entire procedure in a broad range of neurosurgical operations.

Conclusions

We have presented the successful application of brain-shift correction in the operating theater using image registration between pre-operative MR data and intra-operative ultrasound data. The method is fully integrated into the neuronavigation system and can be used for any surgical case where 3D MRA is available pre-operatively and power Doppler imaging can be performed during the procedure. This system has the potential to increase the usability of pre-operative MR, which is particularly important for data that currently cannot be acquired using intra-operative ultrasound (e.g., fMRI and DTI). Accurate correspondence between MR and ultrasound data during most of the surgical procedure will also ease the interpretation of the ultrasound images and the MR data will provide a more accurate and realistic anatomical overview of the surgical field not available using ultrasound data alone. More intuitive and easier interpretation of intra-operative ultrasound images will hopefully also make this image modality more attractive and result in more widespread use in neurosurgery.