Keywords

1 Introduction

Laparoscopic surgery is a minimally invasive alternative to conventional open surgery and has advantages that include improved outcomes, less scarring, and faster patient recovery. It has become the standard of care for certain surgical procedures such as cholecystectomy. Real-time video of the surgical field obtained using a laparoscopic camera is the primary imaging technique that guides laparoscopic surgeries currently. Despite the increasing application of laparoscopy to treat various pathologic conditions, visualization of the surgical field remains challenging. The majority of laparoscopes used in operating rooms (ORs) are two-dimensional (2D) and can provide only a relatively flat representation of three-dimensional (3D) anatomy and thus lack important depth cues. Moreover, although the current technology is able to provide intraoperative video with rich surface detail of the surgical anatomy, structures beneath the exposed organ surfaces, such as blood vessels and solid lesions, cannot be visualized in the video and might not be fully recognized by the operating surgeon during the surgery, causing avoidable medical complications.

Fig. 1.
figure 1

(a): our current AR system based on optical tracking. (b): planned positions for embedding EM sensors (c): an example of the bending of the head of the LUS transducer.

Full size image

Several groups [1–3] have reported augmented reality (AR) methods with the goal of enhancing intraoperative visualization of minimally invasive laparoscopic procedures. For laparoscopic surgeries, these methods overlay tomographic imaging data on intraoperative video to reveal internal anatomical structures not visible in the video images. Our group has built an AR system using a laparoscopic ultrasound (LUS) scanner (flex Focus 700, BK Medical, Herlev, Denmark), which is capable of seeing beneath the surface of organs in real time, for visualizing hidden structures [4, 5]. To cope with inherent limitations of 2D cameras, our team has adopted stereoscopic (i.e., 3D) visualization (VSII, Visionsense Corp., New York, NY, USA), which is emerging now as a visualization option for laparoscopic surgeries. With the use of a commercial optical tracking system (Polaris, Northern Digital Inc., Waterloo, ON, Canada), we have further developed the capability to overlay real-time LUS data on real-time stereoscopic video accurately to provide 3D AR visualization without the prevailing problem of depth ambiguity. Figure 1(a) shows our current AR system based on optical tracking. Through successful demonstration of our AR system in animal and human studies, we have been gathering feedback from collaborating laparoscopic surgeons regarding the clinical feasibility and usefulness of the AR system. The feedback has focused on the use of optical tracking in a surgical setting. For our application, one limitation of using optical tracking is that the optical markers have to be placed outside the patient’s body because of the line-of-sight requirement. For this, we designed a fixture to mount the optical markers on the handle of the LUS transducer (Fig. 1(b)). To maintain a rigid relationship between the marker and the LUS image, our current AR system does not allow four-way articulation (bending) of the imaging tip of the LUS transducer (Fig. 1(c)), which is a very desirable feature of the imaging device. In fact, a metallic cover is placed over the LUS transducer to prevent its tip from bending (Fig. 1(b)).

To incorporate this feedback of clinicians, we intend to replace optical tracking in our current AR system with electromagnetic (EM) tracking - a widely used real-time tracking technology without the line-of-sight restriction. We plan to embed an EM sensor on the tip of the LUS transducer (Fig. 1(b)) such that it can be allowed to bend freely and tracked. As with optical tracking, we intend to place an EM sensor on the handle of the 3D laparoscope, since it does not have a flexible tip and cannot be bent during surgery. The purpose of this study was to evaluate the tracking accuracy of a commercial EM tracking system made for OR-based applications. The result from this study will guide us in appropriately embedding EM sensors into the two imaging devices in the future.

In a typical EM tracking system, a field generator (FG) is used to create a local magnetic field of known geometry to localize positions and orientations of small sensors (diameter around 1 mm) inside the magnetic field. A thin wire is often required to connect the sensor to the control unit of the tracking system. In spite of many advantages over optical tracking, EM tracking is generally considered less accurate and less stable, especially when applied to clinical settings. This is mainly due to the fact that its magnetic field can be easily distorted by surrounding ferrous metals or conductive materials in the OR. These distortions affect the sensor position and orientation readings. Many investigations have focused on evaluating the accuracy of EM tracking systems in various environments. One common approach is to use a board phantom with drilled holes [6]. The distances among the holes are known (5 cm) and serve as the ground truth. Another popular method is the “scribbling” approach [7], in which sensor position data are collected by moving freely on a plane board with various elevations. A 180 mm\(^3\) cube phantom with 225 holes of known geometry is introduced by Wilson et al. to measure position errors of EM tracking [8]. It is worth noting that several studies used inexpensive and easily available LEGO\(^{\circledR }\) basic bricks and building plate to design their experiments [7, 9]. Moore et al.’s study [10] assessed EM tracking accuracy with the sensors embedded in a transesophageal echocardiography (TEE) probe. However, they did not take dynamic effects into account.

In general, errors of an EM tracking system can be classified into: (1) static errors - errors generated when the sensor is stationary for a certain period of time within the working volume of the FG - and (2) dynamic errors - errors generated when the sensor is moving or the environment is changing. For static errors, common measurements include precision, which measures jitter error (random noise); and accuracy, which measures exactness of relative positions. For each source of error, both positional and orientational errors can be measured. We designed our experiments to evaluate precision and both static and dynamic accuracies in three different situations: sensor by itself, and when the sensor is attached to the planned positions on the two imaging devices (Fig. 1(b)). Measurements in this study were restricted to positional errors.

2 Experimental Setup

A commercial EM tracking system with a 3.4 cm thick tabletop FG (Aurora, Northern Digital Inc., Waterloo, ON, Canada) was used in this study. Tabletop FG is specially designed for OR applications, and is supposed to be placed between the patient and the surgical table. The FG suppresses distortions caused by conductive or ferromagnetic materials located under it. Compatible 6 degrees of freedom (DoF) sensors (Aurora Catheter, Type 2; 1.3 mm diameter) were used for all experiments. In order to simulate a clinical setting, the tabletop FG was placed on a standard surgical table (Fig. 2(a)). In addition, the LUS machine and 3D laparoscopic visualization system were placed near the table.

Fig. 2.
figure 2

Experiments for assessing the EM tracking system. (a): setup of the EM tracking system. (b): specially designed fixture with two EM sensors fixed on the handle of the 3D laparoscope. (c): setup to measure the static error with the 3D laparoscope turned on. (d): setup to measure the static error with the LUS transducer turned on. (e): EM sensor taped to the tip of the LUS transducer. (f): setup to measure the dynamic error with the 3D laparoscope turned on.

Full size image

A fixture to be fixed onto the handle of the 3D laparoscope was designed for our experiments (Fig. 2(b)). It is comprised of a cylindrical mount and a long straight bar with slots at 1 cm interval for placing the sensors. The diameter of the slot matched exactly the diameter of the sensor such that the sensor could be firmly fixed in the slot. Two sensors were placed, using tape, into the first (i.e., the slot closest to the scope handle, and is referred to as the “First Sensor”) and the fifth slot (referred to as the “Second Sensor”), respectively. The tips of the two sensors were exactly aligned with the edge of the fixture (Fig. 2(b)). Since the tracking system reports positions of the sensor tip, this sensor placement yielded a 5 cm distance between the two sensor tips, which was used as the ground truth for accuracy measurement.

If we denote a Cartesian coordinate system centered at the centroid of the surface of the tabletop FG (i.e., the centroid has coordinates (0, 0, 0)), the working volume of the FG can be expressed as (in mm) \(x\in [-210,210]\) (i.e., the width range of the FG), \(y\in [-300,300]\) (i.e., the length range of the FG), and \(z\in [120,600]\) (i.e., the height range above the FG). Note that the working volume of the FG we tested is in fact an ellipsoid-shaped volume, and thus, the sizes we refer to here are the maximum lengths in each dimension. We sampled the working volume with a total of 15 test points: 3 heights at z = 25 cm, 35 cm, and 45 cm; at each height, five points at coordinates \((0,0), (0,-185), (0,185), (-130,0), (130,0)\). It was less interesting for us to study positions with height below 25 cm, since in practice, our AR system is supposed to work with the patient lying on the tabletop FG.

We assessed the performance of EM tracking system in three situations: using the fixture (with two sensors) alone; fixing the fixture onto the handle of the 3D laparoscope (Fig. 2(b)); and stick the First Sensor to the surface of the head of the LUS transducer. For each situation, we measured jitter error, static accuracy and dynamic accuracy. Jitter error applies to single sensor, and in this case the one referred to as the First Sensor. Accuracy was obtained by comparing recorded distances between the two sensor tips to the 5 cm ground truth. To have consistent measurement of static errors, we aimed to position the tip of the First Sensor close to the target test point. This was achieved by utilizing LEGO\(^{\circledR }\) basic bricks and building plates. Six 10 inch by 10 inch plates were connected and taped on the surface of the FG. The fixture and the handles of the two probes were attached to LEGO\(^{\circledR }\)-made mounts using double-sided tapes. These mounts were elevated and positioned to the designated locations in a way such that the distance between the actual location of the tip of the First Sensor and the target test point was less than 5 mm. The Second Sensor maintained a fixed relative position to the First Sensor due to the rigid body of the fixture phantom. Figure 2(c) and (d) show the setup for measuring static errors when the fixture is attached to the working 3D laparoscope and LUS transducer, respectively. For easier positioning of the probes, two orientations of the probes (i.e., in Fig. 2(c) and (d), respectively) were used and kept (or held) consistently among different testing situations. As shown in Fig. 2(e), the First Sensor was stuck to the head of the LUS transducer using double-sided tape. The attachment location was selected to be the farthest location from the tip of the transducer, which could still yield bending of the transducer head.

Precision/jitter is defined, as the deviation of measured positions while one sensor is stationary for a certain period of time. At location \(x\), it is calculated as the Root Mean Square (RMS) [11]

$$\begin{aligned} \text {Pre.}(x)=\sqrt{\frac{1}{N}\sum _{i=1}^Nd(\bar{p},p_i)^2} \end{aligned}$$
(1)

where \(d(\cdot ,\cdot )\) is the Euclidean distance, and \(\bar{p}\) is the mean position of \(N\) recorded positions \(p_i\), \(i=1,\ldots , N\). The accuracy was calculated by

$$\begin{aligned} \text {Acc.}(x)=\frac{1}{N}\sum _{i=1}^N\left[ d(p^1_i,p^2_i)-d_{\text {truth}}\right] \end{aligned}$$
(2)

where \(p^1\) and \(p^2\) were recorded positions with regard to the First and the Second Sensor, respectively, and \(d_{\text {truth}}=5\) cm. For experiments of measuring static error, we recorded tracking for 20 s with a sampling interval of 1 s.

To measure dynamic errors, the fixture was moved freely with the operator’s hand on a glass plate (Fig. 2(f)). The fixture was kept on the plate while moving. We tried to maintain a uniform speed of about 10 cm/s. Experiments were carried out for three situations same as above, and at three different heights (i.e., z = 25 cm, 35 cm, 45 cm). Tracking of the sensors were recorded for 30 s with a sampling interval of 1 s, and the accuracy was calculated using Eq. 2.

3 Result

The mean and maximum errors for three measurements in three situations are given in Table 1. The mean precision was calculated as the average of jitter errors (which was calculated using Eq. 1) at 15 locations. Similarly, the mean static accuracy is the mean of accuracies (calculated using Eq. 2) at 15 locations. The mean dynamic accuracy was the average accuracy over three heights. It should be noted that the maximum value in each case is not the maximum of instant values, but rather the maximum value (averaged according to Eq. 1 or Eq. 2) of 15 positions (for static) or 3 heights (for dynamic). For positions generating extreme values, e.g., 2.08 mm as the maximum static accuracy error for the case without the probe, we repeated the same experiments several times and took the mean value as the result. In a similar manner, errors grouped according to different heights are summarized in Table 2.

Table 1. Mean (maximum) errors for three different situations.
Full size table
Table 2. Mean (maximum) errors at three different heights.
Full size table

4 Discussion

The results we obtained in this study are consistent with results reported previously by other groups. Maier-Hein et al. [12] evaluated the same tracking system, i.e., NDI Aurora Tabletop FG, using standardized board phantom [6] with 5 cm distance as the ground truth. Our 0.56 mm static accuracy (without probe) lies between their reported laboratory accuracy (0.30 mm) and accuracy in a CT suite (0.90 mm), which is reasonable due to our simulated OR setting. In addition, Nafis et al. [7] assessed dynamic errors for a tabletop FG, i.e., 3D Guidance medSAFE\(^{\text {TM}}\) Flat Transmitter (Ascension Technology, Shelburne, VT, USA), and they reported greater error with increased height from the FG, which is similar to what we have found. Besides these comparisons, we further noticed that the dynamic error is generally greater than the static error, which is as expected. Furthermore, the 0.23 mm jitter error when the sensor is attached to the 3D laparoscope is higher than the 0.18 mm error found in the other two situations.

Regarding incorporating EM tracking into our laparoscopic AR system, NDI Aurora Tabletop FG delivers satisfactory tracking accuracy according to our results, and is suitable for clinical applications due to its tabletop design. Although all three error measurements increase when the EM sensors are attached to either of the probes, the increased error is still acceptable. The evaluation results give us valuable insights for further embedding the EM sensors into the two probes. For 3D laparoscope, we could design a fixture similar as the one we used in this study but without the long straight bar, so that a sensor could be fixed at a location close to the handle of the probe. For LUS transducer, we intend to embed the sensor within the transducer head, approximately the same position as where we stuck the sensor in this work. The results from this study also suggest that the tracking system works better at lower heights, and this information is helpful to us in further design of our experiments for 3D camera and LUS calibration, as well as evaluation of the complete EM-tracked AR visualization system.

In conclusion, we have evaluated positional precision and accuracy, both statically and dynamically, for a commercial EM tracking system. The assessment experiments account for situations when just using the sensors alone and when they are attached to one of the two probes used in our stereoscopic laparoscopic AR visualization system. The results suggest that the tracking system has high accuracy and the attachment of the sensor to the planned positions on the probes is promising. These results will serve as the basis and benchmark and guide us in appropriately embedding the sensors into both imaging devices in our continued development of a superior laparoscopic visualization technology.