Computer-Assisted Navigation for Ultrasound “3-D Ultrasound”

Abstract

Computer-assisted navigation systems have been researched for the past 50 years although they are just beginning to be incorporated into clinical practice. A number of navigation systems utilizing ultrasound will be discussed which incorporate elements of computerized positional tracking into standard ultrasound platforms. This chapter will focus on these different systems, how they work, and how they incorporate ultrasound into their navigation system.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This comprehensive textbook is uniquely designed with surgeons in mind, with the understanding that ultrasound can be a somewhat difficult imaging tool to master. Simply visualizing ultrasound images for the first several times, which often appear as black and white abstract portraits without well-defined borders or distinct anatomic landmarks, can be challenging. Recall, if you will, the first time you, as a fresh out of the classroom medical student, gazed upon an ultrasound image and thought, “I have no idea what I am looking at.” And although most experienced general surgeons can easily identify a gallbladder filled with stones on an abdominal ultrasound, interpreting less common or more complex images is often difficult. Even more so is the art of using ultrasound as a tool for guidance of a diagnostic or therapeutic endeavor, such as a core-needle biopsy, central line placement, or tumor ablation. It would be a tremendous understatement to say that mastering ultrasound takes significant dedication, practice, and perseverance. Despite being mentioned as a curriculum requirement by the American Board of Surgery, formal training in ultrasound for surgical residents in the majority of programs in the United States has not yet become standard. Furthermore, many surgeons who completed surgical training prior to the development of more user-friendly ultrasound machines and widespread use of these machines during residency find it difficult to adopt the new skills required for ultrasound. It is not uncommon, even today in 2013, to hear that even accomplished surgeons call a radiologist to the operating room to interpret ultrasound images. This textbook aims to eliminate the need for this practice and to help enable practicing general surgeons with the skills necessary to perform ultrasound with confidence and proficiency.

It is hard for most of us to imagine a world without computers, which have become fixtures in our daily lives. Some of these computers are more subtle than others and embedded in standard household refrigerators or microwave ovens, items that we often overlook. A growing percentage of automobiles on the roads today have GPS (Global Positioning System) navigation built in or have add-on aftermarket devices to help us get around without getting lost. Cell phones are no longer just a means of communication; they have become powerful computers that allow us to search the Internet, find the nearest coffee shop, or direct us with infinitely detailed maps and “GPS” directions. Almost all air traffic in the United States, commercial or private, utilize computerized navigation devices that have become so accurate as to virtually eliminate older means of manual navigation (using compasses and maps). Computers have also become standard in fields such as automobile and aircraft construction as well as parts manufacturing, where they enable precision and efficiency that exceeds human capacity. In some cases these computers allow humans to perform tasks quicker and safer, while in other cases the computers can take over and perform the task completely without human intervention. Surgical robots and unmanned military combat aircraft (drones) are becoming household vocabulary. Such is the world we are entering.

Computer-assisted navigation systems are not new to medicine; there has been research in this field for over 50 years [1]. As they apply to ultrasound in particular, there are a number of systems coming on the market which will be discussed which incorporate elements of computerized positional tracking into standard ultrasound platforms. This chapter will focus on these different systems, how they work, and how they incorporate ultrasound, specifically, into their navigation system. The preceding chapters have extensively discussed the theory and means by which to perform ultrasound examination of the liver, pancreas, and biliary tree. Transabdominal, open surgical, and laparoscopic ultrasound techniques all require skills and practice in order to consistently identify target anatomy with precision and efficiency. Simply identifying a small neuroendocrine tumor of the pancreas or a small hepatoma in a cirrhotic liver can be difficult. Doing this with a flexible-tip laparoscopic ultrasound probe is doubly so. Placing a biopsy needle or ablation antennae into a small tumor such as the ones mentioned above and using a laparoscopic approach represents the ultimate achievement and requires multiple skills which, quite simply, are not easily taught or attained. The surgeon must mentally fuse information from the laparoscopic camera, commonly displayed on one flat-panel monitor, with information from the ultrasound probe, which is often displayed on a second monitor or on the ultrasound machine itself. Then, in what is commonly referred to as “biaxial” image interpretation, the surgeon must move their instrument in the direction desired [1]. What “biaxial” means, in plain English, is that the operator is looking at a television monitor, not down at their hands. This is a skill that many older surgeons, brought up before the age of video games or laparoscopic surgery, have never fully mastered. It is probably safe to say that this is a skill required from all graduating surgical residents. Add to this the ultrasound image, which often is not oriented in the same position as the laparoscopic image or the patient’s body. Finally, the ultrasound image itself is a thin, two-dimensional data set and will not identify a biopsy needle or ablation antennae until they actually cross the ultrasound “plane.” It is this complex amalgam of data – laparoscopic and ultrasound image, target organ location, and surgeon’s hand and instrument positioning – that must be processed by the surgeon’s brain in this procedure (see Fig. 18.1). It is this process or skill that computers, can play a significant role in making it easier for the surgeon to first locate a tumor and then successfully target the lesion [1].

Fig. 18.1

Laparoscopic, hand-assisted, ultrasound-guided ablation of a hepatic malignancy. The patient had undergone a sigmoid resection and already had a lower abdominal hand port in place. Notice the surgeon is performing the procedure by looking back over his shoulder in the opposite direction

“Computerized Proprioception”

There are several commercially available computer-assisted navigational systems currently available, or under development, which have been developed specifically with liver and pancreas surgeons in mind. These systems include those produced by InnerOptic Technology (Hillsborough, NC), Pathfinder Technology (Nashville, TN), and CAScination AG (Bern, Switzerland). They all share some technical design characteristics, and even have some identical third-party components in common, but perhaps nothing more important than the similar concept behind the hardware. Essentially, these systems allow the computer to “know” where certain things are in space, such as a biopsy needle, an ablation antenna, or an ultrasound handpiece transducer. For lack of a better term, we might call this computerized proprioception. We begin by creating a three-dimensional “space” in the computer construct and allow the computer to place certain items in correct position and orientation, in this case an ultrasound probe and its corresponding ultrasound 2-D image. The computer then can add additional objects into that space in a location and orientation determined by the tracking system (also called the localization system) that it employs [1]. These objects, ranging from biopsy needles to microwave ablation antennae to surgical instruments, are computer-generated models (or avatars) of the actual instruments (see Fig. 18.2). This allows the surgeon to visualize the particular instrument relation to the ultrasound image and target lesion long before intersecting the plane of the ultrasound. (Remember that the surgeon does not normally see the antenna once it disappears into the target organ, until it crosses the ultrasound plane. Even then, it is often difficult to actually “see” the instrument.) Early systems utilized actual mechanical arms or calipers, which relayed information about instrument position to the computer to determine the location and orientation of certain items held in those arms. These systems utilized what is referred to as “mechanical digitizers” in order to relay positional information about the end instrument’s location and position to the computer [1] (see Figs. 18.3 and 18.4). One of these systems, which was developed at the University of North Carolina, Chapel Hill, Department of Computer Science, even added stereoscopic three-dimensional goggles (or head-mounted displays) to allow the surgeon to “look” directly at the target organ, rather than at a television monitor [2, 3].

Fig. 18.2

Ultrasound guidance system image including 2-D ultrasound image in the left upper corner and stereoscopic 3-D image with microwave antenna avatar in the center. Notice the purple square target on the ultrasound image

Figs. 18.3 and 18.4

Breast biopsy system (ca 1996). A mechanical arm is used to track the position and orientation of the ultrasound probe, and the live U/S scan appears inside the breast, via an augmented-reality head-worn display. Left: conceptual sketch (Courtesy of Andrei State). Right: view from the head-mounted display, showing the U/S scan on a breast phantom, with an aspiration needle (Courtesy of Andrei State)

Although functional and certainly revolutionary, these systems had the drawback of being somewhat bulky and impractical for certain OR environments. With the advancement of technology, infrared cameras and optical sensors became the systems of choice for several of the image guidance systems and made the mechanical arms somewhat obsolete. These systems, collectively termed optical tracking systems (OTSs), have been used by the above-mentioned companies, yet the actual hardware (infrared camera and optical reflectors) was developed by a third-party company (Northern Digital Inc., Ontario, Canada). Several OTSs for orthopedic and neurosurgical procedures, as well as for general surgery, are currently available and all share certain similar tracking components, although not all systems utilize ultrasound. The Brainlab® system (Feldkirchen, Germany), for example, uses CT and MR imaging of the patient’s skull and brain along with an optical tracking system to help perform complex and precise neurosurgical procedures.

At least two companies have used similar strategies specifically for computer-navigated liver surgery, namely, Pathfinder® (Nashville, TN, USA) and CASination® (Bern, Switzerland). Both of these companies utilized sophisticated computer software to first construct complex 3-D models of each patient’s liver, including vascular anatomy and tumor characteristics. Both systems incorporate an OTS described above to co-register the patients’ actual liver and surgical instruments to the CT-based, computer-generated, 3-D model of the patient’s liver on a video monitor. In this way, the surgeon is able to “see” how close a particular instrument is to certain vital structures such as a major portal vein branch or hepatic vein. Both of these systems have been employed in actual human clinical surgeries for open hepatic resections and/or ablations with remarkable efficacy. These systems depend on static, preoperative CT- or MR-generated models rather than real-time intraoperative ultrasound (IOUS). With the mobilization and manipulation of the liver during open surgery, there often is distortion of the actual organ and the relationship of, say, a tumor to internal hepatic structures. Furthermore, there is continual movement of the patient’s liver during surgery from mechanical ventilation and diaphragmatic motion. As such, it was critical for these navigation systems to integrate live ultrasound.

The first system to successfully integrate real-time intraoperative ultrasound and an OTS for the purpose of liver surgery was produced by InnerOptic (Hillsborough, NC). This system grew out of the earlier research in the Department of Computer Science at the University of North Carolina, Chapel Hill, and has undergone multiple improvements and modifications since the first prototype was developed, some out of trial and error and some out of continuous improvements in hardware technology [2, 3]. The initial systems utilized rather large infrared cameras and optical LEDs mounted on clip-on adaptors for the ultrasound handpiece and the microwave ablation antennae (see Fig. 18.5). Yet this system proved functional enough to produce significant targeting improvements both in the laboratory setting and, after multiple generations of refinements and modifications, in the operating room. This data was presented at the American Hepato-Pancreato-Biliary Association annual meeting in 2008 and subsequently published, demonstrating an improvement in targeting of small phantom tumors in gelatin models by both experience and novice operators [4] (see Fig. 18.6). Furthermore, this system was shown to be both accurate and safe in an actual OR environment consisting of open hepatic ablation procedures [5]. Once again, lessons learned in both the laboratory and in the OR led to design modifications and improvements. Some of the other systems, previously described, which initially relied solely on preoperative CT or MR eventually modified their systems to include ultrasound. These systems now allow the surgeon to visualize both the preop CT mapping of the liver with real-time intraoperative ultrasound, all on a single flat-panel monitor (see Fig. 18.7).

Fig. 18.5

Early guidance system prototype (ca. 2007) utilizing “active” optical sensors attached to ultrasound probes and microwave antennae and including a “head positional mount.” This system was modified and refined over time

Fig. 18.6

Results from targeting studies utilizing a 3-D guidance system for ultrasound, with optical tracking, in gelatin agar targets

Fig. 18.7

Screenshot of a guidance system incorporating coronal and axial CT images, an ultrasound image, and a computer model of the hepatic vascular anatomy and phantom ablation antenna (Image courtesy of Pathfinder Technologies)

“What Do You Mean by 3-D?”

A bit of clarification is in order regarding what is meant by “3-D,” in ultrasound navigation systems. To begin with, most ultrasound transducers in use today by surgeons (to include BK Medical, Aloka®, and SonoSite®) all utilize a single linear array of crystals and therefore produce a single-plane, two-dimensional ultrasound image. Much more sophisticated ultrasound systems utilize a grid of crystal transducers, or have a linear array, and a motor to quickly sweep it back and forth inside the ultrasound probe housing and can produce true multi-planar, three-dimensional ultrasound images. These machines are commonly used in obstetrics, where eager parents-to-be can see hauntingly detailed, 3-D images of their developing baby. However, when we speak of 3-D ultrasound guidance systems, we do not mean to imply a true 3-D ultrasound image; rather, some aspect of the navigation system is in 3-D. When we see a typical ultrasound image on a typical ultrasound monitor, we are seeing a 2-D image on a 2-D screen. And when we see a typical laparoscopic image, say during a laparoscopic cholecystectomy, we are seeing a 2-D image on a 2-D monitor. The InnerOptic system utilizes a high-resolution stereoscopic 3-D monitor that affords the surgeon depth perception not possible on even a high-definition 2-D television monitor. In these systems, the ultrasound image itself remains a 2-D image within a larger 3-D monitor “window.” Imagine, if you will, the ultrasound image is a sheet of paper floating in a virtual “box” in the 3-D monitor. That ultrasound image, as a sheet of paper, can be rotated and adjusted by the surgeon to the optimal position for the procedure. The computer then can a 3-D computer-generated model of the particular device in the precise position and orientation determined by the systems tracking components. Essentially, there are both 2-D and 3-D components within a 3-D space. The combination of these elements gives the surgeon the information needed to target a tumor with greater ease and precision (see Fig. 18.8). (Keep in mind that images depicted in this chapter are, in fact, 2-D representations of actual stereoscopic 3-D computer images).

Fig. 18.8

3-D optical tracking system incorporating a high-resolution 3-D monitor and overhead infrared camera (Polaris, NDI, Waterloo, Ontario, Canada)

Open Optical Tracking Systems (OTS)

As described above, there are several systems available which incorporate optical tracking systems (OTS) including optical reflectors and infrared (IR) cameras for navigation in hepatic surgery. And although each system has unique features, all systems share the same basic concepts. These cameras emit infrared light which bounces off of the small, round, optical reflectors and back to the camera. Each camera system actually utilizes multiple infrared emitters and multiple receivers, which are needed to “triangulate” the position of the reflectors and their spatial orientation. These optical “reflectors” are a type of a “passive system” of targeting that eliminated the need for wires to the surgical instruments. Older systems utilized “active” reflectors, which actually consisted of LEDs, which were connected by wire to the OTS computer (Fig. 18.5). Additionally, there are multiple reflectors clustered in unique geometric patterns on specially designed clip-on mounts. These mounts, in turn, are attached to the surgical instruments, biopsy needles, or ultrasound handpieces (see Figs. 18.9 and 18.10). When the camera receives the infrared light from the reflectors, it then transmits this information to the systems computer, usually a laptop computer. The computer then processes this data and is able to determine the precise location and spatial orientation of each reflector, and in turn, each instrument. These OTSs function similar to the way sailors of old were able to determine their location in the middle of oceans by the position of the stars in the sky. The real advantage of these systems is their ability to incorporate a computer-generated three-dimensional model of an ablation antenna (or other device) and place that image into the 3-D television monitor for the surgeon to see. Some systems have even added trajectory tools to their systems, which allow the surgeon to simply aim the ablation device or biopsy needle at the target and advance straight in.

Fig. 18.9

Optical reflectors attached on plastic mounts to an ultrasound handpiece and ablation antenna. This system demonstrated superior accuracy when targeting small tumor targets in gelatin models (see Fig. 18.6)

Fig. 18.10

Optical tracking system used during open liver ablation trials in porcine models

Yet as good as some of these systems were for aiding surgeons in performing ultrasound-guided tasks, there were limitations to each. One common difficulty was “line of sight” limitations. What this means, simply, is that when a surgeon performs an ultrasound as part of an open liver operation, their hand is often obscured from view by the right costal margin or by the surgical retractors or drapes. This is particularly true when the surgeon’s hand and ultrasound transducer are placed over the dome of the liver. These factors limited the ability of the infrared camera to “see” the optical reflectors, which prompted many successive tweaks and modifications to the shape of the reflectors (e.g., making the handles much longer, wider, etc.). Yet, despite continual changes to the designs, ergonomic and logistical limitations persisted and to a certain degree hampered the surgeons’ ability to perform ultrasound-guided procedures.

From Open to Laparoscopic

Technology marches on and waits for no one. Even as many of these systems were developed, put through preclinical and clinical trains, and passed FDA approval, many have become obsolete. This happened, in part, because technology improved but, equally so, because surgical practice has evolved. Over the past several years, there has been a significant trend toward performing liver tumor ablations in a minimally invasive fashion, to the point that, at many high-volume hepatobiliary centers, the vast majority of ablations are now performed laparoscopically [6, 7]. At Carolinas Medical Center, a 900-bed tertiary referral center for liver surgery, we perform approximately 100 liver tumor ablations per year. The vast majority of these procedures are now performed laparoscopically [8]. And because many surgeons who perform laparoscopic liver tumor ablations utilize an articulating or flexible laparoscopic ultrasound probe, the position of the transducer head cannot be determined using externally applied optical tracking reflectors. (Although it should be said that utilizing a rigid ultrasound probe, it is possible to use the OTSs described above, using IR cameras and optical reflectors. This system was actually tested in 2008, but its benefits were limited, because of the lack of articulation of the ultrasound probe, and because surgeons sometimes had to hold the instruments at awkward angles to keep line of sight between the reflectors on the handles and the IR tracking cameras [unpublished researched]) (see Figs. 18.11 and 18.12).

Fig. 18.11

Prototype of laparoscopic version of optical tracking system in porcine model, utilizing a rigid ultrasound probe (left hand)

Fig. 18.12

Video monitor view of laparoscopic image and superimposed ultrasound image (from Fig. 18.11)

Another problem with using the optical reflectors attached to the handle of ablation antennae is that these devices have a fair amount of flexibility. If these antennae or needles are not placed extremely carefully, there can be a “deflection” of the tip by up to roughly a centimeter. The computer will not be able to account for this deflection, and thus accuracy will suffer. As a result in this rather significant change in surgical practice, and in addition to the limitations discussed above, systems employing infrared cameras and optical reflectors have, according to some opinions, become obsolete. Systems had to be totally rethought and redesigned to accommodate the evolution to minimally invasive approaches.

Electromagnetic Tracking Systems (EMTSs)

As opposed to optical tracking, electromagnetic tracking systems do not require line of sight between the surgical instruments and cameras. Instead, these systems consist of a magnetic field generator, either mounted to a cart or the operating table, and tiny sensor coils, inside the surgical instruments. These systems utilize the principle of electromagnetic induction – which underlies almost all modern electrical technology: transformers, motors, radio, etc. Imagine a source coil, inside the field generator on a cart. If we run an alternating electrical current through the source coil, the coil generates a varying magnetic field, which passes through the patient (see Fig. 18.13). Imagine a second, much smaller, receiver/sensor coil, inside the surgical instrument, nearby the source coil. The magnetic field induces a small electrical current signal in the sensor coil. The strength of this signal depends approximately on the sensor’s distance from the source coil and on sensor’s orientation to the magnetic field. Modern electromagnetic tracking systems have several source coils inside the field generator, at different positions and or orientations relative to the operating table. A computer drives each of these source coils with (possibly) different frequencies and strengths of currents and measures the resulting current signal in the sensor coil, to estimate the position and orientation of the sensor coil. Some systems incorporate feedback – based on where they last found the sensor coil, the computer might alter the signals that drive the source coils, to more accurately home in on subsequent small movements of the sensor coil [9].

Fig. 18.13

Schematic representation of the authors’ estimate of the electromagnetic field produced by tabletop field generator which envelopes the patient’s body

These electromagnetic systems were first developed in the 1970s by Polhemus Navigation Systems as a way to track a pilot’s helmet in an aircraft cockpit, so that the pilot could aim weapons or steer radar with his head motion. In the 1980s the applications were expanded to include capturing the motion of actors for movies and capturing a person’s head and limb positions for virtual reality [10]. In the 1990s, the first systems for medical procedures appeared. Today, there are several companies making medical electromagnetic tracking systems, particularly for interventional radiology and cardiac catheterization procedures. These include Ascension (Milton, Vermont), Northern Digital Inc. (Ontario, Canada), superDimension (Minneapolis, MN), and Biosense Webster (Diamond Bar, CA).

Metal objects (such as in the operating table and in the surgical instruments) can be a problem for electromagnetic tracking systems, because the varying magnetic field, produced by the source coil, can cause eddy currents inside the metal objects near the patient. These unintended eddy currents produce their own magnetic fields that can distort the primary field detected by the sensor coils, which in turn can lead to inaccurate position readings or tracking system failures. Various manufacturers have developed different proprietary and confidential techniques for handling metal materials in the area of the procedure. Regarding the patient table, some tracking systems require keeping the field generator and sensor coils close to each other and far from the metal in the patient table. Other systems have a large field generator that is magnetically shielded from the table and must be positioned underneath the patient. Regarding metals in the surgical instruments, most modern tracking systems tolerate stainless steel and titanium, but not ferrous metals or aluminum. The EMTS-based guidance system developed by InnerOptic Technology Inc. (Hillsboro NC) was first tested at Carolinas Medical Center in 2010 and performed well in preclinical testing and subsequent IRB-approved human clinical trials [4, 5, 11]. The system incorporated a thin, flat magnetic field generator (NDI, Ontario, Canada) with source coils that was designed to be placed under the patient (see Figs. 18.14 and 18.15). Tiny sensor coils were placed into the tip of an articulating ultrasound probe (BK Medical, Denmark) as well as microwave ablation antennae (Microsulis, England) (see Fig. 18.16). The large, overhead OTS cameras were thus eliminated, as were the clumsy, clip-on optical reflectors on the surgical instruments. Furthermore, since the miniature sensor coils are placed in the actual tip of the instruments, the bending or deflecting of the antennae virtually becomes less of a potential for error (see Figs. 18.17, 18.18, and 18.19). As of the time of writing of this chapter, several companies are developing EMTSs to be placed into their clinical inventory, and these will likely replace the older OTSs as the modality of choice for image guidance for ultrasound.

Fig. 18.14

Essential components of an electromagnetic tracking system (EMTS). A large, flat field generator, an articulating ultrasound probe, and a microwave ablation antenna

Fig. 18.15

Tabletop, flat EM field generator in position on the OR table, underneath the foam padding and patient. The device itself measures only a few cm in thickness and produces a relatively large, homogenous EM field

Fig. 18.16

Sensor coils which can be incorporated into ultrasound probes or surgical instruments. These probes function similarly to the optical reflectors in OTSs (Image courtesy of Northern Digital Inc.)

Fig. 18.17

Laparoscopic liver ablation performed using an EMTS during human clinical trials

Fig. 18.18

Video monitor image of a 3-D EMTS showing a standard 2-D ultrasound image on the left and stereoscopic 3-D image with computer model (avatar) of motion-tracked microwave antenna on the right

Fig. 18.19

Intraoperative photo of a laparoscopic liver tumor ablation using the EMTS. Note side-by-side monitors with stereoscopic 3-D ultrasound navigation image and laparoscopic image. Some of the surgeons are wearing lightweight passive 3-D stereo glasses

Future Directions

Despite many years of research and development and incorporation into functioning, clinically available products, ultrasound guidance systems remain a relative rarity in the operating rooms of the overwhelming majority of surgeons throughout the world. Part of this is due to the relative lack of widespread availability of these products, and part of this is due to the actual cost of the systems. Some of the systems discussed in this chapter reportedly have sticker prices of almost 500 thousand US dollars, several times the cost of many modern surgical ultrasound machines. It is not hard to understand that in this current economic healthcare climate, adoption of these types of systems will be extremely difficult. Perhaps the optimal solution would be the incorporation of ultrasound imaging, a 3-D image guidance EMTS, and an ablation device, into a single, affordable unit. But that would take cooperation from multiple medical products companies, not always an easy task. It is clear, however, that these guidance systems will continue to play an ever-growing role in the future of medicine (see Fig. 18.20).

Fig. 18.20

Illustration of the EMTS developed for minimally invasive hepatic tumor ablations (Courtesy of Andrei State and InnerOptic)