Medical Robotics and Computer-Integrated Surgery

Abstract

The growth of medical robotics since the mid-1980s has been striking. From a few initial efforts in stereotactic brain surgery, orthopaedics, endoscopic surgery, microsurgery, and other areas, the field has expanded to include commercially marketed, clinically deployed systems, and a robust and exponentially expanding research community. This chapter will discuss some major themes and illustrate them with examples from current and past research. Further reading providing a more comprehensive review of this rapidly expanding field is suggested in Sect. 63.4.

Medical robots may be classified in many ways: by manipulator design (e. g., kinematics, actuation); by level of autonomy (e. g., preprogrammed versus teleoperation versus constrained cooperative control), by targeted anatomy or technique (e. g., cardiac, intravascular, percutaneous, laparoscopic, microsurgical); or intended operating environment (e. g., in-scanner, conventional operating room). In this chapter, we have chosen to focus on the role of medical robots within the context of larger computer-integrated systems including presurgical planning, intraoperative execution, and postoperative assessment and follow-up.

First, we introduce basic concepts of computer-integrated surgery, discuss critical factors affecting the eventual deployment and acceptance of medical robots, and introduce the basic system paradigms of surgical computer-assisted planning, execution, monitoring, and assessment (surgical GlossaryTerm

CAD

/GlossaryTerm

CAM

) and surgical assistance. In subsequent sections, we provide an overview of the technology of medical robot systems and discuss examples of our basic system paradigms, with brief additional discussion topics of remote telesurgery and robotic surgical simulators. We conclude with some thoughts on future research directions and provide suggested further reading.

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Core Concepts

A fundamental property of robotic systems is their ability to couple complex information to physical action in order to perform a useful task. This ability to replace, supplement, or transcend human performance has had a profound influence on many fields of our society, including industrial production, exploration, quality control, and laboratory processes. Although robots have often been first introduced to automate or improve discrete processes such as welding or test probe placement or to provide access to environments where humans cannot safely go, their greater long-term impact has often come indirectly as essential enablers of computer integration of entire production or service processes.

1.1 Medical Robotics, Computer-Integrated Surgery, and Closed-Loop Interventions

Medical robots have a similar potential to fundamentally change surgery and interventional medicine as part of a broader, information-intensive environment that exploits the complementary strengths of humans and computer-based technology. The robots may be thought of as information-driven surgical tools that enable human surgeons to treat individual patients with greater safety, improved efficacy, and reduced morbidity than would otherwise be possible. Further, the consistency and information infrastructure associated with medical robotic and computer-assisted surgery systems have the potential to make computer-integrated surgery as important to health care as computer-integrated manufacturing is to industrial production.

Figure 63.1 illustrates this view of computer-integrated surgery (GlossaryTerm

CIS

). The process starts with information about the patient, which can include medical images (computed tomography (GlossaryTerm

CT

), magnetic resonance imaging (GlossaryTerm

MRI

), positron emission tomography (GlossaryTerm

PET

), etc.), lab test results, and other information. This patient-specific information is combined with statistical information about human anatomy, physiology, and disease to produce a comprehensive computer representation of the patient, which can then be used to produce an optimized interventional plan. In the operating room, the preoperative patient model and plan must be registered to the actual patient. Typically, this is done by identifying corresponding landmarks or structures on the preoperative model and the patient, either by means of additional imaging (x-ray, ultrasound, video), by the use of a tracked pointing device, or by the robot itself. If the patient’s anatomy has changed, then the model and plan are updated appropriately, and the planned procedure is carried out with assistance of the robot. As the intervention continues, additional imaging or other sensing is used to monitor the progress of the procedure, to update the patient model, and to verify that the planned procedure has been successfully executed. After the procedure is complete, further imaging, modeling, and computer-assisted assessment is performed for patient follow-up and to plan subsequent interventions, if any should be required. Further, all the patient-specific data generated during the planning, execution, and follow-up phases can be retained. These data can subsequently be analyzed statistically to improve the rules and methods used to plan future procedures.

Fig. 63.1

Fundamental information flow in computer-integrated surgery

1.2 Factors Affecting the Acceptance of Medical Robots

Medical robotics is ultimately an application-driven research field. Although the development of medical robotic systems requires significant innovation and can lead to very real, fundamental advances in technology, medical robots must provide measurable and significant advantages if they are to be widely accepted and deployed. The situation is complicated by the fact that these advantages are often difficult to measure, can take an extended period to assess, and may be of varying importance to different groups. Table 63.1 lists some of the more important factors that researchers contemplating the development of a new medical robot system should consider in assessing their proposed approach.

Table 63.1 Assessment factors for medical robots or computer-integrated surgery systems (after [63.1])

Broadly, the advantages offered by medical robots may be grouped into three areas. The first is the potential of a medical robot to significantly improve surgeons’ technical capability to perform procedures by exploiting the complementary strengths of humans and robots summarized in Table 63.2. Medical robots can be constructed to be more precise and geometrically accurate than an unaided human. They can operate in hostile radiological environments and can provide great dexterity for minimally invasive procedures inside the patient’s body. These capabilities can both enhance the ability of an average surgeon to perform procedures that only a few exceptionally gifted surgeons can perform unassisted and can also make it possible to perform interventions that would otherwise be completely infeasible.

Table 63.2 Complementary strengths of human surgeons and robots (after [63.1])

A second, closely related capability is the potential of medical robots to promote surgical safety both by improving a surgeon’s technical performance and by means of active assists such as no-fly zones or virtual fixtures (Sect. 63.2.3) to prevent surgical instruments from causing unintentional damage to delicate structures. Furthermore, the integration of medical robots within the information infrastructure of a larger CIS system can provide the surgeon with significantly improved monitoring and online decision supports, thus further improving safety.

A third advantage is the inherent ability of medical robots and CIS systems to promote consistency while capturing detailed online information for every procedure. Consistent execution (e. g., in spacing and tensioning of sutures or in placing of components in joint reconstructions) is itself an important quality factor. If saved and routinely analyzed, the flight data recorder information inherently available with a medical robot can be used both in morbidity and mortality assessments of serious surgical incidents and, potentially, in statistical analyses examining many cases to develop better surgical plans. Furthermore, such data can provide valuable input for surgical simulators, as well as a database for developing skill assessment and certification tools for surgeons.

1.3 Medical Robotics System Paradigms: Surgical CAD/CAM and Surgical Assistance

We call the process of computer-assisted planning, registration, execution, monitoring, and assessment surgical CAD/CAM, emphasizing the analogy to manufacturing CAD/CAM. Just as in manufacturing, robots can be critical in this CAD/CAM process by enhancing the surgeon’s ability to execute surgical plans. The specific role played by the robot depends somewhat on the application, but current systems tend to exploit the geometric accuracy of the robot and/or its ability to function concurrently with x-ray or other imaging devices. Typical examples include radiation therapy delivery robots such as Accuray’s CyberKnife [63.2] (Accuray, Inc., Sunnyvale, CA), shaping of bone in orthopaedic joint reconstructions (discussed further in Sect. 63.3.2) and image-guided placement of therapy needles (Sect. 63.3.3).

Surgery is often highly interactive; many decisions are made by the surgeon in the operating room and executed immediately, usually with direct visual or haptic feedback. Generally, the goal of surgical robotics is not to replace the surgeon so much as to improve his or her ability to treat the patient. The robot is thus a computer-controlled surgical tool in which control of the robot is often shared in one way or another between the human surgeon and a computer. We thus often speak of medical robots as surgical assistants.

Broadly, robotic surgical assistants may be broken into two subcategories. The first category, surgeon extender robots, manipulate surgical instruments under the direct control of the surgeon, usually through a teleoperation or hands-on cooperative control interface. The primary value of these systems is that they can overcome some of the perception and manipulation limitations of the surgeon. Examples include the ability to manipulate surgical instruments with superhuman precision by eliminating hand tremor, the ability to perform highly dexterous tasks inside the patient’s body, or the ability to perform surgery on a patient who is physically remote from the surgeon. Although setup time is still a serious concern with most surgeon extender systems, the greater ease of manipulation that such systems offer has the potential to reduce operative times. One widely deployed example of a surgeon extender is the da Vinci system [63.3] (Intuitive Surgical Systems, Sunnyvale, CA) shown in Fig. 63.2. Other examples (among many) incude the Sensei catheter system [63.7] (Hansen Medical Systems, Mountain View, CA.). the Johns Hopkins University (GlossaryTerm

JHU

) Steady Hand microsurgery robot [63.4, 63.5, 63.6] shown in Fig. 63.3 and discussed in Sect. 63.3, the Rio orthopaedic robot [63.8] (Mako Surgical Systems, Ft. Lauderdale, Florida), the GlossaryTerm

DLR

Miro system [63.9], the Surgica Robotica’s Sergenius system (Surgica Robotica, Udine), and Titan Medical’s Amadeus System (Titan Medical, Toronto, Canada).

Fig. 63.2

The da Vinci telesurgical robot (after [63.3]) extends a surgeon’s capabilities by providing the immediacy and dexterity of open surgery in a minimally invasive surgical environment (photos courtesy of Intuitive Surgical, Sunnyvale)

Fig. 63.3a–c

The Johns Hopkins Steady Hand microsurgical robot (after [63.4, 63.5, 63.6]) extends a surgeon’s capabilities by providing the ability to manipulate surgical instruments with very high precision while still exploiting the surgeon’s natural hand–eye coordination. (a) The basic paradigm of hands-on compliant guiding. The commanded velocity of the robot is proportional to a scaled difference between the forces exerted by the surgeon on the tool handle and (optionally) sensed tool-to-tissue forces. (b) Current laboratory setup, showing the robot, stereo video microscope, stereo display with information overlays, display console for optical coherence tomography (OCT ) system, and a sensorized tool. (c) An earlier recent version of the Steady Hand robot currently being used for experiments in microcannulation of $100\mathrm{\mu }\mathrm{m}$ blood vessels

A second category, auxiliary surgical support robots, generally work alongside the surgeon and perform such routine tasks as tissue retraction, limb positioning, or endoscope holding. One primary advantage of such systems is their potential to reduce the number of people required in the operating room, although that advantage can only be achieved if all the tasks routinely performed by an assisting individual can be automated. Other advantages can include improved task performance (e. g., a steadier endoscopic view), safety (e. g., elimination of excessive retraction forces), or simply giving the surgeon a greater feeling of control over the procedure. One of the key challenges in these systems is providing the required assistance without posing an undue burden on the surgeon’s attention. A variety of control interfaces are common, including joysticks, head tracking, voice recognition systems, and visual tracking of the surgeon and surgical instruments, for example, the Aesop endoscope positioner [63.10] used both a foot-actuated joystick and a very effective voice recognition system. Again, further examples are discussed in Sect. 63.3.

It is important to realize that surgical CAD/CAM and surgical assistance are complementary concepts. They are not at all incompatible, and many systems have aspects of both.

2 Technology

The mechanical design of a surgical robot depends crucially on its intended application. For example, robots with high precision, stiffness and (possibly) limited dexterity are often very suitable for orthopaedic bone shaping or stereotactic needle placement, and medical robots for these applications [63.17, 63.18, 63.19, 63.20] frequently have high gear ratios and consequently, low back-drivability, high stiffness, and low speed. On the other hand, robots for complex, minimally invasive surgery (GlossaryTerm

MIS

) on soft tissues require compactness, dexterity, and responsiveness. These systems [63.21, 63.3] frequently have relatively high speed, low stiffness, and highly back-drivable mechanisms.

2.1 Mechanical Design Considerations

Many early medical robots [63.17, 63.20, 63.22] were essentially modified industrial robots. This approach has many advantages, including low cost, high reliability, and shortened development times. If suitable modifications are made to ensure safety and sterility, such systems can be very successful clinically [63.18], and they can also be invaluable for rapid prototyping and research use.

However, the specialized requirements of surgical applications have tended to encourage more specialized designs. For example, laparoscopic surgery and percutaneous needle placement procedures typically involve the passage or manipulation of instruments about a common entry point into the patient’s body. There are three basic design approaches. The first approach uses a passive wrist to allow the instrument to pivot about the insertion point and has been used in the commercial Aesop and Zeus robots [63.21, 63.23] as well as several research systems. The second approach mechanically constrains the motion of the surgical tool to rotate about a remote center of motion (GlossaryTerm

RCM

) distal to the robot’s structure. In surgery, the robot is positioned so that the RCM point coincides with the entry point into the patient’s body. This approach has been used by the commercially developed da Vinci robot [63.3], as well as by numerous research groups, using a variety of kinematic designs [63.24, 63.25, 63.26]. Finally, a third approach uses an active external wrist [63.17, 63.9]and thus supports robot-assisted interventions that do not require a pivot point, potentially extending robotic surgery to other field of surgery.

The emergence of minimally invasive surgery has created a need for robotic systems that can provide high degrees of dexterity in very constrained spaces inside the patient’s body, and at smaller and smaller scales. Figure 63.4 shows several typical examples of current approaches. One common response has been to develop cable-actuated wrists [63.3]. However, a number of investigators have investigated other approaches, including bending structural elements [63.11], shape-memory alloy actuators [63.27, 63.28], microhydraulic systems [63.29], and electroactive polymers [63.30]. Similarly, the problem of providing access to surgical sites inside the body has led several groups to develop semiautonomously moving robots for epicardial [63.31] or endoluminal applications [63.32, 63.33].

Fig. 63.4a–f

Dexterity enhancement inside a patient’s body: (a) The Intuitive da Vinci Si system and wrist with a typical surgical instrument (Photos courtesy of Intuitive Surgical) (after [63.3]); (b) The end-effectors of the JHU/Columbia snake telesurgical system (after [63.11]); (c) Two-handed manipulation system for use in endogastric surgery (after [63.12]); (d) five-degree-of-freedom 3 mm wrist and gripper (after [63.13]) for microsurgery in deep and narrow spaces; (e) concentric tube robot (after [63.14, 63.15]); (f) Columbia/Vanderbilt high dexterity system for single port access surgery (after [63.16])

Two growing trends MIS are natural orifice transluminal surgery (GlossaryTerm

NOTES

) [63.34, 63.35] and single port laparoscopy (GlossaryTerm

SPL

) [63.36]: the idea is to get access to the abdominal cavity by using natural orifices and internal incisions (in NOTES) or existing human scars (e. g., the navel, in SPL). From the mechanical design viewpoint, there is the need to develop deployable surgical instruments or accessorised endoscopes and to combine flexibility (to reach the target) and stability of the platform (to achieve precision). Clashing of instruments and difficulty in triangulation are the main limitations which companies and research groups try to approach [63.37, 63.38].

The problem of distal operation, already present in MIS, is becoming more dramatic in NOTES and SPL and several solutions for helping surgical tasks requiring triangulation have been developed by different research groups [63.39]. They are based on magnetic fields which can generate an internal force without constraining the internal tool to the access port [63.40, 63.41, 63.42].

Another significant development in recent years has been the emergence and widespread deployment of three-dimensional (GlossaryTerm

3-D

) printing and other rapid prototyping technologies for clinically usable medical devices and medical robot components, as well as for construction of realistic patient-specific models [63.43, 63.44]. This trend has promoted very rapid progress in medical robot design and will be increasingly important in coming years.

Although most surgical robots are mounted to the surgical table, to the operating room ceiling, or to the floor, there has been growing interest in developing systems that directly attach to the patient [63.45, 63.46], and clinically deployed examples exist [63.47]. The main advantage of this approach is that the relative position of the robot and patient is unaffected if the patient moves. The challenges are that the robot must be smaller and that relatively nonintrusive means for mounting it must be developed.

Finally, robotic systems intended for use in specific imaging environments pose additional design challenges. First, there is the geometric constraint that the robot (or at least its end-effector) must fit within the scanner along with the patient. Second, the robot’s mechanical structure and actuators must not interfere with the image formation process. In the case of x-ray and CT, satisfying these constraints is relatively straightforward. The constraints for MRI are more challenging [63.48].

2.2 Control Paradigms

Surgical robots assist surgeons in treating patients by moving surgical instruments, sensors, or other devices in relation to the patient. Generally, these motions are controlled by the surgeon in one of three ways:

• Preprogrammed, semi-autonomous motion: The desired behavior of the robot’s tools is specified interactively by the surgeon, usually based on medical images. The computer fills in the details and obtains the surgeon’s concurrence before the robot is moved. Examples include the selection of needle target and insertion points for percutaneous therapy and tool cutter paths for orthopaedic bone machining.

• Teleoperator control: The surgeon specifies the desired motions directly through a separate human interface device and the robot moves immediately. Examples include common telesurgery systems such as the da Vinci [63.3]. Although physical master manipulators are the most common input devices, other human interfaces are also used, notably voice control [63.21].

• Hands-on compliant control: The surgeon grasps the surgical tool held by the robot or a control handle on the robot’s end-effector. A force sensor senses the direction that the surgeon wishes to move the tool and the computer moves the robot to comply. Early experiences with Robodoc [63.17] and other surgical robots [63.25] showed that surgeons found this form of control to be very convenient and natural for surgical tasks. Subsequently, a number of groups have exploited this idea for precise surgical tasks, notably the JHU Steady Hand microsurgical robot [63.4] shown in Fig. 63.3, the Rio orthopaedic robot [63.8] (Mako Surgical Systems, Ft. Lauderdale, Florida) and the Imperial College Acrobot orthopaedic system [63.49] shown in Fig. 63.5c,d.

  Fig. 63.5a–d

  

  Clinically deployed robots for orthopaedic surgery. (a,b) The Robodoc system (after [63.17, 63.18]) represents the first clinically applied robot for joint reconstruction surgery and has been used for both primary and revision hip replacement surgery as well as knee replacement surgery. (c,d) The Acrobot system of Davies et al. (after [63.49]) uses hands-on compliant guiding together with a form of virtual fixtures to prepare the femur and tibia for knee replacement surgery

These control modes are not mutually exclusive and are frequently mixed. For example, the Robodoc system [63.17, 63.18] uses hands-on control to position the robot close to the patient’s femur or knee and preprogrammed motions for bone machining. Similarly, the IBM/JHU LARS robot. [63.25] used both cooperative and telerobotic control modes. The cooperatively controlled Acrobot [63.49] uses preprogrammed virtual fixtures (Sect. 63.1.3) derived from the implant shape and its planned position relative to medical images.

Each mode has advantages and limitations, depending on the task. Preprogrammed motions permit complex paths to be generated from relatively simple specifications of the specific task to be performed. They are most often encountered in surgical CAD/CAM applications where the planning uses two- (GlossaryTerm

2-D

) or three-dimensional (3-D) medical images. However, they can also provide useful complex motions combining sensory feedback in teleoperated or hands-on systems. Examples might include passing a suture or inserting a needle into a vessel after the surgeon has prepositioned the tip. On the other hand, interactive specification of motions based on real-time visual appreciation of deforming anatomy would be very difficult.

Teleoperated control provides the greatest versatility for interactive surgery applications, such as dexterous MIS [63.21, 63.26, 63.3, 63.50] or remote surgery [63.51, 63.52]. It permits motions to be scaled, and (in some research systems) facilitates haptic feedback between master and slave systems. The main drawbacks are complexity, cost, and disruption to standard operating room work flow associated with having separate master and slave robots.

Hands-on control combines the precision, strength, and tremor-free motion of robotic devices with some of the immediacy of freehand surgical manipulation. These systems tend to be less expensive than telesurgical systems, since there is less hardware, and they can be easier to introduce into existing surgical settings. They exploit a surgeon’s natural eye–hand coordination in an intuitively appealing way, and they can be adapted to provide force scaling [63.4, 63.5]. Although direct motion scaling is not possible, the fact that the tool moves in the direction that the surgeon pulls it makes this limitation relatively unimportant when working with a surgical microscope. The biggest drawbacks are that hands-on control is inherently incompatible with any degree of remoteness between the surgeon and the surgical tool and that it is not practical to provide hands-on control of instruments with distal dexterity.

Teleoperation and hands-on control are both compatible with shared control modes in which the robot controller constrains or augments the motions specified by the surgeon, as discussed in Sect. 63.2.3.

2.3 Virtual Fixtures and Human–Machine Cooperative Systems

Although one goal of both teleoperation and hands-on control is often transparency, i. e., the ability to move an instrument with the freedom and dexterity he/she might expect with a handheld tool, the fact that a computer is actually controlling the robot’s motion creates many more possibilities. The simplest is a safety barrier or no-fly zone, in which the robot’s tool is constrained from entering certain portions of its workspace. More sophisticated versions include virtual springs, dampers, or complex kinematic constraints that help a surgeon align a tool, maintain a desired force, or maintain a desired anatomical relationship. The Acrobot system shown in Fig. 63.5c,d represents a successful clinical application of the concept, which has many names, of which virtual fixtures seems to be the most popular [63.53, 63.54]. A number of groups are exploring extensions of the concept to active cooperative control, in which the surgeon and robot share or trade off control of the robot during a surgical task or subtask. As the ability of computers to model and follow along surgical tasks improves, these modes will become more and more important in surgical assistant applications. Figure 63.6 illustrates the overall concept of human–machine cooperative systems in surgery, and Fig. 63.7 illustrates the use of registered anatomical models to generate constraint-based virtual fixtures. These approaches are equally valid whether the surgeon interacts with the system through classical teleoperation or through hands-on compliant control. See also Chap. 43.

Fig. 63.6

Human–machine cooperative systems (HMCS ) in surgery

Fig. 63.7

Human–machine cooperative manipulation using constraint-based virtual fixtures, in which patient-specific constraints are derived from registered anatomical models (after [63.53])

Both teleoperation and hands-on control are likewise used in human–machine cooperative systems for rehabilitation and disability assistance systems. Constrained hands-on systems offer special importance for rehabilitation applications and for helping people with movement disorders. Similarly, teleoperation and intelligent task following and control are likely to be vital for further advances in assistive systems for people with severe physical disabilities. See Chap. 64 for a further discussion of human–machine cooperation in assistive systems.

2.4 Safety and Sterility

Medical robots are safety-critical systems, and safety should be considered from the very beginning of the design process [63.55, 63.56]. Although there is some difference in detail, government regulatory bodies require a careful and rigorous development process with extensive documentation at all stages of design, implementation, testing, manufacturing, and field support. Generally, systems should have extensive redundancy built into hardware and control software, with multiple consistency conditions constantly enforced. The basic consideration is that no single point of failure should cause the robot to go out of control or to injure a patient. Although there is some difference of opinion as to the best way to make trade-offs, medical manipulators are usually equipped with redundant position encoders and ways to mechanically limit the speed and/or force that the robot can exert. If a consistency check failure is detected, two common approaches are to freeze robot motion or to cause the manipulator to go limp. Which is better depends strongly on the particular application.

Sterilizability and biocompatibility are also crucial considerations. Again, the details are application dependent. Common sterilization methods include gamma rays (for disposable tools), autoclaving, soaking or gas sterilization, and the use of sterile drapes to cover unsterile components. Soaking or gas sterilization are less likely to damage robot components, but very rigorous cleaning is required to prevent extraneous foreign matter from shielding microbes from the sterilizing agent.

Careful attention to higher levels of application protocols is also essential. Just like any other tool, surgical robots must be used correctly by surgeons, and careful training is essential for safe practice. Surgeons must understand both the capabilities and limitations of the robot and of the surgical process as well, since safety is a systemic property. This adds new requirements to training programs, which must include robotic capabilities and nontechnical skills (Sect. 63.3.8 ). In surgical CAD/CAM applications, the surgeon must understand how the robot will execute the plan and be able to verify that the plan is being followed. If the surgeon is interactively commanding the robot, it is essential that the robot interpret these commands correctly. Similarly, it is essential that the robot’s model of its task environment correspond correctly to the actual environment. The availability of task models is necessary to the development of autonomous execution of robotic gestures as well as the formal verification of task correctness [63.57]. Although careful design and implementation can practically eliminate the likelihood of a runaway condition by the manipulator, this will do little good if the robot is badly registered to the patient images used to control the procedure. If the robot fails for any reason, there must be well-documented and planned procedures for recovery (and possibly continuing the procedure manually).

Finally, it is important to remember that a well-designed robot system can actually enhance patient safety. The robot is not subject to fatigue or momentary lapses of attention. Its motions can be more precise and there is less chance that a slip of the scalpel may damage some delicate structure. In fact, the system can be programmed to provide virtual fixtures (Sect. 63.2.3) preventing a tool from entering a forbidden region unless the surgeon explicitly overrides the system.

2.5 Imaging and Modelling of Patients

As the capabilities of medical robots continue to evolve, the use of computer systems to model dynamically changing patient-specific anatomy will become increasingly important. There is a robust and diverse research community addressing a very broad range of research topics, including the creation of patient-specific models from medical images, techniques for updating these models based upon real-time image and other sensor data, and the use of these models for planning and monitoring of surgical procedures. Some of the pertinent research topics include the following:

• Medical image segmentation and image fusion to construct and update patient-specific anatomic models

• Biomechanical property measurement and modelling for analyzing and predicting tissue deformations and functional factors affecting surgical planning, control, and rehabilitation

• Optimization methods for treatment planning and interactive control of systems

• Methods for registering the virtual reality of images and computational models to the physical reality of an actual patient

• Methods for characterizing treatment plans and individual task steps such as suturing, needle insertion, or limb manipulation for purposes of planning, monitoring, control, and intelligent assistance

• Real-time data fusion for such purposes as updating models from intraoperative images

• Methods for human–machine communication, including real-time visualization of data models, natural language understanding, gesture recognition, etc.

• Methods for characterizing uncertainties in data, models, and systems and for using this information in developing robust planning and control methods.

An in-depth examination of this research is beyond the scope of this article. A more complete discussion of these topics may be found in the suggested further reading in Sect. 63.4.

2.6 Registration

Geometric relationships are fundamental in medical robotics, especially in surgical CAD/CAM. There is an extensive literature on techniques for coregistering coordinate systems associated with robots, sensors, images, and the patient [63.58, 63.59]. Following [63.59], we briefly summarize the main concepts here. Suppose that we have coordinates

$${\mathit{v}}_A^r=(x_A,y_A,z_A){\mathit{v}}_B^r=(x_B,y_B,z_B),$$

corresponding to comparable locations in two coordinate systems Ref${}_{\mathrm{A}}$ and Ref${}_{\mathrm{B}}$. Then the process of registration is simply that of finding a function T${}_{\mathrm{AB}}\mathrm{\left(}\mathrm{\cdots }\mathrm{\right)}$ such that

$${\mathit{v}}_B={\mathit{T}}_{AB}\left({\mathit{v}}_A\right).\mathit{v}$$

Generally, ${\mathit{T}}_{\mathrm{AB}}\left(\mathrm{\cdots }\right)$ is assumed to be a rigid transformation of the form

$${\mathit{T}}_{AB}\left({\mathit{v}}_A^r\right)={\mathit{R}}_{AB}{\mathit{v}}_A^r+{\mathit{p}}_{AB}^r,$$

where R_AB represents a rotation and p_AB represents a translation, but nonrigid transformations are becoming increasingly common. There are hundreds of methods for computing ${\mathit{T}}_{\mathrm{AB}}\left(\mathrm{\cdots }\right)$. The most common for medical robotics involve finding a set of corresponding geometric features Γ_A and Γ_B whose coordinates can be determined in both coordinate systems and then finding a transformation that minimizes some distance function $d_{\mathrm{AB}}=\mathrm{distance}[{\mathrm{\Gamma }}_{\mathrm{B}},{\mathit{T}}_{\mathrm{AB}}({\mathrm{\Gamma }}_{\mathrm{A}}\left)\right]$. Typical features can include artificial fiducial objects (pins, implanted spheres, rods, etc.) or anatomical features such as point landmarks, ridge curves, or surfaces.

One common class of methods is based on the iterated closest-point algorithm of Besl and McKay [63.60], for example, 3-D robot coordinates a_j may be found for a collection of points known to be on the surface of an anatomical structure that can also be found in a segmented 3-D image. Given an estimate T_k of the transformation between image and robot coordinates, the method iteratively finds corresponding points b_j on the surface that are closest to ${\mathit{T}}_k{\mathit{a}}_j$ and then finds a new transformation

$${\mathit{T}}_{k+1}=\arg \underset{T}{min}\sum _j({\mathit{b}}_j-\mathit{T}{\mathit{a}}_j{)}^2.$$

The process is repeated until some suitable termination condition is reached.

3 Systems, Research Areas, and Applications

Medical robots are not ends in themselves. As the late Hap Paul often remarked, the robot is a surgical tool designed to improve the efficacy of a procedure. (Paul was the founder of Integrated Surgical Systems. Along with William Bargar, he was one of the first people to recognize the potential of robots to fundamentally improve the precision of orthopaedic surgery.)

3.1 Nonrobotic Computer-Assisted Surgery: Navigation and Image Overlay Devices

In cases where the role of the robot is placing instruments on targets determined from medical images, surgical navigation is often a superior alternative. In surgical navigation [63.61], the positions of instruments relative to the reference markers on the patient are tracked using specialized electromechanical, optical, electromagnetic, or sonic digitizers or by more general computer vision techniques. After the relationships between key coordinate systems (patient anatomy, images, surgical tools, etc.) are determined through a registration process (Sect. 63.2.6), a computer workstation provides graphical feedback to the surgeon to assist in performing the planned task, usually by displaying instrument positions relative to medical images, as shown in Fig. 63.8a. Although the registration is usually performed computationally, a simple mechanical alignment of an image display with an imaging device can be surprisingly effective in some cases. One example [63.62] is shown in Fig. 63.8b.

Fig. 63.8a–d

Information enhancement for surgical assistance. (a) Display from a typical surgical navigation system, here the Medtronic StealthStation; (b) the JHU image overlay system (after [63.62]) uses a mirror to align the virtual image of a cross-sectional image with the corresponding physical position in the patient’s body; (c) Sensory substitution display of surgical force information onto da Vinci surgical robot video monitor (after [63.63]); (d) Overlay of laparoscopic ultrasound onto the da Vinci surgical robot video monitor (after [63.64])

The main advantages of surgical navigation systems are their versatility, their relative simplicity, and their ability to exploit the surgeon’s natural dexterity and haptic sensitivity. They are readily combined with passive fixtures and manipulation aids [63.65, 63.66]. The main drawbacks, compared to active robots, are those associated with human limitations in accuracy, strength, ability to work in certain imaging environments, and dexterity inside the patient’s body (Table 63.2).

Because these advantages often outweigh the limitations, surgical navigation systems are achieving widespread and increasing acceptance in such fields as neurosurgery, otolaryngology, and orthopaedics. Since much of the technology of these systems is compatible with surgical robots and since technical problems such as registration are common among all these systems, we may expect to see a growing number of hybrid applications combining medical robots and navigation.

3.2 Orthopaedic Systems

Orthopaedic surgery represents a natural surgical CAD/CAM application, and both surgical navigation systems and medical robots have been applied to orthopaedics. Bone is rigid and is easily imaged in CT and intraoperative x-rays, and surgeons are accustomed to doing at least some preplanning based on these images. Geometric accuracy in executing surgical plans is very important, for example, bones must be shaped accurately to ensure proper fit and positioning of components in joint replacement surgery. Similarly, osteotomies require both accurate cutting and placement of bone fragments. Spine surgery often requires screws and other hardware to be placed into vertebrae without damage to the spinal cord, nerves, and nearby blood vessels.

The Robodoc system shown in Fig. 63.5a-d a,b represents the first clinically applied robot for joint reconstruction surgery [63.17, 63.18]. Since 1992, it has been applied successfully to both primary and revision hip replacement surgery, as well as knee surgery. Since this system exhibits many of the characteristics of surgical CAD/CAM, we will discuss it is some detail. In the surgical CAD phase, the surgeon selects the desired based on preoperative CT images and interactively specifies the desired position of the implant components. In the surgical CAM phase, surgery proceeds normally up to the point where the patient’s bones are to be prepared to receive the implant. The robot is moved up to the operating table, the patient’s bones are attached rigidly to the robot’s base, and the robot is registered to the CT images either by use of implanted fiducial pins or by use of a 3-D digitizer to match bone surfaces to the CT images. After registration, the surgeon’s hand guides the robot to an approximate initial starting position. Then, the robot autonomously machines the desired shape with a high-speed rotary cutter while the surgeon monitors progress. During cutting, the robot monitors cutting forces, bone motion, and other safety sensor, and either the robot controller or the surgeon can pause execution at any time. If the procedure is paused for any reason, there are a number of error recovery procedures available to permit the procedure to be resumed or restarted at one of several defined checkpoints. Once the desired shape has been machined, surgery proceeds manually in the normal manner.

Subsequently, several other robotic systems for joint replacement surgery have been introduced or proposed. The references in Sect. 63.4 provide numerous examples. Notable hands-on guided systems include Rio surgical robot [63.8] (Mako Surgical, Ft. Lauderdale, Florida) and the Acrobot [63.49] system for knee surgery shown in Figs. 63.5c,d. Similarly, several groups have recently proposed small orthopaedic robots attaching directly to the patient’s bones [63.45] or completely freehand systems such as the NavioPFS surgical system (Blue Belt Technlogies, Pittsburgh, Pa.) which combine surgical navigation with very fast on-off control of a surgical cutter [63.67, 63.68]. A recent example of a hybrid passive-active robot is [63.69].

3.3 Percutaneous Needle Placement Systems

Needle placement procedures have become ubiquitous in image-guided intervention, typically performed through the skin but also through cavities. These procedures fit within the broader paradigm of surgical CAD/CAM, where the process involves use of patient images to identify targets within the patient and planning needle trajectories; inserting the needles and verifying their placement; performing some action such as an injection or taking a biopsy sample; and assessing the results. In most cases, an accuracy of 1–2 mm is acceptable, which is not easy to achieve freehand, because the target is not directly visible, soft tissues tend to deform and deflect, and needles often bend. The procedures typically rely on some form of intraoperative imaging (x-ray, CT, MRI, and ultrasound) for both guidance and verification. The surgical motion sequence typically has three decoupled phases: place the needle tip at the entry point, orient the needle by pivoting around the needle tip, and insert the needle into the body along a straight trajectory. These motions are often performed freehand, with varying degrees of information feedback for the physician. However, passive, semiautonomous, and active robotic systems have all been introduced. Figure 63.9 shows several clinically deployed systems for needle placement.

Fig. 63.9a–e

Clinically deployed systems for in-scanner needle placement. (a,b) The Neuromate system (after [63.19]) for stereotactic procedures in the brain uses a novel noncontact sensing system for robot-to-image registration; (c) Johns Hopkins system for in-CT needle placement (after [63.70, 63.71]); (d,e) Manually activated device for in-MRI transrectal needle placement into the prostate (after [63.72])

3.3.1 Freehand Needle Placement Systems

Freehand needle placement with CT and MRI guidance uses skin markers to locate the entry point [63.62], reference to the scanner’s alignment laser to control needle direction, and markers on the needle to control depth. With ultrasound, the primary reliance is on surgeon experience or the use of some sort of needle guide to drive the needle to target while it passes in the ultrasound plane. Tracked ultrasound snapshots guidance combines orthogonal viewpoints with frozen ultrasound image frames [63.73]. Mechanical needle guides [63.61], hand-held navigation guidance [63.74], and optical guides have been combined with most imaging modalities. These include laser guidance devices [63.75], augmented reality systems [63.76]. Augmented reality with 2-D ultrasound [63.77] and CT/MRI slices [63.62] (Fig. 63.8b) was developed, where a semi-transparent mirror is used together with a flat-panel display to create the appearance of a virtual image floating inside the body in the correct position and size.

3.3.2 Passive and Semiautonomous Devices for Needle Placement

Passive, encoded manipulator arms were proposed for image-guided needle placement [63.78], where following a registration step, the position and orientation of a passive needle guide is tracked and the corresponding needle path is displayed on CT or MRI images. Semiautonomous systems allow remote, interactive image-guided placement of needles, such as transrectal prostate needle placement in MRI environment [63.72] with an actuated manipulator from outside the scanner bore, while the needle driver is tracked in MRI with active coils.

3.3.3 Active Robots for Needle Placement

Neurosurgery was one of the first clinical applications of active robots [63.19, 63.20, 63.22], a natural application for surgical CAD/CAM. The entry and target points are planned on CT/MRI images, the robot coordinate system is registered to the image coordinate system (typically with markers affixed to the patient’s head), and then the robot positions a needle or drill guide. The marker structure may be a conventional stereotactic head frame or, as in the Neuromate system [63.19], registration is achieved by simultaneous tracking of the robot and markers attached to the patient’s skull. Spatial constraints in needle placement led to the development of structures achieving remote center of motion (RCM) or fulcrum motion [63.24, 63.25]. In these systems, the RCM is positioned at the entry point, typically with an active Cartesian stage or a passive adjustable arm, and the robot sets the needle direction and (sometimes) the depth. To speed up imaging, planning, registration, and execution, the robot can work concurrently with imaging devices, such as variants of an RCM-based system that was deployed with x-ray [63.24] and CT guidance [63.70, 63.71]. In [63.71], a marker structure was incorporated in the needle driver to register the robot with a single image slice. MRI has an excellent potential for guiding, monitoring, and controlling therapy, invoking intensive research on MRI-compatible robotic systems for needle placement [63.79] and other more interactive procedures [63.50]. Ultrasound guidance offers many unique advantages: is relatively inexpensive and compact, provides real-time images, does not involve any ionizing radiation, and does not impose significant materials constraints on the robot design. Several robotic systems have been proposed for prostate interventions [63.80] using transrectal ultrasound guidance. For other ultrasound-guided needle placement applications, examples include experimental systems for liver [63.64, 63.81], gallbladder [63.82], and breast [63.83]. Figure 63.8d shows one example of the use of information overlay to assist in needle placement in a telesurgical application [63.64]. Whatever form of image feedback is available, steering flexible needles to hit desired targets while avoiding obstacles is a ubiquitous problem, having led to several novel approaches [63.84, 63.85] and reviewed most recently in [63.86]. Concentric tube robots (also interchangeably called active cannulas due to their usefulness in medicine), consist of several precurved elastic tubes nested within one another. Actuators grasp these tubes at their bases and extend them telescopically while also applying axial rotations. These movements cause the overall device to elongate and bend, providing a needle-sized device that moves in a manner analogous to a tentacle. Chronologically, a precursor to current concentric tube robots was an early steerable needle design where a curved stylet was used at the tip of a needle to induce steering [63.87]. These robots fall within the broader class of continuously flexible robots called continuum robots (see [63.15] for a review.) Over the past few years, mechanics-based models of active cannulas have rapidly matured due to the simultaneous parallel efforts of several research groups [63.14, 63.88]. Today, these models provide the basis for an active subfield of research on teleoperative control, surgery-specific design, and motion planning research. Surgical applications suggested for concentric tube robots and are in various stages of development, ranging from purely conceptual to animal studies, which will make the next several years exciting for translational clinical research.

3.4 Telesurgical Systems

The concepts of telemedicine, telesurgery, and telepresence in surgery date from the 1970s. Since then, the potential for telesurgical systems to facilitate effective interventions in remote or hostile environments such as the battlefield, space, or thinly populated areas has continued to be recognized [63.89], and there have been some spectacular demonstrations including a transatlantic cholecystectomy [63.51] in 2001, experiments in Italy [63.90] and Japan [63.91] as well as more nearly routine use in Canada [63.52]. The operational difficulties due to the intrinsic communication delay of long distance tele-surgery affect usability and safety and make the regular use of tele-surgery quite uncommon.

However, the primary uses of telesurgical systems have been with the surgeon and patient in the same operating room. Teleoperated robots have been used for over 15 years in MIS, both as auxiliary surgical support systems to hold endoscopes or retractors [63.23, 63.25, 63.92, 63.93, 63.94] and as surgeon extender systems to manipulate surgical instruments [63.26, 63.3]. There has also been recent work to develop telesurgical systems for use within imaging environments such as MRI [63.95].

A primary challenge for auxiliary support systems is to permit the surgeon to command the robot while his or her hands are otherwise occupied. Typical approaches have included conventional foot switches [63.23], instrument-mounted joysticks [63.25], voice control [63.10, 63.25], and computer vision [63.25, 63.96, 63.97].

A common goal in surgeon extender systems is to provide a measure of telepresence to the surgeon, specifically, to give the surgeon the sensation of performing open surgery from inside the patient. In early work, Green et al. [63.98] developed a successful prototype system for telesurgery combining remote manipulation, force feedback, stereoscopic imaging, ergonomic design, etc. Subsequently, several commercial telesurgical systems have been applied clinically for MIS. Of these, Intuitive Surgical’s da Vinci [63.3] has been the most successful, with over 2500 systems deployed as of 2013. Experience with these systems has demonstrated that a high-dexterity wrist is often critical for surgeon acceptance. Although originally targeted at cardiac surgery, as well as more general interventions, the most successful clinical applications to date are radical prostatectomies, where significant improvements in outcomes have been reported [63.99], and hysterectomy, where the clinical benefits compared to conventional laparoscopy are still being studied [63.100].

One emerging area for research exploits the inherent ability of telesurgical systems to act as flight data recorders during surgical procedures. Several authors [63.101, 63.102, 63.103, 63.104] have begun analyzing such data for such purposes as measuring surgical skill, learning surgical gestures and motions, and providing data for surgical simulators. Another emerging area for research [63.105] focuses on semi-automation of surgical gestures between the surgeon and the robot, often based on learned models. Other research [63.106, 63.107, 63.108] exploits augmented reality methods to enhance the information available to the surgeon during telesurgical procedures.

3.5 Microsurgery Systems

Although microsurgery is not a consistently defined term, it generally indicates procedures performed on very small, delicate structures, such as those found in the eye, brain, spinal cord, small blood vessels, nerves, or the like. Microsurgical procedures are commonly performed under direct or video visualization, using some form of magnification (e. g., microscope, surgical loupes, high-magnification endoscope). The surgeon typically has little or no tactile appreciation of the forces being exerted by the surgical instruments and physiological hand tremor can be a significant factor limiting surgical performance. Robotic systems can help overcome these human sensory-motor limitations, and efforts to develop specialized systems for applications such as ophthalmology [63.109, 63.110, 63.111, 63.112, 63.113, 63.114, 63.115, 63.6]) and otology [63.116, 63.117, 63.118], Several groups have also experimented with magnetic manipulation for these applications [63.113, 63.119]. There have been several efforts to compare microsurgical anastamosis procedures using laparoscopic telesurgical systems to conventional microsurgery. Schiff et al. [63.120] among others reported significant reductions in tremor with either robot and significantly improved technical quality and operative times compared to conventional microsurgery. As the nubber of da Vinci systems has proliferated, such applications are increasingly common [63.121]. A number of groups have implemented telesurgery systems specifically for microsurgery [63.122, 63.123, 63.124, 63.13] [63.109, 63.95]. These systems are in various stages of development, from laboratory prototype to preliminary clinical experimentation.

Not all microsurgical robots are teleoperated. For example, the cooperatively controlled JHU Steady Hand robots [63.4, 63.5] [63.115, 63.6] shown in Fig. 63.3 are being developed for retinal, head-and-neck, neurosurgery, and other microsurgical applications. A modified version of this system has also been used for microinjections into single mouse embryos [63.125].

There have also been efforts to develop completely hand-held instruments that actively cancel physiological tremor, for example, Riviere et al. [63.126, 63.127, 63.128] have developed an ophthalmic instrument using optical sensors to sense handle motion and adaptive filtering to estimate the tremulous component of instrument motion. A micromanipulator built into the instrument deflects the tip with an equal but opposite motion, compensating the tremor. Simple mechanical devices [63.129] for reducing tremor in specific tasks have also been developed.

An additional type of hand-held microsurgical and micro-therapeutic devices is reported in [63.130], which describes an active microendoscope for neuroendoscopy and therapy of the spinal cord able to safely navigate in the subarachnoid space and to avoid dangerous contact with the internal delicate structures thanks to a system based on hydrojets. Hydrojets come from the lateral surface of the catheter and, appropriately tuned and oriented, allow the tip of the endoscope to proceed without touching the spinal cord internal walls. The shared control system of the neuroendoscope, based on processing, segmentation, and analysis of the endoscopic images, assists the safe advancement of the tool in real time [63.131].

3.6 Endoluminal Robots

The term endoluminal surgery was first coined by Cuschieri et al. [63.132] as a major component of endoscopic surgery. Endoluminal procedures consist of bringing a set of advanced therapeutic and surgical tools to the area of interest by navigating in the lumina (i. e., the tube-like structures) of the human body, such as the gastrointestinal (GlossaryTerm

GI

) tract, the urinary tract, the circulatory system, etc. Currently, most endoluminal robots are designed for gastrointestinal applications, although there has been some initial work for other areas. There are several advantages (from a robotics research perspective) of working in the GI tract. The GI tract is not sterile and relatively large in diameter. Further it can be punctured intentionally to reach other abdominal cavities in NOTES approaches.

Traditionally, catheters and flexible endoscopes for endoluminal procedures have been inserted and manipulated manually from outside the body with the assistance of one or more visualization systems (e. g., direct endoscopic video, x-ray fluoroscopy, ultrasound). One major challenge is limited dexterity making it difficult to reach the desired target. Typically, flexible endoscopes have a bendable tip that can be steered by means of cable drives, and catheters may have only a fixed bend on a guide wire. There is also the inherent difficulty of pushing a rope, which some companies are trying to address by using external magnetic field (e. g., [63.133]). Once the target site is reached, these limitations become even more significant. Very simple instruments can be inserted through working channels or slid over guide wires, but dexterity is severely limited and there is no force feedback beyond what can be felt through the long, flexible instrument shaft.

These limitations have led a number of researchers to explore integration of more degrees of freedom in the catheter/endoscope body, as well as the design of intelligent tips with higher dexterity and sensing capabilities. Early work by Ikuta et al. led to the development of a five-segment, 13 mm-diameter sigmoidscope using shape-memory alloy (GlossaryTerm

SMA

) actuators. Subsequently, Ikuta et al. developed 3 mm-diameter active endovascular devices using hydraulic actuators incorporating a novel band pass valve fabricated using micro-stereolithographic techniques [63.29].

Several examples exist of instrumented catheter tips with force sensors [63.134] that allow the right branch of the circulatory systems to be found by estimating the force generated between the tip and the vessel walls. Basically, these sensorized endoluminal devices belong to the larger group of micro-electromechanical systems (GlossaryTerm

MEMS

)-instrumented surgical devices and the same sensing technologies can be also exploited for microsurgery. A survey article by Rebello [63.135] provides an excellent overview of sensorized catheters and other MEMS-based devices in endoluminal and microsurgical applications.

Another approach to endoluminal robots is represented by systems that move under their own power through the body, rather than being pushed. Early work on such systems is well summarized in [63.136]. In 1995 Burdick et al. developed an inchworm-like mechanism for use in the colon. This device combined a central extensor for propulsion and inflatable balloons for friction enhancement with the slippery colon tissue. A more advanced inchworm design for a semiautonomous robotic colonoscope was developed by Dario et al. [63.33] (Fig. 63.10). This device consists of a central silicone elongator, two clamping systems based on suction and gentle mechanical grasping of the colon tissue, and a silicone steerable tip integrating a complementary metal-oxide-semiconductor (GlossaryTerm

CMOS

) camera and a light-emitting diode (GlossaryTerm

LED

)-based illumination system. Thanks to its intrinsic flexibility, the robotic colonoscope applies forces on colon tissues that are ten times lower than those produced by traditional colonoscopies. This system is now clinically tested [63.137], and similar devices combining flexibility and painless operation have been proposed by some companies [63.138, 63.139]. Although the application is not endoluminal, the HeartLander system of Riviere et al. [63.31] shown in Fig. 63.11a,b shares many of the characteristics of these systems. It uses an inchworm-like gait to traverse the surface of the heart and to perform simple operation. An instructive paper on the combination between flexibility and stiffness of endoscopic devices for allowing painless manoeuvrability and stable anchoring when performing surgical tasks has been recently published [63.140]. It presents a number of design criteria and solutions for transforming endoscopic devices from diagnostic tools to stable surgical paltforms.

Fig. 63.10a,b

Medical robot for colonoscopy (after [63.33]): (a) the gait cycle of the robot, consisting of: (1) proximal clamping, (2) elongation, (3) distal clamping, and (4) retraction; (b) a recent working prototype used for clinical trials

Fig. 63.11a–d

Mobility inside the body. (a,b) HeartLander device for crawling across the surface of the heart (after [63.31]). (c) Legged capsule for gastrointestinal diagnosis and therapy (after [63.148, 63.28]), (d) magnetic capsule for exploration of the GI tract, showing left capsule and components and right magnetic dragging platform based on a permanent magnet driven by a robotic manipulator (after [63.143])

The natural evolution of GI endoscopic devices is represented by endoscopic capsules [63.141] which keep the promise to make endoscopy of the GI tract a screening method highly tolerated by patients. For transforming simple CMOS swallowable cameras with illumination and transmission functionalities into useful diagnostic devices, several research groups have explored variegated solutions for active capsule locomotions. An example of legged locomotion capsules for GI application is illustrated in Fig. 63.11c and described in [63.142, 63.28]. In order to overcome dramatic powering problems, magnetic capsules propelled by permanent magnets [63.143] or by modified MRI fields [63.144] have been proposed. An earlier application of electromagnetic manipulation of an object within the body was the video tumor fighter of Ritter et al. [63.145].

3.7 Sensorized Instruments and Haptic Feedback

Surgical procedures almost always involve some form of physical interaction between surgical instruments and patient anatomy, and surgeons are accustomed to use their haptic appreciation of tool-to-tissue interaction forces in performing surgical procedures. In situations such as where the clumsiness of instrumentation or human sensory-motor limitations limit the surgeon’s ability to feel these forces, surgeons have been trained to rely on visual or other cues to compensate. Apart from the need for haptic feedback for diagnostic procedures and tissue identification, it has been demonstrated that reduced tactile and force information can result in the application of unnecessarily large forces on the patient during surgery, with possible harm to the patient [63.146]. The quality of haptic feedback available in currently deployed surgical robots is poor or nonexistent. Current research addresses the limitations firstly by integrating force and other sensors into surgical end-effectors and secondly by developing improved methods for processing and conveying the sensed information to the surgeon. However, the debate of the usefulness of force feedback in robot-assisted surgery is still open and is made more complex by the many scientific and technical difficulties due to the intrinsic presence of (possibly variable) communication time delay in the robotic system [63.147].

Although the most obvious way to display haptic information in a telesurgical system is directly through the master hand controllers, this method has several limitations, including friction and limited bandwidth in the hand controllers. Although these issues may be addressed through specialized design (which may raise costs and require other compromises) and improved control, there has been considerable interest in sensory substitution schemes [63.149, 63.150, 63.151] in which force or other sensor information is displayed visually, aurally, or by other means. Figure 63.8c shows one example of sensory substitution for warning when a da Vinci robot is about to break a suture [63.63].

Starting in the 1990s, several groups [63.151, 63.152, 63.153] have sensorized surgical instruments for microsurgery and MIS by equipping them with force sensors. Generally, these efforts relied on sensory substitution to display force data, either for freehand or telesurgical application. For example, Poulose et al. [63.152] demonstrated that a force sensing instrument used together with an IBM/JHU LARS robot [63.25] could significantly reduce both average retraction force and variability of retraction force during Nissen fundoplication. The first surgical robot with force feedback used in in vivo experiments was the GlossaryTerm

JPL

-NASA GlossaryTerm

RAMS

system, which was tested on animal models for vascular anastomosis [63.154] and carotid arteriotomies [63.155]. Rosen et al. [63.153] developed a force-controlled teleoperated endoscopic grasper equipped with position sensors and actuated by direct-current (GlossaryTerm

DC

) motors whose output torque is sensed and fed back through motors integrated in a grasping handle. A similar approach was used by Menciassi et al. [63.156] for a microsurgery gripper equipped with semiconductor strain gauges and a PHANTOM (SensAble Technologies, Inc.) haptic interface. More recent work at Johns Hopkins has focused on incorporation of optical fiber force [63.157, 63.158, 63.159, 63.160] and OCT [63.161, 63.162, 63.163] sensors into microsurgical tools. Both video and auditory sensory substitution [63.164], as well as direct feedback to robotic devices have been used to help surgeons control tool-tissue interaction forces and distance.

Several researchers [63.165] have focused on specialized fingers and display devices for palpation tasks requiring delicate tactile feedback (e. g., for detecting hidden blood vessels or cancerous tissues beneath normal tissues). There has also been work to integrate nonhaptic sensors into surgical instruments, for example, Fischer et al. have developed instruments measuring both force and tissue oxygenation levels [63.166]. This information can be used for such purposes as assessing tissue viability, distinguishing between different tissue types, and controlling retraction so as not to cause ischemic tissue damage.

Finally, it is important to note that sensorized surgical tools have important application beyond their direct use in surgical procedures, for example, one use is in biomechanical studies to measure organ and tissue mechanical properties to improve surgical simulators.

3.8 Surgical Simulators and Telerobotic Systems for Training

Medical education is undergoing significant changes. The traditional paradigm for surgical technical training is often summarized as see one, do one, teach one. This method can be effective in open surgery, where the surgical trainee directly observes the expert surgeon hands, sees the instrument motion, and follows the organ manipulation. However, in endoscopic surgery it is difficult to observe the surgeon’s hand movements (outside the body) and the surgical tool actions (inside the body and only visible on a video monitor). In addition, endoscopic surgery requires different skills than open surgery, such as spatial orientation, interpretation of 2-D images in 3-D, and manipulating instruments through entry portals. These considerations led to introduction of surgical simulation systems of varying degrees of complexity and realism for endoscopic and other minimally invasive procedures. Nowadays, training simulators have achieved widespread acceptance in the field of anaesthesia, intensive care, flexible endoscopy, surgery, interventional radiology, and other fields. The use of simulators for training is so common that working groups have been set up in order to evaluate these training systems based on shared guidelines [63.167] and many teaching hospitals have extensive simulation training centers. Simulators are being validated on their basic parameters (face, content and construct) [63.168], whereas results on concurrent and predictive validity of simulation training are still not available.

Survey [63.169] divides training simulators into three groups, depending on the type of technology used: mechanical, hybrid, and virtual reality.

Mechanical simulators are boxes where objects or organs are placed and manipulated using surgical instruments. The manipulation is observed through a laparoscope and a screen. The simulated surgical task is observed by an experienced surgeon, who gives feedback to the trainee. Generally, there are no registration processes and the simulator must be reassembled after any training session. The LapTrainer with SimuVision (Simulab Inc., Seattle, USA) is a training system with a simulated laparoscope that consists of a boom-mounted digital camera in an open box trainer.

A hybrid simulator uses a box with objects or organs as a mechanical simulator, but in addition the performance of the trainee are monitored by a computer which gives guidance for the task execution and an objective feedback based on preplanned metrics. Thanks to this feedback, the assistance and judgement of an experienced surgeon are not strictly necessary, for example, the ProMIS (Haptica Inc., Boston, USA) is a typical hybrid simulator for training basic skills such as suturing and knot tying. Surgical instruments are passed through dedicated ports and the trainee receives the same haptic feedback as in real surgery during manipulation in the box. In addition, ProMIS analyzes the trainee’s performance by tracking the instrument position in 3-D and by measuring the execution time, path length, and smoothness of task execution. Another recent hybrid simulator is the Perk Tutor open-source platform for ultrasound-guided percutaneous needle insertion training [63.170], which has been successfully applied in teaching of ultrasound-guided facet joint injections [63.171]. Specific to robotic surgery is the RoSS system [63.172], which simulates the operator console of the da Vinci robotic system. Intuitive Surgical currently markets the dv-Trainer [63.173] simulator for the da Vinci system, comprising the surgeon console from a da Vinci Si system and a computer back end to simulate the patient-side manipulators, and its efficacy is being evaluated [63.174].

Finally, virtual-reality training systems combine visualization and haptic interfaces to enable surgeons to interact efficiently and naturally with real-time computational models of patient anatomy [63.175]. The development of these systems is inherently a multidisciplinary effort, including real-time computer graphics, the development of high-bandwidth haptic devices, real-time biomechanical modeling of organs, tool–tissue interactions, expertise in training assessment, and human–machine information exchange, etc. [63.176]. Research in these areas is closely related to and synergistic with comparable developments in technology and systems for performing real interventions, for example, modeling of organ motion in response to forces is necessary to improve the accuracy of needle placement procedures. Haptic feedback devices must meet similar requirements whether the forces displayed are simulated or measured directly in telesurgery, and so on. Finally, as noted earlier, sensorized instruments and real-time imaging are critical sources of data needed to create realistic biomechanical models. The use of the computer graphical processor (GlossaryTerm

GPU

) allows the simulation of organs at rates in excess of 10 kHz [63.177], which enables good haptic feedback. Similarly, new physics-based graphical libraries support the development of low cost virtual reality simulators that correct render the interaction of instruments and anatomical parts, and thus can support large training classes [63.178].

The variety of interface devices and virtual reality laparoscopic simulators is quite wide and increasing numbers of systems are becoming commercially available. The Phantom interface is used in conjunction with virtual simulators to provide users with realistic haptic feedback (SensAble Technologies Inc., Woburn, MA, USA). The Xitact LS500 laparoscopy simulator (Xitact S.A., Lausanne, Switzerland) is a modular virtual-reality simulation platform with software for training and assessing performance in laparoscopic tasks. It is an open system including all or some of these subsystems: laparoscopic tools, mechanical abdomen, a personal computer (GlossaryTerm

PC

) providing the virtual-reality scenario, a haptic interface, a force feedback system and a tracking system for the tools. Several other systems for virtual reality simulation exist that exploit the hardware from Xitact or Immersion Medical, Inc. (Gaithersburg, MD, USA) and that are dedicated to specific surgical tasks: Lapmentor [63.179], the Surgical Education platform [63.180], LapSim [63.181], Procedicus MIST [63.182], EndoTower [63.183], the Reachin laparoscopic trainer [63.184], Simendo [63.185], and the Vest system [63.186]. Specifically for training eye surgeons, the surgical simulator EYESi [63.187] uses advanced computer technology and virtual reality to simulate the feel of real eye surgery, making it possible for surgeons at all levels to acquire new skills and perfect their techniques in preparation for surgery on the human eye. Training for dexterity skills in robotic surgery is supported by the new virtual simulator XRON developed at the University of Verona, which interfaces physics-based simulation with several commercial joysticks and provides evaluation metrics of interactive tasks [63.188].

Making a comparison between these different categories of simulators is not trivial. Basically, box trainers and hybrid simulators require some experienced technicians for the set up and some organizational logistics due to legal and ethical factors related to the storage of freshly explanted organs. The most evident advantage of these simulators is that the tactile response from the manipulated objects is the same as in real surgery and complicated models of organs and tissue–tool interaction are not required. On the other hand, completely virtual-reality trainers are potentially very flexible, can take advantage of powerful graphical engines, but they are limited by the availability of realistic calibration data for the anatomical and biomechanical models. Although demonstrations exist of the ability of simulators to record, objectively score, and hone the psychomotor skills of novice surgeons [63.189], the debate about their value is still open because no multicentre trails has yet been conducted to determine their efficacy. Furthermore, it has been shown that simulation training must be included in structured curricula in robotic surgery, where trainees are also introduced to basic concepts of robotics and, in particular, to the so called non technical skills, e. g., organization, leadership and communication, which significantly affect the surgical outcome [63.190].

3.9 Other Applications and Research Areas

The research areas described above illustrate major themes within medical robotics, but they are by no means exhaustive. Many important application areas such as otolaryngology [63.191, 63.192, 63.193, 63.194] and radiation therapy have necessarily been excluded for space reasons. For a fuller exploration, readers should consult the further reading suggestions in Sect. 63.4.

4 Conclusion and Future Directions

Medical robotics (and the larger field of computer-integrated interventional medicine) has great potential to revolutionize clinical practice by:

• Exploiting technology to transcend human limitations in treating patients

• Improving the safety, consistency, and overall quality of interventions

• Improving the efficiency and cost-effectiveness of robot-assisted patient care

• Improving training through the use of validated simulators, quantitative data capture and skill assessment methods, and the capture and playback of clinical cases

• Promoting more effective use of information at all levels, both in treating individual patients and in improving treatment processes.

From being the stuff of late-night comedy and science fiction 20 years ago, the field has reached a critical threshold, with clinically useful systems and commercial successes. The scope and number of research programs has grown exponentially in the past 8 years, and this chapter is by no means a comprehensive survey of the field. In fact, such a survey would be practically impossible in less than 100 pages. Many important emerging systems and applications are necessarily left out. Interested readers are urged to refer to the further reading section for more complete treatments. In particular, the survey articles and books in listed at the end of this section collectively contain somewhat fuller bibliographies than space permits here.

In the future, we can expect to see continued research in all aspects of technology and system development, with increasing emphasis on clinical applications. As this work proceeds, it is important that researchers remember several basic principles. The first, and arguably most important, principle is that medical robotics is fundamentally a team activity, involving academic researchers, clinicians, and industry. Each of these groups has unique expertise, and success comes from effective, highly interactive partnerships drawing upon this expertise. Building these teams takes a long-term commitment, and the successes in recent years are largely the pay-off from investments in creating these teams. Second, it is important to work on problems with well-defined clinical and technical goals, in which the criteria for measuring success are ultimately related to real advantages in treating patients. In working toward these goals, it is important to have measurable and meaningful milestones and to emphasize rapid iteration with clinician involvement at all stages. Finally, it is essential that all team members recognize the level of commitment that is required to achieve success and that they enjoy what they are doing.

Video-References

:

Da Vinci Surgery on a grape available from http://handbookofrobotics.org/view-chapter/63/videodetails/823

:

Da Vinci Xi introduction | Engadget available from http://handbookofrobotics.org/view-chapter/63/videodetails/824

:

Intuitive surgical Da Vinci single port robotic system available from http://handbookofrobotics.org/view-chapter/63/videodetails/825

:

SPORT system by Titan Medical available from http://handbookofrobotics.org/view-chapter/63/videodetails/826

:

Robot for single port surgery by Nebraska University available from http://handbookofrobotics.org/view-chapter/63/videodetails/827

:

Magnetic and needlescopic instruments for surgical procedures available from http://handbookofrobotics.org/view-chapter/63/videodetails/828

:

CardioArm available from http://handbookofrobotics.org/view-chapter/63/videodetails/829

:

Snake robot for surgery in tight spaces available from http://handbookofrobotics.org/view-chapter/63/videodetails/830

:

IREP robot – Insertable robotic effectors in single port surgery available from http://handbookofrobotics.org/view-chapter/63/videodetails/831

:

Variable stiffness manipulator based on layer jamming available from http://handbookofrobotics.org/view-chapter/63/videodetails/832

:

Reconfigurable and modular robot for NOTES applications available from http://handbookofrobotics.org/view-chapter/63/videodetails/833

:

SPRINT robot for single port surgery available from http://handbookofrobotics.org/view-chapter/63/videodetails/834

:

A micro-robot operating inside eye available from http://handbookofrobotics.org/view-chapter/63/videodetails/835