Keywords

1.1 Introduction

The human musculoskeletal system is an organ system that includes the bones of the skeleton and the cartilages, ligaments, and other connective tissues that bind tissues and organs together. The main functions of this system are to provide form, support, stability, and movement to the body. Bones, besides supporting the weight of the body, work together with muscles to maintain body position and to produce controlled, precise movements. Musculoskeletal disease is among the most common causes of severe long-term disability and practical pain in industrialized societies [1]. The impact and importance of musculoskeletal diseases are critical not only for individual health and mobility but also for social functioning and productivity and economic growth on a larger scale, reflected by the proclamation of the Bone and Joint Decade 2000–2010 [1].

Both traumatology and orthopaedic surgery aim at the treatment of musculoskeletal tissues. Surgical steps such as the placement of an implant component, the reduction and fixation of a fracture, ligament reconstruction, osteotomy, tumour resection, and the cutting or drilling of bone should ideally be carried out as precisely as possible. Not only will optimal precision improve the post-operative outcome of the treatment, but it will also minimize the risk factors for intra- and post-operative complications. To this end, a large number of pure mechanical guides have been developed for various clinical applications. The pure mechanical guides, though easy to use and easy to handle, do not respect the individual patient’s morphology. Thus, their general benefit has been questioned (see for example [2]). Additionally, surgeons often encounter the challenge of limited visibility of the surgical situs, which makes it difficult to achieve the intended procedure as accurately as desired. Moreover, the recent trend towards increased minimally invasive surgery makes it more and more important to gain feedback about surgical actions that take place subcutaneously. Just as a Global Positioning System (GPS)-based car navigation provides visual instruction to a driver by displaying the location of the car on a map, a computer-aided orthopaedic surgery (CAOS) module allows the surgeon to get real-time feedback about the performed surgical actions using information conveyed through a virtual scene of the situs presented on a display device [3, 4]. Parallel to the CAOS module to potentially improve surgical outcome is the employment of surgical robots that actively or semi-actively participate in the surgery [5].

Introduced more than two decades ago [3,4,5], CAOS has emerged as a new and independent area and stands for approaches that use computer-enabled tracking systems or robotic devices to improve visibility to the surgical field and increase application accuracy in a variety of surgical procedures. Although CAOS modules use numerous technical methods to realize individual aspects of a procedure, their basic conceptual design is very similar. They all involve three major components: a therapeutic object (TO in abbreviation, which is the target of the treatment), a virtual object (VO in abbreviation, which is the virtual representation in the planning and navigation computer), and a so-called navigator that links both objects. For reasons of simplicity, the term “CAOS system” will be used within this article to refer to both navigation systems and robotic devices.

The central element of each CAOS system is the navigator. It is a device that establishes a global, three-dimensional (3-D) coordinate system (COS) in which the target is to be treated and the current location and orientation of the utilized end effectors (EE) are mathematically described. End effectors are usually passive surgical instruments but can also be semi-active or active devices. One of the main functions of the navigator is to enable the transmission of positional information between the end effectors, the TO and the VO. For robotic devices, the robot itself plays the role of the navigator, while for surgical navigation a position tracking device is used.

For the purpose of establishment of a CAOS system through coactions of these three entities, three key procedural requirements have to be fulfilled. The first is the calibration of the end effectors, which means to describe the end effectors’ geometry and shape in the coordinate system of the navigator. For this purpose, it is required to establish physically a local coordinate system at the end effectors. When an optical tracker is used, this is done via rigid attachment of three or more optical markers onto each end effector. The second is registration, which aims to provide a geometrical transformation between the TO and the VO in order to display the end effect’s localization with respect to the virtual representation, just like the display of the location of a car in a map in a GPS-based navigation system. The geometrical transformation could be rigid or non-rigid. In literature, a wide variety of registration concepts and associated algorithms exist (see the next section for more details). The third key ingredient to a CAOS system is referencing, which is necessary to compensate for possible motion of the navigator and/or the TO during the surgical actions to be controlled. This is done by either attaching a so-called dynamic reference bases (DRB) holding three or more optical markers to the TO or immobilizing the TO with respect to the navigator.

The rest of the chapter is organized as follows. Section 1.2 will review the state-of-the-art examples of basic elements of CAOS systems. Section 1.3 will present clinical fields of applications. In Sect. 1.4, future perspectives will be outlined, followed by conclusion in Sect. 1.5.

1.2 Basic Elements of CAOS Systems

1.2.1 Virtual Object

The VO in each CAOS system is defined as a sufficiently realistic representation of the musculoskeletal structures that allows the surgeon to plan the intended intervention, as exemplified in Fig. 1.1a Intra-operatively, it also serves as the “background” into which the measured position of a surgical instrument can be visualized (see Fig. 1.1b for an example). Though most of the time VO is derived from image data of the patient, it can also be created directly from intra-operative digitization without using any medical image data. Below detailed examples of different forms of VOs will be reviewed.

Fig. 1.1
figure 1

Example of CT-based navigational feedback. These screenshots show a CT-based CAOS system during pre-operative planning (a) and intra-operative navigation (b) of pedicle screw placement. (Courtesy of Brainlab AG, Munich, Germany)

When the VO is derived from medical image data, these data may be acquired at two points in time: either pre-operatively or intra-operatively. Two decades ago, the VOs of majority CAOS systems were derived from pre-operatively acquired CT scans, and a few groups also tried to use magnetic resonance imaging (MRI) [6, 7]. In comparison with MRI, CT has clear advantages of excellent bone-soft tissue contrast and no geometrical distortion despite its acquisition inducing radiation exposure to the patient. Soon after the introduction of the first CAOS systems, the limitations of pre-operative VOs were observed, which led to the introduction of intra-operative imaging modalities. More specifically, the bony morphology may have changed between the time of image acquisition and the actual surgical procedure. As a consequence, the VO may not necessarily correspond to the TO any more leading to unpredictable inaccuracies during navigation or robotic procedures. This effect can be particularly adverse for traumatology in the presence of unstable fractures. To overcome this problem in the field of surgical navigation, the use of intra-operative CT scanning has been proposed [8], but the infrastructural changes that are required for the realization of this approach are tremendous, often requiring considerable reconstruction of a hospital’s facilities. This has motivated the development of navigation systems based on fluoroscopic images [9,10,11]. The image intensifier is a well-established device during orthopaedic and trauma procedures but has the limitations that the images generated with a fluoroscope are usually distorted and that one-dimensional information gets lost due to image projection. To use these images as VOs therefore requires the calibration of the fluoroscope which aims to compute the image projection model and to compensate for the image distortion [9,10,11]. The resultant systems are therefore known as “fluoroscopy-based navigation systems” in literature [9,10,11]. Additional feature offered by a fluoroscopy-based navigation system is that multiple images acquired from different positions are co-registered to a common coordinate system established on the target structure via the DRB technique. Such a system can thus provide visual feedback just like the use of multiple fluoroscopes placed at different positions in constant mode but without the associated radiation exposure, which is a clear advantage (see Fig. 1.2 for an example). This technique is therefore also known as “virtual fluoroscopy” [11]. Despite the fact that in such a system, only two-dimensional (2-D) projected images with low contrast are available, the advantages offered by a fluoroscopy-based navigation system preponderate for a number of clinical applications in orthopaedics and traumatology.

Fig. 1.2
figure 2

Example of fluoroscopy-based navigation. This screenshot shows the fluoroscopy-based navigation for distal locking of an intramedullary nail. (Courtesy of Brainlab AG, Munich, Germany)

In order to address the 2-D projection limitation of a fluoroscopy-based navigation system, a new imaging device was introduced [12] that enables the intra-operative generation of 3-D fluoroscopic image data. It consists of a motorized, isocentric C-arm that acquires series of 50–100 2-D projections and reconstructs from them 13 × 13 × 13 cm3 volumetric datasets which are comparable to CT scans. Being initially advocated primarily for surgery at the extremities, this “fluoro-CT” has been adopted for usage with a navigation system and has been applied to several anatomical areas already [13, 14]. As a major advantage, the device combines the availability of 3-D imaging with the intra-operative data acquisition. “Fluoro-CT” technology is under continuous development involving smaller and nonisocentric C-arms, “closed” C-arm, i.e. O-armTM design [15, 16], faster acquisition speeds, larger field of view, and also flat panel technology.

A last category of navigation systems functions without any radiological images as VOs. Instead, the tracking capabilities of the system are used to acquire a graphical representation of the patient’s anatomy by intra-operative digitization. By sliding the tip of a tracked instrument on the surface of a surgical object, the spatial location of points on the surface can be recorded. Surfaces can then be generated from the recorded sparse point clouds and used as the virtual representation of the surgical object. Because this model is generated by the operator, the technique is therefore known as “surgeon-defined anatomy” (SDA) (Fig. 1.3). It is particularly useful when soft tissue structures such as ligaments or cartilage boundaries are to be considered that are difficult to identify on CTs or fluoroscopic images [17]. Moreover, with SDA-based systems, some landmarks can be acquired even without the direct access to the anatomy. For instance, the centre of the femoral head, which is an important landmark during total hip and knee replacement, can be calculated from a recorded passive rotation of the leg about the acetabulum. It should be noted that the generated representations are often rather abstract and not easy to interpret as exemplified in Fig. 1.3. This has motivated the development of the so-called “bone morphing” techniques [18, 19], which aim to derive a patient-specific model from a generic statistical forms of the target anatomical structure and a set of sparse points that are acquired with the SDA technique [20]. As the result, a realistic virtual model of the target structure can be presented and used as a VO without any conventional image acquisition (Fig. 1.4).

Fig. 1.3
figure 3

Navigation using surgeon-defined anatomy approach. This virtual model of a patient’s knee is generated intra-operatively by digitizing relevant structures. Although a very abstract representation, it provides sufficient information to enable navigated high tibial osteotomy

Fig. 1.4
figure 4

An example of bone morphing. Screenshots of different stages of an intra-operative bone morphing process. (a) Point acquisition; (b) calculation of morphed model; and (c) verification of final result. (Courtesy of Brainlab AG, Munich, Germany)

1.2.2 Registration

Position data that is used intra-operatively to display the current tool location (navigation system) or to perform automated actions according to a pre-operative plan (robot) are expressed in the local coordinate system of the VO. In general, this coordinate system differs from the one in which the navigator operates intra-operatively. In order to bridge this gap, the mathematical relationships between both coordinate spaces need to be determined. When pre-operative images are used as VOs, this step is performed interactively by the surgeon during the registration, also known as matching. A wide variety of different approaches have been developed and realized following numerous methodologies [21].

Early CAOS systems implemented paired-point matching and surface matching [22]. The operational procedure for paired-point matching is simple. Pairs of distinct points are defined pre-operatively in the VO and intra-operatively in the TO. The points on the VO are usually identified pre-operatively using the computer mouse, while the corresponding points on the TO are usually done intra-operatively with a tracked probe. In the case of a navigation system, the probe is tracked by the navigator, and for a robotic surgery, it is mounted onto the robot’s actuator [23]. Although paired-point matching is easy to solve mathematically, the accuracy of the resultant registration is low. This is due to the fact that the accuracy of paired-point matching depends on an optimal selection of the registration points and the exact identification of the associated pairs which is error prone. One obvious solution to this problem is to implant artificial objects to create easily and exactly identifiable fiducials for an accurate paired-point matching [23]. However, the requirement of implanting these objects before the intervention causes extra operation as well as associated discomfort and infection risk for the patient [24]. Consequently, none of these methods have gained wide clinical acceptance. The other alternative that has been widely adopted in early CAOS systems is to complement the paired-point matching with surface matching [25, 26], which does not require implanting any artificial object and only uses the surfaces of the VO as a basis for registration.

Other methods to compute the registration transformation without the need for extensive pre-operative preparation utilize intra-operative imaging such as calibrated fluoroscopic images or calibrated ultrasound images. As described above, a limited number of fluoroscopic images (e.g. two) acquired at different positions are calibrated and co-registered to a common coordinate system established on the target structure. A so-called “2-D-3-D registration” procedure can then be used to find the geometrical transformation between the common coordinate system and a pre-operatively acquired 3-D CT dataset by maximizing a similarity measurement between the 2-D projective representations and the associated digitally reconstructed radiographs (DRRs) that are created by simulating X-ray projections (see Fig. 1.5 for an example). Intensity-based as well as feature-based approaches have been proposed before. For a comprehensive review of different 2-D-3-D registration techniques, we refer to [21].

Fig. 1.5
figure 5

An example of CT-fluoro matching. Screenshots of different stages of a CT-fluoro matching process. (a) Preregistration for CT-fluoro matching and (b) results of CT-fluoro matching. (Courtesy of Brainlab AG, Munich, Germany)

Another alternative is the employment of intra-operative ultrasonography. If an ultrasound probe is tracked by a navigator and its measurements are calibrated, it may serve as a spatial digitizer with which points or landmarks on the surfaces of certain subcutaneous bony structures may be acquired. This is different from the touch-based digitization done with a conventional probe which usually requires an invasive exposure of the surfaces of the target structures. Two different tracked mode ultrasound probes are available. A (amplitude)-mode ultrasound probes can measure the depth along the acoustic axis of the device. Placed on the patient’s skin, they can measure percutaneously the distance to tissue borders, and the resulting point coordinates can be used as inputs to any feature-based registration algorithm. The applicability of this technique has been demonstrated previously but with certain limitations which prevent its wide usage [27, 28]. More specifically, the accuracy of the A-mode ultrasound probe-based digitization depends on how well the probe can be placed perpendicularly to the surfaces of the target bony structures, which is not an easy task when the subcutaneous soft tissues are thick. Moreover, the velocity of sound during the probe calibration is usually different from the velocity of sound when the probe is used for digitization as the latter depends on the properties of the traversed tissues. Such a velocity difference will lead to unpredictable inaccuracies when the probe is used to digitize deeply located structures. As a consequence, the successful application of this technique remains limited to a narrow field of application. In contrast to an A-mode probe, a B (brightness)-mode ultrasound probe scans a fan-shaped area. It is therefore able to detect also surfaces that are examined from an oblique direction, though the errors caused by the velocity difference still persist. In order to extract the relevant information for the registration of pre-operative CT scans, the resulting, usually noisy images need to be processed [29]. As for the intra-operative processing of fluoroscopic images, the use of B-mode ultrasound for registration is not reliable in every case and consequently remains the subject of CAOS research [30, 31].

It is worth to point out that if the VO is generated intra-operatively, registration is an inherent process [21]. This is due to the fact that since the imaging device is tracked during data acquisition, the position of any acquired image is recorded with respect to the local coordinate system established on the TO. The recorded device position, together with the additional image calibration process, automatically establishes the spatial relationship between the VO and the TO during image acquisition, which is a clear advantage over the interactive registration in the case of pre-operative images serving as VOs. Therefore, registration is not an issue when using intra-operative CT, 2-D, 3-D fluoroscopy or O-arm, or the SDA concept.

Radermacher et al. [32] introduced an alternative way to match pre-operative planning with the intra-operative situation using individual templates. The principle of individualized templates is to create customized templates based on patient-specific 3-D bone models that are normally segmented from pre-operative 3-D data such as CT or MRI scan. One feature about the individual templates is that small reference areas of the bone structures are integrated into the templates as the contact faces. By this means, the planned position and orientation of the template in spatial relation to the bone are stored in a structural way and can be reproduced intra-operatively by adjusting the contact faces of the templates until an exact fit to the bone is achieved. By integrating holes and/or slots, individualized templates function as tool guides, e.g. for the preparation of pedicle screw holes [32] or as cutting jigs used in total knee and hip replacement surgery [33,34,35].

1.2.3 Navigator

Registration closes the gap between VO and TO. The navigator enables this connection by providing a global coordinate space. In addition, it links the surgical end effectors, with which a procedure is carried out, to the TO that they act upon. From a theoretical standpoint, it is the only element in which surgical navigation systems and surgical robotic systems differ.

1.2.3.1 Robots

For this type of CAOS technology, the robot itself is the navigator. Intra-operatively, it has to be registered to the VO in order to realize the plan that is defined in the pre-operative image dataset. The end effectors of a robot are usually designed to carry out specific tasks as part of the therapeutic treatment. Depending on how the end effectors of a robot act on the patient, two different types of robots can be found in literature. The so-called active robots conduct a specific task autonomously without additional support by the surgeon. Such a system has been applied for total joint replacement [5], but their clinical benefit has been strongly questioned [36]. For traumatology applications, the use of active robots has only been explored in the laboratory setting [37, 38]. One possible explanation is that the nature of fracture treatment is an individualized process that does not include many steps that an active robot can repetitively carry out.

In contrast to active robotic devices, passive or semi-active robots do not carry out a part of the intervention autonomously but rather guide or assist the surgeon in positioning the surgical tools. At present there are two representatives of this class, both for bone resection during total knee arthroplasty (TKA). The Navio system (Blue Belt Technologies Inc. Pittsburgh, PA, USA) [39] is a hand-held semi-active robotic technology for bone shaping that allows a surgeon to move freely in order to resect the bone as long as this motion stays within a pre-operatively defined safety volume. The Mako system [40] is a passive robotic arm system providing oriental and tactile guidance. Both the Navio and the Mako systems require additional tracking technology as described in the next sub-section. During the surgical procedure, the system is under the direct surgeon control and gives real-time tactile feedback to the surgeon. Other semi-active robots such as SpineAssist (Mazor Robotics Ltd., Israel) can be seen as intelligent gauges that place, for example, cutting jigs or drilling guides automatically [41, 42].

1.2.3.2 Tracker

The navigator of a surgical navigation system is a spatial position tracking device. It determines the location and orientation of objects and provides these data as 3-D coordinates or 3-D rigid transformations. Although a number of tracking methods based on various physical media, e.g. acoustic, magnetic, optical, and mechanical methods, have been used in the early surgical navigation systems, most of today’s products rely upon optical tracking of objects using operating room (OR) compatible infrared light that is either actively emitted or passively reflected from the tracked objects. To track surgical end effectors with this technology then requires the tools to be adapted with reference bases holding either light-emitting diodes (LED, active) or light-reflecting spheres or plates (passive). Tracking patterns with known geometry by means of video images has been suggested [43, 44] as an inexpensive alternative to an infrared-light optical tracker.

Optical tracking of surgical end effectors requires a direct line of sight between the tracker and the observed objects. This can be a critical issue in the OR setting. The use of electromagnetic tracking systems has been proposed to overcome this problem. This technology involves a homogeneous magnetic field generator that is usually placed near to the surgical situs and the attachment of receiver coils to each of the instruments allowing measuring their position and orientation within the magnetic field. This technique senses positions even if objects such as the surgeon’s hand are in between the emitter coil and the tracked instrument. However, the homogeneity of the magnetic field can be easily disturbed by the presence of certain metallic objects causing measurement artefacts that may decrease the achievable accuracy considerably [45, 46]. Therefore, magnetic tracking has been employed only in very few commercial navigation systems and with limited success.

Recently inertial measurement unit (IMU)-based navigation devices have attracted more and more interests [47,48,49,50,51]. These devices attempt to combine the accuracy of large-console CAOS systems with the familiarity of conventional alignment methods and have been successfully applied to applications including TKA [47, 48], pedicle screw placement [49], and periacetabular osteotomy (PAO) surgery [50, 51]. With such devices, the line-of-sight issues in the optical surgical navigation systems can be completely eliminated. Technical limitations of such devices include (a) relatively lower accuracy in comparison with optical tracking technique and (b) difficulty in measuring translations.

1.2.4 Referencing

Intra-operatively, it is unavoidable that there will be relative motions between the TO and the navigator due to surgical actions. Such motions need to be detected and compensated to secure surgical precision. For this purpose, the operated anatomy is linked to the navigator. For robotic surgery this connection is established as a physical linkage. Large active robots, such as the early machines used for total joint replacement, come with a bone clamp that tightly grips the treated structure or involve an additional multi-link arm, while smaller active and semi-active devices are mounted directly onto the bone. For all other tracker types, bone motion is determined by the attachment of a DRB to the TO [52], which is designed to house infrared LEDs, reflecting markers, acoustic sensors, or electromagnetic coils, depending on the employed tracking technology. Figure 1.6 shows an example of a DRB for an active optical tracking system that is attached to the spinous process of a lumbar vertebra. Since the DRB is used as an indicator to inform the tracker precisely about movements of the operated bone, a stable fixation throughout the entire duration of the procedure is essential.

Fig. 1.6
figure 6

Dynamic reference base. A dynamic reference base allows a navigation system to track the anatomical structure that the surgeon is operating on. In the case of spinal surgery, this DRB is usually attached to the processus spinosus with the help of a clamping mechanism. It is essential that it remains rigidly affixed during the entire usage of the navigation system on that vertebra

1.3 Clinical Fields of Applications

Since the mid-1990s when first CAOS systems were successfully utilized for the insertion of pedicle screws at the lumbar and thoracic spine and total hip replacement procedures [3, 4], a large number of modules covering a wide range of traumatological and orthopaedic applications have been developed, validated in the laboratory and in clinical trials. Some of them needed to be abandoned, because the anticipated benefit failed to be achieved or the technology proved to be unreliable or too complex to be used intra-operatively. Discussing all these applications would go beyond the focus of this article. Thus, here we focus on a review of the most important applications with the most original technological approaches.

While there was clearly one pioneering example of robot-assisted orthopaedic surgery – ROBODOC [5] – the first spinal navigation systems were realized independently by several research groups, almost in parallel [3, 4, 52,53,54,55,56]. These systems used pre-operative CT scans as the VO, relied upon paired-point and surface matching techniques for registration, and were based on optical or electromagnetic trackers. Their initial clinical success [57,58,59] boosted the development of new CAOS systems and modules. While some groups tried to use the existing pedicle screw placement systems for other clinical applications, others aimed to apply the underlying technical principle to new clinical challenges by developing highly specialized navigation systems [60, 61]. With the advent of alternative imaging methods for the generation of VOs, the indication for the use of one or the other method was evaluated more critically. For instance, it became evident that lumbar pedicle screw insertion in the standard degenerative case could be carried out with fluoroscopy-based navigation sufficiently accurate, thus avoiding the need for a pre-operative CT.

A similar development took place for total knee replacement. Initially, this procedure was supported by active [36, 62] and semi-active or passive [39, 40] robots, as well as navigation systems using pre-operative CTs [63], but with a few exceptions, the SDA approach [64] is today’s method of choice.

Fluoroscopy-based navigation still seems to have a large potential to explore new fields of application. The technology has been mainly used in spinal surgery [65]. Efforts to apply it to total hip arthroplasty (THA) [66] and the treatment of long-bone fractures [67] have been commercially less successful. The intra-operative 3-D fluoroscopy or O-arm has been explored intensively [13,14,15,16]. It is expected that with the advent of the flat panel technology, the use of fluoro-CT as a virtual object generator will significantly grow [16].

Recently, computer-assisted surgery using individual templates has gained increasing attention. Initially developed for pedicle screw fixation [32], such a technique has been successfully reintroduced to the market for total knee arthroplasty [33, 68, 69], hip resurfacing [34, 70], total hip arthroplasty [35], and pelvic tumour resection [71, 72] (see Fig. 1.7 for an example). It should be noted that most of the individual templates are produced using additive manufacturing techniques, while most of the associated implants are produced conventionally.

Fig. 1.7
figure 7

Patient-specific instrumentation for pelvic tumour resection surgery. These images show the application of patient-specific instrumentation for pelvic tumour treatment. Implant and template manufactured by Mobelife NV, Leuven, Belgium. (a) A pre-operative X-ray radiograph, (b) the implant; (c) the patient-specific guide; (d) a post-operative X-ray radiograph. (Courtesy of Prof. Dr. K Siebenrock, Inselspital, University of Bern, Switzerland)

1.4 Future Perspectives

Despite its touted advantages, such as decreased radiation exposure to the patient and the surgical team for certain surgical procedures and increased accuracy in most situations, surgical navigation has yet to gain general acceptance among orthopaedic surgeons. Although issues related to training, technical difficulty, and learning curve are commonly presumed to be major barriers to the acceptance of surgical navigation, a recent study [73] suggested that surgeons did not select them as major weaknesses. It has been indicated that barriers to adoption of surgical navigation are neither due to a difficult learning curve nor to a lack of training opportunities. The barriers to adoption of navigation are more intrinsic to the technology itself, including intra-operative glitches, unreliable accuracy, frustration with intra-operative registration, and line-of-sight issues. These findings suggest that significant improvements in the technology will be required to improve the adoption rate of surgical navigation. Addressing these issues from the following perspectives may provide solutions in the continuing effort to implement surgical navigation in everyday clinical practice.

  • 2-D or 3-D image stitching. Long-bone fracture reduction and spinal deformity correction are two typical clinical applications that frequently use the C-arm in its operation. Such a surgery usually involves corrective manoeuvers to improve the sagittal or coronal profile. However, intra-operative estimation of the amount of correction is difficult, especially in longer instrumentation. Mostly, anteroposterior (AP) and lateral fluoroscopic images are used but have the disadvantage to depict only a small portion of the target structure in a single C-arm image due to the limited field of view of a C-arm machine. As such, orthopaedic surgeons nowadays are missing an effective tool to image the entire anatomical structure such as the spine or long bones during surgery for assessing the extent of correction. Although radiographs obtained either by using a large field detector or by image stitching can be used to image the entire structure, they are usually not available for intra-operative interventions. One alternative is to develop methods to stitch multiple intra-operatively acquired small fluoroscopic images to be able to display the entire structure at once [74, 75]. Figure 1.8 shows an image stitching example for spinal intervention. The same idea can be extended to 3-D imaging to create a panoramic cone beam computed tomography [76]. At this moment, fast and easy-to-use 2-D or 3-D image stitching systems are still under development, and as the technology evolves, surgical benefits and improved clinical outcomes are expected.

    Fig. 1.8
    figure 8

    Image stitching for spinal interventions. Several small field-of-view C-arm images are stitched into one big image to depict the entire spine

  • Image fusion. Fusion of multimodality pre-operative image such as various MRI or CT datasets with intra-operative images would allow for visualization of critical structures such as nerve roots or vascular structures during surgical navigation. Different imaging modalities provide complementary information regarding both anatomy and physiology. The evidence supporting this complementarity has been gained over the last few years with increased interest in the development of platform hardware for multimodality imaging. Because multimodality images by definition contain information obtained using different imaging methods, they introduce new degrees of freedom, raising questions beyond those related to exploiting each single modality separately. Processing multimodality images is then all about enabling modalities to fully interact and inform each other. It is important to choose an analytical model that faithfully represents the link between the modalities without imposing phantom connections or suppressing existing ones. Hence it is important to be as data driven as possible. In practice, this means making the fewest assumptions and using the simplest model, both within and across modalities. Example models include linear relationships between underlying latent variables; use of model-independent priors such as sparsity, non-negativity, statistical independence, low rank, and smoothness; or both. Such a principle has been successfully applied to solving challenging problems in a variety of applications [77]. Despite the evident potential benefit, the knowledge of how to actually exploit the additional diversity that multimodality images offer is currently at its preliminary stage and remains open for exploration.

  • Statistical shape and deformation analysis. Statistical shape and deformation analysis [78] has been shown to be useful for predicting 3-D anatomical shape and structures from sparse point sets that are acquired with the SDA technique. Such a technique is heavily employed in so-called “image-free” navigation systems that are commercially available in the market, mainly for knee and hip surgery. However, with the availability of statistical shape models of other anatomical regions, the technique could be applied to any part of the skeleton. Such approaches bear significant potential for future development of computer navigation technology since they are not at all bound to the classical pointer-based acquisition of bony features. In principle, the reconstruction algorithms can be tuned to any type of patient-specific input, e.g. intra-operatively acquired fluoroscopic images [79] or tracked ultrasound [30], thereby potentially enabling new minimally invasive procedures. Figure 1.9 shows an example of bone surface reconstruction from calibrated fluoroscopic images and a statistical shape model. Moreover, prediction from statistical shape models is possible not only for the geometric shape of an object. Given statistical shape and intensity models, “synthetic CT scans” could be predicted from intra-operatively recorded data after a time-consuming computation. With more and more computations shifted from CPUs to graphics processing units (GPUs), it is expected that statistical shape and deformation analysis-based techniques will be used in more and more CAOS systems [80].

    Fig. 1.9
    figure 9

    An example of statistical shape model-based 2-D-3-D reconstruction. Reconstruction of bone surface from two calibrated fluoroscopic images and a statistical shape model using deformable registration

  • Biomechanical modelling. Numerical models of human anatomical structures may help the surgeon during the planning, simulation, and intra-operative phases with the final goal to optimize the outcome of orthopaedic surgical interventions. The terms “physical” or “biomechanical” are often used. While most of existing biomechanical models serve for the basic understanding of physical phenomena, only a few have been validated for the general prediction of consequences of surgical interventions.

    The situation for patient-specific models is even more complex. To be used in clinical practice, ideally the exact knowledge of the underlying geometrical tissue configuration and associated mechanical properties as well as the loading regime is required as input for appropriate mathematical frameworks. In addition these models will not only be used pre-operatively but need to function also in near real time in the operating theatre.

    First attempts have been made to incorporate biomechanical simulation and modelling into the surgical decision-making process for orthopaedic interventions. For example, a large spectrum of medical devices exists for correcting deformities associated with spinal disorders. Driscoll et al. [81] developed a detailed volumetric finite element model of the spine to simulate surgical correction of spinal deformities and to assess, compare, and optimize spinal devices. Another example was presented in [82] where the authors showed that with biomechanical modelling the instrumentation configuration can be optimized based on clinical objectives. Murphy et al. [83] presented the development of a biomechanical guidance system (BGS) for periacetabular osteotomy. The BGS aims to provide not only real-time feedback of the joint repositioning but also the simulated joint contact pressures.

    Another approach is the combined use of intra-operative sensing devices with simplified biomechanical models. Crottet et al. [84] introduced a device that intra-operatively measures knee joint forces and moments and evaluated its performance and surgical advantages on cadaveric specimens using a knee joint loading apparatus. Large variation among specimens reflected the difficulty of ligament release and the need for intra-operative force monitoring. A commercial version of such a device (e-LIBRA Dynamic Knee Balancing System, Synvasive Technology, El Dorado Hills, CA, USA) became available in recent years and is clinically used (see, e.g. [85]). It is expected that incorporation of patient-specific biomechanical modelling into CAOS systems with or without the use of intra-operative sensing devices may eventually increase the quality of surgical outcomes [86]. Research activities must focus on existing technology limitations and models of the musculoskeletal apparatus that are not only anatomically but also functionally correct and accurate.

  • Musculoskeletal imaging. Musculoskeletal imaging is defined as the imaging of bones, joints, and connected soft tissues with an extensive array of modalities such as X-ray radiography, CT, ultrasonography, and MRI. For the past two decades, rapid but cumulative advances can be observed in this field, not only for improving diagnostic capabilities with the recent advancement on low-dose X-ray imaging, cartilage imaging, diffusion tensor imaging, MR arthrography, and high-resolution ultrasound but also for enabling image-guided interventions with the introduction of real-time MRI or CT fluoroscopy, molecular imaging with PET/CT, and optical imaging into operating room [87].

    One recent advancement that has found a lot of clinical applications is the EOS 2-D/3-D image system (EOS imaging, Paris, France), which was introduced to the market in 2007. The EOS 2-D/3-D imaging system [88] is based on the Nobel Prize-winning work of French physicist Georges Charpak on multiwire proportional chamber, which is placed between the X-rays emerging from the radiographed object and the distal detectors. Each of the emerging X-rays generates a secondary flow of photons within the chamber, which in turn stimulate the distal detectors that give rise to the digital image. This electronic avalanche effect explains why a low dose of primary X-ray beam is sufficient to generate a high-quality 2-D digital radiograph, making it possible to cover a field of view of 175 cm by 45 cm in a single acquisition of about 20s duration [89]. With an orthogonally co-linked, vertically movable, slot-scanning X-ray tube/detector pairs, EOS has the benefit that it can take a pair of calibrated posteroanterior (PA) and lateral (LAT) images simultaneously [90]. EOS allows the acquisition of images while the patient is in an upright, weight-bearing (standing, seated, or squatting) position and can image the full length of the body, removing the need for digital stitching/manual joining of multiple images [91]. The quality and nature of the image generated by EOS system are comparable or even better than computed radiography (CR) and digital radiography (DR) but with much lower radiation dosage [90]. It was reported by Illés et al. [90] that absorbed radiation dose by various organs during a full-body EOS 2-D/3-D examination required to perform a surface 3-D reconstruction was 800–1000 times less than the amount of radiation during a typical CT scan required for a volumetric 3-D reconstruction. When compared with conventional or digitalized radiographs [92], EOS system allows a reduction of the X-ray dose of an order 80–90%. The unique feature of simultaneously capturing a pair of calibrated PA and LAT images of the patient allows a full 3-D reconstruction of the subject’s skeleton [90, 93, 94]. This in turn provides over 100 clinical parameters for pre- and post-operative surgical planning [90]. With a phantom study, Glaser et al. [95] assessed the accuracy of EOS 3-D reconstruction by comparing it with 3-D CT. They reported a mean shape reconstruction accuracy of 1.1±0.2 mm (maximum 4.7 mm) with 95% confidence interval of 1.7 mm. They also found that there was no significant difference in each of their analysed parameters (p > 0.05) when the phantom was placed in different orientations in the EOS machine. The reconstruction of 3-D bone models allows analysis of subject-specific morphology in a weight-bearing situation for different applications to a level of accuracy which was not previously possible. For example, Lazennec et al. [96] used the EOS system to measure pelvis and acetabular component orientations in sitting and standing positions. Further applications of EOS system in planning total hip arthroplasty include accurate evaluation of femoral offset [97] and rotational alignment [98]. The low dose and biplanar information of the EOS 2-D/3-D imaging system introduce key benefits in contemporary radiology and open numerous and important perspectives in CAOS research.

    Another novel technology on 2-D/3-D imaging was introduced in [99], which had the advantage of being integrated with any conventional X-ray machine. A mean reconstruction parameter of 1.06±0.20 mm was reported. This technology has been used for conducting 3-D pre-operative planning and post-operative treatment evaluation of TKA based on only 2-D long leg standing X-ray radiographs [100].

  • Artificial intelligence, machine learning, and deep learning. Recently artificial intelligence and machine learning-based methods have gained increasing interest in many different fields including musculoskeletal imaging and surgical navigation. Most of these methods are based on ensemble learning principles that can aggregate predictions of multiple classifiers and demonstrate superior performance in various challenging problems [77, 101, 102]. A crucial step in the design of such systems is the extraction of discriminant features from the images [103]. In contrast, many deep learning algorithms that have been proposed recently, which are based on models (networks) composed of many layers that transform input data (e.g. images) to outputs (e.g. segmentation), let computers learn the features that optimally represent the data for the problem at hand. The most successful type of models for image analysis to date are convolutional neural networks (CNN) [104], which contain many layers that transform their input with convolution filters of a small extent. Deep learning-based methods have been successfully used to solve many challenging problems in computer-aided orthopaedic surgery [105,106,107,108]. Figure 1.10 shows an example of the application of cascaded fully convolutional networks (FCN) for automatic segmentation of lumbar vertebrae from CT images [108]. It is expected that more and more solutions will be developed based on different types of deep learning techniques.

    Fig. 1.10
    figure 10

    A schematic view of using cascaded fully convolutional networks (FCN), which consists of a localization net and a segmentation net for automatic segmentation of lumbar vertebrae from CT images

1.5 Conclusions

More than two decades have passed since the first robot and navigation systems for CAOS were introduced. Today this technology has emerged from the laboratory and is being routinely used in the operating theatre and might be about to become state of the art for certain orthopaedic procedures.

Still we are at the beginning of a rapid process of evolution. Existing techniques are being systematically optimized, and new techniques will constantly be integrated into existing systems. Hybrid CAOS systems are under development, which will allow the surgeon to use any combinations of the above-described concepts to establish virtual object information. New generations of mobile imaging systems, inherently registered, will soon be available. However research focus should particularly be on alternative tracking technologies, which remove drawbacks of the currently available optical tracking and magnetic devices. This in turn will stimulate the development of less or even non-invasive registration methods and referencing tools. Force-sensing devices and real-time computational models may allow establishing a new generation of CAOS systems by going beyond pure kinematic control of the surgical actions. For keyhole procedures there is distinct need for smart end effectors to complement the surgeon in its ability to perform a surgical action. The recent advancement on smart instrumentation, medical robotics, artificial intelligence, machine learning, and deep learning techniques, in combination with big data analytics, may lead to smart CAOS systems and intelligent orthopaedics in the near future.