Introduction

With the advent of Minimally Invasive Surgery (MIS), intraoperative imaging started to play a crucial role in different fields, such as neurosurgery,121 urology81 and nephrectomy,35 to access hidden targets, allow intraoperative optical biopsy, guide navigation and, in general, to guarantee minimal invasiveness and maximal safety. In the last decades, several advancements have been done in the field of intraoperative imaging, leading to real-time (or quasi-real time) systems with higher resolution, efficiency, lower costs and able to execute complex data analyses.116

Intra-Operative UltraSound (iOUS), X-ray, Optical Coherence Tomography (OCT), intra-operative Magnetic Resonance Imaging (iMRI), Nuclear Medicine (NM), endo/laparoscopy, PhotoAcoustic (PA), and Raman imaging are among the most rapidly evolving modalities, even if with different levels of diffusion in clinics. In Fig. 1, exemplary intra-operative images are shown. These imaging modalities are commonly exploited for different surgical tasks and in different surgical phases, according to their specifications. Tables 1 and 2 summarize this information, while Table 3 highlights the main clinical applications for each modality.

Figure 1
figure 1

Surveyed imaging modalities: X-rays, OCT (Optical Coherence Tomography), PA (PhotoAcoustic) imaging, endo/laparoscopy, iMRI (intra-operative Magnetic Resonance Imaging), iOUS (intra-Operative UltraSound), nuclear medicine, and Raman.

Full size image

Considering how fast the field of intra-operative imaging is evolving, the motivations behind reviewing such a topic resides in the fact that, by analyzing the relevant state of art, we found that the majority of published reviews are either focused on technical aspects (e.g. AR,203 anatomy segmentation,126 deep-learning processing111) or limited to a specific imaging modality (e.g. OCT,31 Endo/laparoscopy,113 iMRI,14 Raman.160)

The closer work to ours is the one presented in Ref. 5, which, however, only surveys emerging imaging modalities (i.e. fluorescence, PA, Raman and nuclear imaging). As a result, considering such information, the importance of intraoperative imaging in the surgery of the future will come to light. The goal of the review is, instead, to provide a compact and updated source of information for young researchers who are approaching the wide field on intra-operative imaging, and a reference overview document for those already working in the field.

This review article discusses the basic principles and development directions of intra-operative imaging modalities and is not intended to be a comprehensive review of intra-operative imaging applications. Eight imaging modalities are surveyed: iOUS (section “Intra-operative Ultrasound (iOUS) Imaging”), X-ray (section “X-ray”), OCT (section “Optical Coherence Tomography (OCT)”), (section “Intra-operative Magnetic Resonance Imaging (iMRI)”), Endo/laparoscopy (section “Endo/laparoscopy”), PA imaging (section “Photoacoustic Imaging”), Nuclear medicine (section “Nuclear Medicine”), Raman spectroscopy (section “Raman Spectroscopy”). To conclude this review, an overview of integrated surgical rooms, as well as a survey of real-time/quasi real-time image processing techniques for intraoperative applications, is presented (section “Hybrid Surgical Rooms and Real-Time/Quasi Real-Time Image Processing”). This way, we aim at providing the reader with useful information about the forthcoming trend to install ad-hoc operating rooms (ORs).

Table 1 Imaging technique specifications.
Full size table
Table 2 Enhanced tissues.
Full size table
Table 3 Principal surgical applications.
Full size table

To limit the overlap with previous survey papers, we selected the articles according to the following criteria:

  • Papers about clinical application, mainly reported in Tables 4 and 5, had to be published from 2010 onward; no restriction for papers introducing general concepts about imaging physical principles;

  • Papers not strictly discussing intra-operative applications (such as diagnosis and follow-up and clinical trials) were not considered.

The 40% of the cited articles discuss the technical aspects of the investigated modalities, the remaining 60% are about clinical applications.

Table 4 Application samples for intra-operative ultrasound (iOUS), X-ray and optical coherence tomography (OCT).
Full size table
Table 5 Application samples for interventional magnetic resonance imaging (iMRI), endo/laparoscopy, photoacustic (PA) imaging, nuclear medicine and Raman spectroscopy.
Full size table

Intra-operative Ultrasound (iOUS) Imaging

UltraSounds (US) are a succession of rarefactions and compressions transmitted due to elastic forces between adjacent particles. Most diagnostic US has frequencies in the range 2–20 MHz.2 The way elastic waves are reflected provides information about internal tissues.

US imaging techniques have been introduced as intra-operative imaging modalities (iOUS) thanks to their real-time acquisition, reduced OR encumbrance and limited costs, which allow full in-room compatibility.

Technological Advancements

Recent technological advancements of iOUS are related to:

  • Probe miniaturization, down to few mm in diameter, which allows the probe insertion in hollow cavities, such vessels in vascular or cardiac procedures and in the patient abdomen trough the trochar port during minimal invasive surgery. This led to Intra-Cardiac Echocardiography (ICE), TransEsophageal Echocardiography (TEE), TransRectal US (TRUS) and IntraVascular US (IVUS). On this regard, an interesting comparative study IVUS vs OCT has been recently published.151

  • Probe navigation and 3D probes realization, which allows the visualization of a volumetric dataset, rather than a planar slice.

  • Signal processing capabilities, which allow for real-time visualization of inner anatomical structures and surgical tools.

  • High focused US implementation, for precise targeting of therapy (see paragraph 5.2).

Volumetric US imaging is surely among the most impacting advancements of iOUS systems in the actual clinical practice. A review on real-time 3D US imaging technology has been recently published.86

Volumetric US imaging can be achieved using 3D probes17 and spatial localizing the probe with external measuring devices and properly calibrated.52 Alternatively, US volumetric probes can be rigidly attached to robot end-effectors and provide intra-operative guidance of surgical interventions.119

Deep learning has been recently employed to reconstruct the 3D volume without any external tracking device.149 3D volume reconstruction can be achieved with frame rates up to 120 frames/s.33

The technological pharmacological combination of capsule endoscopy with US-mediated Targeted Drug Delivery (UmTDD) carries new potential for treatment of diseases throughout the gastro-intestinal tract. Finally Contrast-Enhanced US (CEUS) are used during robotic-assisted kidney surgery6 to enhance the visualization of macro and microvasculature of the kidneys.

Limitations and Open Issues

IOUS based devices are portable and low-cost systems for obtaining intra-operative information. Some open issues are still limiting their adoption in some clinical procedures, such as:

  • Poor tissue contrast due to low Signal-to-Noise Ratio (SNR), despite the adoption of contrast media (e.g. such as microbubbles). This is particularly limiting neurosurgical navigation, since the planning phase is currently done on pre-operative CT or MRI sequences.

  • Limited spatial resolution and FoV (inverse relationship, both are function of the excitation frequency of the transducer).

X-ray

X-ray based imaging techniques take advantage of the capability of high-energy photons to penetrate the matter. Radiations are artificially generated by means of X-ray tubes and it is possible to adjust the beam energy depending on patient size and desired tissue contrast. Information about the internal anatomy of the subject are revealed by photon attenuation through the matter.179 Projective (2D) or tomographic (3D) images can be generated depending on the device configuration. Both type of images can be acquired over time, although only 2D digital radiography (i.e. fluoroscopy) offers true real time imaging. High contrast is obtained for bone and air; soft tissues can be enhanced by injecting a radiocontrast agent. The use of X-rays is limited by the maximum patient’s radiation exposure stated by radiation protection limits.

Technological Advancements

X-rays for intraoperative use were introduced in the ’50, when Philips developed the first flexible and portable device known as C-arm. Nowadays, digital flat panel detectors replaced traditional image intensifiers, since they offer higher transducer efficiency with lower dose, higher spatial and radiometric resolution (100–200 \(\mu\)m and 14–16 bits respectively), fast sample rate (25–40Hz), larger FoV, and lower image degradation over the period of use. C-arms are traditionally used for static and cine 2D acquisition. However, since the arm can revolve 360 °C around the patient, Cone Beam Computed Tomography (CBCT) can be acquired for 3D volume reconstruction. In addition, in room CT devices are also available:

  1. 1.

    On rail intraoperative CT (iCT), which are normal diagnostic CT scanners that can be moved into the room through ceiling rails. In some cases, the scanner is fixed and the surgical couch can be moved inside the CT device;

  2. 2.

    Small and portable CT scanners which can be moved in and out from the surgical room.

The last technological advancements are now directed towards the possibility to enhance soft tissue contrast without injecting contrast mean. This can be obtained by means of: (1) dual energy X-ray sources; (2) use of different ionizing radiations, such as proton or carbon ion beams, to obtain proton radiography and tomography.166, 175 Even if such cutting edge technologies are not currently used for intraoperative applications, it is very likely they will be the next frontier of this in room modality.

Technological advancements about X-rays for guidance are related also to radiotherapy applications where X-ray beams are used not only for treatment, but also for guidance. To this purpose, stereoscopic radiographs for 3D reconstruction and CBCT are widely employed to verify patient’s position and localize the tumor.23, 177 In some centers, on rail CT and iCT are also employed for performing optimal adaptive radiotherapy treatments with the same quality of planning CT.141 Linear accelerator have been integrated with CT scanner as for TomoTherapy® and robotic X-ray arms as for CyberKnife™ for high precision radiosurgery treatments.162 Another promising technique relies in exploiting Cherenkov emission during irradiation in order to visualize, in real time, surface dose on the patient skin. This method has been proved for breast radiotherapy93 and total skin electron therapy,10 demonstrating the improvement of the irradiation quality assessment.

Limitations and Open Issues

The principal limitation of intraoperative X-ray based imaging is the invasiveness of ionizing radiation for biological tissues. In the last years, particular attention has been payed to reduce X-ray dose delivered both to patient and staff.1, 64 Especially for pediatric patients, other imaging techniques are preferred, when possible, to minimize the radiation exposure. From the image quality point of view, an important issue is represented by the presence of metal inserts which generate artifacts, especially for high density material.178 Many efforts are also made to improve the quality of CBCT reconstruction. In fact, due to the conic aperture of the beam, photon scatter represents a serious issue for image degradation. Many scatter correction algorithms have been proposed in literature.135, 222 However, standard practical solutions still remain inadequate.

Optical Coherence Tomography (OCT)

Optical coherence tomography85 is an imaging technique able to provide 1D (also named A-scan), 2D (B-scan) and 3D representations of biological tissue. It takes advantage of the optical reflection of light to obtain spatial information of the sample structure. By exploiting this physical propriety, it is possible to acquire high resolution images (axial resolution in the range of \(\mu\)m) without any tissue damage nor ionizing radiation dose delivered. Tissue details are revealed by time of flight of transmitted/reflected light signal, that is related to sample structure and composition. Ultrashort laser pulse, as well as low-coherence light, can be used as energy source. 2D images over time can be acquired and directly shown, meanwhile 4D representation (volumes over time) has been recently introduced.

Real-time OCT imaging has been made possible by Graphics Processing Units (GPU) computational power31 and spectral-domain paradigm.

OCT has full in-room compatibility, since no risks exist for patients and operators. Anyway, as discussed in Ref. 57, metallic surgical tools can affect OCT image quality (e.g. introducing shadow). For this reason, in order to allow a real-time intraoperative OCT, instead of a “stop and scan“ approach, instruments made of alternative materials (such as plastics and silicone) can be used.

Technological Advancements

Since the presentation of this new modality, reducing acquisition time and improving image quality were the most important challenges to deal with. The introduction of spectral-domain OCT as an alternative to time-domain OCT, allowed to reduce scanning time, making easier the investigation of bigger volume sample210, 218 and facilitating a real intraoperative usage. In addition to the spectral-domain strategy, another important improvement was the possibility to join the probe with microscopes, surgical instruments (such as needles), and laser modules.31

Limitations and Open Issues

Currently, main OCT limitations are due to the narrow FoV (including reduced depth of penetration) achievable by means of this modality. However, in Ref. 174, a possible methodology to overcome this limitation has been successfully tested, enabling an acquisition of a FoV up to \(20\times 20\) cm. If compared with IVUS, OCT has a smaller depth of penetration, that in turn affects the FoV.

Intra-operative Magnetic Resonance Imaging (iMRI)

MRI is based on the interaction of H+ proton spins immersed in a magnetic field and stimulated by Radio Frequency waves (RF pulse).

Tissues containing mobile protons, such soft tissues, present very high contrast in MRI. The contrast can be even modified in a process called pulse sequence, where a certain number of RF pulses and magnetic field gradients is set and combined to obtain an image with anatomical or functional appearance, such as for Perfusion MRI (Pe-MRI), MR Angiography (MRA), MR Venography (MRV), Diffusion Weighted Imaging (DWI), functional MRI (fMRI).80

Due to the high combination of parameter setting, MRI is a very versatile technique. It provides high image quality in terms of spatial and contrast resolution, it combines morphological and physiological information, it features multiplanar 2D acquisition in any direction and orientation, as well as 3D isotropic voxel acquisition. Moreover, it does not involve ionizing radiations, thus being less invasive than X-ray based imaging. On the other end, MRI is prone to several artifacts, most important being motion and magnetic field distortion and it can be dangerous for the patient in presence of metal implants and active implantable medical devices.

Table 6 Main characteristics of iMRI scanners.
Full size table

Technological Advancements

From the equipment perspective, the current possible configurations for iMRI can be grouped into 2 classes,83 whose main characteristics are reported in Table 6:

  • Low field scanners: with a static magnetic field \(\le\)1T, they are small and portable devices90 with a gap to allow access to the patient during the surgical procedures.

  • High field scanners: with a static magnetic field \(\ge\) 1.5T (closed bore). They are introduced to the OR by means of ceiling rails (or the patient is moved inside the scanner by means of a movable operative table.94) The main advantage is the higher image quality and the possibility to acquire non-anatomical images (DWI, Pe-MRI, MRA and fMRI).

Specific pulse sequences allowing rapid imaging have been developed for real-time or quasi real-time imaging (10–20 frames/s4,36).

Technological advancements led iMRI to be used as guide during radiotherapy and US based treatments. Image guidance in radiotherapy plays a crucial role for correct patient positioning, organ and tumor motion assessment during radiation delivery. By now, the scene has been dominated by US, optical tracking systems and X-ray based techniques, both for photon91 and proton based treatments.167 However, very recently, LINear ACcelerators (LINAC) have been integrated with MRI, giving birth to the first LINAC-MRI systems.

The great advantage provided by MRI guidance is the possibility to clearly contrast the cancerous tissue without use of any implanted or external surrogate point. Due to the promising results, the current trend in radio/particle therapy is to move toward MRI based treatments.109,147

US energy, finally, can be used to heat, store the heat and then release the heat over time into the tissue to be treated.159 The focal point can be localized using pre-operative MRI (MR-guided focused US or MRgFUS). Intra-operatively,76 introduced focal spot localization using Harmonic Motion Imaging (HMI). The motion of the organs can be compensated using robotic end-effectors.55

Limitations and Open Issues

iMRI systems, especially in the high field configuration, are still very expensive and require a re-arrangement or a complete new installation of the OR and the use of specific MRI-safe devices.

In many cases, the time required for operations increases compared to the standard navigation and the involved personnel need a specific training to work in presence of magnetic field. These issues limit the spread of iMRI to specialized clinical institutions or very big hospitals.

It is possible to foresee in the future a higher presence of iMRI in the OR, especially with the new trend of multi-modal operative rooms.

Endo/Laparoscopy

With the spread of MIS procedures, endo/laparoscopic imaging has become one of the most popular intraoperative imaging modality. Laparoscopic imaging is an optical, non-invasive and non-ionizing technology that provides surgeons with 2D images, with three (e.g., in case of RGB) or more channels, of the surgical scene. With respect to other imaging modalities (such as MRI and X-ray), endo/laparoscopic imaging is also fully compatible with standard OR instrumentation.182

Besides standard RGB imaging, powerful solutions include barrow band imaging, which is an optical technique where a filtered light enhances the visualization of epithelial and subepithelial microvascular patterns.125 This technique exploits the physical property that the depth of penetration of light is dependent on its wavelength. Narrow-Band Imaging (NBI) filters select the blue and green light with wavelengths of 415 and 540 nm, respectively, that correspond to the peaks of absorption of hemoglobin. These filtered wavelengths penetrate, respectively, the epithelium, thus highlighting the capillary network and the deeper levels, enhancing the subepithelial vessels.

Technological Advancements

Within this context, Multi-HyperSpectral Imaging (MHSI) has drawn the attention of the medical-imaging community, even if its use inside the OR is still limited.41 MHSI enables to capture both spatial and spectral information on structures. MHSI provides images that generally have dozens (multispectral) or hundred (hyperspectral) of channels, each corresponding to the reflection of light within a certain wavelength band.107 Multispectral bands are usually optimized to encode the informative content which is relevant for a specific application.212 Similarly to NBI systems, the measured reflectance spectrum is influenced by the optical properties of tissues, including the concentration of absorbers, such as hemoglobin, and scatterers, such as cells or structural connective tissues. However, MHSI allows higher resolution than NBI and often guarantees more accurate tissue analysis.107, 129

As a natural evolution of MHSI, Multi-HyperSpectral Fluorescence Imaging (MHSFI) is also becoming more and more spread.100,117,211 By combining MHSI and fluorescence molecular techniques (mostly based on fluorescein/fluorescein isothiocyanate or indocyanine green molecules), MHSFI is particularly suitable when dealing with tissues with multiple fluorescent labels that, however, have similar color and texture appearance (according to the human eye) and are localized in spatially overlapping areas.

Recently, fluorescence spectroscopy provided by 5-aminolevulinic acid (5-ALA) is showing promising results in assisting neurosurgeons during tumor resection. On this regard, studies were conducted to compare 5-ALA and iMRI, and the impact of a combined usage of these techniques.44,70

Large interest is today given to the development of near-infrared fluorescent probes for tumor margin assessment intraoperatively.122,191,193,204 Fluorescent probes may allow to detect lesions at an early stage, where conventional imaging may fail, lowering patients morbidity and mortality.

Label-free fluorescence lifetime imaging (FLIm) is a novel surgical-guidance technique, which relies only on tissue autofluorescence, without requiring exogenous contrast agents. By exploiting time-resolved measurements, FLIm overcomes the limitations of steady-state fluorescence, where non-uniform tissue illumination, and variable presence of endogenous absorbers may interfere with the fluorescence signal of interests. Preliminary results are already available for applications in surgery.7,72,196,209

Limitations and Open Issues

With advances in high-energy pulsed lasers, hardware cameras, image analysis methods, and computational power, many exciting applications in the medical field have been proposed in the endo/laparo-scopic fields.

MHSI and MHSFI offer a straightforward measurement of tissue characteristics (e.g. texture and perfusion), as long as the visualized tissue is close to the surface. This actually limits the use of MHSI/MHSFI when deeper tissues need to be investigated.

When dealing with steady-state fluorescence imaging (i.e., FLIm is excluded here), a further potential issue is represented by tissue autofluorescence, which is present in many living, non-cancerous cells. The autofluorescence causes non-specific background fluorescence, which may interact with the true cancer-specific fluorescent signal, and limit the imaging quality. With FLIm, this issue is not present. Open issues here deal with tissue motion and acquisition setup preparation, which may still require heavy time-consuming manual correction.

Selecting the most MHSI and MHSFI informative spectral bands and the most discriminative fluorescence molecules is crucial to allow the best visualization of structure of interests.100,212 MHSI/MHSFI systems could cover ultraviolet (200 to 400 nm), visible (400 to 780 nm), near (780 to 2500 nm) and mid infrared (2500 to 25000), depending on applications. However, visible and near infrared are the most widely used spectral ranges.113

A further issue is related to real-time data acquisition. Depending on hardware set-up and number of recorded spectral channels, acquisition time can range from a few seconds to several minutes. This could lead to misalignment in the multispectral stacks, resulting in noisy and blurred multispectral images, where the same pixel measured at different band could correspond to different tissues. Lens distortion and noisy image borders should also be considered when visualizing and processing multispectral data. Considering the high number of image channels, computational-time issues arise also when processing MHSI/MHSFI data, e.g. for segmentation purposes.

The translation of MHSI/MHSFI into the actual clinical practice is still limited by costs, even if now cheaper and cheaper sensors are becoming available. Moreover, the general lack of surgical guidelines and training could explain the slow introduction of MHSI/MHSFI in the OR.

Photoacoustic Imaging

PA imaging is emerging as a new biomedical imaging modality based on the photoacoustic effect. In photoacoustic imaging, non-ionizing laser pulses are delivered into biological tissues (when radio frequency pulses are used, the technology is referred to as thermoacoustic imaging). Some of the delivered energy will be absorbed and converted into heat, leading to transient thermoelastic expansion and thus wideband (i.e. MHz) ultrasonic emission. The generated ultrasonic waves are detected by ultrasonic transducers and then analyzed to produce images.

Technological Advancements

Thus, PA is naturally a 3D imaging modality. To lower costs and acquisition time associated to volumetric US detectors, other strategies can be used, such as using 2D US detectors focused on a plane or spherically-focused US detectors for sampling one point in the FoV at a time.185

Technological advancements in parallel detection and fast tuning of optical parametric oscillators allowed real-time multispectral PA, pushing its use in the clinical practice.99,184,216

Limitation and Open Issues

PA imaging is evolving fast but, although many exciting applications have been proposed in the medical field, large clinical trials are still lacking. One relevant issue is the PA signal attenuation, which prevents using this technology for imaging small and deep tissues. Hard tissue imaging (e.g. human brain imaging) is also prevented due to aberration processes of US wave-fronts.

Nuclear Medicine

Nuclear medicine based imaging provides information about the metabolism and functionality of tissues and organs, rather than anatomical details. It exploits the possibility to mark with a radioactive substances a given molecule involved into a physiological/pathological process. The obtained compound (also named radiopharmaceutical) is administered to the patient and then, by directly tracking the signal emitted by the radioactive element, functional details of the tissues can be revealed (both in 2D and in 3D).

As well as other imaging modalities, also nuclear medicine has been used to provide intraoperative information to the surgeon.75 In such a scenario, however, in room devices could significantly differ from diagnostic scanners. In fact, for intraoperative applications, 2D images are usually obtained by a hand-held probe. In addition, by combining localization system and 2D hand probe, it is possible to extract intraoperative volumetric representation of the radiopharmaceutical distribution.138

Technological Advancements

Over the years, the main technological advancements in this field were about the detector (commonly called “gamma camera”). Similarly to the diagnostic scanners, also the hand probe devices relies on detectors that can be classified as belonging to two different classes: scintillators (such as NaI(Tl) and bismuth germinate (BGO)) and semiconductors (Cadmium-Zinc-Telluride (CZT)). Both solutions offer advantages and disadvantages and both have been commercially used.82

Limitations and Open Issues

Although nuclear medicine probes can provide intra-operative information about tissues metabolism and lead to more accurate surgery procedure, some drawbacks still remains. Such limitations are mainly related to the physical working principle behind this modality. In particular, the main disadvantages are:

  • Patient and operators are exposed to ionizing radiation.

  • The system generates images with limited spatial and temporal resolution, low SNR and small FoV.

Raman Spectroscopy

Raman spectroscopy has emerged as a potential tool for detecting biochemical differences between cancerous and healthy tissue, improving the accuracy of tumor surgery since it is fast, non-destructive and non-invasive.77,160

In this modality, a laser light interacts with tissue sample and, due to the Raman effect, a portion of this light undergoes to an energy shift. The amount of the energy shift is informative about molecular composition of the tissue, resulting into a full characterization of the sample.

Raman spectroscopy does not require any special tissue preparation and staining or labelling, thus being cheap and fast. Moreover, the biochemical interpretation of the biological samples assists in the objective and quantitative evaluation about the tissue, overcoming the issues of the more subjective histopathological diagnosis performed by a single or panel of pathologists.

Technological Advancements

Recent advancements in the field include Surface-Enhanced Raman Spectroscopy (SERS)39 and Raman-Encoded Molecular Imaging (REMI)206 that, by exploiting nanoparticles delivered to the sample, allow both to amplify Raman signal (by a factor of ~ 10 orders of magnitude), and to speed up the acquisition process. On this regard, the design of ad-hoc nanoparticles, able to provide improved signal intensity, can further help Raman spetroscopy to better detect different types of tumor.79 Finally, an unique triple-modality MRI/PA/Raman has been developed and tested.98

Limitation and Open Issues

The main limitation and open issue of intraoperative Raman spetroscopy is about the safety in evaluating not excised patient’s tissue. In fact, since both SERS and REMI require nanoparticles tags directly applied on the tissue to analyze, the toxicity of this procedure should be carefully investigated.

Hybrid Surgical Rooms and Real-Time/Quasi Real-Time Image Processing

With the increasing need of image guidance in surgery and therapy, most of the modern surgical rooms are equipped with multimodal imaging systems. These are referred as hybrid surgical (or operating) rooms (or theatres). The most advanced present a multi-room layout to allow the presence of high field iMRI and CT or Positron Emission Tomography (PET)/CT scanner. Hybrid surgical rooms offer the advantage of performing different procedures in the same place. This is also a safety benefit from the patient side: if something goes wrong during a planned intervention the lay-out can easily converted to a more complicated surgical procedure. From the surgeon and medical staff side, these rooms offer the state of the art advancements in terms of imaging integration, real time data extraction and, in some case, voice and hand gesture control.

A representative example is the Advanced Multimodal Image-Guided Operating (AMIGO) suite (see Fig. 2), at Brigham Women Hospital in Boston (USA), which was launched in 2011. AMIGO consists of three adjacent rooms. The central room is the OR and it is equipped with MRI-compatible anesthesia delivery and monitoring systems; a surgical microscope with near-infrared capability; surgical navigation systems that track handheld tools; a ceiling-mounted C-Arm X-Ray system and 3D ultrasound devices. The side rooms include a high field (3T) iMRI scanner and a PET/CT scanner respectively. The iMRI can be moved into the OR by ceiling rails. The PET/CT is fixed and the patient is transferred from the OR through a shuttle system. Since its launch, more than 2000 (by January 2019) MIS procedures have been performed in AMIGO, mostly being neurosurgeries, ablations and biopsies.186

The trend has pushed companies like Siemens Health CareFootnote 1 (Erlangen, Germany) and IMRISFootnote 2 (Winnipeg, Canada) to invest on hybrid surgical rooms for different applications. Besides the advantages that a hybrid OR offers, its cost is still very high, ranging from 1 million to 4 million dollars, and it often requires re-restructuring the existing space. Moreover, with the fast technological advancement, these suites have to flexible to rapid changes and renovations. So, we can say that the future of OR is going to be hybrid, but still some year is required to have them as clinical practice.

Figure 2
figure 2

Overview of AMIGO surgical room. Courtesy of the Surgical Planning Laboratory, Brigham and Women’s Hospital, USA. Reprinted with permission from the authors.

Full size image

On the other hand, taking advantage of image-processing algorithms, intraoperative images can be enriched by (i) computing and showing supplementary information extracted from the image itself (ii) merging different and complementary acquisitions of the same anatomical district.

A straightforward solution to achieve these goals is using augmented and virtual reality.21

However, the low image quality of some intraoperative images and the real-time or quasi real-time processing to be guaranteed pose technological challenges. Intraoperative processing algorithms can be grouped as:

  • Structure segmentation, identification and tracking:

    Anatomical structures, as well as surgical tools, can be automatically identified (segmented) or tracked over time to provide surgeons with decision support and context awareness.

    Exemplary applications include vertebrae156 segmentation on fluoroscopy images, tissues and surgical tools tracking132 in 3D US, vessel segmentation,123 organ segmentation and tumor margin assessment in laparoscopic imaging,104,124,128 surgical tool detection in video laparoscopy,45 cancerous tissue219 and organs at risk,29,152,183 panorama stitching to enlarge the field of view,127 surface reconstruction in plastic surgery,142 identification in planning radiotherapy CT, brachitherapy220 and biopsy120 needles segmentation in iMRI, and pyramidal tract reconstruction.24

  • Physiological parameter estimation: medical images have been used also to esteem some physical and physiological parameters not directly measurable. Examples include iOUS-based flow estimation78 and assessment of right ventricular function,192 oxygenation level assessment on MHSI.213

  • Workflow analysis: automatic methodologies, strongly relying on OR video images and able to recognize and to analyze each phase of the operation, could promptly and automatically detect possible incidents and/or document the whole procedure.140,181

Finally, in the last years, algorithms for converting one image modality to another one have been developed, in particular for radiotherapy application (e.g. MRI to CT, and CBCT to CT).146,176,190,195

Discussion and Conclusion

Nowadays, several image modalities are available, each of which offers different characteristics (resolution, invasiveness, surgical compatibility, cost) and different contrast among tissues. The best modality to use for the specific use case is decided by the surgeon by considering and evaluating all the specific peculiarities of each of them. Depending on the chosen modality, adopting some preventive measure to guarantee the safety of both operators and patient could be necessary. This has also to be considered in robotic-assisted surgery scenarios. An increasing number of clinics have started to increment the type of imaging devices usable by physicians into the OR, especially in large hospital centers. Meanwhile, the last frontier of science in the field is represented by real-time processing of the acquired images to provide the surgeon with additional information. However, the majority of the developed technology is still for research purpose only, without any Food and Drug Administration and/or European Conformity approval.

The aim of this review was to provide the reader with an updated overview about currently available imaging modalities for intraoperative guidance (iOUS, X-ray, OCT, iMRI, video-endoscopy, NM, PA, and Raman spectroscopy). For each modality, physical working principle, technological advancements, and relevant pros and cons were reported and discussed, highlighting sample applications in several surgical scenarios. In view of such information, supported also by a survey about pioneering hybrid surgical rooms and real time image processing algorithms, the importance of image guided surgery for achieving better therapy come to light.

To conclude, we drew a path for helping students, scientist and health care worker, to guess, design and choose the surgical room of the future.