Introduction

A continuous and rapid evolution of technology has changed the face of trauma and orthopedic surgeries in the past decades. Especially, minimally invasive techniques are widely accepted for treatment of bone fractures in spine and pelvis, thanks to the development of modern imaging technology and computer-aided navigation systems. The benefits of minimally invasive orthopedic surgeries are the reduction in blood loss, collateral tissue damage, and overall operating duration [5]. However, these techniques usually yield a higher X-ray exposure for both patient and clinical staff and may increase fatigue and frustration due to the difficulty in continuous repositioning of the mobile X-ray machine (C-arm) [1, 32].

Fig. 1
figure 1

a Lateral view of the hip in pelvic surgery: Several skin punctures demonstrate the number of failed attempts to place the K-wire. b Anteroposterior X-ray image of hip region in pelvic surgery. The narrow superior pubic ramus requires a precise placement of the K-wire, especially considering that a misplacement could cause severe damage to external iliac artery or vein

Full size image

The main challenge during percutaneous K-wire placement and screw fixation is the mental alignment of patient, medical instruments, and the intra-operative X-ray images [27, 29], which also requires the frequent repositioning of the C-arm [31]. For instance, in pelvic acetabulum fractures, the surgeon needs to find the correct trajectory of the K-wire through a small bony structure, namely the superior pubic ramus. The misplacement of the K-wire could cause severe damage to the external iliac artery and vein, obturator nerve, or structures such as the inguinal canal and intra-articular hip joint [8]. It is not unusual that a single K-wire placement for one screw takes up to ten minutes [30].

The standard treatment procedure for undisplaced superior pubic ramus fractures requires several K-wire placements and subsequent screw insertions. For each K-wire, the surgeon first locates the entry point location and performs a skin incision at the lateral side of the hip, which requires several intra-operative X-ray images from various perspectives to confirm the exact tool orientation. It is common to correct the K-wire placement, as shown in Fig. 1a). While advancing the K-wire through soft tissue and into the bone, X-ray images from various perspectives are acquired to constantly validate the trajectory. Fig. 1b) shows the narrow path through the superior pubic ramus. After the K-wire is placed, the procedure concludes by drilling and placing a cannulated screw.

Computer-aided surgical navigation systems have been introduced to assist the placement of K-wires and screws. Current solutions use preoperative computed tomography (CT) volumes, external optical tracking systems, and tracked markers as reference on medical instruments, the patient, and the C-arm. Navigation systems then provide intra-operative information on the spatial relation of surgical instruments and medical images. The validation of the K-wire placement is performed using conventional X-ray imaging.

The benefits of navigation systems are controversial. Some publications indicate a reduction in the radiation dose and an increase in accuracy [6, 7], while a more recent study shows no clear advantage of using navigation systems in some procedures [14]. A major drawback of navigation systems is the high cost, which limits the availability of such systems to major hospitals and research facilities [7, 12]. The cost is driven by external hardware, which constitutes a logistical problem due to the bulkiness and consumption of space in the OR. Beyond hardware requirements, the systems also impose a change in the surgical workflow  [31]. In summary, after two decades of surgical navigation systems, expert surgeons are starting to realize that these systems have failed to provide the advantages promised. They do not reduce the required OR time, show no systematic, significant influence on the patient outcome, and reduce the frustration of the surgeon and staff. The additional efforts required to use modern surgical navigation systems often outweigh the benefits in many scenarios. Therefore, interventions are frequently performed without surgical navigation systems even though navigation would be available and theoretically present a benefit [13], which has been especially researched for spine surgery [11].

An alternative solution, which is comparatively inexpensive, contained in existing equipment and intuitive, has been proposed in [19]. This solution adds a mirror and video camera to a C-arm, such that the X-ray and optical views align. After a single calibration and warping step, the video view can be augmented with the X-ray images, which provides an intuitive optical feedback to the surgeon. In cadaver studies, this system leads to reduced radiation dose and increase in surgical efficiency in terms of duration and accuracy [16, 33]. During orthopedic and trauma procedures, the use of a camera augmented intra-operative X-ray system resulted in improved incisions, reduced radiation exposure of the surgeon, and simplified instrument tool alignment [3, 18].

However, the mirror construction reduces the free moving space of the surgeon, which can be overcome in mounting the camera next to the X-ray source [22]. That setup will only be able to augment the video view with warped X-ray images, which are clinically less relevant. Both approaches require the X-ray source to be positioned on the top rather than below the surgical table, which is an unusual setup and may increase the exposure of the surgeon to scatter radiation.

In continuation to [19], in [9] an red–green–blue depth (RGBD) camera was mounted to a C-arm instead of a video camera. Similarly to an RGB camera, an RGBD camera provides a 2D color image and additionally provides a depth value for every pixel which represents the distance between the observed object and the camera origin. This allows to reconstruct the 3D surfaces of an object. The system using the RGBD camera, rather than the RGB camera, enables an offline 3D/2D mixed reality visualization of X-ray on the reconstructed patient surface. The main limitation of this work is due to 2D projective nature of the X-ray image. As soon as the display viewpoint of the surface is different than the X-ray source optical axis, the visualization is physically wrong. Using CBCT may allow to overcome this issue, since a new simulated X-ray (DRR) corresponding to the viewpoint can be generated dynamically. In [10], two RGBD sensors were mounted on a mobile C-arm in order to synthesize the video as seen from the X-ray source viewpoint without the need of a mirror construction.

The integration of a stereo camera near the X-ray detector enables tool tracking within the working space of the C-arm. If CT images are transferred to the inter-operative setup, a digitally reconstructed radiograph (DRR) can be computed and augmented onto the one camera view [25]. This system has been presented as a good combination of augmented reality visualization and surgical navigation systems, but requires markers on the patient and tools. The change in the augmented view requires the movement of the entire system and may introduce errors of the alignment of CT, and optical view in case the patient marker is occluded.

Systems with augmented video may benefit of the use of RGBD cameras, which allows the positioning of the virtual cameras and renderings of the patient surface from arbitrary perspectives [9]. RGBD information can also be used to improve the understanding of the environment and enhance the augmentation [24].

In this paper, we present a preclinical usability study to provide a more comprehensive understanding whether enhanced C-arm systems provide a clinically relevant benefit. We will compare K-wire placement using (i) conventional X-ray imaging, (ii) 2D RGB video augmented with X-ray images, and (iii) a novel 3D RGBD video augmented with DRRs generated from cone beam CT (CBCT). The later system allows the surgeon virtually rotate the entire scene (DRR, patient surface, and tools) and simultaneously view the scene from different perspectives. A total of 21 K-wire placements are performed by seven surgeons, ranging from residents to attending physicians. We compare the system usabilities in terms of surgical efficiency, which is defined by the number of X-ray images, duration, accuracy, and surgical task load.

Method

In this section we first describe the imaging systems to be compared. These include conventional intra-operative X-ray imaging, X-ray image augmented 2D video, and a novel 3D RGBD view augmented with DRR. Finally, we present the questionnaires and statistical methods to perform the usability study.

Imaging systems

To evaluate the usability of mixed reality visualization techniques, we acquire a baseline using conventional intra-operative X-ray imaging. Examples of the three visualizations are shown in Fig. 2.

Fig. 2
figure 2

Same stage in the K-wire placement has been recreated using the different image-guidance systems. In a, the K-wire is placed under conventional C-arm guidance, which requires frequent imaging and may result in a higher radiation dose for the surgeon. b The X-ray image is augmented onto a live video stream, and the surgeon can update the X-ray image at his discretion. c The use of an RGBD camera and DRR computed from a CBCT allows for the simultaneous visualization of the patient from different views. The surgeon can choose which views should be displayed, which will be updated using live RGBD information

Full size image

Conventional intra-operative X-ray imaging This imaging method using a standard C-arm provides the baseline performance as it is the most commonly used system to perform image-guided K-wire placement. The images are obtained in the digital radiography (DR) mode. This allows for a single, brief exposure at higher than normal mA to capture a higher-quality single image. For reasons of comparability between subjects, we limit the functionality of the C-arm to this mode.

2D RGB video and X-ray visualization To achieve a fused RGB and X-ray visualization, we attached a camera near the X-ray source. Using a mirror construction, the X-ray source and optical camera centers are virtually aligned as described in [19]. To be able to observe the surgical site using the RGB camera, the X-ray source and camera are positioned above the patient.

The X-ray images are obtained using the standard C-arm in DR mode. After camera calibration [34], the alignment registration of optical and X-ray images is performed using a single plane phantom with radiopaque markers that are also visible in the optical view [20].

Finally, this first augmented reality system allows the simultaneous display of live RGB video overlaid with DR images obtained at the user’s discretion. Additionally, we provide the user with the option to control the alpha blending to change the transparency to be able to focus on the X-ray image or video background.

3D RGBD and DRR via CBCT visualization The previous system requires the repositioning of the C-arm in order to change the optical and X-ray view. To overcome this limitation, we introduce a novel system using an RGBD camera and cone beam CT (CBCT) volumes, which allows the simultaneous visualization of the patient and medical data from multiple arbitrary views. As the RGBD camera is rigidly mounted to the X-ray detector, the X-ray source can be positioned under the surgical table as done during conventional image-guided surgery.

To calibrate the RGBD information and the CBCT volume, we simultaneously acquire the CBCT and the surface information using the RGBD camera of an arbitrary object. We extract the surface from the CBCT by simple thresholding, and reconstruct the surface observed by the RGBD camera as described in [21], resulting in a smooth and precise surface mesh. The calibration is obtained by means of surface matching [15].

After the calibration is obtained, the CBCT and patient’s surface scan are acquired. These data are fused into a mixed reality scene, in which the patient’s surface, DRR from CBCT, and live RGBD data (e.g., hand or tool) are visualized. The surgeon can now define multiple arbitrary views of the fused DRR and RGBD data. The system allows perspectives that are usually not possible using conventional X-ray imaging, as the free moving space is limited by the patient, surgical table, or OR setup. The live RGBD data provide an intuitive understanding of the relation of CBCT volume, patient’s surface, surgeon’s hand, and medical tools.

Evaluation method

During the usability study, we evaluate the performance achieved using each system. Our hypothesis states that the mixed reality visualizations improve the surgical efficiency. Our data cannot be assumed to be of normal distribution, but are ordinal. Using Friedman’s ANOVA [4], we test whether the differences in observations are coincidental or statistically significant. Additionally, we need to test whether the individual systems yield a significant difference in terms of the surgical efficiency. As a normal distribution of our data cannot be assumed, these post hoc tests are performed using the Wilcoxon signed-rank tests with Bonferroni correction [35].

Surgical efficiency measure Together with our clinical partners, we identified following measures to express the surgical efficiency. First, the duration of each K-wire placement is of importance. During hip surgeries, this process is often the most time-consuming and is followed by a relatively quick drilling step and screw placement. Surgical navigation systems often do not yield the advantage of reducing the overall OR time. Next, the number or X-ray images and cumulative area dose product is of importance to the both patient and surgeon. During conventional C-arm guided placement, a large number of X-ray images are acquired during the planning and propagation of the K-wire. One of our systems will acquire a preincision CBCT, for which we will include the dose into our statistics. Finally, the error is defined by the medical need of the K-wire remaining in the superior pubic ramus. We will compute the average distance between the ideal path, which is the center line of bone phantom, and the placement of it. However, as all study participants are trained surgeons, we do not expect that any significant improvement will be possible.

Surgical task load The workload is measured using a standardized questionnaire, namely the Surgical Task Load Index (SURG-TLX) [36]. This test is designed to evaluate the mental demands, physical demands, temporal demands, task complexity, situational stress, and distractions during surgical interventions. It is specifically designed and validated to analyze implications for categorizing the difficulty of certain procedures and the implementation of new technology in the operating room.

Experiments

The K-wire placement through the superior pubic ramus (acetabulum arc) is a complex and cumbersome procedure, which is performed frequently and in case of an undislocated fracture usually minimally invasive [17]. In our experiments, we mimicked this scenario by designing adequate radiopaque phantoms. The surgeons each performed three K-wire placements using the image-guidance systems in a randomized order.

Phantom design The superior pubic ramus is a thin tubular bone with an diameter around 10 mm. In case of an undislocated fracture, a 2.8-mm-thin K-wire needs to be placed through a narrow safe zone [2, 26]. Later, a 7.3 mm cannulated screw is inserted  [28]. Our phantom was created out of methylene bisphenyl diisocyanate (MDI) foam, which is stiff, lightweight, and not radiopaque. The bone phantom was created out of an thin aluminum mesh filled with MDI. The begin and end of the bone were marked with a rubber radiopaque ring. Therefore, the bone phantom is very similar to the superior pubic ramus in terms of haptic feedback during K-wire placement, as the K-wire will easily exit the bone without significant resistance. The orientation of the bone within the phantom was randomized and phantoms were not reused for other experiments.

Experimental setup and design In all our experiments, we use a CBCT enabled C-arm, SIEMENS ARCADIS Orbic 3D from Siemens Healthcare GmbH, which automatically computes the cumulative area dose for the patient for each imaging session. The second and third system use an optical video camera, Manta G-125C, from Allied Vision Technologies, or an RGBD camera, Intel RealSense Camera (F200), Intel Corporation, respectively.

Each surgeon was asked to perform three independent K-wire placements using the different imaging modalities. The order of the modalities was randomized, but for simplicity we will refer to the first (S1), second (S2), and third system (S3) in the order presented in “Imaging systems” section. Figure 3 shows the experimental setup for the 3D RGB and DRR visualization system (S3). Using a 2.8 mm K-wire, the surgeons identified the entry point on the phantom’s surface, drilled toward the begin of the bone phantom, and passed through the tubular bone structure. When finished, the K-wire was removed from the drill and a CBCT was acquired to measure the error of the K-wire placement postoperatively.

Fig. 3
figure 3

During the experiments, the surgeons drilled a K-wire into a phantom. This figure shows the experimental setup during a procedure guided by the 3D RGBD and DRR visualization (S3). The RGBD camera is mounted on the C-arm X-ray detector, and the surgeon is watching the live 3D RGBD and DRR views on the monitor while drilling into the phantom containing the bone model

Full size image

Results

We observed a total of 21 minimally invasive K-wire placements using different image-guidance systems. Table 1 shows the observed time in seconds, number of acquired X-ray images, cumulative area dose product (dose) in \(\mathrm{cGycm}^2\), error relative to the ideal path in mm, and surgical task load index for each participant and system used. Note that the task load is a accumulative scale, for which the score of 5 and 100 represents the lowest and highest possible load, respectively.

Table 1 This table presents all observed values for each study participant and system used
Full size table
Table 2 Accumulated values and standard deviations for the observations (Table 1) are presented in this table
Full size table
Fig. 4
figure 4

This plot illustrates duration of the intervention, number of X-ray images taken, radiation dose, K-wire placement error, and surgical task load, where each bar shows the accumulated values using one of the systems (conventional X-ray, RGB/X-ray fusion, or RBGD/DRR). Each measure is normalized relative to the maximum value observed. The \(*\) symbols indicate significant differences

Full size image

The aggregated observations are presented in Table 2. When comparing the use of a conventional C-arm to the use of a mixed reality system, a clear tendency toward a decreased operation time, lower number of X-ray images acquired, reduced dose, and reduced task load can be observed, as shown in Fig. 4, is observed. The measure of the dose includes the acquisition of the CBCT volume required for the RGBD and DRR visualization (system 3). The accuracy does not improve, as it is already in an acceptable range.

Statistical evaluation

Statistical tests were performed to study the changes in the surgical efficiency measures. Significance is achieved for p-values lower than 0.05, indicating that the chance of the change being coincidentally observed is less than 5 % [23]. A Friedman test was calculated to compare each measure as a normal distribution of the data could not be assumed. We found a significant difference in time (\(\chi ^2(3) = 11.14\), \(p<0.01\)), number of X-ray images (\(\chi ^2(3) = 12.29\), \(p<0.01\)), and radiation dose (\(\chi ^2(3) = 6.00\), \(p<0.05\)) depending on the kind of assistance that was provided to the subjects. The post hoc tests were computed using the Wilcoxon signed-rank tests with Bonferroni correction.

Time The tests show significant differences between the first system (S1: conventional C-arm) and the third system (S3: RGBD and DRR Visualization) (\(Z = -2.366\), \(p < 0.05\)), and significant differences between second system (S2: RGB and X-ray visualization) and S3 (\(Z = -2.366\), \(p < 0.05\)). This indicates that the 3D placement of the K-wire is best supported with a multi-view 3D visualization.

X-ray images All combinations of S1, S2, and S3 show a significant reduction in the number of X-ray images acquired: S1 to S2 (\(Z = -2.117\), \(p < 0.05\)), S2 to S3 (\(Z = -2.375\), \(p < 0.05\)), and S1 to S3 (\(Z = -2.366\), \(p < 0.05\)).

Radiation dose Although we have included the dose caused by the CBCT in S3, the tests show that the intervention using the conventional C-arm causes a significantly higher cumulative area dose product: S1 to S2 (\(Z = -2.197\), \(p < 0.05\)), S1 to S3 (\(Z = -2.197\), \(p < 0.05\)). However, the dose difference between S2 and S3 is not significant.

Error No significant difference in error between based on the use of different systems can be observed. Therefore, the reported changes in accuracies are most likely coincidental.

Surgical task load index Similarly to the changes in the duration of the intervention, the reduction in task load evaluated using the SURG-TLX is only significant between S1 and S3 (\(Z = -2.197\), \(p < 0.05\)).

In conclusion, S3 yields better results in terms of all observed surgical efficiency measures except for the accuracy, for which the difference is not statistically significant. Even though S3 is not a fully developed product, our usability study indicates that there are clear advantages over the conventional C-arm system when guiding K-wire placement.

Discussion and conclusion

In this paper, we presented a usability study using three different mixed reality visualization systems to perform K-wire placement into the superior pubic ramus. This procedure was chosen because of the high clinical relevance, frequent prevalence, and the especially challenging minimal invasive surgical technique.

Our attention was focused on the usability and clinical impact of the three different visualization systems. For that reason, we were interested not only in the quality of a procedure (e.g., accuracy), but also in the workload and frustration that the surgeons experienced while using the different systems. We observed the 21 interventions performed by seven surgeons and used the Surgical TLX to evaluate the task load.

Our results show that the 3D visualization yields the most benefit in terms of surgical duration, number of X-ray images taken, overall radiation dose, and surgical workload. This is despite the fact that the mixed reality visualizations currently do not provide an augmentation of a tracked tool. The conventional C-arm constitute the system yielding the poorest results, indicating a high potential for improvements to the currently used image-guidance systems. In all scenarios, the surgeons placed the K-wire within clinically relevant tolerance. The change in accuracy of the placed K-wire is not significant, which shows that all three systems provide sufficient support in terms of placement quality.

This study also showed the clear necessity to continue research and development of the mixed reality systems. For instance, movement of the C-arm or surgical table may lead to loss of tracking, which results in an outdated mixed reality visualization. However, in a clinical scenario, the failure of the mixed reality system is immediately visible and the surgeon can continue using the conventional X-ray imaging capabilities.

In our evaluation, we have not take the learning curve under consideration as we frequently observed that surgeons unfamiliar to the mixed reality system adopted very quickly. Perhaps an initial training phase would further emphasize the advantages of the augmentations.

Future studies will include other complex K-wire placement procedures, such as performed in case of an iliosacral fracture, or pedicle screw placement. We will attempt to include even more surgeons during the next studies, which will allow for a more detailed statistically analysis. Additionally, we will investigate the usability for other procedures, such as Jamshidi needle placement or general needle biopsies.

Our usability study showed that mixed reality systems have great potential to increase surgical efficiency, and should be in the focus of research on computer-assisted interventions. The integration of better visualization techniques or tool tracking and identification may yield more opportunities in assisting surgeons to conduct interventions more efficiently and reduce the task load.