Introduction

The most common intra-operative imaging modality used in orthopaedics is two-dimensional (2D) fluoroscopy. Identification of fractured bone fragments, assessment of reduction and guidance of surgical tools are typically performed under 2D fluoroscopy scans taken from different directions [9, 25]. Path deviations can result in severe injury, additional operating time and radiation to correct implant placement, with one study reporting an average 12 mins of radiation exposure per case for pelvic fixation [19], which raises important safety concerns not only for the patient but for the operating staff as well [19].

Ultrasound (US) imaging is one of the safest and cheapest modalities available today and has the potential to become a valuable alternative to X-ray-based imaging for intra-operative use since it provides non-ionizing, fast, portable and inexpensive real-time 3D imaging capability. Unfortunately, US is relatively uncommon in orthopaedic surgery due to the poor quality of the bone boundaries. Nonlinear characteristics of US, low signal-to-noise ratios and other imaging artifacts make it difficult to accurately and reliably determine the location and shape of the bone surface [17]. For US to become a practical modality for orthopaedic interventions, bone surfaces have to be extracted automatically, accurately and quickly from US scans taken in clinical settings.

We are primarily interested in developing US-based procedures to assist the treatment of bone fractures, particularly pelvic ring injuries. To develop the methodology and technology, we chose to image 2 types of injuries: Distal radius fractures and Pelvic ring injuries. Distal radius fractures were used in this development due to its frequency [25], and the bone surfaces are close to the US transducer face decreasing the typical US artifacts cause by the soft tissue interface. Pelvic ring injuries are less frequent [6], but these procedures are frequently long and challenging [15]. Over 40 % of patients who require pelvic stabilization experience long-term complications, usually related to neurological and urological issues or pain [15, 23].

Previous groups have attempted to use US in orthopaedics [2, 3, 8, 24]. These studies demonstrated decreased total radiation dose and improved surgical accuracy. Various algorithms to automatically extract bone surfaces from US scans based on intensity and local gradient image information have been reported in [7, 8, 17, 18]. Aside from the fact that most work to date has been limited to 2D images, the performance of these methods remains highly sensitive to variations in data and imaging parameter settings due to the typical US imaging artifacts [13, 17].

To address some of the challenges of the previously proposed bone extraction methods we aimed to develop and validate a new image processing method that allows for automatic, accurate and fast extraction of bone surfaces from both 2D and three-dimensional (3D) US scans using intensity-invariant image phase information. Phase information has been previously used to enhance soft tissue interfaces from US data [4, 10, 21]. In our initial work, we established that bone surfaces of phantom preparations can be localized with sub-millimetric accuracy [11–14]. In [11] we have reported the extension of our previously developed phase-based image processing method [13] from 2D to 3D. Log-Gabor filters, developed using empirical filter parameters, were used to extract the local phase information from US images. The method was validated on phantom and ex vivo experiments. Automatic 2D Log-Gabor filter parameter selection was reported in [12]. Quantitative validation was performed on phantom and 3 patient scans. In our most recent work [14] we reported the extension of the automatic parameter selection work [12] to 3D. Accuracy experiment performed on phantom and 10 patient scans showed an improvement on the order of 60–75 % compared to our previous works [11–13]. Therefore, our objective for this study was to determine whether our newly developed image processing method would allow us to accurately extract the surfaces of fractured and intact bones from 3D US volumes acquired in a live clinical setting and understand the full potential of this method. We report herein the capability of this new image processing technique on enhancing bone surfaces from 29 patient scans obtained from trauma patients showing the potential of US in orthopaedic surgery.

Materials and methods

Patients qualified for our study if they presented to a Level 1 Trauma Facility (Vancouver General Hospital, Vancouver, BC, Canada) between April 2010 and June 2013 for operative care of a distal radius fracture or pelvic ring injury referred to the orthopaedic trauma team. All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2008 (5). We excluded patients with skin conditions or allergies, which precluded the use of ultrasound gel, patients who had sustained a previous pelvis or distal radius fracture, and patients unable to provide informed consent. All patients had CT images taken as part of their standard clinical care. 3D US scans were subsequently acquired either in the operating room or on the ward. For the radius, we obtained dorsal, volar, and radial views of the injured limb (Fig. 1). For pelvic ring injuries we imaged at the iliac crest/iliac fossa. Bilateral scans were done when scans were obtained in surgery under anaesthetic, and the non-injured side alone was scanned when we obtained images pre-operatively to avoid patient discomfort.

Fig. 1
figure 1

Clinical scanning of a patient with a distal radius fracture scanned in the operating room. a Our researcher performing the scan, b scan on dorsal surface of radius, c 2D B-mode US image; red arrow indicates fracture location

Full size image

We used a commercially available US machine (GE Voluson 730, GE Healthcare, Waukesha, WI) with a 3D RSP5-12 transducer (Fig. 1). This is a mechanized probe in which a linear array transducer sweeps through an arc of \(20^{\circ }\). The time to obtain one US volume is 10 s. The reconstructed US volumes were \(199 \times 174 \times 180\) voxels (lateral \(\times \) axial \(\times \) elevational) for the distal radius scans. This size varied between \(94 \times 198 \times 104\)–\(138 \times 184 \times 138\) (lateral \(\times \) axial \(\times \) elevational) voxels for pelvic scans, depending on the patient’s body mass. The resolution of the US scans varied between 0.21 and 0.42 mm, for all directions, again depending on the imaged anatomical region and patient. This corresponded to a scanning area of \(4\times 3.7\times 3.7\,\hbox {cm}^{3}\) and of pelvis scans is \(5.7\,\hbox {cm}\times 5.7\)–\(8\times 5.7\,\hbox {cm}^{3}\) for the \(\hbox {lateral}\times \hbox {axial}\times \) elevational directions. CT scans had an in-plane resolution that varied between 0.5 and 0.8 mm (in plane) and 1–2 mm in the axial direction.

CT data are used in this study solely for validation purposes, as shown in the flowchart of Fig. 2. Future clinical applications may or may not require preoperative CT data.

Fig. 2
figure 2

Quantitative validation: flowchart showing the image processing modules performed to compare our US-based bone extraction results to the ground truth as extracted from CT

Full size image

Initial alignment of the CT dataset to 3D US volume was performed using the AMIRA software package (TGS, San Diego, CA, USA) and its anatomical landmark-based registration module. This registration algorithm minimized the sum of the squared distances between the corresponding landmark points identified in both datasets. The anatomically corresponding landmarks were manually digitized in both image sets to allow for initial coarse alignment. Following this initial registration, bone surfaces from the CT scans were extracted using a standard thresholding approach that minimizes the intra-class variance [11] (Fig. 3c). The resulting effect of different automatic thresholding methods on the surface fit error had a range of range 0.04–0.1 mm [11].

Fig. 3
figure 3

a–e Image processing steps for quantitative validation. a Example central 2D B-mode US slice from a volume. b Local phase symmetry (PS) surface extracted from a using our local phase processing method. c Segmented CT surface (shown in black) used in the quantitative validation overlaid to the actual CT slice. The sketch of the US transducer and the red arrows (US signals) display the propagation of the US signals inside the tissue. Since most of the signals will be reflected from the bone/tissue interface back to the transducer, imaging inside the bone with US is not possible. d Extracted local PS surface from b overlaid on the reference (gold standard) CT surface. e Signed distance map where the reference surface (0 signed distance value) is shown as the transition from yellow colour to black. Note that positions to the left and right of the spatially limited reference surface are irrelevant to the analysis since the US scan is not obtained from that region

Full size image

Due to the high acoustic impedance at bone-soft tissue interfaces, US bone imaging always results in the reflection of most of the US signal at the first encountered bone surface making only the top surfaces visible, i.e. imaging inside the bones with US is not possible. To compare this top surface with that extracted from CT data, we only segment the top bone surface of the CT scans along the direction of the US signal propagation. This allowed us to generate corresponding bone surfaces from the US and CT dataset pairs. The bone surfaces were extracted from the US volumes using our recently developed image processing method, which uses image phase information rather than image intensity information [11, 14]. The algorithm’s filter parameters were automatically optimized using our previously developed framework that is adaptive to image content [14]. This phase-based image processing approach has been shown to localize bone surfaces at an accuracy of better than 0.6 mm in phantoms [11].

Following the extraction of bone surfaces from both CT and US, the initial landmark-based alignment was further refined using a surface-based registration algorithm [21]. In this registration method, the point sets, obtained from US and CT datasets, are represented as Gaussian Mixture Models (GMM). The registration is solved by an expectation maximization algorithm where the centroids of the GMMs belonging to the US point set are transformed by a set of transformation parameters to best fit the GMMs belonging to the CT dataset [21].

To compare our 3D “gold standard” CT surface to the 3D US surface identified using our phase-based processing method, a signed distance map was computed around the 3D CT bone surface. Each nonzero value in the phase-processed US image was then mapped to its corresponding location in the CT image so as to identify the signed distance value associated with that location. This produced a set of intensity/distance pairs. High-intensity values confined to a zone near the zero distance (zero level set) indicate a perfect US surface localization (Fig. 3e). To estimate the surface fit error (SFE), we identified the highest phase intensity value in each column of the processed 3D US volume and defined the SFE as the average signed distance value corresponding to these maximum phase intensity values. This analysis was repeated for all scans obtained from all patients.

Results

In total 29 patients were scanned, comprising 4 with distal radius fractures and 25 with pelvic ring injuries. Two of the pelvic cases were not included in the validation since the CT scans could not be obtained from the initial hospital where the patients were taken prior to being transferred to Vancouver General Hospital. The initial anatomical landmark registration error had a mean value of 0.32 mm (SD 0.16 mm) for pelvic patients and 0.27 mm (SD 0.32 mm) for the distal radius cases. For the pelvic ring injury patients, the mean of the SFE across subjects was 0.62 mm (SD 0.42 mm). For the distal radius patients, the mean SFE was 0.21 mm (SD 0.14 mm).

Thirteen of the 23 pelvic scans were obtained on the ward. Of the 13 cases scanned on the ward, five had markedly larger mean errors compared to the remaining 18 scans illustrated in Fig. 4. For these patients, the mean SFE was 1.35 mm (SD 0.23 mm), while the remaining 18 scans had a mean SFE of only 0.49 mm (SD: 0.23 mm). The main reason for this was likely suboptimal palpation/orientation of the US transducer with respect to the bone surface due to the patient being conscious and in pain. In these scans, fiducial-based alignment did not include key features from the ilium or iliac fossa regions that we normally used (Fig. 5a). Subsequently, the extracted local phase bone features were also affected by this suboptimal scanning, resulting in the extraction of soft tissue interfaces together with bone surfaces (Fig. 5a, bottom). For US scans obtained from the iliac crest region, where the US probe was not oriented properly, the extracted local phase-based bone surfaces were also affected by the soft tissue interfaces (Fig. 5b, bottom). This is mainly due to bone surfaces being oriented in the same directions as the soft tissue interfaces. In such situations, a post-image-processing technique (such as bottom up ray casting Fig. 5c) could be used to enhance the bone surfaces or, alternatively, the filter parameter optimization framework reported in [12, 14] could be adjusted accordingly.

Fig. 4
figure 4

Clinical pelvis scan box plot. The horizontal red bar represents the mean SFE (0.68 mm), and the red dots represent the individual SFEs. The five red dots with SFE larger than 1 mm indicate scans obtained on the wards rather than in the emergency department

Full size image

Figure 6 shows qualitative results obtained from the pelvic scans collected in the OR. Our image processing approach appears to produce sharper and cleaner continuous bone surfaces when compared to the unprocessed US volumes. In addition, the majority of the signal from overlying soft tissues is eliminated. Furthermore, the correspondence between the US- and CT-derived surfaces is excellent, as illustrated by the registration overlay (Fig. 6c, d). There were no problems encountered during the scanning of the distal radius cases. In Fig. 7 we demonstrate how our image processing approach produces sharper continuous bone surfaces compared to the unprocessed US images. The fracture location is also clearly visible in the extracted bone surfaces (also shown with red arrows), though we cannot obtain direct visualization of the intra-articular aspects of the fracture as these lie behind the proximal bone interface.

Fig. 5
figure 5

a–c US scans obtained in pelvic injury fracture of patients. a Top Central slice of a B-mode US volume with no bone features present from the ilium or iliac fossa regions (indicated by the red arrow) due to incorrect orientation of the US transducer with respect to the underlying bone surface (see text for explanation). Bottom Central slice of a B-mode US volume from the iliac crest region. Soft tissue interfaces having the same orientation as the bone surface are shown with yellow arrows. b Local phase bone surfaces corresponding to a where soft tissue artifacts are also present. b Ray casting of b obtained by selecting the first 4 pixel values in the bottom up vertical direction, which are nonzero. Suboptimal palpation/orientation of the US transducer with respect to the bone surface due to the patient being conscious and in pain results with soft tissue artifacts present in the US images, which affects the SFE

Full size image
Fig. 6
figure 6

a–d Qualitative results obtained from a second example pelvic ring injury patient. a B-mode US volume where the scan site includes reflections from the bone surfaces in the region of the iliac fossa. b Extracted 3D bone surface using our phase-based image processing method. c Registration overlay where the local phase bone surface is registered to the pre-operative CT scan (axial slice). d Zoomed-in version of c

Full size image
Fig. 7
figure 7

a–d Qualitative results obtained from one of the patients with a distal radius fracture. a B-mode US volume scan where the scan site includes fractured distal radius. b Extracted 3D bone surface using our local phase image processing method. c Registration overlay where the local phase bone surface is registered to the pre-operative CT scan. d Zoomed-in version of c. Red arrows indicate location of fracture

Full size image

Discussion

Current diagnostic and surgical planning methods in orthopaedic trauma surgeries are mainly based on X-ray (plain and fluoroscopic) and CT imaging and thus have significant limitations in terms of radiation exposure, dimensionality of image data and availability in the OR. US imaging is a powerful potential imaging alternative that is ionizing radiation-free, easy to use in pre- and intra-operative settings, is comparatively inexpensive and capable of producing 3D image data. However, the appearance of bone surfaces in US is limited because of the typical imaging artifacts, low signal-to-noise ratio (SNR) and regions corresponding to bone boundaries appear blurry with several millimetres in thickness. To enable practical use of US in computer-assisted orthopaedic surgery applications, advanced automatic image processing for extracting bone surfaces is essential for, accurate near-real-time operation. We have recently developed a local phase-based image processing technique that appears to meet these criteria in phantom tests [13]. In this study, we evaluated the potential of our technique in a live clinical setting, specifically, in the context of trauma patients who had suffered fractures of either the distal radius or the pelvis. For distal radius fractures the extracted surfaces could be used to assess the fracture by investigating the extracted surface information. Furthermore, the extracted surfaces could be used to assess the location of the fracture as well. The symmetry of the pelvis is one of the common indicators of successful reduction in pelvic ring fractures. 3D surfaces extracted from tracked US volumes (collected from both sides of the iliac crest and pubic symphysis) could be used to assess this symmetry during surgery. For complex fracture cases where the imaging of the bone fragments is not possible due to the imaging depth and US signal attenuation limitations, the extracted bone surfaces (obtained from a more accessible bone surface) could be using to register to a pre-operative plan developed using CT or MRI images.

We successfully obtained 3D surface scans from 29 patients enrolled in our study (two were excluded from the quantitative analysis due to unavailability of their pre-operative CT scans). On average, we found sub-millimetric surface fitting errors between our US-derived surfaces and the ‘gold standard’ surfaces derived from the CT scans. In a fraction of the pelvic ring injury cases (5 of 23), the patients’ pain levels prevented us from performing an adequate scan; this would not be a limitation in an operative setting due to the use of anaesthesia. All of the distal radius US data were collected in the operating room, and we did not encounter any difficulties in obtaining suitable scans. We therefore believe that our local phase image processing technique can produce more accurate bone surfaces under anaesthesia than while the patient is awake. Ultimately the accuracy was high, which is promising finding as we plan this technology’s next steps.

In this study, we did not acquire the images following reduction and do not suggest that this approach be used to assess realignment of fractured fragments but are only able to suggest that this approach is very promising in that context. One potential limitation during the reduction would be the use of a US coupling gel, which is usually required to improve acoustic coupling between the transducers and the skin surface in order to obtain an image with acceptable quality. In a trauma situation, the patient may have a bandage or a cast that could potentially interfere with the ability to place the coupling gel and US transducer in a reasonable position, or wounds that would prevent the examination altogether. The US transducer used in this study uses an internal motor to rotate an array of piezoelectric elements to create a stack of images that are assembled into a single volume. This dedicated 3D US transducer offers a faster acquisition rate compared to tracked freehand 2D US transducers [16], though the field of view of a single volume is relatively limited. In some applications, it may be desirable to weave a set of volumes acquired from a tracked 3D US transducer into a larger ’stitched’ volume. We have not yet implemented such an algorithm, although similar algorithms do exist [22]. We note that matrix array US transducers could also be used to generate 3D image sets with faster acquisition rates than the mechanical transducers used in this study, though they would generally be more costly.

In this study we only computed the surface fit error (SFE) due to the unavailability of well-defined and localized fiducials that are visible in both CT and US. Therefore, we have not been able to demonstrate the absolute accuracy of the bone surface extracted from the US images. However, the SFE is consistent with earlier phantom studies our group has performed, which were validated with multi-modality fiducial-based registration [11–14] where mean errors of 0.3–0.6 mm were reported, so we believe that the accuracy would be similar to that found in our earlier studies. Our SFE is also consistent with results previously reported using some automatic or semi-automatic 2D US surface extraction techniques [7, 8, 18]. In particular, Foroughi et al. [8] and Kowal et al. [18] showed mean errors in the range of 0.3–0.6 mm. However, previous studies were obtained using 2D US transducers, in which the transducer was optimally aligned with respect to the bone surface, and were only validated on cadaveric bovine specimen [18] or on scans obtained from a limited number of volunteer subjects [8]. Since our patient series was drawn from a single hospital population, it is possible that our patient set is not broadly representative of orthopaedic trauma patients in all jurisdictions. However, we did obtain accurate scans from a significant number of patients, so it is reasonable to conclude that the technique is promising for at least a significant subset of orthopaedic trauma patients.

In our current proof of concept MATLAB (The Mathworks Inc., Natick, MA, USA) implementation, average processing of one volume takes approximately 47 seconds on an Intel Core i7 870 CPU @ 2.93 GhZ with 8 GB of RAM. Recently, we have investigated alternative coding methods to implement local phase-based image processing and have demonstrated a 15-fold speed up in computation time when run on a graphic processing unit (GPU) [1]. We are currently pursuing implementations of the proposed method using GPU for real-time processing. A near-real-time implementation would also allow the user to observe the extracted bone surfaces and decide whether the US transducer is positioned properly, which in turn would result in the extraction of minimum soft tissue interfaces.

In conclusion, our results demonstrate that 3D US images obtained in clinically relevant orthopaedic trauma cases can be automatically processed to produce accurate bone surfaces with sub-millimetric errors. This suggests the feasibility of using US as an intra-operative imaging modality in computer-assisted orthopaedic surgery applications. Furthermore, the automatically extracted bone surfaces could be registered to a pre-operative plan, which in turn will allow the development of an image guided surgery system with a goal to improve surgical accuracy. This new system could also decrease the total amount of radiation exposure by limiting the use of traditional fluoroscopy. Reports showing an average 12 mins/case of radiation exposure in percutaneous pelvic ring surgery offer a great motivation towards this goal.

Although we did not establish in this study that the method can be implemented in near-real-time, clinical implementation will require that critical step. Recently we have also shown that the extracted bone surfaces could also be used for intra-operative registration to match a pre-operative plan to the patient anatomy [5]. These results taken collectively suggest that our goal of building US-based navigation applications may well be within reach.