1 Introduction

The use of biplane fluoroscopic systems has become increasingly popular for evaluating joint kinematics in vivo [1,2,3,4,5,6]. Glenohumeral joint kinematics can be quantified from biplane fluoroscopic images using radiostereometric analysis (RSA) or a 3D–2D registration approach. Custom biplane fluoroscopic systems, which are not FDA-approved for clinical use, have been commonly used for these approaches [2, 7]. Clinical biplane systems are FDA-approved but are inherently not well-suited for biplane analyses due to the fact that images are acquired asynchronously. Asynchrony in image acquisition has the advantage of reducing cross-scattering effects of one X-ray source onto the other [8]; however, asynchronous acquisition introduces errors into the registration process, as there is joint movement between the two asynchronous images. Moreover, the magnitude of the error depends on the speed of movement which is not known prior to the scan and the fluoroscopy frame rate. Thus, achieving accurate 3D kinematics using conventional techniques on temporally offset data sets is challenging.

In the present study, our goal was to improve the accuracy of image registration by generating simulated corresponding fluoroscopy images. We introduce an interpolation algorithm to generate missing images in biplane image sequences thus producing interpolated-synchronous image pairs. The main objective of this study was to validate the proposed approach using an artificial shoulder model and clinical biplane fluoroscope. We performed a quantitative comparison of 3D–2D registration using (a) asynchronous biplane image registration and (b) interpolated-synchronous biplane image registration.

2 Methods

2.1 Image Acquisition

We imaged radiopaque humerus and scapula bone models (Reference part #1020-100-1, Sawbones, Pacific Research Laboratories) with radiopaque beads rigidly attached to the surface of each. Seven and nine beads were fixed on the humerus and scapula, respectively. A clinical flat-panel biplane fluoroscope (Siemens Artis Zee; Forchheim Germany) was used for all 2D imaging. The scapula was fixed in a custom apparatus that allowed free movement of the humerus (Fig. 1), and the glenohumeral joint was centered in the imaging volume of the biplane fluoroscope. Biplane radiographic images of the shoulder were acquired at 30 frames/s (15 frames/s temporally offset for each source; an oscilloscope was used to check the offset for the biplane system and to ascertain if it was constant), while manually elevating the humerus relative to the scapula in the frontal plane (approximately 2.66 s from rest to full elevation).

Fig. 1
figure 1

Sawbones radiopaque shoulder bone model, fixed in a custom apparatus, was imaged using a clinical biplane fluoroscope, while manually elevating the humerus relative to the scapula

Full size image

Fluoroscopic images were also acquired at multiple static humeral elevation angles (73 kVp, plane 1: 39 mA, plane 2: 92 mA) using the custom apparatus to evaluate the effect of calibration error. This will be explained in more detail below.

The flat-panel detectors remained stationary and were positioned 90° from each other for all image acquisitions. A custom calibration cube was imaged using the fluoroscopic setup to define the orientation of the X-ray sources and detectors. Following fluoroscopic imaging, CT imaging was acquired on a clinical CT scanner (Siemens Definition FLASH; Forchheim, Germany), with slice thickness: 1 mm; kVp: 140; FOV: 426 mm; and pixel image size: 512 × 512, to obtain the 3D bone models and 3D positions of the beads.

2.2 Image Analysis

3D models of the humerus and scapula were manually segmented from reconstructed CT volumes (AnalyzePro; Mayo Clinic). Coordinate systems were computed for each 3D bone model using anatomical features as recommended by the International Society of Biomechanics (ISB) as shown in Fig. 2 [9]. Forty paired biplane digital fluoroscopic images, corresponding to one elevation movement, were analyzed.

Fig. 2
figure 2

Humerus and scapula coordinate systems based on ISB recommendations

Full size image

To quantify bone rotations and translations, the 3D–2D image registration was performed using open-source software (JointTrack Biplane, sourceforge.net/projects/jointtrack/) [10]. To perform a comparison between 3D–2D registration and RSA analysis, the 3D positions of the beads were determined relative to the bone coordinate systems by segmenting them directly from the CT volumes.

The 3D positions of the beads were tracked from the biplane fluoroscopy images using a freely available software (XMA Lab; Brown University) [11], and then used to calculate frame-by-frame motion for each bead set and thus the corresponding bone model.

2.2.1 Asynchronous Biplane Registration

To quantify bone rotations and translations, the 3D bone models were registered to each set of two biplane images under the assumption that they were synchronous, despite the joint movement between the two asynchronous images (Fig. 4a). The 3D positions of the beads were also determined from the asynchronous biplane images using the RSA technique.

2.2.2 Generating Interpolated Images

Since the proposed method relies on generating interpolated images from the 3D data, a contour projection algorithm was developed. Following the segmentation of the CT image volumes, the 3D bone models are defined as a set of vertices and triangle meshes. The silhouette of the 3D bone models were identified by the contour edges, which are common sides of triangle meshes that have the normal pointing in opposite directions from the X-ray source (Fig. 3) [12]. The silhouette was then projected on the image plane to create the missing images. A Matlab script (version 9.0.0.341360 (R2016a), The Mathworks Inc., Natick, Massachusetts) was used to identify the silhouette of each bone model and project it onto the image plane to create the missing images. Since JointTrack software uses a contour-based registration technique, the projection of the silhouette of the bone models is enough for the image registration.

Fig. 3
figure 3

2D perspective silhouette of bone models was projected to create simultaneous corresponding fluoroscopy images

Full size image

2.2.3 Semi-Synchronous Biplane Registration

In the proposed interpolation algorithm, the bone models were registered independently to each view from two X-ray sources. The bone kinematics from sets of two successive images for view 1 (X-ray source 1) were measured using single plane image registration and then interpolated to determine the bone pose at the time between the two acquisition times (Fig. 4b). Pose interpolation was performed as the interpolation of rigid body transformations between two registrations. First, the rigid body transformations were converted to kinematic parameters (three translations along and three rotations around X, Y, and Z axes). Then each kinematic parameter was averaged between two registrations, assuming the bone models moved with no acceleration during each time interval to determine the interpolated pose.

Fig. 4
figure 4

An interpolation algorithm was used to generate the missing images in biplane image sequences to produce interpolated-synchronous image pairs: (a) asynchronous biplane images; (b) generate missing images in view 1; (c) generate the missing images in view 2; (d) interpolated-synchronous biplane images

Full size image

Since the single plane registration accuracy for in-plane bone translation and rotation is much higher than the registration accuracy for the out-of-plane motions, the interpolated pose is only reliable in the in-plane direction for the corresponding view (view 1 in this case). We can therefore project the bone model with the new interpolated pose, given the known position of the view 1 source and detector and the surface geometry of the bone models. This generates the missing image which coincides with the corresponding fluoroscopy images from view 2 (Fig. 4b). The same process was repeated for view 2 (Fig. 4c). The 3D bone models were then matched to each set of interpolated-synchronous biplane images in order to determine the joint movement (Fig. 4d). The same technique was used for the RSA analysis in order to generate interpolated-synchronous biplane images of the beads.

2.3 Comparison Analysis

Since our RSA (gold standard) results were also determined from the interpolated-synchronous biplane images, we first needed to evaluate the accuracy of the RSA results. We measured reprojection error as an indication of the accuracy of the RSA for the two registration methods. The reprojection error is a geometric error corresponding to the image distance between a projected point and a measured one and is used to quantify how closely an estimate of a 3D point recreates the point’s true projection. However, the reprojection error contains the calibration error, in addition to the error due to the asynchronous acquisition. To measure the reprojection error due to the calibration error alone, we analyzed images that were acquired statically at multiple static humeral elevation angles. Static images can be assumed as synchronous biplane images, considering no object motion occurs between the acquisitions.

We evaluated the accuracy of the 3D–2D registration by computing the absolute error between the 3D–2D registration results and the RSA analysis, for each of six poses (three positions and three orientations) of the humerus and scapula, in each image pair.

3 Results

For the asynchronous RSA analysis, mean reprojection error of all the beads was 1.0 ± 0.7 pixel (max 4 pixels), compared to the 0.4 ± 0.3 pixel (max 1.7 pixels) error for the interpolated-synchronous RSA analysis. The average reprojection error due to the calibration error alone was 0.06 ± 0.05 pixel (max 0.2 pixels).

For the asynchronous biplane acquisition, the mean absolute error between the 3D–2D registration approach and the RSA analysis were 0.42 ± 0.03 mm and 1.03 ± 0.22° (max error: 1.24 mm and 5.73°) for the humerus, and were 0.37 ± 0.04 mm and 0.28 ± 0.03° (max error: 1.43 mm and 1.11°) for the scapula.

For the interpolated-synchronous biplane acquisition, the mean absolute error between the 3D–2D registration approach and the RSA analysis were 0.18 ± 0.01 mm and 0.4 ± 0.13° (max: 0.6 mm and 1.7°) for the humerus and were 0.25 ± 0.02 mm and 0.17 ± 0.02° (max: 0.9 mm and 0.7°) for the scapula. The absolute error for all frames is shown for each kinematic variable in Fig. 5.

Fig. 5
figure 5

The frame-to-frame absolute error (circles) and mean absolute error (arrows) for each kinematic variable (X: anterior–posterior translation; Y: superior–inferior translation; Z: medial–lateral translation; Xr: elevation (abduction–adduction); Yr: axial rotation, Zr: flexion–extension) at different elevation angles: (a) humerus kinematics from asynchronous; (b) scapula kinematics from asynchronous; (c) humerus kinematics from interpolated-synchronous; (d) scapula kinematics from interpolated-synchronous registration

Full size image

4 Discussion

The notion of registering fluoroscopic images to 3D data has been studied for several decades. Early findings are reported in [13]. Many approaches have been proposed; however, nearly all of them assume synchrony between images. Single plane approaches have been proposed where the third dimension is geometrically modeled [14]; however, errors were high [15]. Moreover, when motion is introduced the accuracy of 3D bone pose, reconstructed from either the RSA technique or the 3D–2D image registration, is even more influenced by the synchronicity of the biplane images.

Our studies showed that the introduced interpolation algorithm improves this accuracy by creating interpolated-synchronous biplane images. The interpolated-synchronous biplane registration was expected to perform superiorly. With 3D–2D image registration, movement of the bone models along the out of plane axis produces relatively little change in the model projection. By using the simultaneous image pair, registration in one view serves to correct the depth estimate in the other, and vice versa. The reprojection error in the RSA analysis due to asynchronous biplane acquisition was also reduced using the same interpolation technique. The interpolated-synchronous biplane registration technique produced average errors (0.18 mm and 0.4° for the humerus and 0.25 mm and 0.17° for the scapula) that are in the range of published results of other studies that have validated the 3D–2D registration technique at the shoulder using custom biplane (synchronous biplane) fluoroscopes [2, 3, 7]. Therefore, our interpolation approach is able to provide acceptable accuracy for shoulder kinematics. The difference between the two approaches becomes more pronounced with faster bone movement. This is suggested by the greater improvements in accuracy seen in humerus tracking compared to the scapula, which underwent slower movement. One limitation of the study is that we assumed the bone models moved with no acceleration in each time interval. However, this was an acceptable assumption for the small time intervals between two successive frames and the movement velocity examined in this study.

In order to quantitatively demonstrate the accuracy of the proposed method, this study was conducted with artificial bone models and a single shoulder activity. Results may differ for a human shoulder joint with surrounding soft tissue, and with different shoulder motions. To address some of these issues an intensity-based registration scheme may be used where a CT image is registered to the X-ray images by comparison of the pixel intensity between digitally reconstructed radiographs (DRR) and the X-ray image [2]. The use of an intensity-based registration approach also reduces the dependency of registration on the accuracy of segmentation [16].

Future work includes the investigation of accuracy of the proposed technique in different shoulder activities using a cadaveric model and the addition of digitally reconstructed radiographs (DRRs) to our protocol for intensity-based 3D–2D image registration. The introduced interpolation algorithm would be applicable to a variety of 3D–2D registration applications, and is of interest to activities with fast bone movements.