Introduction

The human spine is made up of complex anatomical structures. Therefore, pathological deformations of the spine are difficult to diagnose using two-dimensional imaging techniques. They have traditionally been evaluated in two dimensions using frontal (postereo-anterior) and lateral radiographs [1]. The advanced 3D imaging modalities like computed tomography (CT) and magnetic resonance imaging (MRI) have limitations on assessment of 3D spinal deformities as they are obtained in supine position [2]. The supine position can change the actual deformation. Although CT can provide more information for the deformity assessment, the subjects are exposed to higher levels of radiation, which leads to ethical considerations. Like CT scanning, MRI is also expensive. Since the subjects with spinal deformities may have surgical implants and corrective tools, use of MRI has limitations as a diagnostic tool [3]. Hence, stereo-radiography is the most widely used technique for 3D reconstruction of the human spine.

In earlier applications of stereo-radiography, 3D reconstruction techniques were based either on the triangulation of stereo-corresponding points (SCPs) [4, 5] or together with the non-stereo-corresponding points (NSCP) [6, 7]. In the SCP method, six stereo-corresponding points were identified on biplanar radiographs and reconstructed in 3D using a triangulation method with the help of known calibration parameters. Calibration parameters were computed with the help of a calibration bench. The number of reconstructed points was very low in this process. Hence, NSCP, a method that uses points available in only one of the radiographs for reconstruction process, was developed. Though the number of reconstructed points increased, it ignored many important image features available in each radiograph. Also, a lot of manual intervention was required to digitize numerous landmarks identified on the radiographs. Hence, this approach was more suitable for research purposes, but with a restricted usage in the clinical environment.

To automate these processes and use many other image features, several attempts have been made. Statistical and geometric inferences derived from the human vertebrae were used by Pomero et al. [8] to develop an automated process. Longitudinal inferences were added to this process by Humbert et al. [9]. A semi-automated approach that interpolates and optimizes the vertebral contours of a few vertebrae was developed by Dumas et al. [10]. These methods use a dedicated specialized system in which two radiographs are acquired simultaneously. However, neither the accuracy of vertebrae location nor of clinical indices was evaluated completely. Zhang et al. [11] have used spine midline and vertebral body contours for spine reconstruction. The use of fundamental matrix and epipolar geometry limits the process to a projective reconstruction. Also, it was not validated with standard reconstruction techniques. At present, to the best of our knowledge, no technique is capable of performing accurate reconstructions of the spine automatically, or with very limited user input.

In this study, to improve visibility, a novel 3D reconstruction method is developed using the angles at which vertebrae are oriented in the biplanar radiographs. The radiographic environment is fully calibrated. Vertebral orientations are automatically extracted from the radiographs using several image-processing steps [12]. The frontal and lateral radiographs of scoliotic subjects are acquired in a calibration bench. The radiographic environment is calibrated using the direct linear transformation method. From the radiographs, six stereo-corresponding points are identified using a semi-automatic procedure. Using the calibration information, the 3D positions of these points are computed by triangulation. The spine midline is estimated by joining the center of the vertebral endplates.

A generic spine model is developed according to the specifications available from literature [13] and is deformed to fit all the SCPs. The projections of this model along frontal and lateral planes are computed and vertebral orientations are automatically calculated. The deformation of the generic model is performed until the projected angles match with the angles calculated from the radiographs, for each vertebra. This refinement of the SCP-reconstructed model will result in a more realistic spine model. It is validated by back projecting the reconstructed 3D model on the biplanar radiographs. The whole procedure is semi-automatic as the user interaction is limited to initialization of vertebral landmarks and vertebral positioning. Hence, it reduces observer variability. Also, the low-cost generic model and the use of back projection for validation result in an economic 3D reconstruction procedure for the quantification of spinal deformities in any clinical environment.

Materials and Methods

The stereo-radiographic 3D reconstruction method presented here is based on the deformation of the generic spine model according to the points obtained from SCP reconstruction and the geometric inferences available in each radiograph. This procedure is explained in the following section. Figure 1 shows the flowchart of the entire process.

Fig. 1
figure 1

Procedure for 3D spine reconstruction

Full size image

SCP Reconstruction

The 3D reconstruction method from stereo-corresponding points (SCPs) uses biplanar radiographs of the spine acquired from a subject in standing position. The method proposed by Dansereau et al. [14] is used for calibration of radiographic environment. A simple, patient-friendly calibration bench shown in Fig. 2 was developed for this purpose. Although the method suffers from errors due to the involuntary motion of patients between radiograph acquisitions, it has lesser reconstruction errors compared to self-calibration techniques [15]. Methods involving the calibration bench give results which are close to gold standards [16]; hence, this approach is followed.

Fig. 2
figure 2

Calibration bench

Full size image

It consists of a positioning apparatus and a calibration object. The positioning apparatus consists of a rotary platform and a stabilization device. The rotary platform enables the subject’s rotation during biplanar radiograph acquisitions. Frontal and lateral radiographs can be obtained by this procedure. The stabilization device encompasses the subject and prevents position and posture changes due to the movements of the subject. Hence, it minimizes patient motion between acquisitions. A calibration object consists of embedded radio-opaque markers (steel beads) on the acrylic sheets that are attached to the three sides of the positioning apparatus. Global coordinates are as shown in Fig. 2. Steel beads of 2 mm diameter are used as control points for calibration. They are embedded on acrylic sheets in all the three dimensions. The first marker from the left in the lowest row of the YZ-plane is considered as the origin and the 3D coordinates of all the remaining markers are measured with respect to this by means of a vernier (accuracy 0.02 mm).

A linear regression technique called direct linear transformation (DLT) [17] is used for calibrating the radiological environment. The major advantage of the DLT method is due to its linearity and low computational complexity. Also, it does not depend on a static setup and is independent of the skills of the radiographic technician. DLT computes the mapping between space coordinates and the 2D image coordinates. It needs a set of markers whose coordinates are already known. Steel beads with known coordinates on the calibration bench are used for this purpose. The corresponding image points due to these markers on the biplanar radiographs are used to estimate the mapping.

Six corresponding points are sufficient to solve this system of equations. But, the large calibration bench, which can surround the subject, is used in order to reduce DLT extrapolation error. When the object is radiographed on this calibration bench, the biplanar radiographs contain projections of the markers (with known 3D coordinates) as well as the anatomical structures (whose 3D locations have to be detected). Using the corresponding markers, the radiological environment is calibrated. Twenty markers are used in order to reduce the identification errors and increase the robustness of the algorithm. The accuracy of the calibration is measured by computing the re-projection error. It is defined as the error between the observed projection of a point to the projection of the same point under the calibration matrix. A mean error of 0.5 mm is observed in five subject cases. The Levenberg-Marquardt algorithm [18] is then applied to minimize the re-projection error. The algorithm iteratively computes the correction and updates the parameters until correction becomes negligible. Hence, it reduces the re-projection error. Thus, the error between the observed projection and re-projection is reduced, which in turn increases the reconstruction accuracy. Since the X-ray source is used, the lens distortion is not considered during the calibration. Since the extrinsic parameters change for each subject, the calibration is performed each time before the reconstruction.

Five subjects (female) suffering from idiopathic scoliosis with S-shaped spinal curves have volunteered for this study. Subjects are of the age group 10–14 years. The local ethical committee has approved this study. Two radiographs (frontal and lateral) are obtained from these subjects. Six stereo-corresponding points per vertebra are identified on the radiographs using a semi-automatic method. These points are the top and bottom of the pedicles and the center of the vertebral end plates. The procedure is explained in detail in our previous work [19]. Using the optimized calibration matrix, 3D positions of all the landmarks are computed by linear triangulation using the DLT method. This procedure is repeated for the remaining vertebrae of scoliotic subject (T1 to L5) to get the SCP reconstruction of the spine. The result obtained from subject 3 is shown in Fig. 3 along with the biplanar radiographs having markers for calibration. The six-point model thus obtained fails to give the complete picture of the human spine. All the landmarks are obtained from the vertebral body, whereas the important structures that decide the shape of the spine are present in the posterior arch. Hence, to improve the visibility, a new method has been proposed.

Fig. 3
figure 3

Radiographs and corresponding 3D reconstructed spine of subject 3. a Frontal view. b Lateral view

Full size image

Automatic Extraction of Vertebral Orientation Features

Initially, biplanar radiographs are enhanced using multiscale mathematical morphology and vertebral boundaries are segmented using a gradient vector flow (GVF) snake algorithm. Enhancing a low-contrast noisy radiograph is a challenging task. Global enhancement techniques fail in this process as the dark and bright features are unevenly distributed in the radiographs. Hence, multiscale mathematical morphology [20] which extracts the scale-specific features to enhance the local contrast of noisy radiographs is used for enhancement. The gray scale opening function eliminates the brighter regions that are larger than the structuring element. The opened image is subtracted from the original image to obtain bright features. This is called top-hat transformation. Thus, the multiscale structuring element extracts the dark and bright features from top-hat transformations. They are combined to form an enhanced radiograph. Mathematically, it can be expressed as:

$$ \overset{\sim }{g}\left(r,c\right)=g\left(r,c\right)+0.5{F}_B^o\left(r,c\right)-0.5{F}_B^c\left(r,c\right) $$
(1)

Here, \( \overset{\sim }{g}\left(r,c\right) \) is the image after enhancement, g(r, c) is the original image, and F o B (r, c) and F c B (r, c) are the bright and dark features obtained from the top-hat transformation. A constant weight assigns impartial weightage to all the features and results in an enhanced image.

Due to better convergence properties, the gradient vector flow (GVF) snake algorithm [21] is used to segment the vertebral body contours from the radiograph. In this algorithm, a curve x(s), s ϵ [0, 1] is initialized near the object to be segmented. It moves under the influence of internal and external forces. The energy minimization function is given by:

$$ E={\int}_0^1\frac{1}{2}\Big(\alpha {\left|{x}^{\prime }(s)\right|}^2+\beta {\left|{x}^{\prime \prime }(s)\right|}^2+{E}_{\mathrm{ext}}\left(x(s)\right)ds $$
(2)

Here, α and β are controlling parameters for tension and rigidity, x′ and x″ are first and second derivatives. E ext is the external GVF force “v” derived from the image. It is defined as a solution of the following diffusion equation:

$$ \frac{\partial v}{\partial t}=\mu {\mathit{\nabla}}^2v-\left(v-\mathit{\nabla}f\right){\left|\mathit{\nabla}f\right|}^2 $$
(3)

In this equation, μ is the regulation parameter, f is the edge map image, and ∇ is the gradient operator. The GVF snake has the capability of convergence, although it is initialized far from the boundary. Also, it is capable of converging into the boundary concavities. Hence, the vertebral body contours are accurately segmented due to this algorithm. Morphological operators are then applied on segmented radiographs to retain only the vertebral boundaries and clear the remaining data. Since the horizontal edges have to be extracted, a Sobel [22] operator is applied to each of the radiographs for vertebral edge detection. The Hough transform [23] is used to measure the angles of orientation of every vertebrae in each radiograph.

Figure 4a shows the segmentation results. Morphological operators are used to retain only the vertebral body contours in the radiographs. Figure 4b shows the horizontal edges detected using a Sobel operator.

Fig. 4
figure 4

a Segmentation results. b Edge detection

Full size image

Generic Spine Model

A generic spine model is developed using the geometric details given by Panjabi et al. [13]. A 3D coordinate measuring machine is used for measurements from several cadaveric spines. The mean values of linear and angular dimensions and the area of vertebral body, pedicle, spinous, and transverse processes are obtained for each vertebra. These parameters are used for developing a generic spine model. The complex vertebral shapes are approximated to simpler versions and a generic feature-based model is developed using CATIA V5 software (Dassault Systemes, France). The dimensions are kept within the limits of standard errors. Due to the irregularity in different cross sections of various regions of vertebrae, necessary shape approximations are done ensuring a sufficient visual agreement with the actual vertebrae. The articular facets are assumed to have no lateral inclination and are modelled so as to provide a clear distinction between thoracic and lumbar vertebrae. Deformations from the normal shape are neglected. The model consists of T1-L5 vertebrae, as most of the spinal deformities are observed in this area. Figure 5 shows the generic spine model designed for this study.

Fig. 5
figure 5

Generic spine model

Full size image

Personalized 3D Reconstruction of the Spine

In the SCP reconstruction method, 3D positions of the midpoints of each vertebral endplate are computed as landmarks. The spine midline is formed by fitting a cubic B-spline curve to these points. Initially, vertebrae in the generic model are resized and aligned along the spine midline. Small translations and rotations are applied to each vertebra of the generic model to fit other available SCP landmarks in least square matching procedure.

Now, the angular features are used for the step-by-step deformation of the generic spine model starting from the T1 vertebra. Coordinate convention is the same as that of the global coordinates assumed during the calibration procedure. The T1 vertebra in the generic model is first moved under the constraint of spine midline (i.e., its center along the midline, along Z-axis) within an average intervertebral distance. Then, it is rotated in small angles around X- and Y-axes to match the angles obtained from frontal and lateral radiographs, respectively. The projections of this model along the frontal and lateral plane are obtained.

The edges are extracted from the projections using a Sobel operator. These edges after deformation are compared to the horizontal edges obtained from the radiographs. For comparing the similarity in the orientation of vertebrae, a histogram of the gradients is computed. If orientations are alike in these two images, only one peak is observed in the histogram. This indicates the similarity in the orientation of the vertebra in the generic model and the actual deformity of the subject. The geometric transformations (rotations and translations) are applied in small steps until a single peak is observed in gradient histogram.

This procedure is repeated for T2-L5 vertebrae of the spine. The deformed model obtained after this procedure is the personalized 3D-reconstructed spine model corresponding to the frontal and lateral radiographs of the subject. Figure 6 shows the more realistic 3D-reconstructed model of the scoliotic spine of subject 3. Intervertebral distances can be adjusted to improve the vertebral alignment.

Fig. 6
figure 6

Improved stereo-radiographic 3D reconstruction of subject 3

Full size image

Results

The proposed 3D reconstruction procedure is applied to biplanar radiographs obtained from five subjects with spinal deformities in K. M. C. Hospital, Manipal. Results obtained from subject 3 have been already discussed in the previous section. A similar procedure is repeated for the radiographs of remaining subjects to obtain the personalized 3D spine models. Figure 7 shows the 3D-reconstructed models of all the subjects. In order to validate the result, both qualitative and quantitative approaches have been followed. In the qualitative approach, the 3D spine model is projected on frontal and lateral planes. Their midlines are computed by fitting B-spline curves between vertebral and plate centers. These curves are superimposed on the already available spine midlines from frontal and lateral radiographs as shown in Fig. 8. A close similarity can be observed between these two curves and similar results are obtained for all the five cases. The vertebral shape comparison is not possible in these cases as the feature-based generic model which has a fixed shape is used.

Fig. 7
figure 7

Frontal radiographs and corresponding 3D models

Full size image
Fig. 8
figure 8

Superimposing the projection of the 3D model on the spine midline of two views of subject 3 radiographs

Full size image

In the quantitative approach, a comparison of mean error between the corresponding points for six SCPs from 3D spine projections and respective radiographs is carried out. The 3D spine is projected on frontal and lateral planes and 2D positions of the six SCPs are identified. First, the mean error between endplate centers of the projection and the radiograph is calculated. As the endplate centers form the spine curve, the mean error between the corresponding points on two curves is obtained from the Euclidian distance d i between corresponding \( \mathrm{points}=\frac{\sum {d}_i^2}{N} \), where N is the total number of points [24]. The standard deviation is given in Eq. 4.

$$ \mathrm{S}\mathrm{D}=\sqrt{\frac{1}{N}{\displaystyle \sum_{i=1}^N{\left({x}_i-\mathrm{mean}\right)}^2}} $$
(4)

For subject 3, mean error between corresponding endplate centers is found to be 1.62 ± 0.86 mm. Similarly, the mean error between corresponding pedicle landmark positions is computed and it is found to be 3.46 ± 2.65 mm. Then, the global mean error between all the six landmarks and their corresponding model projections are computed. It is found to be 2.52 ± 2.43 mm. This global mean error computation procedure is repeated for all the five subjects. Their global mean errors are comparable to that of subject 3 (2.46 ± 2.54 mm).

For further validation, an expert is asked to measure the Cobb angle manually from the radiographs. It is compared with the angles obtained from the projection of the 3D spine model on frontal and lateral planes. The mean absolute error of 2.5° ± 1.2° is found in the five subjects.

Discussion

At present, 3D reconstruction of the spine is performed using six stereo-corresponding points and twenty or more non-stereo-corresponding points identified on the biplanar radiographs. These points are reconstructed in 3D and used for deforming the generic model obtained from CT model to get the personalized model of the spine. The proposed method uses only six SCPs and automatically derives vertebral orientations to deform the generic model for 3D reconstruction. Thus, the geometric inferences are effectively used for improving the visibility of SCP reconstruction without using additional corresponding points. Thus, the proposed method is computationally inexpensive compared to NSCP method. The calibration bench used here is simple in design and economic. Also, the small radio-opaque steel beads occlude the very least part of the anatomical structures of interest in the radiographs. The landmark identification procedure is semi-automatic and reduces the observer variability [19]. A simple feature-based CAD model is used as the generic model and thus the method is a low-cost alternative to the current approach.

The qualitative comparison of spine midlines in radiographs and projections of 3D models on frontal and lateral planes has been carried out. It shows a close similarity in all the five subjects. The Cobb angles measured by the expert are similar to the angles obtained from the 3D model projections. The quantitative comparison of these curves was also done. The vertebral endplate centers, which form the spine midline show a less than 2 mm mean error in all the five subjects. The mean error in pedicle landmark positions is less than 4 mm for all the five subjects. The mean error in the pedicle landmark position is slightly larger. This is due to the fixed positions of the pedicles in the generic model. The global mean error obtained for all the five subjects is compared with the results obtained by similar reconstruction methods from literature (Table 1). The standard deviation is greater than the corresponding mean error due to the rigid generic model used for reconstruction.

Table 1 The accuracy of spine reconstruction with SCPs
Full size table

A proposed work has a similar error magnitude to that of Aubin et al. [5]. However, they use 18 landmarks per vertebra which are manually identified. This results in a complex procedure and causes observer variability. It also requires a geometric model built using CT scan models of typical vertebrae for reconstruction and a 3D scanner for validation. Hence, the cost of reconstruction is very high compared to the proposed work. Delorme et al. [16] use 12 points per vertebra for reconstruction. Though, the number of points per vertebra is more than the proposed work, accuracy is less. Humbert et al. [9] use only 2 splines per patient, but it requires 13 additional actions from the user for each view. Though, the accuracy of this method is high, the authors did not perform an accuracy study on vertebrae location and orientation. Zunhua Zhang et al. [11] have validated their result with an SCP reconstruction, which already has an error of 1.21 ± 1.4 mm. Also, the epipolar geometry errors are added which will result in a poorer accuracy compared to the proposed work.

Hence, the proposed work can be considered as a close approximation of the human spine when only stereo-corresponding points are used for reconstruction. The procedure is semi-automatic and the vertebral orientation adjustment is also manual. If these procedures are automated, this method can be used in clinical practice.

Conclusion

A novel 3D spine reconstruction method that uses geometrical inferences has been proposed. It increases the visibility of SCP reconstruction without considering the additional corresponding points. Thus, it reduces the observer variability and, in turn, improves the accuracy of the 3D reconstruction. Due to the use of a CAD-based generic model, low-cost calibration bench, and the exclusion of a 3D scanner for validation, the entire process is economical. Geometric inferences like the orientation of vertebrae are used for manual deformation of each vertebral model. The automatic vertebral deformation is the next step to achieve a fully automatic reconstruction procedure. More geometric inferences can be incorporated to get a more realistic spine model. Our future work is to quantify the spinal deformity using the proposed model. Thus, a low-cost, patient-friendly, visually sound 3D spine reconstruction technique has been proposed. It can be a useful tool for the diagnosis of the patients with spinal deformities.