Keywords

1 Introduction

The hip joint center (HJC) is an important reference point for decisions regarding the treatment of hip dysplasia. Unlike prominent bony landmarks, the HJC cannot be palpated, and thus, its determination must be calculated [1]. In the clinic, the HJC location is manually estimated by expert orthopedists, which is time consuming and labor intensive. It has become increasingly important to automatically and efficiently determine the HJC coordinates.

Current state-of-the-art methods for HJC determination can be categorized into three classes: Mose circle methods, functional methods and regression methods. (1) The Mose circle method is a quantitative approach used to estimate the HJC by using a template of concentric circles [2], and some modified Mose circle techniques have been proven to provide a more accurate approximation of the HJC than manual prediction [3]. However, when the ossification of the femoral skull epiphysis is insufficient, the HJC position determined by this method may be eccentric. (2) Functional methods calculate the trajectories of markers attached to the thigh and then locate the HJC by estimating the relative motion of the thigh and pelvis [4]. Many optimization approaches have been proposed to improve the accuracy and calculation time [5], such as the quadratic best sphere fitting [6], bias-compensated quartic best fitted sphere [7], and Reuleaux [8] methods. However, the estimation of the HJC relies heavily on the quality and range of movement during calibration trials. Due to soft tissue artifacts and limited circumduction movement, it is difficult to establish a reliable local coordinate system (LCS) of the femur or pelvis. A functional method would result in HJC location errors of 26 mm when performing substantially limited hip circumduction movement [4]. Functional calibration is challenging for those with impaired movement control or reduced range of motion, and recent studies suggest that it may be less accurate in clinical populations [9,10,11]. (3) Regression methods use empirical regression equations between palpable bone landmarks and the HJC as an anthropometric quantity function, and regression parameters are obtained by pelvis samples isolated in vivo and via medical imaging. However, these equations are not applicable universally since HJC-related measurements are quite variable between races or even communities [12]. The mean error is estimated in the range of 15–30 mm when using available regression equations [5, 10, 11, 13, 14]. Thus, more accurate and adaptive prediction approaches are desired.

However, within the context of HJC determination, the challenges stem from three aspects. (1) The first aspect is large-scale cartilage. Due to insufficient ossification of the femoral head, the acetabular curvature displayed on the orthographic image of the hip joint is far smaller than the actual curvature. This property brings a new challenge to general regression methods. (2) The second aspect is intraclass variations. There are obvious differences between study subjects, such as the width and height of the pelvis, the ossification degree of the acetabulum and femur, inconsistent abduction/adduction performance, and an unexpectedly tilted or rotated pelvis when examinations are performed. These differences often fail to yield effective results. (3) The third aspect is adaptive HJC prediction. Different empirical regression equations have been proposed in the past to identify the HJC, but these methods are suitable only for certain groups of people [15]. How to adaptively predict HJC using intelligent learning models remains unsolved. Note that the deep learning approaches have been widely used in the field of medical images [16,17,18], but there are few studies to explore the deep learning approaches for the HJC prediction.

To accurately identify the HJC, this paper advocates the idea that static to dynamic prominent bony landmarks contribute to object localization. This idea is motivated by the following intuitive concepts: pelvis bone landmarks remain unchanged regardless of what movement the legs perform, and bone landmarks from the hip joint are dynamic when the legs perform large abduction/adduction and flexion/extension movements. Thus, it is desirable to investigate static to dynamic landmarks to achieve better predictions.

To this end, we propose an innovative hybrid framework for HJC identification. It is equipped with a fast deep stacked network (FDSN)-based fast prediction module and a dynamic registration graph-based accurate positioning module with the following four aspects.

  1. (1)

    Comprehensive prominent bony landmarks are extracted. Our method is based on X-ray images, and the extraction of the most comprehensive bone anatomical landmarks is desired. The coordinate system is built on multipose X-ray images (including abduction images and adduction images), and eight anatomical landmarks on each hip are annotated by expert orthopedists.

  2. (2)

    We introduce a new attribute optimization module, guiding the network to focus on useful anatomical landmarks. By ranking the correlation coefficients between the prominent bony landmarks and the HJC using the grey relational analysis (GRA) method, we select the high-importance attributes to train our model since some landmarks may be damaged.

  3. (3)

    The proposed FDSN method considers adaptive HJC prediction to tackle the problem of automated model optimization. Specifically, a regularized extreme learning machine (RELM) is employed to train the modules, and random feature mapping is shared among the modules to speed up the FDSN training. We initialized the model with optimal parameters obtained by the multiverse optimizer (MVO) method [19]. This MVO can be easily added to model architectures, increasing prediction accuracy.

  4. (4)

    The proposed dynamic registration graph-based positioning method is based on multipose X-ray images. A two-step registration approach is used to align two pelvis images, and we fit the circle center in a dynamic abduction/adduction manner for accurate HJC positioning. The proposed innovative hybrid framework features better accuracy and reliability than previously developed approaches.

2 The Proposed Method

2.1 Method Architecture

Our goal is to automatically and efficiently determine the position coordinates of the HJC. This goal requires that both static and dynamic anatomical landmarks be investigated for the best prediction of the HJC. To this end, we propose an innovative hybrid prediction framework, as shown in Fig. 1. It involves one landmark extraction module for the comprehensive prominent bony landmarks, one GRA optimization module for the importance measure of landmarks, one FDSN module that achieves fast HJC prediction in static cases and one dynamic registration strategy to fit the HJC in dynamic cases. Next, we will detail these modules.

Fig. 1.
figure 1

The overall flowchart of our proposed method.

Full size image

2.2 Landmark Extraction

Comprehensive prominent bony landmarks should be extracted from multipose X-ray images of hip joints. Considering that there were obvious differences between the study subjects, such as the width and height of the pelvis, the ossification degree of the acetabulum and femur, and inconsistent abduction/adduction performance, we built a coordinate system on each image according to the posterior inferior iliac spine on both sides, as shown in Fig. 2(a). Specifically, the subjects were asked to perform dynamic abduction/adduction movements. Sixteen prominent bony landmarks were annotated by orthopedists, as displayed in Fig. 2(b) and Table 1, and the HJC was determined by Mose circles with a template of concentric circles, as displayed in Fig. 2(c) and (d).

Fig. 2.
figure 2

Image processing.

Full size image
Table 1. Annotated anatomical landmarks.
Full size table

2.3 Grey Relational Analysis

GRA is a quantitative analysis and development trend comparison method, and it is used to analyze the relevance between input attributes and output predictions. The basic idea of the GRA method is to judge the closeness degree based on the geometric similarity between the reference sequence and the multiple comparison sequences [20]. In our study, the input attributes are the coordinates of prominent landmarks (also called the comparison sequence), and the output predictions are the coordinates of the HJC determined by Mose circles (also called the reference sequence). Note that our model takes the most prominent bone anatomical landmarks as the predictors, and the GRA optimization module can exclude some incorrect landmarks, such as damaged or broken landmarks.

The core ideas of GRA are as follows: the reference sequence is described as \({{\varvec{x}}}_{0}=\left\{{x}_{0}\left(1\right),{x}_{0}\left(2\right),\dots {x}_{0}(n)\right\}\), and the comparison sequence is represented by \({{\varvec{x}}}_{i}=\left\{{x}_{i}\left(1\right), {x}_{i}\left(2\right),\dots {x}_{i}(n)\right\}\), where n represents the number of the samples and i represents the i-th input attribute. We normalize the original sequence and can obtain the initialized reference sequence and the comparison sequences, \({{\varvec{y}}}_{0}=\left\{{y}_{0}\left(1\right),{y}_{0}\left(2\right), \dots y(n)\right\}\) and \({{\varvec{y}}}_{i}=\left\{{y}_{i}\left(1\right),{y}_{i}\left(2\right),\dots {y}_{i}(n)\right\}\), respectively. After the nondimensional transformation, the correlation coefficient between y0(t) and yi(t) at time k can be calculated:

$$ \xi_{i} (k) = \frac{{\mathop {\min }\limits_{i} (\Delta_{i} (\min )) + \beta \mathop {\max }\limits_{i} (\Delta_{i} (\max ))}}{{\left| {y_{0} (k) - y_{i} (k)} \right| + \beta \mathop {\max }\limits_{i} (\Delta_{i} (\max ))}} $$
(1)

where Δi\(\left(min\right)={min}_{k}(\left|{y}_{0}\left(k\right)-{y}_{i}(k)\right|)\), Δi\(\left(max\right)={max}_{k}(\left|{y}_{0}\left(k\right)-{y}_{i}(k)\right|)\), and β represent the distinguishing coefficients. The grey correlation grade γi between the reference sequence and the multiple comparison sequences is the average value of all the correlation coefficients.

If the coordinates of a prominent landmark obtain the highest grey correlation grade with the coordinates of the HJC, this landmark is most relevant to the HJC. Therefore, we introduce a new attribute selection module, guiding the network to focus on meaningful landmarks at input variables. The correlation coefficient is estimated to determine whether prominent landmarks should be added to train the models. By ranking the correlation coefficients between the prominent landmark and the HJC using the GRA method, we select the high-importance landmarks to train the model.

2.4 Fast HJC Prediction Module

The proposed FDSN method is shown in Fig. 3, which takes the prominent landmark coordinates as model inputs and achieves fast HJC prediction. Specifically, a regularized extreme learning machine (RELM) is employed to train the modules of the FDSN. This idea is motivated by the following intuitive concepts: (1) As an efficient approach to train neural networks, ELMs have low computational cost and good generalization performance [21]. (2) ELM-based random feature mapping is shared among the modules in the FDSN to speed up the training process. In the FDSN, the first module is trained using the RELM-based approach with additive nodes: random weights W1 and biases v1. Matrix H1 is calculated as shown in Eq. (2).

$$ \left\{ {\begin{array}{*{20}l} {{\varvec{H}}_{1} = {\varvec{G}}{(}{\varvec{\eta}}_{1} {)}} \hfill \\ {{\varvec{\eta}}_{1} = {\varvec{X}}_{1} {\varvec{W}}_{{1}} { + }{\varvec{N}}_{{1}} } \hfill \\ \end{array} } \right. $$
(2)

where G is an operator, X1 is the input, and N1 is the v1 vector in each row. As the weight matrix connecting the output and hidden layers, B1 is calculated using H1 and the chosen RELM-based algorithm. The first module output is described by \(\overline{{\varvec{T}}}_{{1}} = {\varvec{H}}_{{1}} {\varvec{B}}_{{1}}\).

Note that the modules of the FDSN are stacked on top of each other, and the input and the output of a module make up the input of the following module [22]. The FDSN can suffer from a badly chosen initial weight since some weights and calculations are reused when accelerating the training of the DSN and reducing memory usage. The FDSN algorithm has three extremely important parameters (including the number of hidden neurons (N), regularization parameter (r), and maximal number of modules (s)), and the manual tuning of these parameters can be a time-consuming and tedious process. To automatically and efficiently determine the optimal FDSN model parameters, we added an MVO global optimization module to the framework. The main inspirations of this MVO algorithm are based on three concepts in cosmology: white holes, black holes, and wormholes. A universe with a high expansion rate is considered to have white holes, and a universe with a low expansion rate is considered to have black holes. Each universe has wormholes to transport its objects through space randomly, and the objects are transferred from the black hole in the source universe to the black hole in the target universe. Mathematical models of these three concepts are developed to perform exploration, exploitation, and a local search.

In the MVO algorithm, a set of random universes is created to start the optimization process. During each iteration, objects in the universes with high inflation rates tend to move to the universes with low inflation rates via white/black holes. Note that every single universe performs random teleportation in its objects through wormholes towards the best universe [19]. The computational complexity of the MVO algorithms mainly depends on four factors (including the number of iterations, number of universes, roulette wheel mechanism, and universe sorting mechanism). Note that universes are sorted in every iteration, and the Quicksort algorithm is employed in this study, which has the complexity of O(nlog n) and O(n2) in the best and worst cases, respectively. We conduct roulette wheel selection in every universe over the iterations and can obtain the complexity O(n) or O(logn) based on implementation. Finally, the overall computational complexity can be obtained:

$$ \left\{ {\begin{array}{*{20}l} {O(MVO) = O(l(O({\text{Quick sort}}) + n \times d \times (O(roulette{\text{ wheel))))}}} \hfill \\ {O{\text{(MVO)}} = O{(}l{(}n^{2} + n \times d \times \log n))} \hfill \\ \end{array} } \right. $$
(3)

where n represents the number of universes, l represents the maximum number of iterations, and d represents the number of objects. This MVO module can be easily added to model architectures for the optimal model parameters and increasing prediction accuracy.

Fig. 3.
figure 3

Fast HJC prediction module based on the FDSN.

Full size image

2.5 Accurate HJC Positioning Module

Calculating the trajectories of annotated bone landmarks under dynamic behaviors is desired. However, this calculation also introduces two issues. One issue is that bone landmarks are manually annotated by orthopedists. The manual tuning of landmark coordinates can be a time-consuming and tedious process. Moreover, it is difficult to obtain optimal fixed points for accurate registration. The other issue is that two pairs of dynamic points (dynamic landmarks in hip joints in abduction images and adduction images, as shown in Table 1) are too weak to identify the HJC, leading to an amplified distance error after fitting circles. Thus, it is necessary to further tune the control point locations (static landmarks in the pelvis in Table 1) and select enough dynamic landmarks (dynamic landmarks in hip joints in Table 1) as informative spatial locations for HJC localization.

To align two pelvis images (abduction images and adduction images), a two-step registration approach is used in this study. First, we tune the control point locations (static landmarks in the pelvis in Table 1) using the cross-correlation method, and the adjusted coordinates are accurate up to one-tenth of a pixel, which more closely matches the positions of the fixed landmarks than previous methods can achieve [23]. Note that the cross-correlation method is used to obtain subpixel accuracy from the image content and coarse control point selection. Second, we fit a geometric transformation to the adjusted control point pairs (adjusted static landmarks in the pelvis in abduction images and adduction images). Finally, every two pairs of dynamic landmarks (landmarks 5, 6 and 8 in abduction images and adduction images) are fitted to a circle center, and the average value of all the centers is taken as the final HJC, as shown in Fig. 4.

Fig. 4.
figure 4

HJC positioning module based on the dynamic registration graph.

Full size image

3 Experimental Results

3.1 Data Processing and Analysis

To show the effectiveness of our proposed HJC prediction method, we validated it on a dataset from the Wuhan Women and Children’s Health Care Center in China. In 100 subjects’ X-ray images and a total of 200 hips, a coordinate system was established on each X-ray image according to the posterior inferior iliac spine on both sides. Sixteen prominent bony landmarks were annotated by orthopedists. Taking the left hip joint as an example, the correlation between the coordinates of each landmark and the HJC was analyzed using the GRA method, as shown in Fig. 5. A relational grade value closer to 1 indicates a better corresponding landmark. Those landmarks with strong correlation (comprehensive relational grade value > 0.7) were retained as the input variables of the model.

Fig. 5.
figure 5

Grey relational analysis results.

Full size image
Fig. 6.
figure 6

Convergence curve of the MVO.

Full size image

3.2 Experimental Settings

Our static prediction module is based on the implementation of the FDSN. By considering the number of landmarks and the HJC, the end-to-end parameters of our model include the number of input neurons (16) and the number of output neurons (2). Note that 70% of the samples are used to train the model, and the remaining 30% of the samples are used for testing. Our optimization objective function is the average root mean square error (RMSE) from ten 10-fold validation processes performed on the training data using the FDSN method. Figure 6 shows the convergence curve of the global optimization obtained with the MVO. The optimal parameters obtained by the MVO include the number of hidden neurons (443), regularization parameter (1) and maximum number of modules (10). Our dynamic positioning module takes the average value of three fitted circle centers as the final HJC. All experiments are performed on the same computer with an Intel Core i7-8700 3.20 GHz CPU and 16 GB memory.

3.3 Comparison with State-of-the-Art Methods

The experimental results of our proposed FDSN method are shown in Fig. 7. We compared the proposed FDSN model with the Elman [24], radial basis function (RBF) [25], ELM [21], RELM [26], stacked ELM (SELM) [27], SELM-based autoencoder (AE-SELM) [27], and DSNELM [28] models. All experiments are performed with optimal parameters obtained by the MVO algorithm. We used the RMSE, mean absolute percentage error (MAPE), and R-squared (R2) value as evaluation metrics. We used the mean and standard deviation (SD) to analyze the results. We ran the experiment 1000 times (independently), and the average results for the testing sets are shown in Table 2. Obviously, (1) our FDSN method outperformed the seven state-of-the-art methods. Our method achieved the smallest RMSE, the smallest MAPE, and the largest R2 value. In addition, our algorithm achieved the smallest standard deviation among the three indicators. These results imply the effectiveness and robustness of our algorithm. (2) The ELM model achieved inferior performances. The reason for this is that the ELM is a single-layer feedforward network (SLFN) and is prone to underfitting or overfitting.

Fig. 7.
figure 7

Experimental results of our proposed method.

Full size image
Table 2. Average results of the 1000 independent runs for the testing set.
Full size table

In addition, we compared the proposed dynamic registration approach with traditional functional and regression methods. The proposed dynamic registration graph method does not participate in the training process and achieves the following results: RMSE = 0.4139 mm, MAPE = 1.4660, and R2 = 0.9962. The traditional functional method results in unsatisfactory location errors of up to 26 mm [4], the mean error of the traditional regression method is in the range of 15–30 mm [5, 10, 11, 13, 14], and the modified Ranawat method and pelvic height ratio method have estimation errors <5 mm [29]. We can clearly observe that the proposed static FDSN model and dynamic registration approach have better accuracy than traditional functional methods and traditional regression methods; in particular, the proposed dynamic registration graph features superior accuracy (RMSE = 0.4139 mm, MAPE = 1.4660, and R2 = 0.9962). These findings show that our framework obtains coordinates much closer to the real HJC than the previously developed approaches.

3.4 Ablation Analysis

To better demonstrate the effectiveness of the proposed FDSN method, we performed ablation experiments on our model, as shown in Table 3. The results indicate that (1) the model performance is evidently improved by combining the static landmarks in the pelvis and dynamic landmarks in the hip joint. Clearly, both the static landmarks and dynamic landmarks can help identify the HJC, and our fusion strategy can provide richer feature representations than single static/dynamic landmarks. (2) Applying an MVO global optimization module further optimizes the prediction of the proposed FDSN method (complexity analysis: 9.2598e−04 s with the MVO module and 2.8210e−04 s without the MVO module). Obviously, our proposed FDSN method greatly enhances the effectiveness and robustness in HJC identification.

Table 3. Average ablation results obtained with our proposed method.
Full size table

4 Conclusion

This paper proposes a novel framework to address the problem of HJC determination. First, comprehensive prominent landmarks from multipose X-ray images are extracted, and grey relational analysis (GRA) is used to guide the network to focus on useful variables. Then, a fast deep stacked network (FDSN) with a multiverse optimizer (MVO) is designed to achieve fast HJC prediction, and it considers adaptive HJC prediction to tackle the problem of automated model optimization. Finally, using a dynamic registration graph, the HJC localization ability of the whole framework is enhanced. Validation on a dataset from the China Wuhan Women and Children’s Health Care Center demonstrates the effectiveness of our hip joint center (HJC) determination method on X-ray images.