Introduction

Orthopedic robot systems (ORSs) have been widely applied to clinical surgery of knees, hips, spines, skulls, zygomas, maxillofacial, teeth, etc., greatly improving the safety, accuracy and efficiency of traditional artificial surgery [1,2,3,4]. During ORS-assisted surgery, surgical planning is first customized by the surgeon using medical images of patient’s anatomy and then guides the robot system to perform the required manipulation tasks (e.g., osteotomy and target positioning). For this purpose, patient-to-image registration (hereinafter referred to as registration) is necessary, which largely determines the surgical outcome. Its essence is to find a rigid transformation between the intra-operative patient’s anatomy (called patient space) and the corresponding CT images (called image space) using an appropriate registration method [5]. Generally, marker-based paired-point registration (MBPPR) and surface-based marker-less registration (SBMLR) methods have been developed to adapt to different surgical objects, conditions and procedures [6, 7].

MBPPR is commonly acknowledged to be high-accuracy [8, 9]. No fewer than three non-collinear artificial markers which significantly differ from patient’s anatomy in X-ray absorptivity, are frequently inserted into the bone [10,11,12], or fixed out of patient’s anatomy with an extending bracket [3], or glued onto the skin surface [13]. The paired fiducial points (FPs) in patient and image spaces determined by these markers can be manually or automatically obtained [14], and then the transformation between them can be solved by the representative least-squares fitting method [15]. However, MBPPR faces several practical challenges involving: introducing invasive bone-implanted markers not only requires time-consuming preparation, but also causes discomfort even complications to patients [11]; skin-adhered markers may slip or fall off, which negatively impacts the registration accuracy [5]; the target registration error (TRE) may be significantly amplified with the distance between the external markers and the patient’s anatomy [3]; artifacts of metal markers produced in CT images severely limit the segmentation and detection of markers [16].

Accordingly, SBMLR attracts more attention. The patient point cloud (PPC) acquired from the anatomic surfaces using a scanner, or a stereo camera [6, 17,18,19,20], or a digital touch probe [9, 11, 21] is frequently registered to the corresponding image point cloud (IPC) extracted from the CT images via the most representative iterative closest point (ICP) algorithm [22]. Generally, the PPC acquired by scanners or stereo cameras contains tens to hundreds of thousands densely distributed FPs to characterize a relatively complete anatomic surface, which is closest to the IPC in the number of FPs, thus greatly increasing the computing time. Furthermore, numerous disturbances (e.g., noises, outliers) existing in the PPC and/or IPC, caused by the uncertainty of sampling and image segmentation of bone surface, have adverse effects on the registration accuracy and robustness. Some reported algorithms [23,24,25,26] aiming at solving similar issues require the establishments of complex metrics or models, which increase the calculation complexity and programming difficulty. In contrast, the PPC collected by successively touching the exposed bone surface using a digital touch probe, usually contains only fewer than one hundred sparsely distributed FPs due to the limited accessibility and operation time [6], which is seriously asymmetric to the IPC in the number and distribution. In this case, numerous FPs regarded as possible disturbances in the IPC that cannot accurately match with those in the PPC, which may lead to a wrong registration trend. Besides, another marker-less scheme is to pick up the paired anatomic landmarks in image and patient spaces [5, 27, 28].

The goal of this study is to provide a simple, feasible, robust and high-accuracy (i.e., the TRE is about 1 mm) registration scheme for robot-assisted long bone osteotomy, which has rarely been systematically reported to our knowledge. Other practical requirements mainly involve that (1) invasive bone-implanted and skin-adhered markers are undesirable; (2) the exposed bone surface should be as small as possible; (3) the intra-operative real-time registration and tracking performance of the patient’s bone needs to be guaranteed; however, scanner-based and stereo camera-based schemes are usually subject to this limitation due to large amounts of data to be processed; (4) anatomic landmark-based method is not absolutely reliable because the long bone surfaces are usually too smooth to find distinguishing features.

Based on the consideration of the reported methods and practical requirements, we present a hybrid feature-based patient-to-image registration method termed CR-RAMSICP and integrate it into our self-developed ORS. In this paper, the ORS architecture is first outlined. Then, the proposed CR-RAMSICP is elaborated, in which coarse and fine registrations adopt different fiducial data and algorithms, respectively. The initial alignment between the PPC collected by the tracked optical probe and the IPC extracted from the CT datasets is accomplished, relying on the retro-reflective markers (RRMs) attached to the tracked rigid body which is rigidly fixed out of the bone with an extending bracket. The fine transformation between the PPC and the IPC is solved by an improved ICP algorithm based on the range-adaptive matching strategy (termed RAMSICP). Subsequently, we experimentally evaluate the accuracy of the proposed method and discuss some important impact factors. The registration results of isolated porcine iliums indicate that CR-RAMSICP is effective for accuracy improvement without bone-implanted markers, and exhibits great robustness to the CR errors, and low sensitivity to sampling with the tracked probe. Finally, the conclusions and prospects are given.

Context

The self-developed ORS is modularized into tracking, patient, image and robot subsystems (see Fig. 1). A high-accuracy infrared optical tracking system (Polaris Vega®, NDI, Canada) is selected as the tracking subsystem, including a set of near-infrared (NIR) sensors, a tracked rigid body and a tracked probe. Each tracked tool is localized by multiple attached wireless RRMs which can be tracked by the NIR sensors automatically. To construct the patient subsystem, the tracked rigid body is rigidly fixed out of an isolated porcine ilium (instead of a real patient) by inserting an extending bracket into the porcine ilium. Note that in clinical application, a special bone clip equipped on the extending bracket can avoid the invasiveness to a real patient. The image subsystem mainly provides an integrated surgical navigation software programmed by Python based on the open source software platform “3D Slicer” to guide the operation processes, which supports image processing, three-dimensional (3D) visualization, surgical planning, tool calibration, registration, intra-operative real-time navigation, etc. The developed software runs on a computer with an Intel (R) core (TM) i7-1065G7 (1.30 GHz) processor, 16 GB RAM, and the Windows 10 operating system. The robot subsystem is constructed by a six-degree-of-freedom robot (UR5, Universal Robots, Denmark) equipping a surgical tool, which can perform a full-automatic osteotomy process in strict accordance with the planned tool trajectory.

Fig. 1
figure 1

The self-developed ORS composition and the transformation relationships between CSs. G: global CS determined by the NIR sensors. P: patient CS determined by the RRM centers. TP: probe CS. B: robot base CS. T: surgical tool CS. I: image CS

Full size image

Each rigid transformation between any two coordinate systems (CSs) determined by the corresponding subsystems is defined and illustrated in Fig. 1, and mathematically described by a homogeneous transformation matrix:

$$ {}_{{{\text{ref}}}}^{{{\text{tar}}}} {\mathbf{T}} = \left[ {\begin{array}{*{20}c} {{\mathbf{R}}_{{3 \times 3}} } & {{\mathbf{t}}_{{3 \times 1}} } \\ {{\mathbf{O}}_{{{\text{1}} \times {\text{3}}}} } & 1 \\ \end{array} } \right]_{{4 \times 4}} = \left[ {\begin{array}{*{20}c} {n_{x} } & {o_{x} } & {a_{x} } & {t_{x} } \\ {n_{y} } & {o_{y} } & {a_{y} } & {t_{y} } \\ {n_{z} } & {o_{z} } & {a_{z} } & {t_{z} } \\ 0 & 0 & 0 & 1 \\ \end{array} } \right] $$
(1)

where \({}_{{{\text{ref}}}}^{{{\text{tar}}}} {\mathbf{T}}\) represents the transformation matrix from the reference CS ref to the target CS tar and R and t are the rotation matrix and translation vector of \({}_{{{\text{ref}}}}^{{{\text{tar}}}} {\mathbf{T}}\), respectively. The solution principles for several key matrices are briefly described as follows:

  1. (1)

    \({}_{G}^{P} {\mathbf{T}}\) and \({}_{G}^{{TP}} {\mathbf{T}}\) can be tracked by the NIR sensors directly, and then \({}_{P}^{{TP}} {\mathbf{T}}\) can be calculated as follows:

    $$ {}_{P}^{{TP}} {\mathbf{T}} = ({}_{G}^{P} {\mathbf{T}})^{{ - 1}} \cdot {}_{G}^{{TP}} {\mathbf{T}} $$
    (2)
  2. (2)

    \({}_{B}^{T} {\mathbf{T}}\) can be computed by combining the robot kinematics and the four-point calibration method [29].

  3. (3)

    To calculate \({}_{G}^{B} {\mathbf{T}}\), an optical calibrator was customized (see Fig. 2). First, the calibrator was installed on the surgical tool to ensure that the current tool center point ptcp of the surgical tool coincides with the calibration groove bottom point pgbp which can be directly tracked by the NIR sensors. Their relative movement was constrained by a clamping device. Then, the tool center point was controlled by the robot to reach any three non-collinear positions p1, p2 and p3 in turn which were used to establish the auxiliary CS A. Obviously, \({}_{B}^{A} {\mathbf{T}}\) and \({}_{G}^{A} {\mathbf{T}}\) are easy to determine, and thus \({}_{G}^{B} {\mathbf{T}}\) can be calculated as follows:

    $$ {}_{G}^{B} {\mathbf{T}} = {}_{G}^{A} {\mathbf{T}} \cdot ({}_{B}^{A} {\mathbf{T}})^{{ - 1}} $$
    (3)
    Fig. 2
    figure 2

    Schematic diagram of calibration between the tracking and robot subsystems

    Full size image
  4. (4)

    All surgical tool poses planned in image space should be eventually transmitted to the robot controller. Consequently, \({}_{I}^{B} {\mathbf{T}}\) is necessary and can be calculated as follows:

    $$ {}_{B}^{I} {\mathbf{T}} = ({}_{G}^{B} {\mathbf{T}})^{{ - 1}} \cdot {}_{G}^{P} {\mathbf{T}} \cdot ({}_{I}^{P} {\mathbf{T}})^{{ - 1}} $$
    (4)

where \({}_{I}^{P} {\mathbf{T}}\) represents the patient-to-image registration matrix to be solved by our proposed method.

Hybrid feature-based registration

Figure 3 illustrates the main procedures of the proposed hybrid feature-based registration method.

Fig. 3
figure 3

Overview of the proposed method

Full size image

Data acquisition

  1. (1)

    FPs acquisition in image space. The CT datasets of the patient subsystem were first scanned by a C-arm (ARCADIS Varic, Siemens, Germany), characterized by a 0.625 mm slice thickness and a 0.5 mm pixel size, then were filtered and denoised. Next, four RRMs attached to the tracked rigid body and the isolated porcine ilium were segmented out using a threshold-based method [30], and Fig. 4a shows their reconstructed virtual models. Subsequently, the IPC (see Fig. 4b) was extracted from the surface of the virtual porcine ilium model, represented as Y = {yj} for j = 1, …, NI.

    Fig. 4
    figure 4

    a 3D virtual model of the patient subsystem and placed virtual fiducials (blue points). b The IPC with 194 568 FPs. c The PPC with 40 FPs (blue points)

    Full size image
  2. (2)

    FPs acquisition in patient space. Each FP was collected successively by a single touch of the porcine ilium surface with the tracked probe. These FPs were mapped into the patient CS using the transformation illustrated in Fig. 1 to compose the PPC, represented as X = {xi} for i = 1, …, NP, and their distribution pattern shown in Fig. 4c satisfies that (i) eight sub-areas (SAs) denoted as SA1-SA8 were set on the unilateral smooth surface of the porcine ilium; (ii) each SA contains five randomly distributed FPs.

It should be clarified that during above-mentioned procedures, there must be no relative movement between the tracked rigid body and the porcine ilium. Besides, during clinical practice, the medical fixator attaching reference markers usually needs to be clamped on the real patient’s bone with a special bone clip intra-operatively, and thus intra-operative imaging is necessary, which provides better reliability and safety than pre-operative imaging.

Coarse registration

The RRMs can provide more reliable features than anatomic landmarks, and their geometric centers FP = {FP1, FP2, FP3, FP4} (see Fig. 1) can be tracked and mapped into the patient CS automatically. In image space, four virtual spherical fiducials with the same 11.5 mm diameter as the real RRMs were interactively placed through the built-in tool “Fiducials” of 3D Slicer, and their centers FI = {FI1, FI2, FI3, FI4} were adjusted to be consistent with those of the corresponding RRM models as much as possible (see Fig. 4a). Note that the virtual RRM models show slight deformations and partial defects, and thus the interactive placement of the virtual fiducials is prone to large artificial errors. The coarse registration (CR) matrix \({}_{I}^{P} {\mathbf{T}}_{{{\text{coarse}}}}\) represented by the transformation between FP and FI can be calculated by the singular value decomposition (SVD) algorithm [8]. Then the PPC X and the IPC Y can be initially aligned by updating X to CRX = {CRxi}, namely

$$ ^{{{\text{CR}}}} {\mathbf{X}} = {}_{I}^{P} {\mathbf{T}}_{{{\text{coarse}}}} \cdot {\mathbf{X}} $$
(5)

Fine registration using RAMSICP algorithm

The fine registration matrix \({}_{I}^{P} {\mathbf{T}}_{{{\text{fine}}}}\) represented by the transformation between CRX and Y is calculated using the proposed RAMSICP algorithm which is an extension of standard ICP algorithm to avoid falling into a local optimum and wrong registration tendency caused by poor CR and the large asymmetry of CRX and Y in the number and distribution. Figure 5 shows the basic principle of the RAMSICP algorithm.

Fig. 5
figure 5

Schematic diagram of the RAMSICP algorithm

Full size image

The k-distance neighborhood (k-DN) si of CRxi is selected as the matching sub-range, excluding numerous possible disturbances (e.g., outliers or noises) from X. Define the complete matching range set S as follows:

$$ {\mathbf{S}} = \left\{ {s_{i} |\left| {s_{i} - {}^{{{\text{CR}}}}x_{i} } \right| < k_{i} } \right\}{\text{ for }}i = {\text{1}}, \ldots ,N_{P} $$
(6)

where ki represents the size of si. In iterations, only the FPs limited in S can participate in registration. Thus, Y is updated to MRY = {MRyil(i)}, namely

$$ {}^{{{\text{MR}}}}{\mathbf{Y}} = {\mathbf{S}}{ \cap }{\mathbf{Y}} \ne {\emptyset } $$
(7)

Then, Each FP in MRY closest to CRxi is searched out to compose the point set SY = {Syi}. ki determines the IPC samples in iterations and thus affects the registration accuracy. We assume that whether ki is so large that numerous outliers and noises exist in MRY or too small to retain essential FPs in MRY, it is likely to cause mismatches. Consequently, a suitable ki seems necessary to minimize the registration error. In this study, considering the various CR results and local distributions of the IPC samples near each FP in the PPC, ki is adjusted adaptively by multiplying a positive scale factor Li and the Euclidean distance Di between the corresponding closest point pairs CRxi and Syi, namely

$$ k_{i} = L_{i} \cdot D_{i} $$
(8)

where Li is set as a uniform value L for simplification.

Actually, certain mismatched point pairs may be unavoidable due to the asymmetry of the PPC and the IPC. However, in this study, we disuse the rejection strategy [23] due to the few FPs in the PPC, and omit the weighting process due to the insignificant contribution to the accuracy improvement [31]. The objective function dh is defined to iteratively update the current transformation:

$$ d_{h} = \sqrt {\min \sum\limits_{{i = 1}}^{{N_{P} }} {\left| {{}^{{\text{S}}}{\mathbf{Y}}_{{i,h}} - {}_{I}^{P} {\mathbf{T}}_{{{\text{fine,}}h}} \cdot {}^{{{\text{CR}}}}{\mathbf{X}}_{{i,h}} } \right|^{2} } } $$
(9)

where h is the iteration number, and \({}_{I}^{P} {\mathbf{T}}_{{{\text{fine}}}}\) is solved by the SVD method [8]. Iterations are terminated when \(d_{{h + 1}} - d_{h} < \varepsilon\) where ε is set to 10−5 mm. Eventually, \({}_{I}^{P} {\mathbf{T}}\) can be calculated as follows:

$$ {}_{I}^{P} {\mathbf{T}} = {}_{I}^{P} {\mathbf{T}}_{{{\text{coarse}}}} \cdot {}_{I}^{P} {\mathbf{T}}_{{{\text{fine}}}} $$
(10)

Results and discussion

Experimental preparation

A series of experiments were performed to demonstrate the effectiveness of our proposed CR-RAMSICP in improving the registration accuracy characterized by the TRE, by comparison with that of CR and CR combined with the standard ICP algorithm (termed CR-ICP). First, depending on the planned osteotomy trajectory, a small bone block was resected from the compete isolated porcine ilium using our self-developed ORS (see Fig. 6), and the retained part was used for experiments. Then, follow our aforementioned procedures to acquire FPs. The TRE was computed using four distinct endpoints (see Fig. 6b) on the resected surface of the retained part, which can be expressed mathematically as:

$$ {\text{TRE}} = \sqrt {\frac{1}{4}\sum\limits_{{n = 1}}^{4} {\left| {Y_{n} - {}_{I}^{P} {\mathbf{T}} \cdot X_{n} } \right|^{2} } } $$
(11)
Fig. 6
figure 6

a Osteotomy scene. b Effects of planned and actual osteotomy. The target points in image and patient spaces are marked with red and blue, respectively

Full size image

where Xn represents the nth target point collected by the tracked probe in patient space and Yn represents the corresponding one placed interactively in image space. It should be clarified that the operation processes of collecting the target point coordinates were very carefully carried out, and each real target point coordinate was estimated by the mean of the corresponding measured ones in ten repeated measurements.

Effect of the scale factor

To investigate the impact of the scale factor L involved in the RAMSICP algorithm on the registration accuracy, FPs located in three different SA combinations (denoted as Combs. 1, 2 and 3) were selected to register with the IPC, respectively. The TRE changes corresponding to Combs. 1, 2 and 3 with different scale factors varying from 1 to 5 are shown in Fig. 7, which first decrease and then increase. Note that an optimal scale factor range Ropt corresponding to the minimum TRE can be reached in each case, but is different in different registrations.

Fig. 7
figure 7

The TRE changes with the scale factor

Full size image

However, it is very difficult to preset an optimal scale factor Lopt located in Ropt. To enhance the fault tolerance of the algorithm, in all subsequent experiments, the reference range (1 to 5) of L was discretized with an increment ∆L = 0.2 to calculate the TRE corresponding to each assigned L, and Lopt that yields the minimum TRE was picked out directly. It should be clearly stated that in this case, the longest calculation time is less than 10 s, far less than that of the whole operation process (about 8–15 min).

Reproducibility and robustness to different CR errors

  1. (1)

    Reproducibility. To evaluate the reproducibility of accuracy for the three aforementioned methods, fifteen prepared experimental subjects were registered by different operators, respectively. The corresponding results shown in Fig. 8 indicate that the mean TREs for CR, CR-ICP and CR-RAMSICP are 1.33 ± 0.27 mm, 0.89 ± 0.14 mm and 0.56 ± 0.08 mm, respectively. Obviously, CR-RAMSICP exhibits the lowest mean and standard deviation of the TREs and thus is more accurate and reproducible than CR and CR-ICP.

    Fig. 8
    figure 8

    Comparison of the TREs for different registration methods

    Full size image
  2. (2)

    Robustness. To investigate the robustness of CR-RAMSICP to CR errors, we artificially adjusted the placement deviations of the virtual fiducials to extend the CR error range which was limited within 1 to 5 mm and divided into eight levels (denoted as Levels 1 to 8) with a 0.5 mm interval. For each CR error level, five groups of experiments were performed, and the TREs for CR-RAMSICP and CR-ICP are shown in Fig. 9. The results indicate that the TREs for CR-ICP display a general increasing trend proportional to the CR errors, which are always higher than those for CR-RAMSICP; in contrast, the TREs for CR-RAMSICP are stable within 1 mm under the CR errors at Levels 1 to 6, but sharply increase and gradually tend to approach those for CR-ICP when the CR error is more than 4 mm. In general, compared with CR-ICP, CR-RAMSICP can enhance the robustness to poor CR errors to a certain extent.

    Fig. 9
    figure 9

    Comparison of the TREs for CR-RAMSICP and CR-ICP under different CR errors

    Full size image

Sensitivity to sampling with the tracked probe

During practical surgical intervention, sampling with the tracked probe is limited by the accessibility of exposed anatomic surfaces, and thus fewer points and more compact sample distribution are expected on the premise of ensuring the accuracy. Therefore, we compared the sensitivity of CR-ICP and CR-RAMSICP to the number and distribution of FPs collected by the tracked probe, respectively.

  1. (1)

    Sensitivity to the distribution of FPs collected by the tracked probe. The only five FPs located in each SA (see Fig. 4c) were separately extracted for registration, and the experiments were repeated 10 times by different operators. The TREs for different methods under different distributions of FPs collected by the tracked probe are shown in Fig. 10a. The results indicate that SA3, SA4 and SA5 show low TREs, which may be associated with the short distances from the measured target points; however, SA7 and SA8 exhibit high TREs due to the long distances from the target points. Generally, for each SA, the TREs for CR-RAMSICP exhibit lower values and smaller differences than those for CR-ICP.

    Fig. 10
    figure 10

    Comparison of the TREs for CR-RAMSICP and CR-ICP under different distributions and numbers of FPs collected by the tracked probe. a The case for different FP distributions. b The case for different FP numbers

    Full size image
  2. (2)

    Sensitivity to the number of FPs collected by the tracked probe. 1, 2, 3 and 4 FPs were selected from each SA, respectively; that is, a total of 8, 16, 24 and 32 FPs in patient space were used for registration, respectively. The experiments were repeated 10 times by different operators. The TREs for different methods under different numbers of FPs collected by the tracked probe are shown in Fig. 10b. The results illustrate that the TREs of CR-RAMSICP and CR-ICP decreases with the increase of the number of FPs; however, CR-RAMSICP shows lower TREs and sensitivity to the FP number. Besides, it can be deducted that CR-RAMSICP has the potentials to provide higher accuracy with fewer FPs because, for example, the TREs for CR-RAMSICP with 8 FPs are generally lower and more stable than those for CR-ICP with 32 FPs.

Conclusion

In this paper, a hybrid feature-based registration method CR-RAMSICP is proposed, which sufficiently meets the required accuracy of 1 mm without invasive bone-implanted markers. Four RRMs attached to the tracked rigid body which is rigidly fixed out of the bone with an extending bracket are used for CR, and their centers can be tracked automatically in patient space and placed interactively in image space. The proposed RAMSICP algorithm is adopted for fine registration to precisely match the PPC collected on the bone surfaces by the tracked probe and the IPC extracted from the corresponding CT datasets. It does not require the establishments of complex metrics and models except the point-to-point distance, which is convenient to program. A series of experiments based on isolated porcine iliums were performed to preliminarily verify the effectiveness of CR-RAMSICP in accuracy improvement. The results illustrate that CR-RAMSICP is not only more accurate and reproducible than CR and CR-ICP, but also exhibits better robustness to CR errors, and lower sensitivity to the distribution and number of FPs collected by the tracked probe than CR-ICP. Besides, it should be clarified that when collecting the PPC with the tracked probe, without loss of generality, we avoided the tendency to select rich features on the porcine ilium surfaces, but the relatively smooth areas. Therefore, from this point of view, CR-RAMSICP may provide the clinical potentials to be extended to other types of long bones.

In future work, we will study robust bone surface segmentation and the determination of optimal IPC resolution to further improve the registration accuracy. More practically, the human phantom experiments will be performed to further evaluate the applicability of CR-RAMSICP and provide relevant practical guidelines for clinical trials.