Introduction

Needle-based procedures are among the most common diagnostic or therapeutic gestures, from “simple” drug injection to cancer biopsy. Most often, the practical conditions or the difficulty to visualize the surgical zone and the anatomical structures, reduce the overall accuracy of the percutaneous action. In some cases, this accuracy may critically influence the success of the clinical procedure. In clinical practice, the needles must be rigid, and clinicians try to avoid needle bending during insertion, for instance by rotating the needle. Rarely, in specific cases, the clinician may apply forces on the needle base to bend it in order to reach areas usually inaccessible because of the presence of bones or critical structures. The development of medical robotics has led to the introduction of a novel approach based on automatic flexible needle steering in order to generate curved trajectories towards a target in presence of obstacles. Such trajectories can be produced by the application of forces on the needle base to bend it. This has a significant drawback since the required intensity increases with the insertion depth. The resulting stress on the tissue might create traumas. Another way of bending the needle is obtained by biological tissue manipulation and deformation but several clinical applications are not compatible with such an approach. A third steering approach—the one we selected—makes use of asymmetric bevel-tip needles. Forces applied by biological tissue reaction on the tip during insertion deflect the needle in the direction of the sharpened side of the bevel. Finally, alternative approaches consist in developing specific needles17 or other devices such as concentric tubes.12 More extensive citations can be found in review papers.26,33

In all cases, robotic needle steering requires real-time abilities to detect a needle and a potentially moving target from images, to plan needle trajectory and to determine robot’s inputs to reach that target.

Trajectory Planning

Earlier research projects used the application of forces and torques on the needle base in order to bend it. This required accurate modeling of the needle insertion and bending in living tissue: finite element models11 or virtual spring methods13 have been proposed. The latter reached real-time performances. More recently, the same ``base manipulation’’ approach using 2D ultrasound (US) as feedback was proposed.19

In Ref. 27, a beveled needle is modeled as a cantilever compliant beam which oscillation modes are estimated. Most often, the beveled needle deformation has been approximated using a non-holonomic model called unicycle model.34 This model is used for planning by several groups.5,22 To make it computationally efficient, a 3D algorithm based on a rapidly-exploring random tree (RRT) was introduced.38 RRT is well-adapted for fast re-planning from real-time information (electromagnetic feedback24 or 2D image feedback8). These last two works generate the robot control inputs using the duty-cycling approach for 2D36 and 3D37 automatic needle steering. Duty cycling allows controlling the curvature of the needle during insertion by alternating high-speed rotation and no-rotation periods with a specific duty-cycle. During rotation periods, the needle follows a straight line while it bends during insertion with no rotation. The resulting trajectory can be modeled by a circular path with a specific curvature. This method simplifies path planning algorithms by using arcs with different radii of curvature.

Needle Tracking

Needle localization in soft tissue is a challenging issue. It can be partly addressed using localizers which are commonly used during computer-assisted medical intervention. However, because they cannot give any information about needle deformation during insertion, Ref. 1 proposed to use mechanical strain sensors integrated on the needle shaft. Although promising, this requires complex calibration and/or the integration of sensors inside medical devices and it is not yet available for clinical routine.

Medical imaging modalities remain the most common way to detect a needle and a mobile target. X-ray imaging offers low visibility of soft tissues. It is also associated with risks of irradiation for patient and clinical staff. Magnetic resonance imaging (MRI) has good performance in visualizing soft tissues23 but it suffers from incompatibility with metallic objects, reduced working space and availability issues for interventions. Ultrasound is a safe, convenient and low-cost imaging modality. Recently, two groups2,3 developed needle steering systems using RRT re-planning and US feedback in ex vivo tissue. However, in both systems, the needle detection algorithms constrain the needle orientation relative to the ultrasound image plane.

US noise and artifacts make needle detection difficult. This has stimulated extensive research for the past years. Needle segmentation in 2D US images has been addressed using multiple Hough transforms.20 More recently, a new needle segmentation algorithm based on ultrasonic spectral analysis was proposed.6 The method benefits from the clinician’s hand tremor when holding the needle. Unfortunately, it does not work in real-time. Regarding 3D US images, the first challenge is to quickly process large and noisy volumes. 2D detections by Hough transform in planar projections of the volume may be combined for 3D localization.31 In Ref. 2, a motor translates the 2D US probe to detect and follow the needle tip during insertion. Other authors4 proposed to reconstruct the entire needle shaft using a frame-by-frame algorithm associated with an unscented Kalman filter and color-Doppler imaging. These last two algorithms both constrain the needle orientation with respect to the image acquisition plane, which may limit their clinical applicability. Finally, the Random Sample Consensus algorithm (RANSAC) was also introduced for direct 3D flexible needle detection with short computation time32 and applied to needle steering.9

In our previous work,18 we have associated a 3D RANSAC algorithm with a mechanical-based model to predict a region of interest and to filter the segmentation result. This method makes the needle detection more robust. This robust feedback is then used for needle steering using a brachytherapy robot prototype and rapid-replanning RRT approach. Preliminary testing has been conducted using paraffin gel phantoms and PVC phantoms.

The step from phantoms to ex vivo tissue is very challenging. First, ex vivo tissue is generally highly heterogeneous, leading to large differences between planned and actual paths. Moreover, needle segmentation is very difficult. This is due, among other reasons, to: miscellaneous tissue structures visible in ultrasound, ultrasound signal attenuation with depth, artifacts or needle orientation relative to the US propagation direction. Some existing methods allow enhancing needle visibility. For instance US waves can be oriented in a direction perpendicular to the needle to maximize the reflected signal and to increase needle visibility.10 This method is already embedded in commercial products of Ultrasonix, Sonosite or General Electric. However, this requires prior knowledge about needle orientation and it only works with straight rigid needles. Echogenic needle design has also been explored. Microstructures located on the surface of the needle have been used to diffuse the incoming US waves and to increase signal reflection back to the transducers. The microstructures are generally grooved along the needle shaft but could also be embedded in a polymer coating.7 References 16 and 21 describe the use of echogenic needles during ultrasound hand-guided nerve block interventions. To the best of our knowledge, none of the robotized needle steering systems used echogenic needles to achieve controlled insertion in biological tissue, and the impact of these modified needles on automatic needle detection has never been quantified.

Contributions

In this paper, we introduce a needle steering system based on a medical robotic prototype dedicated to transperineal needle insertion.14 We describe how this medical robot is controlled using a RRT-based rapid re-planning approach and a unicycle model (see Table 1). The aim is to obtain an advanced integrated system for flexible needle insertion. The approach is explained in “Materials and Methods” section. The feedback used for needle guidance is based on a needle tracking algorithm using a RANSAC algorithm combined with the computation of a dynamic region of interest (ROI) using a mechanical-based prediction model. This novelty allows determining the future position of the ROI from one US volume to another. We also introduce the use of echogenic needles for improving detection performances. Finally, we evaluate in “Results” section the full system on a significant number of experiments on phantoms and ex vivo biological tissue.

Table 1 Main existing flexible needle-steering systems and innovations.
Full size table

Materials and Methods

General Approach

The control of needle deformation during its insertion in a synthetic or biological tissue is based on rapid re-planning: the needle path is continuously updated from information about the needle and target positions obtained by processing 3D US images. This rapid re-planning control can compensate for disturbances, such as tissue deformation or patient motion as well as potential modeling errors or simplifications. It can provide high reliability despite imperfect modeling of biological tissue, needles and their interactions.

As shown in Fig. 1, the controller is divided in two modules. The high-level control is composed of needle segmentation, filtering, path planning and duty-cycling control (DCC). The low-level control is based on the PROSPER robot14 itself.

Figure 1
figure 1

Schematic of the needle steering approach based on rapid re-planning.

Full size image

Trajectory Planning and Duty-Cycling

Trajectory planning is based on combining the unicycle model and a 3D RRT algorithm, inspired by previous work.2,8,24 The developed method generates paths with at most two arcs. This is justified by the low curvature achievable in biological tissue and the narrow field of view offered by the US probe. Searching for simpler feasible paths reduces computation time without loss of generality.

Each time a US volume has been acquired, reconstructed and processed, the algorithm selects the best path among all the possible paths generated from the current position to the target. The optimal path has a minimum length and a maximum distance to the obstacles. The method returns three parameters (L, α, κ) representing the selected circular path A (see Fig. 2) to be executed by the robot for the next motion. L is the insertion length, between 0 and Lmax, the length of the arc that reaches the target. κ is the arc curvature and is bounded by κmax, the maximum curvature of the beveled-tip needle. α in [0, 360] is the angle defining the orientation of the plane in which the arc is included. These three parameters are then transformed in robot commands by the DCC algorithm. The duty-cycle DC corresponds to the rotation time of the needle divided by the insertion time. A simple equation allows computing DC from the curvature of the planned arc: DC = 1 − κ/κmax. DC is then converted into insertion and rotation time for PROSPER taking into account velocity limits when specified.

Figure 2
figure 2

Parametric description of generated arcs: insertion length L (a), curvature κ (b) and path angle α (c).

Full size image

Needle Detection and Tracking

As mentioned, RANSAC can rapidly detect a curved needle in 3D without constraining the needle orientation. Its input is a set of candidate voxels resulting from 3D image thresholding. The threshold was chosen in order to select a given percentage of brighter voxels in the region of interest, based on the needle size. For our experiments, 5% was the optimal choice. The candidate voxels contain not only the needle but also noise and artifacts. To determine the needle position, RANSAC randomly selects a set of voxels and fits a polynomial curve to them; then it tests all the voxels of the initial set in order to classify them as inliers (on the needle) or outliers, depending on their distance to the curve. The maximum acceptable distance for inliers is half the needle diameter as visible in an US image. In our set-up, the needle diameter in the image is about 3 mm. This process is repeated n times and RANSAC selects among the n curves, the one having the largest number of inliers. This algorithm has proved to be efficient when the image is good enough but it may fail when the signal coming from the needle is low. Thus, we proposed to increase the segmentation robustness in US images by defining a curvilinear region of interest (ROI)—see Fig. 3—around the needle prediction using Kalman filtering. Two simple models were tested for predicting next needle position: one kinematic model based on needle tip position and insertion velocity and one mechanical model describing needle-tissue interaction by virtual springs. Parameters of this model were selected from force measurements during insertions of needles in phantoms. These models and the parametrization, as well as first performance evaluation are described in Ref. 18.

Figure 3
figure 3

Schematic of the curvilinear 3D ROI selection using a side margin of 2 pixels and a tip margin of 1 pixel.

Full size image

Because needle visibility in biological tissue could results in much lower needle detection performance we also investigated the use of echogenic needles (see “Hardware and Software Environments” section).

Target Tracking

The target corresponds to a desired location computed by a planning system: for instance based on the dose planning for brachytherapy. In our needle steering experiments, the user manually defined the desired final location of the needle tip in the initial US volume. Because biological tissues are deformable and may move during the needle insertion, the target has to be tracked. The block-matching algorithm29 that we used consists in exhaustively searching for the target neighborhood of the previous image in a largest region of interest of the current image. This algorithm has been validated using ex vivo tissue (see “Hardware and Software Environments” section).

Hardware and Software Environments

US-Guided Brachytherapy Robot and Computing Environment

The experimental setup is composed of the PROSPER robot, an Ultrasonix RP ultrasound device and a 4DEC9-5/10 endorectal 3D US probe. The US voxel size is 0.4 × 0.4 × 0.4 mm. The US machine, running Windows, integrates an IntelCore 2(1.8 GHz) processor with 2Go RAM. Each frame of the 3D US volume is acquired using the Ultrasonix library and is sent to an external computer with a TCP/IP connection. This computer, which runs the application, is a Windows IntelCore i7 (3.4 GHz) with 16Go RAM and equipped with a NVidia Quadra 600 graphical card with 1 Go memory. The code written in C++ was developed in the open source framework CamiTK (Computer-assisted medical intervention ToolKit—cf. http://camitk.imag.fr/).

The external computer is responsible for 3D volume reconstruction, needle segmentation, path planning and robot control. 3D US acquisition and reconstruction lasts about 1 s. The segmentation takes between 300 and 700 ms (depending on the size of the ROI) and the planning time is around 150 ms. Thus, the acquisition time is longer than the segmentation and planning time and runs in parallel to them. The feedback frequency is therefore limited by the time required to acquire and compute each US volume. In the described experiments, the maximum needle insertion speed was set to 0.5 mm/s.

The PROSPER robot has seven motors organized in two independent modules. Five of them are dedicated to the needle positioning out of the patient. The two other motors control the insertion and the rotation of the needle inside the body (see Fig. 4). Previous work14 demonstrated the robot ability to position the needle tip on predefined physical targets (beads) in prostate deformable phantoms.15 Before entering the phantom, the needle was aligned with the physical target. The mean distance between the needle tip and the target was 2.98 mm, measured on more than 100 insertions. With straight needles, the robot could only compensate prostate motion and deformation in the insertion direction. Along this direction, the accuracy was in average 1.61 mm.

Figure 4
figure 4

Experimental environment for needle steering on ex vivo tissue. (a) Prosper general architecture, (b) set-up.

Full size image

Echogenic Steerable Needles

We used two different types of 30° beveled needles for steering experiments: nitinol needles (Ø 0.5 mm) and stainless steel biopsy needles (Ø 0.71 mm). Nitinol, a super-elastic Nickel/Titanium alloy, allows the needle to make highly curved paths. Biopsy needles are less deformable but correspond to clinically used needles. We have not quantified the achieved curvature since 3D steering with double bending makes it quite difficult. Nitinol needles with 0.5 mm diameter typically have a radius of curvature of about 500/600 mm—it depends on the elasticity of the medium in which the needle is inserted. Biopsy needles have a larger radius of curvature. As proposed in previous work,35 all needles used in this study were pre-bent 3 mm distally from the tip to artificially increase bending. This resulted in smaller radii of curvature: in some of the experiments we have observed locally a radius of curvature of 30–50 mm for nitinol and around 100 mm for steel needles.

The signal reflected from a needle to the US sensor is maximum when the main axis of the needle is perpendicular to the US wave propagation direction. To make needles more visible, modifying their surface quality provides scattering effects that may compensate for a reflected signal loss. CO2 laser etching makes it possible to groove the needle in order to obtain regularly spaced diffusing structures (see Fig. 5). The diffusion process requires a spatial frequency of the structure (d) smaller than the ultrasound wavelength λ. λ = ν/f where f and ν are the transducer frequency and the sound velocity in the medium. In our case, f is 7 MHz and a common estimate of the sound velocity in soft tissue is 1540 m/s. Thus, the spacing of the scattering structure must be less than 220 μ. We chose 150 μ, a value used for the EchoBright™ needle sold by Halyard.

Figure 5
figure 5

Schematic (a) and picture (b) of the Nitinol needle after fiber laser etching.

Full size image

Experiments

Detection and Tracking

The ground truth is given by manually fitting the needle model—a Bezier curve—to the images acquired during and after insertion. Visibility improvement of echogenic needles was quantified in 2D and 3D images with three different environments: water, paraffin phantom and ex vivo tissue (pork tenderloin). For each medium, a standard needle and an echogenic one with the same characteristics (diameter, material, etc.) were inserted. The insertion angle was the same for the two conditions (standard vs. etched) and did not influence needle visibility. The image threshold and RANSAC parameters (number of iterations, criterion to select inliers, etc.) were also similar. The RANSAC algorithm was applied 100 times for each needle. Since the choice of the ultrasound plane could influence the needle visibility in 2D, we applied the same process to 3D volume with both types of needles. In 2D and 3D cases, the result of each detection was classified as successful or unsuccessful depending on its distance to the ground truth. An error at the needle tip greater than 2 mm in position and 10° in orientation was declared unsuccessful.

Regarding needle tracking, in previous works,18 we compared in a single experience the kinematic and mechanical models for ROI prediction with Kalman filtering when this paper concerns 20 different image sequences of needle insertions (7 in ex vivo tissue, 13 in paraffin phantoms).

Target Tracking

The block matching was implemented on a GPU. It reached a run time of 150 ms, using a block size of 5 × 5 × 5 mm and a searching area of 10 × 10 × 10 mm. The method was validated on an ex vivo tissue (pork tenderloin) covered by paraffin gel. A target motion was simulated by moving the US probe with respect to the phantom. The US probe was attached to the PROSPER robot end-effector and a controlled displacement was performed describing the 15 mm edges of a planar square. During this movement, the probe head was always in contact with the paraffin gel. A visible target was selected in the image of the ex vivo tissue at the beginning of the experiment. The detected motion was compared to the controlled displacement.

Needle Steering

Three different environments were used to evaluate the ability to steer a flexible needle: homogeneous phantom, heterogeneous phantom and ex vivo tissue (pork tenderloin).

The heterogeneous phantom was composed of paraffin gel with Vybar™ polymer (from Baker Hughes). The polymer modifies the phantom elasticity. Several layers were created with different Vybar™ concentrations to mimic the brachytherapy environment including the skin, the perineal tissue and the prostate capsule. Figure 6 shows a schematic of the layers and US shear-wave elastography measurements with the Aixplorer™ from Supersonic Imagine.

Figure 6
figure 6

Experimental setup ready for needle steering on ex vivo tissue and elastography measurements.

Full size image

For these three types of environment, echogenic needles were inserted under several conditions regarding the presence of obstacles and the motion of the target.

For all experiments, a target position was defined manually by the user in the initial image. In order to show the potential of needle steering in compensating for unexpected inaccuracies and perturbations, the needle initial trajectory (before insertion) was given an artificial misalignment to the target. The definition and value of this initial misalignment are given in Table 2. The initial position of the needle was reached using the “positioning module” of the robot while insertion experiments were conducted using the two degrees of freedom of its “insertion module”.

Table 2 Initial misalignments (distance between the target and the initial needle direction—mean ± standard deviation [min; max], in mm).
Full size table

In the phantoms, block matching could not be used for target tracking because of a lack of background structures (due to phantom transparency to US). Thus, we applied a virtual displacement to this target, during the insertion, to simulate an additional perturbation as it could be detected with a real target. In the ex vivo tissue, both static and moving targets were tested. In the latter case, the tracking algorithm updated the target positions.

In both homogeneous and heterogeneous phantoms, the needle steering experiments were conducted with and without virtual obstacles. In contrast, needle steering was tested without obstacles in ex vivo tissue. Indeed, the observed natural radius of curvature of needles (around 150 mm) and the maximum insertion length allowed by PROSPER (60 mm) make obstacle avoidance very difficult in this case.

Obstacles, tested for phantoms, were spheres whose diameter ranged from 8 to 20 mm. Most of them were positioned in the first half of the space separating the initial tip position from the target.

Needle steering was also tested using echogenic biopsy needles in ex vivo tissue. Five insertions towards mobile targets were conducted without considering obstacles. The target was tracked using the block-matching method. Since the biopsy needle is more rigid than the nitinol one and has a larger radius of curvature, the introduced misalignments were smaller than for the nitinol needle experiments. These experiments are presented separately since the initial conditions are not similar to experiments with nitinol needles.

Results

Detection and Tracking

Figure 7 shows the US images corresponding to each medium with both echogenic and standard needles. The luminance profiles displayed for each needle clearly show that the luminance is higher for echogenic needles in each image. In addition, standard needles appear to create more artifacts resulting in a worse overall appearance.

Figure 7
figure 7

Ultrasound image of echogenic and standard needles and luminance profiles in water (a), paraffin phantom (b) and pork tenderloin (c). The blue and red dotted lines in the US images correspond respectively to the echogenic and standard needle detections. The other bright line in the images corresponds to the interface of the phantom with the plexiglas support.

Full size image

Returned results, in terms of successful detections, for water, paraffin phantom and pork tenderloins are 2, 1 and 6% respectively with standard needles against 100, 100 and 90% for echogenic ones. The detection rate is definitely better for the echogenic needle in each environment both in 2D and in 3D. This confirms the benefit of using echogenic needles.

Regarding the tracking by optimizing the ROI, using the kinematic model only reached 65% of success whereas the tracking of the needle using the mechanical model reached 100% of success. This shows that the mechanical model significantly increases the detection robustness in the 20 sequences.

Concerning target tracking, the error between the measured motion of the target in the image and the square trajectory of the probe was 0.11 ± 0.9 mm.

Nitinol Needle Steering Experiments

The reported positioning results correspond to the final distance separating the needle tip from the target. The results of all the experiments are summarized in Table 3. For each case, five insertions were performed. The mean, standard deviation and extreme values are given and discussed in the next sections.

Table 3 Accuracy of the nitinol needle tip position in different media with different conditions (five trials per medium and per condition − mean ± standard deviation [min;max], in mm).
Full size table

In the homogeneous phantom, the final positioning error between the needle tip and the target is 1.2 ± 1.1 mm. With one obstacle, the final error increases up to 2.4 ± 1.4 mm due to the high curvature of the path and the difficulty of reaching the target after obstacle avoidance. The results are very promising considering the US volume quality and its spatial resolution.

In the heterogeneous phantom, the perturbation generated by the heterogeneity of the material increases the positioning error up to 2.1 ± 1.1 mm without obstacle, which can also be considered as a good result taking into account ultrasound quality. With the presence of obstacles, this error increases up to 2.4 ± 1.8 mm, which is approximately the same as in homogeneous phantom. The results could be explained by the large perturbation caused by the obstacle that obviates the influence of phantom heterogeneity.

In the ex vivo biological tissue, the results are promising with an average error of 3.8 ± 1.7 mm with a static target and 4.4 ± 1.0 mm with a mobile target tracked using the block-matching algorithm. The measured motion of the target was 1.1 ± 0.5 [0.4; 1.6] mm. A typical experiment is illustrated in Fig. 8.

Figure 8
figure 8

Result of a needle guidance in ex vivo tissue. (a) XY projection of the final pose of the needle, (b) XZ projection of the final pose of the needle and (c) needle tip trajectory during insertion.

Full size image

In ex vivo experiments, the high heterogeneity of biological tissue greatly disturbs the needle steering and it results in increasing the positioning error. Moreover, an important phenomenon has been observed during ex vivo needle insertions: in some cases, unexpected changes of needle direction occurred, particularly when the needle rotates. These unexpected movements could be explained by the winding of nerves or muscle fibers around the needle that results in force application on the needle shaft. These disturbances greatly influence the final error in some cases.

Biopsy Needle Steering Experiments

Finally, similar experiments on ex vivo tissue with a tracked target and no obstacle were performed using biopsy needles. The measured motion of the target was 1.0 ± 0.9 [0.1; 2.4] mm. The tip position error for five steering experiments are 2.2 ± 0.6 [1.5; 2.8] mm.

All these experiments are demonstrated in the associated video.Footnote 1

Discussion

Steering experiments on ex vivo tissue are more challenging than phantom experiments due mainly to needle visibility limitation and to the complex structure of the environment. Using echogenic needles is a way of dealing with this visibility issue. In this first study, the echogenic needles had a very simple design and were grooved only on one side. The experiments have nevertheless shown that such echogenic needles offer improved visibility in several insertion environments. This even allowed guiding needle in ex vivo tissue with high robustness. IFuture works include the test of our needle steering system on living tissue for full validation.

However, it should be noted that, for approximately 15% of the ex vivo needle steering experiments, the grooved needles broke due to a large bending. In this case, the experiments had to be repeated with new needles. This phenomenon never occurred with standard needles used in previously published work nor during our experiments in phantoms with modified needles. We assume that our process of needle etching weakened the needle by creating cracks in the material. The winding of tissue around the needle, adding to this weakness, probably results in this amount of needle braking. In the future, mechanical tests should be performed on echogenic needles to evaluate changes in their mechanical properties. An optimized etching process should also be investigated in order to make the needle stronger while keeping flexibility and visual enhancement. It is also necessary to ensure via histopathology analysis that the grooves do not increase the damage to the tissue compared to a conventional needle.

We also observed that the rotation of the needle in ex vivo tissue sometimes induced unpredictable needle bending. Our hypothesis is that this bending is due to the winding of muscular or nervous fibers around the needle during rotation. This creates unpredictable external forces applied to the needle. In previous work,3 similar unexpected deviations were observed when the needle collided vessels in a beef liver. Reference 30 also mentions this issue and describes how its occurrence is detected using a force sensor located on the needle base. This observation, coupled with the potential increase of tissue damage caused by needle rotation, provides another argument against the use of continuous rotation to guide needle insertion into biological tissue. In the near future, our control must be optimized28 to reduce the needle rotation, for instance by replacing it with alternating rotations (clockwise and counterclockwise). We believe that this improvement could greatly increase the steering precision of our system by avoiding unpredictable deformations. We also believe that it will reduce the potential damage done to the tissue due to needle rotation. Torsional models25 could also be investigated for increased accuracy.

The direct comparison to previous similar experiments (US, ex vivo) is made difficult due to the different experimental conditions and way of reporting. Reference 2 used targets in ex vivo chicken tissue embedded in a gelatin phantom. Experiments include obstacle avoidance. The insertion distance is in average much longer that ours since it ranges between 86 and 103 mm; let us remind that we are limited by the size of the 3D US volume. Reported accuracy is 1.82 ± 0.58 mm. Since a very significant part of the trajectory is in the gelatin phantom, this accuracy has probably to be compared to our heterogeneous phantom experiments. In Ref. 3, experiments conducted in beef liver result in an average error of 1.57 mm for six insertions. As reported by the authors, many other trials resulted in unexpected deviations of the needle generating much larger errors. These “failures” are not included in the statistics whilst our reported statistics include all experiments except when the needle broke. It should also be underlined that, in the two previously cited papers, the reported experiments do not include any initial misalignments where we introduce significant initial errors. Finally, we can compare our performance to linear insertion of a biopsy needle initially aligned with the target using the Prosper robot.14 In the heterogeneous phantom, the average accuracy was 2.98 mm compared to 2.2 mm, measured here, when steering the biopsy needle in ex vivo tissue (see “Biopsy Needle Steering Experiments” section) with a significant initial misalignment (see Table 2). Therefore, the new method demonstrates its ability to reach a target by steering a needle despite large perturbations simulated by the initial target misalignment.

We developed a needle steering system based on 3D ultrasound images. The use of echogenic needle greatly improves the detection rate of our needle detection algorithm. This improvement allows guiding a needle in biological tissue in closed-loop control using a RRT based rapid re-planning. This development, integrated with a prostate brachytherapy robot prototype, is a significant step toward clinical application of robotic needle steering. Even under the presence of unpredicted disturbances, probably due to tissue winding around the needle, our statistics on needle tip positioning error are promising. This work shows that needle steering can be an asset for many needle-involved robot-assisted surgical interventions. However, significant work certainly still remains to prove the non-invasiveness of the approach in living biological tissue.