1 Introduction

In 2016, the American Cancer Society estimates the numbers of new cancer cases and deaths that will occur in the USA, with the survey findings showing that death rates are increasing for cancers of the liver, pancreas, and uterine corpus, and cancer is now the leading cause of death in 21 states [90]. Thus, accelerating progress against cancer requires technology development not only in the medical aspect but also in engineering science, which is a pretty important assistance in medicine.

In recent years, the magnetic resonance (MR) images and computerized tomography (CT) images are widely used in clinical tests and treatments due to the higher resolution of the medical images. Ultrasound images also have good quality in real-time navigation. Thus, image guidance has been a popular technique used in percutaneous surgery, in which image guidance can give feedback of the tip pose. Nowadays, experimental researches are using the technique of image guidance to get well-fitted data with the clinical trials. In [59], we have a figure which describes different kinds of the tip pose estimation. Orthogonal cameras and 2D and 3D ultrasound images were usually used to estimate the tip pose. Researchers can develop a platform of software, which performs the image filtering, segmentation, and construction automatically and then estimates the tip position and orientation on real time. Actually, some electromagnetic position sensors and fiber Bragg grating sensors were embedded into the flexible needle for the measurement of the tip pose or bending deflections. However, electromagnetic position sensors will sometimes interfere with the signal, which results in a higher position error. Also, needles are usually needed to be redesigned to put the fiber Bragg grating sensors inside the needle shaft. Thus, it is more convenient to get the tip pose and needle deflection by image guidance. In addition, before the experiments, we need to get the coordinate of the target and obstacles to define the position of the target and obstacles. If the obstacles and target are embedded in gelatin or tissue, we cannot get their accurate position without the image scanning of the mockup. With the scanned image, the shape of the target and obstacles can be redrawn and coordinates can be calculated by image segmentation and reconstruction.

However, medical staffs are inconvenient to enter the imaging system and perform the operation in real time, which is affected by the structure, radiation, and field intensity of the imaging device. With the urgent therapy demands of the neoplastic disease and the therapeutic experience accumulation of the medical robots [22, 23], the image-guided robot-assisted surgical system has been increasingly attracting extensive emphasis and recognition in the application of biopsy or brachytherapy [9, 42, 55, 103, 104]. In image-guided robot-assisted percutaneous surgery, long and thin flexible needles are inserted into soft tissues which have large deformation and nonlinear characteristics in order to reach the target region not accessible using traditional techniques. The research focuses on the security and accuracy of the percutaneous surgery, because once the vessels, nerves, or other vital organs are damaged during the insertion, it will result in serious complications [37, 82].

With the development of medical imaging technology, surgical robotics, and the technology of computer science, surgical needles have been widely used in many minimally invasive surgeries, including but not limited to image-guided biopsy, image-guided brachytherapy, and tumor ablation. The development of tracking technology, registration algorithm, and 3D reconstruction as well as the software design and interaction [32] all made these surgical applications more accurate and popular. The success rate of the surgeries has been increased to a large extent. Patients and physicians as well as the medical staffs that participated in the surgeries will not suffer the damage of the radiation from the imaging machine. Here, what we have to demonstrate is that even though these surgical applications have been progressing very well, some disadvantages may not be neglected.

Image-guided biopsy is the primary aspect in diagnostic pathology. In the percutaneous surgery, 14G–20G biopsy needles are used to extract partial nidus tissue or rotary-cut in order to allow surgeons to prepare a pathological observation and make a clinical diagnosis. However, the puncture may fail and patients may suffer serious damage when the biopsy is processed in some tiny lesions with a diameter less than 2 cm, which is adhered in the abdomen-thorax, due to the influence of complex anatomical structures, natural breathing, or the cardiac pulse during tissue extraction. The success rate of the single needle puncture is merely 70% because of the target deviation caused by tiny errors of the puncture angle, human factors, imaging limitations, and dynamic tissue reactions including soft tissue deformations and the sliding of multi-layer structures, with a higher false-negative rate than false-positive rate [58], which will delay the therapies of patients.

Even though the diagnostic biopsy is inerrant and treatment planning is opportune enough, the curative effect is significantly affected by the targeting accuracy. For instance, brachytherapy is a branch of inner radiotherapy, which implants the radioactive seeds into the tumor tissue or lymphatic drainage region by using 18G needles for percutaneous surgery through the image-guided technique and treatment planning system. Nevertheless, owing to the needle deflection and tissue deformation, the accurate positions and intervals of seeds cannot be guaranteed. The targeting error of seed implantation is a key problem of the brachytherapy improvement. Therefore, the focus to solve the problem is to enhance the targeting accuracy for biopsy and therapy, which can be realized by analyzing the needle-tissue interactive mechanism, planning the needle trajectory, and steering the needle access.

The needle-tissue interactive mechanism includes the force modeling of needle insertion [4], force data to calculate the tissue deformation and needle deflection [95] during insertion, and needle-tissue interactions for computer-based surgical simulation [71]. Needle steering contains the needle design [94], kinematic modeling of surgical robots [99], online path planning, and steering control [33, 43] of the flexible needle, which are demonstrated in the previous reviews where the specific research aspect for needle-tissue insertion or needle steering is discussed perfectly.

This review is focusing on the relationship of the mechanical interactions and steering control; through the comprehensive effect of these two aspects, targeting accuracy of the minimally invasive surgery will be improved more or less. To our best knowledge, the steering control is most executed utilizing the kinematic modeling, with proper translations and rotations of the needle base with respect to the insertion point, except for a few literatures written by Misra et al. [68], Abayazid et al. [2], Glozman et al. [44], and Neubach et al. [74] reporting that the steering control can be performed by the mechanics study of the needle-tissue interactions. Without mechanical foundation, nonlinear tissue deformation would not be properly incorporated in the kinematic model and path replanning which now only supposes the target moves constantly [97]. In addition, the research of computer-based surgical simulation is not contributing in the mostly used steering control algorithms.

Through the needle-tissue interaction research, the interaction force and deformation results are input to the online path planning and needle steering. Combining the image-guided technique, it will result in avoiding the vital tissues, reducing the tissue damage [93], and reaching the target. A schematic diagram of the image-guided robot-assisted surgery is shown in Fig. 1. When a needle is inserted percutaneously, haptic feedback is required to enhance the clinical operation, which contains the force modeling and surgical simulation research. Visual feedback demonstrating the needle deflection and tissue deformation results in path planning and steering control. The models obtained from visual and haptic data can be used for intraoperative path planning and trajectory generation.

Fig. 1
figure 1

Schematic diagram of the image-guided robot-assisted surgery; part 1 shows the image processing in the treatment planning system, while part 2 displays the intraoperative surgery including the research of the needle-tissue interactive mechanism and steering control

Full size image

This review is organized as follows. The first section is an overview of the force modeling of the needle-tissue interactions. In the second section, surgical simulation of the needle-tissue interactive mechanism is summarized and analyzed by retrospecting the literatures. In the third section, the path planning of the flexible needle is proposed and followed with the modeling of needle steering and application in the closed-loop system. The existing techniques are discussed, and limitations and recommendations are proposed in the end of each section. Finally, conclusions and future work are presented.

2 Force modeling of the needle-tissue interactions

Force modeling of the needle-tissue interactions can offer the clinical relevance and guide the trajectory planning and control strategy for percutaneous puncture surgery, as it can identify the insertion position [56] and detect the tissue characteristics [107] according to force feedback. The force modeling and model optimization using influence factors are shown as follows.

2.1 Modeling of the insertion forces

Okamura et al. [75] separated the force data into components from different sources. We repeat the needle insertion into the phantom, and the sample force versus position data (Fig. 2) is acquired, which reveals one primary puncture and subsequent internal punctures. The phantom used in our experiments is a transparent polyvinyl alcohol (PVA) hydrogel with the composition of 3 g PVA, 17 g de-ionized water, 80 g dimethyl sulfoxide, and five freeze/thaw cycles, which was authenticated to have a similar microstructure, a similar Young’s modulus to porcine liver tissue, and the same deformation property as prostate tissue [53]. The first rupture event is designated by a peak in stiffness force after a steady rise, shown in the deformation phase. Subsequent variations in force are due to friction and cutting forces, which are demonstrated in the insertion phase. The last rupture happened at the puncture event of another side of the phantom, which only results in friction force. In the extraction phase, friction force occupies the main force part.

Fig. 2
figure 2

Needle insertion forces versus position data during insertion into and extraction from the phantom, including the deformation phase, rupture event and the insertion phase, as well as the extraction phase

Full size image

Okamura et al. [75] validated their force model by data collection from bovine livers using a one degree of freedom (DOF) robot equipped with a load cell and needle attachment. In their results, there are significant variations in liver geometry generally and the internal structure is complex, both of which may make a perfect match impossible. Nevertheless, results still show that the overall shape of the force model is similar to the measured data. The force model developed by Okamura et al. [75] and Simone et al. [91] included in three parts listed as the following: (1) capsule stiffness, produced in the phase of large deformation; (2) friction, which always appears in the insertion phase; and (3) cutting, the force that cut the front tissue. Capsule stiffness is modeled as a nonlinear spring model, friction force is constructed as a modified Karnopp model, while cutting force is viewed as a constant for a given tissue. The force data collected was a summation of stiffness, friction, and cutting force

$$ {f}_{\mathrm{needle}}(x)={f}_{\mathrm{stiffness}}(x)+{f}_{\mathrm{friction}}(x)+{f}_{\mathrm{cutting}}(x). $$
(1)

But in this method, collisions with small interior structures such as blood vessels were not included in the model. These may add additional peaks and stiffness forces to the data.

On the basis of the three-part force model, Carra et al. [29] modified the stiffness force using a Hunt-Crossley model, which stated the contact model varied nonlinearly without taking tissue motion into consideration. Friction force was proposed using a Dahl model with the feature of capturing presliding displacement. The model also allowed describing friction in the low-velocity regimes. Li et al. [61] developed a contact model (Fig. 3a) that calculated the stiffness force in multi-layer tissue, which was resolved from a systematic use of Hankel transforms and the theory of Sneddon dual integral equation. Stiffness force was shown as

$$ {f}_{\mathrm{stiffness}}=\frac{2}{\pi }{E}_{\mathrm{r}}\tan \left(\alpha \right){h}^2, $$
(2)

where α is the radius of the contact circle and h is the insertion depth of the soft tissue. Er represents the reduced modulus of the soft tissue and flexible needle. The friction force was modeled as a modified Winkler’s foundation with a linear stiffness coefficient (Fig. 3b); thus, the normal force Fn along the needle shaft due to tissue deformation can be expressed as Fn = khΔ, where h and Δ are parameters that stand for the contact length and settling amount, respectively; k is the foundation modulus of an elastic beam developed by Biot [26]. The friction force was viewed as Coulomb friction, with the expression of ffriction = μFn, in which μ is the friction coefficient between needle and soft tissue. An example of μ is selected as 0.015; the simulated stiffness force and friction force are shown in Fig. 4.

Fig. 3
figure 3

Stiffness force and friction force of the needle-tissue interaction biomechanical model. a Contact model prior to the rupture event. b Friction force using modified Winkler foundation after the rupture event [61]

Full size image
Fig. 4
figure 4

Simulated stiffness force, friction force, and cutting force of the surgical needles

Full size image

Li et al. [61] and Carra et al. [29] were focused on the research of multi-layer force modeling. Suppose that surgical needles are inserted through the skin, the fat, and the muscles, then the complete force model was not only a summation of the stiffness force, friction force, and the cutting force. Young’s modulus and the Poisson ratio of the skin, fat, and muscle should be considered in each corresponding phase. When the needle passed through the skin layer, the new insertion force at the current time should be the summation of the friction force in the skin layer and the stiffness force in the fat layer. When a new rupture appeared in the fat layer, the new insertion force should be the summation of the friction force in the skin layer and the fat layer, as well as the cutting force in the fat layer (Fig.  3 a). They validated their force model by data collection from the PVA hydrogels using a 1-DOF robot fitted with a 6-DOF force/torque (F/T) sensor. Results show that the predicted stiffness force fit the measured data well, with a standard deviation of 0.1355 N in the experiments. But in this method, the inhomogenous characteristic was not considered.

Asadian et al. [18] modeled the force using nonlinear dynamics based on a modified LuGre model that captured all stages of needle-tissue interaction including puncture, cutting, and friction forces. In their later research, they modified the model using multiple Kalman filters to characterize the total contact force, through which they made the system highly adaptable for capturing the force evolution during a percutaneous needle access in standard operating conditions [17].

During percutaneous puncture procedures, transitions between tissue layers often lead to rupture events that involve large forces and tissue deformations also produce uncontrollable crack extensions [62]. The interactive force will suffer a sharp decline in the moment of tissue fracture. However, fracture mechanics was not considered in the modeling of the three-part force. Therefore, researchers in the flowing study developed a fracture model which is prior to the friction force and after the stiffness force. Mahvash et al. [62] has proposed a fracture model using the energy-based J integral method retrieved from fracture mechanics, in which rupture events were viewed as sudden crack extensions. The sudden crack extension is caused by the exceedance of the release rate J of strain energy concentrated at the tip of the crack than the fracture toughness of the material. Fracture toughness is to characterize the material ability to prevent crack propagation, which is a quantitative measure of the toughness of the material. Fracture toughness was also investigated by many research teams [19, 45]. In the phase of crack extension during rupture, dx = 0, it yielded the needle force necessary to initiate rupture as \( {f}_{\mathrm{n}}\propto \sqrt[m]{\frac{R}{K_{\mathrm{c}}}}{A}_{\mathrm{c}} \), where Kc is the crack energy-intensification factor, Ac is the size of the contact area, R represents the fracture toughness of the soft material, and m depends on the nonlinearity of the material. In the phase of crack extension during cutting, the shape of the crack extension dA was idealized as rectangular prior to deformation, with wc as the width of the crack (Fig. 5). Then, the needle force for cutting was derived as fn = Rwc. In the phase of force drop during rupture (point 1 to point 2 in Fig. 2), the needle force at the end of rupture is derived as

$$ {f}_{\mathrm{n}2}={f}_{\mathrm{n}1}/\left(\sqrt[m]{\frac{R_1{K}_{\mathrm{c}2}}{R_2{K}_{\mathrm{c}1}}}\frac{A_{\mathrm{c}1}}{A_{\mathrm{c}2}}\right), $$
(3)

in which Ac1 and Ac2 are contact areas between the needle and tissue.

Fig. 5
figure 5

Rupture and the previous step of a needle insertion; in this idealization, a single microcrack close to the needle tip and of initial area A extends to produce an increase in area of dA [62]

Full size image

2.2 Model optimization using the influence factors

It is obvious that the interaction force is influenced by numerous variable parameters, which are divided into three categories: needle geometries including needle diameter [108], needle tip type [30, 96], or the bevel angle [96, 108]; insertion methods containing insertion velocity [76, 96], drive mode [65, 80], or rotation [8]; and tissue characteristics involving different tissues [65] such as skin [47, 49], muscle, fat, vessel, nerve [89, 108], organ capsule [51, 95], or ex vivo and in vivo tissue [64].

These factors have significant influence on the interaction force, which were investigated in Van Gerwen et al.’s [95] review in detail. We will not repeat them. Methods of force modeling and comparison among these references in terms of analysis of the influence factors are listed in Table 1. However, in the listed literatures, parameter investigation was often performed after the force modeling or implemented directly in experimental validation. Sometimes, the investigated factors even do not exist in the force modeling research. So far, the experimental study of the influence factors has not been used in the theoretical evolution. Here, comprehensive statistics of the investigated influence factors are done by analyzing the literature research and the factors are verified to be considered and regarded as optimization parameters to construct more accurate needle-tissue interaction force models in the future. The recommendatory technique rote is shown in Fig. 6.

Table 1 Methods and comparison of force modeling of the needle-tissue interactions
Full size table
Fig. 6
figure 6

Force modeling and statistical analyses of the influence factors that constitute the recommendatory technique rote in the following research of modeling of insertion forces, partly of the figure was redrawn from [51, 62, 75]

Full size image

3 Surgical simulations

The interactive virtual needle insertion simulation is an effective manner to investigate the needle deflection and tissue deformation during the percutaneous surgery [31, 36, 38]. Through the research on the deformation law, efficacious compensation measures can be performed to accomplish the real-time compensation, which result in reducing the targeting errors and improving the targeting accuracy [48]. Research on the needle deflection and tissue deformation is shown as follows.

3.1 Needle deflection

Commercial surgical needles used in percutaneous surgery are usually flexible and beveled-tip needles with an asymmetrical tip. The asymmetry of the bevel tip produces a resultant transverse load which causes a flexible needle to naturally bend when it is inserted through a soft medium. This phenomenon is not observed in needles with symmetric tips [68]. Thus, this research is a transcendental point to predict the needle deflection when it is inserted into the soft tissues. Researchers utilized different methods to investigate the needle deflection; some developed theoretical models based on the mechanical interaction or the energy mode [46, 87], and others measured the deflection through experiments using sensors [49, 57].

In the needle deflection part, because the simple assumption does not fully match the real deflection of the surgical needles, Kataoka et al. [54] suggest that an additional DOF, a moment or a rotational force, should be acted on the needle. In the following research, Abolhassani et al. [5] added the moment in their needle deflection model and then they minimized the needle deflection [6]. They confirmed their model by inserting a beveled-tip needle into animal tissue. The needle deflection at the target was reduced by about 90%. In addition, the minimization of needle deflection can reduce the tissue deformation. Afterwards, Misra et al. [68] proposed the energy-based method to predict the needle deflection. They validated the energy-based formulations through a series of experiments with different needle bevel angles, diameters, and gelatins. Simulation results matched well with those observed in experimental studies, which indicate that the energy-based method is an effectual technique to predict the needle deflection. Models are shown as follows:

Figure 7 shows a force-deflection model of a needle on penetration, which was proposed by Kataoka et al. [54] who described that a beveled-tip needle goes forward while compressing the tissue around the tip; then, the needle is deflected (with a deflection of g(l, d)) due to the uneven stress on each side caused by the asymmetric needle tip. In the model, l is the axial position of the needle whose origin is at the needle tip without deflection and d is the diameter of the needle. Here, the predicted deflection of the beveled-tip needle was calculated through the integral of the infinitesimal resistance force per length, which was obtained through the differential of the transverse force at the fixed end of the needle Fend, that is, w(l, d) = dFend/dlm, where lin is the insertion depth of the flexible needle. Fend can be measured by a three-axis load cell. Thus, the needle deflection g(1, d) caused by w(l, d) can be described by solving the following equation:

$$ \frac{d^2\left(g\left(l,d\right)\right)}{dl^2}=\frac{1}{EI}\int \frac{d\kern0.5em {F}_{\mathrm{end}}}{d\kern0.5em {l}_{\mathrm{in}}}\kern0.5em l\kern0.5em dl, $$
(4)

in which E is Young’s modulus and I is the moment of inertia of the needle.

Fig. 7
figure 7

A force-deflection model of a needle on penetration [54]

Full size image

Figure 8 shows the needle deflection model developed by Abolhassani et al. [6], in which the flexible needle was considered as a flexible prismatic beam with clamped support at the base. Then, the relationship between the needle tip deflection and forces as well as moments acting on the needle was constructed. Abolhassani et al. [6] simplified the bending moment equation by neglecting the effect of the maximum intensity of the triangularly distributed resistance force, q0. Thus, the simplified actual amount of deflection at the needle tip can be calculated using

$$ v=\frac{1}{6 EI}\cdot \sum \limits_{i=0}^n\left(3\varDelta {M}_{r_i}{L}^2-\varDelta {F}_{r_i}{L}^3\right), $$
(5)

where n is the number of steps to reach a certain depth inside the tissue. L is the total needle length. Parameters \( {M}_{r_i} \) and \( {F}_{r_i} \) are measured at the base of the needle using a 6-DOF force/torque sensor. \( \varDelta {M}_{r_i} \) and \( \varDelta {F}_{r_i} \) are changes in the amount of moment and force values between two consecutive steps. i is the current step.

Fig. 8
figure 8

Needle deflection model with bending moment M at a cross section [6]

Full size image

Misra et al. [68, 69] proposed an energy-based model that described the deflection of a beveled-tip needle inserted in an elastic medium, in which the tissue-specific parameters such as rupture toughness, nonlinear material elasticity, and interaction stiffness, as well as needle geometric and material properties, are incorporated. The model was favorable for the virtual needle-tissue insertion simulation. They used the Rayleigh-Ritz method in which the minimum of a potential defined by the sum of the total energy and work done by the system were calculated. Figure 9 [68] shows their needle-tissue interaction model with two increments and two sub-steps each. In increment i, sub-step 1, the needle is initially at configuration as. Then, needle goes from configuration as to bs due to its interaction with the surrounding medium, the applied input force, and the geometric tip constraint. Sub-step 2 captures the rupture of the tissue due to tip forces and calculates the resulting deflection of the needle. At the end of sub-step 2, the configuration is updated as cs. The resultant reaction force due to the presence of the roller boundary constraint in the global configuration in increments i is R i . In increment i + 1, sub-step 1, the needle is initially at configuration cs, which is the end of the increment i, sub-step 2. It deflects to configuration es at the end of increment i + 1, sub-step 2. Ri + 1 shown in Fig. 9 is the resultant reaction forces in increments i + 1. The needle deflects to configurations bs and ds in increments i and i + 1, respectively, due to the presence of these reaction forces and the insertion force, Pinput. β is the cut angle and ϕ i is the angle between the local and global tip configurations. The functional form of the needle transverse deflection, v i , and axial deflection, u i , are initially assumed for sub-steps 1 and 2, which are shown as

$$ {v}_{is}=\left\{\begin{array}{ll}{a}_{0s}+{a}_{1s}x+{a}_{2s}{x}^2+{a}_{3s}{x}^3& 0<x<{l}_i/2\\ {}{b}_{0s}+{b}_{1s}x+{b}_{2s}{x}^2+{b}_{3s}{x}^3& {l}_i/2<x<{l}_i\end{array},\kern0.62em {u}_{is}={c}_{0s}+{c}_{1s}x,\right. $$
(6)

where s stands for sub-steps 1 and 2 and i is the current increment. x is the insertion depth and l i is the needle length, which are all shown in Fig. 9. a ks and b ks (k = 0, 1, 2, 3) and c js (j = 0, 1) are coefficients of the deflections, which can be evaluated by minimizing the system potential at each sub-step.

Fig. 9
figure 9

Needle-tissue interaction model shown for two increments with two sub-steps each [68]

Full size image

Roesthuis et al. [86] expanded the mechanics-based model to predict needle deflection for a needle undergoing multiple bends during insertion into soft tissue, which has lower errors in tip deflection than the kinematic-based model [100]. The needle was modeled as a cantilever beam and divided into n elements, each described by their own shape function. Needle rotation was performed at x r , and tip force acted on the needle to make it deflect in the opposite direction. The double bend needle shape was modeled by fixing the part of the needle before rotation with a series of springs.

In the experiment area, fiber Bragg grating sensors are usually used to estimate the needle shape and deflection. Park et al. [77] developed a MRI-compatible biopsy needle instrumented with optical fiber Bragg gratings for measuring bending deflections of the needle. The prototype design with a modified inner stylet incorporated with the optical fibers is shown in Fig. 10. The inner stylet had three lengthwise grooves, and each groove held an optical fiber with two fiber Bragg grating sensors. Using this method, needle deflections were displayed in real time with greater band width and accuracy than that when viewing in MR images. Roesthuis et al. [85] also fabricated a prototype of a flexible nitinol needle (φ 1.0 mm and length 172 mm) integrated with an array of 12 fiber Bragg grating sensors to measure the axial strain and calculate the needle curvature. Then, the three-dimensional needle shape was reconstructed from the curvature data.

Fig. 10
figure 10

Prototype design with modified inner stylet incorporated with three optical fibers. Three identical grooves at 120° intervals are made on the inner stylet to embed fiber Bragg grating sensors along the needle length. a Midpoint cross section. b Magnified view of an actual groove. c Tip of the stylet. d Fixture design for electrical discharge machining (EDM) parallel grooves in the biopsy needle stylet [77]

Full size image

3.2 Tissue deformation

It is necessary to determine the biomechanical properties of the soft medium and investigate the constitutive laws for modeling the soft tissue and predicting the tissue deformation [7]. The material parameters associated with the hyperelastic model are usually experimentally derived using tensile, compression, shear, or biaxial tests [72].

Extensive studies have been conducted on simulations of tissue deformations that occur during needle insertion in soft tissue. Misra et al. [72] measured the rupture toughness and nonlinear material elasticity parameters of several soft tissues. These physical parameters were incorporated into a finite element model that included both contact and cohesive zone models to simulate tissue cleavage. In their research, uniaxial compression experimental data were measured and fit the Neo-Hookean model well. Subsequently, they found that the organ geometry and boundary constraints surrounding the organ were the most important factors influencing the deformation [70]. They used the Mooney-Rivlin model to investigate the constitutive laws and construct the hyperelastic model of the soft tissue. Mousavi et al. [73] also considered the organ shape and developed the FE model using a mapping function, which was suitable for real-time estimation of tissue deformation of specific organs.

Except for the hyperelastic model, Ahn et al. [13] developed a 3D viscoelastic model using the force-time experimental data. The relaxation function G(t) was a scalar function of time and expressed using the Prony series. Rigidity modulus k i was added in the model; then, the expression of the Prony series can be shown as

$$ {\displaystyle \begin{array}{c}G(t)={G}_0\left(1-{\overline{g}}_1^p\left(1-{e}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\tau}_1$}\right.}\right)\right)+{\overline{g}}_2^p\left(1-{e}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\tau}_2$}\right.}\right)\\ {}={k}_0+{k}_1{e}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\tau}_1$}\right.}+{k}_2{e}^{-\raisebox{1ex}{$1$}\!\left/ \!\raisebox{-1ex}{${\tau}_2$}\right.}\end{array}}, $$
(7)

and Prony series parameters were obtained as

$$ {\overline{g}}_i^p=\frac{k_i}{k_0+{k}_1+{k}_2},\left(i=1,2\right). $$
(8)

Basafa et al. [21] then developed an improved real-time simulation model of the nonlinear viscoelastic deformations due to a surgical indenter. The mechanical realization conventional mass-spring damper models were improved using nonlinear springs and nodal dampers; the high computational efficiency was maintained using an adapted implicit integration algorithm. Mahvash et al. [62] also used a nonlinear viscoelastic Kelvin model to predict the relationship between the tissue deformation and the rupture force.

4 Steering control

4.1 Path planning of the flexible needles

Nowadays, during operation in percutaneous surgery, even the most experienced surgical teams may still experience difficulties in finding the optimal port placement for each intervention [28]. The main reasons behind this difficulty are the variability of anatomical structures between patients and the large quantity of information to be considered (patient traits, interventional variations, operational settings, etc.) as well as the deformation among the soft tissues [12]. In percutaneous surgery, a key issue is to decide how to control the flexible needle to reach a desired target while avoiding vital tissues. Therefore, path planning of the flexible needle should be investigated to offer effective control category when the automatic needle insertion is acted on the soft tissue.

Currently existing path planning algorithms of the flexible needle are mainly divided into three categories: numerical method, inverse solution method and search method [109], which are shown in Fig. 11.

Fig. 11
figure 11

Path planning algorithms containing the numerical method, inverse kinematics method, and search method, with the examples, disadvantage, and applicability of each algorithm [14, 16, 37, 39, 40, 52, 78, 80, 84, 105]

Full size image

The numerical method is divided into the probability density function method and the objective function method in detail. Park et al. [78] firstly generated the probability density of the reachable region by considering the Gaussian white noise. They then used a diffusion-based motion planning algorithm to numerically compute a path. Alterovitz et al. [14, 15] developed a motion planner computing optimal steering actions that can maximize the probability of reaching the desired target, with the capability of considering uncertainty in the insertion due to individual differences. Suppose there are N discrete states; the motion planning problem was to determine the optimal action u i for each state x i , i = 1, …, N. Then, the optimal action can be expressed as

$$ {p}_s\left({x}_i\right)=\max \left\{\sum \limits_{j=1}^N{P}_{ij}\left({u}_i\right){p}_s\left({x}_j\right)\right\}, $$
(9)

where P ij (u i ) is the probability of entering state x j after executing action u i at current state x i .

Using the objective function method, Alterovitz et al. [16] also developed a motion planner combining soft tissue modeling and numerical optimization to generate an optimal trajectory plan for the surgical needles, in which the simulated tissue deformations were compensated, polygonal obstacles were locally avoided, and the distance of needle insertion was minimized. The initial given parameters are the location, orientation, bevel rotation, and insertion distance. Using a similar method, instead of discretizing the configuration space, Duindam et al. [40] discretized the control space, such that the needle trajectory could be expressed analytically without approximate numerical simulation, which found a locally optimal trajectory in a 3D environment with obstacles with just a few seconds of computational time. Dehghan et al. [34, 35] developed a gradient-free and fast-convergence optimization method based on iterative simulations, through which the optimal insertion point and orientation were calculated. Even though the numerical method is accurate in calculation, sometimes it is not adaptable in the complicated environment.

Duindam et al. [39] presented a constant-time motion planning algorithm for steerable needles based on explicit geometric inverse kinematics. They split the problem into two parts: first, the needle was turned and inserted such that its instantaneous line of motion intersects the line describing the goal position and direction. Second, the remaining planner problem was solved by utilizing the 2D inverse kinematics. However, sometimes the inverse algorithm cannot guarantee solvability.

The search method of the flexible needle mainly includes the artificial potential field method, the roadmap method, and the rapidly exploring random tree method. DiMaio et al. [36, 37] constructed a path planning technique in a geometrical environment based on an artificial potential field by establishing a comprehensive potential field, in which the obstacles generated repulsive field Urep while the target generated an attractive field Uatt, which were shown as follows:

$$ {\displaystyle \begin{array}{l}{U}_{\mathrm{att}}(p)=\frac{1}{2}{k}_{\mathrm{att}}{\rho}_{\mathrm{att}}^2(p)\\ {}{U}_{\mathrm{rep}}(p)=\left\{\begin{array}{cc}\frac{1}{2}\eta {\left(\frac{1}{\rho_{\mathrm{rep}}(p)}-\frac{1}{\rho_0}\right)}^2& \mathrm{if}\;{\rho}_{\mathrm{rep}}(p)\le {\rho}_0\\ {}0& \mathrm{if}\;{\rho}_{\mathrm{rep}}(p)>{\rho}_0\end{array}\kern0.5em \left({\rho}_{\mathrm{rep}}(p)\ne 0\right)\right.,\end{array}} $$
(10)

where katt and η are positive scaling factors. ρatt and ρrep are the distances between the needle tip and the target or the obstacles. Jiang et al. [52] and Li et al. [60] modified the artificial potential field and solved the local minimum problem where the flexible needle would vibrate continuously due to the local zero potential point. So far, the artificial potential field was used off-line, while the road map method and the rapidly exploring random tree can be used online and accomplish path replanning. Reed et al. [84] developed a stochastic motion roadmap-based planner and generated an optimal path. The planning system guides steerable needles to a desired target without touching the obstacles around. It is worth mentioning that a torsion compensator in the system controlled the needle tip orientation about the axis of the needle shaft. The rapidly exploring random tree is mostly used in the path planning nowadays, owing to the high efficiency and real-time planning. Given a set of targets, Xu et al. [105] proposed an algorithm to quickly explore the configuration space by building a forest of rapidly exploring random trees and to find feasible plans for multiple steerable needles from a single entry region. The final optimal plan among all feasible outputs was selected by comparing the plan with minimal twists and the plan with minimal insertion region. Patil et al. [80] achieved orders of magnitude speedup compared to other approaches using the similar method based on the rapidly exploring random tree.

Extensive calculation and numerical optimization are performed in the numerical method, which results in a higher targeting accuracy. However, because of the great deal of computation, this method will not be adaptable in some complicated environments, especially in a clinical experiment which contains many kinds of irregular obstacles. It will be more applicable in some geometric environments which have some obstacles with regular shapes. The inverse kinematic method sometimes cannot guarantee the solvability. Also, it will be applicable in geometric environments. The search method is a relatively faster algorithm which can search several feasible paths simultaneously. But it cannot guarantee the optimal path. Some optimization should be performed to select the best path for the needles. The search method can be applicable in some complex environments which contain obstacles with irregular shapes. Thus, it is appropriate to be used in clinical experiments. Also, because of the relatively faster speed, the search method can achieve replanning simultaneously which can be used in online planning.

However, nowadays, the planning methods are mostly used in geometrical environments and static environments. Even if the search method is applicable in clinical environments, most of the researches are still focused on the geometric environments. We suggest that researchers investigate the path planning of flexible needles when they are inserted into an anatomical environment [59, 88, 92] and a deformable environment [98] due to the tissue deformation. Recommendatory research of the path planning in the future research can be seen in Fig. 12.

Fig. 12
figure 12

The future research focus that we suggest, including the deformable environment and anatomical environment [14, 80, 88, 92, 98]

Full size image

4.2 Needle steering

An accurate model of needle steering is an essential first step toward closed-loop needle control to compensate for targeting and entry-angle error, as well as tissue deformation [100]. Using the mechanics theory, Misra et al. [68], Abayazid et al. [2], Glozman et al. [44], and Neubach et al. [74] developed several models to offer the needle steering, which was shown in the introduction part.

Whereas, the mostly used model was a nonholonomic kinematic model (Fig. 13a) proposed by Webster et al. [100], which generalized the standard 3 DOF nonholonomic unicycle and bicycle models to 6 DOF using the Lie group theory. The bicycle model which had a constant front wheel angle ϕ and wheel base l1 was modeled as a kinematic steering for the beveled-tip needle. Frame A was viewed as a stationary world frame, and fames B and C were regarded as the front and back wheels of the bicycle. The control inputs were assumed as u = (u1, u2); n(t) = [x(t), y(t), z(t)]T was the tip position of the flexible needle in x-, y-, and z-directions at time t. It was shown as

$$ n(t)={R}_{ab}(t){l}_2{e}_3+{p}_{ab}(t), $$
(11)

where l2 is the location along the bicycle that is attached to the needle tip n; the unit vectors e1, e2, and e3 ∈ ℝ3 represent x-, y-, and z-axis unit vectors, respectively; and R ab and p ab represent the 3D rotation and translation components of the homogeneous transformation matrix, \( {g}_{ab}(t)={g}_{ab}(0){\exp}^{\left({u}_1{\hat{V}}_1+{u}_2{\hat{V}}_2\right)t}, \) which expresses the rigid transformation between A and B. Vector V1 and V2 ∈ ℝ6 denote the twist coordinates of the pure needle insertion and shaft rotation.

Fig. 13
figure 13

Kinematic model with the different types of duty-cycled spinning, a configuration of model parameters during insertion of a flexible beveled-tip needle [100], b continuous insertion with unidirectional rotation, continuous insertion with bidirectional rotation, duty-cycled flipping with insertion stops during rotation, respectively [64]

Full size image

Minhas et al. [67] incorporated duty-cycled spinning into Webster’s kinematic model during needle insertion, which provided proportional control of the curvature of the needle trajectory through tissue. In order to incorporate a duty cycle of D = trot/T into the kinematic model, where trot was the time of rotation and T was the duty cycle period, the inputs were adjusted as

$$ {u}_1(t)=v,{u}_2(t)=\left\{\begin{array}{cc}w,& jT\le t<T\left(j+D\right),\kern0.5em j=0,1,2\dots \\ {}0,& else\end{array}\right., $$
(12)

where w is the rotation speed in the rotation period and v is the insertion speed. The rotation speed must be greater than the insertion speed, ω >  > υ , to prevent helical needle trajectories, though it was possible to exploit these trajectories for needle control [64].

Then, Engh et al. [41] and Minhas et al. [66] performed the experimental validation during needle insertion into brain tissues. Majewicz et al. [64] proposed a different type of duty-cycled spinning including continuous insertion with unidirectional rotation, continuous insertion with bidirectional rotation, and duty-cycled flipping with insertion stops during rotation, respectively (see Fig. 13b). When unidirectional rotation was used, the insertion velocity input u and rotation velocity input w were defined for each iteration, i ∈ (0, k) , as follows:

$$ {u}_{\mathrm{i}}(t)= ds/{t}_{\mathrm{rev}},{w}_i(t)=\left\{\begin{array}{l}2\pi /{t}_{\mathrm{rev}\kern0.72em }\kern0.5em iT\ge t<T\left(i+D\right),\kern0.5em i=0,2,\dots \\ {}\begin{array}{ccc}-2\pi /{t}_{\mathrm{rev}}& iT\ge t<T\left(i+D\right),& i=1,3,\dots \end{array}\\ {}\begin{array}{cc}0,\kern1.8em & else\end{array}\end{array}\right., $$
(13)

where ds is the sub-steps of size and trev is the rotation period. When duty-cycled flipping was used, the duty cycle factor D was defined in terms of the time the needle was inserted in its flipped orientation, Tflipped, to its unflipped orientation, Tunflipped.

$$ D=1-\frac{T_{\mathrm{flip}}-{T}_{\mathrm{unflip}}}{T_{\mathrm{flip}}+{T}_{\mathrm{unflip}}}. $$
(14)

Wood et al. [101, 102] proposed a tracking control law of needle steering via duty-cycled rotation in 2D and 3D based on the Stanley method. Then, the modified curvature, k, of the flexible needle after correction of the heading error θ e and cross-track error, ect, was expressed as

$$ k={k}_2\tan \left({\theta}_e+{\tan}^{-1}\frac{k_1{e}_{ct}}{ds}\right), $$
(15)

where k1 and k2 are tuning parameters.

In the application, Bernardes et al. [24], Swaney et al. [93], Abayazid et al. [1, 3], Vrooijink et al. [97], Adebar et al. [10, 11], and Patil et al. [79] performed the experimental validation in different environments and achieved lower targeting error using the duty-cycled spinning. The specific conditions and results are shown in Fig. 14.

Fig. 14
figure 14

Experimental validation using duty-cycled spinning in environments with different obstacles [3, 10, 24, 79, 93, 97]

Full size image

5 Discussion and conclusion

In this paper, we have provided an overview of the needle-tissue interactive mechanism and steering control in image-guided robot-assisted minimally invasive surgery. As can be seen from this review, we have investigated the force modeling of needle-tissue interactions with the analyses of the influence factors, virtual surgical simulation which predicts and simulates the needle deflection and tissue deformation, path planning of the flexible needles, and the accurate steering control. The techniques for interaction and steering are described, and limitations as well as recommendations for future research are discussed.

Force modeling of the needle-tissue interactions is concentrated on the stiffness force, fracture force, friction force, and cutting force. Analyses of the influence factors are carried on three categories: needle geometry, tissue characteristics, and insertion method. It was verified that these factors have significant influence on the interaction force. However, in the current research model, a parameter investigation was often performed after the force modeling or implemented directly in experimental validation without force modeling. In some literatures, the investigated factors even do not exist in the force modeling research. But comprehensive analyses of the influence factors can provide exhaustive consideration for the force modeling or the model optimization. So far, the experimental study of the influence factors has not been used in the theoretical evolution. We recommend that factors should be considered and regarded as optimization parameters to construct more accurate needle-tissue interaction force models in the future.

Surgical simulation of the needle-tissue interactions is consisting of the prediction of needle deflection and modeling of the constitutive laws of soft tissues. In most of the researches, the hyperelastic model or the viscoelastic model is applied in the condition of supposing the soft tissues are continuous, incompressible, homogeneous, and isotropic materials. However, for the nonlinear heterogeneous soft tissue [25] or the multi-layer tissues [83], there are only few studies about the modeling and simulation. We think the research teams should focus on the modeling and simulation of the heterogeneous soft tissues in the future, as it is closest to the actual situation when a needle is inserted into human soft tissues during percutaneous surgery. In addition, prestress was verified as an optimal biomechanical parameter for needle penetration, which minimized skin deflection and allowed for needle actuation to the targeted penetration depths with minimum variability [27]. Thus, various influence factors [106] should also be investigated to analyze the effect to needle deflection and tissue deformation. Through the parameter optimization and adjustment, the efficiency of the surgical simulator will be improved. In the virtual surgical simulation, needle deflection and tissue deformation have been investigated, while not much research were concentrated on the compensation measures to reduce the targeting errors. We suggest that researchers combine the compensation in the surgical simulator and improve the accuracy as well as the efficiency.

Nowadays, the planning methods are widely applied in a geometrical environment and a static environment. Due to the high efficiency of the search method, it is applicable in clinical environments or even can be used in real-time replanning when the clinical environment changes such as tissue deformation appears. Here, we suggest researchers may need to focus on investigating the path planning of flexible needles when they insert into an anatomical environment and a deformable environment which has the characteristic of tissue deformation. Needle steering control is now increasingly used in anatomical environments or deformable environments, which are the same as the research trend of the path planning. In the deformable environments, target motion and obstacle motion caused by the interaction force of the flexible needles and soft tissue during the needle steering are considered by researchers nowadays. However, the motion was only supposed as a uniform motion with a constant velocity [97]. According to the surgical simulation in Section 3, we know that tissue deformation was usually presenting nonlinear characteristics. Thus, it will be improper to suppose the target and obstacles are moving constantly. We suggest that when needle steering is controlled in a closed-loop system, the haptic feedback should be used. In addition, the needle-tissue interactive mechanism should be incorporated in the kinematic model, which will offer real-time tissue deformation with nonlinear characteristics. Nonholonomic modeling combined with duty-cycled spinning for needle steering is validated by several researchers in the closed-loop steering control systems, which tracks the tip position in real time and compensates for the deviation error. This algorithm is also recommended to be applied in the anatomical and deformable environments with nonlinear tissue deformation in future research.