1 Introduction

Virtual reality based surgery simulation is expected to provide benefits in many aspects of surgical procedure training and evaluation. To this end, a significant amount of research efforts have been dedicated to simulating the behaviours of soft tissues. These approaches can be divided into two classes. One class of investigations is focused on real-time simulation such as mass-spring models [8, 41] and spline surfaces used in deformation [6, 36]. The advantage of these methods is that the computation is less time consuming and the algorithm is easier to implement. However, these methods do not allow accurate modelling of material properties, and more importantly, increasing the number of springs leads to a stiffer system. The other class of studies focuses on deformation simulation using techniques such as finite element method (FEM) [4, 12] and boundary element method (BEM) [19, 27]. In FEM or BEM, rigorous mathematical analysis based on continuum mechanics is applied to accurately model the mechanical behaviours of soft tissues. However, these methods are computationally expensive and are typically simulated off-line, and only the equilibrium can be modelled without transient states. The pre-calculation [19], matrix condensation [7], the space and time adaptive level-of-detail [15] and the explicit finite element [13] techniques are used to enhance the computational performance.

In general, most of the existing methods for simulation of soft tissue deformation are fully built on an elastic model that is an ideal tool to describe the deformation. However, the behaviours of human tissues and organs are extremely non-linear [17, 21]. The common deformation methods, such as mass-spring, FEM and BEM, are mainly based on linear elastic models because of the simplicity of linear elastic models, and also because linear elastic models allow reduced runtime computations. However, linear elastic models cannot accommodate large-range geometric deformations which may allow displacements to be more than 10% of the deformable tissue size [17, 25]. Although few methods based on the non-linear elastic model can handle the large-range deformations [33], the use of quadric strains generally requires a very expensive computation for real-time simulation. The runtime assembly of all the force terms for every element limits the interactivity to only a few hundred elements. In addition, only geometric non-linearity is modelled other than non-linear material properties, extra work often needs to be performed for anisotropic deformations [32, 33]. The survey on soft tissue deformation can be found in [26].

This paper presents a new methodology for soft tissue simulation by drawing an analogy between autowaves and soft tissue deformation. The deformation is formulated as the propagation of autowaves. The potential energy stored due to a deformation caused by an external force is calculated and treated as the wave energy injected into the system, as described by the first law of thermodynamics. The potential energy generated by the external force is propagated among mass points by non-linear autowaves. An improved autowave model is developed to propagate the energy generated by the external force in the natural manner of autowaves. A cellular neural network (CNN) to solve the autowave model is established to satisfy the real-time requirement of deformations. A wave flux based method to derive the internal forces from the potential energy distribution is also presented. A haptic virtual reality system has been developed to mimic soft tissue deformation with force feedback. Examples are presented to demonstrate the efficacy of the proposed methodology. The methodology proposed in this study can not only deal with large-range deformations, but also can accommodate isotropic, anisotropic and inhomogeneous materials easily through simply modifying diffusion coefficients.

Autowaves have been successfully used in image processing [24, 31] and trajectory generation and control of robot motions [1, 2]. However, there are no investigations that combine autowaves with deformation simulation. To the best of our knowledge, this study is the first to directly use autowave techniques to simulate soft tissue deformation under externally applied loads. The contribution of this paper is that non-linear material properties, such as creeping, hysteresis and non-linear load-deformation relationship, are modelled with non-linear autowaves other than geometric non-linearity such as non-linear strain, and autowave techniques are established to naturally propagate the energy generated by the external force and further to extrapolate the internal forces from the natural energy distribution for deformations.

The remainder of this paper first introduces the concept of autowaves, analyzes the physical analogy between autowaves and soft tissue deformation, and describes the design of the autowave model in Sect. 2. The construction of the autowave model is then discussed in Sect. 3, including the source formulation and the CNN formulation of the autowave model. The discussion shifts in Sect. 4 to the internal force derivation, the weight functions for obtaining the optimum internal forces and the issue of model dynamics. Implementations, verifications, results and discussions of deformation simulation are then provided in Sect. 5. Finally, Sect. 6 concludes this paper with a brief description of future research work and improvements.

2 Autowaves

2.1 Originality

Autowaves represent a particular class of non-linear waves, which spread in an excitable medium. The term “autowave” was first coined by Khokhlov [18] to refer to autonomous waves. Autowaves are characteristic of strongly non-linear active media. They are self-sustained signals that induce local release of stored energy in an active medium, which is spent to trigger the same process in adjacent regions. The fundamental difference of autowaves from classical waves is that autowaves are propagated at the expense of energy stored in an excitable medium [14]. Typical examples of autowaves include combustion waves, waves of phase transitions, concentration waves in chemical reactions, and many biological autowave processes such as propagation of nerve impulses and excitation waves in heart muscle. These examples stress the importance of the autowave phenomena.

The activity of autowaves in an excitable medium is described by a reaction-diffusion type partial differential equation [28]:

$$ \nabla \bullet (D\nabla U) + q = \frac{{\partial U}}{{\partial t}}, $$
(1)

where ∇ • is the divergence operator, ∇ is the gradient operator, U is the potential of the observed point at time t, D is the diffusion coefficient at the observed point, and q is the source, which describes the active local reaction in small volumes of the medium. Therefore, the reaction-diffusion equation can be described as the rate of the change of the potential is induced by the kinetics of local reaction plus the potential diffusion from adjacent regions.

For isotropic materials, Eq. 1 becomes:

$$ D\nabla ^{2} U + q = \frac{{\partial U}}{{\partial t}}, $$
(2)

where \(\nabla ^{2} = ({{\partial ^{2}}}/{{\partial v^{2}}}) + ({{\partial ^{2}}}{{\partial w^{2}}})\) is Laplace operator that encodes each node relative to its neighbourhood.

The wave flux \(\vec{J}\) is the negative gradient of potential:

$$ \vec{J} = - D\nabla U.$$
(3)

2.2 Physical analogy

Excitable media that consist of locally connected active elements capable of forming pulses are very frequent in the nature, especially in biological tissues such as nerve fibers, cardiac muscle, and brain tissues, etc. The most intriguing property of excitable media is the existence of autowaves, i.e. non-linear spatiotemporal structures propagating through the medium. The non-linear physical process of autowaves provides a way to model the non-linearity of biologically soft tissues.

In accordance with Huygens’ principle of propagation, autowaves have the nature of wave propagation and provide a natural manner for energy propagation. The propagation of autowaves only depends on the inherent property of a medium, i.e. the diffusion coefficient, under a given external environment. The diffusion coefficient determines the energy distribution in the medium. It is similar to the elastic constants of elastic materials, which describe the physical behaviours of elastic materials. Therefore, the property of the diffusion coefficient can be used to deal with isotropic, anisotropic and inhomogeneous deformations. It must be emphasized that this paper addresses deformation modelling by using the diffusion coefficient, rather than elastic material constants.

Autowaves do not reflect from either the medium boundaries or inhomogeneous media. Therefore, autowaves do not generate oscillations at boundaries, and can accommodate inhomogeneous materials.

The behaviour of autowaves can be dramatically controlled by wave fluxes. The wave fluxes are similar to internal forces since the deformation behaviours of soft tissues are controlled by internal forces under a given external environment.

Soft tissue deformation is actually a process of energy propagation. When a soft tissue is deformed by an external force, the potential energy of the soft tissue is changed. The potential energy is distributed among mass points of the soft tissue. The potential energy should be zero when the soft tissue is in its natural state, and the energy should grow larger as the soft tissue gets increasingly deformed away from its natural state [39]. According to the law of conservation of energy, the change in potential energy is due to the work done by the external force [34]. Therefore, the deformation process can be treated as a process of (i) applying the energy generated by an external force to the soft tissue and (ii) propagating the energy among mass points of the soft tissue at the expense of the energy stored among the mass points as the potential energy of the soft tissue. The process of soft tissue deformation is very similar to the energy propagation process of autowaves, in which autowave energy is generated by a source and propagated in an excitable medium at the expense of the energy stored in the medium.

In the proposed autowave analogy, the deformation of soft tissues by an external force is treated as an autowave process with a source at the contact point. The surface of soft tissues is treated as an excitable medium. The energy generated by the external force is treated as the wave source. As a result of the propagation of autowaves, a potential field is developed on the tissue surface. The wave flux that originates from the source always flows from the point with a high potential energy to the point with a low potential energy. Therefore, such a wave field with a source, wave fluxes and flow of wave fluxes can be seen as a communication medium among an external force, internal forces and deformation.

To solve Eq. 1 or 2, it is necessary to determine the boundary conditions. The boundary conditions determine the character for autowave propagation over the boundary. In the case of soft tissue simulation, the boundary can be treated as the part of the soft tissue that connects/contacts with other tissues/objects. For simplicity, we choose Dirichlet boundary condition, i.e. the specified potential on the boundary Ω :

$$ U = U_{\Omega}\quad \hbox{on}\;\Omega, $$
(4)

where U Ω is the given boundary potential function.

For the sake of convenient description, the soft tissue refers to an isotropic soft tissue unless we declare it as an anisotropic soft tissue.

3 Autowave model

3.1 Source formulation

When a soft tissue is deformed under an external force, there is a displacement experienced. The deformation is the consequence of the work done by the external force. According to the first law of thermodynamics, the work done by the external force can be transformed into the equivalent wave energy injected into the system. For the convenience of describing the “work”, we assume that the external force does not change while the displacement is taken place. Therefore, the source q is:

$$ q = \frac{W}{{V_{F}}} = \frac{{\vec{F} \bullet \vec{d}}}{{V_{F}}}, $$
(5)

where • represents the dot product, W is the work done by an external force \(\vec{F}\) causing a displacement \(\vec{d},\) and V F is the volume on which the external force is applied.

Since only the surface of soft tissues is considered in the deformation formulation, the volume degrades as an area with a unit thickness. Therefore, Eq. 5 becomes:

$$ q = \frac{{\vec{F} \bullet \vec{d}}}{{A_{F}}}, $$
(6)

where A F is the area on which the external force is applied.

If the external force is applied to a point, or the area that the force is applied on is very small, the source q may be regarded as the elastic strain energy density at the contact point:

$$ q = \sigma \varepsilon = {\sum\limits_{i = 1}^3 {{\sum\limits_{j = 1}^3 {\sigma _{{ij}}}}}}\varepsilon _{{ij}}, $$
(7)

where σ is the stress tensor and ɛ is the strain tensor at the contact point.

The strain tensor in Eq. 7 may be a linear or non-linear strain tensor. The commonly used and simple strain tensor is the linear Cauchy strain tensor [11] described by:

$$ \varepsilon = \frac{1}{2}\left(\frac{{\partial u_{i}}}{{\partial x_{j}}} + \frac{{\partial u_{j}}}{{\partial x_{i}}}\right). $$
(8)

A straight forward approach that leads to a linear relationship between these two tensors is provided by Hooke’s law:

$$ \sigma = {\mathbf{C}}\varepsilon, $$
(9)

where C is a tensor approximating the constitutive law of a material. For isotropic materials, C has only two independent coefficients, i.e. Young’s Modulus and Poisson’s Ratio.

From Eq. 9, the displacement caused by the external force can be deduced as:

$$ \varepsilon = {\mathbf{C}}^{{- 1}} \sigma. $$
(10)

Subsequently, the source q can be described as a non-linear function of a stress:

$$ q = \sigma {\mathbf{C}}^{{- 1}} \sigma. $$
(11)

For traditional autowaves, the source at each point has the same value. For the purpose of soft tissue deformation, the energy generated by the external force is treated as a source and propagated to other mass points along the tissue surface. Therefore, the obtained source is set only at the contact point of the external force, and the source values at other mass points are set to zero, i.e.

$$ q = \left\{{\begin{array}{*{20}c} {{\sigma {\mathbf{C}}^{{- 1}} \sigma}}\quad \hbox{at the contact point}\\{0}\quad \;\hbox{at other points}\\ \end{array}}\right.. $$
(12)

3.2 CNN modelling of autowave model

The autowave model can be easily discretized on the object surface by using a finite difference scheme [20] or a finite volume method [40]. The discretized autowave equations are usually solved by numerical methods, such as refined Gaussian elimination and Gauss–Seidel iteration. The drawback of the numerical methods is that the numerical solution may develop oscillations unless certain precautions are taken [5]. Most importantly, the computing time of the numerical methods is expensive due to the iterative solution process. Therefore, the numerical methods are not suitable to solve the autowave propagation equations for real-time soft tissue deformation.

Comparing to numerical solutions, a CNN offers an incomparable speed advantage due to the collective and simultaneous activity of all cells [35]. The CNN is an analogue dynamic processor array, made of cells, which are simple circuits containing linear capacitors, linear resistors, and linear/non-linear current sources. One significant feature of a CNN is that the individual cells are non-linear dynamical systems, but that the coupling between them, i.e. the local connectivity of cells is linear [38]. The feature makes CNNs very suitable for modelling non-linear physical processes, such as solving non-linear partial differential equations (PDE), since CNNs conserve the physical properties of a continuous structure.

Both PDE and CNNs are time-continuous dynamic, and they share the same property that their dynamic behaviours depend on their local interactions. The inherent analogies between the CNN dynamic equation and the autowave equation have been made evident in the CNN literature [10, 23, 35, 38]. Furthermore, it has been proven that CNNs represent an excellent approximation to non-linear PDE including the autowave equation, and many important PDE with an appropriate finite-difference approximation can be efficiently solved by CNNs [10, 23, 35, 38].

The CNN model can be applied to different grid types. Without loss of generality, we consider the CNN on a rectangular grid with M rows and N columns. Each node on the grid is occupied by a cell. The dynamics of the CNN model is described by the following equations and conditions [9]:

$$ C\frac{{{\rm d}v_{{xij}} (t)}}{{{\rm d} t}} = - \frac{1}{{R_{x}}}v_{{xij}} (t) + {\sum\limits_{(k,l) \in N_{r} (i,j)} {A(i,j;k,l)v_{{ykl}} (t)}} + {\sum\limits_{(k,l) \in N_{r} (i,j)} {B(i,j;k,l)v_{{ukl}} +}}I $$
(13a)
$$ v_{{yij}} (t) = \frac{1}{2}\left(|v_{{xij}} (t) + K| - |v_{{xij}} (t) - K|\right), \quad{{K\,\geqslant\,1}} $$
(13b)
$$ |v_{{xij}} (0)|\,\leqslant\,K, \quad |v_{{uij}} |\,\leqslant\,K $$
(13c)
$$ N_{r} (i,j) = \{(a,b)|\max \{|a - i|,|b - j|\}\,\leqslant\,r,\begin{array}{*{20}c} \\ \end{array} 1\,\leqslant\,a\,\leqslant\,M,\begin{array}{*{20}c}\\ \end{array} 1\,\leqslant\,b\,\leqslant\,N\}\quad (1\leqslant i \leqslant M;\begin{array}{*{20}c} \\ \end{array} 1\,\leqslant\,j\,\leqslant\,N),$$
(13d)

where (i, j) refers to the cell associated with the node under consideration, (k, l) to a cell in the neighborhood of cell (i, j), namely N r (i, j), within a radius r of cell (i, j) (r  = 1 in our case). C is the capacitance of a linear capacitor. Since the capacitor is a scalar coefficient, we can set C  = 1 without the loss of generality. R x is the resistance of a linear resistor, and I is the current of the independent linear/non-linear current source. A is the feedback template and B is the control template, whose values depend only on the relative positions of cells (i, j) and cells (k, l). v uij (t), v xij (t), and v yij (t) denote the input, state and output of cell (i, j) at the time t, respectively. v yij (t) is a non-linear sigmoid function of v xij (t), and it is bounded by a constant K, which is equal to or greater than one.

Without inputs, Eq. 13a becomes an autonomous CNN:

$$ \frac{{{\rm d}v_{{xij}} (t)}}{{{\rm d}t}} = - \frac{1}{{R_{x}}}v_{{xij}} (t) + {\sum\limits_{(k,l) \in N_{r} (i,j)} {A(i,j;k,l)v_{{ykl}} (t)}} + I. $$
(14)

The CNN templates which define the CNN model are obtained from the discrete autowave model. For example, consider the discrete autowave model at the point \(\vec{P}_{{i,j}}\) shown in Fig. 1:

$$ \frac{{2DU_{{i + 1,j}} (t)}}{{\Delta u_{i} (\Delta u_{{i - 1}} + \Delta u_{i})}} + \frac{{2DU_{{i - 1,j}} (t)}}{{\Delta u_{{i - 1}} (\Delta u_{{i - 1}} + \Delta u_{i})}} + \frac{{2DU_{{i,j + 1}} (t)}}{{\Delta w_{j} (\Delta w_{{j - 1}} + \Delta w_{j})}} + \frac{{2DU_{{i,j - 1}} (t)}}{{\Delta w_{{j - 1}} (\Delta w_{{j - 1}} + \Delta w_{j})}} - \frac{{2DU_{{i,j}} (t)}}{{\Delta u_{{i - 1}} \Delta u_{i}}} - \frac{{2DU_{{i,j}} (t)}}{{\Delta w_{{j - 1}} \Delta w_{j}}} + q = \frac{{\partial U_{{i,j}} (t)}}{{\partial t}}, $$
(15)
$$ \Delta u_{{i - 1}} = {\left\| {{{\overrightarrow{P_{i - 1,j} P_{i,j}}}}} \right\|}\quad\Delta u_{i} = {\left\| {{{\overrightarrow{P_{i,j} P_{i + 1,j}}}}} \right\|}\quad\Delta w_{{j - 1}} = {\left\| {{{\overrightarrow{P_{i,j - 1} P_{i,j}}}}} \right\|}\quad\Delta w_{j} = {\left\| {{{\overrightarrow{P_{i,j} P_{i,j + 1}}}}} \right\|}, $$

where U i,j (t) is the potential of point \(\vec{P}_{{i,j}}\) at time t, and \({\left\| {{{\overrightarrow{P_{i - 1,j} P_{i,j}}}}} \right\|}\) and other similar terms represent the magnitudes of vector \({{{\overrightarrow{P_{i - 1,j} P_{i,j}}}}}\) and other similar vectors.

Fig. 1
figure 1

Discretization of the autowave model on a rectangular grid

Full size image

The analogy between the autonomous CNN given by Eq. 14 and the discretized autowave model given by Eq. 15 can be established by associating the state of a CNN cell v xij (t) with the potential U i,j (t), and the CNN templates can be further obtained as Eq. 16. The output of a cell can be further obtained from the non-linear sigmoid function Eq. 13b. Therefore, the time evolution of the cell array’s transient directly provides the solution to the autowave equation.

$$ A = {\left({\begin{array}{*{20}c}&I=q&\\ {0}& {{\frac{{2D}}{{\Delta w_{j} (\Delta w_{{j - 1}} + \Delta w_{j})}}}}& {0} \\ {{\frac{{2D}}{{\Delta u_{{i - 1}} (\Delta u_{{i - 1}} + \Delta u_{i})}}}}& {{\frac{1}{{R_{x}}} - \frac{{2D}}{{\Delta u_{{i - 1}} \Delta u_{i}}} - \frac{{2D}}{{\Delta w_{{j - 1}} \Delta w_{j}}}}}& {{\frac{{2D}}{{\Delta u_{i} (\Delta u_{{i - 1}} + \Delta u_{i})}}}} \\ {0}& {{\frac{{2D}}{{\Delta w_{{j - 1}} (\Delta w_{{j - 1}} + \Delta w_{j})}}}}& {0} \\ \end{array}} \right)}. $$
(16)

The initial value and the boundary condition of the autowave equation are also incorporated in the CNN model. The initial potential value of the autowave equation is directly associated with the initial state of the CNN. The Dirichlet boundary condition is realized by using fixed-state cells.

4 Internal forces and model dynamics

4.1 Derivation of internal forces

The internal force is described by wave flux, which is the negative gradient of potential with respect to the change in position, as described below:

$$ \vec{f} = - D\nabla _{{\vec{P}_{i}}} U, $$
(17)

where \(\vec{P}_{i}\) is a point with position \((\begin{array}{*{20}c} {{P_{{ix},}}}& {{P_{{iy},}}}& {{P_{{iz}}}} \\ \end{array}).\)

For anisotropic materials, Eq. 17 becomes:

$$ \vec{f} = - \nabla _{{\vec{P}_{i}}} DU. $$
(18)

Since a flux flows along an edge from a high potential point to a low potential point, an internal force exists between any two adjacent points and the internal force at a point is further derived from the neighbouring points of this point.

The force between two adjacent points \(\vec{P_i}\) and \(\vec{P}_{j}\) is:

$$ \vec{f}_{{ij}} = D\frac{{{\left| {U_{{\vec{P}_{j}}} - U_{{\vec{P}_{i}}}} \right|}}}{{{\left\| {\vec{P}_{j} - \vec{P}_{i}} \right\|}}}{{\overrightarrow{P_{i} P_{j}}}}. $$
(19)

The detailed derivation of Eq. 19 is described in Appendix.

The internal force \(\vec{g}_{i}\) at a given point \(\vec{P}_{i}\) is the sum of the internal forces from all adjacent points of point \(\vec{P}_{i}.\)

$$ \vec{g}_{i} = {\sum\limits_{j \in N(\vec{P}_{i})} {\vec{f}_{{ij}} = D{\sum\limits_{j \in N(\vec{P}_{i})} {\frac{{|U_{{\vec{P}_{j}}} - U_{{\vec{P}_{i}}} |}}{{{\left\| {\overset{\lower0.5em\hbox{$\smash{\scriptscriptstyle\rightharpoonup}$}} {P} _{j} - \vec{P}_{i}} \right\|}}}{{\overrightarrow{P_{i} P_{j}}}}}}}}, $$
(20)

where \(N(\vec{P}_{i})\) is the neighbouring points of \(\vec{P}_{i},\) and \(\vec{f}_{{ij}}\) is the force between point \(\vec{P}_{i}\) and its neighbouring point \(\vec{P}_{j}.\)

4.2 Optimization of internal forces

The internal force between two points is associated with the gradient of potential, i.e. it is proportional to the potential difference between these two points and is inversely proportional to the distance between these two points. The internal force increases with the increment of the potential difference, and the increment of the internal force further results in the decrement of the distance between these two points. The decrement of the distance subsequently causes the increment of the internal force at the next iteration. Consequently, a large enough discontinuity will finally occur, resulting in undesirable effects such as instabilities in the numerical integration and visual discontinuity points.

To overcome this problem, a neighborhood [R min, R max] is defined as shown in Fig. 2 as the allowable range of a neighbouring point \(\vec{P}_{1}\) of the observed point \(\vec{P}_{0}.\) A weight function is used to control the potential between these two points in order to locate the neighbouring point \(\vec{P}_{1}\) within the allowable range during the deformation process. The weight function is to ensure that the potential and hence the internal force is decreased when the distance between these two points is less than R min or is increased when the distance is greater than R max. In addition, the weight function should be continuous at the two ends of its parameter interval to avoid the occurrence of visual discontinuity points.

Fig. 2
figure 2

The neighborhood of a point

Full size image

Due to the simplicity, monotony and continuity of polynomial functions, one straightforward way is to construct a piecewise polynomial function with respect to the distance between two points as the weight function. As shown in Fig. 3, the weight function s(l) is defined as a piecewise continuous polynomial function:

$$ \begin{aligned} s(l) &= \left\{{\begin{array}{*{20}l} {{b_{1} (\varphi)\begin{array}{*{20}c} \\ \end{array} \begin{array}{*{20}c} \\ \end{array} \hbox{if}\begin{array}{*{20}l} {{l\, < \,R_{{\min},} \begin{array}{*{20}c} \\ \end{array} \begin{array}{*{20}l} {{\hbox{letting}\begin{array}{*{20}l} {{\varphi = \frac{{R_{{\min}} - l}}{{R_{{\min}}}}} \hfill} \\ \end{array}} \hfill} \\ \end{array}} \hfill} \\ \end{array}} \hfill} \\ {{1\begin{array}{*{20}c} \\ \end{array} \begin{array}{*{20}c} \\ \end{array} \begin{array}{*{20}c} \\ \end{array} \begin{array}{*{20}c} \\ \end{array} \begin{array}{*{20}c} \\ \end{array} \hbox{if}\begin{array}{*{20}c} {{R_{{\min}}\,\leqslant\,l\,\leqslant\,R_{{\max}}}} \\ \end{array}} \hfill} \\ {{b_{2} (\varphi)\begin{array}{*{20}c} \\ \end{array} \begin{array}{*{20}c} \\ \end{array} \hbox{if}\begin{array}{*{20}c} {{l > R_{{\max},} \begin{array}{*{20}c} \\ \end{array} \hbox{letting}\begin{array}{*{20}l} {{\varphi = \frac{{l - R_{{\max}}}}{{R_{{\max}}}}} \hfill} \\ \end{array}}} \\ \end{array}} \hfill} \\ \end{array}} \right.,\\ l &= {\left\| {\vec{P}_{1} - \vec{P}_{0}} \right\|},\\ b_{1} (\varphi) &= - 6\varphi ^{5} + 15\varphi ^{4} - 10\varphi ^{3} + 1.0,\\ b_{2} (\varphi) &= 6\varphi ^{5} - 15\varphi ^{4} + 10\varphi ^{3} + 1.0,\\ \end{aligned} $$
(21)

where b 1 (φ) is chosen as a monotonically decreasing polynomial and b 2 (φ) as a monotonically increasing polynomial over the interval [0,1], and the first and second derivatives of both b 1 (φ) and b 2 (φ) are zero at the ends of the interval. Therefore, the weight function is continuous and smooth in the first and second derivatives. The smoothness and continuity conditions are important for well behaved numerical integration.

Fig. 3
figure 3

Potential weight function

Full size image

With the potential refined by the weight function, the internal force is further refined as:

$$ \vec{f} = - D\nabla _{{\vec{P}_{i}}} s(l)U(l). $$
(22)

Since

$$ \nabla _{{\vec{P}_{i}}} s(l)U(l) = s(l)\frac{{{\rm d}U}}{{{\rm d}l}}\nabla _{{\vec{p}_{i}}} l + U(l)\frac{{{\rm d}s(l)}}{{{\rm d}l}}\nabla _{{\vec{p}_{i}}} l $$
$$ \nabla _{{\vec{p}_{i}}} l = - \frac{{\vec{P}_{j} - \vec{P}_{i}}}{{{\left\| {\vec{P}_{j} - \vec{P}_{i}} \right\|}}} = - {{\overrightarrow{P_{i}P_{j}}}} $$
(23)

the force between point \({\vec{P}_i}\) and its neighbouring point \(\vec{P}_{j}\) is

$$ \vec{f}_{{ij}} = D{\left[ {s(l_{{ij}})\frac{{U_{j} - U_{i}}}{{{\left\| {\vec{P}_{j} - \vec{P}_{i}} \right\|}}} + (U_{j} - U_{i})\frac{{{\rm d}s(l_{{ij}})}}{{{\rm d}l}}} \right]}{{\overrightarrow{P_{i} P_{j}}}}. $$
(24)

4.3 Incompressibility constraint and model dynamics

Soft tissues such as living tissues are usually incompressible, i.e. they have the property of volume preservation. In order to model the property of volume preservation, similar as [41], an external incompressibility constraint is introduced to soft tissue models. The volume variation is penalized by applying an incompressibility constraint force to nodes along the normal of each node. The incompressibility constraint force at point \({\vec{P}_i}\) is:

$$ \vec{f}^{{\rm incomp}}_{i} = \rho {\sum\limits_{j \in N(\vec{P}_{i})} {{\left({\frac{{V_{j} - V^{0}_{j}}}{{V^{0}_{j}}}} \right)}}}^{2} \vec{n}_{i}, $$
(25)

where \(V^{0}_{j}\) and V j are the initial and current volumes of the prismatic element that adjacent to node \(i, \vec{n}_{i}\) is the normal at point \({\vec{P}_i},\) and ρ is the scalar factor.

When an external force is applied to a soft tissue, the contact point of the external force is replaced with a new position. As a result, the other points not directly influenced by the external force are in an unstable state. The energy generated by the external force is propagated through autowaves among mass points to establish a new equilibrium state by generating the corresponding internal forces. Based on the equilibrium state, the new position of each point is obtained. The dynamic process is governed by the Lagrangian equation of motion of each node:

$$ m_{i} \frac{{{\rm d}^{2} \vec{P}_{i}}}{{{\rm d}t^{2}}} + \gamma _{i} \frac{{{\rm d}\vec{P}_{i}}}{{{\rm d}t}} + \vec{g}_{i} + \vec{f}^{{\rm incomp}}_{i} = \vec{F}_{i}, $$
(26)

where \(\vec{P}_{i}\) is the position vector of node i, and m i and γ i are the mass and damping constants of node i, respectively. \(\vec{g}_{i}\) is the net internal force applied to node i at time t, \(\vec{f}^{{\rm incomp}}_{i}\) is the incompressibility constraint force applied to node i, and \(\vec{F}_{i}\) is the external force applied to node i at time t.

The solution of Eq. 26 can be computed either by an implicit integration scheme or an explicit integration scheme. Although implicit integration has the advantage of being unconditionally stable, which means it allows large time steps to be used, it is computationally intensive and requires inverting a sparse matrix at each iteration [3]. Therefore, the explicit integration scheme is used to solve Eq. 26 The advantage of the explicit scheme is that no matrix inversion is required for updating each vertex position.

5 Performance evaluations

In this section, the deformation based on the proposed methodology is evaluated comprehensively in three aspects: (i) mechanical behaviours, (ii) interactive deformation simulation with force feedback, and (iii) the computational performance. The comparison with the existing deformation methods is also discussed. For the convenience of investigating different deformation behaviours effected by the autowave model, the same elastic constants \(\left(E=2.5 \times 10^{5}\, \hbox{kg/cm}^{2}\;\hbox{and}\;\nu= \frac{1}{3}\right)\) are used in the following examples.

5.1 Mechanical behaviours

Realism of soft tissue simulation depends on the accuracy of the utilized deformation methods. Ideally, a deformation method should enable the computer-generated organs to reproduce the biomechanical behaviours of real tissues. Unfortunately, the complexity of soft tissues makes accurate modelling prohibitive. Microscopically, soft tissues are composed of various biomaterials with different compositions and arranged into different structures, and thus exhibiting complicated mechanical characteristics when studied from a macroscopic perspective. Given the intricacies, the validation of the proposed methodology is focused on the typical mechanical behaviours of soft tissues, including non-linearity, visco-elasticity, anisotropy and inhomogeneity.

Due to the different behaviours between living tissues and non-living tissues, the living tissues have to be used to test the proposed methodology. However, such an experiment with living tissues raises ethical issues and technical difficulties. Validation of deformable models still remains a challenge and there is not a technique suited to capture the full range of tissue behaviours [22, 30, 37]. Therefore, similar to most research efforts in soft tissue deformation [8, 12, 41], empirical evidences on mechanical properties of living tissues are used to verify the simulated behaviours of virtual tissue models.

5.1.1 Soft tissue responses

To assess the ability of the proposed methodology to model real tissues, a set of experiments have been conducted to verify if the proposed methodology could predict the typical mechanical behaviours of soft tissues, including two common features of visco-elasticity, i.e. strain creeping and hysteresis, and non-linear load-deformation relationship [17, 21, 25].

The creeping has been verified by applying a constant force to a node of a surface grid. Eight materials modelled with different damping constants (from top to down in Fig. 4, the damping is 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0 and 10.0, respectively) are tested. Figure 4 shows the variation of displacement against time under a constant force. The results in Fig. 4 demonstrate that the displacement builds up asymptotically towards a maximum value. The observed responses are similar to the creeping measurements obtained from real tissues [21].

Fig. 4
figure 4

Creeping property

Full size image

The hysteresis effect has been verified by applying an external force to a node of a surface grid. Two materials modelled with two different damping constants (the damping is 2.0 and 9.0 in Fig. 5 respectively) are tested. The deformation is calculated when the force is increased at a constant rate from zero and then down to zero again. As shown in Fig. 5, a solid line shows the deformations under the loading process and a dot line shows the deformations under the unloading process. The results in Fig. 5 demonstrate that the loading and unloading process follow two distinct paths, which are in accordance with the hysteresis phenomenon exhibited by living tissues [17].

Fig. 5
figure 5

Hysteresis property

Full size image

The proposed methodology has been tested to verify the non-linear load-deformation relationship. Eight materials modelled with different damping constants (from right to left in Fig. 6, the damping is 1.0, 2.0, 4.0, 5.0, 6.0, 7.0, 9.0 and 10.0, respectively) are tested. The deformation is calculated when the force applied to the model is increased at a constant rate. The results in Fig. 6 demonstrate that the deformation varies non-linearly with the applied force [17, 21].

Fig. 6
figure 6

Non-linear load-deformation relationship

Full size image

With reference to the results presented in Figs. 4, 5, 6, the proposed methodology reproduces the typical mechanical behaviours of real tissues, and the simulated mechanical properties are similar to those exhibited by soft tissues.

5.1.2 Isotropic, anisotropic and inhomogeneous deformations

Isotropic, anisotropic and inhomogeneous deformations have been achieved by using the property of diffusion coefficient. Figure 7 illustrates the deformation of an isotropic material (the number of mass points = 360, diffusion coefficient = 0.05, mass = 1.0 and damping = 1.0, the external force = 20 kg). The undeformed shape of the material is a cylinder. Figure 7a–d are different views of the material deformed under a compressive force.

Fig. 7
figure 7

Isotropic deformations

Full size image

The anisotropic property of soft tissues can be easily simulated by the proposed methodology through simply setting different diffusion coefficients in different parameter directions. Figure 8 illustrates the deformation of an anisotropic cylindrical material. The material has the same material parameters with the material in Fig. 7 except that the diffusion coefficients are different in the parameter directions (diffusion coefficient on the lines parallel to the cylinder’s axis = 0.05 and diffusion coefficient on the segment lines = 0.1). Compared to the isotropic deformation shown in Fig. 7, the anisotropic deformation of the material is illustrated in Fig. 8, where the cylinder is deformed more on the segment lines and the cylinder becomes narrower around the deformation area.

Fig. 8
figure 8

Anisotropic deformations

Full size image

The inhomogeneous property of soft tissues can also be simulated by setting different diffusion coefficients at different points. Figure 9 illustrates the deformation of an inhomogeneous cylindrical material. The material has the same material parameters with the material in Fig. 7 except that the portions with white grids (diffusion coefficient = 0.1) have a different diffusion coefficient from the green portion (diffusion coefficient = 0.05). As shown in Fig. 9, some of the portions with white grids are deformed correspondingly during the deformation, and they also highlight the difference from the isotropic deformation shown in Fig. 7. Compared to the anisotropic deformation shown in Figs. 8, 9 illustrates that the cylinder is deformed differently at the different points with different diffusion coefficients.

Fig. 9
figure 9

Inhomogeneous deformations

Full size image

5.1.3 Local and large-range deformations

Local deformation is happened commonly in surgical operations. The local deformation can be easily predicted by the proposed methodology.

It must be recalled that wave propagation is proportional to diffusion coefficient values. This property can be used to limit the deformation range. A very low diffusion coefficient can be set at the points where small deformations are expected. Figure 10 illustrates an example of local deformations, where the portions with white grids of the images have a very low diffusion coefficient. Figure 10a–d show different views of the cylinder deformed under a compressive force, where the portions with white grids are not deformed.

Fig. 10
figure 10

Local deformations

Full size image

The proposed methodology can deal with large-range deformations. With the above deformation examples, all of the deformation ranges exceed 10% of the original mesh sizes. The non-linear load-deformation relationship also reveals that the proposed methodology can deal with large-range deformations. Further, the large-range deformations can be achieved by the property of the source.

It is noted that the range of wave propagation is proportional to the source. As discussed in Sect. 3.1, the source value generated by an external force is only set to the contact point, while the source values at other points are set to zero. If there are more points set to the source value at the contact point, there is more wave energy propagated from the source in the wave field. Therefore, large deformation range can be further obtained by setting the source value to the points around the contact point. Figure 11 shows an example for the large-range deformation of a cylinder by setting the source value to multiple points, where the source value generated by the external force is set to the points on the white line.

Fig. 11
figure 11

Large-range deformations by using the property of the source

Full size image

5.2 Deformation simulation with force feedback

Laparoscopic surgery is much more difficult for surgeons than classical surgery. Surgeons lose the third dimension and use two long tools instead of their fingers. Therefore, they must use all the clues provided to them to understand and reconstruct the scene in their minds. Force feedback is one of these clues, and its introduction in the surgery simulation greatly improves the overall realism of the simulation.

An interactive prototype system has been developed with the proposed methodology for interactive deformation simulation with force feedback. As shown in Fig. 12, a PHANToM haptic device is configured with the system to carry out the deformations of soft tissues. A user can use the device to touch the deformable organ and deform it by pushing, pulling and dragging the surface in a natural 3D environment.

Fig. 12
figure 12

System configuration

Full size image

As shown in Fig. 13, the system has been implemented based on a multi-threaded platform, where deformation simulator, graphics rendering and haptic rendering are executed as individual processes. Deformation simulator performs collision detection and carries out deformation simulation if collision occurs. Graphical rendering is achieved by the OpenGL graphics library. Haptic rendering is achieved by the point-based haptic rendering technique [16], which simulates PHANToM’s stylus tip as the haptic interface point (HIP) to interact with virtual tissues. The point-based haptic rendering is a convenient abstraction for haptic interactions, and the speed is fast. During the process of deformation simulation, the collision detection algorithm checks if the HIP collides with the virtual tissue. If collided, a penetration depth is computed as the distance between the current HIP and the corresponding surface contact point, and the external force applied at the contact point is further calculated from the penetration depth. Under the external force, deformation simulator calculates internal forces from the internal energy distribution established by the CNN model, and also calculates nodal positions. The nodal positions are passed to the graphics rendering thread for graphics refresh. The internal forces are output to the PHANToM as the PHANToM’s reaction forces to provide the feeling of force feedback to the operator. Since the internal force is actually wave flux and the flow of wave flux from a high potential point to a low potential point is a natural optimum consequence generated by energy propagation, the benefit of the force feedback is that the natural distribution of internal forces on the surface of the deformed tissue can be felt through the haptic device.

Fig. 13
figure 13

System framework

Full size image

Real-time interactive deformation of virtual human organs with force feedback has been achieved with the proposed methodology. Figure 14 shows the deformations of a kidney model. Figure 14a–c illustrate the simulated deformations of the kidney model with a virtual probe. Figure 14d and e illustrate two different views of the deformed kidney model. Figure 15 shows the deformations of a liver model. Figure 15a and b illustrate the simulated deformations of the liver model with a virtual probe. Figure 15c–f illustrate different views of the deformed liver model.

Fig. 14
figure 14

Deformations of a virtual human kidney model

Full size image
Fig. 15
figure 15

Deformations of a virtual human liver model

Full size image

5.3 Computational efficiency

The timing performance of the proposed methodology has been carried out with an Intel Pentium (R) 2.8 GHz and 1.0 G memory PC with an nVidia GeForce FX5200 graphics card. With one external stimulus, the computational time of one deformation iteration utilizing different numbers of points is given in Fig. 16. It can be seen that the computation time is increased with the increment of the mesh points. The visually satisfactory refresh rate of 25 Hz to maintain a realistic visual feedback [32] is achieved by meshes with less than about 1200 grid points. Since the correct visualization of a soft tissue such as the human liver requires at least 600 grid points [27], it is sufficient to provide realistic visual feedback with the proposed methodology.

Fig. 16
figure 16

Computational performance

Full size image

For the force feedback to be realistic, the haptic device requires forces to be updated at the rate of at least 1,000 Hz [29]. The haptic fresh rate can be achieved by using a small time step to ensure that the surface grid simulated is always near the HIP when there is a collision. When the computational speed cannot cope with the haptic fresh rate any more, force extrapolation [32] is employed to improve the realism of force feedback by generating the missing forces from the previous time step. On the other hand, since force sensing depends on the deformable nature of the model, the realism of force feedback corresponds to the physical realism of soft tissue deformation [12]. Therefore, the verification of the simulated behaviours in Sect. 5.1.1 is also evidence of the physical realism of the force feedback.

The fundamental difference of the proposed methodology from most of the existing deformation methods such as mass-spring, BEM and FEM, is that soft tissue deformation is carried out from the viewpoint of energy propagation. The methodology can perform large-range displacements through its non-linear load-deformation relationship, while only small-range displacements through the linear load-deformation relationship are carried out in the most existing deformation methods. Although the computational speed may be a problem if compared with the common deformation methods to deal with small-range deformation, it can be further greatly enhanced by the digital hardware implementation of the CNN algorithm [9]. Compared with the explicit non-linear FEM [33], non-linear material properties are modelled with non-linear CNN other than geometric non-linearity, and the CNN model is easier to be formulated than the complex non-linear elastic models. The transient states are also physically modelled by the real transient states of CNN, while the transient states are modelled only through the dynamic numerical integration in the explicit non-linear FEM. Only surface mass points are involved in computation and rendering without any inside points, while the interior has to be meshed and calculated in FEM. The deformation process in the proposed methodology is caused naturally by an external force, while the deformation process is caused by a displacement without any external force existed in the non-linear explicit FEM. The proposed methodology exhibits the typical mechanical behaviours of real tissues and provides a natural force feedback, while there is not any discussion on the verification for the mechanical properties of real tissues and on force feedback in the explicit non-linear FEM. Furthermore, the methodology can easily accommodate anisotropic and inhomogeneous materials by simply setting diffusion constants, while extra work has to be performed in explicit FEM.

6 Conclusions

This paper presents a new methodology to simulate the deformation of soft tissues by drawing an analogy between autowaves and soft tissue deformation. The contribution of this paper is that (i) non-linear materials are modelled with non-linear autowave propagation other than geometric non-linearity, and (ii) autowave techniques are established to naturally propagate the energy generated by the external force and further to extrapolate the internal forces from the natural energy distribution for deformations. An improved autowave model is developed for the natural propagation of the energy generated by the external force. A CNN model is constructed to solve the autowave model for real-time deformations of soft tissues. A wave flux based method is presented for deriving the internal forces from the potential energy distribution. A haptic virtual reality system has been developed to simulate soft tissue deformation with force feedback. This proposed methodology can not only deal with large-range deformations, but also can accommodate isotropic, anisotropic and inhomogeneous materials through simply modifying diffusion coefficients.

Ongoing research work focuses on the extension of the proposed methodology to 3D solid volumes. Currently, only surface models are treated in the proposed methodology. Due to the lack of a defined interior, it is difficult to simulate complex surgical operations, such as cutting and tearing operations, and it is also difficult to simulate the deformation behaviours of 3D pathologic organ models, such as the human organ models with tumors inside. The limitation can be overcome by extending the proposed methodology to 3D volume models. In essence, the autowave model can be easily extended to 3D volume models, and the CNN based on the mass points of a 3D object surface can also be extended to the CNN based on the 3D finite elements of a 3D volume. Another aspect of improvement in the near future is the global validation of simulated deformations against real in vivo data of soft tissues. The measurement of in vivo mechanical data of soft tissues would be of great value for a full validation but is much more challenging to achieve experimentally. A minimally invasive experimental setup is being established to use normal indentation to acquire the force-displacement response of soft tissues.