1 Introduction

Laparoscopic surgery has become popular because it offers many benefits to patients over conventional open surgery, such as short recovery time, less scarring, and reduced pain. However, it takes a long time for surgeons to acquire the skills needed to perform laparoscopic surgery. Laparoscopic surgery requires surgeons to maneuver long instruments while watching laparoscopic camera images. Moreover, laparoscopic surgery is generally difficult because of the required hand-eye coordination, the pivoting motion of the laparoscopic instruments through tiny incisions, and the lack of sense of distance on the 2-dimensional display of the laparoscopic camera. Virtual simulation has been developed to overcome the difficulty of learning laparoscopic surgical skills [1, 2]. Novice surgeons or residents can practice in the convenient environment provided by the virtual simulator, which was not possible in traditional laparoscopic surgery training methods using animals or patients. Some studies have shown the effectiveness of laparoscopic surgical training using virtual simulation [3–5]. The haptic rendering technique is often used during virtual laparoscopic surgical training to provide high-fidelity simulation to enhance the realism [6, 7].

Among some open source libraries for virtual medical simulation, simulation open framework architecture (SOFA) provides highly sophisticated real-time medical simulation results [8, 9]. SOFA provides many state-of-the-art graphical and haptic functions such as force fields and numerical integration methods for deformable behavior, collision detection, loading standard data, and surface model representation. Users can create medical simulation software by combining new user-specific plug-in algorithms with original pre-loaded algorithms in SOFA. Many research groups have successfully developed various medical simulation systems using SOFA [8–12].

Cholecystectomy, one of the major surgeries performed using laparoscopic surgical procedures, is the most common method for treating symptomatic gallstones and gallbladder inflammation. Laparoscopic cholecystectomy, which is associated with a lower incidence of infection and pain, has now replaced open surgery [1]. The main procedures are as follows [2]: (a) identification of the cystic duct and cystic artery, (b) clamping and (c) cutting of the cystic duct and artery, and (d) dissection of the connective tissue between the liver and the gallbladder (Fig. 1). Although some products regarding cholecystectomy simulation have been developed [13–15] and tested [16, 17], studies with detailed technical descriptions of the virtual cholecystectomy are not easy to find, except for a basic simulation of bile duct exploration [18]. In this paper, we describe detailed techniques for each laparoscopic cholecystectomy step, including a new simulation technique of deformable carving simulation and adequate algorithms of state-of-the-art computer graphics. Our user test results showed that the presented system realistically simulates the cholecystectomy procedures and that it can be used to effectively train surgeons.

Fig. 1
figure 1

Cholecystectomy procedures: a identification of the cystic duct and cystic artery, b clamping, c cutting of the cystic duct and artery, d dissection of connective tissue between the liver and gallbladder, and e human organs involved in the cholecystectomy simulation

Full size image

2 Methods

2.1 Visual simulation of cholecystectomy: mesh deformation

Human organ models for cholecystectomy simulation include the liver, gallbladder, cystic duct, cystic artery, fat tissue covering the cystic duct and artery, bile duct, and connective tissue between the liver and the gallbladder (Fig. 1). The mesh models of the organs were created using Visible Human Korea data [19] and then manually modified for use in the simulation.

The organs are generally deformable models in cholecystectomy. The finite element method (FEM), one of the typical physically based simulation methods, was used for the liver and gallbladder model deformation. When a trainee virtually touches, grasps, and pulls the organs with the laparoscopic instruments, the liver and gallbladder mesh models are deformed in real time using the tetrahedral FEM. We used co-rotational FEM [20] to create the real-time mesh deformation [Eq. (2)]. Co-rotational FEM is a modified FEM, which first applies rotation to each element to avoid unrealistic results in large deformations. In co-rotational FEM, a modified form of Lagrangian equation [Eq. (2)] is solved by implicit time integration scheme such as conjugated gradient method:

$$\begin{aligned}&f(u+du)= f^0(u) + Kdu\end{aligned}$$
(1)
$$\begin{aligned}&M\ddot{u}+C\dot{u}+Kdu = f - f^{0} \end{aligned}$$
(2)

where, \(M\), \(C\), and \(K\) denote the mass, damping and stiffness matrix, respectively. \(u\) is the displacement from rest position to actual position and \(f\) is displacement force vector for all vertices. \(du\) is a small displacement around the current configuration of the deformable body, and \(f^{0}\) is force correction vector for the element rotations.

The prepared original triangular meshes were converted to tetrahedral meshes for the liver and gallbladder using isosurface stuffing [21], which fills an isosurface with a uniformly sized tetrahedral mesh.

In medical simulations using deformable models, both instrument-organ collision detection and organ-organ or self-collision detection are required for realistic simulation. To avoid interpenetration between the instruments and the organs, a god tool control system [22] is used, in which the real position is directly linked with the device and the simulated position. The real position fetched from the device is not constrained by the collision. Thus, the simulated tool is linked to the ghost by a spring and is affected by collisions. This design creates forces on the organ, causing it to deform, and generate forces on the device to return to the correct position.

In our simulation, we used some graphical techniques to create a realistic visualization. First, texture images from actual laparoscopic images were mapped on the surface mesh of each organ model. Although typical texture mapping with actual laparoscopic images was applied to most of the organs, a tri-planar texture technique [23] was used for the fat tissue covering the cystic duct and artery. This allowed us to avoid the need to update the texture coordinates while the mesh model was being carved. Tri-planar texturing is appropriate for the fat tissue model because it can be represented using repetitive patterns. Second, we simulated the burning effect that occurs during dissection using an electric cautery hook. When the instrument touches organs, the contacted region of the organ gradually turns black as described by Bruyns and Montgomery [24], which provides a realistic cauterizing simulation of the textured mesh model. Third, a smoke effect was simulated in the visualization by passing a transparent textured plane with one of several prepared smoking images in front of the virtual camera in a random direction with decreasing alpha values. Fourth, we optimized OpenGL parameters such as lighting, material, shining, and shading to make them similar to those of real laparoscopic images. All of these graphical techniques were processed by the graphics processing unit (GPU).

2.2 Identification of the cystic duct and artery: deformable mesh carving

Identification of the cystic duct and artery is an initial process in the cholecystectomy procedure and involves removal of the fat tissues covering them. It is important to carefully remove the fat tissues using a cautery hook without damaging the cystic duct and artery. To simulate this identification process, we proposed a new volumetric model for mesh carving with deformation [25]. In a related study, the volume-based approach for mesh carving was developed for a dental training simulation by Kim and et al. [26], who applied Mauchs closest point transform (CPT) [27] and Velho’s adaptive polygonization method [28] in a tooth-drilling simulation. Similar to our approach, Jeřábková et al. [29] developed an interactive cutting method for deformable objects, while we use potential fields for effective carving simulation.

First, the potential field is created from the original fat tissue mesh using CPT algorithm (Fig. 2a). The potential values near the mesh surface vary from \(-\)1 to +1 according to the distance to the surface. Next, the mesh is regenerated using the adaptive polygonization algorithm (Fig. 2b). During the carving process, collision with the cautery tip is monitored. If it touches the surface of the fat tissue whose potential value is 0 in the volume, those potential values are updated and the mesh is regenerated. This adaptive polygonization algorithm efficiently recalculates the mesh surface using fewer triangles in the simply shaped regions and more triangles in the complex regions. In our application, we also needed to add deformation to the carving simulation because fat tissue deforms when touched by the instrument. Deformation of the model is computed by hexahedral co-rotational FEM with its bounding grid model (Fig. 2c). Because the grid data are necessary for CPT and adaptive polygonization, hexahedral mesh is appropriate for the deformation. The carved mesh model (triangular mesh) and the grid model for the deformation (hexahedral mesh) are combined using barycentric mapping [8] (Fig. 2d). This mechanism efficiently maps the simulated model (surface model) and the deformation model (hexahedral model). The barycentric mapping calculates forces on the surface model according to the barycentric coefficients from the deformation model. Separation of the two mechanisms can provide an efficient and realistic freeform deformable simulation on a complex model. The mesh model deformed using the hexahedral model is shown in Fig. 2e, while a detailed view of carving with deformation is depicted in Fig. 2f.

Fig. 2
figure 2

Algorithm of volume-based carving with deformation: a original triangular mesh for the fat tissue and closest point transform (CPT) algorithm [27], b mesh after adaptive polygonization [28], c bounding grid model for deformation, d mapping between the mesh for carving and the grid for deformation, e deformed data, and f detailed view of carving (red line) with deformation

Full size image

For robust simulation, the potential values and the mesh surface are recomputed using the rest positions of the grid vertices, which are the initial positions of grid vertices before deformation. First, the distances from the carving tool’s surface to the deformed grid vertices are computed and saved in each grid vertex. For fast computation of the distance, the implicit surface representation is used for the carving tool. Then, the potential values of the grid vertices in the rest position are updated by the computed distance to the tool’s surface. The mesh surface is regenerated by the adaptive polygonization algorithm using the rest positions and updated potential values of the grid. Updated region of the mesh surface mapped to the hexahedral model. Finally, the mesh surface is deformed according to the deformed hexahedral model.

2.3 Connective tissue removal: mesh burning

After the cystic duct and artery are cut, the gallbladder is removed from the liver. To detach the gallbladder from the liver, connective tissues (fibrous tissues) between the liver and the gallbladder are dissected using a cautery hook. The gallbladder is pulled up by graspers (typically held in the surgeon’s left hand) to reveal the connective tissues, while the cautery hook is held in the right hand for connective tissue removal.

In our simulation, the connective tissues were modeled using spring models to create the deformation, which connect the gallbladder vertices to the liver mesh. The end-points of the spring were connected to the closest tetrahedron of the gallbladder or liver model using barycentric mapping. The surfaces of the connective tissues for visualization were created by connecting several neighboring springs. Dissection of the connective tissues was simulated by removing the springs after the detection of collision with the cautery hook model. If the instrument touches the connective tissue, the alpha values of the connective tissue transparency decrease; below a threshold value, they are completely deleted. During this process, a burning (darkening) effect is applied to create a realistic visualization. The detailed algorithm on connective tissue removal can be found in our previous work [30]. We used BEM for liver and FEM for gallbladder modeling not using SOFA in [30], while the current software employed SOFA’s co-rotational FEM for both liver and gallbladder models with more stable results.

2.4 Haptic rendering of cholecystectomy

To realize high-fidelity and efficient haptic rendering, we used linear complementarity problem (LCP) haptic rendering [31] in the SOFA framework. This method permits precise computation for contact responses and asynchronization between the haptic loop and the simulated loop. In general, 500–1,000 Hz for haptic rendering is required for realistic simulation, whereas 25–30 Hz is required for visual rendering. Asynchronization of the haptic and visual rendering loops can solve this problematic instability. The LCP haptic rendering model uses a free motion pre-loop devoid of actual contact and stores the resultant displacement on the preliminary loop. The simulation loop then occurs (with contact computation), and the contact response is obtained using the result of the previous pre-loop. The LCP state can also be used to compute haptic forces multiple times between the two steps during the simulation (Fig. 3).

Fig. 3
figure 3

Asynchronization between the haptic rendering and visual simulation loops

Full size image

3 Results and discussion

3.1 Visual evaluation

The simulation results of the cystic duct and artery identification are shown in Fig. 4. When the cautery hook touches the fat tissue covering the cystic duct and artery, the volume-based fat tissue model is carved using the proposed algorithm. During the identifying operation, the carved mesh of fat tissue is updated in real time (\(>\)25 Hz), and the behavior of the cystic duct and artery is computed using beam modeling. Figure 4 shows that the texture of the part that is removed from the fat tissue model is updated at each time step by the GPU-based tri-planar texture projection.

Fig. 4
figure 4

Results of fat tissue removal simulation using the volumetric model

Full size image

We also compared the proposed volume-based deformable mesh carving method. Figure 5 shows that the visual quality of the carved mesh by the proposed method (Fig. 5b) is better than that by the ordinary tetrahedral mesh-based carving method in SOFA (Fig. 5a). We found that tetrahedral mesh-based carving produced more sharp edges. With the models shown in Fig. 5, mesh quality inspection was performed using Rapidform XOR3 [32] (INUS Technology Co.). The results showed that the tetrahedral mesh model-based approach had isolated triangles (14 dangling facets and 32 small clusters), whereas the volume-based approach had no isolated triangles. Moreover, the overall opinion from surgeons supported this judgment in a user study described in Sect. 3.3 below. If the number of tetrahedra in the tetrahedral mesh-based carving method is increased to improve carving quality, the computational costs escalate dramatically.

Fig. 5
figure 5

a Comparison of the soft tissue removal simulations: a tetrahedral mesh model-based approach and b volumetric model-based approach. The small images in the bottom right corners were captured from RapidForm \(XOR_3\) [32] to inspect the mesh quality

Full size image

Figure 6 presents the results of the connective tissue removal simulation. The connective tissue between the gallbladder and the liver is being burned using a cautery hook in the surgeon’s right hand while the gallbladder is being pulled by graspers in the left hand. The graphical effects of the blackening in the operating region are shown in the figure. Smoking effects are also simulated during the connective tissue removal process. A realistic visual simulation is achieved by optimizing graphical rendering parameters such as lighting, material, and shine to resemble the actual laparoscopic images. Figures 6g, h show the images captured from the commercial systems SEP [13] (SimSurgery Co.) and LAP Mentor [14] (Simbionix Co.), respectively. The visual simulation results are comparable to those of the commercial products. A two-handed haptic device for laparoscopic surgical training was designed with consideration of the actual laparoscopic instruments, was integrated with developed software (Fig. 7).

Fig. 6
figure 6

a–f Connective tissue removal simulation from the proposed method: the connective tissue between the gallbladder and the liver is being dissected. Images from g SEP, SimSurgery Co. [13] and h LAP Mentor, Simbionix Co. [14]

Full size image
Fig. 7
figure 7

Integrated laparoscopic surgical training simulation system. (NT Research Co.) [33]

Full size image

3.2 Performance evaluation

The simulation speed was tested, and the results are listed in Table 1. A PC with an Intel Core i7 3.70 GHz CPU and a GeForce GTX 570 graphic card was used for the experiment. As expected, the results show that deformation computation and collision detection require more time when more tetrahedral elements are used in the liver and gallbladder models. In our experiment in which we checked time at each step of the simulation loop, the bottleneck of the simulation was found to be collision detection and response rather than the number of tetrahedra itself. As the number of tetrahedra increased, the computation time of the internal forces and the time spent to solve the system increased dramatically. The cost of mapping (between the tetrahedral mesh for the mechanical model and the surface triangular mesh for the collision and visualization) increased noticeably because of the increased number of points. Trading off simulation realism and speed, we chose case B (1,489 tetrahedra for the liver model and 121 tetrahedra for the gallbladder model), which shows acceptable speed and deformation behavior (Fig. 6). Thus, we ascertained that sufficient speed for the visualization was achieved in the simulation.

Table 1 Overall performance test results of various tetrahedral models
Full size table

3.3 User study

A user test of the developed system was conducted, which included a novice group and an expert group. Ten interns/residents with no experience in laparoscopic surgery were recruited in the novice group. Ten expert surgeons who had performed several hundred laparoscopic surgeries were selected for the expert group. This study had the limitation that we could not perform a comparison test with other commercial systems due to lack of availability of such systems. The developed laparoscopic training simulator was evaluated in terms of construct validity and content validity by the novice and expert groups.

3.3.1 Construct validity

Construct validity was first tested to verify that the system could properly evaluate an operator’s skills. That is, an actual laparoscopic surgery expert operating the simulation system should score higher than a novice. The task performance of the two subject groups was then measured. Before performing the tasks, both groups of subjects were allowed to become accustomed to the device for approximately 15 min. Each subject’s task performance was recorded and measured in terms of time, accuracy, error, and mistakes. The checkpoints of the given tasks and the results are listed in Table 2. The checkpoints during the given task were as follows: (a) elapsed time for finishing the virtual cholecystectomy, (b) total trajectory distance of the left instrument, (c) total trajectory distance of the right instrument, (d) remaining fat tissue after cystic duct and artery identification, (e) collision time with the cystic duct and artery, (f) remaining connective tissue after the gallbladder removal, (g) amount of liver tissue burned by the cautery hook, and (h) amount of gallbladder tissue burned during the task. The performance test results of the two groups showed that the experts scored higher than the novices at every checkpoint. Experts performed the tasks more quickly, safely, and accurately than the novices. Thus, we can conclude that the developed system can simulate actual laparoscopic surgery and that it can be used as a training tool for educational purposes.

Table 2 Task checkpoints in the construct validity test (average values of the two groups are listed)
Full size table

3.3.2 Content validity

Content validity was performed to measure user satisfaction by obtaining subjective opinions such as realism of simulation, training effectiveness, ease of use, usefulness, and general appeal. After the performance test for construct validity was performed, the content validity test was conducted for both user groups. The questions used are listed with the survey results in Table 3. A Likert scale was used in the survey (1, very bad; 2, bad; 3, neutral; 4, good; and 5, very good). No significant differences in content validity results were observed between the expert group and the novice group. Given the content reaction from the expert group, we can conclude that the proposed simulator’s visualization is realistic. The subjects felt slightly better about the software than the hardware in terms of realism and training effectiveness.

Table 3 Participant perception of the developed system using a Likert scale (1, very bad; 2, bad; 3, neutral; 4, good; and 5, very good)
Full size table

In terms of educational effectiveness, the novice group rated the system more valuable than the expert group. This can be thought to be a result of novices considering the developed system helpful for practicing their laparoscopic surgical skills. Ease of use and overall appeal were rated satisfactory by both groups.

4 Conclusion

Here, we presented our newly developed laparoscopic cholecystectomy training system. Various state-of-the-art computer graphical techniques were used for realistic visualization, and a new algorithm of deformable mesh carving was proposed. The experimental results proved that the presented system simulates the cholecystectomy procedures in real time with high degree of realism and fidelity. The user test results showed that the developed system was an effective educational tool for the development of surgical skills. We expect that the proposed methods in this paper can be applied to many other medical simulations.