Minimally invasive surgery (MIS) offers advantages for the patient, but is hindered by prolonged learning curves for surgeons compared with open surgery. MIS is more difficult due to indirect 2D vision, limited haptic feedback, reduced degrees of freedom with the long instruments as well as pivoting and fulcrum effect [1, 2]. Various training modalities have been introduced which allow for training of MIS in a safe environment outside the operating room (OR). Most common training modalities include live or cadaveric animal models, box trainers, and virtual reality (VR)-trainers. These training modalities differ in their degree of realism, costs, availability, and trainee preference [3, 4]. box trainers such as the Pelvi-trainer or the pulsatile organ perfusion (POP) trainer (Optimist, Innsbruck, Austria) provide training on real tissue with real instruments and haptic feedback. Setup time is short, and the usage of both animal and artificial material is possible. VR trainers have the potential to simulate different scenarios of a certain procedure and give feedback based on metric data. As a matter of fact, novices can train independent from supervision and learn important aspects in a safe environment, outside the OR.

The POP trainer has additional value because it offers the perfusion of organs, e.g., the liver. Therefore, the simulation of real complications, such as bleeding, is possible [5–7]. This potentially brings the benefit of procedural and scenario-based training for both the novice and the expert surgeon. On the one hand, novices can train basic laparoscopic procedures, e.g., laparoscopic cholecystectomy (LC), and become aware of which motions can potentially cause injuries. On the other hand, experts can explicitly train the handling of complications. These properties make the POP trainer an excellent training modality for simulation of operations and for performance testing, including performance under pressure if complications occur. It includes a perfusion system with a pump and connections for blood vessels to simulate bleeding with colored water.

The objective of all training modalities is to shorten learning curves, to provide the novice with skills, and to increase patient safety. A lot of studies have established face and construct validity for a wide spectrum of training devices [8, 9]. Face validity tests whether a measurement procedure indeed appears to measure what it should, seen from a test person’s subjective perspective [10]. The POP trainer is thought to offer a training alternative for different experience levels and must therefore offer a useful (MIS skills must be improved) and realistic (learned skills must be transferable to the OR) training setting outside the OR. Construct validity, in contrast, evaluates whether a test can reliably distinguish between different performance levels of trainees, which was done in an earlier study by our group [11–13]. However, there is currently no study establishing face validity for the POP trainer for LC. LC is one of the most commonly performed procedures in the USA and in Europe in abdominal surgery [14]. Thus, LC has a high impact on the healthcare sector.

The primary objective of this study was to establish face validity of the POP trainer as a training modality for LC. Secondary aims include the ranking of training modalities, as well as exploration of their specific advantages and disadvantages.

Materials and methods

Course and setting

The present study was performed using data collected from MIS training courses for basic and advanced participants offered by the Department of General, Visceral and Transplantation Surgery at Heidelberg University between January 2010 and December 2014. The participants received information about the study, and informed consent was obtained. Teaching faculty from Heidelberg University were excluded from the study. The local ethics committee at Heidelberg University approved the study protocol before inclusion of the participants (S-334/2011). Participants trained in the fully equipped MIS training center that is equipped with a VR trainer (Lap Mentor™, Simbionix ©, Cleveland, USA), eight laparoscopy units (KARL STORZ GmbH & Co. KG, Tuttlingen, Germany) with Box- and POP trainers, and two fully equipped OR tables for live animal training (Figs. 1, 2). After introduction of basic techniques by teaching faculty, the participants had to train on all training modalities, and each of the participants (n = 52) was asked to perform at least one LC on a cadaveric porcine liver with the POP trainer. Participants were assigned to either the novice or expert group according to their laparoscopic experience: experts (n = 15) who had performed more than 50 laparoscopic procedures and novices (n = 37) with less than 50 procedures. The participants evaluated the training modalities after completion of the multimodality training course at the end of the second day (POP trainer, box trainer, VR trainer, live animal training) (Fig. 3).

Fig. 1
figure 1

Participant performing LC on the POP trainer

Full size image
Fig. 2
figure 2

Participant performing basic laparoscopic task on the box trainer

Full size image
Fig. 3
figure 3

Flowchart for MIS training course

Full size image

Evaluation

After training, all participants were asked to fill out a questionnaire concerning face validity of the POP trainer. The questionnaire consisted of ten items concerning realism (items 1–7) and usefulness (items 8–10) of the POP trainer (Table 1). The scale reached from “completely disagree” (1) to “completely agree” (5). A value of 3 was considered to be a neutral answer, while any deviation was interpreted as partial agreement or disagreement. In addition, the participants had to rank the training modalities and name specific advantages and disadvantages of each modality. Participation in the survey was voluntary. All participants were employees of Departments for General Surgery in Germany.

Table 1 Overall face validity evaluation of the POP trainer on a five-point Likert scale (1 = completely disagree, 5 = completely agree)
Full size table

Statistical analysis

All data are reported as mean value ± standard deviation. To compare opinions between expert and novices, an exact two-tailed Mann–Whitney U test was performed. IBM SPSS® Statistics software was used for data analysis (version 22.0, IBM SPSS Inc., Chicago, IL, USA). A p value <0.05 was considered statistically significant.

Results

Participants

A total number of 52 participants responded to the questionnaire with the ranking of training modalities. For the evaluation, they were divided into two groups based on their experience in the OR: those who had performed more than 50 laparoscopic procedures (15 experts) and less than 50 (37 novices). Furthermore, 42 participants (9 experts and 33 novices) answered the questionnaire for the POP trainer. There were no significant differences in gender distribution between the two groups. The expert group was significantly older and had more laparoscopic procedures and more laparoscopic cholecystectomies performed (Table 2). Lastly, 38 participants (11 experts and 27 novices) answered the questionnaire for advantages and disadvantages.

Table 2 Demographics and laparoscopic experience of participants
Full size table

Face validity

The average score of realism for the POP trainer was 3.8 ± 0.9, and for usefulness, it was 4.6 ± 0.9. In the questionnaire, all questions were answered with high scores, with the lowest scores for “The degree of freedom of movement of the instruments is realistic” and “The haptic feedback of tissue/organs is realistic” and the highest score for “It would be desirable to have a POP trainer at my own hospital” (Table 1).

Novices rated the POP trainer significantly better for its overall usefulness than the experts did (4.6 ± 0.8 vs. 4.3 ± 1.0; p = 0.040). Both groups rated the overall realism equally high (3.8 ± 0.9 vs. 3.6 ± 0.9; p = 0.194). The experts agreed less than the novices that “The POP trainer resembles reality” (experts: 3.1 ± 0.8 vs. novices: 3.8 ± 0.7; p = 0.010) and that “The operation on the POP trainer is realistic” (experts: 3.4 ± 1.1 vs. novices: 4.5 ± 0.8; p = 0.003). Novices rated the question “It would be desirable to have POP trainer at my own hospital” higher than experts (experts: 4.2 ± 1.1 vs novices: 4.8 ± 0.8; p = 0.040) (Fig. 4).

Fig. 4
figure 4

Face validity of the POP trainer

Full size image

Ranking of training modalities

With regard to the overall ranking of the training modalities, the participants rated the animal training best in all categories with a mean ranking of 1.1 ± 0.3. On average, the POP trainer placed second (2.3 ± 0.9), whereas the box trainer and the VR trainer placed third (2.8 ± 1.1; 2.8 ± 0.9). For the group by group ranking, novices ranked the animal training first (1.1 ± 0.3), the POP trainer second (2.2 ± 0.8), and the box trainer (2.8 ± 1.1) and the VR trainer (2.8 ± 0.9) third. In contrast, experts also ranked the animal training first (1.1 ± 0.4) and the POP trainer (2.6 ± 0.9) second, but the VR trainer (2.7 ± 0.8) placed third and the box trainer fourth (2.9 ± 1.0). The POP trainer was ranked significantly higher on average by novices than by experts (2.2 ± 0.8 vs. 2.6 ± 0.9; p < 0.001). There was no statistical significance between the experts and novices in the ranking of the animal training, the box trainer or the VR trainer (Fig. 5).

Fig. 5
figure 5

Average evaluation for training modalities

Full size image

Looking only at the POP trainer in particular, it achieved the highest overall rankings for “Generally helpful for training laparoscopy” (2.2 ± 0.8), “Helpful for training instrument coordination” (2.2 ± 0.9), and “Helpful for training tissue preparation” (2.2 ± 0.7). It never placed last compared with the other training modalities. Participants gave the lowest ranks for the POP trainer for “Exact simulation of intraoperative situation” (2.4 ± 0.7), “Helpful for training knot tying” (2.4 ± 1.0), and “Helpful for training full operations” (2.4 ± 0.8). Even for these three aspects, the POP trainer was better than at least one other modality and never placed last.

There were differences in rankings between experts and novices for “Generally helpful for training laparoscopy” (2.7 ± 1.0 vs. 2.0 ± 0.7; p = 0.018), “Exact simulation of intraoperative situation” (2.7 ± 0.7 vs. 2.3 ± 0.6; p = 0.042), “Helpful for training instrument coordination” (2.6 ± 1.0 vs. 2.0 ± 0.8; p = 0.048) and “Helpful for training complete operations” (2.9 ± 0.7 vs. 2.2 ± 0.7; p = 0.003). There were no significant differences for the other statements (Table 3).

Table 3 Ranking of training modalities
Full size table

Advantages and disadvantages of training modalities

When asked for advantages, 34 participants mentioned the high grade of realism for the operation on animals. The second most named aspect (15) was the helpfulness of the box trainer for the acquisition of basic laparoscopic skills. The POP trainer was mentioned for two aspects with 13 votes each for being highly realistic and as helpful for learning basic laparoscopic skills. The fifth and sixth most named aspects (9 votes) were the box trainer for its helpfulness in learning suturing and knot tying and the VR trainer for the reason that no further material or animal organs are needed (Fig. 6).

Fig. 6
figure 6

Most named advantages by modality

Full size image

In terms of disadvantages, the participants most often named the low grade of realism (16) in the box trainer, followed by ethical concerns when it comes to operations on animals (15). The VR trainer was thought to be unrealistic (10), and the lack of tissue feeling was seen as a disadvantage (6). Technical problems occurred during the training of five participants with the VR trainer (Fig. 7).

Fig. 7
figure 7

Most named disadvantages by modality

Full size image

Discussion

In order to establish face validity for the POP trainer, we conducted a prospective trial with laparoscopic experts and novices assessing usefulness and realism of the POP trainer for LC in a dedicated MIS training center. After analysis of the questionnaire, participants granted the POP trainer face validity with high scores for both usefulness and realism. Overall, the participants ranked animal training first and the POP trainer as second in the comparison of all training modalities. The high degree of realism in animal training was the most important advantage of all training modalities. Helpfulness for learning basic skills with the Box- and POP trainer and realistic simulation with the POP trainer were named as important advantages.

Face validity

In our study, both experts and novices considered the POP trainer to have face validity with high degrees of realism and usefulness. On average, the novices rated the POP trainer better than the experts did, even though significant differences in opinions between expert and novices were only found for usefulness. This might be explained by the fact that experts can make a better comparison due to their experience in the OR. They possibly take aspects into their consideration which novices do not know about. Novices, in contrast, would like to use any chance of training their laparoscopic skills outside the OR, independent of its efficiency. One crucial part of performing MIS safely is that one must be familiar with the concept of the critical view of safety [15]. This concept aims to identify possible anatomic variations with the paradigm “two and only two structures enter the gallbladder.” The surgeon has to clearly identify the anatomy before clipping or cutting the structure in order to avoid damage to vital structures such as the common bile duct [16]. For explanations, the top of the POP trainer can be removed and the principles can be discussed with direct view and in 3D, which is not possible during animal operations. Furthermore, training together with experts and receiving feedback might make the training more useful for novices and training in pairs may be beneficial due to peer effects [17]. Experts, on the other hand, must be challenged, and only advanced procedural tasks, e.g., fundoplication, suturing of anastomoses, and the handling of bleeding complications, are of real interest for them. Bleeding, for example, does not usually occur with experts, but in case it does during a novice operation, it is up to the expert to handle this. Therefore, in the safe environment of the POP trainer, bleeding can be purposely caused, and experts can train the handling without risking the patients’ health. Therefore, the POP trainer offers specialized and challenging training for experts as well.

Ranking of training modalities

Animal training was ranked first for each particular question. This can be explained by the fact that training on animals is a real operation, rather than a simulation. Each aspect can be trained including port placement, installation of pneumoperitoneum, anatomy, tissue dissection, and removal of the specimen. The POP trainer was ranked second in the overall ranking of training modalities. A study by Katz et al. [18] showed equally high satisfaction of trainees with cadaveric versus porcine models for laparoscopy training. In another study by Madan et al. [19] investigating the opinions of participants for different training devices, no statistical differences in opinions concerning realism and helpfulness were found between VR trainer and box trainer. Diesen et al. and Munz et al. compared the progress of laparoscopic skills while training on either a VR trainer or a box trainer. They found no significant differences for the training success between the VR group and the box group [20, 21]. Compared with the high investment and daily costs, such as specialized facilities, of the animal training, the POP trainer has better availability as well as lower costs. It can therefore be integrated into daily routine more easily. It can be concluded that specific tasks require specific training modalities and should ideally be combined in multimodality training programs.

Advantages and disadvantages of training modalities

The majority of participants named the high degree of realism of animal training as an advantage. Remarkably, the aspect that no animal material is needed with the VR trainer was also important for the participants. For the POP trainer, the participants named different advantages such as a realistic simulation, training on real tissue with haptic feedback, and the helpfulness for training basic skills. A compromise between a high degree of realism and the reduction in the used animal material could be to further optimize VR trainers. However, the POP trainer is valuable due to its flexibility and wide range of procedural applications. This is important from an economic point of view especially in smaller hospitals when only one training modality can be funded. This can be underlined by a study by Scott et al. [22], suggesting that previous laboratory training is more cost-effective than operating room time. The advantage of simulating complications with the POP trainer was explicitly mentioned by van Velthoven and Hoffmann [23] underlining its importance. To sum up, training on animal models remains the most realistic setting so far, while new technologies are evolving for MIS training [24]. When asked for disadvantages, participants named ethical concerns for animal training and the low degree of realism for the VR trainer. Hammoud et al. [25] reported further disadvantages for animal training such as availability, infectious concerns, high costs, and moral and ethical concerns. Furthermore, Undre and Darzi [26] mentioned differences in anatomy compared with the human body for animal training. In literature reviews, the limited feedback options and a need for direct observation for full assessment are disadvantages of box trainers [26].

Overall, the highly realistic animal training remains the gold standard for training MIS, but new technologies are on their way to becoming a serious alternative and are less ethically questionable. The POP trainer can be seen as a compromise between a realistic simulation and working on real tissue on one side, and the reduction in animal material by using left over material from the slaughterhouse on the other side.

Limitations

Limitations of the present study result from the fact that face validity is based on subjective impressions. In order to further evaluate its usefulness, performance assessments before and after training on the POP trainer should be done as it was for other training modalities [27–29]. In another study of our group, construct validity for the POP trainer for LC was established [13]. It should be evaluated later whether the learned skills can be translated to the OR to increase patient safety. However, establishing face validity is the first essential step for validation of a training device. Differences in opinions could also result from the lack of novices’ specific procedural knowledge, which is important in order to make a sufficient evaluation about usefulness and realism. We solved this possible bias by giving a thorough introduction in how to perform LC at the beginning of the course. In other studies, differences such as inconsistent interrater reliability among non-experts versus expert raters were found. Standardized questionnaires were used after a thorough introduction to the operative technique for all participants to minimize the above-mentioned effects in the present study [30].

In the present study, face validity of the POP trainer was established for training of LC with a high degree of realism and usefulness. Training on animals remains the gold standard, being rated the most realistic training modality. The POP trainer was seen as a good alternative, offering features which are not implemented in common training modalities but are important to trainees and experts. These included training on real tissue, perfusion of organs, and simulation of complications. Low investment costs and a short setup time make its introduction into the daily routine easy. With their specific advantages and disadvantages, the training modalities should ideally be combined in multimodality training programs (e.g., POP trainer for complications, box trainer for suturing). The use of live animals should be restricted to find a compromise between the best training quality and ethical concerns. The POP trainer offers excellent training opportunities for MIS and shows face validity for LC. The impact of training with the POP trainer on patient safety should be evaluated in further studies.