Introduction

Endoscopic surgery brought the great merits such as less operative scar, early recovery and cosmetic advantage to not only adult patients but also to pediatric patients. In the pediatric surgery, endoscopic surgery began in the laparoscopic pylorotomy and laparoscopic cholecystectomy [1, 2]; many kinds of endoscopic surgical procedures were applied for children and established. The working area of the forceps is narrow space and ports are near each other in the operation of the children. Pediatric surgeons are forced to manipulate the forceps in the constrained operative field [3]. Therefore, highly advanced skills are required of pediatric surgeons in endoscopic surgery because of a wide variety of operative procedures, but their experienced cases were limited in number as compared with general surgeons [4].

Nevertheless, there is no specific skill evaluation system for pediatric endoscopic surgery. The Japanese Society of Pediatric Endoscopic Surgeons developed an endoscopic surgical skill qualification (ESSQ) system from 2008 [5]. However, this is a video-based subjective evaluation system. So we developed and validated an objective system for pediatric surgeons. The aim of this study was to validate our objective endoscopic surgical skill assessment system for pediatric surgeons.

Methods

In the ESSQ system, the task operation is laparoscopic fundoplication [5]. Therefore, we set up a suture ligature model of the crura of the diaphragm for infant fundoplication. Therefore, suture ligature of the crura of the diaphragm is the most technically difficult component of fundoplication.

Endoscopic surgical skill evaluation system

Commercial-based suture evaluation simulator system was developed by Waseda University and Kyoto Kagaku Co., Ltd. (http://www.kyotokagaku.com/products/detail01/m57.html). This system is composed of a personal computer, suture pad, USB camera, and suturing unit. The suture pad was modified to be a suture ligature model of the crura of the diaphragm, as shown in Fig. 1. A rubber tube was applied as an esophagus 8 mm in diameter. The hepatic branch of vagus nerve was also attached the left side of esophagus (Fig. 2). A pressure sensor was installed in the suture pad, and a box trainer (Endowork Pro II; MC Medical, Tokyo, Japan) was used. An image showing the inside of the box was projected through a CCD camera onto the monitor. The box trainer incorporated forceps and a needle holder (Karl Storz, Tuttlingen, Germany) that is used during clinical endoscopic surgery. The suture pad image was captured by USB camera and used for assessment, as shown in Fig. 2.

Fig. 1
figure 1

Overview of suture ligature model of the crura of the diaphragm

Full size image
Fig. 2
figure 2

Detail of suture ligature model of the crura of the diaphragm

Full size image

Examinees

Examinees were divided into two groups. The expert group (E) had four expert pediatric endoscopic surgeons certified by the ESSQ system and six trainers from the Kyusyu University endoscopic surgery training center. The novice (N) group had 16 trainees from the pediatric endoscopic surgery training course held at Kyusyu University endoscopic surgery training center. The novice who specializes in pediatric surgery has not experienced over 20 cases of fundoplication.

Evaluation for endoscopic surgical skills

The participants had to perform two suture ligatures of the crura on both the ventral and dorsal side of the esophagus using an intracorporeal knot in the box trainer. Suture material is 3-0 ethibond, 15 cm in length (Ethicon Endo-Surgery, Cincinati, USA). The task time was 15 min, and the time score was calculated using the residual time. All participants experienced this evaluation task only once in this trial. The evaluation points were time, force on the tissue, suture tension, stitch spacing, equidistance (width), mean score, and total score (Fig. 3).

Fig. 3
figure 3

Captured suture unit for image analysis

Full size image

Statistical analysis

All data are expressed as the mean ± standard deviation. The statistical analysis was performed with the two tail paired and unpaired Student’s t test, and p values of less than 0.05 were considered to be statistically significant.

Results

All 26 participants completed the evaluation task. The time score for E and N was 63.50 ± 26.52 and 6.56 ± 12.66 (p < 0.0001), respectively. The score for force on the tissue for E and N was 73.70 ± 29.78 and 43.63 ± 30.93 (p < 0.0001), respectively. The suture tension score for E and N was 98.00 ± 3.37 and 78.31 ± 39.65 (p = 0.0616), respectively. The stitch spacing score for E and N was 60.10 ± 29.76 and 20.00 ± 27.94 (p < 0.05), respectively. The equidistance score for E and N was 70.20 ± 16.96 and 46.81 ± 33.83 (p < 0.005), respectively. The mean score for E and N was 63.32 ± 12.90 and 34.09 ± 18.38 (p < 0.0001), respectively, and the total score for E and N was 67.50 ± 10.46 and 40.19 ± 21.05 (p < 0.0005), respectively (Table 1). The time and the force scores on the tissue had a strong correlation.

Table 1 Evaluation score
Full size table

Discussion

In the present study, we designed the objective endoscopic surgical skill assessment system for pediatric surgeons and investigated its validation based on a quantitative and qualitative analysis between expert group (expert surgeons) and novice group (pediatric surgeons in training). The major findings of this study are as follows: (1) Expert group including expert pediatric surgeons who had ESSQ showed better score than those of novice group. (2) As regarding the analysis of data, the score of time, force on the tissue, the stitch spacing, the equidistance, the mean value, and the total value for expert group were significantly higher than those of novice group. (3) The score of suture tension was not significant between expert group and novice group. (4) The time sore and the sore of force on the tissue had a strong correlation. (5) All examinees including the experts and the novice had their performance on this task evaluated only once, then no bias would exist.

The Japan Society for Endoscopic Surgery (JSES) has established an Endoscopic Surgical Skill Qualification (ESSQ) System and started examination in 2004. Two kinds of criteria, namely common and procedure-specific, were prepared. The common criteria were designed to evaluate set-ups, autonomy of the operator, display of the surgical field, recognition of surgical anatomy, co-operation of the surgical team. The procedure-specific criteria depend on the surgical fields. The main goal of the system is to decrease complications of endoscopic surgery by evaluating the surgical skills of each applicant and subsequently certify surgeons with adequate skills to perform endoscopic operations safely. In other word, ESSQ system in Japan is the safe endoscopic surgery. This system was introduced for safe and precise endoscopic surgery in the field of general surgery [6], urology [7], gynecology, orthopedics (mainly spine surgery) and pediatric surgery [5].

A committee of pediatric ESSQ created a checklist to assess the applicant’s endoscopic surgical skills. Skills are assessed in a double-blinded fashion by evaluating an unedited video recording of a laparoscopic fundoplication for pediatric gastroesophageal reflux disease [5]. There is no ESSQ system in the world that can rival. But this system was not completely objective evaluation.

The objective surgical skill evaluation in clinical operation is difficult. Operation quality could not quantify the degree of subjective belief in the surgical skills. In general, operation time, blood loss and complication rate were used for the index of evaluation.

On the other hand, virtual reality simulators are used for the tool of objective surgical skill evaluation [8, 9]. But skill level in the virtual reality simulators does not always reflect the ability of the practical surgery. Endoscopic operative procedure contains many kinds of surgical skill component such as display of the surgical field, recognition of surgical anatomy, dissection, ligation, suturing, selection of energy device and co-operation of the surgical team. Suture ligature of the crura of the diaphragm is one component operation of the laparoscopic fundoplication. It is important for the pediatric patients that pediatric surgeons who were certified their skills objectively should perform the endoscopic surgery.

In summary, our obtained results from this paper ensured the consistency between the ESSQ system and the objective endoscopic surgical skill levels of surgeons.

Conclusions

The results revealed that the expert group possessed gentle and speedy skills compared with that of the novices. Using this validation study, our established model could be used to objectively evaluate the endoscopic surgical skills of pediatric surgeons. An objective evaluation not only facilitates assessing the trainee achievements, but also reveals the basics. Moreover, the progress made along with repeated trials encourages trainees and promotes effective learning.