The increasing use of Image Guided Surgery (IGS), e.g., laparoscopy, for its numerous benefits to patients in the hands of skilled surgeons has made the need for effective training crucial [1]. Surgical coaching has always been challenging; especially in IGS. The learning curve for trainees tends to be greater and more arduous as the surgical field cannot be directly visualized or palpated. Instructions, directions and information are typically communicated verbally which can be time sensitive due to rapidly changing field of view leading to confusion and misinterpretation [2]. This might affect outcomes [1] as well as educational value for the trainee.

In recent years, significant advancements in virtual reality (VR) and augmented reality (AR) technologies have catalyzed their adoption across various domains beyond their traditional confines in gaming and research. This has attracted the attention of surgeon educators who have been studying the application of this novel technology as a tool in surgical education. Many VR and AR simulation platforms capable of telestration exist and are being studied for their application in surgical education [3,4,5] from basic surgical skill training [6] to minimally invasive surgical training [3,4,5].

The aim of this study was to explore the educational value of a novel telestration device in [1] technical skill acquisition, and [2] subjective experience of the trainees.

Materials and methods

Trial design

Institutional ethics board approval was requested and received prior to initiation of this randomized controlled trial. CONSORT guidelines were used for reporting the trial [7]. The trial was designed with 2 parallel arms and a 1:1 allocation ratio. All current surgical trainees (post graduate year 1–5) at our Institution were eligible to participate in the trial. An email invitation about the study was sent out to all residents by the research coordinator. The study was performed in the research dry lab of the institution.

Study task

The study task was the performance of a laparoscopic small bowel anastomosis on a standard dry lab model prepared by the research team. The model was prepared as shown (Fig. 1-top left). The following laparoscopic instruments were provided: a bowel grasper, a pair of needle drivers, a stapler, a Maryland grasper and scissors. The stapler, Endo GIA™ Ultra Universal Stapler (Medtronic, Minneapolis, MN, USA) was preloaded and kept ready with one cartridge (45 mm) for each time the task had to be performed. Following the creation of the small bowel anastomosis the common enterotomy was closed using a 25 cm long 2.0 silk suture (Fig. 1-bottom right), in a continuous fashion. The study task was performed three times at points [A], [B], and [C] as shown in Fig. 2. First as a Baseline assessment, second during the coaching session and third during the repeat assessment following the coaching session. In the control arm of the study the coaching session consisted of verbal guidance to complete the task. In the intervention arm (telestration) the coaching session consisted of verbal guidance with the addition of the telestration platform.

Fig. 1
figure 1

Top left: The standard dry lab model is prepared as shown. Top right: stapling step: the participant creates the small bowel anastomosis using the laparoscopic stapler. Bottom left: suturing step: the participant closes the common enterotomy with using a 2.0 silk suture with a taper needle, in a continuous fashion. Bottom right: cutting of the suture marks the completion of the task

Full size image
Fig. 2
figure 2

A schematic diagram of the various steps in the trial

Full size image

The telestration device

The MDN-Pen (Fig. 3), developed along with Haply Robotics Inc. (Montreal, CA) is an augmented reality platform capable of annotations on real time video feeds and inserting virtual surgical instruments into live video feeds for the purpose of surgical coaching [8] (Fig. 4).

Fig. 3
figure 3

The MDN-Pen

Full size image
Fig. 4
figure 4

Figure shows the coach on the right using the MDN-Pen telestration device in their right hand and annotating on the screen for the participant on the left

Full size image

Measurements

Video recordings (laparoscopic view only) of the Baseline and Post Coaching task performance were de-identified for bias mitigation. Furthermore, no audio recordings were captured during the Baseline and Post Coaching assessments. Two blinded raters were given the videos to rate. The assessments were rated using a modified Objective Structured Assessment of Technical Skills (OSATS) scoring system where the “Use of an assistant” component was not evaluated as it did not apply to this task [9, 10]. Both raters rated all the videos and an average of the two ratings was used for statistical analysis.

During the coaching session, both video and audio feeds from the room were captured and analyzed by the research team. The video feed was used to gather objective data such as time taken for various steps of the task. The audio feed from the room and the video feed were used to collect qualitative data on mentoring. Described as mentoring moments, these included questions asked by the participants and clarifications given by the coach. Finally, after completion of the Post Coaching session, the participants were given a feedback survey with questions to be answered using a 5-point Likert scale.

Statistical analysis

Randomization was done using the Clinical Trial Randomization Tool from the National Cancer Institute (NIH, USA available at https://ctrandomization.cancer.gov/tool/).

IBM® SPSS® Statistics (IBM Inc., NY, USA) was used for performing independent sample t tests and paired t tests on the performance data.

Results

Twenty-four residents participated in the study (11 male and 13 female). There were 11 in the telestration group and 13 in the control group respectively. There were 11 junior residents (PGY1 and PGY2) and 13 Senior Residents (PGY3, PGY4 and PGY5). Tables 1 and 2 show the non-technical results of the coaching sessions. During the coaching sessions, the audio feed recordings of three participants within the control group failed but the objective metrics derived from the video feed analysis were still possible; adjusting the group sizes accordingly in Table 1.

Table 1 Mentoring moments
Full size table
Table 2 Time taken for the coaching session
Full size table

There was no statistically significant difference in the mean Baseline assessment scores (modified OSATS scores out of a maximum possible score of 25) of the telestration and control groups; 20.50 (SD ± 0.393) and 20.62 (SD ± 3.04), respectively. Post Coaching OSATS scores were also not significantly different between the two groups (telestration group 21.41, SD ± 3.12 and control group 23.85, SD ± 2.94, p = 0.064) Table 3.

Table 3 OSATS scores (telestration group vs control group): [baseline OSATS]—a comparison of the baseline scores of the 2 groups; [post coaching scores]—a comparison of the outcome scores of the 2 groups; and [change in score]—a comparison of the improvement within each group
Full size table

In Table 4 the mean change within the control group from Baseline to Post Coaching significantly improved in all parameters of the OSATS except in “Respect for tissue”. In the telestration group the mean change from Baseline to Post-coaching significantly improved in the “Knowledge of instruments” (p = 0.004).

Table 4 Change in scores on the 5 individual categories of the OSATS from baseline (pre-coaching) to post coaching within the control group and within the telestration group
Full size table

In Table 5, within the control group, junior residents showed a significant improvement in “Instrument handling” from a mean of 3.00 (SD 0.41) to 3.71 (SD 0.64) [p = 0.035]. Senior residents showed significant improvements in “Knowledge of instruments” from a mean of 4.10 (SD 0.58) to 4.92 (SD 0.20) [p = 0.011]. “Flow of operation” from a mean of 3.58 (SD 0.58) to 4.42 (SD 0.20) [p = 0.011] “Knowledge of specific procedure” from a mean of 3.42 (SD 0.74) to 4.58 (SD 0.38) [p = 0.005] and Total OSATS scores from 21.10 (SD 3.37) to 25.75 (SD 1.37) [p = 0.014]. Within the telestration group (Table 6), Junior residents showed a significant improvement in “Knowledge of instruments” from a mean of 3.87 (SD 0.48) to 4.50 (SD 0.41) [p = 0.069].

Table 5 A comparison of the change in OSATS scores of the control group participants; junior residents and senior residents taken separately
Full size table
Table 6 A comparison of the change in OSATS scores of the telestration group participants, junior residents and senior residents taken separately
Full size table

The results of the feedback survey were generally positive for both groups, as shown in Table 7 Participants were asked to answer the questions using a 5-point Likert scale of agreement. Satisfaction (Strongly agree and Agree) with the telestration system varied from 91 to 64% on the questions asked. The subjective measure of confidence was considerably higher among the participants of the telestration group compared to the control group.

Table 7 The responses “Agree” and “Strongly Agree” have been grouped together as positive responses
Full size table

Discussion

The study results show a general trend towards an improvement in technical performance with coaching in both groups. There is significant improvement in some categories of the OSATS, which is promising. Senior residents improved in more areas than junior residents. The engagement between the coach and participant was greater as well as the duration of the coaching sessions were longer in the telestration group. Subjectively the participants in the telestration group felt much more comfortable and confident about their ability to perform the task.

Effective communication and coaching strategies for guidance in image-guided surgery, such as laparoscopy are crucial [11]. Telestration as a tool for surgical education is here to stay and currently there are many different systems being studied which show promising initial results. Head Mounted Displays (HMDs) with AR such as the STAR [5] which requires the HMD, along with large touchscreens showed that students while being guided by the system performed better than those who read a set of instructions. This study evaluated the results of sessions which were coached by the mentor. Another system, the iSurgeon which uses a camera to capture the trainer’s hand movements and overlay it on the operative field on the screen, has shown good results too. Here multiple sessions were evaluated, and all the sessions were done under supervision of the coach [4]. These studies have demonstrated that additional use of telestration results in better communication.

The MDN-Pen AR platform; a handheld annotation tool, uses mostly commercially available hardware which draws overlays on the laparoscopic monitor. This study focused on the communication and interaction between the participant and the coach, retention of knowledge by the participant, and the educational value of an additional coaching tool. In this study there was a trend towards improvement in performance and the comfort and confidence of participants who were in the telestration group was much higher.

The results of this study need to be interpreted by keeping in mind some of its limitations. The sample size was relatively small and the close repetition of the tasks themselves might’ve introduced additional bias This trial accounted for some of the existing challenges of resident’s availability and recruitment, therefore it was designed as a one-time appointment with the participants being required to perform the same task repeatedly. Multiple coaching sessions and post-coaching sessions were not feasible due to time and resource constraints. Fatigue might have been a confounding factor with some residents coming in at the end of a long day while others may have come on an off day. Prior experience of performing the task could have been another significant factor. Despite the limitations, this study demonstrates the educational value of coaching and the addition of a telestration device. Residents' perception and the increased engagement demonstrates this value. Future research is needed with larger sample sizes and in settings other than the simulator lab (e.g., operating room) to further demonstrate the added value of coaching using a telestration device.

Conclusion

Telestration is a valuable tool for surgical coaching. It helps in filling gaps in communication between the participants and the coach by increasing engagement. It leads to increased comfort and confidence among the participants in their abilities to perform the tasks. Studies on its value in coaching beyond the lab; the feasibility of using this in the operating room, with larger sample sizes, and with probably more follow-up training and practice sessions would elucidate the value of telestration more clearly.