Laparoscopic and minimally invasive surgical (MIS) skill acquisition involves a steep learning curve due to the niche skillsets required, such as two-dimensional depth perception, hand–eye coordination, and spatial awareness. Laparoscopic suturing is an advanced skill that is important for managing complex operative situations and intraoperative complications. It is viewed as one of the most challenging techniques in minimal access surgery and opportunities to practice are few due to the limited exposure to cases that involve laparoscopic suturing [1].

Simulation-based training is used extensively in MIS education to improve acquisition of laparoscopic skills by increasing opportunities for practice. The Fundamentals of Laparoscopic Surgery (FLS) course by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) or LapPass by the Association of Laparoscopic Surgeons of Great Britain and Ireland (ALSGBI) are examples of standardized didactic programs which utilize simulation for practicing and assessing laparoscopic skills. Despite these opportunities, many trainees still feel uncomfortable performing laparoscopic suturing [2]. This is due to a lack of effective feedback and standardized training in advanced MIS skills such as laparoscopic suturing. A qualitative study of 25 MIS surgeons and surgical residents found that the main challenges surrounding the acquisition of laparoscopic suturing skills were complexity, training misalignment, variability of opportunities, inconsistency of techniques, lack of feedback, and differing expectations [3].

Structured feedback is a crucial component in surgical training and is routinely provided during simulation-based training and clinical practice as part of the surgical education curriculum [4, 5]. In laparoscopic surgery, due to the videoscopic nature of the procedure, box trainers and virtual reality machines are used in simulation-based training to great benefits. This enables easy audio–visual recording of simulated tasks and procedures, which offer opportunities for reflection and delivering video-enhanced feedback.

Video feedback serves as an effective educational tool in coaching trainees. Visual cues have been shown to elicit considerably more attention during the learning process [6]. Both the trainer and trainee are able to directly visualize the errors made during the procedure, instead of having to recall from memory. Trainees will be better able to identify specific deficits and trainers will be able to provide effective feedback with greater clarity. Video-based feedback and coaching have been adapted into many fields such as sports, music, rehabilitation, teaching, and workforce development with promising results [7,8,9,10,11]. Backstein et al. conducted two of the earliest randomized trials assessing the effect of video feedback on the acquisition of surgical skills, which failed to show any significant improvement with video feedback [12, 13]. However, recent studies have suggested potential benefits of incorporating video feedback in surgical education. Three studies demonstrated the superiority of video feedback compared to verbal feedback or online didactic modules [14,15,16]. Several studies also included self-assessment as an experimental arm [17,18,19]. A review of 12 studies comparing the correlation of self-appraisal against expert scores found that the accuracy of self-assessment improved with the use of video feedback [20]. With video recording technologies, self-assessment can be used in conjunction with video review as a convenient educational tool.

Self-assessment refers to the ability to appraise one’s capabilities and limitations. It stems from the psychological concept of an individual’s ability to assess or evaluate their actions, attitudes, or performance [21]. A solely faculty-driven training program is resource-intensive and requires significant time-commitment from practicing surgeons, which reduces their availability for clinical work. Self-assessment enables independent learning and serves as an alternative to expert feedback. This is of significant benefit, especially in an era where health services are facing time pressures on service provision. Furthermore, didactic programs such as FLS and LapPass, which involve significant self-directed learning (SDL), can benefit from self-assessment. Previous studies assessing the efficacy of self-assessment correlated self-assessment scores to the trainees’ performances or expert assessment scores. Due to the heterogeneity of outcomes, there were conflicting results. However, when narrowed down to technical skills simulation in practical procedures, there were promising results as a substantial number of studies demonstrated a positive correlation [22,23,24,25].

The current evidence demonstrates the potential advantages of video feedback and self-assessment in surgical education. Conflicting evidence is likely due to a lack of standardization in the feedback method [25]. Hence, there is a need to investigate the impact of structured video feedback and self-assessment on surgical training. No study has yet directly compared video feedback, self-assessment, and verbal feedback using the same task and outcome measures. The aim of this study was to establish the most effective form of structured feedback on laparoscopic suturing skills performance in novices, by comparing the effects of expert verbal feedback, video feedback (video review with expert verbal feedback), and self-assessment (video review with objective self-assessment).

Materials and methods

Study design

The trial is a primary study with a prospective randomized blinded trial design. The trial was performed at G.08 Simulation Lab, Joseph Rotblat Building, Barts Cancer Institute, Queen Mary University of London, Charterhouse Square, London EC1M 6BE, United Kingdom.

Participants

University students and junior doctors were invited to participate. The inclusion criteria are novices in laparoscopic surgery, defined as individuals who have never undertaken any previous sessions of formal laparoscopic skills training and never performed any laparoscopic suturing. The exclusion criteria are individuals who have undertaken at least one session of formal laparoscopic skills training or performed at least one laparoscopic suturing task, or individuals who have not provided consent or who have been unable to complete the familiarization process. Eligibility was assessed via an online recruitment form, which also included fields for participants to indicate their name, contact details, age, gender, institution, level of training, and hand dominance. Participants were provided with a participant information sheet. Participants had to indicate on the online recruitment form that they had read and understood the participant sheet and provide consent to their participation in the study. Ethical approval was obtained from the Queen Mary University of London Research Ethics Committee.

Equipment set-up

Each experimental set-up comprised a box trainer, a video camcorder with memory cards, and a display monitor. The instruments included a laparoscopic needle holder, a Maryland dissector, and a pair of laparoscopic scissors. Suturing materials included a suturing pad and silk 2‐0 sutures. A laptop was used for video replay and review.

Familiarization process and pre-feedback practice

Participants underwent a familiarization process, which included a short lecture introducing them to the basic principles of laparoscopy, the equipment, and instruments. They also performed several repetitions of a peg-transfer task to familiarize themselves with the equipment. Next, participants viewed two videos detailing the steps of the suturing with intracorporeal knot-tying task and were provided with an instruction sheet summarizing all the steps required. They were then asked to practice the task three times and were given a maximum time of 2 h to familiarize themselves with the task. Participants were allowed to pause briefly between attempts if they felt fatigued.

Pre-feedback assessment

After familiarization and practice, each participant was given a candidate number, which was recorded on the feedback proforma. Participants had to perform one iteration of the laparoscopic suturing with intracorporeal knot-tying task with a time limit of 10 min. The start time was defined as the instance when both instruments were in view on the monitor and the stop time when both instruments were removed with the participant acknowledging their completion or if the 10-min time limit was reached. The participants’ performances were video recorded, and this was noted as the pre-feedback performance. A unique video randomization code was displayed at the beginning of each video recording to blind assessors to the participants’ allocated groups.

Feedback session

Participants were then given feedback in one of three ways: verbal feedback, video feedback, or self-assessment. All three groups used identical proformas as a framework for the feedback session, which were designed according to a simplified version of Pendleton’s rules for giving effective feedback [26]. Participants in the verbal feedback and video feedback groups received feedback from an expert familiar with the suturing task without and with a video review of their performance, respectively. Participants in the self-assessment group reviewed their own videos. Participants were given a maximum of 12 min to receive feedback or self-assess.

Post-feedback practice and assessment

After the feedback session, participants were encouraged to take a short break of at least 10 min before returning for the second round of practice. They were asked to practice the task three more times and were given a maximum time of 1.5 h. At the conclusion of the post-feedback practice session, participants were again tasked with performing one iteration of the laparoscopic suturing with intracorporeal knot-tying task with a time limit of 10 min, as in the pre-feedback assessment. This was noted as the post-feedback performance. Similar to the pre-feedback assessment session, unique video randomization codes were displayed at the beginning of the video recordings.

Questionnaire and end of study

At the end of the study, participants were given a short questionnaire comprising four questions regarding their confidence in performing laparoscopic suturing before and after the training session, as well as their knowledge for improvement and their experience of the training session. Questions were rated on a Likert scale ranging from strongly disagree, disagree, neutral, agree, and strongly agree (scored 1–5, respectively). The end of study was defined as the instance when all videos and post-study questionnaires have been analyzed.

Randomization

Randomization was performed using an online randomizer tool. Participants were randomized to the verbal feedback, video feedback, and self-assessment groups with a ratio of 1:1:1 and a block size of 3. The unique randomization codes for the video recordings of each participant both pre- and post-feedback were also generated using the same tool. The allocation sequence was concealed to the feedback provider until participants were allocated by the experimenter to receive their respective feedback after the first (pre-feedback) assessment.

Outcomes

The primary outcomes were task-specific checklist scores and global scores for Objective Structured Assessment of Technical Skills (OSATS), which measured pre-feedback and post-feedback performances in all three groups. The assessment tool was adapted from an OSATS tool developed by the Department of Obstetrics and Gynecology at Harvard Medical School and is similar to the tool used by the Royal College of Surgeons of England for the Core Skills in Laparoscopic Surgery course [27]. The secondary outcome was the qualitative analysis of the post-study questionnaire responses, based on the Likert scale of 1–5. Videos recorded during the assessment attempts, both before and after feedback, were reviewed and marked by two independent trained expert assessors who were blinded to the participants’ allocated group. The marking criteria were discussed within the research group and between assessors before study commencement to reduce variability in the interpretation of the assessment tool. Criterion issues were frequently addressed throughout the marking process without revealing the identity of video attempt.

Statistical analysis

Data were compiled on Microsoft Excel (Microsoft Corporation, Redmond, WA, USA). All statistical analyses were carried out on SPSS Statistics Version 25 (IBM Corporation, Endicott, NY, USA). The power calculation was carried out based on previous similar studies [14, 18, 28]. Based on a two-tailed test, with an α level of 0.05, power (1−β) of 0.8, and a predicted improvement in overall performance score by the video feedback intervention group of 30%, a minimum of 10 subjects was required in each arm. Analysis of demographic data was carried out using the non-parametric Kruskal–Wallis test. The Cronbach’s α reliability analysis was used to test for the expert scorers’ interrater reliability. Pre- and post-feedback OSATS scores within each group were analyzed using a two-tailed paired samples t test. OSATS score improvements were analyzed using a one-way ANOVA when comparing across groups and the Tukey’s HSD test was used for post hoc analyses. The Kruskal–Wallis test was used to analyze the post-study questionnaire outcomes. For all tests, a p value of < 0.05 was considered to be statistically significant.

Results

Demographics

Fifty-one participants were enrolled and randomly assigned to the three arms of the study with no losses or exclusions (Fig. 1). There were no differences in baseline characteristics between the groups after randomization (Table 1). There were 9 males (53%) and 8 females (47%) in the verbal feedback group, 11 males (65%) and 6 females (35%) in the video feedback group, and 6 males (35%) and 11 females (65%) in the self-assessment group. Mean ages were 21.9 years in both the verbal feedback and video feedback group, and 23.2 years in the self-assessment group. The verbal feedback and self-assessment groups contained 1 left-handed participant (6%) each, with the remaining 8 participants (94%) in each group being right-handed. All 17 participants (100%) in the video feedback group were right-handed. The majority of participants (84%) were medical students in their pre-clinical and clinical years, with 10% being non-medical students, and 6% being doctors in their first and second foundation years (FY1/2).

Fig. 1
figure 1

CONSORT flow diagram for the trial

Full size image
Table 1 Baseline participant characteristics
Full size table

Baseline scores

The pre-feedback OSATS scores were analyzed as baseline scores. Mean (± SD) pre-feedback checklist scores in the verbal feedback, video feedback, and self-assessment groups were 64.6% (± 16.9%), 58.2% (± 13.0%), and 61.3% (± 15.3%), respectively. Mean (± SD) pre-feedback global scores were 43.8% (± 10.2%), 40.1% (± 9.3%), and 41.5% (± 9.6%) in the verbal feedback, video feedback, and self-assessment groups, respectively. A one-way ANOVA comparison of pre-feedback scores showed no statistically significant differences in the pre-feedback checklist (F = 0.742, p = 0.482) and global scores (F = 0.628, p = 0.538) between the three groups. This indicates that the baseline scores were similar across all groups.

Intragroup pre- vs. post-feedback performance comparisons

Comparisons between pre-feedback and post-feedback scores within each group demonstrated statistically significant mean checklist and global score improvements in all three groups (Table 2). The video feedback group had the largest mean (± SD) checklist and global score improvements of 17.1% (± 9.9%) and 14.7% (± 9.3%), respectively, followed by the self-assessment group at 9.1% (± 15.1%) and 6.6% (± 9.4%). The verbal feedback group had the smallest mean (± SD) checklist and global score improvements of 9.0% (± 17.3%) and 4.6% (± 7.9%), respectively. These results demonstrate statistically significant improvements in the average task performance for all three groups, with video feedback having the largest margin of improvement, followed by self-assessment, then verbal feedback.

Table 2 Results of intragroup pre-feedback and post-feedback comparisons
Full size table

Intergroup performance improvement comparisons

Comparisons of mean score improvements between the three groups demonstrated statistically significant differences in global score improvements (F = 6.177 p = 0.004) but not checklist score improvements (F = 1.745, p = 0.186) (Fig. 2). Post hoc analyses of global score improvements demonstrated statistically significant differences between the video feedback and verbal feedback groups (p = 0.005), as well as between the video feedback and self-assessment groups (p = 0.029) (Fig. 3). However, the differences between mean global score improvements of the verbal feedback and self-assessment groups were not statistically significant (p = 0.779).

Fig. 2
figure 2

Checklist score improvements (mean and 95% confidence interval) and their intergroup comparisons

Full size image
Fig. 3
figure 3

Global score improvements (mean and 95% confidence interval) and their intergroup comparisons. *Statistically significant result (p value < 0.05)

Full size image

This shows that the differences in task improvement between groups were largely attributed to differences in global score improvements and not the task-specific checklist scores. The global score improvement was significantly greater in the video feedback group compared to the verbal feedback group but not the self-assessment group compared to the verbal feedback group.

Post-study questionnaire responses

Overall, participants reported an increase in confidence after the feedback session. Confidence levels were derived from the difference between participant-reported levels of confidence before and after the session. Participants also felt that they knew how to improve their performance in the suturing task between the pre-feedback and post-feedback assessment. All of the participants reported positive experiences regarding the teaching structure in the training program. Between the three groups, there were no significant differences (p > 0.05) in participant-perceived confidence of the suturing task, knowledge for improvements, or experience of the teaching structure (Table 3).

Table 3 Post-study questionnaire results based on the Likert Scale, 1: strongly disagree to 5: strongly agree
Full size table

Discussion

Laparoscopic suturing skills are difficult to acquire and opportunities to practice them in the operative setting are rare. Simulation provides an engaging platform for standardized practice and assessment, paving the way for effective feedback delivery. Audio–visual recordings of simulated tasks and procedures offer opportunities for reflection and feedback. All forms of feedback are valuable in surgical education, as seen from the improvement between pre-feedback and post-feedback performance across all three groups in our study. However, utilizing video technology to deliver video feedback and self-assessment may be more beneficial than using verbal feedback alone. Video feedback from an expert represents external feedback and self-assessment represents internal feedback. Both forms of feedback are important in the development of surgical skills. Many studies have demonstrated the benefits in terms of improvements in surgical skills performance parameters and trainee satisfaction levels. A systematic review of nine studies found that all but two studies reported significant knowledge gain from video-based education techniques and that the addition of video to simulator exercises has beneficial effects on training time, learning duration, acquisition of surgical skills, and trainee satisfaction [29]. The use of video-based technologies in surgical education as a whole has brought many significant benefits for the learning of surgical skills.

Video feedback is used effectively in a variety of sectors, ranging from sports to rehabilitation, which involve motor skills acquisition. Therefore, it is unsurprising that the results of this trial demonstrate the benefit of video feedback in learning surgical skills. In our study, the video feedback group (video review with expert verbal feedback) had the greatest improvement in task performance, and global score improvement was significantly greater than verbal feedback alone, indicating that expert feedback is more effective when delivered with the aid of a video review. Video feedback is likely to be more interactive than verbal feedback as it involves both visual and verbal cues, thus engagement with an expert during the video review can invoke greater reflection. Visual cues in video feedback can also allow trainees to better visualize the operative field during the video review, enabling greater visual–spatial awareness. Visual–spatial ability is reported as positively correlated with surgical skills performance in novices [30, 31].

Video review enables trainees to understand the process and progress of clinical procedures, as they are able to visualize the actions involved in the task in a stepwise manner. This makes video feedback a good teaching tool for acquiring procedural skills in novices with very little theoretical knowledge of the steps required for the procedure especially for complex tasks such as laparoscopic suturing. The repeatability of video review also means that feedback can be given at a later stage and even repeated to refresh the memory. This is also beneficial for self-assessment as trainees are able to replay their performances repeatedly to learn from their mistakes and track their progress independently. Therefore, both video review with expert feedback and with self-assessment maximize the learning opportunities of each practice session, which is especially useful at a time when exposure to surgical cases may be limited. Trainees can record their performance and subsequently review it with an expert or by themselves, which will improve self-directed learning on programs such as the FLS or LapPass courses.

The self-assessment group demonstrates a statistically significant improvement in suturing performance, although the improvement margin is not as large compared to the video feedback group and did not differ significantly compared to the verbal feedback group. This is likely due to the novices’ lack of experience and knowledge with the marking criteria and task requirements. Regardless, the potential for trainees to self-assess can be observed in this trial. With an average age of 22.4 years and the majority of participants being undergraduates or recently graduated, our participants are young novices with relatively little experience. Their ability to self-assess reflects their theoretical knowledge. In the review by Rizan et al., the accuracy of self-assessment is shown to increase with experience and age and Quick et al. showed, by comparing the abilities of residents from PGY 1 to 5, that the trainees’ abilities to self-assess improve with training progression [20, 32]. Our study demonstrates the benefit of utilizing video feedback and self-assessment for the acquisition of a surgical skill in a novice population who have not reached proficiency. This applies to medical students and junior doctors such as residents, trainees, and registrars. Likewise, other studies have also reported the benefits of video feedback and self-assessment specifically in surgical resident populations [16, 23].

The main limitation of this trial is that participants were given a short time to train and were not trained to proficiency. Comparisons of performance improvement between groups only elicited significant differences in the global components and not the task-specific checklist components. This is likely due to the steep learning curve of laparoscopic suturing and more repetitions are required to elicit a significant change in specific elements of the task rather than global elements [33, 34]. It would be useful to evaluate the long-term results of video feedback and self-assessment by analyzing trainees’ learning curves to achieve proficiency. However, the results from this trial are sufficient to detect a change in global performance improvements between groups despite a small sample and short training period. Future trials should include larger sample sizes examined over a longer period of time for training.

The post-study questionnaire did not manage to provide any significant results due to the short training time. Since the participant-reported outcomes are subjective, a more appropriate questionnaire design should include questions for participants to list the specific changes they have made when performing the task post feedback. This will provide a better measure of their knowledge of the task, therefore allowing us to analyze any improvements and compare these changes with the type of feedback given. Future study designs should incorporate a longer period of training time for participants to reach competency as an endpoint to elicit meaningful changes in the qualitative responses.

Conclusion

This study demonstrates the benefits of video-based technologies for feedback in terms of laparoscopic suturing skill acquisition and trainee satisfaction. By utilizing video-based technology, expert feedback and self-assessment can be combined and incorporated formally into the way feedback is delivered on surgical training curricula. This should improve self-directed learning and expedite the learning process. Combining expert video feedback and self-assessment should have the highest positive impact on trainees’ skills acquisition. It is recommended that both techniques should be considered for use in laparoscopic surgical training especially for learning advanced skills.