Competency-based training has become a mainstream approach in medical education, especially in procedural areas such as endoscopy and surgery. This approach goes beyond just the number of procedures that a trainee has to complete in order to be considered competent at performing endoscopy [1, 2]. Obtaining proficiency concurrently in technical and cognitive skills is an essential goal of every training program in endoscopy. However, training programs do not clearly define competencies that trainees have to achieve. To date, competency training and assessment in endoscopy mainly focus on basic fundamental endoscopic skills such as intubation skills and polypectomy [3,4,5,6].

Endoscopic submucosal dissection (ESD) is a therapeutic strategy developed in Japan over a decade ago to remove gastrointestinal neoplasms [7, 8]. It is now widely accepted as an effective strategy to treat patients with cancer or high-grade dysplasia. Using this method, selected neoplasms can be removed en bloc regardless of size and with low risk of cancer recurrence. However, precise dissection in the gastrointestinal submucosa, which is a few millimeters thick, requires advanced endoscopic skills. This procedure is associated with high rates of adverse events such as perforation and bleeding, especially during the initial phase of the learning curve [9, 10]. Adequate training and evaluation of trainee competency are essential to ensure high-quality performance and safe treatment of patients. The current training programs for ESD rely mainly on an apprenticeship model under the guidance of supervising staff, with a lack of standardization in training and assessment. Nowadays, Japan is a leading country in ESD training, and the training model in Japanese institutions starts with learning the basic knowledge of ESD, observing experts performing a certain number of ESD procedures, participating in ESD procedures as an assistant with graded responsibility for a variable number of cases, and finally progressing to perform ESD under the supervision of experts [11, 12]. During ESD training, a step-up approach is recommended, starting with selected cases that are technically easy and moving to more challenging cases based on size, the presence of ulcers, and location. However, there is no guideline as to where trainee skill level should be at each level of training, or how to determine whether trainees are ready to progress to the next level of training, which usually depends on supervisors’ subjective judgment. This type of apprenticeship training model is highly variable, lacking standardization and creates difficulty in ensuring that trainees have adequate exposure to all essential skills [13, 14]. A standardized comprehensive training curriculum is needed to train practicing endoscopists appropriately to acquire an acceptable level of competency.

In order to develop a formal training and assessment platform, there is a need to identify and explicitly define all essential skills required to competently perform ESD [15]. Competency involves an array of skills, including technical skills and cognitive functions (e.g., knowledge, judgment, decision-making, and communication skills). The present study aims to define and establish expert consensus on the essential competencies required to perform ESD safely and proficiently for gastric neoplasms.

Materials and methods

Study design

The methodology, based on Delphi principles, was conducted to establish experts’ consensus regarding the important competencies required to perform ESD safely and proficiently. The Delphi methodology is considered to be one of the ideal methods for gathering information among health care professionals, and is useful for achieving consensus on complicated issues. This technique allows a diverse group of experts to give their honest opinion [16,17,18]. Iterative rounds of questionnaires were sent, and the anonymous responses were analyzed and shared with the group after each round until predefined conditions for consensus were met. The study protocol was approved by the Institutional Review Board at McGill University.

Recruitment of ESD experts

Predefined inclusion criteria for expert selection included the following:

  • Individuals with more than 5 years of experience in independent practice of ESD at high volume centers in Japan;

  • Those who were identified as leading ESD performers, recognized by their contribution to education programs related to ESD within gastroenterological societies such as the Japan Gastroenterological Endoscopy Society, the Japanese Society of Gastroenterology, or the American Society for Gastrointestinal Endoscopy;

  • All of the experts had coaching experiences in ESD hands-on training courses endorsed by the Japan Gastroenterological Endoscopy Society.

A panel of 14 ESD experts from 12 institutions in Japan were invited to participate in this study. Participants were selected from diverse geographic locations and clinical backgrounds to obtain representation of varying practices.

Development of the initial item list

The authors, who are experienced in ESD, used task analysis to generate the initial item list. This methodology is recommended for the development of educational curricula aiming to teach learners complex skills [19, 20]. The ESD procedure was first deconstructed into a series of sequential steps. In addition, a review of the literature and textbooks was conducted to ensure that nothing was missed, and the new items found were added to the initial item list. The items were grouped according to the procedural steps. Two experienced ESD practitioners, with more than 5 years of experience in independent practice, and who have performed more than 100 ESD cases, were asked to review the initial list. The content of the initial list was modified and refined based on their opinions and suggestions through several iterations.

Delphi consensus

The initial item list was sent to the panel of ESD experts using an online survey (https://www.surveymonkey.com). Each survey was administered over a period of 3–4 weeks; a reminder was sent 2 weeks after each survey was sent. The participants were asked to rank each item in terms of importance as an indicator of competence in performing ESD using a Likert scale of 1 (no importance) to 10 (greatest importance). Using a free-text comment box, participants were asked to comment on any item or identify any additional items that they thought should be added to the list. Participants were also asked to complete a questionnaire including demographics and their clinical experience with ESD. The summarized anonymous results for each item from the previous round were provided to the participants as a mean score. Participants were then asked to re-rate the items that did not reach an agreement. After each round, three authors reviewed the results and added the newly generated items. The items and descriptors were refined and redundant items were combined according to the recommendations of the participants. The refined version was then sent back to the participants during the subsequent rounds and they were asked again to provide feedback on the content of the list. The survey was repeated until consensus was reached according to predefined criteria.

Statistical analysis and sample size

Consensus was predefined as 80% or more of the respondents scoring 8 or greater for an item on the 10-point Likert scale. We removed items from the list if they did not meet the predefined criteria two consecutive times. Results are reported as mean (standard deviation) and n (%). In general, a sample size of 10–18 experts is recommended for a Delphi panel [13, 17].

Results

Demographics of the Delphi survey participants

Fourteen ESD experts participated in this study. Fourteen respondents (100%) participated in the first round of the survey and 13 respondents (93%) participated in the second and third rounds of the survey. Among the participants, 13 (93%) have performed more than 300 ESD cases. The demographics and clinical experience of the ESD experts are shown in Table 1.

Table 1 Demographics of the panel of experts participating in the Delphi survey (n = 14)
Full size table

Results of the Delphi survey

Twenty-nine items were generated in the initial list. Ten additional items were added to the list by the experts after the first round of the survey. Two redundant items were combined into one item according to expert opinions. Consensus was established after three rounds of surveys. Of the 39 items, 34 achieved consensus as essential surrogates of competency in performing ESD (Table 2). Table 3 shows the final consensus results, ordered according to the importance level. The maximum score of 10 was given to two items: “General: Stabilizes manipulation of the scope” and “After ESD: Evaluates the pathological finding of the resected specimen and makes treatment plans.” The item “General: Optimizes view” was rated as second most important followed by “Diagnosis: Makes proper diagnosis of the disease range.” Twenty-two items were ranked greater than 9 on the 10-point Likert scale by all of the participants. The items are categorized into 10 sections of the procedure: general (n = 11), diagnosis (n = 1), mucosal marking (n = 1), submucosal injection (n = 4), mucosal incision (n = 3), trimming of the mucosa (n = 2), submucosal dissection (n = 4), hemostasis (n = 2), in case of perforation (n = 2), after ESD (n = 4).

Table 2 Items reached consensus for important surrogates of competency in ESD
Full size table
Table 3 Delphi consensus results for all items, ordered according to importance by final consensus
Full size table

Discussion

Traditionally, case numbers have been used as a surrogate to determine competency in ESD. In the literature, the number of cases thought to be required to achieve competency varies from 30 to 80 cases [21,22,23]. This wide range could be explained by the inconsistency of the types of outcomes used in the literature (e.g., en bloc resection rate, adverse event rates, total procedure time) and complexity of the cases (e.g., size, location, existence of ulcer). The existing data linking competency to case numbers are not well established. The learning curve for trainees is highly variable and thus case numbers cannot in themselves be used to determine competency. A measure of competency should be more accurate, specific and objective. It is important to identify essential indicators of competency required to perform ESD safely and efficiently.

This study is the first attempt at eliciting the essential competencies required to perform ESD. The newly developed list comprises a series of steps essential to perform ESD. The experts reached consensus on 11 general items and 23 procedural steps. Each item has a descriptor to standardize/clarify the competency meant to be assessed. The descriptors are defined in detail with the aim to serve as guidance in teaching or assessing ESD. Many of the assessment tools for endoscopic techniques focus on technical aspects of the procedures. However, training for cognitive functions such as knowledge, communication skills, or clinical judgment also significantly contributes to patient safety and clinical outcomes [24, 25]. Our developed list includes descriptors for technical, cognitive, and integrative (combination of technical and cognitive elements) skills. For example, descriptors include the sequential steps of the procedure that operators should follow, tips for error avoidance, and knowledge for efficiently performing the procedure. It is common that there are some procedural differences that exist among experts. This list may not cover all of the little steps of the procedure, but it includes all of the key items that are consistent among all experts, regardless of their educational background, demographic, or institution.

The aim of this study was to develop a training and assessment tool. In order to develop a comprehensive tool with properly established validity evidence, we started with the tool content, generated through Delphi methodology. Content validity is one of the five sources of the current framework of validity [26], and is the degree to which items of the assessment tool represent variables that are essential to the target construct. Obtaining content validity is also the first step to develop an educational curriculum. Delphi surveys, through iterative rounds among experts, have been used to identify the content areas [13, 18]. The competency list developed in this study can be used as a blueprint to develop an objective ESD assessment tool for summative or formative purposes, through direct observation or video assessment, in both clinical and simulation settings. The integration of an objective assessment tool with established validity evidence is essential as it allows continuous monitoring of a trainee’s progress, provides structured evaluations, constructive feedback, and direction for future learning [13, 27]. An ESD assessment tool was developed using the competency list by our group and studies to address the other sources of validity evidence are in progress. These sources including response process, internal structure, relationship to other variables, and consequences. One of these approaches would be to investigate the correlation between our assessment tool and existing endoscopy metrics; this would be addressing relationship to other variables.

During endoscopy training, as the complexity of the procedure increases, trainees require more detailed guidance and feedback. However, experts may not always provide adequate and specific feedback. One of the reasons is that, as physicians gain competence, their skills become automated and the tasks are performed mostly without conscious awareness [28]. Several experts who participated in this study commented that their participation was a good opportunity for them to organize their knowledge of ESD and verbalize the procedure process clearly. The item list generated in this study could help educators design a curriculum to teach essential principals of the procedure and give practicing endoscopists detailed feedback, reflecting the needs defined by the experts. In addition, the list outlines the main components of the ESD procedure. Therefore, it could be used by the trainees as a self-learning tool to guide them to learn the procedure.

Among the top 10 rated items (Table 3), 4 items were fundamental endoscopic skills which can be learned through basic endoscopic training; ‘Stabilizes manipulation of the scope,’ ‘Optimizes view,’ ‘Makes proper diagnosis of the disease range,’ and ‘Maintains a clear view of endoscopic field.’ Therefore, first, training should strongly emphasize and verify expertise with the more basic skills. Three other items out of the 10 items were related to dealing with adverse events of ESD; ‘In case of perforation: Closes the perforation site,’ ‘In case of perforation: Manipulates the device sequentially and properly,’ and ‘Hemostasis: Identifies the bleeding spot.’ The basics of these techniques can also be taught through polypectomy or EMR training. It is mandatory to obtain the basics of these skills before starting training of ESD to ensure patient safety. However, most of the existing ESD training courses focus on teaching techniques specific to ESD, providing lectures on the basics of ESD, following a live demonstration of ESD either on patients or porcine models and/or hands-on training of ESD using porcine models [29,30,31]. Perforation rates among participants with a mean of more than 10 years of endoscopic experience in the West and Asia were reported as high as 19–65% after 2–3-day ESD courses using live porcine models during gastric ESD [11, 29, 30]. On the other hand, perforation rates for gastric ESD in Japan in real patients, which are more complex than porcine models, are much lower (3–6%). Most authors have reported that the majority of the perforations occurred during their ESD learning curve and some data include the period as far back as the late 1990s when ESD started to gain popularity in Japan [22, 32, 33]. These results suggest that fundamental endoscopic training may differ between Japan and other countries. To develop an effective ESD curriculum, it is important to clarify the gap that may exist in endoscopic training programs. The list we developed includes important fundamental skills required to perform ESD. Thus, a plan to focus on those skills may help clarify and compensate for the gap that may exist between the countries. It would also be important to take into account the participants’ endoscopic competencies, not just by their prior endoscopic experience in terms of years of practice or cases performed, but also by the objective verification of their fundamental endoscopic proficiency using well-established assessment tools such as Global Assessment of Gastrointestinal Endoscopy Skills [3], the Mayo Colonoscopy Skills Assessment Tool [6], Gastrointestinal Endoscopy Competency Assessment Tool [18], or Direct Observation of Polypectomy Skills [5]. Development of a curriculum based on trainees’ competency level could improve trainees’ skills effectively.

This competency list was developed for gastric ESD. However, many of these principles can be used as a baseline to develop a competency list of ESD for other gastrointestinal organs such as esophagus, colon, and rectum, which are technically more challenging than stomach, but have been developed based on gastric ESD. Further investigations are needed to clarify additional competencies for those more technically challenging cases.

A limitation of this study is that the panel of experts who participated in the Delphi survey included only Japanese experts. Japan is the leading country for ESD and many endoscopists from other countries visit Japan to learn the techniques. In addition, many Japanese experts participate in ESD educational programs all over the world. Therefore, it was logical to include Japanese experts for the survey panel. Majority of the experts were male, which reflects the gender distribution of experts in advanced endoscopy in Japan. Further studies are necessary to ensure the results of this study can be generalized to other countries and ESD practitioners.

For the items and descriptors, where procedural differences existed among experts, such as the place of the markings (e.g., either 3 or 5 mm outside the lesion margins), we adopted words such as ‘appropriately,’ ‘properly,’ or ‘adequately’ to make the content concise. However, these words are subjective and have a risk of misinterpretation by the users. We clarified in detail the critical aspects of the procedure like the dissection plane of the submucosa by precisely documenting the correct plane that should be dissected. This study is the first step to clarify the competencies for ESD. Our next step is to develop educational videos to further explain each item in detail so that the users are able to understand the meaning of the terms clearly.

In conclusion, as the demand for effective ESD training curriculum has increased, there is a need for detailed characterization of the competencies required to perform ESD. This study reports the results of an expert consensus on the essential competencies required to perform ESD for gastric neoplasms. This competency list can help promote the development of more effective and comprehensive ESD curricula and form the basis of an effective assessment method for educators, and could also serve as an anchor for trainees to learn and understand ESD procedure effectively.