For more than 20 years, Advanced Gastrointestinal/Minimally Invasive Surgery (Adv GI/MIS) fellowships have offered specialized training to general surgery graduates who seek additional experience using MIS approaches to treat complex GI pathology [1]. These fellowships are not governed by the Accreditation Council for Graduate Medical Education (ACGME) and instead, fall under the jurisdiction of the fellowship council (FC). The FC is a robust organization that oversees more than 150 programs with more than 200 fellowship positions in a variety of specialties including Advanced GI, Adv GI/MIS, Bariatric, Flexible Endoscopy, Hepatopancreaticobiliary (HPB), non-ACGME Colorectal, and non-ACGME Thoracic [2]. To ensure high-quality experiences for its fellows, the FC partners with national surgical societies or expert consortiums to develop educational curricula and accreditation criteria for each of these aforementioned fellowship types [3].

The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) is responsible for defining the content associated with Adv GI/MIS fellowships. Historically, these criteria have been relatively broad and prioritized MIS approaches rather than specific anatomic areas or disease processes. As a result, individual programs were fairly heterogeneous with respect to the complement of cases offered to fellows. This heterogeneity had its advantages; namely, fellows had the opportunity to tailor their experience to areas personal of interest. However, as MIS techniques have become increasingly prevalent across all surgical disciplines, SAGES recognized the need to retool these fellowships to focus on disease-based content that aligns with the mission and expertise of SAGES as an organization.

In an effort to better define core content of the Adv GI/MIS fellowships, SAGES thoughtfully developed a proposal for new educational content and accreditation criteria, including minimum case volumes within six defined categories (foregut, bariatric, inguinal hernia, ventral hernia, HPB/solid organ/colorectal/thoracic, and flexible endoscopy) as well as a novel video-based assessment to evaluate competency in core domains. Full details of the methods used to develop these criteria have been published previously [4]. The purpose of this project was to conduct a pilot study that would allow prospective data collection and subsequent analysis to investigate feasibility and generalizability of the proposed criteria.

Methods

Adv GI/MIS Fellowship Programs Directors (PDs) who also held leadership roles in SAGES were invited to participate in the pilot. Fourteen programs including 17 fellows volunteered (Table 1). The primary outcomes were whether the new defined category minimums could be feasibly achieved by pilot fellows and whether pilot fellows’ data could be generalized to represent non-pilot programs. To assess feasibility of the new criteria, pilot programs’ 2017–2018 case logs were compared to 3 years (2012–2015) of historical fellows’ case logs (n = 326). An increase in the proportion of pilot fellows achieving a defined category minimum compared to historical controls was considered evidence of feasibility. To assess generalizability, pilot programs’ case logs from the academic year prior to the launch of the pilot (2016–2017) were compared to all other Advanced GI/MIS programs (n = 92).

Table 1 Adv GI/MIS fellowship pilot programs
Full size table

Importantly, Adv GI/MIS fellows have historically received credit for all cases performed regardless of role [i.e., teaching assistant (TA), primary surgeon, first assistant], whereas the pilot criteria stipulated that fellows only received credit for defined category cases performed as TA or primary surgeon. Thus, in this analysis, historical fellows (2012–2017) received credit for all cases logged, whereas during the pilot year (2017–2018), fellows only received defined category credit for cases performed as TA or primary surgeon.

During the pilot year, fellows’ case logs reports were compiled at 3, 6, and 12 months and assessed against the proposed defined category minimums. To judge whether fellows were on pace to meet the new minimums, case volumes from the 3-month and 6-month reports were extrapolated to 12-month volumes. For purposes of feedback, each PD received a detailed report on his or her fellows’ performance at each of these intervals, and the group also received de-identified summary data for all pilot fellows. Additionally, SAGES leadership held regular meetings and conference calls with pilot program PDs to discuss progress and troubleshoot issues.

Case volumes are reported as median (IQR) unless otherwise indicated. The proportion of fellows achieving defined category minimums were compared using Fisher’s exact tests. The volume of defined category cases logged were compared with Mann–Whitney U Tests. An α < 0.05 was considered statistically significant. All analyses were performed using Microsoft Excel 2016© with the Data Analysis ToolPak. This was a minimal risk study and was determined exempt from IRB review.

Results

For the pilot year, complete data were available for 16 fellows (one fellow was excluded due to a medical leave of absence). Based on the 12-month data, pilot program fellows were significantly more likely to meet defined category minimum for foregut cases compared to historical controls. Regarding bariatrics, inguinal hernia, ventral hernia, and endoscopy, pilot fellows demonstrated non-significant trends toward higher achievement of these defined category minimums. Pilot fellows were significantly less likely to meet the minimum for HPB/solid organ/colorectal/thoracic cases. These results are displayed in Table 2.

Table 2 Proportion of pilot fellows achieving defined category minimums versus historical controls
Full size table

Pilot fellows logged a median of 251.5 (IQR 217–310) advanced MIS cases and 62.5 (IQR 44–94) endoscopies during their fellowship year. Pilot fellows performed a significantly higher volume of foregut cases and a significantly lower volume of HPB/solid organ/colorectal/thoracic cases compared to controls. Case volumes for the bariatric, inguinal hernia, ventral hernia, and endoscopy categories were not significantly different between groups. The median volumes within MIS defined categories are displayed in Table 3. Review of pilot fellows’ case logs also uncovered a substantial volume of open procedures performed by this cohort. Overall, pilot fellows performed a median of 35.0 (IQR 22–49) open cases during their fellowship year.

Table 3 Comparison of case volumes between groups
Full size table

During this pilot study, SAGES also planned to evaluate the feasibility of a video-based assessment of fellows’ ability to perform a laparoscopic Nissen fundoplication. The methodology of this video-based assessment is still being refined through work done in SAGES committees. Thus, this component of the pilot study could not be included.

In the academic year immediately prior to enrolling in the pilot (2016–2017), there were no significant differences in the achievement of defined category minimums when pilot program fellows (n = 17) were compared to non-pilot program fellows (n = 92). Pilot program fellows showed non-significant trends toward higher achievement in the foregut category and lower achievement in the HPB/solid organ/colon/thoracic category compared to non-pilot program fellows. The two groups demonstrated very similar achievement levels in the bariatric, inguinal hernia, ventral hernia, and endoscopy categories. These results are displayed in Table 4.

Table 4 Proportion of fellows achieving defined category minimums in 2016–2017 academic year
Full size table

Based on these data, SAGES leadership eliminated the HPB/solid organ/colon/thoracic category from the proposed criteria. Because of the significant volume of open cases performed by pilot fellows, SAGES leadership also decided to allow up to 15 open cases to count toward a fellow’s overall goal of 150 cases. The revised criteria along with a three-year staged integration plan were approved by the SAGES Resident and Fellow Training (RAFT) Committee and Board of Governors.

The criteria and implementation plans were then reviewed and approved by the FC Education Committee, Accreditation Committee, and Board of Governors. In collaboration, SAGES and the FC issued a communication to all Program Directors announcing the approval and adoption of these new criteria. For FC program accreditation there will be a two-year grace period in which cases will be tracked according to the revised criteria, but these criteria will not be used for accreditation until the 2020–2021 academic cycle. The revised criteria approved for implementation, are displayed in Table 5. For issuance of SAGES certificates [5] to program graduates, the criteria will be implemented in a staged fashion over the ensuing three academic years. Details of the staged implementation are displayed in Fig. 1.

Table 5 Comparison of pilot criteria and revised criteria
Full size table
Fig. 1
figure 1

Timeline for implementation of new criteria for graduates applying for SAGES certificate

Full size image

Discussion

Almost 3 years ago, SAGES recognized the need to retool the Adv GI/MIS fellowship to meet the educational needs of its fellows. The proposed pilot criteria grew out of work done by the SAGES RAFT Committee based on historical case log data from 2012 to 2015. Full details of the development of the pilot criteria are documented in a prior publication [4]. Our current paper reviews the results of a 1-year pilot study designed to generate feasibility and generalizability evidence to support the implementation of the proposed criteria.

With regard to feasibility, the cohort of Adv GI/MIS fellows from 2012 to 2015 was used as a historical control group. This historical cohort was chosen because these same data were initially used to develop the proposed case log criteria [4]. Additionally, while the historical cohort received credit for all defined category cases logged regardless of surgeon role (TA, primary surgeon, first assistant), the pilot fellows only received credit for defined category cases performed as TA or primary surgeons. Although pilot fellows case logs were held to more stringent criteria than historical controls, we felt that this analysis was appropriate to judge the feasibility of both the proposed defined categories and the new restrictions on surgeon role. Results from the pilot demonstrated trends toward improved achievement of the proposed minimums in all proposed categories except the HPB/solid organ/colorectal/thoracic category. While the reasons for poor achievement of this latter category criteria are not entirely clear, we suspect that fellow may have seen progressively decreased exposure to these case types since the historical data was compiled.

Critics may argue that although there were observed improvements in all other categories, these improvements are not necessarily sufficient to declare that the defined categories are “feasible” for all programs. Specifically, some may argue that the 70–80% achievement rates in the categories of endoscopy, inguinal hernia, and ventral hernia are concerning. SAGES recognizes that there is still work to be done to bring all Adv GI/MIS fellowships up to these new standards; however, the minimum threshold in each of the three aforementioned categories is ten cases. This amounts to less than one case per month during the fellowship year and is felt to be a more than reasonable target even among programs that need to make adjustments to meet the standard. By setting the expectation that programs meet these new requirements within 3 years, SAGES has given significant leeway to allow programs to maintain accreditation while still pushing programs to raise the bar in certain key content areas so that all fellows will have a common core experience during their training.

Since the pilot cohort was made up of volunteer programs, we were concerned that selection bias could limit generalizability of results. To assess whether the 14 pilot programs were in fact representative the non-pilot Adv GI/MIS programs, we compared achievement of the proposed defined category minimums in the year prior to the launch of the pilot for these two groups. This comparison was chosen due to a suspicion that pilot fellows and PD’s would change their case mix and logging practices during the pilot year due to the Hawthorne effect and their desire to fulfill the proposed criteria. Thus, assessing the similarity of pilot and non-pilot programs immediately prior to the launch of the pilot was felt to provide a less biased measure of generalizability. The results of this analysis showed no significant differences in these two groups’ achievement of defined category minimums. These data support the generalizability of the trends seen among pilot programs to the entire cohort of Adv GI/MIS fellowships.

The revised criteria represent a major step forward in the modernization of the Adv GI/MIS fellowships. When the FC fellowships were first established in 1997, MIS techniques were in their infancy, and criteria for accreditation logically prioritized MIS approaches over individual case types [6,7,8]. Knowing that MIS surgery has evolved and become more prevalent, SAGES recognized the need to modernize the fellowship criteria to prioritize disease-based content. With the new criteria approved, the Adv GI/MIS fellowship now represents a core experience in foregut diseases, metabolic surgery, abdominal wall hernias, and flexible endoscopy. In addition to this core, the remaining required cases—which represent more than half of the total experience—can be accrued from a variety of complex case categories. By continuing to allow flexibility in a large portion of fellows’ cases, SAGES has preserved the potential for fellows to tailor their experiences to areas of individual interest, which has been and will continue to be a valuable element of the Adv GI/MIS training paradigm [7, 8].

Furthermore, based on the substantial volume of open cases done by pilot fellows, the revised criteria approved for implementation now allow for up to 15 complex open cases to count for credit toward the total of 150 cases needed during fellowship. These cases must be complex cases within the foregut, bariatric, or hernia content areas to count. Although these cases do NOT count toward defined category minimums (all defined category cases must be performed with MIS approaches), they reward programs who provide exposure to open surgery in the defined core content areas. Indeed, SAGES recognizes that Adv GI/MIS graduates will encounter patients who are too complex for MIS approaches, and as such, well-rounded training in advanced GI disease processes should include exposure to complex open surgery.

Overall, the results of this pilot were quite helpful in establishing revised criteria for Adv GI/MIS fellowships. While these criteria may be challenging for some programs to meet, it is anticipated that they will offer a standardized framework that will enhance the educational focus and benefit of these fellowships [9]. All graduates will be expected to have a foundation of knowledge and skills in a core set of diseases, thus improving consistency. As a result, graduates will be well positioned to both enter broad general surgery practices as well as offer advanced treatment options in several areas [10]. Additionally, the standardized framework will allow these fellowships to seamlessly incorporate new educational methods [11,12,13,14,15]. For instance, the FC is developing Entrustable Professional Activities (EPA’s) for the content covered by all of its fellowship types [16]. The Adv GI/MIS fellowships will be able to apply these new methodologies toward the core content defined by the revised criteria. Thus, the educational content of these fellowships is expected to continue to evolve in a positive fashion with the implementation of new curricular components and modern methods of assessment.

SAGES and the FC worked together in a collaborative fashion to openly discuss the new criteria, review the pilot study results, obtain input from representatives from numerous societies, and agree on their appropriateness. Both organizations realized the need to implement the revised criteria in a staged fashion in an effort to allow programs to modify rotations and educational experiences; it is expected that programs should be able to meet these criteria by the end of the three-year implementation period.

Conclusion

This pilot study provided feasibility and generalizability evidence that allowed inclusion of appropriate defined categories for establishment of the new Adv GI/MIS fellowship criteria. We anticipate that the revised criteria will enhance the educational benefit of these fellowships.