Introduction

Screening for colorectal cancer (CRC) has been widely performed, based on evidence that screening reduces the mortality from CRC [1–4]. There are several modalities of CRC screening, such as fecal occult blood testing [1, 2], colonoscopy [5, 6], and CT colonography [7]. A recent CRC screening guideline recommends that individuals over 50 years of age even without a family history of CRC should undergo CRC screening because they are considered at an average risk of CRC [8]. As the importance of CRC screening has been widely accepted, demands of CRC screening have increased. However, participation rates in CRC screening are low; nearly 40 % of the population in the USA has not been screened [9], and the situation in Japan is worse.

The advent of capsule endoscopy (CE) has dramatically changed the clinical management of small-intestinal diseases. A first generation of colon capsule endoscopy (CCE) procedure was developed in 2006 [10]. Recently, the technology has been improved and a second generation of CCE is available [11]. CCE makes it possible to visualize the entire colon without sedation and air insufflation. Therefore, CCE is considered a plausible option for CRC screening. In Japan, the second generation of CCE was approved by the Ministry of Health, Labour and Welfare in July, 2013 after a multicenter study concerning a new bowel preparation method for Japanese patients [12].

In order to fully utilize CCE as a CRC screening method, establishment of a training system to ensure reading competence of CCE videos has a pivotal role. Currently, training on CCE video reading is performed as a lecture on basic information and a hands-on seminar using several clinical case videos. However, such training methods are time-consuming and difficult to standardize. No formalized training system for CCE has been established. In addition, no standardized system to assess CCE reading competence is currently available.

We developed an electronic learning system for CCE (ELCCE), which was originally designed for the members of the Japanese Association for Capsule Endoscopy (JACE). All training modules are Internet-based and available to trainees at any time (Fig. 1). ELCCE also includes real clinical case examinations using a unique scoring engine, which objectively estimates CCE reading competence. The aims of the present study were to assess the utility of the ELCCE system using an objective scoring engine and to evaluate the efficacy of ELCCE on the acquisition of CCE reading competence.

Fig. 1
figure 1

Image of the JACE electronic learning system

Full size image

Methods

Electronic learning system of colon capsule endoscopy (ELCCE)

ELCCE includes seven steps as summarized in Fig. 2. Step 1 (introduction) provides an overview of CCE. Step 1 consists of six chapters. Participants learn overview of colon capsule endoscopy, including CCE system settings, indications and contraindications of CCE, epidemiology of colorectal cancer, current evidences on CCE, bowel preparation, and how to manipulate CCE review software. Step 2 is a pretest. Participants take clinical case examinations (CCE competence assessment test; CCAT) prior to the training. CCAT includes three full videos of typical clinical cases, as defined by colonoscopy and pathological examinations. Each participant’s CCAT report is automatically scored according to a unique scoring engine. Correct answers of the CCAT pretest are masked until the participant completes the entire training. Step 3 (learning reading theory) teaches participants how to review CCE videos and diagnose findings detected by CCE using short-segment video clips and multiple-choice questions. Step 3 consists of five chapters and an assessment test. The contents of each chapter is noted as follows; introduction of reading theory, how to conduct preview CCE video, how to review CCE video, how to analyze CCE video and make a final diagnosis report, and learning several typical polyps including flat and serrated polyps. After studying five chapters, participants take an assessment test. In order to proceed to step 4, participants have to mark more than 80 % of the assessment test. Step 4 (training CCE with guidance) uses one full video and eight short segment video clips with the guidance to train participants on CCE review. Participants are able to experience reviewing CCE videos and learn how to review CCE videos using clinical case videos. Step 5 (training CCE without guidance) uses full videos in a fixed order but without guidance. Participants review videos and complete procedural reporting in a structured format. When each video is reviewed, participants’ reports are scored automatically by the same scoring engine used in the CCAT. In order to proceed to step 6, each participant must continue reviewing case videos until his/her accumulated score reaches a target predetermined target (in the present study, the target score is 180). Step 6 (final assessment test) is the final clinical case examinations, which includes two full video clips and two short video clips. Each participant’s report is scored automatically by the same scoring engine used in the pretest CCAT. When a desired final score is reached in step 6, ELCCE is completed. After the completion of ELCCE, participants were requested to proceed to step 7 and take CCAT of step 2 again (when blinded to the results of step 2). The expected learning time in each step is as follows; 20 min in step 1, 1–2 h depending on trainee’s baseline CCE skill in step 2, 3 h in step 3, 3 h in step 4, 5–15 h depending on the trainee’s achievement level in step 5, 2 h in step 6, and 1–2 h in step 7.

Fig. 2
figure 2

Structure of the electronic learning system for colon capsule endoscopy (ELCCE)

Full size image

In order to ensure the safety of an Internet connection, the following security-related practices are implemented in ELCCE: (1) Authentication is enforced for all uses; (2) A user is automatically logged off when non-active; (3) Permissions policy allows each user to see only their own data; (4) All data is stored in a protected database and only authorized users have access; (5) User credentials are transferred between system components through secure local transcoding interface protocol.

CCE reading competence assessment test (CCAT)

The CCAT of steps 2 and 7 includes three anonymous full videos of typical clinical cases, as defined by colonoscopy and pathological examinations. Participants reviewed the videos and reported the results in the structured format originally designed for ELCCE. Participant reports were scored according to the following factors: lesion detection (overlooked, over called), diagnosis (location, size, shape of lesions), management recommendations (observe, colonoscopy, surgery), and cleansing level (adequate, inadequate). All lesions in both CCAT and ELCCE were carefully interpreted by experienced expert capsule reader doctors who were known to the colonoscopy diagnosis and findings. In terms of management recommendations and difficulty of diagnosis, it was made by expert doctors who committed a development of ELCCE. Cleaning level was assessed using a two-point grading scale (adequate or inadequate) by Leighton et al. [13]. The scoring system is summarized in Table 1. The scoring system was also used in steps 4–6. All participants took the same CCAT before and after completing the ELCCE reading training.

Table 1 CCE reading competence assessment test (CCAT) scoring criteria
Full size table

Participants

Ten trainee endoscopists were recruited as an initial testing cohort (cohort 1). Participants in cohort 1 were asked to finish ELCCE training within 7 weeks of recruitment. After completing the training, the participants sent feedback regarding the training and the ELCCE was modified based on their feedback. The modifications included minor optimizations of ELCCE and technical term correction. The contents of ELCCE and the CCAT were not changed. After the modifications, 11 new trainee’s endoscopists were recruited to validate the ELCCE (cohort 2). Participants in cohort 2 completed the updated ELCCE within 7 weeks of recruitment.

Statistics

All statistical analyses were performed using the Stata software version 13 (StataCorp, College Station, TX, USA). The main study outcome was the change in CCAT score in step 7 after ELCCE compared to the average CCAT score in step 2. The change in scores obtained during step 5 was also analyzed. In step 5, case video score series were categorized into three groups for each participant: the initial three cases, the last three cases, and intermediate between the initial and the last three cases. In order to assess the association between baseline CCE reading competence and the training effect of ELCCE, participants were divided into two groups according to prestest scores in step 2 (low- and high-score group). Accuracy during step 6 was compared between the low- and high-score groups. In all sample analyses, means were compared using an unpaired Student’s t test and frequency distributions were compared using the Fisher’s exact probability test or the Chi-square test, as appropriate. Changes in the accuracy of case video scores during ELCCE were assessed by linear regression analysis. A p value less than 0.05 was considered statistically significant.

Results

Baseline characteristics of the participants

Baseline characteristics of the study participants are summarized in Table 2. All participants experienced more than 250 colonoscopies and the median number of small bowel capsule endoscopies in cohorts 1 and 2 combined was 25 (range, 0–1000). However, participants had very little prior experience in CCE. Fifteen participants had no experience of CCE at enrollment. All but one participant (cohort 1) finished ELCCE within the study period. The total number of log-ins to the ELCCE Internet system per participant was median (range) 21 (8–53) for cohort 1 and 27 (10–42) for cohort 2. There was no significant difference in the total number of log-ins between cohorts 1 and 2.

Table 2 Previous colonoscopy and capsule endoscopy experience of study participants
Full size table

Comparison of CCAT scores before and after ELCCE

Following completion of ELCCE, average CCAT scores of step 7 significantly increased in both cohorts from 42 ± 18 % to 79 ± 15 % in cohort 1 (p = 0.0004), and from 52 ± 15 % to 79 ± 14 % in cohort 2 (p = 0.0003) (Fig. 3). There was no difference in CCAT scores between cohort 1 and cohort 2. All but one participant demonstrated an increase in CCAT scores after ELCCE.

Fig. 3
figure 3

Colon capsule endoscopy reading competence assessment test (CCAT) scores before (step 2) and after (step 7) electronic learning of colon capsule endoscopy (ELCCE); CCAT colon capsule endoscopy reading competence assessment test, ELCCE electronic learning system of colon capsule endoscopy

Full size image

Training effect on CCE reading competence in step 5

The participants needed to review 8–14 case videos, dependent on their reading performance, in order to reach the target score of 180 in step 5. The change in CCAT score percentages during step 5 is summarized in Fig. 4. The accuracy rate of case video scores gradually increased as participants continued case video studies. Cohort 1 obtained the following percentages: 48 ± 14 % in the initial three case videos, 64 ± 8 % in the intermediate case videos, and 81 ± 15 % in the last three case videos (p < 0.01). Cohort 2 showed the same trend as cohort 1: 45 ± 13 % in the initial three case videos, 69 ± 11 % in the intermediate case videos, and 79 ± 10 % in the last three case videos (p < 0.01). There was no difference in percentage of each categorization between cohort 1 and 2.

Fig. 4
figure 4

Change in CCAT scores during step 5

Full size image

Accuracy rate of final assessment test in step 6 and association between the accuracies in steps 2 and 7

The accuracy rate of the final assessment test in both cohorts 1 and 2 ranged from 53 to 98 %. Cohort 2 had a higher accuracy rate than cohort 1 (75.0 ± 12.6 % in cohort 1 versus 83.1 ± 12.6 % in cohort 2, p = 0.05). Figure 5 summarizes the average accuracy in step 6 scores between the low-score group (n = 11) and high-score group (n = 9) based on the step 2 pretest scores. No difference was shown in the accuracy rates during step 6 between low- and high-score group in step 2 (79.3 ± 13.8 % in the low-score group vs. 77.8 ± 15.2 % in the high-score group, p = 0.81). All participants obtained more than 50 % accuracy irrespective of their scores in step 2.

Fig. 5
figure 5

Association between accuracies in steps 2 and 6

Full size image

Discussion

Several articles have reported that computer-based training or virtual simulator models are useful for improved performance of upper gastrointestinal endoscopy [14] and colonoscopy [15, 16]. The efficacy of computer-based training for small bowel capsule endoscopy (SBCE) has already been reported [17], showing that competence of SBCE operators improved after training.

The current study implemented and validated a web-based training system for CCE and showed a significant improvement in CCE reading competence following training. ELCCE makes it possible for physicians to train on CCE anywhere and anytime. Moreover, because all trainees study the same content and must achieve minimum target scores in order to complete the training, ELCCE maximizes reading competence in CCE.

No sufficient evidence on how best to assess the reading competence of CCE operators has previously been published. Generally, capability of endoscopic diagnosis has been considered difficult to estimate precisely. ELCCE is a new assessment tool of CCE reading competence in which assessment of CCE reading capability is automatically conducted by the training software. Therefore, the system is able to estimate CCE reading competence objectively. Lai et al. reported that there are interobserver variations in the interpretation of CCE results among experienced reviewers and that a second reading by an experienced viewer might improve the diagnostic accuracy of the procedure [18]. However, the present study showed that it is possible to improve CCE reading competence by ELCCE, and this system will be widely distributed among CCE trainees in Japan.

In the present study, all but one participant increased CCE reading competence from the pretest of steps 2–7 after ELCCE irrespective of their scores in step 6, suggesting that ELCCE enhances CCE reading competence regardless of the prior experience level of the trainee. Indeed, the group that had the lowest scores in the step 2 pretest had similar scores in step 6 as the group with the highest scores in step 2. Thus, ELCCE achieves stable reading competence of CCE irrespective of initial CCE reading competence before the training. Interestingly, Postgate et al. also reported a similar finding using SBCE [17]. CCE reading competence should be demonstrated by accuracy of diagnosis rather than by the number of previous CCE procedures performed. ELCCE and CCAT make it possible for us to prove a certain level of CCE reading competence.

The study has several limitations. First, the association between CCAT score and capability of reviewing CCE videos in real-life settings still remains unclear. Unfortunately, there is no data on diagnostic capability of each participant after ELCCE. Theoretically, there is a positive correlation between CCAT score and capability of reviewing CCE videos, because CCAT is composed of two clinical CCE videos. As a matter of fact, it is difficult to compare CCAT score and capability of reviewing CCE videos after the training, because number of review in real-life setting is not equal among trainees. Trainees with larger number of experience after the training get better capability, compared with trainees with smaller number. In addition, it is very difficult to guarantee adequate competence of endoscopy including esophagogastroduodenoscopy, colonoscopy, and capsule endoscopy. It is a continuous process from trainees to experts. Interestingly, the study showed an increase in CCAT score after ELCCE, implicating that ELCCE improves capability of reviewing CCE.

Second, the number of participants was relatively small. Nonetheless, even with a small sample size, the study showed that all but one participant improved his/her CCAT scores, demonstrating an objective estimate of CCE reading competence.

Third, the study recruited only endoscopists (medical doctors). As the demands of CCE increase, the need for pre-reading by medical support staff may increase. Training to co-medical staff is indispensable to conduct CCE nationwide, because there are fewer doctors than co-medical staff. In Japan, the cost of CCE has been compensated by health insurance since 2014. In order to apply CCE as CRC screening, development of human resources for CCE review is required. In order to confirm and extend the present results, we are preparing a new study which recruits a large number of medical support staff.

Finally, the association between an increase in CCAT scores following training and CCE review expertise in daily practice is unclear. There is a paucity of evidence regarding how many clinical CCE reviews are needed to reach an expert level.

However, the ELCCE system includes nearly 30 actual clinical case videos, a volume that is not available to most trainees at their local institutions, and in fact, is not found in most Japanese hospitals. Therefore, ELCCE provides direct training based on actual and representative case examples.

In conclusion, ELCCE was designed for JACE and includes training videos and an automatic scoring engine, making it possible to both train users and objectively assess CCE reading competence at the same time. In the current study, the system improved CCE reading competence among trainee endoscopists. In addition, this web-based system is available to trainees anywhere and anytime. Following the completion of ELCCE, trainees are accredited to review CCE videos.