Keywords

1 Introduction

Trying to improve the statistics (colo-rectal cancer being the second leading cause of deaths in United States [1]), one of the main steps is to certify the high quality of the medical procedures. Different evaluation standards for bowel preparation degree exist in actual medical practice all over the world. Two of the main standards used in medical practice are: the Boston Bowel Preparation Scale (BBPS) [2] and, more recently the Chicago Bowel Preparation Scale (CBPS) [3]. They mainly rely on practitioner experience and subjective evaluation. In order to be more accurate, some automatic attempts in computing the degree of bowel cleansing using colonoscopy video frames have been described in the last decades [4, 5]. These studies used the RGB color space, while we have approached it using the La*b* color space [6, 7], which allows a better and simpler (two-dimensional) color localization [8], making easier to find related colors and to estimate the covered areas. In our previous studies we used a video colonoscopy record with 5100 selected frames.

We continued our study on 17 normal video colonoscopies with diverse pathologies and a colonoscopy with Narrow Band Imaging, in order to identify new peculiarities impeding or alleviating fast and reliable automatic evaluation of video colonoscpies.

2 From Video Colonoscopy to Image Processing

Video colonoscopies together with occult tests are standard analyses recommended worldwide to early detect colorectal cancer and to decrease mortality, according to the actual statistics [9,10,11,12,13]. The colonoscopy has the advantage of permitting local resection and further biopsy of the abnormal polyps or adenomas which are detected. The quality of the colonoscopy and the results of the medical procedure depend primarily on two factors: the expert skills and the proper bowls preparation.

2.1 Boston Bowel Preparation Scale

The video colonoscopy is a laborious procedure, sometimes necessitating sedation, mainly for old people or for patients who were previously submitted to abdomino-pelvic surgery, sometimes being even at risk due to this [14].

The endoscope with the video camera (or, for the new types, cameras [15]) passes through the normal segments of the colon having a different temporization on its way to the terminal ileum and backwards.

Usually, the left segment (rectum, sigmoid, descending colon) has to be browsed in 30% of the insertion phase, transverse colon 30% and right segment 40% (ascending colon, then cecum, towards the terminal ileum). Backwards, experts agree that the withdrawal phase has a different temporisation: 30% right segment, 30% for transverse colon and 40% for the left segment of the colon. The withdrawal phase should last at least 6 min.

Main colon segments cleansing has to be observed, international standards suggesting marks (or ratings) some of them being quite linguistically flue, without standardized definitions [2].

Thus, in Boston Bowel Preparation Scale (BBPS), which is a “10-point scale assessing bowel preparation after all cleansing manoeuvres are completed by the endoscopist” [2], the physician has have to note if the cleansing degree is “poor” or “unsatisfactory”, “fair”, “good” or “excellent”. Some remarks have to be done: it was not clear in BBPC [2] if the quality linguistic ratings have to be given in the insertion phase or in the withdrawal phase as they differ. The former give more clues about the purgative substance quality, while the latter, after washing and suctioning the fluids on the covered surfaces, might be a value also referring to the polyps’ detection rate.

A correspondence to a scale of four-point marks was established [2] (for “more objective” scoring, yet still upon the practitioner appreciation, with intra and inter-practitioner variability), for each colon segment, in the withdrawal phase: 0 is a mark for unprepared colon, covered with solid faecal materials impossible to be cleaned; 1 is a mark given for partial covered areas of the colon segments, with solid residues or opaque liquid; 2 is corresponding to minor residues staining, yet mucosa of the colon segment is visible; 3 when the entire colon segment mucosa is well seen, no residual staining, no fragments, no opaque liquid.

The marks are given for every segment (from 0 to 3) and then summed, thus the final score will be in the range 0–9.

Note: A Korean study states that the BBPS is usually inversely correlated with the colonoscope withdrawal time [16], which is directly proportional with adenoma detection rate [17].

2.2 Pitfalls in Image Acquisition

Trying to clean the covered bowels, the fluids are modifying the images, resulting in a specific video acquisition. This justifies the necessity for image preprocessing and for computer aided systems procedures.

A video colonoscopy might last up to 20 min upon its complexity, yet a duration of 13–17 min is more usual. We might have 40–50000 frames in a split video, and we have chosen a rate of 1/10 images (a satisfactory amount of information which implies an acceptable amount of computing).

The problem arises when we study these frames, a lot of them being blurred, covered with water, reflections and shadows, white speckles and an excessive amount of light.

In order to process the image content we obviously have to select good images, normal/abnormal frames, issue that was the subject of some previous publications [6, 18,19,20,21].

The following figure shows normal, good images among the blurred, non-informative ones, irrelevant for further evaluations and time consuming if we continue to maintain them in the image data base that we have to select. It was often proven that good results rely on good data sets. Discarding the ambiguous, irrelevant data conducts to unexpected improvement for the expected results.

Our method is a primary type of discarding useless images, based on sequential evaluation of image entropy on quarters of the colon picture situated inside. Entropy is well known in physics as a measure of disorder: still environment, “calm” background, is a sign of very low entropy, thus conducting to low quantity of information.

2.3 Informative Versus Non-informative Frames

In the following figure we illustrate the entropy variation on the images selected from the video colonoscopy. We selected a medium quality image (half is blurred, half contains some information about the stool covered areas). The picture given by the colonoscope is analyzed by quarters in a first stage, from upper left side in the hourly sense and a graphic is plotted, showing in different colors each value for each frame.

On the Ox axes we have the number of the selected frames, from the original video colonoscopy, on the Oy axes we have the values corresponding to the entropies computed in Matlab. Very low entropies correspond to totally inutile images as observed in Figs. 3, and 4. Obviously, the selection procedure has to be further enhanced, as sometimes even a half of the image might contain an important informative clue.

Fig. 1.
figure 1

Split video colonoscopy: good (valid) and (second raw) blurred (irrelevant) frames

Full size image
Fig. 2.
figure 2

Image 154/5000 video colonoscopy frames, fair to low quality of the image, characterized by medium entropy, computed for each of the four quadrants, starting from upper left magenta, upper right cyan, lower right blue, lower left yellow, anti-trigonometric sense.

Full size image
Fig. 3.
figure 3

Image 997/5000, very low entropy, poor quality, frame to be discarded

Full size image
Fig. 4.
figure 4

Image 3704/5000, totally irrelevant, very low entropy, discarded frame

Full size image

After this first step more than 10% of the frames have been discarded. Further the procedures differ depending on the purpose we aim to. In order to automatically compute the stool covered areas, which would be more objective than the endoscopist appreciation, subject of variability, the range of colors corresponding to the person (depending on the personal nutrition habits and bladder/liver health or disease) has to be established. Due to luminance it is more easy to work in the La*b* space than in RGB. If our purpose is to automatically detect polyps, adenomas or diverticula, another strategy has to be conceived [6, 31].

2.4 Using La*b* Color Space

La*b* color space is organized along a lightness axe (L*), with perpendicular color planes arranged over a green–red (a*) axe and a blue–yellow (b*) axe, [6,7,8].

The results of La*b* color selection on video colonoscopy frames, are illustrated in Figs. 5 and 6: stool in red, and white light and speckles selection in blue.

Fig. 5.
figure 5

Segmentation results in normal images, obtained by the La*b* method: red-stool presence, blue-white speckles areas

Full size image
Fig. 6.
figure 6

Segmentation on a partly-blurred image: stool covered areas (red), and light (blue)

Full size image
Fig. 7.
figure 7

Narrow Band Imaging (NBI) colonoscopy frames: colors are modified, blood vessels are dark green to brown, inner vessels are cyan, stool traces and some tissues appears in red

Full size image

Counting frame by frame for each colon segment, and reporting to the background surface we might have the objective values we search for.

We will further describe the steps for the software computing of the covered areas. The software implementation in order to objectively compute a score on the Boston Bowel Preparation Scale (BBPS) consists of:

  • the three main colon segments are marked by the endoscopist on the video sequence;

  • the stool color range is identified for every colonoscopy by inspecting relevant frames;

  • the colonoscopy video is parsed frame by frame;

  • entropy is computed frame by frame;

  • low entropy frames are discarded;

  • the stool covered area is computed for each frame and a percent of covering is stored for each frame;

  • stool covered area S is summed;

  • S is compared to the uncovered surface;

  • based on the BBPS and the stored percent of stool covered area, with a special aggregation method, a number is assigned to each segment of the colon;

  • in the event of an extremely high ratio of stool presence in a frame, the corresponding image is stored as evidence of non-conformity to the standards, and its influence on the final score is increased by a reinforcement algorithm;

  • results: aggregation is made, using an impact coefficient for the extreme cases (entirely covered frames);

  • otherwise, each segment BBPS objectively evaluated is computed for the final evaluation.

2.5 Narrow Band Imaging (NBI)

A new approach for colonoscopy consists in using NBI technique [22,23,24,25,26,27,28,29,30].

NBI is relying on light penetration properties, directly proportional to the wavelength. Blue light (415 nm) enhances the visualization of superficial mucosal capillaries while green light (540 nm) increases the visibility of submucosal and mucosal vessels” corresponding to the “primary and secondary light absorption peaks of hemoglobin, respectively” [15, 26, 30].

NBI completely modifies the range of colors in the intestinal observation, with clear advantages for evidentiating blood vessels and adenoma/polyp textures. Citing the OLYMPUS, NBI technology productor [30], “Capillaries in the superficial mucosa appear brown in 415 nm wavelength. The 540-nm wavelength penetrates slightly more deeply into the mucosa and submucosa and makes the deeper veins appear blue-green (cyan). Because most of the NBI light is absorbed by the blood vessels in the mucosa, the resulting images emphasize the blood vessels in sharp contrast with the nonvascular structures in the mucosa” [30].

We are questioning if this technique is also aleviating the automatic BBPS evaluation on video colonoscopy frames?

NBI sends blue and green light on the colon surface and for some reason, stool debris reflects the red wavelength, as well as some membrane components which also turn red under this light, This way, our method had difficulties in separating normal tissue color in NBI from the stool covered areas colors, even if the endoscopist slightly manage to explain the difference, and the results that we primarily obtained for software computing were not encouraging, thus the answer seams to be negative.

Yet, research in this field is in continuous progress, numerous scientific research teams concentrating on this challenge with a big impact on human’s health.

An annual competition is taking place, MICCAI 2019, ENDOVIS, Endoscopic Vision Challenge [32].

More data bases are publically available as CVC Colon DB [33] and ASU MAYO DB [34] together with the previous one, providing data for the competition [32].

3 Conclusions

Previous attempts of automatic Boston Bowel Preparation Score computing have been made using RGB color space, characterized by difficulties in assigning a color 3D volume to be detected, corresponding for stool presence in colon. We developed a complementary method in the LAB color space, which offers an algorithm easy to compute, results being obtained faster. This procedure can be applied to the video recordings, saving time and facilitating the computer-assisted analysis of the cleansing aspects, relevant for colonoscopy. For the NBI technique the results are very promising for vessels and tissue/polyp texture identification, while the BBPS score is better obtained in the normal colonoscopy video frames.