Keywords

1 Introduction

As the advancement of the healthcare and increased longevity, many countries had gradually become the aging societies. This results in an increase of patients with chronic diseases, such as diabetes, hypertension, and hyperlipidemia. In addition to these well-known diseases, chronic kidney disease (CKD) is also a prevalent chronic disease nowadays. According to a previous study [1], there were about 12% of the citizens in Taiwan had CKD and there were only 3.5% of them aware of their diseases. In order to improve the care for CKD patients in Taiwan and decrease the medical expense, the National Health Insurance Administration launched a comprehensive care program for early CKD patients since 2011 to facilitate the early detection and treatment for CKD. Even though, there is still some CKD patients having poor disease control due to that the etiology of the underlying kidney disease is unknown after routine laboratory survey. For these patients, the clinician may suggest renal biopsy for further evaluation.

Renal biopsy is a valuable and irreplaceable diagnostic tool for renal diseases. For patients with unknown cause CKD, heavy proteinuria and unexplainable rapid renal function deterioration, the clinicians may suggest renal biopsy, an invasive examination for further investigation. After local anesthesia, the clinician will obtain a small proportion of kidney tissue with needle biopsy gun under real-time sonography guide. The renal specimen will receive a series of preprocessing and staining techniques before it can be read by pathologist with microscopy. In general, there are three different kinds of image studies, including light microscopy, immunofluorescene, and electron microscopy, need to be thoroughly evaluated before the final diagnosis is made. Among these studies, light microscopy is the most fundamental one and can reveal much important information such as the basic microstructure of the kidney.

In general, the human kidney is composed of four basic microstructures, including glomerulus, renal tubule, interstitial tissue and vessel. Each component plays a different role in the control of renal regulatory mechanism. Glomerulus is the basic functional unit for waste and excess water removal. The appearance of the glomeruli on light microscopic image can provide abundant information for disease diagnosis. Besides, the number of the glomeruli in a renal specimen is also an indicator of the sampling quality. For a sample with only limited glomeruli, the value for further interpretation is minimal. Besides, the appearance of glomeruli alone provides a clue for the underlying renal disease and long-term outcome. During the process of renal pathological image interpretation, the glomeruli definitely play a crucial role.

Due to the importance of glomeruli on accurate renal disease diagnosis, many scholars had devoted themselves to the study of glomerular classification and detection [2,3,4,5,6]. The early approach in this topic is the combination of handcrafted features with a classifier. Yoshihiro et al. proposed the combination of Rectangular Histogram of Oriented Gradient (R-HOG) features with linear support vector machine (SVM) classifiers to detect glomeruli [2]. The dataset used were whole slide renal pathological images of Spontaneously Diabetic Torii (SDT) rats [7]. A new descriptor called Segmental HOG (S-HOG) which can be adaptively fitted to input renal images and perform a comprehensive detection of glomeruli in kidney sections images was proposed in [3]. The result showed that the S-HOG method achieved the higher F-measure of 0.866 in comparison with 0.838 of the R-HOG-based method in the rat dataset. A novel system was proposed to detect the renal corpuscle objects and measure the glomerulus diameter and Bowman’s space width of renal albino rats in [4]. The system used the particles analyzer technique to extract three descriptors (aspect ratio, circularity, and solidity) to detect glomerulus on green channel images.

Due to the rise of big data, deep learning models have been widely used in medical image diagnosis in recent years. Different from hand-crafted features, deep learning techniques enable machines to learn the best features by themselves to classify or detect objects [8, 9]. Gallego et al. applied Convolutional Neural Networks (CNNs) to classify human glomerulus and non-glomerulus patches manually cropped from whole slide images (WSI) [5]. The CNN-based classification approach was further applied to detect glomeruli in WSI by using a sliding window with overlapping [6]. Detection was performed on WSI stained with Periodic Acid Schiff (PAS) at 20x magnification. A total of 98 WSI (with around 4000 glomeruli) and 10 WSI (with 275 glomeruli) were used for training and test, respectively.

This paper considers the detection of human glomeruli in whole light microscopy images. The images were stained with PAS or hematoxylin and eosin (H&E) and taken at 100x, 200x, or 400x optical magnification. The variations in glomeruli colors due to different stains and sizes due to different magnifications making the detection task much more challenging than that studied in [6]. The faster region-based convolutional neural network (R-CNN) is a popular method for end-to-end objection detection with different sizes and shapes [10, 11]. This paper applies the faster R-CNN to this detection task. Experimental results show that an F-measure of 0.889 is achieved for the detection of 563 glomeruli in 482 test images with two stains and three magnifications using a single faster R-CNN.

This paper is organized as follows. Section 2 introduces the collection of training and test data. Section 3 introduces glomerulus detection using the faster R-CNN. Section 4 presents experimental results. Finally, Sect. 4 presents the conclusions.

2 Materials

This study was approved by the Institutional Review Board of Taichung Veterans General Hospital, Taiwan (TCVGH-1073605D). In this study, we use renal biopsy pathological images collected for the purpose of routine case discussion during 2011 to 2017 in Taichung Veterans General Hospital, Taiwan. The images were taken with Olympus Digital Camera System and the image resolution was 4080 × 3072 pixels. We choose light microscopy images with H&E and PAS stains as the material in this study. The images were taken with optical magnifications of 100x, 200x, or 400x on light microscopy. Figure 1 shows the appearances of renal pathology images with different stains and magnifications. In general, the PAS stain can highlight the basement membrane of glomeruli and renal tubules. Images with different stain methods and different magnifications can provide different information to the pathologist. Table 1 summarizes the numbers of images collected for training, validation, and test. The total numbers of glomeruli in the training and test images are 3956 and 563, respectively.

Fig. 1.
figure 1

Appearances of whole renal pathology images of different stains and resolutions, where the arrows indicate the locations of glomeruli and the black bounding boxes are manually annotated glomeruli.

Full size image
Table 1. The distribution and numbers of renal biopsy pathological images collected for training, validation, and test datasets.
Full size table

Figure 2 shows different patches in the images. A normal glomerulus in the light microscopy images is roughly circular on pathological images, as shown in Fig. 2(a). It is observed that the glomerulus is surrounded by Bowman’s capsule, and this unique feature makes glomerulus isolated from other structures. In addition to glomeruli, there are other circular structures in the biopsy images, including renal tubules and blood vessels, as shown in Fig. 2(b), which may cause the model to misinterpret as false positives. If glomeruli have abnormalities, such as global/segmental glomerulosclerosis and crescentic glomerulonephritis, as shown in Fig. 2(c), the shape of glomerular changes, which may result in difficulty for the model to identify it as the glomerulus (false negative). Overall, the variations in glomerulus structures, stains, and sizes in the images make the automatic detection of glomeruli a challenging task.

Fig. 2.
figure 2

Patches of (a) normal glomerulus, (b) other structures that mimic glomerulus, and (c) abnormal glomerulus in the dataset.

Full size image

3 Glomerulus Detection Using Faster R-CNN

3.1 Structure of Faster R-CNN

This section introduces the use of a single faster R-CNN to automate the detection of glomeruli in the whole images with H&E or PAS stain at different magnifications. Figure 3 shows the structure of the faster R-CNN used in the glomerulus detection task. The VGG-16 model is utilized in the faster R-CNN. The feature extraction part in the VGG-16 consists of 13 convolution layers and 4 pooling layers. The classification part of the VGG-16 consists of three fully connected layers. The region proposal network (RPN) is performed on each 3x3 spatial sliding window over the last convolution layer (feature maps) of the VGG-16. The RPN provides the classification scores and region coordinates of proposed objects in the feature maps. The region of interest (ROI) pooling layer adjusts the dimension of the feature maps to ensure a constant dimension size to the fully connected layer. In the end, the score and position adjustments are generated by the fully connected layer.

Fig. 3.
figure 3

The architecture of the faster R-CNN, where the VGG16 model is used.

Full size image

In the collection of the training dataset of the faster R-CNN, only the positive rectangular patches (glomeruli) need to be annotated in the images. Figure 1 shows some of the annotated glomeruli in the whole images with different stains and magnifications. The faster R-CNN automatically and randomly samples negative patches (non-glomerulus) from the whole image, which greatly reduces the effort on data collection. Both the PAS and H&E stained images are collected in the training dataset so that the single faster R-CNN can detect glomeruli in the images with different stains. The sizes of the glomeruli in the images taken with the same optical magnification are similar but have a large variation in the images with different magnifications, as shown in Fig. 1. For this problem, the training dataset also mixes the images at the magnifications of 100x, 200x, and 400x so that the single faster R-CNN is robust in detecting glomeruli with different sizes from different magnifications.

3.2 Training of Faster R-CNN

The technique of transfer learning is used. The transfer learning technique uses a pre-trained model and fine tunes it based on the training dataset of a new classification task. This technique has shown its merit in model training, especially for the classification and detection problems when the available training data is insufficient such as medical images. This technique makes it possible to obtain a good model with a small number of data and is, therefore, used in the glomerulus detection problem. In this paper, the VGG-16 pre-trained using the 2014 ImageNet Large Scale Visual Recognition Challenge (ILSVRC) dataset is utilized. Due to the use of transfer learning, the weights in the convolution and fully connected layers are fine tuned. On this side, the stochastic gradient descent is used. The learning rate and the momentum coefficient are set to 0.001 and 0.9, respectively. The total iteration number is set to 30000. These coefficients are selected according to the performance of the validation dataset.

4 Experimental Result and Discussions

This section presents the experimental result using the faster R-CNN. To evaluate the performance of the proposed detection method, the recall (R) and the precision (P) were used. The R measure is defined as the ratio between the number of glomeruli correctly detected (true positives) and the actual number of glomeruli. The P measure is defined as the ratio between the detected glomeruli (true positives) and the total number of detections (true positives + false positives). In addition, the following F-measure (\( F_{1} \)) [6] was also used to measure the detection performance

$$ F_{1} = 2\frac{R \cdot P}{R + P} $$
(1)

The R, P, and \( F_{1} \) were 91.54%, 86.50%, and 0.889, respectively. Figure 4 shows some of the correct detections from the whole images with H&E or PAS stains at the three magnifications. The single faster R-CNN succeeded in detecting most glomeruli in the images without a priori information of the stains and magnifications. It should be emphasized that the training and test datasets were different from those in [6] and contained higher variations in the glomerulus colors (two stains) and sizes (three magnifications). The window-based scanning method in [6] cannot be directly applied to detect glomeruli with such variations in sizes.

Fig. 4.
figure 4

The correction detections using the faster R-CNN, where the yellow boxes indicate the true positives. (Color figure online)

Full size image

To analyze the false detections, Table 2 shows the numbers of false positives and false negatives distributed over the images with different stains and magnifications. The result showed that the false positives were nearly evenly distributed over different stains and magnifications with no bias to a particular type of image. Most of the false negatives occurred at the images with 100x and 200x magnifications with no bias to the stains. The images with 400x magnification showed a smaller number of false negatives mainly because of the large glomerulus size in the image. Figure 5(a) shows examples of the false negatives, where it was observed that some of them were at the image boundaries and some were abnormal in structure. Figure 5(b) shows examples of the false positives, where some of them were glomerulus-like in structure and some of them were caused by improper detection sizes.

Table 2. The test result (#false positives/#false negatives) distributed over the images with two stains and three magnifications by using the faster R-CNN.
Full size table
Fig. 5.
figure 5

(a) False negatives (red boxes) and (b) false positives (blue boxes) in the images, where the yellow boxes indicate the true positives. (Color figure online)

Full size image

5 Conclusion

In this study, we have demonstrated the feasibility of automatic glomerulus detection on renal pathology images with different stains and magnifications using deep learning. The use of a single faster R-CNN succeeds in detecting most glomeruli in the images with H&E or PAS stains at 100x, 200x, or 400 magnifications. In the future, the collection of more abnormal glomeruli in the training dataset to improve the detection rate will be studied. To automate the whole medical diagnosis process, deep-learning based automatic disease diagnosis following the detection of the glomeruli from the renal pathology images will be studied as well. The proposed approach may improve the efficiency of the pathological diagnosis process if widely used in the clinical settings in the future.