Introduction

Microscopic examination of Peripheral Blood Smear (PBS) is a gold-standard procedure practiced by pathologists over the years to diagnose hematological disorders [1]. Evaluation of blood cells namely erythrocytes, leukocytes and platelets reveals overall health status of a person. Leukocytes are of five types namely lymphocytes, monocytes, neutrophils, eosinophils and basophils as shown in Fig. 1. They vary in size, shape and color. Total leukocyte count and the differential leukocyte count help in diagnosing and monitoring many disorders. For example, excess of immature monocytes or lymphocytes in peripheral blood may indicate leukemia which is a deadly disease. Increased number of eosinophils may indicate allergic reactions. Therefore, differential count of leukocytes plays an important role in diagnosing diseases such as leukemia, eosinophilia, neutropenia, neutrophilia and basophilic disorders, etc.

Fig. 1
figure 1

Leukocyte types; a neutrophil, b eosinophil, c basophil, d monocyte, e lymphocyte

Full size image

Laboratories usually carry out two types of evaluation based on blood count namely Complete Blood Count (CBC) and Differential Count (DC). CBC is a test which provides counts of all the blood cells namely erythrocytes, leukocytes and platelets whereas DC provides counts of each type of leukocyte. An expert observes around 100 microscopic views of a stained blood smear and gives the percentage count of each type of leukocyte. The result of DC may depend upon skill of the person performing the test. Therefore, manual method ends-up in subjective results [2]. Automated method of detection and counting of leukocytes would reduce the burden on hematologists. Also, this eliminates the subjectivity in lab reports. Detection of leukocytes plays an important role for accurate counting of each type of leukocytes. Detection of leukocytes depend on accurate segmentation to study the nucleus and cytoplasm appropriately. Hence, segmentation plays a vital role in the process of automating laboratory procedure [3, 4].

Many methods have been proposed to automate detection of leukocytes using image processing approach. Most of the methods follow the image processing pipeline of pre-processing, segmentation, feature extraction and classification. Many research groups used thresholding method [5,6,7,8,9,10,11] for segmentation of nuclei from PBS images. Nuclei appear darker compared to the other regions in PBS images. Hence, authors concluded that nuclei can be detected by simple thresholding method. Der-Chen et al. [5] used Otsu’s automatic thresholding method by considering RGB and HSV color space representations for segmentation of leukocytes which yielded average Dice score of 0.96. Further, classification of leukocytes was carried out by using genetic algorithm based k-means clustering approach and classification accuracy in the range of 81%- 98% was reported in this study. Madhloom et al. [6] employed image arithmetic and automatic thresholding method on grayscale images of normal cases for detection of nuclei and reported the accuracy between 85% and 98%. Use of image arithmetic operation and thresholding method on grayscale image was also employed in [12,13,14,15] for segmentation of leukocytes to detect acute leukemia with average accuracy between 88% and 93%. Putzu et al. [7] used Zack’s thresholding for background removal and detection of leukocytes considering CMYK and CIE LAB color representations. Otsu’s thresholding method was considered for detection of region of nuclei. Further, separation of grouped leukocytes was considered using watershed segmentation method. Authors used a total of 108 images from ALL-IDB1 dataset and reported the average detection accuracy of 92%. Hayan et al. [16] considered thresholding and morphological operations for segmentation of lymphoblast cells. The authors used 180 images with at least one lymphoblast and sub-images having single cell for segmentation of nucleus and cytoplasm. Segmentation accuracy between 90% and 95% was reported in this study. Yang et al. [8] employed logic operations such as AND and XOR segmentation of leukocytes. Color space representations namely HSI and RGB were used for segmentation. Though average accuracy of segmentation of 97.7% was reported, only 10 representative images were used in their study. Dorini et al. [9] used morphological operations, watershed segmentation and scale-space analysis for segmentation of leukocytes into nucleus and cytoplasm by considering 100 images of normal cases. Li et al. [10] proposed dual-thresholding method to improve the efficiency of segmentation of region of nuclei using RGB, HSV and CIE LAB color space representations. Though the method achieves good Dice score of around 0.97, color and brightness variations were not addressed. Also, the method used sub-images of single leukocyte. Safuan et al. [11] reported segmentation of leukocytes using thresholding method applied on various color space representations such as RGB, CMYK and HSV.

Several methods have been proposed using k-means clustering for segmentation of leukocytes [17,18,19,20,21,22]. Zhang et al. [21] used color based k-means clustering. Also, images of different backgrounds were normalized to match resultant image with a template image. They reported segmentation accuracy of 95.7% and 91.3% for detection of nuclei and cytoplasm respectively. Moradi et al. [22] used k-means clustering and SVM for detection of lymphoblasts. Sajjad et al. [18] proposed a smart phone based cloud assisted method for classification of leukocytes using color based k-means clustering, there by reporting segmentation accuracy around 96%. Soltanzadeh et al. [17] employed k-means clustering and morphological filters for detection of nuclei. Further, an attempt was made to detect nucleoli using curvelet transform. The study reported the use of 100 PBS images and also reported sensitivity around 90% and 84% for detection of nuclei and nucleoli respectively. A method for leukocyte segmentation which is a combination of k-means clustering and watershed transform was proposed by Ghane et al. [19]. Color space representations such as CIE LAB and CMYK were used for the segmentation, thereby obtained similarity measure of 92% and 93% for detection of nuclei and leukocytes respectively. Liu et al. [20] also proposed a method for leukocyte segmentation based on watershed transform and mean shift clustering techniques. Segmentation method was framed using various color representations namely RGB, HSI and CMYK which resulted in overall F-measure of 99% for segmentation of leukocytes. Though the clustering method provides the various regions of similar characteristics, it may lead to over and/or under-segmentation due to overlap of color characteristics of types of leukocytes.

Shape and size variations can be observed depending on the type of leukocytes. Hence, many research groups utilized active contours method which is robust to shape and size of objects [23,24,25,26]. Yang et al. [23] used GVF snake algorithm for segmentation of leukocytes and reported average accuracy of around 94%. A combination of GVF snake algorithm and Zack’s thresholding was used by Sadeghian et al. [25] and reported accuracy of nuclei segmentation around 92%. Rezatofighi et al. [24] used Gram-Schmidt orthogonalization with snake algorithm for nuclei and cytoplasm segmentation. They considered 400 samples in their study and reported the accuracy of nuclei segmentation around 93%. Active contours method without edges was used for segmentation of leukocytes into nuclei and cytoplasm by Mathur et al. [27]. Further, region splitting algorithm was used to count number of lobes of nucleus. Chul et al. [26] employed meanshift clustering for detection of nuclei and GVF snake algorithm for detection of leukocytes, thereby obtaining average segmentation accuracy of 88%. The study reported the use of 100 images consisting of 260 leukocytes. Enas et al. [4] used support vector machine for segmentation of leukocytes. This study reported the use of 100 microscopic images. The authors reported an average accuracy of 95%. All these methods either considered images acquired under controlled environment or images with less variations.

Though a lot of efforts have been made to automate leukocyte analysis, there is a need for robust method which can handle variations present in PBS images. Also, there is need for simple method which finds practical application even in low resource settings. In this study, an automated method for detection of nuclei and leukocyte is proposed. Threshold based method is considered for segmentation of nuclei. A robust method for detection of leukocytes using active contours method is presented. Contributions of the paper are listed as follows.

  1. 1.

    A novel image enhancement method is presented to increase the contrast between background and region of nuclei.

  2. 2.

    Accurate nuclei detection method using a combination of arithmetic operation and Zack’s thresholding is presented.

  3. 3.

    Adaptive approach for generation of masks based on the size and shape of five types of leukocytes for active contours method is proposed.

Materials and methods

In this section, details of data collection, nuclei detection and WBC detection methods are given. The main aim of the proposed method is to detect leukocytes even in the presence of color shade variations. The method is divided into two parts namely detection of nuclei and detection and extraction of leukocytes.

Presence of dark nucleus is the distinguishing features of WBCs in PBS images. Nuclei appear dark compared to the other objects in PBS images. Shape of the nuclei vary from round to lobulated based on the type of WBCs. Cytoplasm regions around the nuclei are pale. These regions consist of various types of granules. Color and density of granules vary among the types of WBCs. These features were used to detect WBCs in the proposed method.

Data collection

To demonstrate the performance of the proposed method to handle images from various sources, two datasets were considered in the study. The first dataset includes images acquired from hematology laboratory, Kasturba Medical College hospital, Manipal, India. Images were acquired using OLYMPUS CX31 microscope considering magnification of 100X with resolution of 1600X1200. The dataset consists of images of different color and brightness variations as shown in Fig. 2. The images contain leukocytes, erythrocytes and platelets. In the present study, 119 microscopic images of normal cases were considered.

Fig. 2
figure 2

Sample images of the dataset

Full size image

The second dataset includes images from ALL-IDB2, a public dataset [28]. We considered a total of 54 images of normal cases from ALL-IDB dataset.

Detection of nuclei

The block diagram representation of the proposed nucleus detection and leukocyte detection method is shown in Fig. 3. The steps followed for detection of nuclei are shown in block A which includes sub-blocks from (a) to (d).

Fig. 3
figure 3

Block diagram of nuclei and WBC detection

Full size image

Nucleus is the dark region of leukocyte in a peripheral blood smear image. Detection of nuclei was considered as the first step in the proposed method of detection of leukocyte. Arithmetic operations as explained by [6] was used for detection of nuclei in the proposed method. Arithmetic operations as explained by [6] will be referred to as Image arithmetic in this paper. Arithmetic operations were applied on grayscale representation of original images and Otsu’s automatic thresholding method was used for nucleus detection in Image arithmetic.

Microscopic images suffer from illumination variations. Therefore, image enhancement plays an important role during segmentation of leukocytes [29]. Hence, a novel image enhancement method was proposed which was used in combination with Image arithmetic to improve the contrast of nuclei compared to the background. Image enhancement was achieved by eliminating R component of input image. We replaced R and G component by normalized G component and B component by normalized B component which is referred to as G’G’B representation as shown in (Fig. 3 (b)). The arithmetic operation and minimum filter of size (6, 6) as per Image arithmetic were applied on the enhanced G’G’B images. The result of minimum filter is a grayscale image which was an approximate region of nucleus as depicted in Fig. 3(c). It can be observed that along with region of nucleus, platelets and small portion of cytoplasm also appeared as a result of minimum filter.

Automatic Otsu’s thresholding method was used in Image arithmetic to obtain nucleus region. In the proposed method, Zack’s thresholding method was used to select appropriate threshold value for detection of nucleus from the grayscale image obtained as a result of minimum filter. It is a histogram based threshold selection method. The histogram of minimum filter output was considered for identifying the threshold using Zack’s thresholding method. In this method, a line is constructed between maximum histogram value and minimum histogram value. The distance is computed between the line and the histogram values. The intensity value corresponding to the maximum is selected as the threshold value [25]. This value changes adaptively based on intensity levels of the filtered image. The obtained threshold value was applied on output of minimum filter which results in binary representation of nucleus. The binary image consists of region of nucleus, platelets and staining artifacts. Further, morphological closing operation with disk shaped structuring element of radius 10 was used to connect lobulated nuclei. Unwanted regions were eliminated using morphological opening and area filter. Results of nuclei detection were evaluated by comparing with the ground truth obtained from an expert.

Detection and extraction of leukocytes

Active contours were used for detection of leukocytes in the proposed method. Active contours work on the basis of energy minimization technique. It is a flexible curve which can adapt to the required object boundaries. It is an iterative process where user has to specify an initial contour and the number of iterations [30]. Kass et al. [31] introduced active contours algorithm with three energy functions as given in eq. (1).

$$ E={\int}_0^1\left[{E}_{int}\left(v(s)\right)+{E}_{img}\left(v(s)\right)+{E}_{con}\left(v(s)\right)\right] ds $$
(1)

where, v(s) = (x(s), y(s)) is the contour of an object, Eint is internal energy, Eimg is image force and Econ is external constraint force.

Internal energy is the sum of elasticity and stiffness, defined such that it is minimum at object boundary. It helps in smoothing the curve. Image energy is calculated from image pixel values, so that it takes small values at the boundaries. The third term in eq. (1) is a constraint which controls external force and also helps in placing the contour near the desired local minimum. The shape and size of leukocytes vary depending on the type. These variations should be taken care for accurate detection of WBCs. Active contour method with proper initialization of the contours can detect the object accurately irrespective of the shape of the object. Therefore, we used active contours with adaptive mask for initialization of the contours to detect and extract leukocytes which is robust to shape and size variations.

The intensity value of the background was found to vary from 240 to 255 depending on lighting conditions of the images. It was not feasible to use a single threshold value for the entire dataset. Hence, background removal was considered using histogram of grayscale representation of original images. Background removed image was the input image used in active contours as shown in Fig. 3 (f). Removal of background makes active contours to converge at the exact boundary of WBCs. For the efficient convergence of active contours, masks were adaptively generated to suit the size and shape of various types of leukocytes. To automatically classify the types of leukocytes, geometrical features namely area and circularity of the nuclei were computed. Circularity reaches the value 1, if the shape of the nucleus is circular. It can be computed using the formula as given in Eq. 2.

$$ Circularity=\frac{Perimeter^2}{4\ast \pi \ast Area} $$
(2)

These nuclei were approximately classified as lobulated, small or large based on the area and circularity. Convex hull area of nucleus was dilated depending on approximate classification of nucleus as depicted in Fig. 3 (e). Disk shaped structuring elements of sizes between 30 and 70 were used to obtain appropriate masks to initiate the active contours. The combination of dilated convex hull of nucleus and active contours detects leukocytes correctly irrespective of lighting conditions of images.

The steps followed for nuclei and leukocyte detection are as follows.

  1. 1.

    Extract R, G and B channels of input image I

  2. 2.

    Apply contrast stretching on G component to obtain G’

  3. 3.

    Replace R and G components by G’

  4. 4.

    Combine G’, G’ and B to obtain G’G’B representation of RGB images IG’G’B (as shown in Fig. 3 (b))

  5. 5.

    Convert IG’G’B images to grayscale images Igray

  6. 6.

    Apply arithmetic operation on Igray as per Image arithmetic (as shown in Fig. 3 (c))

  7. 7.

    Obtain binary representation of Igray using Zack thresholding method Ibw (as shown in Fig. 3 (d))

  8. 8.

    Use Ibw as mask on I to obtain appropriate nuclei region

  9. 9.

    Calculate area and circularity of nuclei obtained in step 6

  10. 10.

    Dilate nuclei (Ibw) using disk shaped structuring element to obtain Imask

    1. (i)

      For area < 40,000 and circularity >0.75, radius of the disk = 30

    2. (ii)

      For area < 50,000 and circularity <0.55, radius of the disk = 55

    3. (iii)

      For other cases, radius of the disk = 70

  11. 11.

    Use Imask as mask for active contours to initiate the contours (as shown in Fig. 3 (e))

  12. 12.

    Use histogram based thresholding to remove background of grayscale representation of I (as shown in Fig. 3 (f))

12a. Determine pixel value ‘p’ at maximum peak.

12b. Replace pixel value > = p by zero

  1. 13.

    Use mask obtained in step 9 on background removed images obtained in step 11 to obtain Iwbc

  2. 14.

    Use Iwbc as mask on original image to obtain leukocyte

Results

In this section, results of nuclei detection and leukocyte detection are demonstrated. Performance of nuclei detection method is given in section “Performance of nuclei detection method”. Also, comparison of results of nuclei detection method with two state-of-the-art methods are provided. Results of leukocyte detection method of the proposed method are given in section “Results of leukocyte detection” and results are compared with that of three state-of-the-art methods. To demonstrate the robustness of the proposed nuclei detection method, it was applied on images from ALL-IDB dataset.

Performance of nuclei detection method

Appearance of leukocytes changes with illumination variations. The results of nuclei detection are illustrated in Fig. 4; expert annotated images are shown in row (a) and images in row (b) are results of the nuclei detection method. As shown, segmented regions of nuclei closely match with that of expert annotated images.

Fig. 4
figure 4

Results of nuclei detection method; row a expert annotated images, row b detected nuclei using the proposed method, row c nuclei detection results as per Image arithmetic, row d nuclei detection results as per TissueQuant method

Full size image

Results of the proposed method were compared with that of the Image arithmetic and the method proposed in [32] which is referred to as TissueQuant method in this paper. TissueQuant tool for color image analysis was demonstrated by Keerthana et al. [32]. TissueQuant is a color image segmentation method which selects a particular color and its shades in an efficient manner. It converts given color image to HSI color space representation and Gaussian weighting functions are applied on each component of the color space in scoring colors based on how much they differ from the selected color. Gaussian weighting function assigns higher score to the color shades which are closer to the selected color. Lower scores are assigned to color shades which are distant from the selected color based on the width of the Gaussian weighing functions and other color shades get score of zero.

In Fig. 4, Images in row (c) are results of nuclei detection as per Image arithmetic and images in row (d) are results of nuclei detection as per TissueQuant method. The detected nuclei as per TissueQuant method also match with that of expert annotated images, but failed to detect the region of nuclei in the case of low contrast at the boundaries of the nuclei. The detected nuclei as per Image arithmetic match with expert annotated images in most of the cases, but the method failed in case of presence of dark regions around the nuclei which is as shown in first column of Fig. 4(c).

Results were evaluated by computing mean and standard deviation (STDEV) of Similarity Measures (SM) namely Dice score, False Positive rate (FP), Sensitivity and Hausdorff Distance (HD). Similarity measures were also computed for the methods proposed in Image arithmetic and TissueQuant method. The similarity measures for the proposed method and methods as per Image arithmetic and TissueQuant method are listed in Table 1. It can be observed that the proposed method performs better compared to both the Image arithmetic and TissueQuant method.

Table 1 Mean and standard deviations of similarity measures of nuclei detection for our dataset
Full size table

The Dice score plots for the proposed method and method as per Image arithmetic are shown in Fig. 5(a). The variation in Dice score is much less for the proposed method. This indicates that, the segmented nuclei using the proposed method match well with the expert annotated images.

Fig. 5
figure 5

Dice score plot, a comparison of the proposed method with method as per Image arithmetic, b comparison of the proposed method with method as per TissueQuant method

Full size image

The comparison of the proposed method with method as per TissueQuant method for Dice score measure is illustrated in Fig. 5(b). It can be observed from the plot that; the Dice score values remain above 0.9 for the proposed method. Also, Dice score values for the proposed method is around 0.99 for majority of the images. In the case of TissueQuant method it varies between 0.85 and 0.98. However, variation in Dice score using the method as per TissueQuant method is smaller compared to that of method proposed in Image arithmetic.

To evaluate the performance of the proposed method, we considered 54 images of normal cases from ALL-IDB2 dataset. The mean and standard deviation values of similarity measures for the dataset are given in Table 2. Segmentation accuracy around 99% was obtained for detection of nuclei for images of ALL-IDB2 dataset. This could be due to less color and illumination variations in the images compared to the images of our dataset. Overall Dice score of 0.97 was obtained for both the datasets. Thus the proposed method is robust to address the color and illumination variations.

Table 2 Mean and standard deviation of similarity measures of nuclei detection for ALL-IDB2 dataset
Full size table

Results of leukocyte detection

Active contours method was used for detection of leukocytes. The dataset contains images of different color shades as mentioned in section “Materials and methods”. The proposed method addressed color and illumination variations in detection of leukocytes. The results of leukocyte detection are shown in Fig. 6; upper row in the figure corresponds to expert annotated images and lower row images are results of the proposed method. Active contours method successfully detected leukocytes in almost all the images. Over-detection can be observed in a few images with overlapped cells as shown in Fig. 6.

Fig. 6
figure 6

Results of leukocyte detection method; upper row expert annotated images, bottom row leukocyte detection using the proposed method

Full size image

Results of leukocyte detection were evaluated by computing similarity measures as listed in Table 3. Average DSC around 0.96 and average sensitivity around 0.94 was obtained for detection of leukocytes. We also considered the performance of the proposed method for ALL-IDB2 dataset. The similarity measures are as listed in Table 3. It can be observed from the table that, Dice score and sensitivity are above 0.97 for ALL-IDB2 dataset. This indicates that the proposed method can handle the variations in PBS images.

Table 3 Mean and standard deviation of similarity measures of leukocyte detection method
Full size table

We compared the results of leukocyte detection of the proposed method with four state-of-the-art methods as shown in Table 4. It can be observed from the table that, the performance of the proposed method for leukocyte detection is higher compared to the state-of-the-art methods.

Table 4 Comparison of the proposed method with state-of-the-art methods for leukocyte detection
Full size table

Discussion

Variations in illumination and staining are the challenges faced during microscopic image analysis. Hence, an automated leukocyte detection method should be robust to manage the color and illumination variations. We addressed these variations using a novel image enhancement method as given in section “Detection of nuclei”. We observed that, G component is distinct between nucleus and cytoplasm, and R and B components are nearly the same. G component has distinctly different values in nucleus and cytoplasm region. Hence to enhance the contrast between nucleus and cytoplasm we considered G’G’B representation. To further enhance the contrast, we used contrast enhancement method. There are two benefits of this image enhancement method; it provides good contrast between nuclei and background and it also introduces robustness in managing color variations. It can be observed from Fig. 7 (upper row) that, color of nuclei and background varies in original images. The proposed image enhancement method minimizes these variations as can be seen in Fig. 7 (lower row). This introduces robustness to manage variations in color and illumination.

Fig. 7
figure 7

Image enhancement; upper row Original images, lower row G’G’B images

Full size image

The results of nuclei detection are demonstrated in section “Performance of nuclei detection method”. The proposed method was compared with two state-of-the-art methods to evaluate the results. The performance of the proposed method was found to be better in comparison with the other two methods. This could be due to the combination of image enhancement method and Zack’s thresholding method to select appropriate threshold value. The results after applying Zack’s method and Otsu’s method are shown in Fig. 8(b) and Fig. 8(c) respectively. It can be observed from the figure that, Zack’s thresholding method results in appropriate threshold value for detection of nuclei. This adaptive way of threshold selection method could differentiate nucleus from other regions around it and improved segmentation results. Thus less variation can be observed in DSC plot as depicted in Fig. 5 and the proposed method can handle illumination and color variations in detection of nuclei.

Fig. 8
figure 8

Results of thresholding methods; a result of minimum filter, b result of Zack’s algorithm, c result of Otsu’s method

Full size image

Leukocyte detection using active contours method was demonstrated in section “Results of leukocyte detection”. We considered active contours to manage shape and size variations of leukocytes. Active contours are robust to shape variations. In the proposed method, we considered masks of sizes between 30 and 70 to initiate the active contours based on area and circularity of the nuclei. Selection of smaller mask for smaller leukocytes such as lymphocyte, eliminates over segmentation and converges in less number of iterations. Larger mask size helps in appropriate detection of neutrophils, monocytes and eosinophils. Large mask size for small leukocytes may lead to convergence of active contours to the background region. Small mask size for large leukocytes may lead to convergence of the active contours to the nuclei boundary. Hence, fixed mask size may result in either over-segmentation or under-segmentation of leukocytes. Use of variable mask size based on the geometrical features of the nuclei helped in appropriate detection of leukocytes.

In the dataset there are erythrocytes overlapped on leukocytes. We eliminated touching erythrocytes using morphological opening operations but separation of overlapped cells is not addressed in the proposed method. More accurate detection of leukocytes can be achieved by pre-processing the image to separate overlapped cells.

We also evaluated the performance the proposed method for detection of leukocytes by considering images from ALL-IDB dataset. It is found that the proposed method performs equally well for both the datasets. This highlights the robustness of the proposed method for variations among datasets.

Conclusion

In this paper, we demonstrated an automated method for detection of nuclei and leukocytes considering color and illumination variations. Segmentation of nuclei were considered as initial step to detect leukocytes because these are distinctly dark objects in peripheral blood smear images. We considered G’G’B representation of original RGB image which highlights nuclei regions. We used arithmetic operations and Zack’s thresholding for detection of nuclei and obtained average DSC of 0.97 and sensitivity of 0.96. Active contours method was used for detection of leukocytes considering nuclei as mask. We considered mask of variable size based on area and circularity of nuclei. This resulted in overall DSC of 0.965. The proposed method achieved an average sensitivity of 0.97 for leukocyte detection. Adaptive way of selecting masks for detection of leukocytes contributed to higher efficiency in leukocyte detection.

Blood cell segmentation plays a vital role in automating laboratory procedure. An automated method must locate the cells correctly irrespective of color, illumination, brightness and staining variations for further processing such as feature extraction and classification. The proposed method addressed these variations by introducing a novel image enhancement method. Further, a method could be developed for separation of overlapped cells for accurate detection of leukocytes.