1 Introduction

Blood cells consist mainly of platelets, white and red blood cells, commonly known as WBCs and RBCs respectively [31]. In the immune system of the human body, WBCs play an essential role. WBCs protect the body from extrinsic invaders and epidemic diseases like leucocytosis, leukopenia [6]. The WBCs can be classified mainly into two groups named Granular cells and Non-Granular cells [28]. Granular cells are the small particles that store enzymes that are released during the infection. It is further classified into three subtypes named Neutrophils, Eosinophil and Basophils. The Non-Granular cells are further classified into two subtypes named Monocyte and Lymphocyte. In the human body, the Monocyte helps in fighting bacteria, viruses, and other infections. Monocytes are the key elements of the immune response. Antibodies can be released by lymphocytes. In the bone marrow, they are developed and contained in the blood and lymph tissue. There is a whole branch of medicine known as haematology that studies the cause, prognosis, treatment and prevention of blood-related diseases. Haematologists usually use their expertise and traditional knowledge of haematology to classify WBCs and their subtypes [5]. This primitive data is consistently used by doctors to determine the seriousness of blood disease. Significant and essential requirements for medical diagnosis has thus become the study of the classification of white blood cells.

Figure 1 indicates the white blood cells composition. To resist foreign organisms, a particular defensive function is associated with each type of white blood cell. It is of two types granular and non-granular which is based on their shape. Further granular is classified into Eosinophil, Neutrophil, and Basophils. And non-granular is further classified into lymphocyte and monocyte. Neutrophils protect against bacteria and are one of the main white blood cells. The parasites are killed by Eosinophil and perform an allergy operation, although basophils are involved in different allergic reactions in the body. The monocyte destroys the body's damaged tissues by entering into the cell. To balance immunity mediated by cells, lymphocytes are very important and complicated cells. Lymphocytes are of two types (B-type and T-type) which are used for memorizing invasive micro-organisms and viruses, therefore the lymphocytes are entirely different from the other WBCs.

Fig. 1
figure 1

White blood cell classification structure

Full size image

Similar to the traditional method on the classification of WBCs, automatic methods are required mainly on the classification of WBCs. In an automatic method some traditional image processing, machine learning and deep learning methods have been proposed. In traditional image processing, some features are extracted and then using mathematical relationships and controlled parameters a threshold function is defined. It is used to discriminate the features of different category which helps to make a better classifier. While in machine learning, an advanced feature like texture is extracted and the model is trained on those features. In deep learning, the automated feature engineering approach derives appearances, textures, patterns and different attributes. That play a primary role in image classification based on WBC characteristics. Image processing methods for classification, segmentation, extraction of features and recognition have been implemented. The model previously introduced by a haematology analyser [33] to identify the WBC images was time-consuming and manual. Often, it causes detection errors. Computer-aided image processing-based diagnostic machine learning algorithms such as decision tree [24], k-means [12], Recurrent Neural Network (RNN) [18], Convolutional Neural Network (CNN) [14] and support vector machine (SVM) [20]. Various types of white blood cells and their subtype classification are compared in Table 1.

Table 1 Comparison of algorithms used for blood cell image classification
Full size table

Using the optical electron microscope on blood samples research has been introduced by Rosyadi et al. [23]. That has the potential to identify WBCs images.

Furthermore, the Ostu classification threshold approach is adopted by the authors with the KNN. Based on their study, it was concluded that discriminatory feature has developed after the implementation of clustering with k-means to identify WBCs. In [8] classification algorithms are used for the diagnosis of leukaemia where leukocyte segmentation is performed using CMYK colour space from two separate blood smears. The classification of support vector machines achieved an accuracy of 86.67%.

In Giemsasmear blood cell images using texture, histogram, and gradient region granulomatous characteristics with classifier as decision tree, Malihi et al. [17] suggested identification of malaria parasites. Pandey, C [4] Proposed Machine learning approach for image classification using SVM. Singh, A. [26] fused these features with ANN approach for classification. In [21] proposed CNN architecture for classification. In [9] another approach is used which have a classifier as naive Bayes for training the algorithm. It includes circularity, perimeter, area, eccentricity and semantic features. The Bayes classifier is used and achieved 80.88 per cent accuracy. In [25] the classification of cells with hyper-spectral imaging using a Gabor-wavelet and a CNN model is shown.

An automated machine learning (ML) approach is proposed in [1] for the identification of blood cell types as platelets, RBCs and WBCs. These types of blood cells are counted correctly by the algorithm provided. In [3] Automatic classification method for blood cells types using a transfer learning system has been suggested. The classification of acute lymphoblastic leukaemia based on WBCs segmentation using maximum feature selection with main component analysis is described in [27] with 94 per cent average accuracy.

In [29], random forest classifiers, SVM, multilayer perceptron (MLP) is compared with CNN classifier to classify WBCs image cells. In [19] for blood cell microscopic images, the automatic algorithm for classification is proposed. Discrete wavelet transformation is applied after image augmentation to obtain frequency information of the WBCs and ultimately CNN is used for classification. SVM and deep learning are used to establish irregular blood cell recognition techniques [2]. A CNN with the combination of RNN is applied in single-cell images with matrix transformation is proposed in [15] for the classification of blood cells. By integrating single image features of the cell nuclei, the CNN-RNN technique resulted in an accuracy of 90.87 per cent. CNN and the combination of CNN and RNN are mainly the subjects of the latest medical image classification research. The researchers have previously proposed different models of CNN capable of improving the accuracy of the classification, especially focusing on automatic feature learning techniques.

Blood cell defects are responsible for numerous health conditions. The conventional technique of manually visualizing White Blood Cells under the microscope is a time-consuming, tedious process and its interpretation requires professionals. This work presents an automatic classification method with the help of machine learning for blood cell classification from blood sample medical images. The proposed method can identify and classify the function of each segmented White Blood Cells cell image as granular and non-granular White Blood Cells cell type. It further classifies granular into Eosinophil, Neutrophil and non-granular into Lymphocyte, Monocyte in various forms. Which helps to identify leucocytosis, leukopenia of these sub cell types.

To classify the WBCs and achieve higher accuracy rates, these models can be further enhanced. The goal of this research is to enhance the accuracy of the classification with the extraction of discriminative features using automatic feature engineering techniques.

This work presents an automatic classification method with the help of machine learning for blood cell classification from blood samples. The proposed method can identify and classify the function of each segmented WBCs cell image as granular and non-granular WBCs cell type. It further classifies granular into Eosinophil, Neutrophil and non-granular into Lymphocyte, Monocyte in various forms. Because of its high precision, the proposed framework includes a neural network model to detect white blood cell types. To improve the accuracy of multiple cells overlap and increase the robustness, data augmentation techniques have been used in the proposed system for training binary classification of blood cell types and multi-classification of blood cell subtypes.

This paper is structured as follows: Sect. 2 describes the dataset used in this experiment Sect. 3 outlines the methods and methodology adopted to achieve the research objectives. In Sect. 4, along with the discussion, the experimental results are discussed. The last Sect. 5 ends with the conclusion and future work.

2 Dataset

This dataset includes 12500 augmented blood cell images (JPEG format) with labels of the cell type (CSV format). This dataset contains approximately 3000 blood cell images for each 4 different sub-types of blood cell types. Granular and non-granular are the cell types. Eosinophil, neutrophil, monocyte, lymphocyte and are the four-cell sub-types. In particular, the dataset contains 2500 enhanced images and 4 additional (JPEG + CSV) subtype labels. This dataset is a publically available standard dataset named as BCCD Dataset (https://www.kaggle.com/paultimothymooney/blood-cells) provided by github/Shenggan.

Figure 2 shows the flow diagram for sample collection and dataset preparation. In this process there are 5 steps. First step is to collect the blood sample from the subject. Second step is to take the blood sample to glass slide which is used to observe using electron microscope in step three. Step four is to acquire a good image for that sample with the help of experts in this filed. Last step is to prepare the dataset and the labelling of that sample for ground truth.

Fig. 2
figure 2

Flow diagram for sample collection and dataset preparation

Full size image

Table 2 describes the augmented dataset used for the experiment in this work. In this, there are major two types of WBCs as granular which contains 6243 images and non-granular 6204 images. Which is further sub-classified into 4 classes. In Eosinophil total of 3120 images are used (2497 images used for training and 623 images used for testing), Neutrophil total of 3123 images are used (2499 images used for training and 624 images used for testing), Lymphocyte total of 3106 images are used (2483 images used for training and 623 images used for testing) and Monocyte total of 3098 images are used (2478 images used for training and 620 images used for testing).

Table 2 Augmented WBCs dataset used for experiment
Full size table

Figure 3 shows the various white blood cell images of different types. (a) Represents the Eosinophil type images. (b) Represents the Lymphocyte type images. (c) Represents the Monocyte type images. (d) Represents the Neutrophil type images.

Fig. 3
figure 3

Samples images of white blood cell images with augmentation

Full size image

3 Proposed method

The main objective of the proposed work is to create an automatic classification model for white blood cell type detection. This section demonstrates the white blood cell image classification approach using the convolutional neural network method. The flowchart for the proposed algorithm is shown in Fig. 4. In this study, the image of blood cell samples is given as input then feature extraction is performed using a deep learning model which extracts deep features from these white blood cell images to achieve a higher rate of classification. This includes pre-processing and development of CNN architecture, accompanied by the discovery of influential features and discriminatory characteristics to differentiate these types of cells and used for training and testing.

Fig. 4
figure 4

Block diagram of the proposed framework for white blood cell type

Full size image

The Flowchart of the proposed WBCs-Net model demonstrates the classification of WBC in the Fig. 4. The strategy is broken down into four parts: (a) collecting samples of blood from subjects or datasets (b) Image pre-processing and data augmentation is performed on image samples like cropping, rotation, scaling etc. (c) Proposed CNN architecture for white blood cell image multi-classification which can help for the identification and classification of Subtypes of blood cells have significant medical applications. The learning of blood cell type is carried out using model training and testing via the proposed CNN architecture. (d) Checked the results of the proposed CNN architecture for multi-classification using professional-generated ground reality.

3.1 Pre-processing on WBCs images

Pre-processing is the basic step in image processing. Dataset has a white blood cell images sample. Image pre-processing is done in this step to enhance accuracy and reduce processing time. In this step, rescaling is performed to convert image data from range 0–255 to 0–1 with the help of normalising the data by dividing the data by 255. The sheer range is used ‘0.2’ and zoom range ‘0.2’ is applied. Horizontal flip augmentation has also been used for better training.

3.2 CNN architecture

A deep learning model is suitably used in a variety of image processing applications. It is a neural network architecture of various kinds of layers. It is inspired by the human brain's biological visual cortex. Each CNN layer is responsible for performing the process of feature extraction with different operations and functions. Weight parameters are obtained by the CNN model by adjusting the initial weights by training on the dataset. Convolution neural networks is an architecture consist of a convolutional layer followed by a pooling and dense layer. This form of the model is useful for both types of input data such as images, audio. In this form, information is transferred from one layer to another layer. These features are processed with different hidden layers. Where the first few layers are processed the low-level features and the deeper hidden layer processed more high-level features are extracted. The CNN style, as shown in Fig. 5 is made up of a stack of different layers that relay data connected to each neuron from the input to the output level.

Fig. 5
figure 5

Proposed CNN architecture for white blood cell type

Full size image

A detailed explanation and formulation of the various layers of the convolutional neural network are given in the following subsections.

3.2.1 The input layer

The input layer takes the raw image as an input to classify them into different types of WBC. These images consist of 3 dimensions such as height width and the number of colour channels. This sample is of size 64 × 64 × 3.

3.2.2 The convolutional layer

The convolutional layer is the main feature extraction layer. The convolution layer highlighted the small changes in the images. This encourages the use of convolution neural networks in nano-scale biomedical images for pattern recognitions and identifies the optimal weights, activation function, and layer architecture. It takes input from the previous layer and extract important features and pass these feature to the next layer. This operation consists of three elements. Input layer feature, kernel, and convoluted features. The kernel slide over input WBC images from top-left to bottom right and extract the convolution features one by one. The output feature map is generated by the kernel which is multiplication over input data and inverted kernel. Also, the rectified linear unit (ReLU) activation function is used to activate/deactivate the neuron based on a function. It will provide irregularity in the neural network.

$$f\left(r\right)=\left\{\begin{array}{c}0, r<0\\ 1, r\ge 0\end{array}\right\}$$
(1)

3.2.3 The pooling layer

The pooling layer is applied after this convolution and activation layer. It is used to down sample the feature map that reduces the computational and over fitting problems. Here max pooling is used with the pooling size of 2 × 2.

3.2.4 The fully connected layer

A fully connected layer gives desired output as a result of the network. The connection of this layer depends on two cases. If it is connected to the previous fully connected layer then a number of the parameter is defined by multiplication of parameter in previous layer and parameter in this fully connected layer. If it is connected to the convolutional layer then a number of the parameter is defined by multiplication of output feature map of the previous layer, size of kernel and number of a parameter in this fully connected layer. The output of different structures involving 2D-convolutional layers was studied while preparing the optimal CNN model for white blood cell image detection. The proposed structure, however, contains seven convolution layers that achieve optimum efficiency in the experiment. Figure 5 and Table 3 define the structure of the ideal CNN in detail.

Table 3 The proposed CNN architecture for blood cell image multi-classification
Full size table

3.3 Training and testing of the proposed model

CNN training is the main step in the neural network model, as the training approach has a significant influence on the subsequent performance of the entire system. First a separate collection of data was used for preparation, validation and testing using the proposed method. Second, using categorical cross-entropy loss, the training process is being optimized. We divide the entire training collection into so-called mini-batches due to memory constraints, and these mini-batches are used to train iterations of CNN. For a batch, 32 images are chosen randomly from the training data (without repetition) and trained for epochs. In a validation set, classification accuracy and fine-tune model parameters are used to evaluate the model, where prediction is made by slicing the test sample into overlapping TF-patches (1-frame hop), making a prediction for each TF-patch and finally selecting the prediction of the sample level as to the class with the highest mean output activation over all frames. Therefore, in 4 separate categories of white blood cell images with a training dataset, this CNN model is trained. These parameters of the model are stored and ported into low level computer systems. That low level system can be used for classification of WBCs images. The high level computers are used here for the training perspective. But this trained model don’t require such type of high specification computers for classification.

For the multi-classification of white blood cell types, the hyperparameters used while training the proposed CNN architecture are shown in Table 4. These are the parameters whose values are defined before the training process to control the learning process. Here ‘categorical cross-entropy’ is used as a loss type, mini-batch size 32 is used, maximum steps per epoch is 800, maximum epoch is 1000, early stopping is used as 25 epoch with no improvement. Model check pointing is used to store best epoch model and ‘adam’ optimization algorithm [11] is used. In the proposed model other parameters, strides, padding, processing, such as filters, kernel size, weights has been empirically determined based on several experiments.

Table 4 Proposed hyper parameters for CNN
Full size table

The suggested model is trained using data from training, with validation data being used to fine-tuning the model parameters. The resulting model is tested using the test dataset to prevent overfitting and better splitting of the results. The trained deep neural network predicts whether the white blood cell images is one of the types (granular and non-granular) for binary classifications and (Eosinophil, Neutrophil, Lymphocyte, Monocyte) for multi-classification. These findings are linked to the principles of ground truth supplied by domain experts.

4 Experiment result and discussion

The convolution layers are applied to the entire white blood cell image dataset before training and validation processes are carried out by deep learning. Multiple cell images are considered instead of single-cell images because using the CNN technique, higher features are extracted and result in high precision. The hardware Using Keras, CNN is implemented in Python, with NVIDIA DGX-2 server, 2 petaFLOPS performance, 2 × Intel Xenon Platinum CPUs, 16 NVIDIA Tesla V100 32 GB HBM2, 1.5 TB DDR4 RAM, 30 TB SSDs, 8 × 100 Gb/sec network speed.

Table 5 Describe the list of software and hardware used for implementation. The operating system used in the experiment is NVIDIA DGX Server Version 4.7.0 with customised NVIDIA Dockers is used in this experiment. NVIDIA customised NGC container is used is for experiment version “Nvidia:TensorFlow” where container tag is “20.02-tf1-py3”. Python 3.7 was installed in this container, with TensorFlow GPU enable 1.15.2 version. The system has 16 NVIDIA tesla V100 GPUs, 81920 NVIDIA CUDA core, 10240 NVIDIA tensor core.

Table 5 List of hardware and software used for experiments
Full size table

Table 6 demonstrates the binary classification confusion matrix based on types of WBCs by the proposed model on the test set. A total of 2453 images are classified correctly out of 2490. It shows the total accuracy the 98.51% on the test set.

Table 6 Confusion matrix of blood cells type identification
Full size table

Table 7 demonstrates the multi-classification confusion matrix based on sub-types of WBCs by the proposed model on the test set which shows very little misclassification of a dataset. It shows the total accuracy the 97.7% on the test set for multi-classification.

Table 7 Confusion matrix of blood cells sub-type identification
Full size table

Table 8 the results of the binary classification based on types of WBCs by the proposed model on the test set show the various other output parameters. A total of 98.51% accuracy has been achieved with 98.51% precision, 98.56% recall, 98.51% specificity and 97.50% F1 score on the binary classification.

Table 8 Various performance parameters and results for binary classification
Full size table

Table 9 the results of the multi-classification based on sub-types of WBCs by the proposed model on the test set show the various other output parameters. A total of 98.91% accuracy has been achieved with 97.59% precision, 97.7% recall, 99.19% specificity and 97.63% F1 score on the multi-classification.

Table 9 Various performance parameters and results for multi-classification
Full size table

Figure 6a shows the plotted graph between the model training accuracy and testing accuracy. Where the blue colour shows the training parameter while the orange colour shows the testing parameter. Figure 6b shows the plotted graph between training loss and testing loss. Where the blue colour shows the training parameter while the orange colour shows the testing parameter.

Fig. 6
figure 6

Model accuracy and loss graph with training and testing samples

Full size image

Table 10 shows the comparison results of different models in 2D CNNs which were experimented on white blood cell images. This table shows that the model has 5 layers with maximum number of filter 32 has achieved the maximum accuracy rate among these. Therefore this model is selected as the final model for this white blood cell type detection problem.

Table 10 Comparison results of different model in 2D CNNs
Full size table

Table 11 shows the comparison of the performance of the proposed work on this subject with the current state of the art. The Livieris et al. [16] Achieved 93.29% accuracy KNN with is a semi supervised learning. Bani-Hani et al. [7] and Zhao et al. [10] Has used the Convolutional neural networks with an accuracy of 91.01 and 87.08% respectively. While Liang et al. [15] Has achieved 90.97% accuracy using fusion of CNN and RNN technique. The recent paper Patil et al. [22] has achieved 95.89% accuracy with CAA applied to CNN and RNN technique. The proposed method has achieved the 98.51% for binary and 97.70% for multi classification on WBCs. This method has resolved the issues of miss-classification of WBCs subtypes. This shows the proposed method has an advantage over the existing method while discriminating the white blood cell type.

Table 11 A comparative table of the proposed approach with the state-of-art
Full size table

5 Conclusion

A completely convolutional neural network model based on deep learning is presented in this work to reduce the miss-classification of white blood cells. The data set augmentation approach is applied to improve the data set. There are convolutions and numerous other layers in the model. 98.51 per cent precision for the test set was reached by the proposed model for WBCs classification. 97.7 per cent precision for the test set was achieved by the proposed model for WBCs subtype classification. Through the use of a deep learning model on white blood cell images, such major improvements in performance indicate the superiority of the proposed system. Future studies will be aimed at recognizing various other kinds of cell types.