1 Introduction

Computer-aided image analysis has become one of the major research subjects in medical imaging. Numerous approaches for detection or diagnosis of various diseases have been proposed recently to assist in the interpretation of medical images [1] and complement the opinion of pathologists. This is mainly motivated by the fact that visual qualitative assessment of medical images is time-consuming and subject to inter- and intra-pathologist variability in diagnosis [2], i.e., different pathologists may come up with diverse interpretations, leading to different diagnosis, or the same pathologist may make different diagnosis at different time for the same set of medical images.

In the case of follicular lymphoma (FL), which is the second most common lymphoma diagnosed in USA and Western Europe, a pathologist categorizes FL cases into histological grades based on the number of centroblasts (CBs) counted per high power field (HPF), i.e., microscopic images acquired at a magnification level of \(\times \)400 [3]. Tissue biopsies of FL are stained with hematoxilin and eosin (H&E), and they are visually inspected by pathologists. In order to account for tissue heterogeneity, the average CB number in ten different HPF images, derived from the same tissue section, is being estimated [4]. Since this manual procedure is highly subjective and requires extensive training, various methods [1, 5–7] for automatic FL grading have been proposed to increase the accuracy and reproducibility of diagnosis.

The main challenge of these methodologies is the accurate segmentation of nuclei and the extraction of a suitable set of features for their classification into CBs or non-CBs. Particularly, the latter requires the modeling of pathologists’ knowledge used in clinical practice, that is, the identification of a number of morphological and textural features. However, the extraction of this information from H&E-stained images using computer-aided image analysis is not an easy task. Therefore, either morphological characteristics of nuclei are used for their classification or their morphology is considered along with the textural variation of nuclei [4–6] or even with graph-based features [8]. In most of these cases, the classification of nuclei considers low-level texture features, which are usually extracted after a statistical analysis based on PCA to quantify the textural variation. The classification accuracies using such an approach range from 75 to \(85\,\%\) [6–8] and therefore may not be sufficient enough for certain clinical applications [9].

Fig. 1
figure 1

Characteristic case of a CB (blue dashed circle) and FL dendritic cell (red solid circle) in a a H&E image and b the corresponding PAX5 image. In PAX5 image, the dendritic cell is blue colored and it can be easily removed through the segmentation (color figure online)

Full size image

A different approach was proposed in [9] aiming to identify several features related to pathologists’ knowledge used in clinical practice of FL. The method was applied only on manually cropped images of CB and non-CB cells and attempted to implicitly extract such information by considering the whole image of a cell with its surroundings as a single feature vector. The redundant features were removed using linear and nonlinear dimensionality reduction approaches. More recently, a different method for the explicit identification of features defined by pathologists was presented in our previous work [10, 11] in which 1-\(\upmu \)m sliced tissues of FL were used for the classification of CBs. Although the use of tissue sections sliced at a lower thickness level of 1–1.5 \(\upmu \)m revealed more information about the interior of nuclei, both segmentation and classification of cells were challenging, since their boundaries were not always well shaped and the discrimination of CB from non-CB nuclei with similar characteristics, i.e., dendritic or endothelial cells, was extremely difficult as shown in Fig. 1.

Taking advantage of the fact that different stains can provide valuable information to aid understanding of the physical or functional properties of tissue [12], in this paper we introduce a novel complete framework for the segmentation, separation and classification of FL nuclei in tissue sections stained with PAX5 and H&E stains and sliced at 1 \(\upmu \)m thickness level (see Fig. 2). The proposed methodology, as shown in the experimental results, not only improves segmentation and separation of nuclei, but also contributes significantly to the discrimination of CBs from other non-CB cells with similar characteristics. In addition, we introduce a novel approach for overlapped nuclei separation inspired by the clustering of large-scale visual vocabularies and we propose a new classifier, based on probabilistic graphical models, for the combination of morphological characteristics of nuclei extracted from PAX5 images with their textural features in H&E images.

Fig. 2
figure 2

Proposed framework for the segmentation, separation and classification of nuclei in PAX5- and H&E-stained FL tissue sections

Full size image

2 Nuclei segmentation

For the segmentation of nuclei in PAX5-stained images, we apply an energy minimization technique based on graph cuts, which is an unsupervised approach that can be employed efficiently to various image segmentation problems and particularly to FL PAX5 images, where the number of foreground and background pixels is balanced, as shown in Fig. 1. Specifically, the segmentation procedure is considered as a labeling problem, where the labels represent different cytological components, i.e., nuclei and cytoplasm. In practice, apart from the cytoplasm the second class contains also the blue-colored nuclei including follicular dendritic cells. In this labeling problem, the image is represented as a graph \(G = \langle V,E\rangle \), where V is the set of all nodes and E is the set of all edges connecting adjacent nodes. Nodes and edges correspond to pixels and their adjacency relationship, respectively. The graph also contains two terminal nodes, which are referred to as the source and the sink. The labeling problem is to assign a unique label \(x_{p}\) for each node V, so as to minimize the following energy:

$$\begin{aligned} E=\mathop \sum \limits _{p\in V} C_p \left( {x_p } \right) +\mathop \sum \limits _{\left( {p,q} \right) \in E} S_{p,q} \left( {x_{p,} x_q } \right) \end{aligned}$$
(1)

where \(C_{p}\) is the color consistency cost, i.e., the negative log likelihood of the pixel p to belong to the cluster corresponding to \(x_{p}\) according to its RGB value and \(S_{p,q }\) is the smoothing cost between two neighboring pixels (p, q). The smoothing cost depends on the labels (\(x_{p}, x_{q})\), i.e., nuclei or cytoplasm, of the neighboring pixels and the existence of an edge at this location of the image (pairs of pixels with different labels are penalized if no edge exists). The cost of the cut (a cut partitions the graph into two disjoined subsets) is defined to be the sum of weights of the edges crossing the cut, whereas the minimum cut problem is to find the cut with the minimum cost. The basic idea is that the minimum cut minimizes the energy either globally or locally. In our experiments, the expansion algorithm [13] was used, which is one of the most efficient algorithms for minimizing discontinuity-preserving energy functions. The algorithm results in the labeling that minimizes the energy of Eq. (1) leading to the segmentation of nuclei from cytoplasm. For the initialization of the algorithm, a k-means approach was adopted for assigning an initial label to each pixel.

3 Separation of clustered nuclei

After the segmentation step, a number of nuclei overlap with each other forming clusters of cells. Splitting of clustered or overlapped nuclei is one of the most critical steps in microscopic image analysis. Semi-automatic approaches [14, 15] have been widely used in the past to improve segmentation results, however, such methods are in general time-consuming and often infeasible [16]. To this end, a variety of techniques have been introduced [17–19] for unsupervised splitting of clustered nuclei, with watershed algorithm [21] to be the most popular. Nevertheless, watershed usually suffers from oversegmentation, as it is affected by the existence of local minima. To suppress the oversegmentation effect, marker controlled approaches have been introduced [15] and morphological operations have been applied to address the problem of spurious markers. However, even in this case the accurate definition of markers is a challenging issue, while the results heavily depend on the size of the structuring elements [19]. On the other hand, blob detection techniques, such as those based on LoG filter [20], allow the detection of nuclei without applying any segmentation step (however in our case, segmentation is important since it contributes significantly to the removal of dendritic cells), but they transform the problem into a local maxima detection problem. These techniques, however, require a tuning of the appropriate parameter setting, while oversegmentation problems appear when nuclei deviate significantly from blob shape or tiny regions are detected as nuclei [20]. More sophisticated techniques attempt to formulate the splitting of the clustered nuclei as an optimization problem [16, 22]. To avoid exhaustive search for the estimation of the optimum number of seeds, in this paper we propose an alternative approach aiming to initially identify and purge spurious seeds from a list of candidate seeds and then detect each nucleus using an ellipsoidal model. Finally, a validation criterion is introduced for the identification of spurious seeds. The proposed methodology is illustrated in Fig. 3.

Fig. 3
figure 3

a Two different cases of clustered nuclei, b estimation of regional maxima after the application of distance transform, c seeds corresponding to local maxima are deleted using regional H-maxima transforms, d the result of EM algorithm, e identification of weak seeds, f the new results of EM for the remaining seeds, i.e., without weak seeds and g the segmented nuclei. As we can see, in the first case (top clustered nuclei) the weak seed is finally pruned, while in the second case the weak seed is considered valid, since the EM solution in (d) provides better results in terms of the validity index than the one in (f)

Full size image
Fig. 4
figure 4

Methodology of the separation of clustered nuclei

Full size image

3.1 Identification of candidate seeds and clusters modeling

To handle the oversegmentation problem, we initially aim to identify a list of candidate seeds. Toward this end, we apply a distance transform and we estimate the regional maxima in the generated distance image D (as shown in the experimental results, the use of distance transform is adequate for the case of FL, however, in more complicated microscopic image analysis problems other preprocessing steps could be adopted to detect the initial set of candidate seeds). The result of this first preprocessing step is the extraction of a number of seeds, some of which correspond to local maxima located mainly in the vicinity of true regional maxima. Since there is a one-to-one correspondence between the regional maxima and the nuclei, only one maximum can be accepted in each neighborhood. To eliminate the local maxima, we sort all candidate seeds in a descending order according to their distance value \(v_{i}\), where \(i=1,2\ldots , n\) is the seed and n is the total number of candidate seeds. Starting from the seed with the highest \(v_{i}\) value, we apply a regional H-maxima transform [23] in order to suppress all local maxima in its vicinity with radius equal to the value of \(v_{i}\). Subsequently, the seeds corresponding to local maxima are deleted from the list and the same procedure is repeated for the remaining maxima. The result of this first preprocessing step allows us to identify a list of k candidate seeds driven by the morphology of clustered nuclei, i.e., the regional maxima (Fig. 4). Based on the hypothesis that nuclei can be spatially modeled as ellipsoids, the pixel coordinates in each cell are modeled using a Gaussian distribution. More specifically, a Gaussian mixture model is applied with the number of clusters being equal to that of candidate seeds.

3.2 Identification of \(\varvec{\textit{weak}}\) seed

Having estimated the ellipsoidal models of nuclei for all seeds, we need to identify the spurious ones from the list of candidate seeds in order to estimate the correct number of nuclei. To do so, we attempt to compute an overlap measure for the estimated clusters. In [24], a clustering method is proposed to construct visual vocabularies for image retrieval. The method attempts to compute the overlapping of neighboring clusters and uses an overlap threshold to purge all redundant clusters. In our case for the segmentation of clustered nuclei, instead of using a threshold, we attempt to identify the weak seed in the cluster and we use a validation process in order to decide whether a cluster is redundant or not. More specifically, for the estimation of the overlapping metric, we compute for each Gaussian component \(p_{i}\) the quantity r, which indicates the responsibility of component for itself relative to the set of remaining seeds \( {\hat{k}} =\left[ {1,\ldots ,i-1,i+1,\ldots k} \right] \):

$$\begin{aligned} r_{{i, {\hat{k}}}} = \frac{{\hat{\gamma }_{{ii}} }}{{\hat{\gamma }}_{{ii}} + \sum _{j \in {\hat{k}}}{{\hat{\gamma }} _{{ij}} }}, \quad \hbox {where}\, i \notin {\hat{k}} \end{aligned}$$
(2)

where \({\hat{\gamma }}_{ij} =\hat{\gamma }_j (p_{i})\in \left[ {0,1} \right] \) is the generalized responsibility of component \(p_{j}\) for component \(p_{i}\) with each component of the mixture to correspond to a unique seed:

$$\begin{aligned} \hat{{\gamma }}_j (p_i )=\frac{\left\langle {p_j,p_i } \right\rangle }{\mathop {\sum }\nolimits _{i=1}^k {\left\langle {p_j,p_i } \right\rangle } } \end{aligned}$$
(3)

with \(\left\langle {p_j,p_i } \right\rangle =\pi _j \pi _i N(\upmu _j |\upmu _i,\Sigma _j +\Sigma _i )\). However, instead of purging all seeds corresponding to components with a value \(r_{i,{\hat{k}}}\) lower than a predefined threshold, in this paper we propose the identification of the weak seed based on the criterion of Eq. (2). That is, the seed whose cluster has the lowest \(r_{i,\hat{k}}\) among all the mixture components in the clustered nuclei is considered as the weak seed, i.e., \(s_\mathrm{w} =\mathop {\arg {\min }}\limits _i (r_{i,\hat{{k}}} )\). This means that the identified \(s_{w}\) is likely to be redundant, as a low \(r_{i,{\hat{k}}}\) value implies that the components in \({\hat{k}}\) can adequately describe component \(p_{i}\).

3.3 Validation process

The above indication, however, is not enough for purging \(s_{w}\) from the list; therefore, a cluster validation process is needed to make the final decision. To this end, we also propose a new validation index in order to evaluate whether seed \(s_{w}\) is spurious or not, that is, whether the new EM solution \({\hat{\varTheta }} \) corresponding to seeds in \({\hat{k}}\) is better than the previous solution \(\varTheta \) of k components. In general, cluster validation is a critical issue in cluster analysis. An objective approach has been proposed in [25], especially for fuzzy clustering, which introduces a combined function of clusters compactness and separation in order to determine the correct number of clusters. In other words, the validation criterion requires the clusters to be well separated and with a compact structure.

However, while the splitting of clustered nuclei can be considered as a cluster analysis problem, there are also specific particularities that should be taken into account. For instance, although the criterion of compactness provides significant information about the structure of a cluster, i.e., large distances between the members of a cluster or the existence of outliers, in the case of nuclei splitting this measurement by itself is not enough to evaluate the clustering result, as it provides little information regarding the relation of the mixture of Gaussians with the nuclei morphology. For this reason, a new validation criterion should be considered to include also the fitness degree of the estimated ellipsoidal components against the clustered nuclei data:

$$\begin{aligned} V(\hat{{k}})=\frac{\hbox {Sep}(\hat{{k}})}{\hbox {Comp}(\hat{{k}})\cdot \hbox {Fitness}(\hat{{k}})} \end{aligned}$$
(4)

In the above equation, \(\hbox {Sep}(\hat{{k}})=\hbox {trace}(S)\) denotes the separation measure of components with S indicating the between-components scatter matrix \(S=\mathop {\sum }\nolimits _{i\in \hat{{k}}} \mathop {\sum }\nolimits _{j=1}^n u_{ij}^r (m_i -\bar{{m}})(m_i -\bar{{m}})^{T} \), where \(m_i \) are the centers of components, \({\bar{m}} \) is the centroid of the clustered nuclei, n is the total number of its elements, i.e., the number of pixels, and \(u_{ij}\) is the grade of fuzzy membership of the jth element to the ith component. Since, crisp clustering may be viewed as a special case of fuzzy clustering, an element can get a membership value of 1 for the component to which it belongs, i.e., the component from which the element has the shortest distance from its centroid, and a value of 0 to all other components [26]. On the other hand, the compactness measure in the denominator of Eq. (4) is given as \(\hbox {Comp}(\hat{{k}})=\sum _{i\in \hat{{k}}{}_{\phantom {0_0}}}^ {\hbox {trace}(\bar{{\Sigma }}_{_i } )} \), where \(\bar{\Sigma _i}\) is the normalized covariance matrix of i. The last factor of Eq. (4) is , which aims to measure how well the ellipsoidal models fit to the elements of their clusters and it is defined as the sum of the normalized fitness scores of individual components. The score Fc(i) denotes the sum of distances of all elements in set \(W_i =E_i\oplus X_i\) (where sets \(E_i \) and \(X_i \) represent the elements of ellipsoidal component i and the elements of the corresponding cluster, respectively, and \(\oplus \) is the symmetric difference operator) from the centroid of cluster i, normalized by the maximum radius of the nuclei:

$$\begin{aligned} Fc\left( i \right) =\frac{\mathop \sum \nolimits _{t=1}^\tau \left( {{\hat{x}}_t -m_i } \right) \left( {{\hat{x}}_t -m_i } \right) ^{T}}{\mathop {\max }\limits _{\forall j} \left( {u_{ij} \left( {x_j -m_i } \right) \left( {{\hat{x}}_j -m_i } \right) ^{T}} \right) } \end{aligned}$$
(5)

where \({\hat{x}}_t \in W_{i} \), \(\tau \) indicates the number of elements belonging to \(W_{i}\) and \(x_{j}\) are all elements of clustered nuclei, respectively. A large value of \(\hbox {Fitness}\left( {\hat{k}}\right) \) indicates that the ellipsoidal models are not well fitted to the clusters.

Based on the calculation of these three metrics (separation, compactness and fitness), we can easily compute the validation index V of the estimated clusters from Eq. (4). Then, we can simply claim that if a new EM solution increases the validity index V, the weak seed \(s_\mathrm{w}\) can be considered as a spurious seed and is deleted from the list. The same procedure is repeated for the identification of other spurious seeds in the list until the new validity index is lower than the previous one. Finally, the segmentation of the clustered nuclei is performed by applying Bayesian classification (Fig. 4), i.e., a pixel x is assigned to cluster \(C_{i}\) with the maximum posterior probability.

4 Nuclei classification

After the segmentation of nuclei, the extra cellular material is removed along with a number of follicular dendritic cells, appearing as blue-colored nuclei in PAX5-stained images. In order to assess the shape of nuclei, the perimeter of each nucleus is extracted and the best fitting ellipse is estimated using the Orthogonal Distance Regression (ODR) algorithm [27]. Subsequently, ellipse residual is being estimated as the average geometric distance of the pixels in the perimeter from the ellipse. This feature is referred to as nuclear regularity, since it estimates the regularity of the shape of the nucleus.

Since morphological characteristics are not enough for the classification of nuclei in CBs or non-CBs, textural information needs to be extracted from the corresponding H&E-stained images. To do so, we first need to align H&E-and PAX5-stained images corresponding to the same tissue section. Our main concern here is to address mostly rotation and translation problems as by using the same camera setup and tissue section (with different stains) we eliminate possible scale or skew issues. Toward this end, we apply intensity-based rigid image registration [28] to find the correct location of the corresponding nuclei in H&E images.

Fig. 5
figure 5

Representative case of a a CB and b a small non-CB cell in a H&E image

Full size image

Having identified the corresponding nuclei in H&E-stained image, textural analysis can be applied to extract features related to the internal and external texture of nuclei as well as the number and the size of nucleoli in their interior. More specifically, while non-CBs have relatively dark nucleus, CBs are brighter and characterized by a higher non-uniformity in their texture, as shown in Fig. 5. This is also justified by the existence of a number of nucleoli in their interior. On the other hand, it has been noticed that CBs do not have sufficient cytoplasm and the exterior of their nuclei appears brighter than that of non-CBs. To encapsulate the properties of internal texture of CBs, we initially estimate the mean value and skewness of the grayscale histogram of nuclei and then we apply the GLRL algorithm [29] to calculate the gray-level non-uniformity (GLN) value, which is a quantitative measure of textural abnormalities in a nucleus. Similarly, for the external texture of nuclei, the variance of the cytoplasm histogram is estimated for the surrounding area of each nucleus. Finally, for the detection of nucleoli, the expectation maximization (EM) algorithm is used to estimate the number of different intensity zones (classes), by using the minimum description length (MDL) criterion [30]. Assuming that intensity values are modeled as a Gaussian mixture model, MDL works by attempting to find the model order which minimizes the number of bits that would be required to code both the input data samples (intensity values) and parameters of the Gaussian mixture. Since, the darkest class corresponds to the nucleoli, circular Hough transform [31] is subsequently applied to detect small dark circles in the nucleus representing nucleoli [10]. After the detection of nucleoli, their number, total area and maximum area are calculated for each nucleus.

To better model the decision making procedure of pathologists, we propose a hierarchical Bayesian network (BN) classifier, in which the final decision is not directly affected by the initial features extracted from the pair of images, but by a level of intermediate nodes aiming to quantify different properties of nuclei. The structure of the proposed directed acyclic graph (DAG) is depicted in Fig. 6. More specifically, the proposed BN classifier receives as input a feature vector containing nine individual features (two morphological and seven textural) extracted after the identification of nuclei in both PAX5- and H&E-stained images. Finally, the decision for the classification of a nucleus is taken based on the conditional probabilities of the parent nodes in the second level of DAG, which quantify different properties of nuclei associated with their morphology, internal texture, nucleoli characteristics and external texture.

Fig. 6
figure 6

Structure of the proposed BN classifier

Full size image

5 Experimental results

We created a dataset consisting of 28 pairs of PAX5- and H&E-stained HPF images (i.e., 56 images of size \(1280\times 960\)) using a Nikon Eclipse E600W microscope and a Nikon DS-Fi1 digital camera. For the preparation of the 28 pairs of images, containing in total more than 25.000 FL nuclei, we adopted the following procedure: Initially, we stained each slide with hematoxylin and eosin. Then, for the removal of H&E stain, we placed each slide in xylene to dislodge the coverslip. The slides were immersed in 0.3 % HCl solution until hematoxylin and eosin are almost completely destained. The procedure of differentiation was used to remove excess colour in overstained sections. The almost unstained section was rinsed in tap water for 5–10 min, in order to remove acid and then immunohistochemistry for PAX5 (rabbit monoclonal, clone RBT-PAX5, dilution 1:30, BioSB, Santa Barbara, USA) was performed, using the Bond Max autostainer (Leica Microsystems, Wetzlar, Germany). The above procedure was repeated four times in different days (leading each time to seven pairs of images) in order to avoid any dependence of the proposed technique to the initial sample preparation. Moreover, we manually annotated three hundred clustered nuclei consisting of two to five cells. In order to crop the images containing the clustered nuclei, we initially ran the energy minimization segmentation algorithm on the HPF images stained with PAX5 and we visually identified the connected cells. The surrounding area was cleaned from smaller objects so that each image contains just one cluster of nuclei.

Fig. 7
figure 7

Evaluation of the proposed validation index against its three components (separation, compactness and fitness) and the validation index proposed in [25] and applied in [16]

Full size image

5.1 Separation of clustered nuclei

To evaluate the proposed method for the separation of overlapped cells, we used the aforementioned dataset consisting of three hundred clustered FL nuclei. Specifically, in this section we elaborate a detailed evaluation analysis of the proposed validation index and we validate the performance of the proposed algorithm against three other state-of-the-art techniques. For the evaluation of the proposed validation index, we compare its performance against the validation index proposed in [25] and applied in [16] for the segmentation of overlapped nuclei on specimen of cervical cells and mammary invasive ductal carcinomas. In addition, we present a detailed analysis of its components, i.e., separation, compactness and fitness, in order to assess their contribution to the segmentation process. As shown in Fig. 7, the proposed validation index outperforms with a correctly segmented nuclei rate of \(95\,\%\). This result is due to the introduction of the new fitness function, which provides significant information regarding the spatial relation of ellipsoidal models with the morphology of clustered nuclei. On the other hand, the validation index in [25] is based only on the estimation of the separation and compactness factors ignoring completely the fitness degree of the Gaussian components to the clustered nuclei. To this end, EM solutions consisting of components with large overlapped areas or areas that exceed the borders of the clustered nuclei may be erroneously selected as the optimum ones. As a result using the validation index in [25], the correctly segmented nuclei rate falls to \(87.67\,\%\).

Figure 8 shows the comparison of the proposed method with three state-of-the-art methods: the GMM-based [11] algorithm proposed in our previous work for cell splitting in H&E-stained images of 1 \(\upmu \)m, the fast radial symmetry transform (FRST) [32], which was applied in [8] for the segmentation of overlapped FL nuclei in 4-\(\upmu \)m slides, and the classical watershed algorithm. The introduction of the responsibility criterion for the identification of weak seeds in conjunction with the proposed validation index reduces significantly the number of spurious seeds and result in a low oversegmentation rate for the proposed method. On the other hand, and as was expected, the watershed algorithm leads to high oversegmentation results, while the proposed separation method seems to be extremely robust to undersegmentation.

Fig. 8
figure 8

Comparison of the proposed separation method against three state-of-the-art methods

Full size image

5.2 Nuclei classification

The proposed methodology for the identification of CBs was evaluated over a dataset of 25 pairs of PAX5- and H&E -tained HPF images, while for the training of BN classifier we used CB and non-CB cells from three additional pairs of images. As shown in Fig. 9, our method provides an average F-score \(96.53\,\%\) (average true-positive—TP—rate of \(94.56\,\%\) i.e., 139 correctly detected CBs out of 147, with a false-positive—FP—rate of \(2.78\,\%\), i.e., around 22 nuclei per slide, which is a relatively low false-positive rate). To compare the performance of the proposed method with a state-of-the-art approach, we decided to apply [11] (which was also developed for tissue sections sliced at a thickness level of 1 \(\upmu \)m) to the same slides stained with H&E, in order to have a fair comparison. Experimental results show that the proposed method achieves improvements up to 10.89 and \(4.10\,\%\) in terms of average detection rate and false-positive rate, respectively.

Fig. 9
figure 9

Average and the individual F-scores for the 25 pairs of PAX5 and H&E images of size \(1280\times 960\)

Full size image

Figure 10 presents the different steps of the proposed methodology in a Region of Interest (ROI) of Slide 10 containing two CBs and a FL dendritic cell. As it can be easily seen in this characteristic example, the identification of the dendritic cell in the H&E-stained image is extremely difficult, since it has similar characteristics with the other two CBs, e.g., large size, nucleoli etc. However, in the PAX5-stained image, the energy minimization algorithm can easily discard it, as shown in Fig. 10c, thus reducing the false-positive rate. It should also be noted that after the segmentation step, one of the CBs forms a cluster with two other nuclei. Although the size of this CB is much larger than the other clustered cells, almost double from the middle one, and its shape is more elongated, the proposed algorithm can successfully segment it. On the other hand, a possible oversegmentation in this stage, e.g., by using Watershed algorithm, could lead to its final misclassification, i.e., it would reduce the detection accuracy of the algorithm. For the extraction of textural information, the two images are aligned by applying intensity-based rigid registration to find the correct location of the corresponding nuclei in H&E stained images (Fig. 10e). Finally, the CBs are detected by combining the morphological and textural characteristics of nuclei using the proposed BN classifier (Fig. 10f).

Fig. 10
figure 10

Different steps of the proposed methodology using a ROI containing two CBs (black circles) and a dendritic cell (red circle). a The PAX5-stained image, b the corresponding H&E-stained image, c segmentation result, d splitting of clustered nuclei, e registration result and f classification result (color figure online)

Full size image

To evaluate the performance of the proposed classifier, we compare it with five other state-of-the-art classifiers using the same training set: Adaptive Neuro-Fuzzy Inference (ANFIS) [11], Adaboost, SVM with radial basis function kernel, Neural Networks (NN) and Classification Trees (CT). As shown in Fig. 11, the proposed BN classifier outperforms providing a higher detection rate than the other classifiers, while at the same time the number of false detections remains relatively low, i.e., \(2.78\,\%\) instead of 2.81, 3.41 and \(2.69\,\%\) for ANFIS, Adaboost and SVM, respectively. On the other hand, while NN and CT provide lower FP rates, their detection rates is prohibitively low.

The contribution of both morphological and textural features extracted from PAX5- and H&E-stained images, respectively, is depicted in Fig. 12. As can be seen, although nuclei morphology leads to higher detection rate than textural features, it also provides many false detections, which is sensible since many non-CBs have similar morphological characteristics with CBs. On the other hand, the extraction of textural features from H&E-stained images seems to contribute significantly to the robustness of the algorithm, although their detection rate is lower. This observation shows that the role of morphological and textural features can be considered supplementary in the classification process and for this reason their combination in the proposed classifier results in improved CB detection (TP) and false detection (FP) rates.

Fig. 11
figure 11

Comparison of the proposed classifier with five other state-of-the-art classifiers

Full size image
Fig. 12
figure 12

Contribution of morphological and textural features in the classification process

Full size image

6 Discussion and conclusions

Typically, computer-aided detection of CBs for FL grading is performed using H&E-stained tissue sections sliced at a thickness of 4–5 \(\upmu \)m. The use of such images facilitates the segmentation of distinct cytological components and, especially, the discrimination of nuclei from the cytoplasm. However, while morphological characteristics of nuclei can be accurately extracted, characteristics related to nucleoli or the internal texture of the nucleus are not easily traceable. As referred to [9], the classification accuracies of methods using tissue sections sliced at this level range from 75 to \(85\,\%\) (e.g., [6] provides an average detection accuracy of \(80.7\,\%\) and average false-positive count of 30 cells per image). By using tissue sections sliced at a lower thickness level, we can increase the detection rate since more information about the interior of nuclei is revealed [11]; however, the false-positive rate remains relatively high. Experimental results showed that the proposed method, based on PAX5 and H&E stains and tissue sections sliced at the thickness of 1 \(\upmu \)m, can increase the detection rate, while providing a relatively low false-positive rate. In the future, we aim to examine the use of histopathology images of consecutive tissue sections stained with different histochemical or immunohistochemical stains in order to avoid the destaining procedure of slides.