1 Introduction

Approximately 14 million people are diagnosed with cancer by an expert each year, and 8 million of them die of it or related complications. The early diagnosis of cancer is of vital importance in surviving the disease. With technological development, the use of technological devices for diagnosis contributes to the early diagnosis of many diseases and further helps to start necessary treatment. Magnetic resonance imaging (MRI) and computed tomography (CT) are devices frequently used by experts because they supply reliable information about the internal human structure and functions by various means. Pathology plays an important role in early cancer diagnosis. Following the pathological pre-processes (staining, etc.), pathologists examine the suspected cancerous tissues in the laboratory. These examinations made by pathologists consist of diagnosis of the disease upon morphological and functional analysis of cellular structures, tissues, and organs. One of the most crucial processes for diagnosis is the determination of cellular structures. The cellular structures of cancerous tissues morphologically differ from the cellular structures of non-cancerous tissues. Analyzing each cellular structure one by one is a difficult and time-consuming process for pathology experts. The aim of this study is to automate this difficult and time-consuming process with the help of imaging equipment and digital image processing techniques. This process is called computer-aided diagnosis (CAD). The purpose of CAD is to establish secondary decision support systems that contribute to early diagnosis by analyzing the digitized histopathological images in a computer environment.

In recent years, many studies have been carried out on the segmentation of cellular structures in histopathological images [13, 25, 30, 32]. These studies were performed using clustering-based, thresholding-based, and graph-based algorithms. In [3], Al-Lahham et al. proposed a method of segmenting cellular structures. First the RGB image was transformed to La*b* color space. Then, a k-means algorithm was performed to find the local regions. Finally, a global thresholding and a number of morphological operations were performed to find the cellular structures. In [31], Xu et al. performed the segmentation process with an adaptive thresholding method. After that, the segmented image was determined by observing the elliptical curve to find near-circular nuclei. Similarly, Lu et al. also proposed a robust method that performs an adaptive filter and observes the elliptical curves to detect the cellular structures [19].

Pathologists perform various pre-processing operations, such as fixation, staining, cutting, and scanning, before examining the tissue obtained by biopsy. The more successful these pre-processes are, the better discriminated the cellular structures are from cytoplasm and fatty tissue. The various disruptions, which can occur at any stage of these pre-processes, lead to the formation of various artifacts that resemble the cellular construction. These artifacts can reduce the segmentation success in computer-assisted analyses. While cellular structures are segmented in digital histopathological images, global assessment of all pixels in the plane may reduce segmentation success. On the other hand, an analysis at the local area may improve the segmentation performance since it favors differences in the cellular structures compared to the environment. From this point of view, the superpixel approach, which focuses on local area of the image, gains importance.

Superpixels are a group of pixels that are merged according to their brightness information and neighborhood relationship. Several superpixel approaches have been proposed that are very successful in object recognition, remote sensing and target tracking [9, 10, 27]. Ochs et al. proposed a superpixel segmentation method for the purpose of object detection in motion data [23]. A distance measure was introduced that incorporates color, spatial information (xyz), and temporal information [6]. The method is successful in detecting moving objects and foreground objects. In [22], a novel segmentation method was proposed by Meng et al. to segment common objects from multiple images. Then, a digraph was constructed to represent the relationships between different local regions based on their similarities. In [24], Schick et al. proposed a foreground segmentation superpixel method as a post-processing framework based on probabilistic superpixel Markov random fields. A robust superpixel method was also proposed for foreground target detection [33]. In that study, it was shown that the use of superpixels provides flexible and effective mid-level cues. A local region around the target was segmented into superpixels, and confidence values were assigned to them to form a confidence map by computing the distance between a superpixel and clusters.

While the superpixel approach has been applied in some biomedical image processing areas, such as brain MRI imaging segmentation, optic disc segmentation, glaucoma screening, etc., [8, 17, 21], there are not many studies on the superpixel approach in digital histopathological image analysis. Superpixel methods used in histopathological image analysis are about segmentation of regions rather than cell segmentation [26]. In [5], a superpixel-based method was used to separate small segments of breast tissue images. Subsequently, the cell nucleus and cytoplasm in each small segment were separated into epithelial and stromal regions by a support vector machine (SVM) classifier. Similarly, superpixel-based SVM has been used to distinguish epithelial regions from stroma in oropharyngeal squamous cell carcinoma [4]. In [2], superpixel classification algorithms were proposed for tumor localization in breast tissue. The bag-of-words method was adapted to incorporate spatial information. In [29], a simple linear iterative clustering (SLIC) superpixel algorithm was used as an initial step for colorectal image segmentation. Each image was divided into 168 superpixels by SLIC. Then, a similarity metric between those superpixels was calculated for further analysis. Finally, a normalized graph cuts algorithm was applied to merge them into one segment. The contribution of the superpixel approach to this area is crucial, especially when cellular structures are thought to have distinctive characteristics from their local region.

This study aims to present four contributions to the field of cell segmentation in histopathological images:

  1. (i)

    To compare the performance of the clustering-based segmentation algorithms (global clustering where all pixels in the plane have equal weight) and the segmentation performance of superpixel segmentation algorithms (where neighbors’ distances in local regions are added as a feature),

  2. (ii)

    To examine the effect of SLIC superpixel segmentation method which is used as a pre-segmentation algorithm to the clustering-based segmentation algorithms in digital histopathological images,

  3. (iii)

    To compare the segmentation performance of well-known superpixels segmentation algorithms,

  4. (iv)

    To perform time performance analysis of both the global clustering algorithms and superpixel segmentation algorithms.

2 Methods

In this section, we present the formulation of the proposed two-staged segmentation method. We first introduce the clustering-based segmentation methods and the superpixels segmentation methods used in this study. In Section 2.3, the detail of the proposed study is explained.

2.1 Global clustering-based segmentation

2.1.1 K-means clustering algorithm

The K-means algorithm is one of the unsupervised machine learning methods used to group data which have no label information [20]. Since there is no label information, it performs grouping according to their similarities by comparing the data with each other. The number of groups is determined by the number of k clusters to be entered by the user. The processing steps of k-means algorithm are as follows:

figure d

2.1.2 Fuzzy c-means clustering algorithm

The fuzzy c-means (FCM) algorithm was first proposed by Donn et al. and improved by Bozdek et al. [7, 11]. Unlike the k-means algorithm, it is based on the probability that each of the samples to be clustered belongs to a cluster center in a certain place, rather than being precisely assigned to a cluster center. The general process steps of fuzzy c-means algorithm are listed below:

figure e

2.2 Superpixel-based segmentation

2.2.1 Simple linear iterative clustering (SLIC) algorithm

Simple linear iterative clustering (SLIC) is a superpixel segmentation algorithm proposed by Achanta et al. [1]. SLIC is a k-means-based algorithm that clusters the neighboring pixels by considering their color and coordinate information. In that study, the color space was transformed to CIE Lab color space since Lab is perceptually uniform for small color distance. The SLIC algorithm does not directly calculate the Euclidean distance of the color and coordinate information in the same formula since one of them could negatively affects the other. Equation 1 represents the distance calculation of the intensity values within the same specific grid size:

$$ d_{lab} = \sqrt{(l_{j}-l_{i})^{2} + (a_{j}-a_{i})^{2} + (b_{j}-b_{i})^{2}} $$
(1)

where j represents the center pixel, and i represents the value to be clustered. The value of dlab is the distance of the corresponding pixel to the center. L,a and b represent the brightness values of the respective pixels. The Eq. 2 also represents the distance of the coordinates of each pixel to the related cluster center,

$$ d_{xy} = \sqrt{(x_{j}-x_{i})^{2} + (y_{j}-y_{i})^{2}} $$
(2)

where xj and yj are the horizontal and vertical coordinate information of each center pixel, and xi and yi values are the coordinate information of each pixel to be clustered.

$$ d_{s} = d_{lab} + m/N * d_{xy} $$
(3)

The value of ds is the sum of the (x, y) plane distance normalized by the grid interval N and the lab distance. Here, normalization is done so that the calculation of the coordinate information does not directly affect the brightness interval. The value of m is defined to set the compactness of superpixels.

2.2.2 Simple linear iterative clustering via density-based spatial clustering of application with noise (SLIC-DBSCAN)

SLIC-DBSCAN is a superpixel segmentation algorithm using the density-based spatial clustering of applications with noise (DBSCAN) [1, 12, 15, 16]. The DBSCAN algorithm has more potential to segment irregular objects in order to create more regular superpixels. In this algorithm, the pixels in the image were merged to get initial superpixels by using the SLIC superpixel segmentation algorithm. Then, these initial superpixels are merged to obtain final segments. The success of this algorithm is due to the use of DBSCAN as a segmentation algorithm to group the similar neighbor superpixels. The DBSCAN algorithm improves the performance of segmentation by adding local geometric boundaries.

DBSCAN is a clustering algorithm based on the measurement of density in regions close to a given object (superpixel). There are two important parameters in the segmentation of the data in the DBSCAN algorithm, epsilon and minimum points (MinPts). The epsilon parameter represents the radius of a circle that encompasses superpixels around a given superpixels center. It is called the 𝜖-neighborhood of x . ’MinPts’ is the minimum number of superpixels within the epsilon distance. Figure 1 demonstrates the simple logic of the DBSCAN algorithm.

Fig. 1
figure 1

Basic representation of DBSCAN algorithm. Point × and the other orange point are core points because the area surrounding these points in an epsilon radius contain at least six points

Full size image

Different types of points (core superpixels, border, and outlier superpixels) are included in the Fig. x is a core point because the neighbor of x is 6, and y is a border point because neighbors of y are less than MinPts, but it belongs to the 𝜖-neighborhood of the core point x. Z is a noise point. An initial SLIC algorithm is used to merge neighbor pixels. Similar superpixels are then merged with the core superpixels if they are in a specific 𝜖 area.

Any pixel in the plate which has a number of neighbors grater than or equal to ‘MinPts’ is called a ’core pixel’. However, x is border point, if the number of its neighbors is less than ‘MinPts’.

2.2.3 Topology preserved regular superpixel (TPRS)

Topology preserved regular superpixel (TPRS) and entropy rate superpixel segmentation (ERS) were used to segment the nuclei in this proposed study. This is aimed at both evaluating the segmentation performance of both algorithms and to compare those algorithms with the proposed method.

TPRS is a comprehensive method proposed by Tang et al. to generate regular superpixels in salience images [28]. It basically consists of three steps: First, initial seeds are arranged under a lattice grid and associated with proper pixels over the boundary map. Second, each seed is relocated to the pixel with locally maximal edge magnitude depending on both distance term and probability term. Finally, local optimal path connected of each relocated seed is generated vertically and horizontally. The detail of the method is well expressed in [28] and the source code of the algorithm is also publicly available.

2.2.4 Entropy rate superpixel (ERS) segmentation

Entropy rate superpixel (ERS) segmentation is a graph-based superpixel segmentation method proposed by Liu et al. Unlike other well-known superpixel segmentation algorithms such as SLIC or SLIC-DBSCAN, ERSS tries to find compact, homogenous superpixels by using a graph-based approach. A novel objective function on the graph topology is presented. This objective function consists of two components: entropy rate and balancing term. While the entropy rate providing the formation of compact and homogeneous clusters, the balancing term provides clusters with similar sizes [18].

2.3 The proposed method for cellular structure segmentation

K-means and fuzzy c-means algorithms assume red, green, and blue intensity values of each pixel in the image as features (1 × 3 feature vector - [R,G,B]) and calculate an Euclidean distance between each pixel and the cluster centers. Since only the color information is used in the global-based segmentation approach, the calculation is done independently from coordinate distance. So, adding the coordinate information as a feature could increase the segmentation performance. This is the motivation behind the superpixel segmentation methods. In this study, we propose a method that combines these two approaches for segmentation of cellular structures in histopathological images.

In the first experiment, segmentation results were obtained by applying k-means and fuzzy c-means methods, which are well known as segmentation algorithms. The k-means and the fuzzy c-means methods assume that all the pixels in the plane are of equal weight. So, the segmentation performance obtained using these algorithms was evaluated by calculating all the pixels in the image independent of the distance. Therefore, it is important to compare the superpixel methods with these algorithms. For evaluating the k-means and fuzzy c-means algorithms, each color image in the data set was smoothed by applying a 5 × 5 median filter before clustering. This process suppressed the noisy pixels in the image and helped to achieve a more successful segmentation. After that, a clustering algorithm with six cluster centers was applied to each color image. At this step, red (R), green (G), and blue (B) intensity values of each pixel were used as input features in segmentation process. Then, a morphological operation was applied to the image obtained by segmentation. Small artifacts resembling cellular structures were eliminated. This process was done both for the k-means and fuzzy c-means algorithms. Figure 2 represents two different types of kidney renal cell carcinoma images taken from the data set and the histogram of the grayscale of those images to mention the pixel distribution. Figure 2a represents a digitized image that does not include many tissue-like structures. However, in Fig. 2b, there are some tissue-like structures (connective structure) that could affect the segmentation performance. Figure 2c and d represent the histogram values of the grayscale of those images. Applying clustering-based segmentation algorithms with three clusters in histopathological images may not always achieve promising results because of noise and artifacts. So, increasing the number of clusters could be a solution to handle this segmentation problem. A total of six clusters are thought to be fat tissue, connective tissue, cellular structure and cell-like, fat-like and connective tissue-like. The parameters ’exponent’ and the maximum number of iterations for FCM were set to 2 and 100, respectively.

Fig. 2
figure 2

The cellular structures in each image may have different darkness depending on the quality of staining and scanning operations. a and b Two images obtained from data sets which have cellular structures with different darkness. c and d Histogram values of the images after transforming to grayscale

Full size image

In the second experiment, the contribution of the superpixel method to cell segmentation as a pre-segmentation process was examined. Figure 3 shows the processing steps followed at this stage. Unlike the first part of the study, the superpixels in the plane were segmented by the k-means and the fuzzy c-means methods. The superpixel segmentation algorithm proposed by Anchanta et al. was used for this purpose [1]. Figure 4 represents (a) an original image obtained from the data set, (b) a superpixel method implemented to this image, and (c) the clustering result of the superpixels created from the image. The minimum superpixel size can be taken as 10 × 10 when the average area of each nuclei is assumed to be 20 × 20. Approximately, \( \frac {ImageWidth \times ImageHeight}{10 \times 10}= 1600 \) could be chosen as initial superpixels for image to be segmented. When all the images in the data set are examined, 4000 superpixels or greater performs good results to extract the cellular structures. After this process, the background information is eliminated when cellular structures are obtained from the segmented image. Finally, artifacts that resemble the cellular structure of the segmented image are eliminated if their area is less than 100 pixels. The value of 100 is not greater than the smallest cellular structure in the ground truth. The RGB color space is used for SLIC superpixel segmentation algorithm in order to make fair comparison with k-means and fuzzy c-means.

Fig. 3
figure 3

Processing steps of the proposed study

Full size image
Fig. 4
figure 4

a The superpixel segmentation result applied to an image. b The clustering result of that image. c The extraction of the cellular structures from the corresponding image

Full size image

In the third experiment, the performance of SLIC-DBSCAN, ERS, and TPRS superpixel segmentation algorithms in histopathological images are represented. Similar to the previous two stages, a median filter with 5 × 5 windows size is applied to the color images. For each superpixel segmentation algorithm, the numbers of superpixels between 500 and 1500 were tested empirically. Since the best segmentation performance was achieved with 1000 superpixels, its results are represented for comparison purpose. In SLIC-DBSCAN, the weighting factor (m) between color and spatial differences was determined as 30. The large value of m enforced the superpixels to be more regular. The radius for merging the region was set to 1. Regions smaller than the radius were merged with the adjacent regions. The resulting superpixels were updated to have average information about the pixels they contained. Then, any superpixels whose brightness value was greater than the threshold level of 130 were eliminated.

ERS and TPRS algorithms were also used for comparison with other segmentation algorithms. Similar processing steps were followed for TPRS and ERS superpixel methods. First, the obtained image is smoothed with 5×5 median filter. Then, the smoothed image is segmented to 1500 and 3500 superpixels for ERS and TPRS, respectively. One point to note when using such superpixel methods in cell segmentation is that less than a certain number of superpixels may reduce the segmentation performance. In this case, since there are more than a certain number of superpixels, segmentation does not significantly affect the performance. Then, superpixels which had a value below 80 were assumed as cellular structures, the rest were assumed as noncellular structures (background information). Finally, artifacts resembling the cellular structure were eliminated.

3 Results

3.1 Data set description

The data set used in this study is obtained from Beck Laboratory at Harvard University. The data set consisted of high-resolution histopathological images of renal cell carcinoma selected from The Cancer Genome Atlas (TCGA) data portal and publicly available for usage. There are 810 high-resolution 400 × 400, histopathological images of ten kidney renal cell carcinomas. Images were scanned using 40 × magnification. TCGA is a high-scale cancer research organization financed by the American National Cancer Institute and the National Human Genome Research Institute. In addition, TCGA conducts surveys to find solutions to the 25 most common cancer types. In addition to collecting molecular and clinical data, TCGA also obtains whole slide images (WSI) under cancer research. Figure 5 represents sample images taken from the data set and the ground truth images annotated by pathologists of those images. The data set was introduced in [14].

Fig. 5
figure 5

Sample histopathological images and the ground truths taken from data set

Full size image

3.2 Performance metrics

Segmentation results of the proposed study were compared with ground truth images, which were labeled by pathology experts. The segmentation metrics used to evaluate segmentation annotation include: true positive (TP), true negative (TN), false positive (FP), false negative (FN), precision, and recall. True positive represents the number of pixels in the cellular structure area that are correctly labeled as positive samples, while true negative represents the number of pixels that are outside of the cellular structure area and labeled as negative. False positive and false negative correspond to the number of pixels outside of the cellular structure area that are, however, labeled as positive and the number of pixels located in the cellular structure area and labeled as negative, respectively. Equation 4 represents the precision value which is the ratio of TP value to the positive predictions.

$$ precision = \frac{TP}{TP+FP} $$
(4)

TPR (recall) value which is the ratio of TP to the all observations in actual class is calculated in Eq. 5.

$$ TPR (recall) = \frac{TP}{TP+FN} $$
(5)

Precision and TPR should be used together to illustrate the overall performance of the evaluated method. Hence, the Eq. 6 represents the F-measure (F-M), which is the harmonic mean of precision and recall.

$$ \mathrm{F-M} = \frac{2 \times precision \times recall}{precision+recall} $$
(6)

True negative rate (TNR) denotes the proportion of negatives that are correctly identified.

$$ TNR = \frac{TN}{TN+FP} $$
(7)

Equation 8 shows the overlap value, which is the ratio of intersection of segmented output image and ground truth image to the union of segmented image and ground truth.

$$ Overlap Ratio (OR) = \frac{A(S) \cap A(G)}{A(S)\cup A(G)} $$
(8)

where A(S) represents the area of segmented image and A(G) represents the area of ground truth.

3.3 Evaluations

Figure 6 represents the (a) precision, (b) recall, (c) f-measure values and the (d) overlap ratio of the proposed method, which uses SLIC algorithm as a pre-segmentation algorithm and k-means as post-segmentation algorithm. The horizontal axis indicates the number of superpixels from 2k (2000) to 20k (20,000) and the vertical axis shows the variation metrics depending on the superpixels numbers. In Fig. 6a, the lowest precision value is obtained with three cluster centers. The precision value increases as the number of cluster centers increases. Figure 6b represents the variation of recall value depending on the number of superpixels. In this case, the greatest recall value is obtained with three cluster centers and the recall value decreases as the number of cluster centers increases. Obtaining high or low value of precision and recall alone is not enough to assess the segmentation performance. Combining the precision and recall is more valuable for the segmentation success than using each one. Therefore, the harmonic mean of precision and recall, which is also known as f-measure, is used for evaluation. Clustering of 6000 or greater superpixels with 4-means gives the best f-measure value according to Fig. 6c. It can be seen from the graphs that use of the superpixels as pre-segmentation algorithm provides less cluster usage and better segmentation performance because superpixels eliminate the local variances of pixels. According to Fig. 6, the overlap ratio of four cluster of 6000 superpixels is also the best while comparing with the other combinations.

Fig. 6
figure 6

Segmentation performance of k-means and FCM clustering methods after using SLIC superpixel algorithm as a pre-segmentation method. a precision, b recall, c f-measure and (overlap) are represented, respectively

Full size image

SLIC superpixels algorithm has compactness value for the regularity of superpixels. Figure 7 represents the segmentation performance of the proposed algorithm with different compactness value to evaluate the effect of the compactness. First, the compactness values were tried from 1 to 20, as mentioned in [1] but an acceptable change could not be observed. A low value of m does not effect distances of intensity and coordinates values, which is mentioned in Eq. 2, so much. Although, very large value of m causes the recall, f-measure, and overlap ratio to decrease.

Fig. 7
figure 7

The effect of compactness value to the proposed method with 6000 superpixels and 4-means, which gives the best results

Full size image

Table 1 represents the segmentation results obtained from the proposed study. The results are quite close when compared to the results of the paper, which introduces the data set [14]. The metrics used to evaluate segmentation annotation include true positive ratio (TPR), precision (Pre.), F-Measure, true negative ratio (TNR), and overlap.

Table 1 Comparison of the segmentation performance for seven algorithms on the kidney renal cell carcinoma data set
Full size table

According to Table 1, the accuracy of estimating the cellular structures as cell of k-means algorithm is 60.00%. The non-cellular structures including fat and blood tissues are segmented with 97.4% performance. On the other hand, the fuzzy c-means algorithm segments the cellular structures by 60.14% success. Fuzzy c-means can segment the non-cellular structures successfully with 97.3%. Although the performances of the algorithms are approximately the same, the fuzzy c-means algorithm works relatively slowly compared to k-means.

The segmentation results of all the seven algorithms are presented in Table 1. From left to right, the columns show the true positive ratio (TPR), precision, F-measure, true negative ratio, overlap, and time performance values of segmentation algorithms. From the results of seven algorithms applied in this study, it can be shown that the best performance results were achieved by SLIC+FCM, SLIC-DBSCAN, and SLIC+K-means, which all achieved F-M scores between 63.7% and 65%. The single K-means and FCM algorithms showed worse performance results with F-M scores of 61.5%. Using the SLIC algorithm as a pre-segmentation method increases the TPR, F-M, overlap, and time performances of the algorithms. The results are also comparable with the results of Irshad et al. (Fig. 8).

Fig. 8
figure 8

Sample results obtained by applying all the seven algorithms to an image taken from the data set

Full size image

The performance criteria in Table 1 are based on the performance criteria in the paper that the data set is published. Accordingly, TPR (true positive ratio) represents the performance of the system that predicts cell region as cell. On the other hand, TNR (true negative ratio) represents the knowledge of pixels not marked as cells by the system and not marked as cells in the ground truth.

3.4 Evaluations for computational time

The experiments were performed on a workstation with a 4.0-GHz Intel core i7-6700K and 48 GB of RAM. All of the seven algorithms are implemented on CPU only, in Matlab 2016a. Superpixels segmentations take 0.1 s per image of size 400 × 400. All computations are represented at the last column in Table 1. Single K-means algorithm clusters all the images in the data set within 60 s. While applying SLIC superpixels algorithm before k-means, the computation time is decreased to 20 s. The FCM algorithm is a slow algorithm while comparing with k-means, respectively. Because instead of directly assigning the label to the candidate pixels, a member function is calculated for each candidate pixel. This operation is a time-consuming process. Similar to the k-means algorithm, applying superpixels before the FCM algorithm decreases the time complexity. Depending on the its complexity, some superpixel segmentation algorithm may perform better than global clustering algorithms. SLIC-DBSCAN performs faster than FCM and SLIC+FCM algorithms but it is slower than the K-means and SLIC+K means algorithms.

4 Discussion

This study aimed to contribute to the cell segmentation process in high-resolution histopathological images by using the SLIC superpixel segmentation method as a pre-segmentation algorithm and to compare the performance of this algorithm to other well-known superpixel segmentation algorithms. SLIC was used as a pre-segmentation algorithm to improve the segmentation performance of the state-of-the-art clustering-based segmentation algorithms. Results of SLIC-DBSCAN, ERS, and TPRS superpixel methods were also compared with the results of the state-of-the-art clustering-based segmentation algorithms, k-means, and fuzzy c-means. ERS and TPRS are chosen as segmentation algorithms since each of them is one of the most well known superpixel segmentation algorithms that use different algorithmic approaches to find the superpixels. ERS is a graph-based superpixel segmentation algorithm and TPRS is an entropy-based superpixel segmentation algorithm. In this study, we also aim to show the segmentation performance of those superpixel segmentation algorithms for the researchers who study histopathological image segmentation. According to the results obtained from this study, the segmentation performance of the superpixel method is successful in high-resolution histopathological images compared to the clustering-based segmentation algorithms. The results showed that clustering of histopathological images with SLIC superpixel algorithm improved both the segmentation performance and computation time while comparing with k-means and FCM. The segmentation performances depending on the parameters of superpixel segmentation algorithms were also represented. It has been observed that the results obtained using the superpixel segmentation methods improved the cell segmentation performance when compared with the study that the data set introduced. The TPRS and ERS methods were also used for comparison to determine the segmentation performance of superpixels methods. In the ERS and TPRS methods, segmenting the given image to less than a certain number of superpixels may affect the segmentation performance negatively. Instead, choosing a number of superpixels greater than a certain value does not significantly affect the segmentation performance. As mentioned in Section 2.3, the initial area of each superpixel should be determined smaller than average nuclei area. Time performance of ERS and TPRS methods was also compared to the other methods. ERS was faster than the FCM, SLIC+FCM, and SLIC-DBSCAN algorithms. However, TPRS was the slowest algorithm in all seven methods because of its performance depending on the quality of the pre-computed boundary maps.

5 Conclusions

In this study, the contribution of superpixel algorithms to cell segmentation in high-resolution histopathological images was examined. The study consists of two parts: first, using the SLIC superpixel algorithm as a pre-segmentation algorithm and using SLIC-DBSCAN superpixel algorithm as an alternative segmentation algorithm to the existing clustering-based segmentation methods. The SLIC superpixel algorithm was applied as a pre-segmentation algorithm to increase the segmentation performance of k-means and fuzzy c-means. There are three main tissues in histopathological images including fat tissue, connective tissue, and cellular structures. So, the optimum cluster number for state-of-the-art clustering-based segmentation algorithm is three. However, various artifacts occur while scanning images and transferring them to a computer environment. These cause the images to have more than three segments. Using six clusters for k-means and fuzzy c-means gave the best results in this study. The details were explained in the first part of Section 3. The advantages of using SLIC superpixel segmentation algorithm is that SLIC smoothens the local variance of neighbor pixels and eliminates the artifacts. This helps k-means and fuzzy c-means to cluster the segmented image with lower cluster centers. Reducing the number of cluster centers also reduces the computation time. The results show that the segmentation performance improved as a result of using the SLIC algorithm as a pre-segmentation algorithm before using clustering-based algorithms, instead of using the single clustering algorithm alone to segment the data. The segmentation performance of the superpixel algorithm was obtained by applying the SLIC-DBSCAN superpixel algorithm. In addition, the effect of superpixel segmentation algorithms on high-resolution histopathological images of renal cell carcinoma obtained from the Cancer Genome Atlas (TCGA) data set was compared to the best-known global-based clustering algorithms, k-means and fuzzy c-means.

While comparing TPRS and ERS with SLIC-DBSCAN, the major advantage of these algorithms is that they are not hyper-parametric. However, the major drawback of TPS is the performance, which depends on the quality of the pre-computed boundary maps.