Keywords

1 Introduction

Malignant melanoma is considered one of the most deadly forms of skin cancer, and the mortality rate caused by it has increased significantly. It is the seventh most common malignancy in women, and the sixth most common in men. However, early detection of it is particularly important, since it can often be cured with a simple excision [23]. Therefore, with an early diagnosis of melanoma, the mortality rate can be reduced.

Dermoscopy is one of the major tools for the diagnosis of melanoma, it is widely used by dermatologists, due to its value in early stages for malignant melanoma detection. It provides better visualization of several pigmented structures such as streaks, dots, pigment networks and blue-white areas, which are invisible to the naked eye [24]. Due to the presence of the hair and several artifacts such as oil, air bubbles and gel drops on the images. In addition, the borders of the lesions are not visible for the radiologists. Computer Aided Diagnosis (CAD) systems became very important and necessary to help the doctors to interpret the images clearly, and support their diagnosis.

In the last two decades, CAD systems of melanoma detection have reduced the gap between the medical and engineering knowledge, since these systems try to mimic the performance of dermatologists when diagnosing a skin lesion area. Thus, help them to differentiate between melanoma and benign lesions in less time [7, 22].

In dermoscopy image analysis, one usually enhances the images first and segments the lesion areas (ROI). This is followed by extracting several features, which could be local or global, and using them to learn an appropriate classifier, in order to predict the lesion label (melanoma or non-melanoma). Each step of above process depends on the previous one. For instance, the classification stage depends on the performance of all previous steps. Therefore, to get a high classification rate and increase the accuracy of the diagnosis of skin cancer, all or most of the previous steps should be implemented with the best strategy. It is well- known that the skin lesion classification methods are usually based on the feature extraction. Therefore, the extraction of representative features of the lesions under analysis is a very important stage for efficient classification [19]. For this purpose, we focused on the best way to extract the optimal features from images, using an improved method to segment the lesion.

In this paper, four main steps have been implemented to build an automatic process for detection of malignant melanoma in dermoscopy images. First, a pre-processing step was applied on each image for the purpose of removing artifacts such as air bubbles, hairs and lightening reflection. Second, the Particle Swarm Optimization (PSO) [13] and the Markov Random Field (MRF) [17] methods are integrated to segment the lesion areas. Therefore, the k-means was applied on segmented image (lesion), in order to separate each homogeneous set of pixels in one group (cluster). Consequently, several features are extracted from the given lesion based on existing clusters. Thus, by using a trained classifier, the lesion is classified in one of two classes of benign and melanoma.

The paper is organized as follows. Section 2 provides an overview of previous works. The proposed approach is explained in detail in Sect. 3. The evaluation of the detection system and the results obtained are discussed in Sect. 4. Finally, conclusions are presented in Sect. 5.

2 Previous Work

In recent years, there has been an increasing interest in early detection of skin cancer using CAD systems. Most of these diagnosis techniques are based on the ABCD rule to analyse four parameters (Asymmetry, Border, Color and Diameter) [4, 6, 10, 11, 20]. In addition, the 7-point checklist criterion is also widely used for the same purpose [3]. Barata et al. [8] proposed a new approach to extract pigment networks from dermoscopy images using a bank of direction filters and many morphological operations. Two distinctive properties: region pigment network intensity and geometry were used, and several features were extracted. Then, an Adaboost algorithm was used to classify the given region as either normal or abnormal. Also, a Bag-of-Features (BoF) model for the classification of melanoma in dermoscopy images was implemented by Barata et al. [7]. The authors used two different types of local descriptors: color and texture, and their performance was evaluated separately, and then compared in order to assess their ability to describe the different dermoscopic features. The same research group presented a new method to classify skin cancer images as either melanoma or non-melanoma [5]. Color features and texture features were used based on color histograms in three different colors (HSV, L*a*b*, and Opponent) and gradient related histogram. All used features were extracted globally and locally from each image. Therefore, the authors investigated the best way to combine the features by applying two strategies (early and late fusion). A Random Forests classifier yielded the best results. Celebi et al. [10] proposed a machine learning method for automated quantification of clinically significant colors in dermoscopy images. The K-means clustering approach was used to cluster each image with an optimal K value, which was estimated separately using five commonly used cluster validity criteria. Eltayef et al. [15] proposed an automated skin cancer diagnosis system on dermoscopy images using pigment network structures. Five features were extracted from the segmented image (pigment network) and used to feed the Artificial Neural Network as classification stage. An automatic framework for detection of melanoma from dysplastic nevi was proposed by Rastgoo et al. [21]. They combined several extracted features such as colour, shape, size and texture features with well-known texture features such as local binary pattern, grey-level co-occurrence matrix, histogram of gradients and the Gabor filter. Support Vector Machines (SVM), gradient boosting and random forest methods were used to evaluate the performance of their work. A new methodology for color identification in dermoscopy images was introduced by Barata et al. [6]. The authors used the Gaussian mixtures model to learn a statistical model for five colors (black, dark brown, light brown, blue-gray and white). Therefore, the learned mixtures were used to assess the colors of a larger set of images. Alfed et al. [2] proposed a new method for melanoma diagnosis. They used a bank of direction filters to segment pigment networks from images, then extracted an few features from the segmented image and used Artificial Neural Network (ANN) as classifier. The same group introduced a new method for improving a bag-of-words approach by combining color histogram features and first order moments with the Histogram of Oriented Gradients (HOG) [1]. Three classifiers methods were used in their work K-Nearest Neighbors (KNN), SVM and AdaBoost, where the SVM achieved the best results.

3 The Proposed Method

The proposed method for skin cancer detection is described and discussed in this section. As an initial step, the quality of the image is improved by detecting and removing several artifacts, such as air bubbles, lightening reflection and hairs. Then, the skin lesion which is suspected to be a melanoma or non-melanoma is segmented from the surrounding healthy skin by applying PSO with MRF methods. Then, the k-means approach was applied on segmented image (lesion) to segment the lesion into sub region. Then, several features are extracted from each sub region. Finally, an Artificial Neural Network (ANN) classifier is trained to classify the skin lesion as either melanoma or normal skin. The overview of the proposed approach is illustrated in Fig. 1.

Fig. 1.
figure 1

Overview of the proposed approach for skin lesions classification.

Full size image

3.1 Pre-Processing of Dermoscopic Images

Usually, dermoscopic images do not have the expected quality to perform the diagnostic analysis. Thus, the step of image pre-processing is very important and necessary, in order to reduce the amount of artifacts and noise in the images. Skin lines, oil, air bubbles, lightening reflection and dark hair are present in almost every image. Therefore, to improve the quality of the images and prevent false positive detections, these kinds of noise must be detected and removed. According to [22], the blue channel of the color images is selected, as it has been experimentally shown to provide the best discrimination in most images.

Reflection Detection. In dermoscopy images, reflections appear as a result of presence of air bubbles caused by placing oil or jell before capturing the image. A sample thresholding method is implemented to classify each pixel as either a reflection artifact or background. The pixel can be classified as a reflection artifact if its intensity value is higher than threshold \(T_{R1}\) as well as, if its intensity value minus the average intensity \(I_{avg}(x,y)\) of its neighbours is higher than threshold \(T_{R2}\), i.e.

$$\begin{aligned} \{ I(x,y)> T_{R1}\} and \{(I(x,y)-I_{avq}(x,y)) > T_{R2}\} . \end{aligned}$$
(1)

where I is the image, \({I}_{avg} (x,y)\) is the average intensity value in surrounding neighbourhood with dimensions 11\(\,\times \,\)11 and \(T_{R1}\), \(T_{R2}\) are set to 0.7 and 0.098 respectively.

Hair Detection and Inpainting. To improve the quality of the image and obtain correct diagnosis of melanoma skin cancer, the hair which is covering almost all lesions in dermoscopy images should be detected and removed, as a normal first stage of medical image processing. This step is required for effective segmentation and classification steps. To do so, the directional Gabor filters are implemented using a bank of 64 directional filters. Various parameters are used in the Gaussian filters at each phase. The images are filtered by each directional filter with various parameters. Therefore, the difference of Gaussians is carried out, followed by finding the local maximum at each image pixel location. Thus, the threshold approach is implemented to classify each pixel as either hair or background. Finally, the image inpainting method [12] is applied to fill in the gaps, which were occurred by multiplying the hair mask with the gray scale images. The details of the method can be found in [8, 14, 15].

3.2 Image Segmentation

The purpose of image segmentation is to separate the homogeneous lesions from the surrounding healthy skin. It is the most important phase for analysing images properly since it affects the subsequent steps accuracy. However, appropriate segmentation in dermoscopy images is a challenging task, because the lesions have large variations in size, shape and color, as well as, existence of low contrast between the lesions and surrounding healthy skin. In this work the image segmentation stage was implemented in two steps, the lesion area was segmented in the first step while the sub region inside the segmented lesion were segmented in the second.

Skin Lesion Segmentation. The automatic segmentation approach was implemented to extract the lesion area. The PSO and the MRF methods were combined, in order to minimize the energy function. The image segmentation is formulated as an optimization problem of the energy function with MRF theory. The POS method is used to perform the initial labeling based on the optimal threshold value, which was obtained by maximizing the fitness function. Then, an additional local search is performed for each segmented image by integrating it with MRF method. The purpose of that is to minimize the energy function or maximize the probability of pixel allocation to a cluster by using Maximum A Post Priority (MAP) [17]. More details of the approach can be seen in [16].

Sub-Region Clustering. Our aim is to divide the segmented lesion into a few clusters, with a more homogeneous distribution of pixels for each clusters. A simple K-means approach is used for this purpose. The outcomes (binary masks or segmented lesions) from the previous step are multiplied by gray scale images (bottom left in Fig. 2), in order to be able to separate the pixels whose located inside the lesion into several groups. The K-means clustering method is used since it is very simple and has low computational complexity. In addition, the number of clusters (k), usually could be determined easily. Hence the number of clusters has been experimentally obtained and set as k = 5.

The Euclidean distance is used to calculate the distance between the image pixels and the centroids of the clusters. Each single pixel was assigned to the appropriate cluster, based on its distance. Thus, the location of each cluster was updated and the pixels were re-assigned. This process continues until no more changes to cluster membership. The final result of this step is several homogeneous clusters, which can be used for the subsequent step of feature extraction. An example can be seen in Fig. 2.

Fig. 2.
figure 2

Segmentation and clustering: Blue scale image (top left), segmented binary image (top right), segmented blue scale image, i.e. the lesion (bottom left) and result of clustering (bottom right).

Full size image

3.3 Feature Extractions

The third phase of the proposed approach aims to extract several type of features from the segmented lesions. The density and the regular distribution of the blue color are the main properties, and can be used to identify the images. In this work, three color moments with color histogram are used as features, in order to determine the skin lesion type.

Color Moments. Color moments can be used to distinguish images based on their color distribution. Typically, probability distributions are characterized by the number of unique moments [1]. Consequently, they can be used as color features.The first color moment can be interpreted as the average color in each sub region inside the lesion, and can be calculated using the following equation:

$$\begin{aligned} E_{i}= \sum _{i=1}^{N}\frac{1}{N}Pi. \end{aligned}$$
(2)

where N is the total number of pixels inside the sub region and Pi is the pixel value. The second color moment used as a feature is the standard deviation, which can be obtained by taking the square root of the variance of the color distribution.

$$\begin{aligned} \sigma _{i}=\sqrt{(\frac{1}{N}\sum _{i=1}^{N}(pi-E_{i})^2)}. \end{aligned}$$
(3)

where \(E_{i}\) is the average value and N is the total number of the pixels inside the sub region.

The third and last color moment used in our approach is the skewness, which means how asymmetric the color distribution is, and therefore, it gives useful information about the shape of the color distribution. Skewness can be calculated as:

$$\begin{aligned} S_{i}=\root 3 \of {(\frac{1}{N}\sum _{i=1}^{N}(pi-E_{i})^3)}. \end{aligned}$$
(4)

Color Histogram. Color histogram is a way to represent the distribution of the composition of colors in images. It shows the number of pixels in each type of color in the image. The histogram associated with the blue color component \(I_{c},c\in \{3\}\) is given by:

$$\begin{aligned} h_{c}(i)=\frac{1}{N}\sum _{x,y} b_{c} (I_{c}(x,y)) i=1,...,B_{c}. \end{aligned}$$
(5)

where N is the number of pixels inside the sub region, i is histogram bin, \(B_{c}\) in the number of bins and \(b_{c}(.)\) is the characteristic function of ith

(6)

The bins are defined by dividing the color component range into intervals with the same width. For all histograms, the number of bins is given by:

$$\begin{aligned} B_{c} \in \{5,10,15,20\}, \end{aligned}$$
(7)

and it was found that the best performance was achieved when the number of bins set as 10. Therefore, the elements of each sub region are sorted into 10 equally spaced bins between the minimum and maximum values of it.

The color moments and color histogram features are concatenated into one vector in order to represent the image. It is worth mentioning that all the extracted features are obtained from only the blue color moment, since it provides the best discriminatory performance. The same number of features was obtained from all images, three features related to the color moment and ten features from color histogram. These features are used as an image descriptor and the images are classified using machine learning techniques.

3.4 Skin Lesion Classification

The classification stage consists of interpretation and identification of skin lesion information based on the extracted features. Features from all images are stored in a database and fed to an Artificial Neural Network (ANN) as a classifier. The output of the classifier is (1) for melanoma and (0) for normal skin. Two-layer ANN (single hidden layer and output layer) are used with 100 neurons in the hidden layer. The Radial Basis Function (RBF) as an activation function given by [9] is used in the network. After training, the learned classifier can be used to classify new images as a melanoma or not.

4 Experimental Results

The proposed approach was evaluated on a public database PH2 [18] which provides 200 dermoscopy images. Each image in the database was classified by the radiologist as either normal or abnormal (the ground truth labels). The extracted features from all images were used as an input to the ANN classifier, with the ground truth labels as the output. The training and testing process was performed by using a 5-fold stratified cross-validation method. The images were split between five subsets, each one of them had approximately the same number of melanoma and non-melanoma. Several criteria have been used to evaluate the performance of the proposed approach.

To evaluate the performance of our method, we used 5 common evaluation criteria i.e. Sensitivity or True Positive Rate (TPR), Specificity or True Negative Rate (TNR), Accuracy (AC), False Positive Rate (FPR), and Mean Square Error (MSE). The AC is defined as the sum of the true positives (images correctly classified as melanoma) and the true negatives (normal skin correctly identified as a non-melanoma), divided by the total number of images. The TPR is defined as the total number of true positives divided by the total number of images marked in the ground truth as melanoma. The TNR is defined as the total number of true negatives divided by the total number of images marked in the ground truth as normal skin. The FNR is calculated as the total number of false negatives divided by the number of images marked as non-melanoma in the ground truth image. It is worth mentioning that a perfect classification would have sensitivity (TPR) of 1 and FNR and MSE of 0.

The obtained results of our method can be seen in Table 1. It indicates that our approach by extracting several features from the sub region inside the segmented lesion achieved better results in terms of sensitivity, specificity and accuracy. In addition, the FPR and MSE are very desirable.

Table 1. Results of lesion classification in dermoscopy images.
Full size table

Quantitative comparison between various methods is difficult since different datasets and criteria have been used. However, we were able to evaluate the performance of our approach against Barata et al. [5, 7, 8], Eltayef et al. [15] and Alfed et al. [1] as they have the same objectives and they are based on the same database. Table 2 shows the comparison results.

To facilitate the performance comparison between our method and the alternate approaches, parameters such as sensitivity (SE), specificity (SP) and accuracy rates (AC) are computed for each method against the ground-truth. Table 2 shows the results of performance comparison. Out of the three criteria, the proposed method performed better than all other methods by SE and AC, but it is not as good as Eltayef [15] and Barata [5], in terms of SP. However, it is worth mentioning that the ground truth was used as the segmented images in both methods, while we did both segmentation and classification from the original images. Also, the work presented in [5] requires more computational time, because a number of image patches are used for feature extraction and this could be expensive when the patch size is large. In addition, the late fusion strategy was used in their work, which needs extra time to classify each type of features separately.

Table 2. Performance comparison with several methods.
Full size table

5 Conclusions

In this paper, a comprehensive approach to melanoma detection in dermoscopy images was developed. The input images are first pre-processed by detecting and removing the noise. Then the lesions are segmented by applying PSO and MRF methods. As opposed to direct feature extraction from the segmented lesions, K-means is applied and the desired features, such as the color moments and color histogram, are extracted at the sub-region (cluster) level. These features are fed into an ANN with Radial Basis Function as an activation function for final melanoma classification. The proposed approach achieved approximately 96.0% accuracy, 99.00% sensitivity and 85.00% specificity on a dataset of 200 images. A comparison against several alternative methods shows that the proposed method achieved overall superior performance in terms of sensitivity and accuracy. Consequently, it has a great potential to detect melanoma in early stage and support the clinical diagnosis.

The main contributions of the work are as follows.

  1. 1.

    A comprehensive method including the whole process of image enhancing, segmentation of lesions and melanoma classification is developed.

  2. 2.

    A method for lesion segmentation is proposed by combining the PSO and MRF methods;

  3. 3.

    Feature extraction at the sub-region level is performed by separating the segmented lesions into homogeneous clusters.