Keywords

1 Introduction

Skin cancer is a major health concern worldwide affecting the life and wellbeing of people. Melanoma is considered as most deadly skin cancer responsible for the majority of deaths in several countries. According to cancer statistics report 2020, around 100,350 cases are diagnosed and 6850 deaths have been reported due to melanoma skin cancer [1]. Early diagnosis and treatment of melanoma are essential for its control and prevention.

Dermoscopy is one of the efficient imaging techniques addressing the early diagnosis of melanoma [2]. Dermatological photographs provide significant information than macroscopic images which are captured from smartphones or digital cameras. An experienced dermatologist is required to use dermoscopy as an effective tool for improving the accuracy of diagnosis.

The manual criteria used by clinicians and dermatologists for the identification of melanoma lesions mainly include ABCD rule and 7-point checklist [3]. Automatic methods for melanoma diagnosis are preferable as they help in early detection and cure by assisting dermatologists in taking correct decisions. These include pre-processing for reducing image imperfections, lesion segmentation for the location of lesions, lesion feature extraction and lesion classification.

2 Image Pre-processing Methods

The image pre-processing step is applied on both clinical as well as dermoscopic images for removing image imperfections like air bubbles, reflections, hair, ruler markings etc. before the segmentation process. Various methods have been adopted for the removal of hair which is common in images. The first method for removal of dark hair based on morphological operators was Dull Razor algorithm proposed in 1997 [4]. Soft color morphology algorithm based on fuzzy logic [5] for removal of hair in color images preserve the original colors without adding any new chromatic values. An improvement of the Dull Razor algorithm was proposed in [6] by which the orientation of hairs in the image is detected using Radon transform and noise and bubbles are filtered by the Prewitt filter method. Color enhancement can also be done in pre-processing using the color space transformation method [7] in which RGB images are first converted to CIE L*a*b* color space and then after transformation, intensity values of channels are adjusted for contrast improvement. The problem of shading caused by the variation in illumination is solved in [8] and local illumination is estimated using morphological closing operation.

3 Lesion Segmentation Methods

Segmentation is an essential step for determining the region of skin lesions from dermoscopic and clinical images. Segmentation is challenging because enough ground truth labels are not available and there are a lot of variations in size, shape, texture and color of lesions. An effective segmentation method holds capacity to solve above problems and boost the accuracy of the classification task [9]. Thresholding, Edge-based segmentation, Region-based segmentation, clustering, region merging, and splitting are basic algorithms developed by researchers in past decades. These are the simple methods applicable only to small datasets and hence cannot be applied to complicated tasks. Hu et al. [10] combined brightness and color saliency map for increasing the image contrast and then adaptive thresholding based on wavelet transform (WT) is applied for segmenting the skin lesion from healthy skin. Markov Random Field (MRF) theory is also showing great potential in solving image segmentation problems [11].

Recent efforts focus on convolutional neural networks (CNN) based methods as a superior tool in the field of image analysis. After the availability of high-performance and less expensive GPUs, CNN models have become quite popular [12] for segmentation and classification tasks. In [13], authors have combined YOLO and grab cut algorithm for detection of lesion location and then post-processing is applied for getting the fine segmented image. An improved fully convolutional neural network (FCN) [14] is proposed for the segmentation of full-resolution images of all skin types. The 19-layer deep fully convolutional network is designed in [15] and fusion of 50-layer U net and ResNet architecture called Res-Unet50 is proposed in [16] for segmentation. These methods reduce training time and reveal good results on the ISBI 2017 and PH2 dataset. Integration of multiple fully convolutional networks is proposed by authors for complicated and challenging skin lesions [17]. The performance of segmentation methods is determined by the evaluation metrics such as Accuracy (ACC), Dice coefficient (DC) and Jaccard Index (JA). Summary of performance of segmentation methods is given in Table 1.

Table 1 Summary of performance of various segmentation methods for skin lesions
Full size table

4 Lesion Feature Extraction Methods

Features from the segmented area is of prime importance in the diagnosis of status of lesions. Extraction of the effective features and removal of redundant features is the main aim of all extraction methods. Shape, color, and textures are the three main skin lesion features [18]. The various shape features are asymmetry and border. Asymmetry can be examined by using several geometrical measures like perimeter, diameter, entropy measures, etc. [19]. Various techniques like wavelet transform [19] and Fourier transform [20] are applied for the determination of border irregularity. Color information of the lesions can be obtained with the help of color spaces, for example, RGB, normalized RGB [21], HSV, and CIELAB [22] color space. Texture analysis can also be done by statistical means like the Gray level co-occurrence matrix (GLCM) and local binary operator (LBP) [23]. Filter based texture analysis methods are wavelet transform, Gaussian filtering and Gabor filtering [24].

5 Lesion Classification Methods

The performance of the classification mainly depends on the extracted features and classification algorithms. A large number of features, imbalance between normal and abnormal samples, variations in feature range are some of the concerns that need to be handled before the start of classification [18]. Classification methods based on machine learning requires hand-crafted features like shape, colors and texture whereas deep learning methods does automatic feature extraction. Shimizu et al. proposed layer and flat classification models which combines color, subregion and texture features for skin lesion classification [25]. Fully connected (FCN) U-net CNN architecture is implemented in [26] which works on sparse coded features for diagnosis of melanoma. The classification performance of four pretrained CNN architectures (DenseNet201, ResNet152, Inceptionv3, Inception ResNet v2) is 11% more accurate than highly skilled professional dermatologists [27]. Oliveira et al. [28] proposed an Optimum Path Forest Classifier (OPF) which gives 92.3% accuracy with hybrid feature extraction techniques. Ensemble of k-nearest neighbor (KNN), support vector machine (SVM) and Deep CNN classifiers is presented in [29] and hybrid (color-texture) feature extraction provides better results with SVM classifier [30]. Clinical profile of patients like age, gender, and lesion visual features are also incorporated in the CNN algorithm [31] for accurate and precise results. The performance of all classification methods is determined by Accuracy (ACC), Sensitivity (SE), Specificity (SP), Precision (Prec) and Area under the region of characteristic curve (AUC). Summary of classifiers is given in Table 2.

Table 2 Classifiers for detection of skin lesions
Full size table

6 Data Sets for Melanoma Diagnosis

Several public and private datasets have been released keeping in view the rising cases of melanoma cancer worldwide. Among the private datasets, the largest one used by Esteva et al. [32] consists of 129,450 clinical images. Other popular databases are Dermofit Image library [33] and Interactive Dermatology Atlas [34]. The major public datasets available are ISIC archive [35], ISBI 2016 [36], ISBI 2017 [37] challenge dataset and PH2 [38] dataset. All these benchmark datasets are labeled by expert dermatologists and shows a fair number of imbalanced classes, difference in size and complexity.

7 Challenges in Melanoma Detection

Melanoma Diagnosis is initially done by dermatologists through the visual screening of skin lesions followed by a dermoscopic analysis [39]. Accurate detection of melanoma lesions of size less than 6 mm is quite difficult [40] and sometimes suspicious lesions required biopsy. Inadequate and imbalanced images in the available benchmark data set is another challenge in deep learning methods as they may lead to overfitting and underfitting problems. Some of the researchers work on non-public data sets therefore replication and validation of their results is a difficult task [41]. Another limitation is that all the available data set in the ISIC repository mainly consists of images of light skinned people and there is lack of benchmark dataset with images of dark-skinned people.

8 Conclusion

This paper presented different techniques of image pre-processing, lesion segmentation, lesion feature extraction and classification required for automatic diagnosis of skin cancer. Pre-processing methods like removal of hair, air bubbles, artifacts help in the extraction of appropriate features. It is observed that deep learning-based segmentation and classification bypass complex pre-processing operations. Pretrained deep learning model provides better results with small datasets. Incorporating ensemble classifiers compensate for the shortcoming of individual classifier. Patient metadata information like age, gender and lesion visual features must be integrated along with images for better training of model. The main purpose of all the automated diagnosis methods is to assist the dermatologists in early diagnosis of skin cancer and decrease the melanoma mortality rate.