Introduction

In this review paper, the most current and innovative solutions using modern digital image processing methods applied for the purpose of skin cancer diagnostics were presented. The choice for this topic is the fact that skin cancer is one of the most common types of cancers [45, 83, 103, 120], in particular in the white population [83, 84], mostly in women [51]. It is also one of the most expensive cancers in treatment [63, 137]. It is said that in the USA only, one in five people will develop skin cancer by the age of 70 [45, 120, 137]; this trend is constantly increasing [56, 57, 137]. Also, early diagnosis and treatment are needed in order to decrease the number of deaths due to this disease.

This paper also contains some information regarding the most common skin cancer types, mortality and epidemiological data for Poland, Europe, Canada and the USA [45, 84, 137], therefore implementation of new, non-invasive methods plays a crucial role in both identification and prevention of skin cancer occurrence [85, 111, 140].

This work also covers the most efficient and modern image recognition methods based on the artificial intelligence applied currently for the skin cancer diagnostics purposes, covered in literature in particular after 2017 [22, 29, 30, 103, 140]. The authors decided to focus on the latest data, mostly due to the rapid technology development and increased number of new methods, which positively affects diagnosis and prognosis. Also, skin cancer, as a result of environmental factors, is becoming more and more frequent [41, 64, 66, 136].

Skin cancer types

It is possible to differentiate the following types of skin cancer [3, 6, 25, 39, 62]:

  • Skin cancer – basal cell carcinoma;

  • Skin cancer – squamous cell carcinoma;

  • Skin cancer – papillary cancer;

  • Merkel neuroendocrine carcinoma;

  • Melanoma;

  • Kaposi’s sarcoma;

  • T type cutaneous lymphoma (mycosis fungoides);

  • Paget’s disease (Paget’s skin cancer).

There is also a simplified, binary classification: as either melanoma or non-melanoma [8, 144]. Non-melanoma skin cancer (NMSC) is one of the most common malignancies among the white population and its occurrence increases annually. It includes basal cell carcinoma (BCC), squamous cell carcinoma (SCC), and many other rare cancers [32, 54, 93]. It is caused by exposure to the UV radiation (both melanoma and non-melanoma) and can be treated with surgery, physical destruction, chemical destruction, and immunomodulatory therapy [54, 93, 144]. Despite the fact of its occurrence increase, the mortality rates were already significantly dropping to \(1--2 \%\) cases in nineties [26, 93], and are still dropping (BCC to \(0.96 \%\) cases and SCC to \(1.25 \%\)) according to [14, 115].

As mentioned above—sunburn is one of the leading causes for skin cancer [55, 104, 123, 144].

Finding appropriate biomarkers as a reliable tool for both melanoma and non-melanoma skin cancer diagnosis plays a crucial role in the whole diagnostics process [39, 144]. It is because of the heterogeneity of skin cancer, which causes an unequal response to the therapy (including primary and acquired resistance to targeted therapies), which translates into the need to use advanced methods of motivating patients and searching for new, more effective and universal biomarkers. This requires an individual approach as part of personalized medicine. In skin cancer, especially in melanoma, biomarkers perform both diagnostic (early detection) and prognostic (in estimating a patient’s prognosis) functions. Promising biomarker is the droplet digital polymerase chain reaction (ddPCR) for the detection and quantification of low-abundance nucleic acids in biopsies [39]. Another promising (rather predictive) skin cancer biomarker is the presence of the V600E mutation in the BRAF gene in neoplastic cells. Promising resistance factors (genetic and non-genetic) also include changes in the PI3K/AKT, MAPK, and RB signaling pathways [108].

The non-melanoma skin cancer (NMSC) is the most common form of skin cancer [8, 27, 34, 64]. Most cases of NMSC are basal cell carcinoma (\(70 \%\)) and squamous cell carcinoma (\(25 \%\)) [27]. Women and people between the age of \(61-90\) are at the greater risk, topographically the NMSC is most commonly found in the head and neck area [27]. Genetic and molecular changes, immunosuppression and ultraviolet radiation lead to NMSC [36]. The immunocompromised patients have a much higher risk of developing NMSC, but non-viral NMSC is usually associated with DNA damage from exposure to UV light. Correctly located cases are effectively treated surgically, and in the case of metastases, drug therapies (pembrolizumab, avelumab, cemiplimab) are becoming more and more effective [12, 60]. For the above-mentioned reasons, early diagnosis and prompt therapeutic intervention are key. Today, drug and gene delivery is increasingly based on micro/nano-sized polymeric carriers and intelligent platforms [75]. Theranostic systems combining diagnostics with therapy are increasingly used.

The melanoma skin cancer (MSC) is known to be the most dangerous skin cancer and, therefore, its detection in the earliest possible stage plays a crucial role in the efficient treatment process [8, 34, 98]. It is because it is more likely to spread to other body parts [8, 34]. The incidence of melanoma has been increasing over the past 50 years due to increased exposure to the sun and UV radiation. Risk factors for melanoma are European ancestry and old age, but also gender, UV exposure and anatomical location [46]. The prevalence of individual melanoma sub-types varies across racial groups [17, 18]. Novel 3D digital skin models can provide a better understanding of the complexity of melanoma and associated risk factors [46]. The past 3 decades brought high incidence rates of the melanoma skin cancer [8, 34, 75], however, the medicine development increased also survival rates [8, 13, 79]. The MSC can appear on any skin surface and is less common in people with dark skin [8, 119, 128].

Skin cancer can be categorized into the three below listed types [103] (Fig. 1):

  1. 1.

    basal cell skin cancer,

  2. 2.

    squamous skin cancer,

  3. 3.

    malignant skin cancer.

Fig. 1
figure 1

Skin cancer classification

Full size image

The non-melanoma skin cancer (NMSC) are basal cancer cell (BCC) and squamous cancer cell (SCC) [97, 103].

The BCC is the least aggressive non-melanoma skin cancer, which appears in a form of a flesh-coloured pearl pump or a pink skin patch [43, 97]. It is caused with sun exposure, so it appears only on sun-exposed areas (e.g. head, neck, limbs), however, it can spread all over the body and grow also in nerves and bones [43, 103]. It is very common and affects \(2-3\) million of people each year, but fortunately, it has a very low death rate and does not require any complex treatments [103].

The SCC is also a non-melanoma skin cancer, which can be found on outer skin surface and appears in a for of a red firm bumps scaly patches [103, 117]. Similarly to the BCC it is also caused due to the sun exposure and can be found on similar body locations. It has a higher death rate than the BCC, but lower than melanoma [103, 117]. Basal and squamous cell carcinomas are the two most common types of skin cancer in the world [97, 103, 117].

Statistics of skin cancer

One of the most common types of cancer is skin cancer. Skin cancer begins with abnormal cell growth mainly due to exposure to sunlight containing UV radiation. The UV radiation wavelength ranges from 100 to 400 nm. It causes genetic changes in DNA and it damages skin cells. At a later stage, the cancer can spread to other parts of the body. In melanoma, cancer cells arise from moles on the skin. The generation of free radicals as a result of the biochemical interaction of ultraviolet with melanin also causes mutations and genetic aberrations [31, 125].

In 2015, there were about 17.5 million cancer cases worldwide and 8.7 million of people died as a result of cancer. Most studies show that rates are increasing significantly worldwide, generally thought to be a result of increased UV exposure [117]. Moreover, in 2017, only 3.3 million people with non-melanoma skin cancer were treated, out of 5.4 million cases. This is due to deficits in health care and diagnostics [84, 103].

Cutaneous melanoma is now less common than NMSC. However, fair-skinned populations have been rising for the last few decades and melanoma is the most rapidly increasing cancer in white populations. Incidence rates are expected to double every \(10-20\) years. In Europe, highest increases of rates were observed in Scandinavian and Western European countries [45, 84]. Moreover, about \(2-4 \%\)cases are suffering from melanoma skin cancer in e.g. India [103]. In Malaysia, for example, skin cancer was ranked as the 10th most common cancer and accounted for \(2.6 \%\) of all cancer cases in this country [103].

In 2018, between two and 3 million non-melanoma skin cancers and 132,000 melanoma skin cancers happen globally every year. The fairer skinned people are at much higher risk of getting melanoma cancer [103]. In 2020, non-melanoma skin cancer was the third incidence cancer affecting males in the world [45].

Currently, mortality due to melanoma is not increasing and has been stabilised. This can be seen in various countries in Europe. This is due to the fact that many intervention strategies have been introduced and early disease detection is used [83, 84].

Image processing-based diagnosis of skin cancer

Using image-based methods for diagnosing skin cancer is non-invasive and has been shown to provide positive results [2, 8]. Computer vision-based methods are increasingly used in various areas of medicine due to their non-invasive nature [8, 65]. These techniques also allow for automatic image analysis with accurate results [8, 95, 101, 129]. The first step is to collect images using dermoscopy [8, 103] which are then pre-processed using various image processing methods, such as Support Vector Machine (SVM) based on Principal Component Analysis (PCA) [8, 23, 127, 142].

The skin cancer diagnostic road-map and the principles and mechanisms of skin cancer detection are summarised in Figs. 2 and 3.

Fig. 2
figure 2

Road-map of skin cancer diagnosis

Full size image
Fig. 3
figure 3

Principles and mechanisms for skin cancer detection

Full size image

The basic methodology of the review carried out and its results are shown in Figs. 4 and 5. The significant increase in publications in the last 10 years is noteworthy: the number of articles published annually has doubled. In addition, 2957 articles were published between 2013 and 2022, i.e. \(63.43 \%\) of all articles published in this field. This demonstrates both the significant impact of modern technology and the rather rapid growth of new knowledge requiring mastery by its adepts.

Fig. 4
figure 4

Results of the literature review

Full size image
Fig. 5
figure 5

Review methodology scheme

Full size image

Image processing-based diagnosis of skin cancer includes optical, photodynamic based, sonography, electrical bio-impedance and thermal imaging techniques, which principles are described in the further part of this paper.

Optical technique

Optical methods use light that passes through the skin tissue. When the light is scattered in the skin tissues, a change in the properties of the reflected light is used for diagnosis. Because the skin is the outer covering of the body, light has easy access and optical techniques can be used to diagnose skin cancer [116]. They are often used for early detection, or to improve other diagnostic tools such as dermoscopy. The optical method can be used to detect the disease without a biopsy [7, 48]. They deliver both spatial and spectral data and other relevant information of skin tissue. Also, the optical methods provide a large amount of data that can be combined with machine learning algorithms to differentiate skin lesions and improve cancer diagnosis [114].

The most known optical diagnostic techniques are optical coherence tomography (OCT), fluorescence spectrometry, reflectance spectrometry, Raman spectroscopy, Multispectral (MS) imaging, 3D topography, Self-Mixing Interferometry (SMI), Polarised Imaging (PI), and confocal microscopy [20, 103, 116].

Photodynamic-based technique

Photodynamic diagnosis (PDD) is a minimally invasive and innovative technique to detect the presence of tumour cells, based on a photodynamic integrative method. A photochemical molecule known as a photosensitizer (PS), is administered to a patient. Then it is selectively absorbed intracellularly by tumour cells only, due to the enhanced permeability retention (EPR) effect. This photosensitive marker is visible under a resectoscope when blue light with a wavelength of 330–400 nm is applied. In such light, cancer cells are fluorescent. Skin lesions using photodynamic diagnosis are based on the fluorescent properties of an exogenous and endogenous compound in response to illumination. This method does not cause damage to cells or tissues, which allows for easy and early identification of a precancerous or cancerous lesion [33, 130].

Sonography technique

Sonography technique uses sound waves, where sound pulses are transmitted into the skin, then some of the sound waves are reflected back to the transmitter and some of them are reflected in other directions [28]. The reflected waves are sensed by the machine and the changes in the property of sound waves shows the structure of the skin. Sonography is mainly used to assess skin lesion depth and margins before doing the biopsy or to classify adjacent lymph nodes, when cancer is diagnosed [76]. It can also help to assess whether it is benign or malignant skin cancer. The main advantage is that it gives the accurate results of measuring skin cancer lesion thickness. Another advantage is the prevention of unnecessary removal of the lymph nodes when they are not affected by disease [103, 139].

Electrical bio-impedance technique

Electrical impedance-based technique is a well-accepted non-invasive technique using electrical impedance [9]. The electrodes are topically applied to the skin and the innate electrical impedance of cells after malignancy is different from that of healthy cells. The stratum corneum has a high resistance so it must be bypassed to measure the living epidermis and dermis. Identification of cancer using electrical impedance is straightforward, objective, fast, and economical [9, 113, 118].

Thermal imaging technique

Thermography or infrared thermal (IRT) imaging is a contactless sensing method that uses a thermal camera to record the infrared radiation of the human skin [9]. There is no ionising radiation, so this method is safe for patient health. It is based on the electromagnetic radiation emitted by the human body. The temperature distribution, based on this radiation, is emitted in the infrared band and is displayed in a thermogram. Then, all temperature abnormalities are detected. For example, the temperature between the cancer tissue and the healthy tissue is different. Thermography can be applied dynamically. Then a thermal stimulus, like heat stress, is applied before the test to increase the temperature differences between the lesion and the surrounding skin, or in a steady state. Compared to other techniques which have false negative reports at a high probability, thermography provides high accuracy [9, 90, 106].

Methods

Medical imaging is a well-developed field of science, therefore there exist numerous methods for collecting data for subsequent medical analysis.

Images

Dengel et al. [35] made a significant discovery regarding the use of photography for skin cancer screening. However, the method is not widely used due to its time and cost implications. This technique involves capturing surface images of the skin to primarily identify suspicious and pigmented lesions in high-risk patients. The captured images are then processed using image processing algorithms to detect any abnormalities in the skin. The segmentation technique used to detect skin lesions is divided into two categories: region-based segmentation and neural edge detection. In the region-based segmentation method, an isodata algorithm is iteratively employed to determine the optimal threshold. In neural network edge detection, an approximate closed elastic curve is fitted between the recognized neural network edge patterns.

Computer-aided decision tools play a crucial role in medical imaging for diagnosing and assessing various diseases. In [67] The high-resolution camera captures an image of the subject, which is pre-processed to eliminate any artefacts. To eliminate artefacts like hair in the image, mean, median, Gaussian, and anisotropic filters are utilised. The next step is lesion detection [40], which uses image segmentation techniques to partition the image into disjoint areas that are homogeneous with respect to a chosen property, such as luminance, colour, and texture. Feature extraction follows this step. The ABCD rule is used in the extraction process, which assesses the symmetry, border, colour, and diameter of the acquired image. Once the lesion is localised, different chromatic and morphological features can be quantified and used for classification. In the classification process, the algorithm in the examination process integrates visual processing with deep learning. In deep learning, the computer is trained to solve a problem rather than having the answers programmed into it.

In recent years, the field of computer-aided diagnosis has expanded to include the detection of skin cancer. Detecting melanoma skin cancer in its early stages is crucial for ensuring proper treatment and saving lives. Abdul Jaleel, Sibi Salim, R. B. Aswin, and others have found that computer-aided detection of skin cancer is based on imaging techniques and artificial intelligence [68, 69]. Computer vision plays a critical role in medical image diagnosis. One paper discusses the use of computer-aided diagnosis for analysing skin lesions and detecting the presence of skin cancer by performing boundary detection and assessing the degree of symmetry. Another paper describes the use of wavelet and texture analysis to diagnose melanoma.

Texture features were derived from wavelet decomposition, and border features were collected from lesion borders using the gain-ratio method. This method was computationally efficient for melanoma diagnosis. Recently, computer-aided diagnosis has expanded to mobile technologies and cloud platforms, enabling the system to classify lesions by identifying moles in skin images and classifying them as melanoma, benign, or nevus lesions [49]. Computer diagnosis can also be used as a preventive tool for skin cancer detection through mobile phone applications. This technique can even separate melanocytes from histopathological images [126]. In particular, these are the various colour models commonly used in computer graphics. The tools described earlier were based on an arbitrary division of the scene, which is an image of its luminance. Typically, such images were saved using a colour model storing only shades of gray. On the other hand, the use of colour models that, apart from luminance (brightness), also carry information about colour in medical imaging is quite rare.

In this field, one of the most important attempts to standardise the recording of information about colour was the initiative described in more detail in [11] which was initiated by the Food and Drug Administration (FDA) and the International Color Consortium (ICC). The result of this initiative was the establishment of the ICC Medical Imaging Working Group. During the first work of the group in 2013, it was noticed that there is a lot to do in terms of colour reproduction in medical imaging. The most important issue was to establish standards for recording colour information in medical imaging. These were the days when the methods of colour reproduction in computer graphics were just beginning to be standardised. The greatest obstacle to the widespread introduction of colour-based medical imaging was the highly imperfect devices for acquisition and reproduction of colour images. Most colour computer monitors were unable to correctly display the RGB colour space with the sRGB profile, which was developed by Microsoft and Hewlett-Packard as early as 1996 [53]. It was the first attempt to standardise the display of colour images on colour computer monitors from various manufacturers. However, it was a spectrally narrow profile of the RGB colour model, as can be seen in Fig. 6.

Fig. 6
figure 6

The spectral width of the sRGB profile relative to the standard observer

Full size image

Dermoscopy

Dermoscopy plays a crucial role in enabling dermatologists to observe epidermal structures, pigment and vascular patterns in order to facilitate lesion examination and clinical decision-making. Research has demonstrated that providing dermatologists with access to dermoscopy images in addition to conventional telemedicine photographs significantly enhances their diagnostic confidence [50]. Moreover, the inclusion of dermoscopic images has been shown to improve effectiveness and cost-efficiency when utilized in skin cancer screening. Nevertheless, proper dermoscopy training is essential for its appropriate and consistent utilization. Diagnostic accuracy is enhanced through user expertise and training, whereas a lack of training can present significant obstacles for providers [153].

The use of handheld devices in dermoscopy enables visualisation of subsurface skin structures and reduces surface reflection, whereas naked eye examination is limited due to the reflective properties of the stratum corneum. Dermoscopy is highly effective in diagnosing skin cancer and outperforms clinical analysis. A report written by [47] demonstrated that dermoscopy techniques have higher sensitivity and specificity than clinical analysis. Dermoscopy is also useful in distinguishing melanoma cells from benign cells using various diagnostic tools, such as pattern analysis, the ABCD rule, Menzies method, and the 7-point checklist. Pattern analysis was found to be more accurate in analysing 20 pigmented skin lesions. Dermoscopy is also used to analyse the vascular structure of melanocytic and non-melanocytic tumours based on their morphological behaviour. In vivo dermoscopy techniques are useful in early diagnosis of malignant melanoma and differential diagnosis of pigmented skin lesions, but it requires a high-resolution camera for capturing images [96]. Dermoscopy training and expertise are necessary for accurate and consistent use.

Ultrasound

Ultrasound can assess the morphology, orientation, internal structure and margins of lesions from multiple planes with high resolution both in predominantly fatty breasts and in dense glandular structures [135]. The application of ultrasound as a noninvasive imaging modality for breast cancer detection was already investigated in the seventies and eighties, and led to the development of ray-based CT. Different categories for various elastographic techniques. Many different elastography techniques are available to measure and display elastography qualitatively or quantitatively, using the displayed modus and different forces. Commonly used techniques are strain elastography (SE), acoustic radiation force impulse (ARFI) imaging, transient elastography (TE), point shear wave elastography (pSWE) and shear wave elastography (SWE). According to the property displays, there are three types: strain or strain rate, displacement and shear wave speed. Strain elastography calculates and displays tissue strain. ARFI imaging detects and displays tissue displacement. TE and pSWE record the shear wave propagation speed(without making an image). Strain elastography uses a hand-held probe with a slightly longitudinal pressing method or respiratory movement and obtains the hardness response information by estimating the deformation along the longitudinal axis and the strain distribution of the internal tissue [156]. Strain elastography technology can be used to qualitatively and semiquantitative study the elastic strain rate ratio of a lesion compared with that of the surrounding normal tissue. Compression technology is easy to implement, although it suffers from higher operator dependence and poor reproducibility. Real-time elastography (RTE), which generates strain imaging by compression, assesses the relative elasticity of the tissue in a specific area of interest, creating an elastogram, that is, a colour-coded map, that is superimposed on the ultrasound image. The relative elasticity may vary according to the tissue studied, the size of the RTE box and the pressure exerted. As tissue is mechanically non-linear, the strain from a given force decreases with increasing force, and the tissue becomes harder as more force is applied. The resolution of strain image changes with different contrast discrimination of the strain and with window size or displacement, strain estimators and the smoothing window, palpation speed and amplitude, persistence, and so forth. There are some artifacts that may influence strain images, such as friction between the transducer and skin, which could decrease the strain of surface tissue; a narrow compressor, which generates limited strain with poor homogeneity and decays rapidly with depth; the artifact of strain concentration, which might be seen when there is a hard inclusion in a soft background and which can explain the high strain at slip boundaries and edge enhancement; and the egg shell, which might occur when soft regions are buried in a stiff background, as stiff tissue prevents the generation of strain inside the egg. The features that help generate good strain images include closeness to the target and some distance to tissue boundaries, the anatomic plane and other structures [92, 150].

Confocal microscopy

Confocal microscopy is a technique that allows for the non-invasive examination of skin at a cellular level using a focused laser beam. This beam illuminates a specific point on the skin, and the reflected light is measured to construct a grayscale image [19, 21, 74]. The microscope contains a light source, condenser, objective lens, and detector, which use point illumination and a pinhole to avoid out-of-focus signals, giving it name confocal. The light source illuminates a small 3-D spot within the sample, and the reflected light is used to produce a pixel. The microscope then scans the illuminated spot horizontally across a 2-D grid to obtain a horizontal microscopic section, known as optical sectioning. This allows for the creation of an image pixel by pixel, with an axial thickness of 2–\(5 \,\upmu \hbox {m}\), enabling the visualisation of a slice in the body of a thick, semi-transparent sample. This is in contrast to conventional microscopes, which visualise all planes simultaneously. Confocal microscopy provides the capacity for direct, non-invasive, serial optical screening for thick, living specimens with least of sample preparation as well as a minimal improvement in lateral resolution, commonly called reflectance confocal microscopy (RCM). In vivo RCM is a non-invasive technique that allows examination of the skin with cellular resolution. Resolution is almost comparable to conventional histology. It has the advantage of allowing the clinician to do a virtual biopsy of the skin and obtain diagnostic clues while minimising unnecessary skin biopsies.

It is clear that valuable information cannot be obtained from 2D cell cultures. In conventional uptake studies, drugs and nanomedicines are applied to cancer cells grown in flat monolayers, and the only obstacle they face in penetrating the intracellular compartment is the cell membrane [82]. On the other hand, 3D cell culture models are a better representation of the biological reality as they simulate the tumour and its microenvironment in vitro. Among these models, multicellular tumour spheroids (MCTS) have gained attention for their ability to imitate key characteristics of non-vascularized tumours, including the close proximity between cells and their self-organisation in layers with varying rates of proliferation. Additionally, cells in spheroids secrete extracellular matrix (ECM) proteins, creating a microenvironment that acts as a biological barrier and can interfere with transport phenomena, blocking the diffusion of drugs and nanoparticles.

Spectroscopy

Depending on the source of equipment, spectroscopy can be checked on: confocal microscopy, Raman spectroscopy and fluorescence spectroscopy [9]. Confocal microscopy is a method that enables the examination of skin with a resolution at the cellular level, without causing any harm. It employs a focused laser beam to illuminate a precise point within the skin and records the light reflected from that point [42, 87, 122]. The technique involves scanning multiple points across an area parallel to the skin surface to generate a grayscale image. The microscope comprises a light source, a condenser, an objective lens, and a detector. By utilising point illumination and a pinhole in an optically conjugate plane before the detector, the confocal microscope can filter out-of-focus signals and collect light from only the single in-focus plane. A small 3-D spot within a sample, such as skin, is illuminated by the light source, and the reflected light is collected to produce a pixel. The spot is then moved horizontally over a 2-D grid to obtain a horizontal microscopic section. This technique is known as optical sectioning, and it produces a series of horizontal planes stacked vertically, forming an image pixel by pixel, with an axial thickness of \(2-5 \upmu \)m. This unique feature enables the confocal microscope to examine a slice in the body of a thick, semi-transparent sample, whereas conventional microscopes visualise all the planes simultaneously [107, 112].

Confocal microscopy is a powerful imaging technique that enables direct, non-invasive examination of thick, living specimens with minimal sample preparation, and offers high resolution imaging, known as reflectance confocal microscopy (RCM). [10, 109] In vivo RCM allows for cellular-level examination of skin lesions, almost comparable to conventional histology, with the added benefit of enabling a virtual biopsy of the skin to obtain diagnostic clues while minimising unnecessary skin biopsies.

Confocal microscopy can detect various skin disorders, including hyper-pigmentary and hypo-pigmentary lesions, and can be combined with Raman spectroscopy at different wavelengths to enhance cellular detail. Recent studies have utilised an extended version of the vivascope, offering better imaging capabilities with a laser source ranging from 488 to 700 nm to non-invasively illuminate tissues and identify their depth and properties. Combining multispectral polarised light imaging (MSPLI) with confocal microscopy can provide even greater accuracy and detail in results compared to either system alone.

Raman spectroscopy is a powerful method that detects various modes in a system, including rotational, vibrational, and other low-frequency modes. The technique utilises the phenomenon of Raman scattering, which involves inelastic collisions between photons of a laser beam and molecules in the sample or tissue being studied. Raman spectroscopy is performed using monochromatic radiations typically from a laser in the visible, near infrared, or near UV range. The obtained spectra can be processed and analysed to provide real-time feedback during measurement. This technique offers high sensitivity and accuracy in differentiating tissues [5, 70, 89].

The classification model used for Raman spectroscopy is probabilistic and automated, relying on feature extraction and a fully adaptive robust feed-forward neural classifier. In vivo Raman microspectroscopy is a non-invasive and real-time diagnostic tool for non-melanoma skin cancer, such as basal cell carcinoma (BCC) and squamous cell carcinoma (SCC). Sample calibration can be achieved in less than one second. Skin lesions can be distinguished using distinctive bands corresponding to specific Raman spectra of lipids and proteins. Partial least regression and discriminant analysis can be used to analyse Raman spectra of various compounds [77, 81]. Raman spectroscopy is useful for studying the static and dynamic properties of biologically significant molecules in living cells, cell cultures, and solutions. Linear least square fitting models can estimate the contribution of various bio-compounds, such as lipids and proteins, in tissue regions affected by skin cancer. By shifting the excitation energy of Raman spectroscopy to the near-infrared region, fluorescence components present within normal cells can be minimised, making it a valuable tool in medical diagnosis.

Fluorescence spectroscopy is a type of electromagnetic spectroscopy that is also referred to as spectrofluorometry or fluorometry. It involves the analysis of fluorescence emitted from a sample after excitation by a light source, which causes the electrons in the molecules to become excited. This technique has been widely used in various fields, including biology, biochemistry, and environmental science. In the medical field, in vivo laser-induced fluorescence spectroscopy has been utilised to detect skin cancer [4, 78].

Multispectral imaging technique

In multi-modal spectroscopy (MMS) or multispectral imaging, various linear polarizers of different wavelengths are used [59, 146]. The spectral and spatial information of the samples are simultaneously recorded by the multispectral image spectrometer, in which the acquired images from a monochrome camera is processed using spectral and polarisation filtering that provides high contrasting images which is useful in identifying the pathological and morphological features of the suspicious skin lesions. The report was made by Hagen Nathan, Kudenov Michael on multispectral imaging based on spectral bands. In this, the image analysis is done automatically and pattern recognition is used to identify lesions which in turns help in further biopsy. It plays an important role in the diagnosis of skin cancer by considering the parameters such as texture, asymmetry, border irregularities etc. [58]. The images are obtained from the affected regions of the skin using the charge coupled camera along with eight narrow band filters ranging from 450 to 800 nm at the interval of 50 nm. The features are extracted from the image using the principal components analysis. The characterizations of malignant and benign tumours are separated by spatial grey level co-occurrence matrix.

The multispectral imaging technique involves in vivo methods in which images are examined at equal wavelength intervals between 483 nm and 950 nm. The technique utilises a neural network classifier for automated skin cancer diagnosis, achieving a sensitivity of \(80.4 \%\) in distinguishing between malignant and benign tumours. To separate pigmented skin lesions, an automatic segmentation algorithm is employed. Typically, images of skin lesions are analysed in both 2D and 3D within the visible to infrared spectrum. The technique allows for the analysis of skin lesions, subcellular pigmentations, and vascular depth, utilising radiometric measurements for tumour analysis. A multispectral camera captures images of skin lesions, which are then analysed in spectral ranges between 450 and 950 nm, allowing for the discrimination of melanoma from nevus cells. Non-contact skin chromophore analysis is performed using self-developed software. The technique can assist in the detection of melanoma skin cancer by considering the melanin index and erythema index [110]. Multispectral imaging also helps in determining whether a biopsy is required from the pigmented lesion using the multispectral digital skin lesion analysis (MSDSLA) device. Analysis of vascular depth in the skin lesion enhances diagnosis and can be interpreted using 6-layered skin models. A combination of multispectral imaging and a 3D imaging sensor has been reported to detect skin cancer [102]. software. The technique can assist in the detection of melanoma skin cancer by considering the melanin index and erythema index [110]. Multispectral imaging also helps in determining whether a biopsy is required from the pigmented lesion using the multispectral digital skin lesion analysis (MSDSLA) device. Analysis of vascular depth in the skin lesion enhances diagnosis and can be interpreted using 6-layered skin models. A combination of multispectral imaging and a 3D imaging sensor has been reported to detect skin cancer [102].

Multi-photon scanning

Multi-photon scanning, also known as two-photon excitation microscopy or non-linear laser scanning microscopy, is a method of three-dimensional imaging that offers several advantages over confocal and deconvolution microscopy. This technique relies on the nonlinear interactions between photons and matter, specifically the interaction of two photons with the same molecule at the same time, which leads to two-photon excitation and subsequent fluorescence [80, 86, 100]. Unlike traditional single photon excitation, two-photon excitation occurs only within the focal spot of the microscope, where photon density is high enough to cause two-photon absorption. Multiphoton scanning is particularly useful for imaging living cells within intact tissues, such as brain slices, embryos, whole organs, and even entire animals. This technique allows for dynamic imaging of living cells in thick specimens, which is not possible with conventional imaging methods. In this way, multiphoton scanning enables many experiments that would otherwise be impossible with other imaging techniques.

Autofluorescence (AF) imaging, which detects endogenous fluorescence, has potential applications as an optical biopsy tool for distinguishing between healthy and diseased tissue based on their inherent fluorescent properties [141]. Unlike exogenous contrast agents, AF imaging is minimally invasive and eliminates the risk of toxicity or reactions to dyes. However, the relatively low signal-to-noise ratio produced by AF imaging can lead to a high false-positive rate, which can be addressed through careful characterization of AF properties. Multiphoton microscopy (MPM) is a promising tool for high-resolution and contrast quantification of tissue AF. Two-photon excited fluorescence (2PEF) is a technique that generates image contrast in MPM by using an endogenous fluorophore that absorbs two photons to create fluorescence emission. This technique is effective in imaging deeper into tissue samples due to the inverse relationship between wavelength and light scattering. Second-harmonic generation (SHG) is another popular method of creating MPM contrast, predominantly for collagen in tissue samples. The use of excitation light of longer wavelengths in MPM reduces the chance of tissue damage and photobleaching during image acquisition while maintaining high image quality [157]. MPM is a powerful imaging modality for accurately assessing the molecular features of tissue specimens, and developing models of DGAST multiphoton fluorescence is a step towards in vivo label-free measurements of these lesions for screening, diagnosis, and staging.

Thermography

Infrared radiation is emitted by every object which possesses the temperature absolute zero point. The thermal imager determines the temperature of the object’s surface based on the intensity of infrared radiation making it visible to the human eye with the thermal image. This process is referred to as thermography. Thermal imager translates the wavelength from the infrared to the wavelength which is visible to the human eye. This is the principle which is used to detect skin cancer using thermography. Thermography has been employed in medicine for various applications. However, thermography overcomes all the shortcomings that other methods had [91, 148].

The medical infrared thermography is utilised in cancer detection because of its advantages such as radiation-free, non-invasive and painless nature. When there is an unexpected increase in temperature, it is an indication of a problem [16, 44, 133, 147]. For example, increased friction causes wear and generates heat, potentially leading to material failure. Similarly, human heat is linked to various conditions such as inflammation and infection, and physicians have used thermo biological diagnostics since the time of Hippocrates. As a living organism, the human body strives to maintain homeostasis, which results in dynamic changes in heat emission. The combination of central and local regulatory systems is reflected in the surface temperature of an extremity. Biomedical infrared thermography detects the emitted radiation on the human body surface and shows the heterogeneous skin and superficial tissue temperature. Infrared emissions from human skin at \(27\,^\circ \hbox {C}\) are in the wavelength range of 2–\(20\, \upmu \hbox {m}\), and peaks at \(10 \,\upmu \hbox {m}\). Body infrared rays, a narrow wavelength range of 8–\(12 \,\upmu \hbox {m}\), are used for medical applications. The use of infrared thermography for skin cancer is optimal because of its noninvasiveness and ability to detect temperature changes and distribution. Melanoma skin lesions are detected through infrared imaging by identifying new blood vessels and chemical changes associated with tumour growth. For other skin tumour types, such as basal cell carcinoma, an encapsulating layer of involved cells acts as a thermal insulator, causing a delayed thermoregulatory process.

Artificial intelligence techniques for cancer detection

In [15, 61, 94, 99, 105, 121, 124, 143] authors describe the use of AI techniques approaches that are utilised to produce and develop computer software programs. AI is an application that can recreate human perception. This application normally requires obtaining input to endow AI with analysis or dilemma solving, as well as the ability to categorise and identify objects. This paper describes various AI techniques, such as support vector machine (SVM) neural network, fuzzy models, artificial neural network (ANN), and K-nearest neighbour (K-NN) used in cancer detection.

The desire to improve the efficacy and efficiency of clinical care continues to drive multiple innovations into practice, including AI. With the ever increasing demand for health care services and the large volumes of data generated daily from parallel streams, the optimization and streamlining of clinical workflows have become increasingly critical. AI excels at recognizing complex patterns in images and thus offers the opportunity to transform image interpretation from a purely qualitative and subjective task to one that is quantifiable and effortlessly reproducible. In addition, AI may quantify information from images that is not detectable by humans and thereby complement clinical decision making. AI also can enable the aggregation of multiple data streams into powerful integrated diagnostic systems spanning radiographic images, genomics, pathology, electronic health records, and social networks.

During skin cancer detection, a dermoscope examination is performed as a standard procedure. Dermoscopic images are the basis for the diagnosis of skin lesions that may be cancerous. Diagnostics may have been aided by Genetic Programming (GP) based feature selection from dermoscopic images [132, 140, 145]. Extracted features can be high-level domain specific features recommended by the dermatologists and low-level Local Binary Pattern (LBP) features. GP helps in selecting the most significant features from the raw data. Then, it can be determined whether the lesion is cancerous or whether it is malignant or not.

When detecting melanoma at an early stage of the disease, a dermoscopy image analysis system can be used [1]. The system analysing the shape, colour, and texture of the skin lesion can help detect cancerous changes. Detection of them at the initial stage of the disease improves the patient’s prognosis. The proposed system achieved classification of the benign, atypical, and melanoma images with accuracy of \(96.3\%\), \(95.7\%\), and \(97.5\%\), respectively [1].

For dermoscopy images classification, neural networks are also used [140, 152]. In [152], a self-generating neural network (SGNN) is used for classifying melanocytic tumours as benign or malignant. Classification is based on features descriptive of tumour colour, texture and border. Then a neural network ensemble model combining back propagation (BP) neural networks with fuzzy neural networks is used. Classification can be done for whole and incomplete lesions when the dermatoscope is unable to prepare an image of the entire lesion [152].

To construct an AI-based diagnostic system, we used a deep neural network architecture called the Single Shot MultiBox Detector (SSD) [88], without altering its algorithm. An SSD is a deep CNN that consists of 16 layers or more. The Caffe deep learning framework, which is one of the most popular and widely used frameworks originally developed at the Berkeley Vision and Learning Center, was then used to train, validate, and test the CNN.

All CNN layers were fine-tuned using stochastic gradient descent with a global learning rate of 0.0001. Each image was resized to \(300 \times 300\) pixels, and the bounding box was also resized accordingly to make CNN analyse optimally. These values were set up by trial and error to ensure all data were compatible with SSD [88].

After constructing the CNN using the training image set, we evaluated the performance through the test image set. When the CNN detected a lesion of gastric cancer from the input data of test images, the CNN outputted a disease name (early or advanced gastric cancer) and its position. A detected lesion was displayed with a yellow rectangular frame on the endoscopic images.

Jones et al. proposed a methodological checklist for the development of AI/ML systems for skin cancer detection. This will facilitate the design, evaluation and implementation of such systems in the future. This is because their review showed a variety of studies and systems implemented to analyze AI/ML-based skin cancer data. In particular, attention is drawn to the large number of incomplete reports in the field of data collection methods, patient demographic data or health economics, as well as small samples of patients. Nevertheless, the AI/ML systems for detecting skin cancer have already achieved good diagnostic accuracy: for melanoma mean \(89 \%\) (range \(59-100\%\)), squamous cell carcinoma: mean \(85 \%\) (range \(71-97 \%\)), and basal cell carcinoma mean \(87 \%\) (range \(70-99 \%\)) [71, 72]. Cochrane meta-analysis showed the advantage of CAD in selected patient populations, however in the case of poor databases the sensitivity of CAD systems is not so high anymore. Pooled data from 22 studies showed the sensitivity of Derm-CAD in detecting melanoma to be \(90.1 \%\) and specificity to be \(74.3 \%\). These results for the 8 multispectral CAD imaging studies (MSI-CAD) were \(92.9 \%\) and \(43.6 \%\), respectively [121, 131]. Among the four classification algorithms (Naive Bayes, Bayes Net, LMT tree and MLP), the highest accuracy in recognizing the image of one of the five types of cancer (actinic-keratosis, benign, solar-lentigo, malignant, and nevus) was achieved by MLP: \(97.13 \%\) [155]. A study by Gouda et al. using the ISIC2018 dataset Inception V3 algorithm achieved the highest accuracy (\(85.8 \%\)) compared to the Inception Resnet (\(84 \%\)), Resnet50 (\(83.7 \%\)) and CNN (\(83.2 \%\)) models [52].

Key identified barriers include the lack of large comparative studies showing evidence of the effectiveness of AL/ML solutions as second opinion systems, including based on data from primary care, and the cost-effectiveness of implementation across countries and healthcare systems [52, 72, 151, 155]. In the case of auxiliary use of AI, there is currently no evidence confirming the effectiveness of such support in dermatoscopic diagnostics [38]. Apart from diagnostics, artificial intelligence systems can also help clinicians make better clinical decisions about skin cancer and perform predictive functions, which will translate into greater effectiveness of therapy [71]. But direct comparison between AI-based second opinion systems in skin cancer diagnosis and prediction is unavailable due to the use of different assessment metrics and image types, the size of datasets or diagnostic classes [138].

On the other hand, the tendency of patients to accept diagnostics and AI-assisted therapy is also important. Patients prefer human doctors and experience more negative emotions because of the use of AI in their therapy [158]. In addition, despite the efforts of engineers and interdisciplinary teams, AI systems can be perceived as a threat to medical students and medical specialists [73]. Nonetheless, according to [30], in 2018, the Food and Drug Administration of the United States (FDA) accepted [22] the applications of artificial intelligence for clinical uses, including devices and software designed to help diagnosing skin cancer.

Optical coherence tomography has shown promising results in the assessment of deep margins of skin tumours and inflammatory skin diseases, but differentiating premalignant from malignant lesions proved to be less effective. Fluorescence spectroscopy proved to be effective in revealing the biochemical composition of tissue; early detection of malignant melanoma was reliable only with stepwise two-photon excitation of melanin, while tumoral margin assessment and differential diagnosis between malignant and non-malignant lesions showed some conflicting results. Characterization of the structural properties of tissue can be made using diffuse reflectance spectrometry, and the values of the specificity and sensitivity of this method are ranging between \(72-92 \%\) and \(64-92 \%\), respectively. Raman spectroscopy proved to have better results both in carcinoma and melanoma diagnosis with sensitivities and specificities above \(90 \%\) and high above \(50 \%\), respectively. Confocal microscopy is the closest technique to pathological examination and has gained the most clinical acceptance, despite the need for a standardisation of the interpretation algorithm.

The specifics of the rater study have been described in detail by Sinz et al. [134]. In a web-based study of 95 human raters (\(51.6 \%\), female; mean age, 43.4 years; \(95 \%\), CI, \(41.0-45.7\) years), participants were divided into 3 groups (according to years of experience with dermoscopy): beginner raters (\(<3\) years), intermediate raters (\(3-10\) years), or expert raters (\(>10\) years). All participants rated 50 cases drawn randomly from the entire test set of 2072 nonpigmented lesions. The random sample was stratified according to the diagnostic category to prevent overrepresentation of common diagnoses. The raters were asked to differentiate between benign and malignant lesions, to make a specific diagnosis, and to suggest therapeutic management. The clinical close-up image was always shown before the dermatoscopic image, and the final evaluation was based on the combination of both imaging modalities.

Summary

The Table 1 summarises the AI methods used to detect skin cancer in publications from recent years and presents their degree of effectiveness. Based on that, the best results from the given ones were provided by the MLP method [155].

Table 1 Summary of the AI-supported diagnostic methods described in this paper
Full size table

Further directions in skin cancer diagnosis

The direction of further research is primarily prospective comparative studies of CAD systems as a diagnostic aid in comparison with, for example, dermoscopy. This is especially true of all studies in real participant populations where the test would be used in practice, e.g. in primary care [121]. There is a need to place special emphasis on the formation and cooperation of interdisciplinary teams (including with the participation of engineers and medical specialists) in the area of AI applications in the diagnosis, therapy and care of skin cancer [73]. By analysing various AI methods (including those based on Deep Learning DL) to support diagnostics, monitoring treatment progress and predicting changes in skin cancer patients, we can see that there is an interest in a structured, light / mobile and multimodal approach. The popularity of DL is growing, but the number of problems to be solved and opportunities for future use is still high [151]. The introduction of advanced immunotherapy, targeted therapies, combination therapies and small molecule vaccines could trigger another revolution in skin cancer therapy that will be reflected in both diagnosis and AI-based data analysis [149].

Conclusions

The current difficult times of pandemic and global crisis are an increasing challenge for healthcare systems, including in the field of diagnosis and treatment of patients with skin cancer. Skin cancers are common all over the world, and the type of cancer and the early stage of the disease at diagnosis are key to good prognosis and the burden of the disease in the patient. Skin cancer causes physical and psychological impacts related to diagnosis and treatment. Despite the increasing use of second opinions and multidisciplinary consultations, traditional skin cancer diagnosis, including histopathology, is limited by the subjectivity of specialists. The genetic susceptibility and progression of cancer is causing new features and multi-criteria analysis to gain importance. A future solution is the use of automated systems based on artificial intelligence to support the daily practice of pathologists and oncologists. They allow information to be shared and circulated more quickly throughout the healthcare system, thereby responding to changes without delay. With the aforementioned reasons, modern AI/ML-based image processing methods that enable the early diagnosis (including screening, as part of regular periodic examinations) of skin cancer, are key to further improving the effectiveness of the therapy. Solutions based on AI/ML are already effective and accurate, but they are still at the threshold of their development and require verification on large groups of patients and further dissemination in clinical practice. All skin cancer stakeholders are involved in analysing and confirming existing knowledge, new data and informing future research [24, 37, 154].