Introduction

At times, landslides may cause more damages and losses of lives than the direct effects of the triggering hazard such as earthquakes (Schuster 1996). However, often the damages caused by landslides triggered by earthquakes (Zhou et al. 2016; Fan et al. 2018; Tsou et al. 2018; Zhao et al. 2019; Goorabi et al. 2020; Barth et al. 2020) are directly reported as earthquake damages; and consequently the records are distorted and the necessary attention is not paid. To explain this issue in numbers, 5–11% of all deaths during earthquakes were caused by landslides triggered by earthquakes (Marano et al. 2010; Daniell et al. 2017), and the landslide events occurred with earthquakes in the period between 1968 and 2008 caused approximately 71,000 fatalities (Jessee et al. 2020). The number of deaths could be predicted by using a detailed inventory triggered by earthquakes covering a 207-year period from 1811 to 2016 (Jessee et al. 2020). This estimation will undoubtedly provide great benefits in terms of spatial planning and efforts to reduce losses due to landslides triggered by earthquakes. However, the most fundamental necessity here is to determine the landslide areas that are likely to be triggered by earthquakes.

If a region is susceptible to landsliding, which is likely to be triggered by earthquakes, regional landslide susceptibility assessments become even more important in terms of reducing earthquake-related damages by appropriate spatial planning. Kumar et al. (2021) also emphasized that mapping the areas prone to coseismic landslides in seismically active regions is essential. The Newmark (Newmark 1965) and/or the infinite slope models have been widely used in the literature for regional landslide susceptibility assessments triggered by earthquakes (Havenith et al. 2006; Das et al. 2013; Rodríguez -Peces et al. 2014; Chen et al. 2020a; Shinoda and Miyata 2017; Shinoda et al. 2019; Li and Su 2021; Nayek and Gade 2021). However, such models require accurate and representative geomechanical parameters of the slope forming materials, which can be used in the regional landslide assessments (Gokceoglu and Aksoy 1996). However, such a process is very difficult and cannot produce accurate results for complex geological environments. Therefore, conventional landslide susceptibility maps (LSMs) still pursue their importance. In the recent literature, it is possible to find studies on regional susceptibility assessments of landslides triggered by earthquakes (e.g. Xie et al. 2018; Chen et al. 2020a, b, c, 2021). However, if there is no landslide inventory prepared immediately after the earthquake, it is still difficult to distinguish the landslides in a region that are triggered by earthquakes. Therefore, the LSM production efforts must consider the coseismic landslide inventory of a region for analyzing the effects of seismic events and for improved landslide hazard assessment. Such inventories are also useful for the evaluation of the prediction performances of LSMs produced with novel machine learning (ML) methods.

Chen et al. (2012) described the large landslides triggered by the Wenchuan earthquake and concluded that large catastrophic landslides can be related to a particular geological setting, where fault type and geometry change abruptly. To prepare an up-to-date LSM of the region, the conditioning factors, in particular the geological and geomorphological characteristics which control landslides, should be considered. Chang et al. (2021) investigated the coseismic landslides triggered by the 2018 Iburi Earthquake and the described key factors on the landslides triggered by the Iburi Earthquake are peak ground acceleration (PGA), lithologhy, distance to fault, distance to the river, altitude, slope, aspect and curvature. Similarly, Mahalingama and Kim (2021) studied the effects of conditioning and triggering factors (such as slope, slope roughness, aspect, land use land cover (LULC), mean annual precipitation, distance to roads, rivers, PGA, and epicenters of the main shock earthquake and aftershocks) of landslides triggered by the Nepal Earthquake. As can be seen from several other studies (e.g., Chen et al. 2017b, 2021; de Oliveira et al. 2019; Wang et al. 2020, 2021; Adnan et al. 2020; Bui et al. 2020; Guo et al. 2021; Liu et al. 2021; Li et al. 2021; Matsakou et al. 2021), various conditioning factors were considered when producing susceptibility maps of landslides triggered by earthquakes, such as altitude, slope, aspect, plan and profile curvature, lithology, distance to faults, distance to rivers, topographic wetness index (TWI), stream power index (SPI), normalized difference vegetation index (NDVI), drainage density, precipitation, LULC, distance to roads depending on the geographical setting.

In the literature, various ML methods and statistical techniques have been proposed to produce LSMs (e.g. Pradhan et al. 2010; Nefeslioglu et al. 2012; Wang et al. 2016; Dagdelenler et al. 2016; Sevgen et al. 2019; Karakas et al. 2020; Kocaman et al. 2020; Medina et al. 2021; Can et al. 2021; Bera et al. 2021; Qi et al. 2021). In recent years, a significant rise in LSMs produced by using data-driven ML methods has been observed. Algorithms such as artificial neural networks (ANNs), support vector machine (SVM), decision tree (DT), random forest (RF), gradient boosting machine (GBM), fuzzy logic (FL) and logistic regression (LR) are among the ML algorithms used to produce LSMs (e.g. see Pourghasemi et al. 2013; Wang et al. 2016, 2020; Sevgen et al. 2019; Karakas et al. 2020; Yanar et al. 2020; Qi et al. 2021; Wu et al. 2020). Wang et al. (2020) compared susceptibility results for landslide prediction using five methods such as LR (0.77), SVM (0.80), RF (0.82), GBM (0.81) and MLP (0.80) in Shexian County. Among the five algorithms, RF yielded the highest accuracy. Bui et al. (2020) compared the accuracy of a deep learning neural network model (DLNN) with state-art-of-the ML algorithms in landslide susceptibility assessment and found that DLNN model obtained higher performance (0.90) followed by the multi-layer perceptron (MLP) (0.87). Adnan et al. (2020) proposed an approach for uncertainty reduction in landslide mapping that evaluated the compatibility of landslide prediction maps produced using four ML algorithms (k-nearest neighbors algorithm (k-NN), MLP, RF and SVM). The prediction results indicated that the RF model had the highest performance followed by the MLP, SVM, and k-NN models. On the other hand, supervised ML algorithms used for the LSM production mentioned above frequently utilize the receiver operating characteristics (ROC) including the area under the curve (AUC) value for the performance evaluation. However, the ROC and the AUC are obtained from the existing landslide inventory (i.e., model training and test data split from the same dataset) and this kind of QA may be insufficient as the LSMs aims at predicting the future landslide activities (Sevgen et al. 2019).

On 24 January 2020, at 20:55, the Elazig earthquake of 6.8 magnitude with a duration of 20.4 s occurred and triggered several landslides. The area affected by the earthquake is located around the Eastern Anatolian Fault Zone (EAFZ) and has mountainous topography, which is prone to landsliding. Two landslide inventories of the region, one before the earthquake and the other one representing the landslides triggered by the earthquake were compiled by Karakas et al. (2021a). Considering the availability of pre- and post-event landslide inventories, high-resolution digital elevation models (DEMs) and volumetric change maps, the present study aimed at evaluating the performances of two different state-of-the-art ML methods, such as the RF and the MLP, comprehensively for the LSM production in the region. Besides the visual investigations on the LSMs and the volumetric change maps, statistical metrics (i.e., ROC curve and AUC) obtained from the evaluated methods were assessed for the QA. The main novelties of the present study are; (i) three different approaches were used to comparatively evaluate the performances of the LSMs obtained from the RF and the MLP, (ii) only a part of the landslide inventory located in the western part of the study area was used for the model training to assess the model transferability, and (iii) the LSMs were produced for the whole study area.

Materials and methods

For the purposes of the study, temporal landslide inventories obtained from the pre-and post-event (i.e., before and after the Elazig earthquake) data derived by Karakas et al. (2021a) using aerial photogrammetric datasets acquired in three different years were employed. The pre-event landslide inventory and the DEM were used in the ML models to produce the LSMs. The pre-earthquake datasets were acquired in two different years (2017 and 2018) during regular photogrammetric mapping campaigns of the General Directorate of Mapping (GDM), Turkey. The temporal difference was caused by the flight planning based on province boundaries (2017 for Malatya and 2018 for Elazig Provinces). Therefore, the ML methods were trained only by using the data that remains in Malatya Province located in the western part of the study area; and the LSMs were produced for both provinces using the same model parameters. Thus, the transferability of the model parameters to the different parts of the same geographical area was also assessed in the study. The preliminary LSM results of the area were obtained with the RF method by using the inventory of both parts for model training (Karakas et al. 2021b) and the AUC results of the test pixels were 0.90 and 0.92 in Malatya and Elazig parts, respectively. A frequency ratio analysis (FRA) between the landslide occurrence and the pre-defined conditioning factors was also presented by Karakas et al. (2021b). In the following sub-sections, the study area characteristics, the datasets, the earthquake event and further methodological details are presented.

Study area and characteristics

The study area is located in the southeastern part of Turkey in the administrative boundaries of Malatya and Elazig Provinces (Fig. 1). As there is high seismicity and active tectonism in the region, the topography is very young and steep. Therefore, the lithological units have weak shear strength characteristics. Due to the topographical and lithological characteristics, the area is prone to landsliding (Sevgen et al. 2019). The FRA presented by Karakas et al. (2021b) indicated that Maden Complex, Puturge Metamorphites and Unconsolidated Gravel, Sand, Silt, Clay lithological units are most prone for landslides. The Puturge Metamorphites of uncertain age (Paleozoic to Mesozoic) in the study area is composed of different metamorphic rock units (gneiss, aphibolite, calcschist, etc.) of different origin, whereas pebblestone, sandstone, limestone, spilitic basalt etc. constitute the lithology of the Maden Complex of early to middle Eocene age (Keskin 2002). The geological characteristics of the study area is provided in Fig. 2. The geological units vary with the altitudes, such as magmatic and metamorphic rocks exist in the upper altitudes whereas young sedimentary rock are observed in the lower altitudes. Figure 1 depicts also the produced DEMs of the study area obtained from two different photogrammetric flight missions carried out in 2017 and 2018 in Malatya and Elazig Provinces, respectively. Thus, the datasets were named after Malatya and Elazig for clarity and processed separately for these two parts. The maps in Figs. 1 and 2 are presented in Universal Transverse Mercator (UTM) projection and World Geodetic System 1984 (WGS84) Datum Zone 37 North with the unit of meters (European Petroleum Survey Group—EPSG code 32637). While the altitude values range between 453 and 2172 m for the Malatya part in the study area, they range between 553 and 2031 m for the Elazig part. The FRA results presented by Karakas et al. (2021b) have shown that approximately 1000–1650 m altitudes are more critical for landslide occurrence. After the Elazig Earthquake, several slope deformations and failures were observed and reported by researchers (Tatar et al. 2020; Gokceoglu et al. 2020; Cetin et al. 2021; Temur et al. 2021).

Fig. 1
figure 1

The location map of the study area with (a) the generic view, (b) the DSM produced for Malatya (left, from the year 2017) and Elazig (right, from the year 2018) parts

Full size image
Fig. 2
figure 2

The geological map of the study area for Malatya (left) and Elazig (right) parts (Akbas et al. 2016)

Full size image

Photogrammetric datasets

The aerial photos were processed in three sets of photogrammetric blocks based on the dates of the acquisitions. The first two sets, i.e., pre-event photos, were acquired in 2017 and 2018 in Malatya and Elazig Provinces, respectively. The last set, the post-event photos, was acquired on 26 Jan 2020, two days after the earthquake event. The part of the inventory including the landslides triggered by the Elazig earthquake was prepared by using the 3D surface models and orthophotos obtained by processing the last set obtained after the event. This inventory was used for validation. These aerial photos have 30 cm spatial resolution and were processed to produce high-resolution digital surface models (DSMs) with 5 m grid spacing and orthophotos with 2 m spatial resolution. The post-event dataset was acquired for the earthquake region for disaster mitigation purposes by the GDM and has a 20 cm resolution. The photogrammetric dataset was explained in more detail in a previous publication (Karakas et al. 2021a). The DSMs were employed here as the elevation information (i.e., DEM) to avoid any errors which may be caused by post-processing to produce a digital terrain model (DTM) as the region has mostly open terrain.

The earthquake event and the landslide inventory

The Elazig earthquake with the Mw of 6.8 occurred on the Hazar-Sincik Segment of the EAFZ on January 24, 2020 (AFAD 2020). The maximum acceleration was measured as 0.293 g by Turkey Disaster and Emergency Management Presidency (AFAD). The landslide inventory was delineated manually using visual interpretations of the orthophotos and the DEMs (Gokceoglu et al. 2020; Karakas et al. 2021a). The smallest and the largest landslide areas are 133 m2 and 3 × 106 m2, respectively. Within the scope of preparing the landslide inventory, the suggestions made by Cruden and Varnes (1996) were taken into consideration. Accordingly, the landslide inventory was classified into four groups such as (i) inactive mass movements (L1), (ii) active mass movements (L2), (iii) areas containing new active zones occurred after 2018 inside the existing landslide (L3), and (iv) newly developed areas after 2018 (L4) (i.e., triggered by the earthquake). In this study, the inactive mass movements (L1) are dormant position. It was concluded that the L3 and L4 activities were induced during the Elazig earthquake (Karakas et al. 2021a). Out of a total of 328 landslides mapped by Karakas et al. (2021a), 316 of them were utilized in the present study (Fig. 3). 21% (67) of the landslides were defined as L1, 57% (180) as L2, 18% (57) as L3 and 4% (12) as L4. In addition, an iso-intensity map of the earthquake event was prepared by Yalcin et al. (2020a) using citizen science methods as proposed by Yalcin et al. (2020b) (Fig. 4). The distribution of the landslides is provided in Fig. 4. As can be seen from the Figure, the study area and the landslides are located in areas with intensity values between 5 and 7.

Fig. 3
figure 3

The DSM of the study area for Malatya (left) and Elazig (right) parts produced from pre-event photogrammetric datasets and the landslides classified by the activity type, such as L1: inactive mass movements, L2: active mass movements, L3: areas containing new active zones inside the existing landslide occurred after 2018, L4: newly developed areas after 2018 (triggered by the earthquake, 24 Jan 2020, Mw 6.8) (Karakas et al. 2021a)

Full size image
Fig. 4
figure 4

The distribution of the landslides on the iso-intensity classes of the Elazig Earthquake produced by Yalcin et al. (2020a)

Full size image

Methodology

The overall methodological workflow is provided in Fig. 5. The topographic derivatives of the study area, such as slope, aspect, plan and profile curvature, etc., were computed from the DEMs of the pre-event acqusitions (i.e. 2017 and 2018). The lithology data were digitized from 1:100,000 scale geological maps (Akbas et al. 2016), and rasterized with 5‐m grid spacing. The landslide inventory and the features extracted from the topographic data were used to produce the LSMs by employing the RF and MLP methods. The LSMs were validated using the test data, which were not utilized in the model training and were also compared with the landslide inventory which was not included in the training. L1 and L2 type landslides, which fall into the Elazig part of the study area were excluded from the training. The L3 and L4 type landslides triggered by the event were omitted from the LSM prediction and used for validation purposes only. In addition, the volumetric change detection maps produced in a previous study (Karakas et al. 2021a) were used for the performance evaluation by visual inspection.

Fig. 5
figure 5

The study workflow

Full size image

Feature extraction for landslide susceptibility mapping

The conditioning factors used for the landslide susceptibility modeling in the study can be investigated in two sub-categories, such as topographical and geological. The PGA or another earthquake-related parameter was not considered here because it is the main trigger of the landslides inventoried in the study. Consequently, the PGA should be used in landslide hazard assessment. The topographic factors considered here include seven parameters, such as altitude, slope, aspect, TWI, SPI, and plan and profile curvatures as summarized previously. The geodata types with their resolution/scale used in the study are shown in Table 1. These parameters were derived from the DEMs of 2017 and 2018. As an additional feature, the lithology data obtained from Akbas et al. (2016) were included in the prediction. The lithology is one of the parameters defining the most important unit in which the landslide occurred. These parameters are frequently used for producing the LSMs in the literature (e.g. see Gokceoglu and Ercanoglu 2001; Brenning, 2005; Nefeslioglu et al. 2012; Pourghasemi et al. 2013; Sevgen et al. 2019).

Table 1 Topographical and geological features used as conditioning factors in the study area and their source geodata
Full size table

The plan and profile curvatures, which are two types of curvatures and are mostly considered, were used in the study. They are the second derivatives of the digital elevation models (DEMs). The SPI defines the erosive power of flowing water (Zakerinejad and Maerker 2015). When calculating the SPI, the slope and the contributing area are used as shown in Eq. (1) (Moore et al. 1991).

$$SPI={A}_{s}\;x\;{\text{tan}}\;{\beta }$$
(1)

where As is the catchment area and β is the slope gradient (°).

The TWI is used in the determination of the hydrological conditions of the topography; and refers to the location and dimensions of the water-saturated areas in the basin area. This approach was first proposed by Beven and Kirkby (1979). Moore et al. (1991) suggested Eq. 2 to calculate the TWI.

$$TWI = \ln \left( {\frac{{A_{s} }}{\tan \beta }} \right)$$
(2)

where As is the catchment area and β is the slope.

The statistical values obtained from the altitude values and the derived topographic features, which include slope, aspect, plan and profile curvatures, TWI, and SPI are provided separately for Malatya and Elazig DEMs in Tables 2 and 3. The statistical summary explains the minimum (Min), Maximum (Max), Mean, Standard Deviation (σ), and Median (Med) values for the whole area (Table 2) and the parts inside the landslide inventory (Table3). The slope values range from 0° to 87° for the study area and the majority of the slopes accumulates between 20 and 25°. The higher slope values were observed along with directions of northeast-southwest (NE-SW) and northwest-southeast (NW–SE). The spatial distributions (maps) are provided in Fig. 6. When the aspect map is analyzed (Fig. 6), no accumulation in the values was observed.

Table 2 The model parameters used RF and MLP algorithms
Full size table
Table 3 Statistical summary of elevations and the topographic derivatives in the study area
Full size table
Fig. 6
figure 6

a The slope map, b the aspect map, c the plan curvature map, d the profile curvature map, e the TWI map and f the SPI map of the study area

Full size image

The histograms of all topographic parameters are presented in Figs. 7 and 8 for the Malatya and Elazig parts, respectively. The peaks in the altitude histograms are caused by the erroneous altitude values over the water surface. These errors are sourced from the false image matches in the DSM production process due to the textureless surface. The distributions of the plan and profile curvatures, TWI and SPI are in Gaussian pattern.

Fig. 7
figure 7

Histograms of the altitude values and the topographical features for Malatya part. a Altitude, b slope, c aspect, d plan curvature, e profile curvature, f TWI, and g SPI

Full size image
Fig. 8
figure 8

Histograms of the altitude values and the topographical features for Elazig part. a Altitude, b slope, c aspect, d plan curvature, e profile curvature, f TWI, and g SPI

Full size image

Landslide susceptibility mapping using MLP and RF methods

In this study, the RF and MLP methods were applied for LSM production and their performances were compared by using different approaches to produce the landslide susceptibility of the study area. The models are data-driven and learns the probability of spatial occurrence of the landslides from the data, i.e. by using landslide and non-landslide samples given for the area. The input features explained previously were used as conditioning factors for the prediction.

The MLP is a feedforward NN (Abiodun et al. 2019) and is currently a popular and widely used a supervised learning algorithm. It consists of two layers as input and output, and one or more hidden layers between them. Each neuron in the hidden layer processes the values from the previous layer with a weighted linear summation. The output layer receives the values from the last hidden layer and produces the output values (Chen et al. 2017a; Zare et al. 2013; Harmouzi et al. 2019; Sahana et al. 2020). In this study, the method was implemented using the Scikit-learn library (Scikit-learn 2021) in a Python programming environment. The final transformation was performed by using a non-linear activation function. The formula for MLP is given in Eq. 3.

$$x_{n}^{m} = f\left( {\mathop \sum \limits_{k = 0}^{i} w_{k}^{m} x_{k}^{m - 1} } \right)$$
(3)

where f represents the activation function, i is the number of neurons in layer m, \({w}_{k}^{m}\) refers the weight of kth neuron in layer m, \({x}_{k}^{m-1}\) is the activation of kth neuron in layer m−1, n is the neuron indices, and j is the layer indices.

The input layer is composed of a set of neurons representing the input features (altitude, slope, aspect, plan and profile curvatures, lithology, TWI and SPI). Here, the input layer has eight neurons containing the conditioning factors. The network was modelled with one hidden layer with 100 neurons in each. The Relu activation function was applied for the hidden layer. The Adam algorithm was used for weight optimization across the nodes. This algorithm is a stochastic gradient-based optimizer proposed by Kingma and Ba (2014). The maximum iteration denotes the number of epochs and used as 150 here. The learning rate approach used for weight updates was adaptive.

The RF algorithm is a powerful ensemble learning method of decision trees (DTs) (Breiman 2001). This algorithm creates DTs by a random selection at the training stage. The DTs are evaluated for the best score based on the average of the results of the trees. The most important features are in principle selected from all trees for classification. The RF minimizes the correlation between trees and provides higher classification performance. The general expression of RF is given in Eq. 4. The number of trees in RF defines how many DTs will be created while the model is being trained. Another important parameter is the maximum depth of the tree. In the present study, the hyperparameters used for running the RF and MLP algorithms were chosen heuristically. Since successful results were obtained using these parameters in previous studies using aerial photograps, similar parameters were also used in this study (Sevgen et al. 2019; Karakas et al. 2020). The parameters used for each algorithm are summarized in Table 4.

Table 4 Statistical summary of the altitudes and the topographic derivatives inside the landslide inventory (all activity types)
Full size table
$$Y\left( x \right) = {\text{arg}}_{z}^{{max}} \sum\limits_{{n = 1}}^{k} {I\left( {y_{n} \left( x \right) = {\text{Z}}} \right)}$$
(4)

where Y(X) represents RF model, \({y}_{n}\left(x\right)\) represents a single decision tree model, Z is the output variable and I (.) denotes the indicative function.

The RF and MLP algorithms were performed using Python scikit-learn library (Scikit-learn, 2021). Only the landslides with activity type 1 and 2 were used to train the model. These landslides were denoted with the black (L1) and the blue (L2) areas in Fig. 9. Due to the imbalanced distribution of the landslide inventory in the study area, a sub-area of Malatya part (marked with red rectangle in Fig. 9) was used for the model training for landslide and non-landslide samples. The ratio of the landslide non‐landslide samples was 1:2 as performed by Yanar et al. (2020). The other landslides, which fall outside the red rectangle in Fig. 9, were not employed during the model training stage but utilized for the performance assessment. The trained model was used for the LSM production for the whole study area including Malatya and Elazig parts. A training/test ratio of 70/30 percent of all samples was used for the model training and testing. Finally, the natural breaks classification was used to reclassify the predicted values into five categories using the Jenks optimization algorithm (Jenks 1967) implemented in ArcGIS software from ESRI Inc. USA. The algorithm minimizes the squared deviations of the class means and has the potential for an improved representation of the spatial distribution pattern in the data.

Fig. 9
figure 9

The model training area (red rectangle) in the Malatya part with the landslide inventory used for the training (L1 type: black polygons; L2 type: blue polygons)

Full size image

Accuracy assessment and validation

Three performance assessment approaches were employed in this study. The first one was the assessment of the prediction performance metrics obtained from the ROC curve, including the AUC; and the visual analysis of the ROC curve. The second assessment approach was the comparison of the prediction results with the landslide inventory, in particular the L3 and L4 types, which occured after the acquisition dates of aerial photogrammetric data (i.e. 2017and 2018) used for the LSM production. Thus, it is expected that the L3 and L4 type landslides must be located in highly susceptible areas.

As the last approach, the performances of LSMs for both parts were was assessed by visual comparison with the volumetric change maps with the same expectation (i.e. the L3 and L4 type landslides can be observed in the changes). Here, it must be noted that the surface changes were not only sourced from the earthquake-induced mass movements; but also include seasonal and land use land cover (e.g. vegetation, infrastructure, buildings, snow cover, etc.) changes in the area in the period of 2–3 years (Karakas et al. 2021a). The LS3D (least square surface matching) software (4DiXplorer AG, Switzerland) developed by Gruen and Akca (2005) was used for the production of volumetric change maps.

Results

Landslide susceptibility maps

The produced LSMs are provided for Malatya and Elazig parts in Fig. 10a (RF) and Fig. 10b (MLP). The LSMs were evaluated in five classes as very low, low, moderate, high and very high (Fig. 10, Table 5) obtained from the Jenks classification algorithm. The probability of occurrences are mostly at higher altitudes, but the slopes close to the water body and the drainage channels are also highly susceptible. Although the LSMs obtained from the both methods have similar patterns, the MLP has in general predicted lower susceptibility values in comparison to the RF especially for high and very high susceptibility areas.

Fig. 10
figure 10

The LSMs of the study area (left: Malatya part, right: Elazig part) produced with the a RF and b MLP methods

Full size image
Table 5 The landslide probability distributions obtained from RF and MLP algorithms
Full size table

Table 5 shows the areal distributions and the percentages of the different susceptibility classes in the predictions. The areas with high and very high susceptibility values obtained from the RF within the Malatya part was 36.94 km2 and 39.87 km2, respectively. The same classes in the Elazig part have coverages of 32.51 km2 and 31.65 km2, respectively. When the areal coverages of high and very-high susceptibility classes obtained from the MLP method for Malatya (high: 35.44 km2, very high: 27.63 km2) and Elazig (high: 26.54 km2, very-high: 33.57 km2) are considered; the RF results show higher susceptibility values especially for Malatya part, even though the ranges obtained from Jenks classification are also higher for the RF. In Elazig part, MLP produced slightly more pixels in the very-high susceptibility class, and less pixels in the high susceptibility class in comparison to the RF.

Quality assessment results

Prediction results

Figure 11 shows the ROC curves and the AUC values obtained from the RF model (Fig. 11a) and the MLP model (Fig. 11b). A total of 350,412 landslide and 700,824 non-landslide pixel samples were used for the training (735,865 pixels) and testing (315,371 pixels). The total number of samples contained in the eight feature sets was 8,409,888. In Fig. 11, the classes 0.0 and 1.0 reflect the non-landslide and landslide pixels. The curves were obtained from the training and test samples obtained from the training area, which is depicted with a red rectangle in Fig. 9. A ratio of 70/30 was preferred between the training and test samples. The AUC values were for both the non-landslide and landslide classes were 0.93 for the RF model. For MLP, these values were 0.87. These results show that the RF method exhibits higher prediction performance.

Fig. 11
figure 11

ROC curves obtained from the (a) RF and (b) MLP methods using the training and test samples. Class 0.0: non-landslide; Class 1.0: landslide

Full size image

Comparison of the LSMs with the post-earthquake landslide inventory

A visual comparison between the landslide inventory and the predictions was carried out using Fig. 12, which shows the LSMs and the inventory data in 3D perspective view. The results were analyzed in 3D geovisualization software. The pixels within the L3 (denoted with green) and L4 (denoted with pink) activity type landslides were not considered in the model training stage. For Malatya Part RF results, when the probability of the susceptibility results in the produced LSM was considered, the probability values between 0.55 and 0.76 indicate high class, and probability values between 0.76 and 1.00 correspond to very high class. For the Elazig Part RF results, these values are between 0.59 and 0.79 in the high class and between 0.79 and 1.00 in the very high class (Fig. 12a). When the MLP results are analyzed in Fig. 12b for Malatya Part, the probability values are between 0.53 and 0.75 in the high class and 0.75–1.00 in the very high class. For Elazig Part, the values are between 0.55 and 0.79 in the high class and between 0.79 and 1.00 in the very high class. In addition, a statistical summary of the susceptibility predictions obtained from a total of 69 L3 and L4 type landslides are provided in Table 6 for both methods. The values were grouped per region and the landslide types. According to the Table, the RF method predicted higher susceptibility values both for Elazig (mean values for L3: 0.62 and L4: 0.74) and for Malatya (mean values for L3: 0.64 and L4: 0.53) in comparison to the MLP. This shows the transferability of model parameters for RF for similar sites. These values show the success of RF and MLP models. On the other hand, according to Fig. 12, the MLP results seem to be a better fit when different geological characteristics are observed between the training and validation sites. As can be seen in Fig. 2, the lithological unit of unconsolidated gravel, sand, slit, clay type exists more in Elazig part and less in Malatya. The area is marked with red dashed ellipse in Fig. 12. Thus, the MLP produced more successful results in this lithological unit in Elazig.

Fig. 12.
figure 12

3D perspective views of LSMs predicted with (a) the RF and (b) the MLP algorithms together with the landslide inventory and the DEMs. The red dashed ellipses denote the area with lithological unit of unconsolidated gravel, sand, slit, clay type

Full size image
Table 6 Statistical summary of probability results obtained from a total of 69 landslides (L3 and L4)
Full size table

Volumetric change detection results

The volumetric change detection was performed previously by Karakas et al. (2021a) using high-resolution DSMs produced from pre- and post-earthquake aerial photos. The Euclidean distances (i.e. discrepancies) between the DSMs were calculated using the LS3D software. Figure 13 shows the Euclidean distance residual plots of Malatya and Elazig parts together with the landslide inventory. The red and the blue colours in Fig. 13 demonstrate the largest elevation changes in meters. The results were assessed visually. Figures 14, 15 and 16 show sub-areas from the different parts with their change detection maps, RF and MLP LSMs, and orthophotos from the pre- and post-event datasets. Note that the L3 activity type landslides are depicted with red polygons instead of green for increasing their visibility on the change detection maps.

Fig. 13
figure 13

The Euclidean distance residual plots of Malatya (left) and Elazig (right) parts

Full size image
Fig. 14
figure 14

The plots of a the Euclidean residuals in Malatya part; b RF LSM; c MLP LSM; d pre-event ortophoto from 2017; and e post-event orthophoto from 2020; all draped on the DSMs

Full size image
Fig. 15
figure 15

The plots of a the Euclidean residuals in Elazig part; b RF LSM; c MLP LSM; d pre-event ortophoto from 2017; and e post-event orthophoto from 2020; all draped on the DSMs

Full size image
Fig. 16
figure 16

The plots of a the Euclidean differences in Elazig part; b RF LSM; c MLP LSM; d pre-event ortophoto from 2017; and e post-event orthophoto from 2020; draped on the DSMs. The area was selected based on the red dashed ellipses, which denote the part with lithological unit of unconsolidated gravel, sand, slit, clay type, given in Fig. 12

Full size image

Discussions

The present study provided a comparative evaluation of various validation approaches for the LSMs, which were produced using two different supervised ML classifiers, i.e. the RF and the MLP methods. The classifications were performed for pixels. The landslide inventory produced by Karakas et al. (2021a) is a comprehensive one reflecting the landslides triggered by the Elazig Earthquake (24 Jan 2020, Mw 6.8) occured in Turkey. Considering the activity type, the landslide inventory was classified into four categories, such as inactive (L1), active (L2), areas containing new active zones occured after 2018 inside the existing landslide (L3), and newly developed areas after 2018 (L4), thus triggered by the earthquake. The LSMs were produced using the pre-event DSMs and the topographic derivatives, the geological characteristics of the area (lithology), and only the two types of landslides (L1 and L2), which were observable in the site in the pre-event datasets and thus were not triggered by the earthquake. The results were validated using the landslides triggered by the Elazig earthquake, i.e. the L3 and L4 activity type landslides. The LSMs were produced separately for the two sub-parts of the study area, i.e. Malatya and Elazig parts, due to approximately 1 year difference between the acquisitions of photogrammetric datasets in these provinces. There were a total of 40 landslides with activity type L3 and L4 for the Malatya part of the study area, and 29 landslides in total for the Elazig part.

The outcomes of the study can be discussed as follows;

  • The conditioning factors derived in the present study are the factors commonly used in the literature (e.g. see Gokceoglu and Ercanoglu 2001; Brenning 2005; Pradhan et al. 2010; Nefeslioglu et al. 2012; Pourghasemi et al. 2013; Sevgen et al. 2019; Dag et al. 2020; Chang et al. 2021; Guo et al. 2021; Mahalingama and Kim 2021). These parameters were found to be sufficient for obtaining high performance in landslide susceptibility modeling here. Although it is possible to employ further parameters, highly correlated conditioning factors may cause multicollinearity problem.

  • There are different supervised ML algorithms used for LSM production in the literature. However, the RF and the MLP algorithms are the most commonly used and also accurate ones among these algorithms (Zare et al. 2013; Chen et al. 2017a, b; de Oliveira et al. 2019; Harmouzi et al. 2019; Sevgen et al. 2019; Adnan et al. 2020; Karakas et al. 2020; Sahana et al. 2020). For this reason, in the study, RF and MLP algorithms were used to produce more accurate and effective LSM.

  • The most important contribution of the study is the detailed comparison of three different approaches for validating the LSMs. The first approach was based on the ROC curve and AUCs obtained from the model outputs. The training/test ratio was 70/30 percent in the training area, which only falls inside Malatya part. The second approach was qualitative and quantitative validations via visual assessments on the LSMs and the L3 & L4 type landslides; and the analysis of prediction statistics for the pixels which fall into these landslides. The last approach was a qualitative evaluation based on the comparison of volumetric change maps and the LSMs.

  • When the first approach is considered, the RF provides higher prediction performance with an AUC value equal 0.93 in comparison to the MLP (AUC = 0.87). The success of DTs and in particular the RF method for LSM production is well-known in the literature (e.g. see Chen et al. 2017b; Sevgen et al. 2019; Adnan et al. 2020; de Oliveira et al. 2019; Karakas et al. 2020). In this study, the RF algorithm has proven to be successful.

  • When the second approach is considered, the quantitative results presented in Table 6 show that the RF could predict higher susceptibility values for the L3 and L4 type landslide pixels with the mean values that fall into high susceptibility class as explained in Fig. 12 and Table 5. For Malatya part, the MLP provided mean susceptibility values for both landslide types that remain in the moderately susceptible class. For Elazig part, the mean susceptibility value (0.56) inside the L3 type landslides fall into the high susceptibility class, whereas for L4 type the mean value was 0.53 and thus moderately susceptible. The standard deviations obtained from both methods are quite similar for both parts and range from 0.11 to 0.14. The median values are comparable to the mean values, which imply a normal distribution of the predictions. It must be noted here that the numbers and the sizes of L3 and L4 type landslides are diverse; and the L4 type landslides are smaller and less frequent.

  • On the other hand, although the quantitative assessment in the second approach indicates the better performance of the RF, MLP was found more successful in a lithological unit which was not included in the model training area, which falls into the Malatya part. As can be seen from Fig. 16, MLP outperformed RF for this unit. Thus, it can be said that although the model parameters can be transferred to another site in the same basin, the RF is more sensitive to the changes in the data and may not be able to predict correctly.

  • When the third approach is considered, a relationship between the residual maps and the LSMs can be observed. It can be thought that small movements inside the susceptible areas could be detected by the high-resolution change detection map. Again, due to the DSM production approach and the temporal difference between the pre- and post-earthquake datasets, the Euclidian residuals presented here have uncertainties caused by the other types of surface changes.

  • When the prediction performance obtained in the present study is compared to the results of Wang et al. (2020), it was observed that the AUC values obtained in the present study are higher (RF AUC = 0.82 and MLP AUC = 0.81). While the MLP results are similar to the study by Bui et al. (2020), the RF (0.83) results were better here. The differences can be associated with the differences in the geographical setting and the data characteristics.

  • When the landslide conditioning parameters applied here are considered, they were found suitable. Although the LULC was not considered as an effective factor in the study, the LSMs can be updated when important LULC changes occur in the region. In addition, the new landslides triggered by the earthquake must be taken into account when the new LSM using the DSM of 2020 is produced.

Conclusions and future work

The study area is a region with high seismicity, which is located along the EAFZ. In addition, the study area is prone to landsliding. In this study, two LSMs were produced considering the data before the earthquake and their performances were assessed using the landslides triggered by Elazig Earthquake (24 Jan 2020, Mw 6.8). The RF and MLP algorithms were applied to produce the LSMs. The landslide inventory prepared by using pre-event datasets were used in the LSM production and validated with the landslide inventory triggered by the earthquake. The ROC curves and volumetric change detection map were used for this purpose. Considering the AUC values, the RF outperforms the MLP. When the susceptibility values inside the landslides triggered by the earthquake were statistically analysed, again the RF provided higher values, which fall into the high susceptibility class. On the other hand, in a lithological unit, which was not used for training, the MLP provided better performance. The results show that the RF can provide higher performance in different sites only if the similarity between the sites are also high. It is also recommended to check the LSMs via visual qualitative assessment.

The study contains two main novelties. One of these is the production of a susceptibility map with a landslide inventory map prepared before a major earthquake, and performance validation using the inventory map prepared after the earthquake. In particular, the existence of a landslide inventory both before and after the earthquake made a significant contribution to the performance evaluation. This contribution is another important novelty of the study. As a result, a procedure for assessing landslide susceptibility triggered by earthquakes was put forward by including these novelties in the study. After this stage, the production of landslide hazard maps and risk management will become more applicable, taking into account earthquake parameters as the triggers.

As a future recommendation, depending on the development of geo-spatial technologies, high-resolution images to be obtained immediately after major earthquakes allow the inventory of triggered landslides to be compiled with high precision. Such inventories will enable the production of LSMs with much higher prediction capacity.