NRIC: A Noise Removal Approach for Nonlinear Isomap Method

Abstract

Nonlinear manifold learning is a popular dimension reduction method that determines large and high dimensional datasets’ structures. However, these nonlinear manifold learning methods, including isomap and locally linear embedding, are sensitive to noise. In this paper, we focus on the noisy nonlinear manifold learning method, such as Isomap. The main problem of the Isomap is sensitivity to noise. Our proposed new method noise removal isomap with a classification (NRIC), is based on the local tangent space alignment (LTSA) algorithm with classification techniques to remove noises and optimize neighborhood structure Isomap. The primary purpose of the NRIC is to increase efficiency, reduce noise, and improve the performance of the graph. Experiments on the real-world datasets have shown that the NRIC method outperforms efficiently and maintains an accurate low dimensional representation of the noisy nonlinear manifold learning data. The results show that LTSA with classification techniques provides high accuracy, mean-precision, mean-recall, and areas under the (ROC) curve (AUC) of the high dimensional datasets and optimizes the graphs.

1 Introduction

Nonlinear Manifold learning is an efficient approach for dimension reduction. Various methods and algorithms have been offered for analyzing the basic structure of the high and large-scale dimensional data, which attracted much more attention in the machine learning area [55]. The basic idea of manifold learning is to transform the high dimensional data into low dimensional space and retain the most important information [17]. In recent years has attracted much more attention to the great importance of studying manifold learning for nonlinear dimension reduction [51, 58].

In 2000, the Isometric Mapping algorithm became a hot research topic in nonlinear manifold learning and information science [54]. Isomap is a nonlinear manifold learning algorithm that is widely used for nonlinear dimension reduction [14]. The basic idea of Isomap is a variant of the Multidimensional Scaling (MDS) metric, which preserves the global intrinsic structure of the data points. It maps the high dimensional data into low dimensional space [45]. Heeyoul and Seungjin [2, 6] proposed the kernel Isomap algorithm to solve the noises and the outlier problem in topological stability. The limitation of the proposed schemes [2, 6] is it destroyed the original Isomap algorithm. This approach preserves topological stability when dealing with outliers and noises. They also suggested the robust kernel Isomap method for topological stability, noises, and outliers problems [8]. They reduced the effect of outliers based on the topological structure with network flow help [7, 8]. H. Chang and DY proposed [5] the robust LLE method for noises and outliers. This method can improve the robustness of LLE, eliminating the outliers and noises in data points. The main drawback of this method outliers and noises are still controlled in data points, and robustness is still reduced to some data points. Kouropteva et al. [23, 24] and Shao et al. [42] proposed the selection of the optimal parameter values method for LLE, and Isomap and Saxena et al. [41] proposed the integrated approach for Isomap and LLE.

In addition, Bo Li et al. [28] proposed the expanded Isomap approach for improving the robust LLE process, and the robustness of the original Isomap was reduced. This method uses the weighted Principal Component Analysis (PCA) [5] to measure the noises and outliers in data points. So every weight point in the data set will be allocated through local robust PCA. To detect weighted noises and outliers, R. McGill et al. use the box statistic method [30]. After de-noising, this method will easily retain the topological structure [28].

In our motivation, we have focused on studying the nonlinear Isomap noise problem. The Isomap algorithm is also noise sensitive. Isomap algorithm is not suitable for real-world datasets, as the datasets are noiseless. We propose a novel approach called Noise Removal Isomap with Classification (NRIC) method for overcoming the Isomap noise problem.We have used the Local Tangent Space Alignment (LTSA) algorithm with classification techniques for the Isomap noise problem. To effectively eliminate the noises in data points, we used the concept of LTSA as a nonlinear manifold learning technique. We have used different classification techniques such as Support Vector Machine (SVM) [49, 50], K Nearest Neighbor (KNN) [16, 29], Naïve Bayes (NB) [20], and Random Forest (RF) [25]. Isomap algorithm can’t easily map high-dimensional data to low-dimensional space by using classification techniques and real-world datasets because it’s very noisy.

Our proposed method results show that the LTSA algorithm with classification techniques can significantly improve the original Isomap in a noisy environment. Also, our proposed method produces accurate results for large and high-dimensional datasets while reducing data point noise. In Sect. 3, we explain in detail the techniques and the algorithms.

Contributions In summary, the contribution of our proposed method is given below:

• 1. We propose the NRIC method for the Isomap noise problem. Our proposed method used an LTSA algorithm with well-known classification techniques. Our proposed NRIC method can easily embed the high dimensional data space into low dimensional space and optimize the neighborhood graph.

• 2. We conduct extensive experiments to analyze our NRIC method on five datasets empirically. We calculate the accuracy, mean-precision, mean-recall, and Area under the ROC (Receiving Operating Characteristics) Curve (AUC) for our proposed method and provide the effective noise removal results.

• 3. We improve the Isomap noise problem’s performance by using the different neighborhood value of K. The experiment section shows the effectiveness of our proposed NRIC method according to K values.

The paper is organized as follows. In Sect. 2, we will give brief details of the Isomap and Machine learning classifier. We will provide details of the proposed method and LTSA, and classification techniques in Sect. 3. Section 4 describes the experimental results on five large scale datasets. Finally, the paper is concluded in Sect. 5.

2 Related Work

Classical Isomap is viewed as a variant of Multidimensional Scaling (MDS) metric to model nonlinear data using its geodesic distance. The primary purpose of Isomap preserves the geometry of data and gets the geodesic distance between all pairs of data points. The geodesic distance is divided into two parts, such as neighborhood data points and faraway data points. In neighborhood points, the Euclidean distances between neighboring points are provided approximated geodesic distance by input-space. In faraway points, the geodesic distances are calculated the approximated by the shortest paths in neighborhood points [28, 37, 38]. The main three steps of Isomap are given below:

Step-1: Build neighborhood graph G Firstly, build the K nearest neighbor (KNN) graph G of manifold learning based on the Euclidean distance d between two data points in the input space $X_{i}$ and $X_{j}$, i.e., d=$X_{i}$, $X_{j}$ = ||$X_{i}$-$X_{j}$|| [28].

Step-2: Calculate the shortest distance When builds the neighborhood graph G, then calculate the geodesic distance matrix between sub-neighborhood faraway data points and computes the shortest path distance between any two data points $X_{i}$ and $X_{j}$ is the graph G by Dijkstra and Floyd’s algorithm [18].

Steps-3: Build a d-dimensional embedding graph Isomap uses the MDS algorithm to compute the low d-dimensional embedding of the data points and make the geodesic distance dense matrix [43].

2.1 Machine Learning Classifier

Machine Learning (ML), which is used in different research fields, including Artificial intelligence, data classification, and Statistic concerned with the automatic acquisition of knowledge datasets. These techniques are capable of improving the performance of datasets from experience [34]. The famous research field of ML is data classification. Data classification is provided various algorithms such as Support Vector Machine (SVM), K Nearest Neighbor (KNN), Random Forest (RF), Naïve Bayes (NB), Artificial Neural Network (ANN), Classification and Regression Tree (CART), Decision Tree, etc. [33]. The primary process of classification is to predict the label’s data points from given datasets. The label data points are sometimes called classes, targets, and categories. Classification Predictive Modeling (CPM) is mapped the input data points through a mapping function and predicts the possible output data points. A detailed description of classification algorithms is given in Sect. 3.

3 Noise Removal Isomap with Classification (NRIC)

In this section, we propose a novel approach, which is called the Noise Removal Isomap with Classification (NRIC) method for the Isomap noise problem. The main idea of our NRIC method is to eliminate the noise quickly, map the high dimensional data into low dimensional space, and then easily optimize the neighborhood graph. We have using the LTSA algorithm with classification techniques in our proposed method. Our NRIC method provided high accuracy rather than Isomap and reduced the noise from data points. We have used classification techniques, such as SVM, KNN, NB, and RF, with different K values. The algorithm 1 of our NRIC method is given below:

3.1 Local Tangent Space Alignment (LTSA) Algorithm

In 2004, Zhang and Zha introduced the nonlinear local tangent space alignment method for embedding [59]. This method can easily embed the high dimensional data into low dimensional space. LTSA can be used for noise problems and very efficiently eliminates noise. LTSA’s central concept is LLE variants and employed the same geometric manifolds as LLE. LTSA uses a distinct method to the embedded manifold space compared with LLE. In LLE, every point of the datasets is locally linearly embedded into the manifold’s linear plot then constructed the low dimensional datasets. So that preserved the locally linear relationships of the original datasets. Moreover, LTSA has built a locally linear patch by using the PCA method on the neighbors, and then the patch can be evaluated as an approximation of local tangent space at the point [52]. The algorithm 2 of LTSA is given below:

3.2 Support Vector Machine (SVM) Classifier

SVM has attracted much more attention and used very actively in several research applications such as regression, learning classification, and ranking function. The basic idea of SVM is dependent on the Structural Risk Minimization (SRM) principle and statistical learning theory and identifying the position of decision space, also called hyperplane, that generates the optimal partition of classes [4, 9, 13, 35]. SVM uses an isolating hyperplane to create an SVM event model classifier. The main issues of SVM cannot be isolated directly in the information space. This method provides a probability function to identify an answer by performing principle information space improvement in high dimensional space, where a perfect portioning hyperplane can be found [39]. In the experiment, we have used the linear kernel model for the SVM classifier. The algorithm 3 of SVM [47] is given below:

3.3 K- Nearest Neighbors (KNN) Classifier

The KNN classifier is the simplest classification method and used in machine learning and data mining. It is beneficial and easy to implement. It does not require a fitting model for classifying various types of datasets and provide the best performance of the multiple types of datasets [21]. In contrast, the best performance of KNN depends on the distance metric for calculating the distance between data points of Euclidean. The KNN data points often use Euclidean distance for similarity [1]. The KNN makes the training samples by itself according to the laws of classification. The KNN algorithm is classifying the objects based on the nearest training samples in the attributes. KNN method is a kind of lazy learning and instance-based learning because the KNN function is locally approximated, and all execution of KNN is delayed until classification [36]. The KNN classifier can easily find the closest samples from training datasets. In the results section, K parameters are used as an optimal value [56, 57]. The algorithm 4 of KNN [44] is given below:

3.4 Naïve Bayes (NB) Classifiers

NB Classifiers are easy probability classifiers based on the Bayes Theorem [27], and the NB classifier is mostly used when input data dimensionality is high. This classifier is efficient for computing the available output data based on the input data. It adds new available raw input data at runtime and has an efficient probabilistic classifier [35]. Different types of NB classifier is accessible by the assumptions on the distribution of features; these are called event models of NB classifier [22], including Bernoulli or multinomial distributions, Gaussian distributions [32], and discrete features. We have used the Gaussian event model in our proposed method for calculating the accuracy. In our proposed method, we used the Gaussian event model to calculate accuracy. It’s also slightly quicker and more efficient than SVM [53]. The NB [40] algorithm 5 is given below:

3.5 Random Forest (RF) Classifier

T. Kam Ho [15] was introduced RF in 1995, which uses the tree as parallel. RF is a collaborative learning classification algorithm (ensemble) combining the same and different types of more than one algorithm to classify the object. RF classifier is a randomly selected subset of the training dataset in the set of decision trees. It is a fast method to train the dataset rather than other techniques such as deep learning, although less slow to predict once trained datasets [3, 25]. The algorithm 6 of RF [46] is given below:

4 Results and Discussion

This section analyzes the effectiveness and efficiency of the NRIC, general experiments on high dimensional and large scale datasets. We have analyzed the proposed algorithms’ performance with an LTSA algorithm and classification techniques such as SVM, KNN, NB, and RF. Moreover, we have also compared our proposed NRIC method with different neighborhood K values.

4.1 Datasets

The experiments were organized on a large scale and high dimensional datasets such as Iris [11], Wine [12], Labeled Faces in the Wild (LFW) people [19], Breast Cancer [31], and Digits [26]. The detailed information on the datasets is listed in Table 1.

Table 1 Experimental datasets

4.2 Evaluation Methods

Various metrics are used to assess the proposed approach and its effectiveness. The classification algorithm provides multiple criteria for evaluating the resulting datasets, such as precision, accuracy, recall, and area under the ROC curve (AUC). Here, TP is the number of correct positive predictions. TN is the number of correct negative predictions, FP is the number of false-positive predictions, and FN is the number of false-negative predictions. The significance of these four parameters is based on the classification application. The division of correct predictions overall predictions is called accuracy. The ratio of correctly positive prediction overall positives predictions is called precision. The ratio of correctly negative predictions overall negative predictions is called recall [25]. The ROC curve area is well known for the classification evaluation method used in machine learning and data mining areas. The ROC curve takes True Positive Rate (TPR) and False Positive Rate (FPR) for a given classification algorithm [48]. ROC graphs are beneficial to organize a better visualize and classifier performance. Also, ROC graphs are present a better relationship between TPR and FPR. AUC can reasonably measure the model’s prediction quality and display the classifier performance in a single value. If the AUC value is greater, the classifier performance is better, and otherwise, the performance is not well [10]. The formulas of accuracy Eq. (1), mean-precision Eq. (2), mean-recall Eq. (3), and area under the ROC (Receiving Operating Characteristics) curve (AUC) Eqs. (4) and (5) are defined below:

$$\begin{aligned} Accuracy= & {} \left[ \frac{ (TP+TN) }{ (TP+FP+FN+TN) }\right] \end{aligned}$$

(1)

$$\begin{aligned} Mean-Precision= & {} \left[ \frac{ (TP) }{ (TP+FP) } \right] \end{aligned}$$

(2)

$$\begin{aligned} Mean-Recall= & {} \left[ \frac{ (TP) }{ (TP+FN) } \right] \end{aligned}$$

(3)

$$\begin{aligned} TPR= & {} \left[ \frac{(TP)}{(TP+FN)}\right] \end{aligned}$$

(4)

$$\begin{aligned} FPR= & {} \left[ \frac{(FP)}{(FP+TN)}\right] \end{aligned}$$

(5)

4.3 Accuracy

Tables 2, 3, 4, 5, 6 shows the calculated accuracy of Iris, Wine, LFW, Breast Cancer, and Digits Datasets from the Eq. (1). We have represented the calculated accuracy of five different high dimensional datasets in Figs. 1, 2, 3, 4, 5. Our proposed method graph consistently achieves high accuracy with different neighborhood K values. We used the different K values such as 10, 15, 20, 25, 30, and 40 for calculating the accuracy of the five datasets. We have compared LTSA with four classification techniques such as SVM, KNN, NB, and RF for five datasets. These classification techniques are work very well with LTSA and provide efficient and effective results for the Isomap noise problem. Therefore, the Isomap method cannot work well with classification techniques rather than our proposed method.

Table 2 Accuracy of iris dataset with SVM, KNN, NB, and RF

For the Iris dataset, we have achieved 100% accuracy for different values of K, such as 10, 15, 20, 25, 30, and 40. These classification techniques also have achieved 100% accuracy for different values of K. According to a comparison of classification techniques; the LTSA algorithm is provided 100% accuracy with SVM, KNN, NB, and RF in Fig. 1. In Table 2, LTSA-SVM is provided the 100% accuracy for values of K (10, 20, 25, and 40). LTSA-KNN is provided 100% accuracy for values of K=10. LTSA-NB and LTSA-RF are delivered with 100% accuracy only on K=10. Therefore, we have compared the Isomap with classification techniques. Still, ISO-SVM has only achieved 92% accuracy for the iris dataset on K=10, ISO-KNN has achieved 93%, ISO-NB has achieved 91%, and ISO-RF has achieved 94%. In Fig. 1, LTSA is performed well with SVM, KNN, NB, RF rather than Isomap with classification techniques.

Fig. 1

Accuracy of Iris dataset with NRIC and isomap methods

Table 3 Accuracy of wine dataset with SVM, KNN, NB, and RF

Table 4 Accuracy of LFW people dataset with SVM, KNN, NB, and RF

For the Wine dataset, we have achieved 100% accuracy only on K=15, 30, and 40 for the LTSA-SVM. In Table 3, LTSA-KNN has gained 100% accuracy on K=15 and 20, LTSA-NB has increased 100% accuracy on K=25, and LTSA-RF has achieved 98% accuracy on K=15. Therefore, we have compared the Isomap with classification techniques. Still, ISO-SVM has only achieved 90% accuracy on K=10 and 20, ISO-KNN has achieved 92% on K=15, ISO-NB has achieved 93%, and ISO-RF has achieved 90% both on k=10. In Fig. 2, we have shown the comparison of NRIC and Isomap method with classification techniques. However, LTSA works well with SVM rather than KNN, NB, and RF, and rather than Isomap with classification techniques in Fig. 2.

For LFW people dataset having 13233 data points and 5828 dimensions. We have used 100 dimensions for calculating the accuracy of the proposed algorithm. In Table 4, the LFW dataset is vast. Therefore, the accuracy of the LFW dataset is reduced very severely. We achieved 93% accuracy only on K=20 for the LTSA-SVM, 88% accuracy on K=30 for LTSA-KNN, and LTSA-NB has achieved 81% accuracy K=40. LTSA-RF has reached 68% accuracy for all K values. Therefore, ISO-SVM has only achieved 88% accuracy on K (20, 40). ISO-KNN has achieved 80% ISO-NB has reached 77% both on the value of K=10, and ISO-RF has reached 65% accuracy. In Fig. 3, we have shown the comparison between NRIC and Isomap method with classification techniques. However, LTSA is performed well with SVM, KNN, and NB rather than RF and Isomap.

Fig. 2

Accuracy of wine dataset with NRIC and isomap methods

Fig. 3

Accuracy of LFW people dataset with NRIC and isomap methods

Table 5 Accuracy of breast cancer dataset with SVM, KNN, NB, and RF

Fig. 4

Accuracy of breast cancer dataset with NRIC and isomap methods

For the Breast Cancer dataset, we have achieved 98% accuracy on K=40 for the LTSA-SVM, 94% accuracy for LTSA-KNN on K=40, 94% accuracy for LTSA-NB, and LTSA-RF on K=30, as shown in Table 5. Therefore, 94% accuracy for ISO-SVM on K (20, 40), ISO-KNN has achieved 90%, ISO-NB has gained 88% both on the value of K=10 ISO-RF has reached 90% accuracy on K=30. In Fig. 4, we have shown the comparison between NRIC and Isomap with classification techniques. However, LTSA works well with SVM, KNN, NB, and RF rather than Isomap.

Table 6 Accuracy of digits dataset with SVM, KNN, NB, and RF

For the Digits datasets having 1797 data points and 64 dimensions. We have used 50 dimensions for calculating the accuracy of the proposed method. Table 6 achieved 97% accuracy for LTSA-SVM on K=30, 95% LTSA-KNN, 94% LTSA-NB, and 84% LTSA-RF on the same value of K=10. Therefore, 95% accuracy for ISO-SVM on K (30), ISO-KNN has achieved 93%, ISO-NB has reached 92%, and ISO-RF has reached 80% accuracy on K=10. In Fig. 5, we have shown the comparison between NRIC and Isomap with classification techniques. LTSA does work well with SVM, KNN, NB, and RF instead of Isomap.

Fig. 5

Accuracy of digits dataset with NRIC and isomap methods

In addition, the NRIC method’s overall performance consistently achieves high accuracy for the five high dimensional datasets. Our NRIC method is much faster than Isomap and very effectively reduced the noise in the dataset. Then the NRIC method easily maps the high dimensional data into a low dimensional manifold. Sometimes, accuracy is high and down according to different K’s values because sometimes the KNN are close to each other and discover quickly. Otherwise, the KNN is far away from each other and recognize hardly them. Moreover, according to different neighborhood K values, some dataset accuracy performance is better, and several other dataset’s accuracy performances are in “V” shape. The “V” shape shows that the K is optimal data-dependent.

4.4 Mean-Precision

Tables 7, 8, 9, 10, 11 shows the calculated mean-precision of Iris, Wine, LFW, Breast Cancer, and Digits Datasets from the Eq. (2). In Figs. 6, 7, 8, 9, 10, we have represented the calculated mean-precision of five different high dimensional datasets with NRIC and Isomap methods. The graph of our proposed NRIC method consistently achieves the high mean-precision with different neighborhood K values. We have compared NRIC and Isomap with four classification techniques such as SVM, KNN, NB, and RF for five datasets. The mean-precision performance of NRIC with classification techniques very well and provided effective results for the Isomap noise problem.

Table 7 Mean-precision of iris dataset with SVM, KNN, NB, and RF

We have attained 100% mean-precision for the Iris dataset for different values of K (10, 15, 20, 25, and 40). Table 7 shows that LTSA-SVM is provided a 100% mean-precision for the values of K (10, 15, 20, 25, and 40); LTSA-KNN, LTSA-NB, and LTSA-RF are provided the 100% mean-precision on the value of K=10. The mean-precision of ISO-SVM is provided 96% on K=10, 95% mean-precision for ISO-KNN on K=10, 94% mean-precision for ISO-NB on K=10, and 93% for ISO-RF on K=10. In Fig. 6, the performance of the LTSA with SVM, KNN, NB, RF is better than Isomap.

Fig. 6

Mean-precision of iris dataset with NRIC and isomap methods

Table 8 Mean-precision of wine dataset with SVM, KNN, NB, and RF

For the Wine dataset, we have attained 100% mean-precision only on K (15, 30, and 40) for the LTSA-SVM. In Table 8, mean-precision of 100% LTSA-KNN (K=15 and 20), 100% of LTSA-NB on K=25, and 97% LTSA-RF (K=15). Therefore, we have compared Isomap with classification techniques such as ISO-SVM is presented with 90% mean-precision, 87% of ISO-KNN, 92% of ISO-NB, and 91% of ISO-RF are provided mean-precision on the same value of K=10. In Fig. 7, we have shown the mean-precision of the NRIC and Isomap methods with classification techniques. However, the mean-precision of LTSA with SVM, KNN, NB, and RF is better than the Isomap method.

Fig. 7

Mean-precision of wine dataset with NRIC and isomap methods

Table 9 Mean-precision of LFW people dataset with SVM, KNN, NB, and RF

Fig. 8

Mean-precision of LWF people dataset with NRIC and isomap methods

For Labeled Faces in the Wild (LFW), we have attained 100% mean-precision on the value of (K=10,15,20,25,30,40) for LTSA-SVM and LTSA-RF, as shown in Table 9. The mean-precision performance of the 90% LTSA-KNN (K=20, 25, 40), 83% LTSA-NB (K=40). Therefore, mean-precision performance of the 89% ISO-SVM (K=20,40), 87% ISO-KNN (K=25), 80% ISO-NB (K=30,40), and 96% ISO-RF (K=10). In Fig. 8, we have shown the comparative performance of the mean-precision of the LTSA and Isomap method with classification techniques. However, LTSA is performed better with all classification techniques rather than Isomap.

Table 10 Mean-precision of breast cancer dataset with SVM, KNN, NB, and RF

Fig. 9

Mean-precision of breast cancer dataset with NRIC and isomap methods

For the Breast Cancer dataset, we have reached a 98% mean-precision for LTSA-SVM, 95% LTSA-KNN both on K=40, 95% LTSA-NB, and LTSA-RF (K=30), as shown in Table 10. Therefore, the mean-precision performance of the ISO-SVM is 95%, 91% ISO-KNN both on (K=40), ISO-NB is achieved 90%, and 93% ISO-RF on the same value of K = 30 in Table 10. The mean-precision performance of the NRIC is higher than the Isomap method. In Fig. 9, we have shown the comparative performance of the mean-precision of the LTSA and Isomap methods with classification techniques. However, LTSA is performed very well with SVM, KNN, NB, RF rather than the Isomap method.

Table 11 Mean-precision of digits dataset with SVM, KNN, NB, and RF

Fig. 10

Mean-precision of digits dataset with NRIC and isomap methods

For the Digits dataset, we have reached high mean-precision 97% on K=25, 30 for LTSA-SVM, as shown in Table 11. The mean-precision of the LTSA-KNN is attained 95% on (K=10, 20), 94% for LTSA-NB, and 87% LTSA-RF on the same value of K=10. Therefore, ISO-SVM is achieved 94% on K=30 and 40, 93% ISO-KNN, 90% ISO-NB, and 84% ISO-RF on the same value of K=10. In Fig. 10, we have shown the comparative performance of the mean-precision of the LTSA and Isomap methods with classification techniques. However, the performance of the LTSA is well with SVM, KNN, NB, and RF rather than Isomap. Moreover, the overall mean-precision performance of the NRIC method consistently attains the high mean-precision for five high dimensional datasets. Sometimes, mean-precision performance is high and low according to the different values of K and classification techniques. Sometimes KNN values are close to each other and find easily. Otherwise, the KNN is far away from each other and find hardly.

4.5 Mean-Recall

Tables 12, 13, 14, 15, 16 shows the calculated mean-recall of Iris, Wine, LFW, Breast Cancer, and Digits Datasets from the Eq. (3). In Figs. 11, 12, 13, 14, 15, we have represented the calculated mean-recall of five different high dimensional datasets. The graph of our proposed NRIC method consistently reaches the high mean-recall of five datasets. We have used the different neighborhood values of K as same as accuracy and mean-precision. The mean-recall performance of LTSA with classification techniques very well and provided effective results for the proposed NRIC method.

We have attained 100% mean-recall for the Iris dataset for different K (10, 15, 20, 25, and 40) of LTSA-SVM. LTSA-KNN, LTSA-NB, and LTSA-RF are provided 100% mean-recall on the value of K=10 in Table 12. Therefore, 93% ISO-KNN mean-recall on K (10, 15, and 20) values, ISO-SVM is provided the 93% mean-recall, 92% ISO-NB, and 94% LTSA-RF, are on the same value of K=10. The performance of Isomap is lesser than the LTSA with classification techniques. In Fig. 11, the mean-recall performance of LTSA with SVM, KNN, NB, and RF is better than the Isomap method.

Table 12 Mean-recall of iris dataset with SVM, KNN, NB, and RF

Fig. 11

Mean-recall of Iris dataset with NRIC and isomap methods

Table 13 Mean-recall of wine dataset with SVM, KNN, NB, and RF

Fig. 12

Mean-recall of wine dataset with NRIC and isomap methods

We have attained 100% mean-precision on (K=15,30, and 40) for the LTSA-SVM wine dataset. In Table 13, the mean-recall of the LTSA-KNN is 100% (K=15 and 20), LTSA-NB is 100% (K=25), and LTSA-RF is 97% (K=15). Therefore, the mean-recall of the ISO-SVM is 90% (K=10), ISO-KNN is 92% on (K=15), ISO-NB is 92% on (K=10), and ISO-RF is 90% on (k=10). In Fig. 12, we have shown the mean-recall of the LTSA algorithm with classification techniques and Isomap. However, the mean-recall performance of LTSA with NB, RF, KNN, and SVM is higher than the Isomap method.

Table 14 Mean-recall of LFW people dataset with SVM, KNN, NB, and RF

Fig. 13

Mean-recall of LWF people dataset with NRIC and isomap methods

For Labeled Faces in the Wild (LFW), we have attained a 93% mean-recall on the value of K=20 for the LTSA-SVM. The mean-recall of the LTSA-KNN is 88% on K=30, 81% LTSA-NB on (K=40), and 68% LTSA-RF have the same values for all K in Table 14. Therefore, the mean-recall of the ISO-SVM is 89% on K=40, 80% ISO-KNN on (K=10), 80% ISO-NB on (K=40) 65% ISO-RF have the same values for all K in Table 14. We have shown the mean-recall of the LTSA algorithm’s comparative performance with classification techniques and Isomap in Fig. 13. However, LTSA performs better with SVM, KNN, and NB rather than the RF and Isomap.

Table 15 Mean-recall of breast cancer dataset with SVM, KNN, NB, and RF

Fig. 14

Mean-recall of breast cancer dataset with NRIC and isomap methods

For the Breast Cancer dataset, we have reached a 98% mean-recall value of K=40 for the LTSA-SVM, 94% LTSA-KNN on the same value of K=10, 94% LTSA-NB, and LTSA-RF on the same value of K=30, as shown in Table 15. In Table 15, therefore, the mean-recall of the ISO-SVM is 98% on (K=20), 88% ISO-KNN on (K=15), 89% ISO-NB on (K=20, 30), and ISO-RF is 91% on the value of K = 10, 30. The ISO-SVM performance is higher than LTSA-SVM on K= 20, and ISO-RF is higher than LTSA-RF on K=10. In Fig. 14, we have shown the mean-recall of the LTSA algorithm’s comparative performance with classification techniques and Isomap.

For the Digits dataset, we have reached a high mean-recall 97% on K=30 for the LTSA-SVM, as shown in Table 16. The mean-recall of the LTSA-KNN is attained 95% on K= (10, 20), 94% LTSA-NB, and 84% for LTSA-RF on the same value of K=10. Therefore, mean-recall of the ISO-SVM is attained 92% on K= (10, 15, 20, and 25), 93% ISO-KNN on K=10, and 90% for ISO-NB on (K=10, 15, 20, 25, and 40), and 80% ISO-RF on the value of (K=10, 20). In Fig. 15, we have shown the mean-recall of the LTSA algorithm’s comparative performance with classification techniques and Isomap. However, the performance of the LTSA is better with KNN, NB, and RF rather than SVM and Isomap. Moreover, the overall mean-recall performance of the NRIC method consistently attains the high results of the five high dimensional datasets. The mean-recall performance is better and down according to the different neighborhood values of K and classification techniques.

4.6 Area Under the ROC Curve (AUC)

Tables 17, 18, 19, 20, 21 shows the calculated area under the ROC (Receiving Operating Characteristics) curve (AUC) of Iris, Wine, LFW, Breast Cancer, and Digits Datasets from Eqs. (4) and (5). In Figs. 16, 17, 18, 19, 20, we have represented the calculated AUC values of five different high dimensional datasets. The graph of our proposed NRIC method consistently reaches the high AUC values of five datasets. We have used the different neighborhood values of K as same as accuracy, mean-precision, and mean-recall. The area’s performance under the ROC (Receiving Operating Characteristics) curve (AUC) of LTSA with classification techniques very well and provided effective results for the proposed NRIC method.

Table 16 Mean-recall of digits dataset with SVM, KNN, NB, and RF

Fig. 15

Mean-recall of digits dataset with NRIC and isomap methods

Table 17 AUC of Iris dataset with SVM, KNN, NB, and RF

Fig. 16

ROC curve of Iris dataset with NRIC and isomap methods

We have computed AUC values for the Iris dataset for different K (10, 15, 20, 25, and 40) values in Table 17. The AUC values of LT-SVM-A, LT-KNN-A, LT-NB-A, and LT-RF-A are 0.734, 0.736, 0.776, and 0.779, respectively, on the different values of K in Table 17. Therefore, the AUC values of IS-SVM-A (0.697), IS-KNN-A (0.693), IS-NB-A (0.761), and IS-RF-A (0.682) on the different values of K. According to Fig. 16, the performance of the ROC curve of LTSA with RF and NB is significant rather than SVM and KNN. In contrast, the high value of AUC is RF and NB. It significantly shows that RF and NB models are improved the performance of our proposed NRIC method. However, the ROC curve performance of the Isomap method with classification techniques is not better than our proposed method.

Table 18 AUC of wine dataset with SVM, KNN, NB, and RF

Fig. 17

ROC curve of wine dataset with NRIC and isomap methods

For the Wine dataset, the AUC values of LT-SVM-A, LT-KNN-A, LT-NB-A, and LT-RF-A are 0.655, 0.584, 0.596, and 0.518, respectively, on the different values of K in Table 18. According to Fig. 17, the ROC curve of LTSA with SVM is better than KNN, NB, and RF because the high value of AUC is SVM. It significantly shows that the SVM model is significantly improved the performance of our proposed NRIC method. Therefore, the AUC values of IS-SVM-A (0.540), IS-KNN-A (0.451), IS-NB-A (0.507), and IS-RF-A (0.500) on the different values of K. However, the ROC curve performances of the Isomap method with classification techniques are not better than our proposed NRIC method.

Table 19 AUC of LWF people dataset with SVM, KNN, NB, and RF

For the LWF People dataset, the AUC values of LT-SVM-A (0.574), LT-KNN-A (0.541), LT-NB-A (0.576), and LT-RF-A (0.737) on the different values of K in Table 19. In Fig. 18, the performance of the ROC curve of LTSA with RF is well rather than SVM, KNN, and NB. It significantly shows that the RF model is significantly improved the performance of our proposed NRIC method. Therefore, the AUC values of IS-SVM-A (0.554), IS-KNN-A (0.480), IS-NB-A (0.507), and IS-RF-A (0.580) on the different values of K. Moreover, the Isomap method works well with the RF model. However, the ROC curve performance of the Isomap method with classification techniques is not better than our proposed NRIC method.

Fig. 18

ROC curve of LWF people dataset with NRIC and isomap methods

For the Breast Cancer dataset, the AUC values of LT-SVM-A (0.744), LT-KNN-A (0.491), LT-NB-A (0.621), and LT-RF-A (0.521) on the different values of K in Table 20. In Fig. 19, the ROC curve of LTSA with SVM is better than KNN, NB, and RF. It significantly shows that the SVM model has improved our proposed NRIC method performance rather than other classification models. Therefore, the AUC values of IS-SVM-A (0.425, 0.547), IS-KNN-A (0.471), IS-NB-A (0.560), and IS-RF-A (0.626) on the different values of K. Moreover, the Isomap method works well with the RF model, but the IS-SVM-A value is higher than the LT-SVM-A on the value of K=15 in Table 20. However, the ROC curve performance of the Isomap method with classification techniques is not better than our proposed NRIC method.

Table 20 AUC of breast cancer dataset with SVM, KNN, NB, and RF

Table 21 AUC of digits dataset with SVM, KNN, NB, and RF

Fig. 19

ROC curve of breast cancer dataset with NRIC and isomap methods

Fig. 20

ROC curve of digits dataset with NRIC and isomap methods

For the Digits dataset, AUC values of LT-SVM-A (0.714 on K=40), LT-KNN-A (0.528 on K=20), LT-NB-A (0.528 on K=20), and LT-RF-A (0.530 on K=25 and 30) in Table 21. In Fig. 20, the ROC curve of LTSA with SVM is better than KNN, NB, and RF. It significantly shows that the SVM model has improved our proposed NRIC method’s performance rather than other classification models. Therefore, the AUC values of IS-SVM-A (0.582 on K=30), IS-KNN-A (0.484 on K=40), IS-NB-A (0.524, 0.528 on K=10 and 20), and IS-RF-A (0.447 on K=10). Moreover, the Isomap method works well with the SVM model, but the IS-RF-A value is higher than LT-RF-A on the value of K=10 in Table 21. However, the overall ROC curve performance of the Isomap method with classification techniques is not better than our proposed NRIC method.

5 Conclusion

This paper proposed a Noise Removal Isomap with a Classification (NRIC) method for the Isomap noise problem. This research paper aims to focus on the noisy nonlinear manifold learning method, such as Isomap. The main problem of the Isomap is sensitivity to noise and cannot easily generate the data after de-noising. Our proposed method can easily map the high dimensional data into low dimensional space. Our NRIC method can identify the noise from data points and eliminate the noise in datasets. In this paper, we compared four classification techniques, including SVM, KNN, NB, and RF, with the LTSA algorithm to improve the accuracy of the noise of Isomap and improve the accuracy of the datasets. We compared four classification techniques on five different datasets to gain insight into what technique is suitable for Isomap noise. We calculated accuracy, mean-precision, mean-recall, and area under the (ROC) curve (AUC) for the NRIC method. Our experiment result shows that our proposed method is much efficient than Isomap and provides highly accurate results of high dimensional datasets, and can easily optimize the graph. In future work, we will analyze the proposed method’s performance on other datasets and calculated the other classification metrics and time complexity analysis. We will be used the same idea of other manifold learning techniques.