1 Introduction

The Cloud Computing (CC) [1] features and advantages attracted the user gradually. The server stores massive amounts of sensitive user information, which are securely shared [2] and accessed by many users. However, the remote accessing and fast sharing of data increases malware attacks and threats in data at cloud servers [3,4,5]. Parallelly, it also attracted malware developers and cyber attackers. Malware is an intrusive software, including spyware, trojan horse, worm, adware, ransomware, virus, etc., that primarily aim to disturb the system. It is categorized into two classes - first-generation malware and second-generation malware. The first-generation malware handles the concept in which malware structure remains unchanged. The second-generation malware changes randomly after each infection and it is very much different from each other. Each malware transactions generate a novel structure in terms of results [6, 7]. The dynamic characteristics of the malware make it hard to quarantine and detect [8, 9]. The key technology for detecting malware is heuristic-based, signature-based, Machine Learning (ML), Deep Learning (DL) Multi-attribute decision-making [10, 11], and normalization [12].The Intrusion detection approach is one of the most promising technology used to detect malware in the cloud network. It is not only applied in the cloud network but also used in the spatially distributed sensors, and many other fields, which is an important research area investigated in [13,14,15,16,17,18,19]. Security analysts and researchers must continually enhance the malware detection system in which endpoint detection and protection are top priorities [20, 21]. Endpoint protection offers a set of security programs involving sandboxing firewalls, anti-spam, URL filtering, and email protection. Especially anti-malware software offers the final layer of defence. There are two kinds of analysis available, namely static and dynamic. The static analysis comprises inspecting an executable without implementation [22]. These two kinds of analysis have limitations and advantages and complement one another. Conventional malware analysis and detection cannot keep pace with variants and new attacks. Organizations are experiencing the serious problem of handling millions of attacks. Additionally, the organization faces a lack of cybersecurity talent and skills [23]. The recognized issue presents a great opportunity for ML to change and impact the cybersecurity landscape considerably. It is because of its capability to deal with the massive number of information [24, 25].

This paper presents an Optimal Encoder-Decoder Driven LSTM Networks for Malware Detection and Classification (OELSTM-MDC) technique. The presented OELSTM-MDC technique applies to pre-processing in the initial stage for data normalization. The Quantum Mayfly Optimization-based Feature Selection (QMFO-FS) technique is derived from selecting an optimal subset of features. Furthermore, the ELSTM classification model is applied to identify and classify malware. Lastly, the Butterfly Optimization Algorithm (BOA) enhances malware detection and classification performance. Moreover, Table 1 represents the acronyms used in the proposed malware detection and classification model.

Table 1 List of acronyms
Full size table

1.1 Motivation

Most of the research works have employed various ML and DL models for efficient and secure malware detection and classification. However, as per the literature [26,27,28,29] associated with the malware detection approaches, the researchers/authors have not tackled the security and privacy issues that can arise while performing malware detection. Due to this, any malicious attacker can forge a system. Thus, there is a need to design a secure and efficient malware detection and classification model with higher accuracy. To resolve the challenges mentioned earlier, we have proposed a quantum mayfly optimization with encoder-decoder-driven LSTM networks for malware detection and classification with higher efficiency and accuracy than the conventional approaches.

1.2 Contributions

The research contributions are summarized as follows:

  • We propose a quantum mayfly optimization with encoder-decoder-driven LSTM networks for malware detection and classification. It mainly consists of the QMFO-FS technique for the initial selection of features that can be used for classifying and identifying using the ELSTM technique.

  • Furthermore, the BOA algorithm is applied to strengthen the performance of malware detection and classification based on the malware and benign class.

  • Finally, the performance of the proposed system has been simulated with the applied OELSTM-DC model. The results yield an accuracy of 97.14% and 98.33% based on the malware and benign class in the training and testing dataset.

2 Related work

Most of the cybersecurity and malware detection solutions say AI-powered antimalware tools efficiently detect modern malware attacks. Research has projected different techniques and learning (ML and DL) technologies [30, 31] for malware detection. The ML technique can derive a classification from limited training instances. Thus, this technique prevents the need to determine signatures explicitly in emerging malware detectors. In previous years, the ML method has triggered a radical shift in several fields, including cyber-security. Over the decade, anti-malware communities and researchers have reported many ML and DL-based models to develop malware detection and analysis schemes.

Fournier et al. [32] implemented and designed an architecture for detecting malware on Android devices to protect financial and private data for the mobile application of the ATISCOM project. Then, they gradually enhanced the presented method for the recently installed application on an Android device. The researchers in [33] presented AdMat - an efficient architecture for characterizing Android applications by processing them as images. The innovation of the study lies in constructing an adjacent matrix for all the applications. This matrix acts as an “input image” to the CNN, allowing them to learn to differentiate between benign and malicious applications and malware families. Damaševičius et al. [34] presented an ensemble classification-based method for detecting malware. A CNN and stacked ensemble of dense can implement the classification.

In [35], a DL-based method is proposed to categorize malware variants according to a hybrid mechanism. The major objective is to present a hybrid structure that incorporates two extensive pre-trained network systems in an enhanced way. This structure comprises four major phases: training the proposed deep neural network architecture, data acquisition, evaluation of the trained deep neural network, and designing deep neural network architecture. The researchers in [36] proposed a malware detection technique based on a supervised ML algorithm. They performed a static analysis of the data extracted from the Drebin dataset. They provided a brief review of other studies in the field. Next, estimate six common classification methods under distinct configurations in terms of i) feature selection and ii) capacity to detect Android malware.

Marin et al. [37] examine the DL techniques on certain problems of classification and detection of malware. They considered raw measurement directly coming from the stream of monitored bytes as input to the presented method. A DL technique, DeepMAL can capture the fundamental statistics of malicious traffic without expert hand-crafted features. It estimates distinct raw-traffic feature representations, including flow-level and packet ones.

Later, Agarkar et al. [26] discussed a malware detection and classification model using machine learning to address the behaviour-based detection methods for malware detection. Then, Eboya et al. [27] investigated a malware detection framework for the IoT ecosystem. However, the authors in [27] did not consider the performance issues of detection. The researchers in [28] introduced a malware classification approach based on the VGG19 network. The authors in [29] discussed a client-server malware detection model utilizing machine learning for android applications. To improve the accuracy of the malware detection system in [38], the authors proposed a flood attacks-based protection model for complex networks.

Manickam et al. [39] presented DDoS attacks-based dataset based on Internet Control Message Protocol with higher detection accuracy and precision. The authors in [40] discussed an efficient method for fine-grained tasks in edge computing along with optimized energy usage. However, as mentioned earlier, the researchers need to focus on the security and privacy issues in the malware detection and classification approaches. Additionally, some of the research works did not consider the accuracy and precision parameters that decide the performance of a malware detection system. Therefore, to meet the mentioned challenges, we have proposed a preserved quantum mayfly optimization with encoder-decoder networks for malware detection and classification with higher accuracy and efficiency. Table 2 presents the comparative analysis of various state-of-the-art malware detection and classification approaches with the proposed system.

Table 2 Comparative analysis of state-of-the-art malware detection and classification approaches with the proposed system
Full size table

3 Proposed model

In this paper, a new OELSTM-MDC algorithm is introduced for the identification and classification of malware. The presented OELSTM-MDC technique undergoes a series of sub-processes: pre-processing, QMFO-based feature subset selection, ELSTM classification, and BOA-based hyperparameter optimization. The utilization of BOA helps to significantly enhance the overall malware detection performance of the ELSTM model. The entire block diagram of the OELSTM-MDC approach is shown in Fig. 1

Fig. 1
figure 1

Block diagram of OELSTM-MDC technique

Full size image

A dataset has been developed that further utilises a feature selection using the QMFO technique. Initially, the data is being collected considering the training and testing dataset, including malware and benign class. Furthermore, the training dataset is pre-processed to remove the missing and null values. Then, OELSTM-DC is applied to classify the training dataset based on the malware and benign class for malware detection. Further, parameter tuning is performed using BOA to enhance the performance of malware detection in terms of efficiency and accuracy.

3.1 Pre-processing

Androguard was a complete package tool infrastructure to interrelate with Android files and has restricted only to the python environment. It could be employed as a tool for reversing engineering single Android applications. Such classification could be vital to select features which require the class a new record is going to. The permission and API calls are removed from all Android applications and integrated as a limited feature in the data set. Thus, a data frame contains a feature (column) and application (row). Every column indicates the specific permission or API call with a binary value. However, rows validate the group of malware and benign APK files. Table 3 shows the used parameters and symbols in the proposed system.

Table 3 Proposed system parameters
Full size table

3.2 Process involved in QMFO-FS technique

The MO technique is presented by Zervoudakis and Tsafarakis [42]. They simulate the mating procedure demonstrated by mayflies (MFs) in nature. In MO technique work primarily by creating two arbitrary population sets demonstrating the female and male sets correspondingly. All the MFs placed from the problem space implies the potential solution to a problem. The place has been demonstrated as ddimensionalvector = (x1,x2,...,xn), and f(x) is the main function to evaluate the performance of all MFs. The MFs place alters their velocity v = (v1,v2,...,vn). However, the flying direction of all MFs is defined as the optimum individual flying experiences of all the MFs (pbest) and optimum swarm social flying experiences (gbest). As the male moved in a swarm and danced on some water meters, it could not move at maximum speed. Therefore, the velocity of male MF has been calculated with the help of Eq. 1.

$$ v_{ij}^{t+1} = v_{ij}^{t} + a_{2} e^{-\beta {r_{p}^{2}}} (pbest_{ij} - x_{ij}^{t}) + a_{1} e^{-\beta {r_{g}^{2}}} (gbest_{ij} - x_{ij}^{t}) $$
(1)

whereas \(v_{ij}^{t}\) refers the male MF velocity, \(x_{i_{j}}^{t}\) indicates the place, j implies the MF number, j = 1,...,n represents the space dimensional, t denotes the time step. However, a1 and a2 are constants executed for corresponding constants to scale the contribution of social and cognitive elements. Also, pbestj represents the optimum place stayed by MF i and N defines the count of male MFs. Lastly, β represents the visibility co-efficient that limits the visibility of MFs to other MFs, but rp and rg indicate the distances amongst xj and pbesti and gbest correspondingly. A novel place of the male is computed as adding the velocity \(v_{i}^{t+1}\) to the present place. It is represented by Eq. 2.

$$ x_{i}^{t+1} = {x_{i}^{t}} + v_{i}^{t+1} $$
(2)

An optimum MFs endure for executing its nuptial dance. So, the optimum MFs have altered their velocity based on the subsequent relation, represented by Eq. 3.

$$ v_{i}^{t+1} = v_{ij}^{t} + d * r $$
(3)

d refers to the co-efficient of nuptial dances and r denotes the arbitrary number between the range of − 1 and 1. These movements present a stochastic element to this technique. The velocity of females is computed with the help of Eq. 4.

$$ v_{i}^{t+1} = \begin{cases} v_{ij}^{t} + a_{3} e^{-\beta r_{mf}^{2}} (x_{ij}^{t} \vert -y_{ij}^{t}), & iff(y_{i}) > f(x_{i})\\ v_{ij}^{t} + fl * r, & iff (y_{i}) \leq f(x_{i}) \end{cases} $$
(4)

whereas \(v_{ij}^{t}\) indicates the female MF velocity, \(y_{ij}^{t}\) refers the place, i is MF number, j = 1,...,n indicates the space dimensional, t denotes the time step. In addition, a3 is a constant executed for scaling the contribution of social and cognitive elements. However, β represents the visibility co-efficient, but rmf refers to the distance between female and male MFs.

At last, fl represents the random walk co-efficient executed in case of attraction between a female and male failed, and r stands for the arbitrary number with − 1 and 1 range. A novel place of female MF was calculated as added velocity \(v_{i}^{t+1}\) to the present place. It is represented by Eq. 5.

$$ y_{i}^{t+1} = {y_{i}^{t}}+v_{i}^{t+1} $$
(5)

The mating procedure amongst MFs is executed with the crossover operator. As stated previously, fitness value has been utilized for selecting the parent to mate, and outcomes in two offspring are created with the help of Eqs. 6 and 7.

$$ offspring1 = L * male + (1-L) * female $$
(6)
$$ offspring2 = L * female + (1-L) * male $$
(7)

In these equations, male refers to the male parent, female indicates the female parent, and L stands for the arbitrary number in an existing range. A primary velocity offspring1 and offspring2 are considered that zero.

The QMFO algorithm has been developed by utilising Quantum Computing (QC) concepts to improve the outcomes of the MFO algorithm. It is a new type of computing model based on quantum theory, such as quantum entanglement, quantum measurement, and state superposition, which adapt the model. The core component of QC is qubit [43]. The two fundamental states |0 > and |1 > form a qubit, represented by Eq. 8 as a linear integration of both states.

$$ \vert Q> = \alpha \vert0 > + \beta\vert1 > $$
(8)

|α|2 denotes the probability of observing state |0 >, |β|2 indicator the probability of observing state |1 >, where |α2| + |β2| = 1. The Quantum is composed of n qubits. According to the nature of quantum superposition, every Quantum comprises 2n possible values. An n-qubits quantum is represented by Eq. 9.

$$ {\Psi} = \sum\limits_{x=0}^{2^{n}-1} C_{x} \vert x>, \sum\limits_{x=0}^{2^{n}-1} \vert C_{x} \vert^{2} = 1 $$
(9)

Quantum gate changes the state of qubits, namely NOT gate, rotation gate, Hadamard gate, etc. The rotation gate is determined as a mutation operator for improving the quanta method and finding the global optimum solution.

The rotation gate can be defined by Eqs. 10 and 11.

$$ \begin{bmatrix} \alpha^{2} (t+1)\\ \beta^{2} (t+1) \end{bmatrix} = \begin{bmatrix} \cos({\Delta}\theta^{d}) & -\sin({\Delta}\theta^{d})\\ \sin({\Delta}\theta^{d}) & \cos({\Delta}\theta^{d}) \end{bmatrix} \begin{bmatrix} \alpha^{d} (t) \\ \beta^{d} (t) \end{bmatrix}, d=1,2, ....,n $$
(10)
$$ {\Delta} \theta^{d} = {\Delta} \times S(\alpha^{d}, \beta^{d}) $$
(11)

Δ𝜃d indicates the rotation angle of the qubit, whereby Δ and S(αd,βd) denote the size and direction of rotation correspondingly.

3.3 Steps involved in ELSTM-based classification

Once the feature subsets are elected, the ELSTM model is utilized to classify the malware. Traditional Recurrent Neural Network (RNN) utilizes preceding context states to determine future states. Bidirectional RNN (BRNN) processes data in two directions with two different hidden states later propagated towards a similar output layer [44]. BRNN employs two RNNs to assist with backward and forward data regarding the sequence at each time step. BRNN calculates the output sequence y, the hidden forward sequence hf and the backward hidden sequence hb by iterating data from the backward layer t = T to t = 1. Next, data in the other networks are propagated from t = 1 to t = T for updating the output layer; once these two networks are integrated, data is propagated bi-directionally. A bi-directional LSTM network (BLSTM) was initially developed for word embedding in NLP for accessing long-range contexts or states in two directions. BLSTM network has been utilized in real-time sequence processing problems, including speech synthesis, phoneme classification, and continuous speech recognition. Figure 2 illustrates the LSTM network structure.

Fig. 2
figure 2

LSTM networks

Full size image

ELSTM is a sequence-to-sequence method for mapping a set length input to a set length output. Therefore, the input sequence of video frames (x1,...,xn), and the output can be the sequence X(t + 1)X(t + 2)X(t + 3)X(t + m) of words (y1,...,ym). Thus, evaluate the conditional possibility of output sequence (y1,...,ym), assumed sequence of input (x1,...,xn), that is p(y1,...,ym|x1,...,xn). In multi-step sequence predicting, the input and output are of parameter length. In the encoding stage, assuming a sequence of input, the ELSTM calculates a sequential hidden layer.

3.4 Optimal hyperparameter tuning using BOA

The BOA has been applied to modify the hyperparameter values in the ELSTM model. The presented BOA replicates the performance of butterflies (BFs) on mating and food source finding [45]. This technique employs two distinct navigation patterns for searching the field. During the exploration stage (r1 ≤ p), BF move towards the optimal BF of the colony, whereas from the exploitation stage (r1 > p), BF performs an arbitrary search inside the searching space by moving toward an arbitrary BF from the colony. Eq. 12, arithmetically expresses this two-searching pattern.

$$ \begin{cases} if r_{1} \leq p \ {^{t+1}}X_{i} = {^{t}}X_{i} ({r_{2}^{2}} \times g^{*} - {^{t}}X_{i}) \times \phi_{i} & Global \ Search\\ if r_{1} > p {^{t+1}} X_{i} = {^{t}}X_{i} ({r_{3}^{2}} \times {^{t}}X_{i} - {^{t}}X_{k}) \times \phi_{i} & Local \ Search \end{cases} $$
(12)

Whereas t and t + 1 indicate the present and upgraded state of the respective parameter. The position of the optimal BF in the colony is represented as g∗. \(t_{X_{j}}\) and \(t_{X_{k}}\) denotes the location of two randomly chosen BFs. r1, r2 and r3 indicates three arbitrary values within [0,1]. ϕi denotes the fragrance factor. It is determined by Eq. 13.

$$ \phi_{i} = cI^{a} $$
(13)

ϕi denotes the fragrance magnitude to ith BF; I and a indicate the intensity of the stimulus and the fluctuating absorption degree, and c represents a coefficient. I refers the related intensity to the main function value, and ith BF can be assumed as f(Xi), whereas f returns objective function value. The a and c coefficients are chosen from the range of [0,1]; p denotes the probability switch that defines the searching behaviour. The BOA approach constructs a fitness function to increase classification performance. It calculates a positive integer to indicate the candidate solutions’ improved performance. The best solution has the lowest error rate, whereas the worst option has a higher error rate. This work’s fitness function minimises the classification error rate, as shown in Eq. 14.

$$ \begin{array}{@{}rcl@{}} fitness(x_{i}) &=& ClassifierErrorRate(x_{i}) \\ &=& \frac{number \ of \ misclassified \ samples}{Total \ number \ of \ samples} \times 100 \end{array} $$
(14)

Figure 3 presents the sequence flow of the proposed system. It initiates with the pre-processing, feature selection and selection using different classification techniques. Now, it is further used for malware detection and classification with the OELSTM-DC model.

Fig. 3
figure 3

Sequence flow of the proposed system

Full size image

4 Experimental validation

In this section, malware detection and classification performance have been evaluated.

4.1 Dataset description

The dataset involves 2000 applications in which 1000 applications fall under the category of malicious class and another 1000 applications fall under the category of benign class. The considered dataset includes various features. We must select the relevant one based on the malware detection and classification requirement. The dataset’s relevant features include 75 features in which classification using the QMFO-FS technique requires 25 features for malware detection based on the malicious and benign classes.

4.2 Malware detection and classification using OELSTM-MDC technique

In this section, the malware detection and classification outcomes of the OELSTM-MDC model are tested using dataset [46, 47]. The dataset consists of 2000 applications divided equally into two classes (malicious and benign), with 1000 applications in each class. The dataset includes 75 features and the QMFO-FS technique has chosen a set of 25 features.

Figure 4 exhibits the confusion matrices generated by the OELSTM-MDC model on 70% of training and 30% of testing datasets. Figure 4a illustrates that the OELSTM-MDC model has effectually recognized 667 samples under the benign class and 693 samples under the malware class. In addition, Fig. 4b shows that the OELSTM-MDC technique has effectually recognized 299 samples under the benign class and 291 samples under the malware class. Table 4 shows the parameters used to implement the OELSTM-DC model considering the dataset for malware detection and classification.

Fig. 4
figure 4

Confusion of OELSTM-MDC technique under the training of 70% and testing of 30%

Full size image
Table 4 Model parameters
Full size table

Table 5 reports a brief malware classification outcome of the OELSTM-MDC model on training data of 70% and testing data of 30%.

Table 5 Result analysis of OELSTM-MDC technique on the training of 70% and testing of 30%
Full size table

Figure 5 offers detailed classifier outcomes of the OELSTM-MDC model on the training dataset. The OELSTM-MDC model has classified the samples under benign class with accuy, precn, recal, Fscore, and MCC of 97.14%, 98.38%, 95.83%, 97.09%, and 94.32% respectively. Moreover, the OELSTM-MDC technique has classified the samples under the Malware class with accuy, precn, recal, Fscore, and MCC of 97.14%, 95.98%, 98.44%, 97.19%, and 94.32% correspondingly.

Fig. 5
figure 5

Result analysis of OELSTM-MDC technique on the training of 70% dataset

Full size image

Figure 6 provides detailed classifier outcomes of the OELSTM-MDC model on the testing dataset. The OELSTM-MDC technique has classified the samples under benign class with accuy, precn, recal, Fscore, and MCC of 98.33%, 98.36%, 98.36%, 98.36%, and 96.67% respectively. Additionally, the OELSTM-MDC model has classified the samples under the Malware class with accuy, precn, recal, Fscore, and MCC of 98.33%, 98.31%, 98.31%, 98.31%, and 96.67% correspondingly.

Fig. 6
figure 6

Result analysis of OELSTM-MDC technique on testing of 30% dataset

Full size image

Figure 7 showcases the classifier results of the ODCNN-RFIC method on the test dataset. Figure 7a depicts that the ODCNN-RFIC technique has showcased effective precision-recall outcomes under the training of 70%. At the same time, Fig. 7b depicts that the ODCNN-RFIC technique has showcased effective precision-recall outcomes under testing of 30%. In addition, Fig. 7c illustrates that the ODCNN-RFIC technique has offered ROC, resulting in a maximum training performance of 70%. Also, Fig. 7d illustrates that the ODCNN-RFIC technique has offered ROC, resulting in maximum performance on testing of 30%.

Fig. 7
figure 7

a) Precision recall-training 70%, b) Precision recall-testing at 30%, c) ROC-training at 70%, d) ROC-testing at 30%

Full size image

Figure 8 depicts the overall accuracy of the OELSTM-MDC system’s results analysis on the test data. The results exhibited that the OELSTM-MDC approach has achieved improved validation accuracy compared to training accuracy. It is also worth noting that the accuracy values get saturated as the number of epochs increases.

Fig. 8
figure 8

Accuracy analysis of OELSTM-MDC technique

Full size image

Figure 9 shows the total loss outcome analysis of the OELSTM-MDC technique on the test data. The figure revealed that the OELSTM-MDC technique had denoted the reduced validation loss over the training loss. Furthermore, the loss values become saturated as the number of epochs increases.

Fig. 9
figure 9

Loss analysis of OELSTM-MDC technique

Full size image

A comparison study with existing models is made in Table 6 [48] to ensure the OELSTM-MDC model’s better outcomes.

Table 6 Comparative analysis of OELSTM-MDC algorithm with existing methods
Full size table

Figure 10 reports a comparative precn investigation of the OELSTM-MDC model with recent models. The results indicated that the GA-SVM and LR-MLP models had lowered precn values of 94.93% and 94.95%, respectively. The IG-Random Forest and RST-PSO algorithms have slightly increased precn values of 95.68% and 95.62%, respectively. Moreover, the CFS-Random Forest model has accomplished a reasonably precn of 96.96%. Though the RDT-XGBoost and E-LSTM models have exhibited considerable precn of 97.58% and 97.85%, the OELSTM-MDC model has depicted a maximum precn of 98.33%.

Fig. 10
figure 10

Precision analysis of OELSTM-MDC technique with recent algorithms

Full size image

Figure 11 defines a comparative recal examination of the OELSTM-MDC model with recent models. The results revealed that the GA-SVM and LR-MLP models have resulted in lower recal values of 94.37% and 95.76%, respectively. Similarly, the IG-Random Forest and RST-PSO algorithms have slightly increased recal values of 94.60% and 94.10%, respectively. Besides, the CFS-Random Forest approach has accomplished a reasonably recal of 96.28%. Lastly, the RDT-XGBoost and E-LSTM methodologies have exhibited considerable recal of 97.30% and 97.38%. The OELSTM-MDC technique has depicted a maximum recal of 98.33%.

Fig. 11
figure 11

Recall analysis of OELSTM-MDC technique with recent algorithms

Full size image

Figure 12 showcases a comparative Accy analysis of the OELSTM-MDC model with recent models. The results revealed that the GA-SVM and LR-MLP models have resulted in lower Accy values of 95.07% and 96.17%, respectively. At the same time, the IG-Random Forest and RST-PSO algorithms have slightly increased Accy values of 97.35% and 97.45%, respectively. Furthermore, the CFS-Random Forest model has accomplished reasonably Accy of 95%. Eventually, the RDT-XGBoost and E-LSTM methods have exhibited considerable Accy of 95.71% and 97.79%. The OELSTM-MDC model has depicted a maximum Accy of 98.33%.

Fig. 12
figure 12

Accuracy analysis of OELSTM-MDC technique with recent algorithms

Full size image

Figure 13 determines a comparative Fscore examination of the OELSTM-MDC model with recent models. The outcomes indicated that the GA-SVM and LR-MLP models have resulted in lower Fscore values of 97.99% and 96%, respectively. Also, the IG-Random Forest and RST-PSO algorithms have reached slightly increased Fscore values of 97.01% and 91.20% correspondingly. In addition, the CFS-Random Forest model has accomplished a reasonably Fscore of 95.86%. At last, the RDT-XGBoost and E-LSTM models have exhibited considerable Fscore of 94.16% and 98.09%. The OELSTM-MDC methodology has depicted a maximum Fscore of 98.33%.

Fig. 13
figure 13

F-score analysis of OELSTM-MDC technique with recent algorithms

Full size image

The simulation results and discussion show that the OELSTM-MDC model has produced the best results compared with the recent algorithms.

5 Conclusion

This paper established a new OELSTM-MDC algorithm to identify and classify malware. The presented OELSTM-MDC technique undergoes a series of sub-processes: pre-processing, QMFO-based feature subset selection, ELSTM classifier, and BOA-based hyperparameter optimisation. The utilization of BOA helps to significantly enhance the overall malware detection performance of the ELSTM technique. A wide-ranging experimental analysis is carried out on the benchmark dataset to examine the enhanced performance of the OELSTM-MDC approach. The comparative analysis reported the improved outcomes of the OELSTM-MDC model on existing techniques. Therefore, the OELSTM-MDC approach is utilized as a proficient approach for malware classification with an accuracy of 97.14% and 98.33% based on the benign and malware class category in the training and testing datasets. In future, hybrid DL models can be applied to boost the efficiency of the OELSTM-MDC technique in a dynamic environment. Furthermore, various DL models can be explored and implemented to improve the overall performance of malware detection and classification.