1 Introduction

In modern life, Email is the best medium for formal communication. Moreover, Email is the easiest way to communicate. Generally, there are two types of emails, those are ham or legitimate email and spam email. An Email contains two parts, those are email body and email header. But, at present time Email has been misused in the name of “Spam”. Spam is also a kind of bulk or unsolicited email that contains an advertisement, phishing website link, malware, Trojan, etc. we all used to receive a lot of emails in a day; out of which, 70%–80% of emails are spam. Spam emails are used to send by spammers with many intentions like hacking, phishing, banking fraud, etc. Social media is the best medium for Spammers for getting the personal data of the user by sending spam emails. Spam is also used to know as “junk emails”. Spam emails are used for content advertisement, offers, phishing website links, anonymous virus-like malware, trojan, etc. “SPAM” is derived as Self Propelled Advertising Material [97]. In 2019, worldwide more than 280 billion spam emails are been sent and received. According to Google, 64% of emails sent and received in 2019 are spam emails and, this rate used to increase every year by 2%–3%. There are two types of spam detection techniques are there. These are sender based spam detection and content-based spam detection [33]. Sender based spam detection mainly happened based on features like Content-Type, Message-ID, MIME-Version, Authentication-Results, and Return-Path [12]. In content-based spam filtering, it checks the text of an email’s message as well as checks the URL of the email with the subject of the email for text classification [86]. In this research work, content-based spam detection has been done. There are three types of spam filters are there, for filtering spam emails, those are Blacklist Filter, Whitelist Filter, and Content-based Filter [6].

The term Machine Learning (ML) defines that the machine learns the characteristics cum behaviour from experience; it’s an application of Artificial Intelligence (AI). Machine Learning is generally classified into three types; those are Supervised Learning, Unsupervised learning and, Reinforcement Learning. There are various machine learning classifiers are there based on a particular algorithm. In this research work, thirteen machine learning classifiers have been implemented. These are Adaptive Boosting, Artificial Neural Network, Bootstrap Aggregating, Decision Table, Decision Tree, J48, K-Nearest Neighbor, Linear Regression, Logistic Regression, Naïve Bayes, Random Forest, Sequential Minimal Optimization and, Support Vector Machine for detecting spam emails from two datasets. Those two data sets are Spam Corpus (http://lpis.csd.auth.gr/mlkd/spam_corpus2.rar), and Spambase (https://archive.ics.uci.edu/ml/datasets/Spambase). Waikato Environment for Knowledge Analysis [30] is open-source software for performing the task of data mining operations like Pre-processing, Classification, Clustering, etc. It was invented by The University of Waikato, Hamilton, New Zealand in 1999. In this research work, WEKA 3.9.4 version has been used.

This research paper is organized as follows. The second section will contain the related work or existing research work on spam email detection using machine learning algorithms. The third section will present all the implemented machine learning classifiers. The fourth section will present the datasets. The fifth section will be delivered the spam detection approach. The sixth section will derive experimental analysis. The seventh section will conclude this research work with a future aspect.

2 Related work

This section will derive the existing research on spam email detection using machine learning classifiers. This survey focused on all those classifiers that were used in past for spam detection. This survey focused on classifiers that are used and which one has the best accuracy. In Table 1 details of the survey have been given below.

Table 1 Literature review of spam detection using machine learning classifiers
Full size table

3 Classifiers

3.1 Adaptive booster

Adaptive Booster (AdaBoost) is a boosting algorithm and a well-known ensemble method in Machine Learning. The term “ensemble” method defines that the classifier takes the results of multiple classifiers together and mixes them for better accurate results. AdaBoost was invented by Yoav Freund and Robert Schapire. AdaBoost is a machine learning classifier used for finding targets by training different classifiers for the same training set to make a powerful classifier [87]. In other words, AdaBoost merges a branch of weak learners to make a single learner that is stronger than a particular learner [32]. AdaBoost is used for object detection in classification. AdaBoost is the first boosting algorithm and also a meta-algorithm from the perspective of machine learning. Generally, AdaBoost can be implemented in three ways; those are using Haar-like features, introducing Local Binary Pattern (LBP), and implementing Field Programmable Gate Arrays (FPGA) [44]. AdaBoost has two special features in terms of classification; those are interpolation and generalization of errors respectively in the process of classification. The AdaBoost classifier is briefly explained in [90].

3.2 Artificial neural network

An Artificial Neural Network is known for information processing which has been the collection of interconnected neurons and is inspired by the human nervous system [35, 92]. The structure of an ANN consists of one input, one output and one or more hidden layers in between [114]. It’s difficult to predict how many numbers of hidden layers and hidden neurons are present in between the input and output layers of an ANN. In terms of a large number of classification, clustering, regression, pattern recognition and prediction, Artificial Neural Network is very successful in many disciplines [2, 68]. In an ANN, given input is used to go next layer and every layer has connections. And, each connection has associated weight. Input values are used to get multiplied with associated weights of connection and summarised to form a new input for the neuron. And, the new input achieves through an activation function. Each neuron of a network used to have a nonlinear activation function [124]. There are seven types of ANN; those are Modular neural network, Recurrent neural network, Generative neural network, Deep neural network, Spiking neural network, Feed-forwarded neural network and Physical neural network [18]. Perceptron is the most known architecture of neural networks [107].n

$$ y(x)=g\left({\sum}_{i=0}^n\left({w}_i{x}_i\right)\right) $$
(1)

In Eq. 1, xi is the ith neuron in the previous layer; wi is the weight deciding parameter that would decide the weight of the neuron; g is the activation function.

3.3 Bootstrap aggregating

Bagging is a powerful widely used ensemble classifier in Machine Learning (ML) and is known as a meta-estimator in terms of classifying datasets. Bagging is derived as “Bootstrap Aggregation”. Bagging makes bootstrap data sets for replacing the actual dataset [39]. Bagging is used to reduce the variance where the dataset contains high variance. In the case of spam email classification, bagging is used for counting the spam functional words in terms of occurrences for training the dataset [19]. Bagging performs n number of classifications based on bootstrap sampling data of training dataset. Lastly, conquers all the results in one as a final prediction. The bagging classifier is fully explained as well as elaborated in [15].

3.4 Decision table

In our research work, Decision Table is the fourth classifier used for detecting spam emails. In terms of numerical prediction, Decision Table is the best accurate classifier for forming decision trees [46]. The decision Table represents the visual model of the classification process in terms of tables with attributes of actual data. Decision Tables can take different decisions or actions based on a set of conditions [119]. Decision Table can easily maintain the data with different versions cum order in terms of classification. The structure of the Decision Table looks like a relational table, in that table, each row contains aggregate, combinations of values, attributes, etc. [8]. Decision Table is popular in terms of classifiers because it’s easy to understand from an overview itself. The decision Table contains two sub-classifier and those are DTMaj (Decision Table Majority) and DTLoc (Decision Table Local) [50]. Decision Table Majority has two components; those are schema and body [49]. Decision Table used to hold more data than the top, mainly following the tree kind of structure. Data of the dataset has been divided and constructed decision table by the decision table classifier [48]. From those various decision tables based on attributes, a decision tree classifier is used to make decisions.

3.5 Decision tree

A Decision Tree is a supervised machine learning technique used for classification and regression. A Decision Tree can be formed using a set of instances through the divide-and-conquer paradigm [84]. A Decision Tree is a supervised tree where internal nodes are testing nodes and leaf nodes are decision nodes [31, 70, 75]. Algorithms like Classification and Regression Tree (CART), Iterative Dichotomiser (ID3), and Chi-Squared Automatic Interaction Detector (CHAID) are useful for creating decision trees. The Decision Tree is useful for clearness and understandability [121]. The decision tree performs the task of finding which attributes will select from each level. Without changing in core logic, a decision tree can scale easily from linear data to non-linear data [43]. A Decision Tree is a graphical representation of all possible solutions to a problem based on given conditions. A Decision Tree is also specified as a hierarchical classifier because it wants multi-level prejudice to decide which class a specific pattern belongs to [117].

3.6 J48

J48 algorithm was invented and developed by Ross Quinlan, and it’s also known as the C4.5 algorithm. The C4.5 algorithm was earlier known as the ID3 algorithm. Moreover, the C4.5 algorithm is an extension of the ID3 algorithm. The ID3 algorithm was also invented by Ross Quinlan. J48 usually form the decision tree by the attributes of the training set [72]. J48 algorithm is used to construct a decision tree for classification. The decision tree of J48 looks like a graph that contains a branching method to show every possible outcome of the decision [65]. By seeing the decision tree of J48, we can predict the approximate outcome of classification as well as it helped to understand the classification. J48 have an advantage like finding missing values, pruning of decision tree, ranges of a continuous attribute value, and rules of derivation [85]. The outcome of the J48 classifier is the combination of multiple decision trees; J48 produces the result by conquering the results of those decision trees. The output of the classification for J48 is always used to present as a binary tree [95]. At the time of classification, the J48 classifier is used to generate a decision tree based on training data, and that’s the best part of the J48 classifier for understanding the classifier for anyone. J48 is the better algorithm compared to several other algorithms for classifying spam emails [3]. J48 algorithm can be implemented on devices for classification as well as useful for detecting diseases also. J48 is the new version of the C4.5 algorithm [122]. Compare to other popular machine learning algorithms like Naïve Bayes (NB), and Support Vector Machine (SVM), J48 always performs better in terms of performance measurement parameters in the context of classification. J48 is the binary tree for classifying [103].

3.7 K-nearest neighbor

K-Nearest Neighbor is a supervised machine learning algorithm, basically used for resolving classification problems. KNN is a k-related algorithm because its classification accuracy depends on the value of k [42]. KNN is used to calculate the distance between the classification point and sample data, then sorted closest k points and, lastly allocates the largest k points as points to be classified. Calculating the distance between training and testing sets using Euclidean distance and Mahalanobis distance method is a general task of a KNN [25]. KNN algorithm is effective in pattern recognition. KNN allocates a test sample to the class which has been voted by k-nearest neighbors in training data [26, 34, 52]. KNN algorithm can’t predict fundamental data and due to that reason, it is called a non-parametric algorithm. The conventional k-nearest neighbor algorithm presumes that the training samples are steadily assigned among various classes [99]. KNN can’t learn from the training set straight away, but it keeps the dataset and executes the dataset at the time of classification. For that reason, the k-nearest neighbor is known as one of the lazy learning techniques [67]. The advantages of the k-nearest neighbor algorithm are simple, easy to implement, and low error rate [60].

3.8 Linear regression

The term linear regression defines the statistical model that shows the relation between a dependent and an independent variable represented in the form of a line equation. Linear regression is a supervised learning algorithm that predicts a certain sample is under the slope or outside of the slope by drawing a lined margin between the samples. It is used to draw the line based on the value of independent and dependent variables. And, by that slope or line, it’s used to classify the samples in terms of their values. The equation of linear regression is easy to understand and, it is used for compromising the capacity between volumetric VAT and anthropometric parameters [57]. There are two types of linear regression, those are simple regression and multiple regression. Multiple regression is a complex kind of linear equation whereas, simple regression is the simpler equation to understand. In the context of numerical prediction, multiple linear regression is easy to implement as well as used in statistical applications [73]. Linear regression is used to deal with complex problems compare to other machine learning algorithms. In linear regression, variable significance is the important element [10]. In simple regression, independent and dependent variables always tried to create something like a correlation but, an exact correlation between those variables was never possible. In simple regression, the linear regression model is trained by all available training data [104].

$$ Y=a+ bX $$
(2)

In Eq. 2, X is a dependent variable, Y is an independent variable, b is the slope of the line and a is the intercept.

3.9 Logistic regression

Logistic regression is a statistical model for prediction and a similar kind of classification technique to linear regression. Logistic regression can be used to model the probability of the sample as true/false. Except that true/false, based on the event or class is used to change like pass/fail, slim/healthy, win/lose etc. Logistic regression is never used to calculate the exact value of the sample; it can only predict whether the sample value is true or false. Moreover, logistic regression and linear regression are similar except for the process of classifying the samples. In logistic regression, data points are used to arrange according to the sigmoid function. Logistic regression is a technique that is used for building a model by using multiple meteorological variables to predict whether precipitation will occur [63]. Generally, logistic regression is used to implement such a scenario where output used to come in binary (0 or 1). Logistic regression is an important model to perform the prediction in a large dataset where important features are selected based on the properties of attributes [118]. Logistic regression comes under the type of regression analysis and is used in a larger dataset where only two types of samples are there like spam filtering (ham or spam). Logistic regression is used worldwide as a classification algorithm [47]. In general, there are three types of logistic regressions; those are binary logistic regression, multinomial logistic regression and, ordinal logistic regression. Logistic regression is popular in statistical learning and machine learning for classifying datasets cum data [59]. Previously, logistic regression was mainly used to solve the binary classification problem. The output of logistic regression is a text segment that is offensive or non-offensive [77]. Logistic regression also comes under the supervised machine learning algorithm category; it is used in regression, multi-classification and, binary classification.

$$ \frac{1}{\left(1+{\mathrm{e}}^{\mathrm{value}}\right)} $$
(3)

This is known as the sigmoid function, and it has been developed by the statistician for the properties of the event or class. In Eq. 3, e is the base of the natural logarithm, and value is the numerical value.

3.10 Naïve Bayes

In the eighteenth century, English mathematician Thomas Bayes discover the ‘Bayes’ theorem. Based on Bayes Theorem, the Naive Bayes classifier was built that is used for computing the unknown classes [45]. Bayes theorem focused on the probability of two events and their conditional probability. The Naive Bayes classifier’s assumption is based on class conditional independence [74]. Naive Bayes is a probabilistic supervised machine learning algorithm that calculates a set of probability on given data set based on counting the frequency and the combination of values. A Naive Bayes classifier is used to utilize the word counts in the Bag of Words (BoW) feature extraction for text classification as well as for having the advantage over classification accuracy [33, 83]. Naive Bayes is a simple and easy algorithm to implement compared to other machine learning algorithms. Except for Support Vector Machine and ID3, the Naive Bayes classifier provides faster results and better accuracy [94]. Except for spam classification, Naive Bayes can be used in sentiment analysis, text classification, cyber-attack detection, real-time prediction, multi-class classification, document classification, natural language processing, etc. Naïve Bayes is an easy model to build and, it’s useful for working with large data sets [61]. In general, there are three types of Naive Bayes models; those are Gaussian, Multinomial, and Bernoulli. In terms of classification, the Bayesian classifier has a similar kind of ability to a decision tree and neural network for classifying spam emails [28]. Naive Bayes is the best classifier for classifying text; moreover, text classification has the best accuracy using a Naive Bayes classifier. Naive Bayes is used to training a probability model, and it will give each word a probability of being a suspicious spam keyword for classifying email [106]. The Naive Bayes classifier assumes that each feature has an independent and equal contribution to the outcome. So, the Naive Bayes classifier can’t learn the relation between features and it’s a disadvantage of the Naive Bayes classifier. In spam filtering, Naive Bayes, Decision Trees, and Support Vector Machine use the vector space method for classical text categorizing [89].

$$ \mathrm{P}\left(\mathrm{A}\left|\mathrm{B}\right.\right)=\frac{\mathrm{P}\left(\mathrm{B}\left|\mathrm{A}\right.\right)\mathrm{P}\left(\mathrm{A}\right)}{\mathrm{P}\left(\mathrm{B}\right)} $$
(4)

Bayes’ Theorem states that the variable y and dependent feature vector x1,….,xn through that,

$$ \mathrm{P}\left(\mathrm{y}\left|{\mathrm{x}}_1\right.,\dots, {\mathrm{x}}_{\mathrm{n}}\right)=\frac{\mathrm{P}\left(\mathrm{y}\right)\mathrm{P}\left({\mathrm{x}}_1,\dots, {\mathrm{x}}_{\mathrm{n}}\left|\mathrm{y}\right.\right)}{\mathrm{P}\left({\mathrm{x}}_1,\dots, {\mathrm{x}}_{\mathrm{n}}\right)} $$
(5)

3.11 Random Forest

Random Forest is the most powerful supervised ensemble tree-based machine learning algorithm that has been used for classification and regression. As the name suggests, the random forest has been consists of many decision trees [41], which are responsible for information retrieval. Each decision tree in a random forest has a result; those results are used to get votes from the decision trees, and the decision tree with the most votes is used to be the outcome of the random forest. Having many decision trees in the random forest makes the random forest algorithm high robustness and for that random forest has high accuracy [20] compare to other machine learning algorithms. Breiman has been proposed the Random Forest algorithm [16] for improving the Bagging algorithm. The random forest algorithm is good in classification, but regression can’t meet the expectation, so random forest is not good for regression tasks. Features of random forest are commendable for the reason it is used worldwide [29]. Random forest algorithm used in prediction, banking sector, stock market, medical science, pattern recognition, etc. Artificial intelligence algorithms are mainly used to solve the problem of classification and regression [11]. The random subspace method and bagging algorithm are combined to create the random forest algorithm. Random features are used to select from the input set by the tree classifier [88]. Random forest solves the problem of over-fitting and it has the scalability and parallelism that’s help to classify large datasets with higher dimensions. Decision tree and random forest work in the same way but, there is a difference between these two is random forest uses an ensemble learning approach [27]. The random forest comes under the category of Classification And Regression Tree (CART). It uses the tree voting method for bootstrapped data and preparing instructional data [64]. The random forest has high robustness and due to its robustness, the random forest can classify or be suitable to perform classification in high dimensional large data sets. The Random Forest algorithm combines multiple decision trees for upgrading the performance [69]. In the random forest algorithm, all the decision trees are used to train with the bagging algorithm. Random Forest has been used so much because it’s easy to implement [76] and for its diversity. Random forest uses the random feature selection method as a dimensionality reduction technique for feature selection. Neural networks and random forests had some similar characteristics [116]. Random forest was introduced in data mining by Ho in 1995 in the name of the random subspace method. The random forest can classify high-quality results without any hyper-parameter tuning [123]. Compare to other machine learning algorithms, random forest selects the features easily and is used to make a good model for predicting by dimensionality reduction technique, which is the reason behind random forest having good accuracy in classification. A random forest is the combination of multiple decision trees, but none of the decision trees is related to each other [56]. In random forest for solving regression problems, it is used to calculate Mean Square Error for organizing data as a node.

$$ \mathrm{MSE}=\frac{1}{\mathrm{N}}\sum \limits_{\mathrm{i}=1}^{\mathrm{N}}{\left({\mathrm{f}}_{\mathrm{i}}-{\mathrm{y}}_{\mathrm{i}}\right)}^2 $$
(6)

In Eq. 6, N is the number of data points, fi is the model returned value, and yi is the value of the data point.

3.12 Sequential minimal optimization

In 1998, John Platt [81] developed the Sequential Minimal Optimization algorithm. For training Support Vector Machine (SVM), an algorithm is needed which can solve the QP programming of SVM. And, from that aspect, only Sequential Minimal Optimization came into the scenario. Sequential Minimal Optimization (SMO) is a fast method to train SVM as well as state-of-the-art solutions for SVM training [55, 66]. The sequential Minimal Optimization algorithm can be divided into two parts; those are the analytical method and the heuristic method. Analytical methods are used to solve the QP problem. The heuristic method is mainly used to recognize the violating pair. Without QP optimization, SMO can solve the SVM QP problem [9, 80, 82]. Sequential Minimal Optimization algorithm is used to solve the problem rise from Support Vector Machine.SMO is used to decrease the calculation period and acquire refined scaling distinctive compared to the SVM training process [24]. SMO solve the QP problem by dividing it into sub-part due to its complexity and then used to solve sub-parts of QP optimization. SMO is a decay method and utilizes the smallest possible working set which can be upgraded successfully [40].SMO is mainly used so much due to its optimization technique. SMO subsequently collect a pair of training samples for join improvisation, which reduces the usage of memory [53].SMO uses an analytical method for avoiding complex iteration processes. SMO can be used as a decay method for training large data sets [93].SMO algorithm depended on heuristics for choosing the variables for optimizing an objective function. This concept helps the algorithm for performing on a large data set. Sequential Minimal Optimization and Support Vector Machine combine perspectives to reduce memory storage, easy to execute with high accuracy [108]. SMO can handle a large training set because SMO consumes the memory for the training set is linear. SMO needs only linear memory for the training set because it didn’t compute a large matrix. SMO has mainly been used to reform only two variables for every co-set [58]. SMO can solve the SVM’s QP optimizing problem without any extra matrix storage. SMO derive the whole QP problem into the QP sub-problem and is then used to solve those small QP sub-problems. Lastly, it uses to combine the result of each QP sub-problems into one like the divide and conquer technique. At every stage of QP optimisation, SMO is used to solve the smallest possible optimization problem.

3.13 Support vector machines

In 1992, Boser, Guyon and Vapnik [13] develop Support Vector Machines based on statistical learning as a supervised machine learning algorithm for performing classification and regression tasks [101]. SVM performs the job of separating two classes based on a hyper-plane. First time SVM has been implemented by Vapnik for solving a quadratic optimization problem [112]. Quadratic programming is used to solve mathematical optimization problems presuming quadratic functions. SVM can be used as developing quadratic problems for training the data set [17]. SVM is used to create a decision boundary for putting new data points in the correct category. In SVM, the best decision boundary is used to call hyper-plane. Generally, SVM processed a set of input data (xi) and predicts (yi) and builds a hyper-plane (H) for separating those classes using a hypothesis space for linear function in high dimensional feature space [5, 21, 23, 54]. In SVM, extreme points or vectors are used to create the hyper-plane. And, those extreme points or vectors are used to call support vectors. SVM is always used to maximize the margin between two classes with the help of support vectors [100, 113]. In SVM, the distance between hyper-plane and vectors is known as margin. And, which hyper-plane has maximum margin known as optimal hyper-plane. The primary idea of SVM is to discover an optimal hyper-plane that categorizes different types of samples [120]. Dimensions of hyper-plane used to rely on features of the dataset. Binary classifications are planned by the standard support vector machines and SVM used a linear separating hyper-plane for binary classification [78, 110]. Generally, SVM has been developed for binary classification. In binary classification, basic support vector machines classifier can work such a way that the kernel function can be pointed out in the input as a high dimensional feature space [22]. SVM is used for text classification, face detection, pattern recognition, hand-written character recognition, etc. SVM has been built for solving the big margin classification problem and, also worked as a statistical learning method based on VC dimensional theory [115]. There are three stages in SVM analysis and those are feature selection, training and testing the classifier, and performance evaluation. SVM is used widely because of its classification accuracy and robustness [14]. SVM performs better with a limited number of samples. There are two types of SVM; those are linear and non-linear SVM [102]. When a dataset is used to get classified into two classes using a straight line, known as linear SVM and this type of SVM is useful for linearly separable data. In linear SVM, problems ranged in their complexity depending on the number of features used [79]. When a dataset can’t be classified using a single straight line, known as non-linear SVM and this type of SVM is useful for non-linear data. SVM can process complex data with high accuracy [38]. In SVM, removing one or more support vectors can change the position of the hyper-plane. SVM mainly stands on the idea of structural minimization, which has been concluded by the generalization error that is bounded based on the sum of the training set and a term depending on the Vapnik-Chervonenkis dimension [111]. Generally, SVM consumes more time compared to other machine learning algorithms for training the model for large data sets. The challenge for the SVM tree classifier is how it separates the classes into two separate subsets for the training algorithm [96].

4 Datasets

In terms of executing the classifier, there are two spam datasets used. Those are Spam Corpus and Spambase. The Spam Corpus data set contains 9324 emails; out of which 2387 emails are Spam emails and the rest of 6937 emails are Ham emails. In terms of percentage, 25.60% of emails are Spam and the rest of the 74.40% of emails are Ham. Besides that, the spam corpus dataset contains 500 features or attributes. Spambase data set contains overall 4601 emails, out of the total number of 2788 emails are Spam and the rest of 1813 emails are Ham. In terms of percentage, 60.59% of emails are spam and the rest of the 39.40% of emails are ham. Except that, the Spambase dataset contains 58 features or attributes. In the following, some of the important features are described (Table 2).

Table 2 Useful features of the datasets
Full size table

5 Detection approach

This section proposed the detection framework for detecting spam emails using thirteen machine learning classifiers; those are briefly discussed in the previous section. In this research work, two datasets have been used and, both the datasets are having two types of instances. Of which some instances are spam emails and the rest are ham emails instances. Moreover, both datasets are labelled as well as sufficient numbers of instances are also there.

In Fig. 1, the spam email detection framework has been illustrated. In the detection framework, the first step is pre-processing of the dataset. In the pre-processing step, the dataset has been analyzed in terms of attributes and instances. In that analysis, several attributes and instances have been discovered. After that, pre-processed dataset gets ready for performing classification. In the next step, a classifier has been implemented for performing classification tasks. In this step, thirteen different machine learning classifiers have been used one at a time. Separately all the classifiers are executed for both datasets in this step. After performing classification, the dataset has been classified into two separate categories, those are Spam emails and Ham emails.

Fig. 1
figure 1

Illustration of spam emails detecting framework

Full size image

Multiple datasets have been used in this research work because of measuring the performance of thirteen machine learning classifiers for detecting spam emails in terms of different sizes of the dataset. In the experimental analysis section, the outcome of thirteen machine learning classifiers has been compared in terms of performance evaluation parameters.

6 Experimental analysis

6.1 Accuracy

Accuracy is the parameter for measuring the percentage of instances classified correctly.

$$ \mathrm{Accuracy}=\frac{\mathrm{Total}\ \mathrm{number}\ \mathrm{of}\ \mathrm{emails}\ \mathrm{classified}\ \mathrm{correctly}}{\mathrm{Total}\ \mathrm{number}\ \mathrm{of}\ \mathrm{emails}\ \mathrm{in}\ \mathrm{the}\ \mathrm{dataset}} $$
(7)
$$ \mathrm{Accuracy}=\frac{\mathrm{True}\ \mathrm{Positive}+\mathrm{True}\ \mathrm{Negative}}{\mathrm{True}\ \mathrm{Positive}+\mathrm{True}\ \mathrm{Negative}+\mathrm{False}\ \mathrm{Positive}+\mathrm{False}\ \mathrm{Negative}} $$
(8)

In Fig. 2, the performance of accuracy for all the thirteen machine learning classifiers has been described. In terms of Accuracy, the Random Forest classifier performs better compared to the rest of the classifiers. Random Forest classifier has an accuracy of 99.91% for detecting spam emails from the Spam Corpus dataset. Out of 9324 instances, the Random Forest classifier detects 9316 instances correctly. Except for the Random Forest classifier, Decision Table and Logistic Regression classifiers are also perform moderately with 99.89% of accuracy for the Spam Corpus dataset. The Naïve Bayes classifier has the least accuracy compared to the rest of the machine learning classifiers. The Naïve Bayes classifier performs with 87.63% of accuracy for the Spam Corpus dataset. Out of 9324 instances, the Naïve Bayes classifier detects only 8171 instances correctly.

Fig. 2
figure 2

Accuracy of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 3, the performance of accuracy for all the thirteen machine learning classifiers has been described. In terms of Accuracy, the Random Forest classifier performs better compared to the rest of the classifiers. Random Forest classifier has an accuracy of 99.93% for detecting spam emails from the Spambase dataset. Out of 4601 instances, the Random Forest classifier detects 4598 instances correctly. Except for the Random Forest classifier, J48 and Bootstrap Aggregating classifiers are also performed moderately with 97.17% and 96.72% of accuracy respectively for the Spam Corpus dataset. The Naïve Bayes classifier has the least accuracy compared to the rest of the machine learning classifiers. The Naïve Bayes classifier performs with 79.53% of accuracy for the Spambase dataset. Out of 4601 instances, the Naïve Bayes classifier detects 3659 instances correctly.

Fig. 3
figure 3

Accuracy of machine learning classifiers for Spambase dataset

Full size image

6.2 Precision

Precision defines as the percentage of correct spam emails classified from the dataset. Precision also is known as Specificity (true negative rate).

$$ \mathrm{Precision}=\frac{\mathrm{True}\ \mathrm{Positive}}{\mathrm{True}\ \mathrm{Positive}+\mathrm{False}\ \mathrm{Positive}} $$
(9)

In Fig. 4, the performance of precision for all the thirteen machine learning classifiers has been described. In terms of precision, Decision Table, Logistic Regression and, Random Forest classifier perform with 0.999 precision for detecting spam emails from the Spam Corpus dataset. The Naïve Bayes classifier has the least precision compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.874 of precision for the Spam Corpus dataset.

Fig. 4
figure 4

Precision of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 5, the performance of precision for all the thirteen machine learning classifiers has been described. In terms of precision, the Random Forest classifier performs with 0.999 precision for detecting spam emails from the Spambase dataset. Except for the Random Forest classifier, J48 and Bootstrap Aggregating classifiers are also performed moderately with 0.972 and 0.967 of precision respectively for the Spambase dataset. The Naïve Bayes classifier has the least precision compared to the rest of the machine learning classifiers. The Naïve Bayes classifier performs with 0.845 precision for the Spambase dataset.

Fig. 5
figure 5

Precision of machine learning classifiers for Spambase dataset

Full size image

6.3 Recall

The recall is the parameter for calculating the percentage of spam emails blocked. The recall is also known as Sensitivity (true positive rate or probability of detection).

$$ \mathrm{Recall}=\frac{\mathrm{True}\ \mathrm{Positive}}{\mathrm{True}\ \mathrm{Positive}+\mathrm{False}\ \mathrm{Positive}} $$
(10)

In Fig. 6, the performance of recall for all the thirteen machine learning classifiers has been described. In terms of recall, Decision Table, Logistic Regression and, Random Forest classifier performs with 0.999 of recall for detecting spam emails from the Spam Corpus dataset. The Naïve Bayes classifier has the least recall compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.876 of precision for the Spam Corpus dataset.

Fig. 6
figure 6

Recall of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 7, the performance of recall for all the thirteen machine learning classifiers has been described. In terms of recall, the Random Forest classifier performs with 0.999 of recall for detecting spam emails from the Spambase dataset. Except for the Random Forest classifier, J48 and Bootstrap Aggregating classifiers are also performed moderately with 0.972 and 0.967 of recall respectively for the Spambase dataset. The naïve Bayes classifier has the least recall compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.795 of recall for Spambase dataset.

Fig. 7
figure 7

Recall of machine learning classifiers for Spambase dataset

Full size image

6.4 F-measure

The F-measure defines the average weight-age of Precision and Recall.

$$ \mathrm{F}-\mathrm{measure}=\frac{2\times \mathrm{Recall}\times \mathrm{Precision}}{\mathrm{Recall}+\mathrm{Precision}} $$
(11)

In Fig. 8, the performance of the f-measure for all the thirteen machine learning classifiers has been described. In terms of the f-measure, Decision Table, Logistic Regression and, Random Forest classifier performs with 0.999 of the f-measure for detecting spam emails from the Spam Corpus dataset. The naïve Bayes classifier has the least f-measure compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.874 of the f-measure for the Spam Corpus dataset.

Fig. 8
figure 8

F-measure of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 9, the performance of the f-measure for all the thirteen machine learning classifiers has been described. In terms of the f-measure, the Random Forest classifier performs with 0.999 of the f-measure for detecting spam emails from the Spambase dataset. Except for the Random Forest classifier, J48 and Bootstrap Aggregating classifiers are also performed moderately with 0.972 and 0.967 of the f-measure respectively for the Spambase dataset. The naïve Bayes classifier has the least f-measure compared to the rest of the machine learning classifiers. The Naïve Bayes classifier performs with 0.797 of the f-measure for the Spambase dataset.

Fig. 9
figure 9

F-measure of machine learning classifiers for Spambase dataset

Full size image

6.5 Mathews Correlation Coefficient (MCC)

Mathews Correlation Coefficient is the parameter for measuring the binary classification of two classes.

In Fig. 10, the performance of MCC for all the thirteen machine learning classifiers has been described. In terms of MCC, the Random Forest classifier performs with 0.998 of MCC for detecting spam emails from the Spam Corpus dataset. Except for the Random Forest classifier, Decision Table and Logistic Regression classifiers are also perform moderately with 0.997 of MCC for the Spam Corpus dataset. The Naïve Bayes classifier has the least MCC compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.667 of MCC for Spam Corpus dataset.

Fig. 10
figure 10

Mathews Correlation Coefficient of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 11, the performance of MCC for all the thirteen machine learning classifiers has been described. In terms of MCC, the Random Forest classifier performs with 0.999 of MCC for detecting spam emails from the Spambase dataset. Except for the Random Forest classifier, J48 and Bootstrap Aggregating classifiers are also performed moderately with 0.941 and 0.931 of MCC respectively for the Spambase dataset. The Naïve Bayes classifier has the least MCC compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.638 of MCC for Spambase dataset.

Fig. 11
figure 11

Mathews Correlation Coefficient of machine learning classifiers for Spambase dataset

Full size image

6.6 Receiver Operating Characteristic (ROC) Area

Receiver Operating Characteristic area measure the performance of the classifier in a general way.

In Fig. 12, the performance of the ROC area for all the thirteen machine learning classifiers has been described. In terms of the ROC area, Decision Table, Logistic Regression and, Random Forest classifier perform 1.00 of ROC area for detecting spam emails from the Spam Corpus dataset. The naïve Bayes classifier has the least ROC area compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.937 of ROC area for Spam Corpus dataset.

Fig. 12
figure 12

Receiver Operating Characteristic area of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 13, the performance of the ROC area for all the thirteen machine learning classifiers has been described. In terms of the ROC area, the Random Forest classifier performs with 1.00 of ROC area for detecting spam emails from the Spambase dataset. Except for the Random Forest classifier, Bootstrap Aggregating and Linear Regression classifiers are also performed moderately with 0.995 and 0.992 of ROC area respectively for the Spambase dataset. The Sequential Minimal Optimization classifier has the least ROC area compared to the rest of the machine learning classifiers. Sequential Minimal Optimization classifier performs with 0.896 of ROC area for Spambase dataset.

Fig. 13
figure 13

Receiver Operating Characteristic area of machine learning classifiers for Spambase dataset

Full size image

6.7 Precision Recall (PRC) Area

The Precision Recall area evaluates the imbalanced dataset in terms of binary classification.

In Fig. 14, the performance of the PRC area for all the thirteen machine learning classifiers has been described. In terms of PRC area, Decision Table, Logistic Regression and, Random Forest classifier perform 1.00 of PRC area for detecting spam emails from the Spam Corpus dataset. The naïve Bayes classifier has the least PRC area compared to the rest of the machine learning classifiers. Naïve Bayes classifier performs with 0.920 of PRC area for Spam Corpus dataset.

Fig. 14
figure 14

Precision Recall area of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 15, the performance of the PRC area for all the thirteen machine learning classifiers has been described. In terms of PRC area, the Random Forest classifier performs with 1.00 of PRC area for detecting spam emails from the Spambase dataset. Except for the Random Forest classifier, Bootstrap Aggregating and Linear Regression classifiers are also performed moderately with 0.995 and 0.992 of PRC area respectively for the Spambase dataset. The Sequential Minimal Optimization classifier has the least PRC area compared to the rest of the machine learning classifiers. Sequential Minimal Optimization classifier performs with 0.866 of PRC area for Spambase dataset.

Fig. 15
figure 15

Precision Recall area of machine learning classifiers for Spambase dataset

Full size image

6.8 False Positive (FP) Rate

False Positive Rate is the parameter for representing the number of times wrongly predicted by the classifier.

In Fig. 16, the performance of the FP Rate for all the thirteen machine learning classifiers has been described. In terms of FP Rate, Decision Table, Logistic Regression and, Random Forest classifiers are having the least FP Rate compared to the rest of the machine learning classifiers. Decision Table, Logistic Regression and, Random Forest classifiers perform with 0.000 of FP Rate for the Spam Corpus dataset. Naïve Bayes and Adaptive Booster classifiers perform with 0.230 and 0.222 of FP Rate respectively for detecting spam emails from the Spam Corpus dataset.

Fig. 16
figure 16

False Positive Rate of machine learning classifiers for Spam Corpus dataset

Full size image

In Fig. 17, the performance of the FP Rate for all the thirteen machine learning classifiers has been described. In terms of FP Rate, the Random Forest classifier is having the least FP Rate compared to the rest of the machine learning classifiers. Random Forest classifier performs with 0.001 of FP Rate for Spambase dataset. Naïve Bayes and Sequential Minimal classifiers perform with 0.148 and 0.117 of FP Rate respectively for detecting spam emails from the Spambase dataset.

Fig. 17
figure 17

False Positive Rate of machine learning classifiers for Spambase dataset

Full size image

7 Conclusion and future work

In this research work, multiple machine learning classifiers have been implemented for detecting spam emails. The proposed framework has been classifying spam emails and ham emails from the datasets. Two well-known datasets have been used for implementing those thirteen machine learning classifiers. Based on the performance of those thirteen machine learning classifiers experimental analysis has been performed. For experimental analysis, eight parameters have been used. In terms of accuracy, the Random Forest classifier performs better compared to the rest of the machine learning classifiers. Random Forest classifier had the accuracy of 99.91% and 99.93% for the Spam Corpus and Spambase datasets respectively. In terms of accuracy, the Naïve Bayes classifier performs poorly compared to the rest of the machine learning classifiers. The Naïve Bayes classifier had the accuracy of 87.63% and 79.53% for the Spam Corpus and Spambase datasets respectively. In terms of other evaluating parameters also the same result reflected that the Random Forest classifier is the best among all classifiers whereas the Naïve Bayes classifier is the worst among all classifiers.

From the experimental analysis section, it is clear that the Naïve Bayes classifier didn’t perform up to the mark for both datasets compared to other machine learning classifiers. In future, planning to improve the performance of the Naïve Bayes classifier based on feature selection for detecting spam emails.