Keywords

1 Introduction

Phishing attacks continue to be very common with 465 brands targeted in Mar 2021 (Statista [2]). According to [3], phishing incidents rose 220% during the pandemic compared to the otherwise yearly average, with 52% of these attacks targeting brand names. 72% of the attacks during the pandemic used a valid HTTPS certificate while almost all of them used TLS encryption. Phishing website detection can help in finding such attempts and keep everyone safe in today’s digital world. A lot of work has been done in the past and this paper analyzes articles related to phishing website detection using deep learning. The motivation of this article is to address the lack of standardization and difficulty in comparing various methodologies in this field. It aims to familiarise its reader with the methodologies, algorithms, and tools used in such studies. It also provides statistical figures to summarize the results and adds suggestions that might encourage easy replication and comparison across similar studies.

Research Question: This paper looks at the following research questions:

What is the current state of the research in phishing website detection using Deep Learning and how can proposed methodologies be made easier to replicate and comparable with other studies?

2 Theoretical Background

Phishing is a cybercrime where an attacker poses as a legitimate institution to lure the target into providing their sensitive data (Phishing.org [4]). Apart from email and website phishing attempts, there are other variants of such types of attacks that use voice calling and text messages. Additional attacks include website forgery, malware, and domain spoofing through which a victim can be trapped in phishing. Typically, Phishing is used against a large number of random targets, while in spear phishing, a targeted version of phishing, the attacks are targeted towards certain individuals who may possess valuable information.

To counter such attacks, various detection models have been proposed. Some of those concentrate on improving the detection rate of phishing URLs while others focus on reducing the detection time. To achieve this, authors use machine learning models like Naive Bayes, Logistic Regression, Random Forest, Decision Trees, Support Vector Machine (SVM), k-Nearest Neighbour, and deep learning models. Deep learning models include Convolutional Neural Network (CNN), Long Short Term Memory (LSTM), Recurrent Neural Network (RNN), and various other novel mechanisms.

Section 3 in this paper discusses the search strategies, inclusion, and exclusion criteria used in this systematic review. Section 4 covers the various algorithms used, paper goals, contents of the datasets, programming tools, and metrics used in these studies. Section 5 concludes the article with proposed suggestions for standardization techniques for future phishing detection research.

3 Methodology

This search was performed on City University of New York’s (CUNY) online library OneSearch. We kept the search criteria broad by searching articles related to phishing and manually narrowed it down to website phishing. This is to ensure that papers that do not have the exact keywords can still be included in the study. Systematic review methodology by Kitchenham [5] is utilized in this study.

3.1 Search Strategy

  • Keyword Search: The following search strings were used to find relevant papers: “Phishing” AND “Detection”

  • Period: Articles published between 2017 and 2021

  • Paper Type: Articles published in conferences or journals

  • Search Database: City University of New York’s online library CUNY OneSearch

  • Inclusion Criteria:

    • Articles written in English.

    • Article scheduled to be published with a pre-print available

    • Articles including keywords in title, full-text or their metadata.

    • Articles implementing or proposing a solution relevant to phishing detection

  • Exclusion Criteria:

    • Book Chapters, Newsletter Articles, Books, and Dissertation.

    • Systematic reviews and literature survey.

    • Articles with pure ML-based implementations.

The search resulted in ninety-two papers out of which twenty-six papers were discarded after applying the exclusion criteria leaving a total of sixty-six papers. Articles that utilized pure Machine learning-based implementation such as kNN, SVM, Random Forest, and Logistic Regression were excluded from this study, except for hybrid and ensemble models where some ML algorithms are combined with deep learning-based algorithms.

4 Discussion/Findings Overview

Once the papers were collected and filtered using the method specified in the previous section, the articles were examined and categorized according to different criteria, including:

  • Datasets: Datasets are used in the training and testing of the model. In phishing detection, the data needs to be continuously updated so researchers list out the methodologies used to fetch data from popular data sources. The Datasets include different features such as URLs, length of URL, domain based-features including the age of domain, DNS record, and HTML based features: number of out links, anchor tags, etc. Table 1 lists the datasets and data sources that are shared and used in multiple studies.

  • Programming tools: Listing out the programming tools helps researchers in reproducing the work and comparing their proposed work against the same environmental specifications. The result in Sect. 4.2 confirms the recent trend in the use of python over other programming languages.

  • Algorithms used: Various. deep learning based algorithms are used in model training. Some researchers fused multiple algorithms in ensemble and hybrid approaches to improve detection accuracy of the model.

  • Research Methods: The different design goals of the covered work are described, which include removing dependencies or minimizing the needed input data as well as improvement of detection rates and reducing training and testing runtime.

4.1 Datasets

While some studies use proprietary datasets, multiple studies include publicly accessible ones, listed in Table 1.

Table 1. Most popular datasets used in phishing detection
Full size table

Datasets Features: Different types of data are used in various research, including:

  • URL: Uniform Resource Locator(URL) and its related information like its length, and use of special characters with or without trimming were used in most of the studies. While most studies use URL repositories, some combine it with additional data listed below

  • Metadata information: Metadata website information includes age of domain, popularity of websites, DNS rank, etc.

  • Webpage content: It includes HTML tags based on information like the number of links in the source code.

  • Images/screenshots: Some datasets have images and screenshots of the website or logo of targeted brands. These images were used along with URL based information to improve the accuracy of the model.

4.2 Programming Tools

Few articles described the programming tools and specifications used in their experiments. Among them almost 35% of the articles used python and approximately 10 % of them used WEKA, Java, and MATLAB-based implementation. The use of these latter languages is decreasing as compared to python. A survey conducted by Kaggle [17] indicated there is a large number of submissions in python and a significantly lower rate of submissions in other languages such as MATLAB and Java. This trend has been observed starting 2013 [18]

4.3 Algorithms

This section lists major deep learning algorithms used in training the classification models. The literature shows three main approaches used by researchers: deep learning models with a single algorithm, hybrid approaches, and ensemble approaches. The latter two approaches are analyzed in more detail in Sect. 4.4

Deep Learning Models: Primary deep learning models used were Deep Neural Networks with hidden layers, CNN, LSTM, and RNN. The models and their accuracy using these algorithms are shown in Table 3.

Ensemble Models: Multiple diverse models are generated and a final prediction is made after aggregating their predictions. Although this model consists of several base models, the model still acts and works like a single model. The ensemble model aims to reduce the generalization error of prediction. Nagaraj et al. [19] used random forest and neural network to get an accuracy of 93.41 on their ensemble model. Another model with LSTM and SVM saw an accuracy in the range of 95.40%–98.50%

Hybrid Models: The model is made by fusing multiple models into a single model. The algorithms used in such models and accuracy ranges are presented in Table 2.

4.4 Research Methods

This section categorizes the research methods used in the surveyed studies. We broadly classify the domain where these improvements were visible into three categories.

  • Novelty in Feature Selection techniques: [20,21,22,23] used various feature extraction methods. [24, 25] introduced novel features and [26] evaluated its model on different feature spaces. [27, 28] used novel feature selection technique like Recursive Feature Elimination(RFE) in their work.

  • Use of Fusion/multilevel architecture: Study by Kazienko et al. [29] shows the use of fusion and multilevel techniques like ensemble model and hybrid model improves the performance of the ML model. While ensemble models can take more time to train the model, a study by Sameen et al. [30] speeds this up through the use of a multi-threaded approach. Different fusion/multilevel architecture models are presented in Table 2.

  • Generation of Adversarial URLs: Evaluating security aspects of a model is useful for evaluating the ability to prevent adversarial attacks. [31] assesses vulnerability of a system while [32] talks about defense against attacks. Adversarial phishing URL were generated by [33,34,35,36,37].

  • Eliminating need of dependencies: Performance of a model can be affected by the interruption of third-party services, language dependencies, etc. Study by Somesha et al. [38], Yang et al. [39], Waziral et al. [27], and Jain et al. [40] eliminated need for third-party services in their work. Webpage content-based features, language dependencies and use manually crafted features were eliminated in [39, 41, 42] respectively.

  • Additional methods: Work by [22, 43,44,45] were directed towards increasing speed of detection. [46, 47] addressed zero-day phishing vulnerabilities, [48] visualized internal working of a DNN while [41, 49, 50] created phishing detection aimed at low-power mobile devices.

Table 2. Hybrid approaches used in studies
Full size table
Table 3. Deep learning algorithms used in studies
Full size table

4.5 Recommendations

  • Use of shared datasets: Shared datasets can help in replication and comparison among different models. It would be useful for researchers to test their data on shared datasets in addition to any proprietary dataset when possible, to help improve the side-by-side evaluation of different algorithms. Sometimes the datasets are not shared due to privacy or ethical issues. The researchers can share their approach to fetching data from a data source for easier replication.

  • Sharing code/algorithm: This will encourage reproducibility of the work and provide a way for researchers to further adapt or expand the current work.

  • Testing on updated datasets continuously. As new phishing websites are continuously introduced, this will provide a way to gauge the performance of successful phishing detection algorithms on newly introduced phishing URL

  • Standardize testing environment documentation: Experimentation is the description of the environment in which the experiment was performed. Creating a standard method of documentation, which will include details regarding the system parameters used and run-time, can help researchers assess the usability of different methods in different attack scenarios as well as recreate the test environment in future studies.

5 Conclusion

Phishing can be done through different techniques. This work focuses on the detection of phishing websites using deep learning neural networks. This study found that there is a growing body of research in this field, utilizing different techniques, datasets, and attack scenarios. This work points to share as well as datasets that continue to update and can be used in future research. It also compares the goals and design details of different studies and the resulting reported performance. This paper suggests methods for standardization of algorithms and testing reports, which can help improve the design of future studies.