Keywords

1 Introduction

In today's digital era, the manipulation of visual content has become incredibly advanced. Deep fake technology uses special computer programs known as Generative Adversarial Networks (GANs). These programs can create computer-generated images and videos that resemble real ones. Earlier, the technology could be used to create art and entertainment that raises serious challenges [1]. Deepfake technology involves the use of convolutional neural networks, particularly Generative Adversarial Networks (GANs), to create fake images or videos by replacing one person's face with another’s person. Recently, social media issues have highlighted instances in which celebrities’ faces were illicitly swapped into explicit content, causing damage to their reputations and long-term harm to the identities of well-known persons and common people in society [2].

GAN model was first introduced by Ian J. Goodfellow in 2014 [2]. GAN was described as a connection between two components i.e., discriminator named as “D” and the generator named as “G.” GAN method is an effective way to generate images. Over time, more advanced GANs such as BEGAN [3], PGGAN [4], StyleGAN [5], and StyleGAN2 [6]emerged. These advanced GANs have been successful in creating images for numerous purposes, such as image enhancement, translation, and image filling in missing parts.

The images generated by GANs are really hard to detect with the human eyes. This level of realism can provide an in-depth insight into particular issues especially when GAN-generated images are utilized for deception purposes. When these images are used to create fake news or to deceive ordinary people on social media by generating fake personal information, several serious challenges arise. Firstly, there are ethical problems, as the planned spread of incorrect information may affect public perception and damage trust in digital content. Secondly, there may be legal implications because these activities may violate laws governing misinformation, defamation, or identity theft. Finally, from a security aspect, GAN-generated images exploited for illicit purposes may lead to various problems, including financial scams and identity theft [7].

There are various detection algorithms have been developed to address the issues mentioned earlier which can be grouped into two categories i.e., Conventional and Deep learning methods. Conventional methods have been developed for extracting image properties such as texture and structural features which rely on manually created methodologies These methods offer a simple and relatable path for detecting fake images [8,9,10,11]. Although, the conventional methods have various drawbacks which require large processing resources to provide excellent performance across various scenarios. In comparison, deep learning methods have gained the interest of researchers due to their excellent outcome in image classification. Deep learning methods [12,13,14,15,16] have the advantage of being less computationally complex compared to conventional methods, but they lack interpretability. These methods require a large amount of data to train efficiently and are susceptible to overfitting, which can limit their practical application. Therefore, there is still a need for continual efforts to develop an efficient method for GAN-generated image detection.

GANs were originally designed for image creation, with a generator network producing synthetic images and a discriminator network discriminating real from fake images. Which involves training a GAN on a dataset that contains both real and GAN-generated fake images. The discriminator network is trained to distinguish between real and GAN-generated images, a process known as adversarial training. This rigorous training improves the network's capacity to identify DeepFakes precisely. Finally, the key contributions of this research are demonstrated by systematic experimentation, which evaluates the performance, accuracy, and reliability of the GAN-based detection model.

The experiment introduces a novel CNN-GAN based architecture designed to detect deep fake images. This innovative approach enhances the capacity to identify manipulated images by training on a preprocessed dataset to reduce the threats of deep fakes. Moreover, it provides an important tool for identifying the real images.

This research explores the use of celebrity image datasets in deepfake detection. The research improves the accuracy of identifying modified images by leveraging renowned celebrities. To employ these datasets provides a model for protecting celebrities from identity theft within deep fakes content.

This study contributes by thoroughly evaluating the proposed CNN-based GAN model's efficiency in detecting DeepFakes images.

The paper is organized in the following manner. Section 2 provides the detailed review of literature on GAN and its types. Section 3 presents the methodology and discusses the proposed detection model of the research work. Section 4 provides details about results and discussion about the experiments performed. Finally, the Section 5 describes conclusion and future work related to the research work.

2 Literature Review

GANs (Generative Adversarial Networks) are type of deep learning model that has revolutionized the field of generative modelling. GANs are used to create new data that is similar to a training dataset. Unlike other generative models that use explicit probability distributions, GANs learn to generate data by training two types of neural networks: a generator network and a discriminator network [17].

Mirsky et al. [18] examined DeepFakes by studying reenactment techniques such as manipulating facial expressions, lips, posture, or even an entire body, as well as conversion methods such as face swapping or transferring. Verdoliva et al. [19] provided an overview, distinguishing between conventional methods based on sensor-based, model-based, and supervised methods, as compared to methods based on deep learning techniques such as vCNN models. Shobhit et al. [20] investigated several images and video manipulation methods i.e., popular methods, and forgery detection methods. Syed Sadaf Ali et al. [21] designed advanced deep learning algorithms to detect double image compression forgeries, whereas Huang et al. [22] used data augmentation and single sample clustering to improve FakeLocator's detection of various DeepFake methods.

Table 1. Comparison of GAN and its types
Full size table

Table 1 illustrates s a comparison of several GAN types, considering their architectural structure, activation functions, number of layer, advantages, and limitations. The table demonstrates that by implementing a configuration consisting of 4 convolutional and de-convolutional layers, along with the activation functions Tanh, ReLU, and Leaky ReLU, the proposed model outperforms compared to others, achieving an accuracy of 95%.

3 Methodology

3.1 Dataset

In this research, the CelebA attributes dataset has been used. CelebA is a huge dataset including over 200,000 celebrity images annotated with 40 attributes. This extensive dataset contains a broad variety of facial data, encompassing a wide range of changes in facial emotions, positions, and backgrounds because of its thorough annotations, which include parameters such as gender, age, the presence of spectacles, and facial expressions, the dataset is widely used in computer vision and machine learning domains. Therefore, the dataset has been selected to perform DeepFakes detection in this research work. The dataset has been distributed into training, testing and validation sets. There are total of 202,599 images in the dataset. Out of total images, 162770 belongs to training set, 19961 belongs to testing data set while 19866 images belong to validation set. Among the dataset, 80% of images are used for training, 10% for tests, and 10% for validation. The details of distribution are given in Table 2.

Table 2. Distribution of training, testing, and validation dataset
Full size table

3.2 Preprocessing

The GAN preprocessing stage includes three critical operations i.e. image resizing, rescaling, and normalization. The goal of rescaling is to bring the dataset values into a consistent and standardized range, specifically between -0.9608 and 0.9608, which is accomplished by increasing the pixel values of each image by 2 and then subtracting 1. The normalization step is critical for accelerating convergence during training and minimizing gradient vanishing or exploding issues. Additionally, the images are resized to 32 x 32-pixel dimensions to maintain uniformity. The dataset resulting from the preprocessing steps, serves as an input for the GAN model while aligning with the model's requirements and supporting effective training and meaningful outcomes.

3.3 Convolutional Based GAN

A Convolutional Generative Adversarial Network is a cutting-edge deep learning model that has transformed generative modeling and image synthesis. Convolutional GANs is a version of the basic GAN architecture that are specifically designed for detecting GAN generated images. In a competitive training procedure, two neural networks i.e., the generator and the discriminator are integrated.

Fig. 1.
figure 1

Flow diagram of proposed Convolutional based GAN

Full size image

Figure 1 demonstrates the process of generating random noise to feed into the proposed model. Later, the noise vector is fed into the generator, comprising of de-convolutional layers and activation functions. The noise is transformed within the generator, eventually resulting in the generation of an image. The generator takes a step-by-step method, improving the image with a series of convolutional layers. The layers are adept at identifying and capturing intricate features in a hierarchical manner, which enable the network to produce realistic images as training progresses. Meanwhile, the discriminator acts as an evaluator, rigorously scrutinizing the generated images in order to distinguish from real ones. The adversarial training dynamic pushes both the generator and discriminator to constantly improve the network.

3.4 Architecture of Convolutional GAN

The proposed model i.e., CNN-based GAN architecture, is a variant of Generative Adversarial Networks (GANs) optimized for image generation tasks. The model combines convolutional layers into both the generator and discriminator networks, significantly improving their ability to analyze image data. The model includes several crucial elements. Firstly, convolutional layers are added into both the generator and discriminator networks, allowing for the extraction of detailed visual features from edges to higher-level patterns. In the generator netwoek, strided convolutions are used to improve the generation of images while gradually increasing spatial resolution from lower to higher scales. In both networks, batch normalization is explicitly performed after convolutional layers, ensuring stable accelerated training via activation normalization. Secondly, there are de-convolutional layerswhich are also known as transposed convolution layers. These layers assist in the process of up-sampling. The layers play an important role in transforming low-dimensional noise vectors to higher-resolution images, gradually increasing their complexity and details. The weights of the generator are iteratively modified based on feedback from the discriminator throughout the training process. Notably, the discriminator's weights remain untrainable, ensuring that the focus is solely on refining the generator's performance. The interaction between the generator and discriminator is crucial to this process. The generator transforms random noise into realistic images by employing a series of convolutional layers, batch normalization, and activation functions such as leaky ReLU. In terms of loss computation, the generator's loss is evaluated using binary cross-entropy loss function. Simultaneously, the discriminator evaluates these generated images critically while distinguishing from real ones. The dynamic interplay is the core of adversarial training, in which the generator aims to outperform the discriminator by constantly improving its image generation capabilities. Figure 2 demonstrates the structural architecture of the proposed model.

Fig. 2.
figure 2

Structural diagram of proposed Convolutional GAN

Full size image

3.5 Parameters for Convolutional GAN

Table 3 provides a detailed summary of the parameters used in the DCC-GAN model. In Table 3, there are several parameters including the kernel size, which determines the filter's coverage in terms of pixels, and the stride, a value that affects the filter's movement over the input image.. Furthermore, the table describes the features associated with each convolutional layer, defining the intricate features learnt during the training process. Besides this, another key feature is batch normalization, which is defined as a technique employed across neural network layers to accelerate training, improve stability, and enable the use of higher learning rates. Moreover, the activation functions play an essential role in assisting the neural network to detect complicated patterns in the image. Notably, the model's generator component uses these parameters to convert a 100-dimensional random noise input into a 32 × 32 × 3 image. Concurrently, the discriminator analyses images with convolutional layers and provides a probability score indicating the authenticity of the input image. These parameter values are required for the Convolutional GAN model's training.

Table 3. Model Parameters of Proposed model
Full size table

3.6 Evaluation Metrics

The model's performance was evaluated using evaluation measures such as True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN). Precision, Recall, Specificity, and Accuracy were calculated using these measures to determine the model's efficacy. The following are detailed descriptions of these measures, as well as their associated mathematical formulas:

Precision

Precision is a measure of how effectively a model predicts a positive a positive outcome, representing the proportion of correct positive predictions produced by the model out of all positive predictions made by the model.

$$percision=True\; positive \div True \;positive +false \; positive$$

Specificity

Specificity is a measure of a model's ability to accurately detect negative cases in the context of binary classification evaluation metrics. It evaluates the proportion of true negative predictions among all actual negative cases.

$$specificity=True \,\,negative \div True \,\,negative +false \,\,positive$$

Recall/Sensitivity

In binary classification, recall, also known as sensitivity or true positive rate, is an important evaluation parameter which measures the model's ability to correctly identify all positive instances in a dataset out of all actual positive instances.

$$recall \,\,or\,\, sensitivity=True \,\,positive \div True\,\, positive +false \,\,negatives$$

F1 Score

The F1 Score is a binary classification metric that combines precision and recall to create a single score that describes a model's performance which is particularly useful when the class distribution is unequal or when false positives and false negatives have distinct costs or implications.

$$F1 \; Score=2 \times percision\times recall \div percision+recall$$

Accuracy

Accuracy is a fundamental evaluation metric used to evaluate a classification model's overall performance. Accuracy is a measure of how well a model performs and measures the ratio of correctly predicted images to the total number of images in the dataset.

$$\begin{aligned} Accuracy =True\,\, positive & +True\,\, negative \div True\,\, positive +false\,\, positive\\ & +false \;nagatives+true\,\, negativ \end{aligned} $$

4 Result and Discussion

The study focuses on identifying CNN-GAN generated images, demonstrating greater performance when compared to other models. The GAN uses convolutional layers to improve its efficiency. In the context of GANs, the generator uses deconvolution to generate images from random variables, while the discriminator uses CNN to identify images as either real or fake. Convolutional layers extract features from images, while de-convolutional layers expand images based on the extracted characteristics.

The experiment was carried out on large datasets using a hybrid architecture that combines the Convolutional Neural Network (CNN) layer with Generative Adversarial Network (GAN) model. The experimentation environment was built on the Jupyter Notebook utilizing the computational capability of an available NVIDIA GPU. Moreover, the Keras 2.3.1 framework, Torch, and the Python-based open-source neural network deep learning libraries were used to implement CNN-GAN. The proposed model represents an enhancement towards exploiting deep learning technology in combination with GAN for the specific dataset. The model was trained across 100 epochs with 128 batch sizes each which allowed the model to learn intricate features and patterns from the dataset. Besides this, the model improved its effectiveness to detect DeepFakes images efficiently.

4.1 Analysis of CNN-GAN Model

The primary goal of the research was to determine the effect of the CNN-GAN model on system performance. The proposed models were evaluated using a large dataset to achieve good results. Moreover, the performance of the proposed model was evaluated based on the evaluation metrics i.e., accuracy and loss. The proposed model's accuracy was measured by its ability to accurately classifying images as real or fake, whereas the loss quantifies inaccuracies throughout the training process.

Fig. 3.
figure 3

Comparison of Loss and accuracy of model over 50 and 100 epochs

Full size image

Figure 3 demonstrates that the discriminator's accuracy and loss curves are unstable at epoch 50. However, when the number of epochs is increased to 100, accuracy and loss demonstrates an upward trend before stabilizing. During the 100th epoch, the model reaches an accuracy of 95.8 percent. In Fig. 3, x-axis represents the number of training epochs, signifying the training cycle through the entire dataset. Meanwhile, the loss and accuracy are represented on the y-axis.

Fig. 4.
figure 4

Result of Classification Probability of Random Test Image

Full size image

In the experiment, a random image has been chosen from the test dataset, and the model predicts whether the image is fake or real, as shown in Fig. 4. The method completely assesses the model's classification capabilities and capacity to generalize across multiple DeepFakes settings. Furthermore, the model aids in gaining insights about the model's proficiency in distinguishing various aspects and attributes associated with real and fake images, as observed through the projected probabilities.

Figure 5 compares confusion matrices produced after the model evaluations at 50th and 100th epochs. Each confusion matrix is a visual aid for evaluating the model's predictions and correspondence with actual images. The matrices provide information about the model's ability to classify images as real or fake correctly. The matrices show how many images are given to each class. Furthermore, the confusion matrices identify patterns and distribution entries that reflect the model's strengths and weaknesses, allowing for future upgrades and fine-tuning to improve overall classification performance.

Fig. 5.
figure 5

Comparison of Confusion Matrices Between 50 and 100 Epochs

Full size image

Table 4 provides an exhaustive overview of major metrics of evaluation important for measuring the effectiveness of proposed Convolutional GAN model. Accuracy, which measures the overall correctness of predictions, is 0.958, which is the ratio of properly predicted instances to entire dataset size. The model's sensitivity (recall) of 0.972 represents the ability to identify true positive instances among all actual positives, which is critical for minimizing false negatives. Specificity, at 0.943, measures the model's ability to distinguish between true negatives among all actual negatives, with a focus on reducing false positives. Precision (0.946) computes the accuracy of positive predictions, emphasizing the model's ability to avoid misclassifying negatives as positives. Finally, the F1-Score (0.958) combines precision and recall into a balanced measure, indicating a model that excels in both high precision and recall.

Table 4. Evaluation matrices measurements for the proposed model
Full size table

5 Conclusion

Generative Adversarial Network i.e., GAN generates synthetic data with a realistic appearance of images by developing an algorithm in a large number of iterations. DeepFakes technology is a hard challenge through which the differentiation can be made between real and fake. The said challenge allows researchers to provide optimal algorithms and solutions to detect DeepFakes. The proposed model is capable of working efficiently with the large dataset. The results depict that the accuracy of the proposed model is outperforming as compared to the other models. In the research work, the loss of discriminator is minimized in a number of iterations as compared to the generator’s loss. The evaluation of the proposed model is performed through sensitivity and specificity. The generalization and convergence of GAN is a promising future research area for improvements.

5.1 Future Direction

There are several limitations of the proposed model i.e., difficulties with mode collapse, issues with gradient descent and usual convergence challenges in GANs. Moreover, there are many future challenges focusing on the improvement and generalization of GAN models.