FakeFilter: A cross-distribution Deepfake detection system with domain adaptation

Abstract

Abuse of face swap techniques poses serious threats to the integrity and authenticity of digital visual media. More alarmingly, fake images or videos created by deep learning technologies, also known as Deepfakes, are more realistic, high-quality, and reveal few tampering traces, which attracts great attention in digital multimedia forensics research. To address those threats imposed by Deepfakes, previous work attempted to classify real and fake faces by discriminative visual features, which is subjected to various objective conditions such as the angle or posture of a face. Differently, some research devises deep neural networks to discriminate Deepfakes at the microscopic-level semantics of images, which achieves promising results. Nevertheless, such methods show limited success as encountering unseen Deepfakes created with different methods from the training sets. Therefore, we propose a novel Deepfake detection system, named FakeFilter, in which we formulate the challenge of unseen Deepfake detection into a problem of cross-distribution data classification, and address the issue with a strategy of domain adaptation. By mapping different distributions of Deepfakes into similar features in a certain space, the detection system achieves comparable performance on both seen and unseen Deepfakes. Further evaluation and comparison results indicate that the challenge has been successfully addressed by FakeFilter.

Keywords

Digital multimedia forensics face swap Deepfake detection domain adaptation

1. Introduction

Digital visual media, such as images and videos, are widely accepted as critical evidence in investigations, military and digital multimedia forensics [13]. Nevertheless, the integrity and authenticity of digital visual media can be undermined by various forgeries, the most harmful of which is the face swap techniques [41]. Such forgery swaps the face in the target image or video with a donor’s face observed in other media, which creates the illusion of a person’s behaviors that do not happen in reality. Adversaries usually employ computer graphics-based methods to swap faces [40], which inevitably reveals some tampering traces. Whereafter, a new type of fake samples is emerging, i.e., Deepfake [41], which is created using deep learning models such as Autoencoder [4] and Generative Adversarial Network (GAN) [16]. Compared with the former ones, Deepfakes are more realistic and high-quality, which retains the target’s facial expressions, head poses and reveals few tampering traces, as illustrated in Fig. 1. With the risks of the applications for creating the Deepfakes, such as FakeAPP1

¹
https://apps.apple.com/cn/app/id413501652

and ZAO,2

https://www.zaoapp.net/

anyone is equipped to implement face swap, which poses serious threats to individual, social and even national security [8].

With increasing concerns on the Deepfakes, Deepfake detection has become an emerging trend in digital multimedia forensics research, and various methods have been proposed. Among them, some research focuses on the macroscopic-level semantics of images, and attempts to identify discriminative visual features to detect Deepfakes, such as head poses [44], details in eyes and teeth [28], and the movements of facial muscles [3]. Nevertheless, affected by objective conditions such as the angle or posture of a face, the discriminative visual features are unlikely available in every image or video frame, which limits the applicability of such methods.

As an alternative to the above visual feature-based methods, based on the difference in the microscopic-level semantics of images, some research focuses on devising effective deep networks to classify real and fake faces [2,11,30]. Deep neural networks are capable of perceiving the differences that are difficult for human eyes, which leads to promising performance of such methods. Nevertheless, these methods are demonstrated with good performance as the testing and training Deepfakes are created with the same methods (abbreviated as seen Deepfakes). Upon the encountering of Deepfakes created with different methods from the training sets (abbreviated as unseen Deepfakes), the performance of deep neural networks will be greatly degraded [27]. That is, the poor generalization on unseen Deepfakes poses a challenge to existing detection methods.

Fig. 1.

Screenshots from the corresponding videos [24]. The generated Deepfake (c) presents the donor’s face in (b) while retaining the target’s facial expressions and head poses in (a).

In this paper, we focus on the deep neural network-based methods in views of their promising performance, and aim to address the existing challenge of unseen Deepfake detection. Therefore, we would like to propose a novel Deepfake detection system, named FakeFilter. Preliminarily, effective classification networks are required to be devised, which is expected to make FakeFilter to possess the capacity to classify real and fake faces. Further, effective generalization strategies are required to be proposed, which is expected to make FakeFilter to effective detect multifold Deepfakes no matter whether they are unseen or not.

With the above expectations on FakeFilter, we mainly make the following contributions:

Based on the observation that seen and unseen Deepfakes share identical labels whereas express different feature distributions for a neural network. We provide a promising perspective of formulating the generalization challenge into a task of cross-distribution data classification, to classify multifold Deepfakes with different feature distributions.

We devise a preliminary detection network of FakeFilter to extract discriminative features and classify real and fake faces at the microscopic-level semantics of images. The preliminary detection network is demonstrated with good performance as encountering seen Deepfakes, e.g., the image-level accuracy, EER and AUC are achieved by 94.00%, 6.02% and 98.56%, respectively.

We address the issue of cross-distribution Deepfake detection with the strategy of domain adaptation [14,15]. Specifically, we devise and augment a domain adaptation component to the preliminary detection network, which enables FakeFilter to extract the features with similar distributions from both seen and unseen Deepfakes, so as to achieve approximately comparable performance on them. It is verified that FakeFilter significantly improves the performance on unseen Deepfakes, e.g., the image-level accuracy is improved by 11.13% and 22.81% on two cross-distribution datasets, and it is appropriate for those Deepfakes created with deep learning-based or computer graphics-based methods.

The reminder of this paper is structured as follows. Section 2 presents a concise introduction to related methods of Deepfake creation and detection. Our proposed detection system is presented on and evaluated in Section 3 and Section 4, respectively. Finally, we conclude this paper and envisage the future work in Section 5.

2. Background and related work

In this section, we first introduce two typical methods to create Deepfakes, and then review their related detection methods.

2.1. Background

Currently, various implementations of Deepfakes are available, and the most typical ones are based on AutoEncoder and GAN.

The AutoEncoder-based method, FaceSwap,3

³
https://github.com/deepfakes/faceswap

creates the first generation of Deepfake, which is based on two autoencoder networks sharing a common encoder, as illustrated in Fig. 2. The encoder is trained to extract latent identity features from the donor and target face, respectively. The detectors are utilized to crop, align and reconstruct faces with the extracted features. To create a Deepfake, the trained decoder for target faces is then applied to a donor face, and thereby the decoder output the reconstructed target face with the identity features of the donor person. Ultimately, an output of the autoencoder and the rest of the image are blended using Poisson image editing [33].

Fig. 2.

Workflow of the Autoencoder-based Deepfake creation method. Based on the latent identify features extracted from a donor face, a Deepfake is created by Decoder 2 trained with target faces.

The GAN-based method, FaceSwap-GAN,4

⁴

https://github.com/shaoanlu/faceswap-GAN

is extended from the Autoencoder-based method, whereas the created fake faces are closer to the real ones. More specifically, the perceptual loss and adversarial loss proposed in VGGFace [32] are augmented to the Autoencoder network. The perceptual loss helps eye movements to be more accordant with input faces, while smoothing out the artefacts in the segmentation mask to obtain high-quality videos and images. Furthermore, the adversarial loss is implemented from CycleGAN [47] with the weights of FaceNet [37], and a multi-task convolutional neural network is utilized for better face detection and alignment [45].

At present, the typical Autoencoder-based and GAN-based methods are extensively leveraged to create multiple Deepfake datasets, which are adopted as critical baselines for Deepfake detection. It is noteworthy that some previous fake faces created with computer graphics-based methods, such as Face2Face [40] and FaceSwap,5

⁵

https://github.com/MarekKowalski/FaceSwap/

are also collectively referred to as Deepfakes. Moreover, the methods that synthesize the completely non-existent faces [21] are different from the Deepfakes in principle and intention, and hence they are beyond the scope of this paper.

2.2. Related work

With the increasing concerns on such threats, Deepfake detection becomes an emerging topic in digital multimedia forensics research. In general, existing typical work can be divided into visual feature-based and deep neural network-based methods.

2.2.1. Visual feature-based methods

Focusing on the macroscopic-level semantics of images, i.e., the difference between real and fake faces visible to naked eyes, some research detects Deepfakes by the artefacts left when they are created.

Intuitively, by observing the artefacts left by splicing synthesized face regions into source images, Yang et al. train a support vector machine (SVM) model, which classifies real and fake faces by comparing their orientation and the position of 3D head poses [44]. Analogously, based on the face warping artefacts left by the matching step to create fake images, Li et al. propose to train a neural network to perceive such artefacts from the face regions and surrounding areas of Deepfakes [26]. Moreover, focusing on the artefacts inevitably left by the face tracking and editing step, including eye color, light reflections, and missing details of teeth or eyes, Matern et al. train a detection network based on these visual features [28].

The above researchers observe the artefacts in the critical steps to create Deepfakes, including face splicing, warping, tracking and editing. Overall, they provide a fundamental strategy of Deepfake detection. Beyond such methods, some research explores some physiological characteristics that are difficult to be imitated by Deepfakes, which are then applied as discriminative visual features to classify real and fake faces.

Specifically, Agarwal et al. collect the occurrence and intensity for 18 facial action units about the movements of facial muscles and heads as visual features for classification, such as mouth stretch, nose wrinkle and cheek raiser [3]. Moreover, Jung et al. observe the significant change in the pattern of blinking, including a person’s biological factors, physical conditions, etc. Therefore, they focus on the visual feature of eye aspect ratio, and classify real and fake videos based on eye blinking counts and periods [22].

Overall, the aforementioned visual feature-based methods, whether based on artefacts or biological characteristics, have provided a feasible perspective for Deepfake detection. Nevertheless, subjected to objective conditions such as the angle or posture of faces, these discriminative visual features sought out are unlikely available in all images or video frames. In practice, fake faces in various conditions are likely to be encountered, which leads to the limitation of the applicability of such methods.

2.2.2. Deep neural network-based methods

In comparison with the visual feature-based methods, the deep neural network-based ones have overcome the above limitations. Focusing on the microscopic-level semantics of images, i.e., the pixel-level difference between real and fake faces, various types of deep neural networks are devised to classify Deepfakes. Such methods have avoided depending on any specific visual traces, and thus will not be affected by the objective conditions of Deepfakes.

Typically, Güera et al. propose a temporal-aware pipeline, in which a convolutional neural network (CNN) is utilized to extract features, and a recurrent neural network (RNN) is devised to identify Deepfakes [17]. Analogously, focusing on the microscopic properties of images, Afchar et al. propose a fundamental CNN-based detection network, named Meso-4 [2], and a upgrade network consists of a variant of Inception module [39], named MesoInception-4. Further, Nguyen et al. present a novel detection system based on the recent Capsule Networks [36], i.e., a light detection network with only a few parameters [30]. Moreover, Dolhansky et al. devise three simple detection networks, i.e., a tiny CNN model consists of 6 convolution layers and 1 fully-connected layer, an XceptionNet model [9] trained with only facial images, and an XceptionNet model trained with complete images, which provide baseline results with the most simple detection networks [12].

The above methods are verified with good performance on specific datasets, which demonstrates deep neural networks with fundamental architectures are enough for the detection task. Beyond these fundamental networks, some research devises more diversified and customized deep neural networks to detect Deepfakes.

Specifically, Zhou et al. propose a two-stream detection network, in which a GoogLeNet model [39] and an SVM model trained with steganalysis features are collaborated applied for Deepfake detection [18]. Moreover, Considering both spatial and motion information, Wang et al. propose a novel detection system based on 3DCNN models, in which the I3D [7] and 3D ResNet [19] approaches are utilized. Especially, to improve the feature maps of classification models, Dang et al. propose to apply the attention mechanism to highlight the informative regions, and devise a customized CNN for Deepfake detection [11].

Overall, the aforementioned deep neural network-based methods, whether their architecture is simple or complex, can perceive the microscopic differences that are difficult for human eyes, and thereby achieve promising results. Moreover, the deep neural network-based methods have avoided the limitations of the visual feature-based ones that depend on specific visual traces, and we thus mainly focus on such methods in this paper. Unfortunately, Upon the encountering of unseen Deepfakes, the performance of these carefully devised deep neural networks is noticed to significantly degrade [27], which poses the poor generalization on unseen Deepfakes remains an urgent issue to be addressed.

To address the above challenge, two typical solutions are currently proposed. Specifically, Nguyen et al. devise a deep neural network that utilizes multi-task learning [46] to simultaneously classify and segment fake faces, and semi-supervised learning is applied to improve its generalization [29]. Moreover, Cozzolino et al. explore the strategy of transfer learning [31] to learn a forensic embedding using an autoencoder network. The learned embedding acts as an anomaly detector, which enables an unseen fake image to be correctly classified if it is mapped adequately far away from the cluster of real ones [10].

In general, the above two typical methods have improved the generalization capability of deep neural networks on unseen Deepfakes. However, they focus on the computer graphics-based Deepfakes, rather than the emerging deep learning-based ones, and are thus still insufficient to deal with the ever-changing Deepfakes.

Compared with the above related work, the work in this paper has the following characteristics:

Rather than the visual feature-based method, this work belongs to the category of the deep neural network-based method applied to Deepfake detection.

Compared with the typical deep neural network-based methods, we mainly focus on the generalization issue on unseen Deepfakes, which remains a challenge to existing work.

Compared with the work with the same goal as ours, we aim at proposing a comprehensive system that applied to both deep learning-based and computer graphics-based Deepfakes, so as to overcome their limitations of applicative ranges.

2.3. Summary

In response to the harm of Deepfakes, effective detection methods are essential for protecting the integrity and authenticity of digital visual media [8]. On one hand, detecting methods focus on identifying discriminative visual features, show limited success. On the other hand, methods have also been proposed to devise deep neural networks for Deepfake detection at the microscopic-level semantics of images. Although promising results have been achieved by the latter ones, the poor generalization on unseen Deepfakes remains a challenge. Although a few attempts are made to address the challenge, they are still insufficient in terms of their applicative ranges.

Hence, in this paper, we will devise effective neural networks to classify Deepfakes, and investigate how to improve its generalization on unseen Deepfakes.

3. Cross-distribution Deepfake detection system

In this section, we elaborate on the proposed detection system, FakeFilter. First, we construct a preliminary neural network for Deepfake detection, and then augment it with a domain adaptation component for better generalization on unseen Deepfakes.

3.1. Preliminary detection network

As mentioned in Section 2, Deepfakes can be created more and more realistically and indistinguishable for human eyes, whereas can be differentiated by deep neural networks. Therefore, we analyze and discriminate the differences between real and fake faces at the pixel level instead of the higher semantic level, and detect Deepfakes by learning a binary-classification deep neural network.

Fig. 3.

The architecture of FakeFilter. A feature extractor, a fake classifier and a domain classifier are collaborated for detecting Deepfakes with domain adaptation.

3.1.1. Preprocessing

In the preprocessing phase, we first utilize a ffmpeg toolkit [42] to split the real and fake videos into frames at a specified frequency. Since mostly the tampering area of an image concentrates on faces and their surroundings, we then utilize a Dlib toolkit [23] to select those frames with faces and extract the facial areas. Ultimately, the cropped facial images are obtained for further implementations.

Subsequently, to put aside the unseen Deepfakes, a preliminary detection network is constructed to detect Deepfakes, with the architecture as in the yellow and blue parts of Fig. 3.

3.1.2. Feature extractor

In this component, a feature extractor maps rich visual representations and optimizes underlying features for other components.

Specifically, a basic convolutional neural network (CNN) [25,43] is adopted, which consists of a series of successive convolutional and pooling layers. Moreover, it utilizes Batch normalization (BN) [20] for regularization and an activation function of Rectified Linear Units (ReLUs).

Formally, with the feature extractor $G_{f} (\cdot)$ , an input image x is mapped into a feature vector f, and the parameters of this mapping are denoted as $θ_{f}$ , i.e., $\begin{matrix} (1) & f = G_{f} (x; θ_{f}) . \end{matrix}$

3.1.3. Fake classifier

Based on the differences in feature distribution between real and fake faces, a fake classifier is constructed to predict the authenticity of input images. Specifically, the fake classifier is composed of a dense network, which utilizes Dropout [38] for regularization and an activation function of Sigmoid.

Formally, the feature vector f is feed into the fake classifier $G_{y} (\cdot)$ which outputs a label y, and the parameters of all layers in this network are denoted as $θ_{y}$ , i.e., $\begin{matrix} (2) & y = G_{y} (G_{f} (x; θ_{f}); θ_{y}) . \end{matrix}$

During the training phase, the parameters of the fake classifier ( $θ_{y}$ ) and the shared parameters ( $θ_{f}$ ) are both optimized to minimize the prediction loss on the training set, which ensures the classification capacity of the fake classifier, and discriminative features can be extracted by the feature extractor.

We define the classification loss $L_{y} (\cdot)$ as a cross-entropy loss function, denote $L_{y}^{i}$ as the prediction loss evaluated at the i-th sample, and denote n as the number of samples. Then, the training phase is expressed as: $\begin{matrix} (3) & min_{θ_{f}, θ_{y}} [\frac{1}{n} \sum_{i = 1}^{n} L_{y}^{i} (θ_{f}, θ_{y})] . \end{matrix}$

At this point, we have constructed a preliminary neural network to detect Deepfakes, and its performance is evaluated in Section 4.2.

3.2. Domain adaptation component

Upon the encountering of unseen Deepfakes, the seen and unseen ones share identical labels (real and fake) while expressing different feature distributions. Based on such observations, we formulate the challenge of unseen Deepfake detection into a problem of cross-distribution data classification, and adopt a strategy of domain adaptation to address the issue of cross-distribution Deepfake detection. The strategy is inspired and extended by the domain adversarial training network [14,15], a fundamental architecture for addressing the cross-distribution data classification in the domain of machine learning.

Specifically, we divide those two types of Deepfakes into two sets based on the concept of domain, i.e., the feature space and distributions of samples [31].

Source domain sample

labelled samples created with the same methods as the training sets.

Target domain sample

unlabelled samples created with different methods from the training sets.

Our ultimate goal is enabling FakeFilter to detect the Deepfakes from the target domain. In the preliminary detection network, discriminative features are extracted from the source domain samples. If these two domains of Deepfakes can be mapped into similar distributions of features, they are deemed as uniform by FakeFilter, which leads to approximately comparable accuracy on both seen and unseen Deepfakes. That is, the extracted features are expected to be both discriminative and domain-invariant.

To measure the dissimilarity of these two types of feature distributions, a domain adaptation component is augmented to the preliminary detection network, as illustrated in the pink part of Fig. 3.

3.2.1. Domain classifier

Based on the features mapped by the feature extractor, a domain classifier is constructed to predict the domain of an input image. Specifically, the domain classifier adopts the same network architecture as the fake classifier. The feature vector f is input into the domain classifier $G_{d} (\cdot)$ which outputs a label d, and the parameters of all layers in this network are denoted as $θ_{d}$ , i.e., $\begin{matrix} (4) & d = G_{d} (G_{f} (x; θ_{f}); θ_{d}) . \end{matrix}$

During the training phase, the parameters of the domain classifier ( $θ_{d}$ ) are learned by minimizing its prediction loss, which ensures the classification capacity of the network. Simultaneously, the shared parameters ( $θ_{f}$ ) are learned by maximizing the prediction loss of the domain classifier. That is, the feature extractor with parameter ( $θ_{f}$ ) is expected to output the feature vector f, which makes the domain classifier unable to classify the domain of inputs as much as possible. In other words, if the domain of inputs cannot be classified, the domain-invariant are learned, which enables FakeFilter to treat Deepfakes from different domains equally.

Analogously, we define the classification loss $L_{d} (\cdot)$ as the cross-entropy loss function, denote $L_{d}^{i}$ as the prediction loss evaluated at the i-th sample, and denote n as the number of samples. Then, the training phase is expressed as: $\begin{matrix} (5) & max_{θ_{d}} [- \frac{1}{n} \sum_{i = 1}^{n} L_{d}^{i} (θ_{f}, θ_{d})] . \end{matrix}$

Subsequently, those three components are considered in a detection network, and the overall optimization objective is then as: $\begin{matrix} (6) & E (θ_{f}, θ_{y}, θ_{d}) = \frac{1}{n} \sum_{i = 1}^{n} L_{y}^{i} (θ_{f}, θ_{y}) - λ \cdot \frac{1}{n} \sum_{i = 1}^{n} L_{d}^{i} (θ_{f}, θ_{d}), \end{matrix}$ where λ controls the trade-off between these two objectives during training, and the optimization procedure is expressed as: $\begin{array}{l} (7) & (\hat{θ_{f}}, \hat{θ_{y}}) = arg min_{θ_{f}, θ_{y}} E (θ_{f}, θ_{y}, \hat{θ_{d}}), \\ (8) & (\hat{θ_{d}}) = arg max_{θ_{d}} E (\hat{θ_{f}}, \hat{θ_{y}}, θ_{d}) . \end{array}$

Ultimately, the optimization involves the updates of these parameters, where the parameters of the fake classifier ( $θ_{y}$ ) and domain classifier ( $θ_{d}$ ) are optimized to minimize their prediction losses, and the shared parameters ( $θ_{f}$ ) are optimized to minimize the fake prediction loss whereas maximize the domain prediction loss. The updates are implemented with the aid of a gradient reversal operation, which connects the domain classifier and feature extractor, as well as takes the gradient from the succedent level and multiplies it by $- λ$ before passing it to the preceding layer during back-propagation.

At this point, the feature extractor has learned its parameters as training the fake classifier and domain classifier, as well as extracted the discriminative and domain-invariant features to make FakeFilter to be appropriate for the Deepfakes from both domains, as evaluated in Section 4.3.

3.2.2. Performance analysis

For a further analysis of FakeFilter, H-divergence [5,6] is introduced to define the dissimilarity between two distributions S and T with a probabilistic bound. To define the H-divergence, we denote it as $d_{H} (S, T)$ and consider it with a classifier that classifies the feature distribution of an input, i.e., the domain classifier, $G_{d} (\cdot)$ . Formally, the H-divergence between S and T is defined as: $\begin{matrix} (9) & d_{H} (S, T) = 2 sup | P_{f \sim S} [G_{d} (f) = 1] - P_{f \sim T} [G_{d} (f) = 1] | . \end{matrix}$

Further, the performance of the fake classifier on the target domain, $ε_{T} (G_{y})$ , is defined as a probability, i.e., $\begin{matrix} (10) & ε_{T} (G_{y}) = P_{f \sim T} (G_{y} (f) \neq y), \end{matrix}$ which is expected to be as small as possible for better performance.

To correlate the performance of $G_{y} (\cdot)$ on the source domain and target domain, $d_{H} (S, T)$ is applied, and a probabilistic relation is expressed as: $\begin{matrix} (11) & ε_{T} (G_{y}) ⩽ ε_{S} (G_{y}) + \frac{1}{2} d_{H} (S, T) + C, \end{matrix}$ here C does not depend on a particular classifier. It is observed that the aim of approximating $ε_{T} (G_{y})$ and $ε_{S} (G_{y})$ depends on $d_{H} (S, T)$ .

Subsequently, considering the defined $d_{H} (S, T)$ in Equation (9), to correlate the formulation with the optimization process of network training, we define $\begin{matrix} (12) & α (G_{d}) = P_{f \sim S} [G_{d} (f) = 0] + P_{f \sim T} [G_{d} (f) = 1], \end{matrix}$ accordingly, $\begin{matrix} (13) & d_{H} (S, T) = 2 sup | 1 - α (G_{d}) | = 2 sup [α (G_{d}) - 1] . \end{matrix}$

During the training phase, $α (G_{d})$ becomes smaller with the iteration of back-propagation, which leads to smaller $d_{H} (S, T)$ , and better approximation of $ε_{T} (G_{y})$ and $ε_{S} (G_{y})$ . That is, the performance of FakeFilter on those two domains gets approximating with domain adaptation.

In summary, by training a preliminary detection network with real and fake samples, the discriminative features are learned, which enables FakeFilter to classify real and fake faces. Furthermore, by training an augmented domain adaptation component with seen and unseen Deepfakes, the domain-invariant features are learned, which enables FakeFilter to treat different domains of Deepfakes equally, and thereby to be effective no matter whether they are unseen or not. In general, we provide a promising perspective of formulating the generalization challenge into a task of cross-distribution data classification, devise and construct the preliminary detection network to discriminate fake faces at the microscopic-level semantics of images. Moreover, inspired and extended by the typical strategy in the domain of machine learning, we construct the domain adaptation component for better generalization on unseen Deepfakes.

4. Experiments

In this section, we first introduce the experimental settings, and then evaluate the capacity of FakeFilter to discriminate fake faces, together with its performance on unseen Deepfakes. Ultimately, we compare our work with other benchmark methods.

4.1. Experimental settings

4.1.1. Datasets

We evaluate this work with four typical datasets, and the samples of which are created with both deep learning-based and computer graphics-based methods. DeepFakeDetection (DFD)

To overcome the challenge of GAN-based Deepfake detection, we adopt the DeepFakeDetection dataset. Based on the FaceSwap-GAN implementation, Google AI 6

⁶
https://www.ai.google/

released this dataset, which involves over 3000 fake videos and 363 real ones recorded by 28 actors.

DeepFakes (DF)

In consideration of its extensive usage for evaluation, we adopt the DeepFakes dataset, the samples of which are created with the first generation of Autoencoder-based Deepfake creation. This dataset is included in the FaceForensic++ project [34], and contains 1000 real videos and 1000 fake ones.

Face2Face (F2F) and FaceSwap (FS)

To further evaluate FakeFilter on the fake faces created without deep learning, we adopt the Face2Face [40] and FaceSwap datasets. Those two datasets were included in the early version of the FaceForensic project [35], which employs two computer graphics-based methods to severally create 1000 fake videos with 1000 real ones.

4.1.2. Evaluation metrics and levels

The metrics of accuracy, EER (equal error rate) and AUC (area under the curve) are used as the performance indicators.

Accuracy

The proportion of correctly classified real and fake samples among the total number of samples.

EER

An equal ratio of false negative rate and false positive rate as varying the discrimination threshold. These two rates refer to the proportions of misclassified real or fake samples among the total real or fake samples, respectively.

AUC

The area under the receiver operating characteristic curve (ROC). ROC is a graphical plot created by plotting the true positive rate against the false positive rate at various threshold settings, where the former rate refers to the proportion of correctly classified real samples among the total real samples.

To comprehensively evaluate this work, the results are evaluated at both image and video levels. Image-level evaluation

The testing images are extracted by splitting an input video at regular intervals.

Video-level evaluation

The testing images are extracted by splitting a testing video into continuous frames, and the testing video is recorded fake if over 50 percent of its frames are classified as fake.

4.1.3. Data and platforms

Based on the above datasets, we prepare the videos and their corresponding images for model training and testing, and their statistics are shown in Table 1. In the table, the numbers on the left and right represent the quantity of videos and images respectively, and only the number of videos are represented in the last two columns for testing at the video level.

Table 1
Statistics of training and testing data

Dataset Training Testing

Image level Video level

Real Fake Real Fake Real Fake

DFD 330/120372 2100/177611 30/13186 80/36851 30 160

DF — 100/43037 30/12987 80/33545 30 160

F2F — 100/36585 30/12025 80/31248 30 160

Dataset	Training	Testing
DFD	330/120372	2100/177611	30/13186	80/36851	30	160
DF	—	100/43037	30/12987	80/33545	30	160
F2F	—	100/36585	30/12025	80/31248	30	160

Specifically, for the training sets, we adopt about 90% of real videos and 70% of fake ones in the DeepFakeDetection dataset, and 10% of fake videos in each of the DeepFakes and Face2Face datasets. For the testing sets, to ensure the fairness of every evaluation, we randomly select the same number of real and fake videos, i.e., 30 real videos and 80 fake videos, from the remaining videos of these three datasets, it is noteworthy that the testing videos are completely independent of the training ones. Moreover, the training images and the image-level testing ones are preprocessed and selected by splitting those videos at regular intervals, where the redundant images are discarded.

In addition, all the implementation and evaluation phases in this paper are carried out on a server with an i7-7820X 3.60 GHz CPU, a 64 GiB memory and three GeForce GTX 2080Ti graphics cards.

4.1.4. Benchmark methods

Among all the typical Deepfake detection methods, we apply the multi-task learning-based method [29] and ForensicTransfer [10] as the benchmark methods for comparison, as introduced in Section 2.

Specifically, we make the decision of employing the above two benchmark methods based on fourfold reasons. Firstly, promising results are achieved by them in terms of accuracy, e.g., the accuracy of 94.47% and 90.00% on seen and unseen Deepfakes are separately achieved by ForensicTransfer. Secondly, they focus on exactly the same goals as we are, as far as we know, among all existing deep neural network-based detection methods, only these two methods explicitly address the generalization issue with promising results. Thirdly, in their methods, they have devised deep neural networks for fake face classification, and apply two typical strategies to improve their generalization on unseen Deepfakes respectively, i.e., multi-task learning [46] and transfer learning [31], such routes of solutions are similar to ours. Finally, they are widely discussed and cited by numerous research [27,34] and have become two of the most typical methods in the field of Deepfake detection.

Overall, the multi-task learning-based method and ForensicTransfer share the same goal and similar solution routes to us, achieve promising results as well as attract wide attentions, and are thus most suitable as benchmark methods for comparison.

4.2. Evaluation of the performance on discrimination

To verify the capacity of FakeFilter to discriminate fake faces, we first evaluate the performance of the preliminary detection network. Specifically, we preprocess and select the real and fake images only from the DeepFakeDetection dataset to train the preliminary detection network, and further evaluate its performance on multiple datasets.

In the testing phase, we first preprocess the testing videos from the DeepFakeDetection, DeepFakes and Face2Face datasets, where the latter two are taken as unseen Deepfakes. Subsequently, we randomly select the testing images for the image-level and video-level evaluations, and input the extracted facial images into the preliminary detection network, the accuracy, EER and AUC are provided in Table 2.

Table 2
Evaluation of the performance on discrimination

Dataset Image level Video level

Training Testing Accuracy EER AUC Accuracy EER AUC

DFD DFD 94.00% 6.02% 98.56% 90.60% 11.36% 88.64%

DFD DF 81.34% 16.10% 89.46% 71.82% 19.39% 80.61%

DFD F2F 60.57% 37.53% 68.28% 43.51% 39.49% 60.51%

Dataset	Image level	Video level
DFD	DFD	94.00%	6.02%	98.56%	90.60%	11.36%	88.64%
DFD	DF	81.34%	16.10%	89.46%	71.82%	19.39%	80.61%
DFD	F2F	60.57%	37.53%	68.28%	43.51%	39.49%	60.51%

In the table, for the samples from the DeepFakeDetection dataset, the accuracy of 94.00% and 90.60% are achieved at the image and video levels respectively, which indicates that the preliminary detection network is qualified for discriminating fake faces, whether they are from images or videos. Moreover, the EER of 6.02% and 11.36% are achieved at the image and videos respectively, which indicates the preliminary detection network seldom misses a fake sample or misclassifies a real sample, and the promising AUC of 98.56% and 88.64% indicates its performance is almost unaffected by the difference in the quantity of real and fake samples. The promising results in terms of EER and AUC further confirms the reliability of the preliminary detection network.

However, observed from the table, as encountering the samples from the DeepFakes and Face2Face datasets, the performance of the preliminary detection network is significantly degraded, which indicates that it is currently difficult for the preliminary network to detect unseen Deepfakes. Especially, the accuracy evaluated at the Face2Face dataset is merely 60.57% and 43.51% at the image and video levels, respectively. Compared with the Deepfakes in the DeepFakes dataset created with a deep learning-based method, the Deepfakes in the Face2Face dataset are created with a computer graphics-based one, and perform greater distribution difference from the training set. Therefore, as encountering the unseen Deepfakes are created with methods that are far from the training set, the preliminary network is even forceless to deal with them, which confirms the challenge addressed in this paper.

4.3. Evaluation of the performance on generalization

To evaluate the generalization of FakeFilter, the domain adaptation component is augmented to deal with unseen Deepfakes. Specifically, we apply the fake images from the DeepFakeDetection dataset to form the source domain, and the fake images from the DeepFakes or Face2Face dataset to form the target domain. The images in the target domain are taken as unseen Deepfakes, thus they are randomly selected and unlabelled. Further, these images from both the source and target domains are incorporated into training the domain adaptation component. It is observed that a small amount of unseen Deepfakes are utilized for domain adaptation, which is expected to enable FakeFilter to be equipped to detect all of such Deepfakes.

In the testing phase, we still utilize the same videos and images collected in Section 4.2, i.e., the testing images are preprocessed and selected from the DeepFakeDetection, DeepFakes and Face2Face datasets. Subsequently, we input the extracted facial images into FakeFilter and provide the accuracy, EER and AUC in Table 3.

To compare the results in Table 2 and Table 3, the performance of FakeFilter is significantly improved when encountering unseen Deepfakes in the DeepFakes and Face2Face datasets. For instance, observed from Table 3, the image-level accuracy of 92.47% and 83.38% are achieved at the DeepFakes and Face2Face datasets respectively, which are increased by 11.13% and 22.81% compared with the corresponding results in Table 2. The promising accuracy indicates that FakeFilter generalizes well to unseen Deepfakes. Moreover, the EER and AUC are also significantly improved, for instance, the image-level EER and AUC are respectively improved by 20.59% and 22.75% evaluated at the Face2Face dataset, which further verifies the reliability of FakeFilter. Especially, in practice, the obtained real and fake samples are commonly with uncertain ratio, the promising AUC demonstrates FakeFilter is reliable in different settings.

Table 3
Evaluation of the performance on generalization

Dataset Image level Video level

Source Target Testing Accuracy EER AUC Accuracy EER AUC

DFD DF DF 92.47% 8.65% 97.09% 93.08% 6.70% 93.30%

(11.13%↑) (7.45%↓) (7.63%↑) (21.26%↑) (12.69%↓) (12.69%↑)

DFD F2F F2F 83.38% 16.58% 91.03% 80.92% 23.78% 76.22%

(22.81%↑) (20.59%↓) (22.75%↑) (37.41%↑) (15.71%↓) (15.71%↑)

Dataset	Image level	Video level
DFD	DF	DF	92.47%	8.65%	97.09%	93.08%	6.70%	93.30%
(11.13%↑)	(7.45%↓)	(7.63%↑)	(21.26%↑)	(12.69%↓)	(12.69%↑)
DFD	F2F	F2F	83.38%	16.58%	91.03%	80.92%	23.78%	76.22%
(22.81%↑)	(20.59%↓)	(22.75%↑)	(37.41%↑)	(15.71%↓)	(15.71%↑)

In general, the experimental results are basically consistent with expectations. Specifically, faced with unseen Deepfakes, FakeFilter is equipped to detect them in terms of the promising accuracy, EER and AUC evaluated at both image and video levels. Moreover, the DeepFakes and Face2Face datasets are created with deep learning-based and computer graphics-based methods respectively. Based on the observations in Section 4.2, the Deepfakes from the Face2Face dataset are more challenging to be detected than the ones from the DeepFakes dataset, owing to their creation methods. The experimental results further confirm that FakeFilter applies to any unseen Deepfakes regardless of the methods to create them.

In addition, we further analyze the results by visualizing the distribution of the mapped features. Specifically, we input 1000 fake images from each of the DeepFakeDetection and Face2Face datasets to FakeFilter before and after domain adaptation, and visualize the activation of the top feature extractor layer with the Principal Component Analysis (PCA) method [1]. As shown in Fig. 4, the distributions of two types of features get much closer after domain adaptation, which indicates similar features are extracted from these two domains of Deepfakes, i.e., the unseen Deepfakes are deemed equally to the seen ones. Although the feature distributions between the source-domain and target-domain samples are not compared by calculated values, owing to the uncertainty caused by the high-dimensional features, it can also be observed from the visualization results that the overlap between the two distributions is positively related to the accuracy for the target domain, which indicates that the idea towards learning domain-invariant features is a feasible strategy to detect unseen Deepfakes.

Fig. 4.

Visualization of the mapped feature distributions. The activation of the top feature extractor layer before and after domain adaptation are visualized in (a) and (b) respectively, where the distributions of the two types of Deepfakes are represented by purple (source domain) and yellow (target domain) points.

4.4. Comparison with benchmark methods

To further evaluate the effectiveness of FakeFilter, we compare it with two state-of-the-art benchmark methods, i.e., the multi-task learning-based method and ForensicTransfer as described in Section 4.1.

To compare with the multi-task learning-based method, we apply exactly the same training and testing data provided by the authors with the same configurations as in [29], and compare FakeFilter with the benchmark method in terms of both capacities of fake face discrimination and generalization. Specifically, 0.28 million real and fake images of 1408 videos from the Face2Face dataset are applied to train the preliminary detection network. Moreover, 1000 fake images of 100 fake videos from the DeepFakes dataset are applied as unseen Deepfakes, which aids FakeFilter to train the domain adaptation component. In the testing phase, 2800 real and fake images of 280 videos from the DeepFakes dataset are utilized, and the performance are evaluated with the image-level accuracy.

To compare with ForensicTransfer, we apply the same configurations utilized in [10], and compare FakeFilter with the benchmark method in terms of fake face discrimination and generalization capacities. Specifically, 0.14 million real and fake images of 700 randomly selected videos from the Face2Face dataset are applied to train the preliminary detection network, and 1000 fake images from the DeepFakes dataset are applied as unseen Deepfakes to aid FakeFilter to learn the domain adaptation component. In the testing phase, 1500 real and fake images of 150 videos from the DeepFakes dataset are utilized, and the performance are evaluated with the image-level accuracy.

Table 4
Comparison with benchmark methods

Method Dataset Image level

Source Target Testing Accuracy

Multi-task learning F2F – F2F 92.77%

F2F FS FS 83.71%

FakeFilter F2F – F2F 91.54%

F2F FS FS 89.89%

ForensicTransfer F2F – F2F 94.47%

F2F FS FS 90.00%

FakeFilter F2F – F2F 94.63%

F2F FS FS 89.84%

Method	Dataset	Image level
Multi-task learning	F2F	–	F2F	92.77%
F2F	FS	FS	83.71%
FakeFilter	F2F	–	F2F	91.54%
F2F	FS	FS	89.89%
ForensicTransfer	F2F	–	F2F	94.47%
F2F	FS	FS	90.00%
FakeFilter	F2F	–	F2F	94.63%
F2F	FS	FS	89.84%

Subsequently, we present the classification results in Table 4. To compare with the benchmark methods, each classification result of FakeFilter, i.e., the bold values in the table, has passed the t-test significance test with p-value < 0.05. As the results, when compared with the multi-task learning-based method, an accuracy improvement of 6.18% is achieved by FakeFilter on generalizing to unseen Deepfakes, and approximate comparable performance is achieved by them on detecting seen ones. In reality, the multi-task learning is indeed one of the effective strategies to address the generalization challenge, and focuses on the generalization of cross-task samples [46]. In contrast, the strategy of domain adaptation is suitable for the generalization of cross-distribution samples, which is more consistent with Deepfake samples and the task of Deepfake detection, and therefore achieves better performance. In addition, when compared with ForensicTransfer, FakeFilter achieves comparable accuracy on detecting both seen and unseen Deepfakes.

In general, compared with these two state-of-the-art methods, FakeFilter performs better than the multi-task learning-based method, and is evenly matched with ForensicTransfer in terms of accuracy, which can further confirm that FakeFilter can effectively classify fake faces and generalizes well to unseen Deepfakes. Furthermore, these two benchmark methods conduct their experiments with the FaceSwap and Face2Face datasets, the fake images of which are both created with the computer graphics-based methods. In contrast, FakeFilter is verified as effective in dealing with both computer graphics-based Deepfakes, as well as the challenging deep learning-based ones, which has made up for the limitations of these benchmark methods.

In addition, similar to these two benchmark methods, a small amount of target-domain fake samples are still required to be incorporated into training the domain adaptation component. In reality, such parameter updates are light and fine-tuned, which makes FakeFilter remember a certain type of unseen Deepfakes quickly through a small amount of them, so as to deal with all of such incoming Deepfakes in the future. In the real world, we might encounter some newfangled Deepfakes created with a newly designed method, whose creation method is blind but a small of them can be obtained or created. In this case, it is necessary to enable FakeFilter to be equipped to detect more such incoming Deepfakes, so as to take precautions before the disaster happens. Therefore, the scenario settings in this paper are in line with the application scenarios in practice, and thus FakeFilter can be applied to the applications in the real world.

5. Conclusion

With the continuous emergence of new methods to create Deepfakes, various detection methods have been proposed as the responses to the risk, which makes Deepfake detection a trendy research topic in digital multimedia forensics research. Although promising results in Deepfake detection have been achieved by various deep neural networks, their performance will be greatly degraded upon the encountering of Deepfakes created with different methods from the training sets, which results in the poor generalization on unseen Deepfakes, and it remains a challenge for Deepfake detection methods.

In this paper, to address the above challenge of unseen Deepfake detection, we propose FakeFilter system with three main contributions:

We attempt this challenge from a promising perspective, by formulating it as a task of cross-distribution data classification, i.e., to classify multifold Deepfakes with different feature distributions.

We devise a preliminary detection network of FakeFilter to extract discriminative features and classify real and fake faces at the microscopic-level of images.

We address the issue of cross-distribution Deepfake detection with the strategy of domain adaptation, which enables FakeFilter to deem different distributions of Deepfakes as uniform, and thereby to achieve comparable performance on seen Deepfakes as well as unseen ones.

Our experimental results confirm that FakeFilter can effectively detect Deepfakes no matter whether they are unseen or not, or whether those Deepfakes are created with deep learning-based or computer graphics-based methods.

Although promising results are achieved by FakeFilter, a small amount of unseen Deepfakes are still necessary to be incorporated into the training process. In the future, we will further investigate how to leverage few or even no unseen Deepfakes to achieve comparable results.

Footnotes

Acknowledgments

This work is supported by National Natural Science Foundation of China (No. 61572469) and Youth Innovation Promotion Association CAS (No.2021155).

We would like to thank the anonymous reviewers whose suggestions have helped us to improve the quality of this paper.

References

Abdi and

L.J.

Williams, Principal component analysis, Wiley interdisciplinary reviews: computational statistics 2(4) (2010), 433–459.

Afchar,

Nozick,

Yamagishi and

Echizen, Mesonet: A compact facial video forgery detection network, in: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), 2018, pp. 1–7. doi:10.1109/WIFS.2018.8630761.

Agarwal,

Farid,

Gu,

He,

Nagano and

Li, Protecting world leaders against deep fakes, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2019, pp. 38–45.

Badrinarayanan,

Kendall and

Cipolla, Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE transactions on pattern analysis and machine intelligence 39(12) (2017), 2481–2495. doi:10.1109/TPAMI.2016.2644615.

Ben-David,

Blitzer,

Crammer,

Kulesza,

Pereira and

J.W.

Vaughan, A theory of learning from different domains, Machine learning 79(1–2) (2010), 151–175. doi:10.1007/s10994-009-5152-4.

Ben-David,

Blitzer,

Crammer and

Pereira, Analysis of representations for domain adaptation, in: Advances in Neural Information Processing Systems, 2007, pp. 137–144. doi:10.7551/mitpress/7503.003.0022.

Carreira and

Zisserman, Quo vadis, action recognition? A new model and the kinetics dataset, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 6299–6308.

Chesney and

Citron, Deep fakes: A looming challenge for privacy, democracy, and national security, SSRN Electronic Journal (2019). doi:10.2139/ssrn.3213954.

Chollet, Xception: Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1251–1258.

10.

Cozzolino,

Thies,

Rössler,

Riess,

Nießner and

Verdoliva, Forensictransfer: Weakly-supervised domain adaptation for forgery detection, arXiv preprint, 2018, arXiv:1812.02510.

11.

Dang,

Liu,

Stehouwer,

Liu and

A.K.

Jain, On the detection of digital face manipulation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 5781–5790.

12.

Dolhansky,

Howes,

Pflaum,

Baram and

C.C.

Ferrer, The Deepfake detection challenge (DFDC) preview dataset, 2019, arXiv preprint arXiv:1910.08854.

13.

Fridrich, Digital image forensics, IEEE Signal Processing Magazine 26(2) (2009), 26–37. doi:10.1109/MSP.2008.931078.

14.

Ganin and

Lempitsky, Unsupervised domain adaptation by backpropagation, in: Proceedings of the 32nd International Conference on Machine Learning (ICML), pp. 1180–1189.

15.

Ganin,

Ustinova,

Ajakan,

Germain,

Larochelle,

Laviolette,

Marchand and

Lempitsky, Domain-adversarial training of neural networks, The Journal of Machine Learning Research 17(1) (2016), 1–35.

16.

Goodfellow,

Pouget-Abadie,

Mirza,

Xu,

Warde-Farley,

Ozair,

Courville and

Bengio, Generative adversarial nets, in: Advances in Neural Information Processing Systems, 2014, pp. 2672–2680.

17.

Güera and

E.J.

Delp, Deepfake video detection using recurrent neural networks, in: 2018 15th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2018, pp. 1–6. doi:10.1109/AVSS.2018.8639163.

18.

Han,

Morariu,

P.I.

Larry Davis et al., Two-stream neural networks for tampered face detection, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017, pp. 19–27.

19.

Hara,

Kataoka and

Satoh, Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6546–6555.

20.

Ioffe and

Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift 37 (2015), 448–456.

21.

Isola,

J.-Y.

Zhu,

Zhou and

A.A.

Efros, Image-to-image translation with conditional adversarial networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 1125–1134. doi:10.1109/CVPR.2017.632.

22.

Jung,

Kim and

Kim, DeepVision: Deepfakes detection using human eye blinking pattern, IEEE Access 8 (2020), 83144–83154. doi:10.1109/ACCESS.2020.2988660.

23.

D.E.

King, Dlib-ml: A machine learning toolkit, The Journal of Machine Learning Research 10 (2009), 1755–1758.

24.

Korshunov and

Marcel, Deepfakes: A new threat to face recognition? Assessment and detection, 2018, arXiv preprint arXiv:1812.08685.

25.

Krizhevsky,

Sutskever and

G.E.

Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in Neural Information Processing Systems, 2012, pp. 1097–1105. doi:10.1145/3065386.

26.

Li and

Lyu, Exposing Deepfake videos by detecting face warping artifacts, 2018, arXiv preprint arXiv:1811.00656.

27.

Li,

Yang,

Sun,

Qi and

Lyu, Celeb-df: A large-scale challenging dataset for DeepFake forensics, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 3204–3213. doi:10.1109/CVPR42600.2020.00327.

28.

Matern,

Riess and

Stamminger, Exploiting visual artifacts to expose deepfakes and face manipulations, in: 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW), 2019, pp. 83–92. doi:10.1109/wacvw.2019.00020.

29.

H.H.

Nguyen,

Fang,

Yamagishi and

Echizen, Multi-task learning for detecting and segmenting manipulated facial images and videos, in: 2019 IEEE International Conference on Biometrics Theory, Applications and Systems (BTAS), 2019, pp. 1–8. doi:10.1109/BTAS46853.2019.9185974.

30.

H.H.

Nguyen,

Yamagishi and

Echizen, Use of a capsule network to detect fake images and videos, 2019, arXiv preprint arXiv:1910.12467.

31.

S.J.

Pan and

Yang, A survey on transfer learning, IEEE Transactions on knowledge and data engineering 22(10) (2010), 1345–1359. doi:10.1109/TKDE.2009.191.

32.

O.M.

Parkhi,

Vedaldi and

Zisserman, Deep face recognition, in: Proceedings of the British Machine Vision Conference, 2015, pp. 41.1–41.12. doi:10.5244/C.29.41.

33.

Pérez,

Gangnet and

Blake, Poisson image editing, ACM Transactions on Graphics 22(3).

34.

Rössler,

Cozzolino,

Verdoliva,

Riess,

Thies and

Nießner, Faceforensics++: Learning to detect manipulated facial images, in: Proceedings of the IEEE International Conference on Computer Vision, 2019, pp. 1–11. doi:10.1109/ICCV.2019.00009.

35.

Rössler,

Cozzolino,

Verdoliva,

Riess,

Thies and

Nießner, Faceforensics: A large-scale video dataset for forgery detection in human faces, 2018, arXiv preprint arXiv:1803.09179.

36.

Sabour,

Frosst and

G.E.

Hinton, Dynamic routing between capsules, 2017, arXiv preprint arXiv:1710.09829.

37.

Schroff,

Kalenichenko and

Philbin, Facenet: A unified embedding for face recognition and clustering, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815–823. doi:10.1109/CVPR.2015.7298682.

38.

Srivastava,

Hinton,

Krizhevsky,

Sutskever and

Salakhutdinov, Dropout: A simple way to prevent neural networks from overfitting, The journal of machine learning research 15(1) (2014), 1929–1958.

39.

Szegedy,

Liu,

Jia,

Sermanet,

Reed,

Anguelov,

Erhan,

Vanhoucke and

Rabinovich, Going deeper with convolutions, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1–9.

40.

Thies,

Zollhofer,

Stamminger,

Theobalt and

Nießner, Face2face: Real-time face capture and reenactment of rgb videos, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2387–2395. doi:10.1109/CVPR.2016.262.

41.

Tolosana,

Vera-Rodriguez,

Fierrez,

Morales and

Ortega-Garcia, DeepFakes and beyond: A survey of face manipulation and fake detection, Information Fushion 64 (2020), 131–148. doi:10.1016/j.inffus.2020.06.014.

42.

Tomar, Converting video formats with FFmpeg, Linux Journal 2006(146) (2006), 10.

43.

Wang,

Jiao,

Yin,

Zhao,

Han and

Sun, Underwater sonar image classification using adaptive weights convolutional neural network, Applied Acoustics 146 (2019), 145–154. doi:10.1016/j.apacoust.2018.11.003.

44.

Yang,

Li and

Lyu, Exposing deep fakes using inconsistent head poses, in: ICASSP 2019 – 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2019, pp. 8261–8265. doi:10.1109/ICASSP.2019.8683164.

45.

Zhang,

Li and

Qiao, Joint face detection and alignment using multitask cascaded convolutional networks, IEEE Signal Processing Letters 23(10) (2016), 1499–1503. doi:10.1109/LSP.2016.2603342.

46.

Zhang and

Yang, A survey on multi-task learning, 2017, arXiv preprint arXiv:1707.08114.

47.

J.-Y.

Zhu,

Park,

Isola and

A.A.

Efros, Unpaired image-to-image translation using cycle-consistent adversarial networks, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2223–2232. doi:10.1109/ICCV.2017.244.

FakeFilter: A cross-distribution Deepfake detection system with domain adaptation

Abstract

Keywords

1. Introduction

1 https://apps.apple.com/cn/app/id413501652

2.1. Background

3 https://github.com/deepfakes/faceswap

2.2.1. Visual feature-based methods

2.2.2. Deep neural network-based methods

2.3. Summary

3. Cross-distribution Deepfake detection system

3.1. Preliminary detection network

3.1.2. Feature extractor

3.1.3. Fake classifier

3.2. Domain adaptation component

3.2.1. Domain classifier

3.2.2. Performance analysis

4. Experiments

4.1. Experimental settings

4.1.1. Datasets

6 https://www.ai.google/

4.1.3. Data and platforms

Table 1 Statistics of training and testing data Dataset Training Testing Image level Video level Real Fake Real Fake Real Fake DFD 330/120372 2100/177611 30/13186 80/36851 30 160 DF — 100/43037 30/12987 80/33545 30 160 F2F — 100/36585 30/12025 80/31248 30 160

4.2. Evaluation of the performance on discrimination

Table 2 Evaluation of the performance on discrimination Dataset Image level Video level Training Testing Accuracy EER AUC Accuracy EER AUC DFD DFD 94.00% 6.02% 98.56% 90.60% 11.36% 88.64% DFD DF 81.34% 16.10% 89.46% 71.82% 19.39% 80.61% DFD F2F 60.57% 37.53% 68.28% 43.51% 39.49% 60.51%

Footnotes

Acknowledgments

References

¹
https://apps.apple.com/cn/app/id413501652

³
https://github.com/deepfakes/faceswap

⁶
https://www.ai.google/

Table 1
Statistics of training and testing data

Dataset Training Testing

Image level Video level

Real Fake Real Fake Real Fake

DFD 330/120372 2100/177611 30/13186 80/36851 30 160

DF — 100/43037 30/12987 80/33545 30 160

F2F — 100/36585 30/12025 80/31248 30 160

Table 2
Evaluation of the performance on discrimination

Dataset Image level Video level

Training Testing Accuracy EER AUC Accuracy EER AUC

DFD DFD 94.00% 6.02% 98.56% 90.60% 11.36% 88.64%

DFD DF 81.34% 16.10% 89.46% 71.82% 19.39% 80.61%

DFD F2F 60.57% 37.53% 68.28% 43.51% 39.49% 60.51%