Project Paper

Deepfake detection through PRNU and logistic

regression analyses
July 5, 2020
Assessor:
Prof. dr. ir. Cees de Laat
[email protected]
Catherine de Weever
[email protected]
University of Amsterdam
UvA ID: 12782750 Supervisor:
ing. Sebastian Wilczek Prof. dr. ing. Zeno Geradts
[email protected] [email protected]
UvA ID: 12837067 Netherlands Forensic Institute (NFI)
Course:
Research Project 2
 Deepfake detection through PRNU and logistic regression analyses
Project Paper

Abstract
Deepfakes, as in creating a video of an impersonated target by replacing the face of an actor with the face of
said target, have evolved in recent years to reach convincing levels of perceived authenticity. Generated forged
imagery of faces has become so advanced that it is getting more complicated to differentiate a manipulated video
from an original by watching it. If Deepfakes can not be told apart from original videos, undesirable effects, such
as media distrust or forensic misevaluation, may occur. Therefore, Deepfake detection methods are required to
ensure that one can mitigate potentially devastating effects of fake media. In this paper, we consider multiple
approaches to distinguish between authentic and Deepfake videos. Among the approaches are Photo Response
Non Uniformity (PRNU), visual artifact, and frequency domain analysis, three methods that do not make use of
neural nets. We discovered that none of the methods show definite proof of Deepfake presence or absence, and even
discovered detection evasion methods. As an alternative, we propose a conceptual system to authenticate original
media instead of discovering Deepfake forgeries.
Keywords— Deepfake, Detection, PRNU, Visual Artifacts, Media Authenticity

1 Introduction anymore, the presentation of a video introduces reasonable

doubt. For example, a video of someone committing a crime
The process known as Deepfake is an emerging technology with could be argued against in a court of law since the person filmed
the goal of replacing the likeness of a person with the portrait might be faked. The same applies to social life and politics. If
of someone else, both visually and audibly. A perfect Deepfake a video of a known person is shared, how can be ensured that
would be indistinguishable from an original representation of the person said or did what is shown in the video?
the impersonated target. In this research paper, we investigate various detection meth-
Especially the concept of replacing a person within a video file ods of Deepfakes. To do so, we consider the evaluation of Photo
has recently gained traction. It is becoming an increasingly Response Non Uniformity (PRNU) patterns as well as the anal-
common practice [6] to take a video or a set of photos of a ysis of video artifacts and frequency domain. Furthermore,
target’s face, projecting them on another, existing video. we consider methods that may assist in Deepfake detection.
Deepfakes are seeing an increasing level of accessibility. While Lastly, we examine an alternative approach of authenticating
not entirely arbitrary to operate, software such as Faceswap original media instead of detecting previously synthesized pen-
[11] and DeepFaceLab [29] enable the creation of faked video dants.
frames with relative ease. Given that one can create Deepfakes
with limited knowledge of the subject, a few concerning, if not
malicious, use cases are enabled through Deepfake. 2 Research Questions
Publications exist of Deepfakes and their creation, impersonat-
ing famous people such as celebrities and politicians. A notable To research the topic, we defined research questions that are
example is a video of the former President of the United States answered by the results of this research. The main question
of America, Barack Obama, which was faked both visually and answered is the following:
audibly [12]. Another example shows the actor Keanu Reeves How can a forged Deepfake video be differ-
actively committing a crime by breaking the perpetrator’s neck entiated from an authentic one, for forensic
during a store robbery [16]. While these publications mostly purposes?
gained traction due to the public interest in the technology,
the shared videos are nonetheless convincing in their content. We divided the aforementioned question into the following sub-
Given that the means of production are free or open-source, one questions:
can assume that they may be used to spread misinformation 1. What detection methods are already avail-
on purpose or to commit fraud. able?
Furthermore, one widespread use case for Deepfakes are adult
videos. Plenty of videos shared on the Internet contain porno- 2. Are these detection methods still applicable
graphic content with the faces of celebrities superimposed on to modern Deepfakes?
the original video [7]. Such videos already have a substantial 3. If these methods are still applicable, can they
impact on both the reputation and mental health of the target be enhanced?
person. Initially, a user of the platform Reddit with the name 4. If these methods are not applicable anymore,
”deepfakes” published the concept. The user later turned it what other approaches could be taken?
into a full platform for sharing creations and resources. The
name of this user is also the etymological origin of the name
Deepfake. 3 Related Work
While one could argue that celebrities are already dealing with
similar issues due to their inherent public life, this technology There are different methods to detect Deepfakes. According
could also apply to private people, given a sufficiently large set to [25], we can divide these methods into three main cat-
of data. There is plenty of room for criminal and civil misde- egories: physical/physiological, signal-level, and data-driven.
meanor, both in intentional fraud and false information spread The methods that fall under the physical/physiological cat-
as well as reputational and personal attacks. egory detect the inconsistencies found in the physical/physi-
Given the potential misuse of Deepfakes, the detection of such ological aspects in Deepfake videos. For example, in [22], a
forgeries becomes a requirement for the future, considering the new method was introduced that makes use of a physiological
legal and social aspects of life. If Deepfakes become so ad- signal, such as eye blinking, while in [33], they made use of
vanced that they can not be distinguished from other media inconsistent head poses.

Catherine de Weever, ing. Sebastian Wilczek page 2 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

The methods that fall under the signal-level category utilize Figure 1: Overview of a Generative Adversarial Network
Deepfake detection algorithms that use signal-level artifacts
introduced during the synthesis process. A few examples of The job of the generator is to introduce new data to the data
these methods are explained in [23] and [35]. set, in this case, a video file frame containing the face or other
The methods that fall under the data-driven category make use to-be-faked likeness of a target. It makes use of random input
of various types of Deep Neural Networks (DNNs) trained on vectors that are influenced by a weight value. The discrimina-
real and Deepfake videos to capture specific artifacts. These tor compares the newly generated frame to an original set of
methods are introduced in [2], [17] and [28]. frames. In the case of Deepfake, those may be video frames
The methods mentioned above make use of DNNs, but there of the target. GAN can also be used in other contexts, for
are also non-neural network methods. In [26], a new method example, to create pictures of people that do not exist [34].
is discussed to detect Deepfakes by exploiting visual artifacts. In this case, the discriminator would compare the generated
This method makes use of the visual features of the eyes, teeth, image to the images of living people. The discriminator cal-
and face to detect Deepfakes. It provides two variants: a small culates the likelihood of the generated picture being genuine,
neural network and a logistic regression model. We made use of non-generated, and returns this loss value. This result propa-
this method in our experiments. From their experiments, they gates back to the generator, which can adjust its weight vectors
were able to achieve an Area under the ROC Curve (AUC) accordingly until the discriminator is sufficiently satisfied. The
score of 0.866. We explain the term AUC in 4 Background. exchange between the two adversarial networks creates contin-
Another category that is not discussed by [25] is detecting uous improvement in the creation of synthetic media.
Deepfakes by looking at the image and video characteristics.
One method that falls under this category is introduced in [19].
This method makes use of PRNU analysis to detect Deepfake 4.2 PRNU Patterns
videos. We also detail PRNU patterns in the following sec-
As previously mentioned, we made use of PRNU patterns to
tion. This analysis makes use of the mean normalized cross-
detect Deepfakes. One can consider the term PRNU as the
correlation to distinguish Deepfakes from real authentic videos.
fingerprint of a digital camera. PRNU patterns are fixed-noise
We also used this method in our experiments.
patterns that the sensor of a camera creates when it converts
Another method that falls under this category is introduced in
the light captured during a photograph or a video recording to
[9], which is based on a classical frequency domain analysis fol-
digital information. This happens because of the inhomogene-
lowed by a basic classifier. They stated and demonstrated that
ity present in the silicon used by the camera sensor. The slight
their approach does not need a large amount of data to achieve
variations of the material lead to responding variations in the
reasonable accuracy. They achieved an accuracy of 100% using
quantum efficiency of each pixel of the sensor, meaning each
20 annotated samples. Because of the high accuracy scores, we
pixel of the sensor converts photons to electrons slightly dif-
decided to use this method in our experiments.
ferent. The result is a two-dimensional pattern with the same
dimensions as the sensor of the capturing camera. [13]
4 Background Given a PRNU pattern of a camera, one can analyze the like-
lihood of an image originating from the given camera source.
In this section, we provide certain background information, to If one can extract the pattern from the image in question, one
put our work into the proper context, and to ensure that our can compute how much the extracted pattern and the pattern
methodology and experiments can be understood. We detail from the considered camera correlate. The higher this value,
the creation process of Deepfakes, the usage of camera-unique the more likely the image originates from the evaluated camera.
patterns, and logistic regression. [5]

4.1 Generative Adversarial Networks 4.3 Logistic Regression

Deepfake operates by making use of a Generative Adversarial Logistic regression is a classification algorithm used to predict
Network (GAN). The term GAN and its underlying concept the probability of a target variable. It is a method used for
were first introduced by Ian Goodfellow, working for the Brain binary classification problems. According to [8] ”The idea of
research team of Google LLC and the University of Montreal logistic regression is to make linear regression produce probabil-
[14]. In its essence, a GAN contains two parts: a generator ities.” In this case, these are class probabilities. This is done
and a discriminator. These are sometimes also referenced as by taking the familiar linear model and giving it to a sigmoid
an encoder and a decoder. Both are neural networks or a sim- function with the use of the following equation [15]:
ilar technology that can generate and distinguish information
based on a set of inputs. Figure 1 shows an overview of a basic
GAN.
y0 = 1
1+e−(z)

where:
• y’ is the output of the logistic regression model
• z is the log odds function: b + w1 x1 + w2 x2 + ... + wN wN

– w values are model’s learned weights

– b is the bias
– x values are feature values.

Catherine de Weever, ing. Sebastian Wilczek page 3 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

The output value will be between 0 and 1. This value is we used both of the models provided.
the probability score. A classification threshold is defined to Analyzing the frequency domain of an image is another way to
map this value to a binary category. One can use the met- detect Deepfakes. This method is called the Frequency Domain
ric accuracy to evaluate a classification model’s predictions. Analysis [9]. The input for this method has some requirements.
Accuracy is the chunk of predictions that the model got cor- For our experiment, we first needed to ensure that our data set
rect. Accuracy is used when the best classification threshold meets these requirements. We used the extracted frames, ran a
is known. The metric (receiver operating characteristic curve face detection on them, and made them squared. This method
(ROC) curve) is used when one evaluates the model across required fake and original videos. The output of this method is
many different possible classification thresholds. Under this an accuracy value. If the accuracy value of the video is higher
curve, there is the Area Under the ROC curve (AUC). The than 0.5 than we classify it as a Deepfake.
AUC gives an aggregate measure of the performance aggre- We also looked into a way to evade the previously mentioned
gated across all possible classification thresholds. AUC ranges detection methods. For the detection methods that make use
from 0 to 1. An AUC score of 0.0 means that the models’ of a logistic regression model, we looked into an adversarial at-
predictions are 100% wrong, while an AUC score of 1.0 means tack using Fast Gradient Sign Method (FGSM). This part of
that its predictions are 100% correct. the research is abstract: our findings are theoretical, and they
reference previous work. We discovered an evasion method for
PRNU analysis during our experiments.
5 Approach and Methodology Given the results of the previous topics, we also considered the
possibilities of authenticating original media. Given the time
In this section, we detail the approaches we have taken through- allocated for this project and the extensive scope of the topic,
out the research. This includes the different considered detec- we decided to keep this part of the research abstract. All find-
tion methods as well as ways to evaluate their success. ings in this paper about media authentication are theoretical,
Please note that most experiments detailed in this paper do referencing previous work in the same field.
not make use of neural networks. This decision is intentional
and part of the scope of this project. Due to time and resource
constraints, we were unable to execute proper experiments in-
volving machine learning. Instead, we opted to consider any
6 Experiments
methodology that does not make use of neural networks for the
In this section, we introduce the experiments we conducted
most part, as detailed below. We only used neural networks
throughout the project. The structure of this section follows
and machine learning to create Deepfakes for our experiments
the different approaches detailed in 5 Approach and Methodol-
and where neural network approaches were detailed as part of
ogy. We define the setup of our experiments, where and how we
any related work. However, we only used these approaches if
retrieved data and information to experiment with, and how
that related work proposed a similar solution that is not using
we executed each experiment. Each experiment aimed to either
neural networks.
retrieve data for other experiments or to evaluate a method to
Previous research into the topic of Deepfake detection showed
detect Deepfakes.
that the analysis of PRNU patterns indicated the presence or
absence of a Deepfake [19]. However, the creation of Deep-
fakes has advanced since the paper was published. Therefore, 6.1 Data Set Retrieval
we reevaluated whether PRNU analysis is still applicable to
modern Deepfakes. To perform detection experiments, we required a sufficiently
To do so, we retrieved and created multiple Deepfakes to eval- large set of Deepfakes. While we created some Deepfakes our-
uate. We divided these Deepfakes into individual frames and selves (see 6.2 Deepfake Creation), the time allocated for the
extracted the PRNU pattern from each frame to perform cross- project did not allow us to create a large number. Therefore,
correlation computations with the extracted patterns of other we made use of two different Deepfake data sets: FaceForen-
files. If the files originated from the same source, there should sics++ [30] and Celeb-DF [24]. We decided to use these two
be a high cross-correlation. Consequently, the computed value sets because of their large amount of files available and their
should be lower for files that one has tampered with using widespread usage in Deepfake detection research.
Deepfakes. We used this computed cross-correlation as an in- FaceForensics++ is a data set published by Google LLC and
dication of detection success. the Technical University of Munich. It consists of 1000 original
As mentioned in the related work section, another way to de- videos containing front-facing faces of humans. Furthermore,
tect Deepfake is to make use of the visual features of the eyes, it includes multiple faked videos, using different tamper meth-
teeth, and face, as in visual artifacts[26]. This method is called ods, one of them being Deepfake. We had to request access to
the visual artifact analysis. We implemented this method and the data set through the Technical University of Munich, which
reevaluated whether we could use it with our retrieved and we did before our experiments. We downloaded the data set to
created data set. This method only accepts images as input, a server of the Security and Network Engineering course at the
so we first needed to extract the frames from the videos. It University of Amsterdam. We used extracts of the data set in
has two variants for classification: A small multi-layer feed- our experiments.
forward neural network and a logistic regression model. If the The Celeb-DF data set was created by the University at Al-
AUC value is higher than 0.5, than the video is classified as bany and the University of Chinese Academy of Sciences. The
a Deepfake. It is also possible to train the model with the newest version, which we used in our experiments, contains
features extracted from the input. We performed two experi- 590 original videos, each with multiple Deepfakes, 5639 altered
ments: one for the untrained model and the other one for the videos in total. Similarly to FaceForensics++, access needed to
trained model. For the untrained model experiment, we only be requested, and we downloaded the complete set to a server,
used the logistic regression model, and for the trained model, using extracts in our experiments.

Catherine de Weever, ing. Sebastian Wilczek page 4 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

6.2 Deepfake Creation plicable, other videos shot using the same camera. We used the
computed values to draw our conclusions about PRNU analysis
To properly check PRNU values, we required Deepfakes, of
accordingly.
which we had the original video belonging to the faked version
and an additional video shot on the same camera, ideally in a
similar setting. Since the data sets available only provide the 6.4 Visual Artifacts Analysis
original versions of the video but not a second, unrelated video Another way to detect Deepfakes is to analyze visual artifacts.
to check against, we had to create our own Deepfakes. For our research, we used the method, abbreviated as VA, in-
To create Deepfake videos, we made use of the cloud computing troduced in [26]. This method captures visual artifacts in the
service Shadow by Blade [4]. The service gave us access to a eyes, teeth, and facial contours of the generated or manipulated
virtual machine running Windows 10. The machine had access face. For example, this considers teeth represented as a white
to a full Intel Xeon E5-2678 v3 CPU, running at 2.50GHz blob. This method has two variants: VA-MLP, which uses a
with eight cores, and an NVIDIA Quadro P5000 GPU with 16 small multi-layer feed-forward neural network for classification,
GB GDDR5 VRAM. This hardware proved to be sufficient for and VA-Logreg, which uses a simple logistic regression model
creating Deepfakes. for classification.
To create Deepfakes, we made use of DeepFaceLab [29]. The It can also detect images from generated faces, Deepfakes, and
software suite comes with the necessary tools to extract faces Face2Face. For our research, we only made use of its detection
from video files, train the model required to generate Deep- pipeline for Deepfakes. This pipeline consists of, first, detecting
fakes, and render the results back to a video file. and segmenting the faces from the frames, second, extracting
For our experiments, we shot videos to be faked and checked. the features of the eyes and teeth, and last, classifying these
All created videos had a length of approximately 10 seconds, features with the previously mentioned models, VA-MLP and
resulting in 220 to 260 frames, and were shot using the front- VA-Logreg. This method can extract the features from the in-
facing camera of a Sony G8341 Xperia XZ1 smartphone. The put and train the models with these features, which we discuss
videos to later check PRNU patterns against were shot in the later in this section.
same manner as the to-be-faked pendants. They had the same As mentioned before, the input for this method needs to be
properties regarding length, encoding, and other comparable images or frames. So we first extracted the frames from the
properties. The faces to be trained as the new faces in the Deepfakes and original videos. We did this for ten videos from
Deepfake were extracted from longer videos of a different cam- the FaceForensics++ and Celeb-DF data set and for one self-
era or taken from existing face sets. made video that consists of a Deepfake and an original in the
DeepFaceLab offers two approaches to train the model, Quick96 same file.
and SAEHD. The former uses fewer resources and takes fewer Then, we processed each frame of the video with the Deep-
iterations to create a working Deepfake, at the expense of pic- fake pipeline mentioned before. The output is a table that
ture quality. Given the limited time of the project, we trained consists of four columns: Filename, Score-MLP, Score-LogReg,
our models using Quick96. We trained the models for about and Valid. The Filename column contains the filenames of the
60000 to 65000 iterations, taking roughly three hours per Deep- frames while the Score-MLP and Score-LogReg columns con-
fake on average. tain the classification score (AUC) of each frame. They can
We merged the resulting video mask containing the face of the range from 0 to 1. If the frame is from a Deepfake, the score
source video onto the original video for every Deepfake. We should be higher than 0.5, and the score should be lower when
did this by using DeepFaceLab. We did not use any external the frame is from an original video. The column Valid contains
video editing software. The mask was color-graded and blurred 0 and 1 values that indicate if the face detection and segmen-
to fit the original video before merging. tation were successful or not. After each frame is processed,
we calculated the average score for each video, excluding the
6.3 PRNU Analysis invalid frames.
For our research, we did the above with untrained and trained
To analyze the PRNU patterns of the Deepfakes retrieved and models. With untrained models, we mean that we calculated
recorded, we made use of an application called PRNU Com- the scores with the default or original classifiers. As mentioned
pare. This software was developed by the Netherlands Forensic before, the features from frames can be extracted and used to
Institute (NFI) [27] and is used by law enforcement agencies to train the models to create new classifiers for each video. So
compare videos and photos, deriving a confidence value if two with trained models, we mean that the scores were calculated
pieces of media originate from the same camera. with the newly created classifiers using the extracted features.
We used the software by importing the video files retrieved and Before we were able to do that, we needed to write a script
created. PRNU Compare automatically extracts the frames of that extracts the ground-truth labels (Filename and Valid col-
a video file, computing the average PRNU pattern over the set umn). So we created a script (see Appendix A) in Python that
of extracted frames. extracts these columns and saves them to a new CSV file called
We then computed the normalized cross-correlation values be- labels. We found out that the part for creating logistic regres-
tween the original and the faked video and the check videos, sion classifiers was not working when we ran the script to create
if available. PRNU Compare is also able to compute a Peak the new classifiers. So we modified the original code to create
to Correlation Energy value. This is another form of corre- new classifiers for the logistic regression model (see Appendix
lation indication, achieving the same goal as computing the B).
cross-correlation. Since the computation of normalized cross-
correlation takes marginally less time, we decided to use these
values for comparisons.
6.5 Frequency Domain Analysis
As mentioned, we compared the PRNU pattern of the original We looked into another way to detect Deepfakes, which is ana-
video to the patterns of both the fake pendant and, where ap- lyzing the domain frequency of an image. For our research, we

Catherine de Weever, ing. Sebastian Wilczek page 5 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

Figure 2: Normalized Cross Correlation of PRNU of Celeb-DF data set extract, with cross-correlation values shown
for each Deepfake per original video

made use of the method introduced in [9]. The pipeline of this experiments mentioned above and the theoretical research into
method contains two main blocks: the pre-processing block further related topics, such as the authentication of original
and the training block. In the pre-processing block, the input media.
is transformed into a convenient domain by performing the fol- Structurally, this section follows the same topics as the previ-
lowing steps. First, the Discrete Fourier Transform (DFT) is ously detailed methodology and experiments. We discuss the
applied to the image where the output is a 2D Amplitude Spec- results of our PRNU analysis and the results relating to visual
trum. The Azimuthal averaging is applied where the output artifact and frequency domain analysis. We then define our
is a 1D Amplitude Spectrum (1D representation of the FFT results regarding the evasion of Deepfake detection techniques
power spectrum). One can see the last step as compression. In and our findings in the area of media authentication.
the training block, these new transformed features are used to
determine if a face is fake or not. This method makes use of
two classification algorithms, Logistic Regression and Support
7.1 PRNU Analysis
Vector Machines (SVMs). We only used the Logistic Regres-
sion algorithm for better comparison to the previous method As previously mentioned, we imported the available Deep-
for our experiments. fakes and original videos into PRNU Compare for compari-
Both real and fake images are needed to train this classifier. son. Given that the computation of PRNU patterns and their
For this method, one needs the same amount of real and fake cross-correlation takes time, we opted to use only extracts of
images, so there needs to be a balance. So for our experiment, the available data sets.
we used 36 videos (eighteen real and eighteen fake) from the
The data set of Celeb-DF contains multiple Deepfakes per orig-
FaceForensics++ data set and ten videos (five real and five
inal video. However, some of the Deepfakes have slightly al-
fake) from the Celeb-DF data set. This method only accepts
tered aspect ratios, making a direct cross-correlation compu-
frames or images as input, so we used the frames that we have
tation impossible. We went through the set and extracted the
extracted for the previous method. Before we could use these
originals and Deepfakes that have the same aspect ratio un-
images as input, they first needed to meet the following re-
til we had a reasonably-sized subset. We then computed the
quirements: the (fake) face needs to be the dominant part of
normalized cross-correlation for each original video against its
the input, and the images need to be squared. For our images
Deepfakes. The results can be seen in Figure 2. The graph
to meet the requirements, we ran a face detection on them and
shows all computed cross-correlation values per original video
made them square images using the program autocrop [20].
in relation to multiple Deepfake videos derived from the origi-
After that, we inserted them into the pipeline. The output nal.
of this method is the average classification rate per video (ac-
Each bar represents the cross-correlation of a given original
curacy). The higher the average, the better, while there is a
video against one of its Deepfakes. As can be seen, the cross-
balance in the features between the fake and real video. If the
correlation values range in-between 0.4 and 0.78. These are
average is higher than 0.5, we classified the video as fake.
inconclusive values. Because there are no other original videos
taken with the same camera as the original to compare against,
7 Results it is impossible to deem any of the analyzed videos as an orig-
inal or a fake.
In this section, we detail the results we uncovered throughout We observed similar behavior in the FaceForensics++ data set.
this project. The results contain the measurable values of the This set only contains one Deepfake per original video. The re-

Catherine de Weever, ing. Sebastian Wilczek page 6 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

sults can be seen in Figure 3. While analyzing the lower cross-correlation values, we realized
that image stabilization influenced our results. We filmed the
videos created up to this point while holding the smartphone
in-hand. The slight movement of the hand triggered the built-
in stabilization techniques that are increasingly common with
smartphone cameras. Moving the camera blurs the resulting
PRNU pattern. This blur results in cross-correlations between
images to be lower, even between two images or videos recorded
right after each other using the same camera.

We created another Deepfake to evaluate this theory. We

mounted the smartphone to a tripod on stable ground. The
recording was triggered remotely to prevent any slight collision
and resulting blur when starting the recording. Other than
that, we created the Deepfake in the same manner as the pre-
vious ones. The result of analyzing this new Deepfake against
Figure 3: Normalized Cross Correlation of PRNU of Face- the new and existing check videos is seen in Figure 5.
Forensics++ data set extract

Similar to the results of the Celeb-DF sets, the resulting values

range in the upper half, between 0.52 and 0.95 percent. Espe-
cially the higher values make it impossible to deem a video as
a fake, given that a higher cross-correlation indicates the likely
equivalence of the video’s origin.
However, the difference between the cross-correlation of a faked
video and a video originating from the same camera against
an original video might indicate Deepfakes in a file. As men-
tioned, neither the Celeb-DF nor the FaceForensics++ data
set contain such check files. Therefore, we analyzed Deepfakes
we created ourselves, together with such check videos. In Fig-
ure 4, one can see the cross-correlation of the Deepfake videos
and multiple check videos against two original videos.

Figure 5: Normalized Cross Correlation of PRNU of cre-

ated Deepfake with stabilization

The results show that the values are indeed higher if the camera
is stable during recording, resulting in a more precise PRNU
pattern. However, the results also show that the lower cross-
correlation values of the Deepfakes previously analyzed were
a coincidence. In this particular case, the cross-correlation of
Deepfake and original video is even higher than the correlation
between the original and a check video.

Figure 4: Normalized Cross Correlation of PRNU of cre-

ated Deepfakes
We attempted to further refine the experiments by compar-
While a difference between the Deepfake and check videos can ing PRNU patterns extracted from videos that we previously
be observed, all computed cross-correlation values are consid- cropped to the face of the depicted subject. Since we used
erably lower than the results retrieved from the available data DeepFaceLab for the creation of our own Deepfakes, the aligned
sets. Furthermore, the difference between the cross-correlation and cropped images of both original and faked faces were avail-
of the fake video and that of the check videos is only marginal. able to us. In the same manner, we extracted the faces from
Again, we could not extract conclusive evidence about the au- the check video. We computed the PRNU patterns of all three,
thenticity of a video. shown in Figure 6, 7 and 8.

Catherine de Weever, ing. Sebastian Wilczek page 7 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

subtle, but this does not apply to full Deepfake imagery.

Overall, none of the PRNU analysis resulted in a definite proof
of Deepfake or authenticity.

7.2 Visual Artifact Analysis

As mentioned in the previous section, the frames from the
Deepfake and original videos were processed to calculate the
classification scores. The number of frames for each video
ranges from 300 to 800 frames rounded up. We calculated
the average for each video to get an overall score. The videos
classified as Deepfake need to have a score higher than 0.5, and
the videos classified as an original need to have a lower score.
We did not include all frames in the calculation, only the ones
classified as valid. Occasionally, almost no valid frames were
Figure 6: PRNU Pattern of Original Video recognized, even just one in a particular experiment. That in-
fluences the visible results, and might also indicate that there
is a forgery (less valid frames with fakes). However, it also
happens with originals occasionally, for example, CDF 0001
Original only had one valid frame.

7.2.1 Untrained models

Three of the five original videos from the FaceForensics++
data set have a score of 0.5 and lower, as can be seen in Fig-
ure 9. These three videos were classified as original. FF 002
Original was classified the best with a score of 0, rounded down.
FF 001 Original was classified the worst with a score of 0.94,
meaning it classified as a Deepfake, despite being original me-
dia.
For the Deepfake videos, four of the five have a score higher
than 0.5, meaning that they classified as Deepfakes (see Fig-
Figure 7: PRNU Pattern of Deepfake Video ure 9. FF 002 Fake has a score of 0.95, so it was classified the
best. Though, the gap between FF 002 Fake and the other
three (FF 000 Fake, FF 001 Fake and FF 004 Fake) is small.
FF 003 Fake scored the worst and is the only Deepfake video
classified as original.
In Figure 9, it is depicted that four of the five original videos
from the Celeb-DF data set are classified as original. Though
they all have a score of 0.5 or lower, the range contains val-
ues higher than 0.5. It means that these videos contain frames
that had a score higher than 0.5. CDF 0000 Original has the
lowest score, lower than 0.5 with the range of standard devi-
ation included, so this video is classified the best. CDF 0001
Original is classified the worst with a score of 1, meaning that
the pipeline classified this original video as a Deepfake with no
margin for error.
Four of the five Deepfake videos were classified as a Deepfake.
CDF 0003 Fake has a score of 0.51 and the rest a score of 0.8
Figure 8: PRNU Pattern of Check Video and higher (see Figure 9). CDF 0003 Fake contains frames
with a score of up to 0.92, while the others contain frames
The cross-correlation values of the patterns depicted above are scoring higher than 1.0, meaning that these frames classified
below 6%. There is only minimal similarity given PRNU val- as a Deepfake with no margin for error. CDF 0001 Fake clas-
ues for the faked as well as another original video if we crop sified the best by having the highest score, while CDF 0000
the videos to faces. The main issue with this approach is that Fake classified the worst by having the lowest score, although
the face in the check video is not necessarily at the same spot it contains frames that classified as a Deepfake.
as in the original and Deepfake videos. Therefore, an extract To depict the score difference between the frames from a Deep-
of the PRNU pattern at a different part of the whole image is fake and the ones that are original, we created a video that
slightly distorted when compared to the extract of the origi- contains frames from the Deepfake and the source video. We
nal and the Deepfake, which are at the same place, given that processed these frames and retrieved the following result de-
only the face changes when comparing the videos, not the face picted in Figure 10. For frame number 183, a drop to 0 can be
position. Using this approach, therefore, only shows if the two seen where the frames start being from the original video. The
faces are the same. This may be valuable if the changes are other drops to 0.5 or less beforehand depict the inconsistencies

Catherine de Weever, ing. Sebastian Wilczek page 8 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

Figure 9: Average logistic regression per video of FaceForensics++ and Celeb-DF data set extracts

in the results. both FaceForensics++ and Celeb-DF all have a score of 0,

rounded down, meaning that they all classified as original. So
using a trained MLP model improves the classification of the
original but worsens the classification of Deepfakes. It is the
opposite of the trained version of the logistic regression model.
We also used both of the trained models with the video that
contains frames from a Deepfake and an original file and got
the following results. As expected, the score using MLP is 0,
rounded down, whereas the score using the logistic regression
model is 1. If we compare the results with the untrained model
results, we observe that there is no drop, making the untrained
model more reliable, even though it has some inconsistencies.

7.3 Frequency Domain Analysis

Because of the ambiguous results from the previous method,
Figure 10: Logistic regression per frame of partial Deep- we also decided to look into the frequency domain analysis that
fake video also uses a logistic regression model, as mentioned before, the
input consists of images from real and Deepfake videos. This
method already calculates the average, so we did not need to
do it ourselves. A note to point out is that after running a
7.2.2 Trained models
face detection on the images, only a few images were rejected
As mentioned in the previous section, we trained the VA- (no face was detected). In this method, a lot more images were
Logreg model using the features from the videos. We trained valid than for the visual artifact analysis method. This method
the model one time. For the original videos both from Face- also compares the real images with fake images, so there is no
Forensics++ and Celeb-DF, all have a score of 1.0 as seen in separate average for the real images.
Figure 11. This also applies to the Deepfake videos. So using For the FaceForensics++ data set, all videos are classified as
a trained logistic regression model improves the classification a Deepfake as seen in Figure 12. FF 018 had the lowest score,
of Deepfakes but worsens the classification of the original files. while FF 011 had a 100% accuracy. Ten of the 19 videos had
After receiving these ambiguous results, we also decided to an accuracy of 70% and above, which leads to a high overall
train the VA-MLP model. The original and fake videos from accuracy for this data set.

Catherine de Weever, ing. Sebastian Wilczek page 9 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

Figure 11: Average logistic regression using trained approach per video of FaceForensics++ and Celeb-DF data set
extracts

For the Celeb-DF data set, it is a different story. As depicted where X = original input image
in Figure 12 all videos have a score of 1. It may seem to have Xadversarial = adversarial image
positive results, but it is not. This phenomenon occurs when ∇ = gradient of cost f unction with respect to X
there is no balance between the features of fake and real im- J = Loss f unction
ages. In our case, there was no spectrum found on the fake Y = model output f or X ( input label )
images. We retrieved this information by plotting the spectra  = parameter to scale noise (small number)
of these videos. It returns a score of 1 because there is no fake
data to compare. So this method did not work on the Celeb-DF First, perturbations (random noise) are created with the sign
data set. We discuss the reason for this in 9 Discussion. function. This is done by using the gradients of the loss with
respect to the input image. Then the perturbations are added
to the original image, X. The output is the image with the
7.4 Detection Evasion random noise added. An example is depicted in Figure 13.
A way to bypass the detection method is to modify Deepfakes In [18], where they use the logistic regression model, the error
adversarially. This way, the detection system (classifier) can be rate grew from 2.9% to 7.6% when epsilon has a value of 0.2
fooled and classify the Deepfake as the original by reducing the and 22.6% when epsilon has the value 0.5. Because the visual
AUC value. There are many different attacks to achieve this. artifact analysis and the frequency domain analysis use the lo-
These attacks can perform in a white-box and a black-box ap- gistic regression model, we can conclude that FGSM can also
proach. For a white-box approach, the attacker has full access be applied to increase this model’s error rate.
to the detection system, including the Deepfake pipeline, the As previously mentioned, we observed that any internal camera
architecture, and the classification model’s parameters. For a stabilization influences the results of PRNU analysis negatively.
black-box approach, the opposite applies: the attacker does By moving the camera, the resulting video’s extracted pattern
not have full or limited access to the classification infrastruc- becomes blurry, even with slight camera motions. The blurred
ture and its parameters. or distorted pattern would result in a lower cross-correlation
In our case, an attack that can be performed is the FGSM. score when comparing against another pattern, even if one ex-
FGSM is a white-box attack which ensures misclassification. tracted the said pattern from the same camera. In other words,
According to [18], ”FGSM is to add the noise whose direction if the camera moves while recording the destination video to
is the same as the gradient of the cost function with respect to have a face edited in, PRNU analysis is less likely to detect a
the data.” It is done in one single step, and uses the following Deepfake. Both original and Deepfake media will have lower
equation: cross-correlation scores, making it impossible to differentiate.
Therefore, camera movements can be seen as evasions methods
Xadversarial = X + .sign(∇x J(X, Y )), to prevent Deepfake detection through PRNU analysis.

Catherine de Weever, ing. Sebastian Wilczek page 10 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

Figure 12: Average logistic regression using frequency domain approach per video of FaceForensics++ and Celeb-DF
data set extracts

7.5 Media Authentication mation. Theoretically, provenance could be created directly at

the source. Organizations, such as news and governmental of-
The question of how to differentiate between original and syn- fices, could sign files that they deem authentic using a form of
thetic media still stands since we could not determine any public key infrastructure. However, every reading party, such
definitive way to detect Deepfakes in video files. Even if one as social networks or other organizations, would have to know
could propose a highly successful way of detecting Deepfakes, the public key. They would have to retrieve the entire file to
the result would be of limited value. As soon as the detec- be able to validate the signature.
tion method becomes public knowledge, one can enhance the
Furthermore, many organizations modify media while warrant-
discriminator of a Deepfake-generating GAN with said detec-
ing authenticity. For instance, news organizations regularly en-
tion method. Over time, Deepfakes would take these results
code media files in different formats or add other forms of vi-
into account, learning to avoid generating images that one can
sual and audible information to a video, such as text banners or
detect through these means.
backing commentaries. While not the same file, the originally
In other words, Deepfake detection may prove to be dysfunc-
depicted videography is nonetheless authentic. In this case,
tional in the future. Instead of attempting to detect Deepfakes,
the file would have to be resigned, and provenance information
an alternative approach would be to recognize the original files.
would be lost since there is no reference to the original file.
The primary issue caused by Deepfakes is the impact on the
This approach is also not applicable to streaming media since
person falsely depicted. If the source of the video is uncer-
the entire file needs to be available at the signature-checking
tain and perhaps not trusted, one could mitigate parts of this
party.
issue. To acquire the source of a video file, said files require
provenance information. [21] To mitigate these issues, Microsoft Corporation proposed a sys-
Provenance is the record of origin and ownership of a product. tem called AMP [10]. The system is designed to provide and
Historically, the term stems from the art world but can apply interact with provenance information on the Internet. The
to other items, such as video files. Provenance for digital me- system makes use of authentication and watermarking. To au-
dia includes the source of production, the chain of modification thenticate media, one can create manifests containing hashes
done to the original file, and information about the publisher of the original file and the metadata of a publisher. For stream-
or owner. able media, hashes are created for file chunks.
If provenance were known for every authentic piece of digi- Furthermore, these manifests can contain pointers to other
tal media, Deepfakes could easily be spotted because they are source files. If a video such as a news report uses multiple
missing provenance, or the provenance is pointing to an un- recordings, the report’s manifest can point to these sources,
known or non-trusted source. However, there is only limited ensuring a chain of provenance information. These manifests
literature and projects available in this field. ”The problem are stored on a public blockchain ledger, to be immutable and
of seeking information provenance has received little attention identifiable.
in comparison with its counterpart, the study of information The goal of AMP is to create a system where every piece of
propagation.” [3] digital media carries a manifest as well. However, since current
The problem lies in how one propagates the provenance infor- Internet media systems are not equipped to do so yet, they also

Catherine de Weever, ing. Sebastian Wilczek page 11 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

Figure 13: An example of how the Adversarial image is created using FGSM [31]

provide a queryable database to check manifests. Additionally, where machine learning was proposed as a subset of a broader
spread-spectrum watermarks may be inserted. These water- methodology proposal, for example, while doing visual artifact
marks add noise to the file that is imperceptible to the human analysis.
eye. This watermark carries an identifier for a manifest, for a The analysis of PRNU patterns has shown drastically varying
user to later retrieve said manifest, enabling authentication. results. Cross-correlation values shifted considerably between
AMP is not the only system attempting to introduce prove- different Deepfake and original videos, with no clear indication
nance to consumers. The research and development group of of an analyzed video being fake or authentic. Furthermore, we
The New York Times Company introduced The News Prove- have seen cross-correlation values being higher for Deepfake
nance Project [32]. Like AMP, it uses blockchain to store and videos than for other original media. We could not determine
retrieve information about the origin of media. In extent, the a cut-off value for a proper indication of Deepfake presence.
project is also concerned with the consumer itself. They are re- Additionally, PRNU analysis also has the significant draw-
searching how consumers evaluate media and their information back of requiring comparison media. To determine a cross-
of origin. correlation value in the first place, the PRNU pattern of the
Furthermore, considerations such as if media without prove- supposed original camera needs to be present, for example, by
nance information can still be trusted are taken into account. analyzing another known piece of media. While this might be
Research such as this is emerging broadly in the context of dig- possible in some cases, for instance, in the case of statically
ital media. Other projects, such as the Content Authenticity placed sources such as surveillance cameras, this leaves out cir-
Initiative by Adobe Inc. [1], are considering similar aspects, cumstances where the original camera may not be retrievable,
and will likely continue to do so in the future. such as private smartphones or news broadcasts.
Given the proper implementation of a system such as AMP and The analysis of visual artifacts has shown varying results when
enhancement of video players available, such as the ones on so- using the non-trained model for both data sets. Most Deepfake
cial networks and websites, it could be possible to introduce videos were classified as a Deepfake; however, a considerable
provenance information to the user. Given this information, a amount of frames from these videos were classified as invalid
consumer can draw an educated conclusion about whether to and not included in the calculation of the AUC score. This
trust the displayed media. makes the visual artifact analysis method unreliable though an
advantage of this method is that it only needs Deepfake videos
for classification. The results when using the trained model
8 Conclusion were ambiguous, which led us to the conclusion that this part
of this method did not work properly.
In the following paragraphs, we detail the conclusions we can For the analysis of the frequency domain, the results differ de-
draw from the experiments mentioned above and their results. pending on the data set used. For the FaceForensics++ data
Structurally, we will follow the research questions mentioned set, it had a high accuracy score, meaning that it classified the
at the beginning of this paper. Deepfake videos as a Deepfake. While for the Celeb-DF data
Our research into related work on the topic of Deepfake de- set, it returned a biased score, meaning that something did not
tection returned many different approaches. One thing most work properly. A drawback of this method that it needs both
approaches had in common was their use of neural networks. original and Deepfake videos, unlike the previous method.
Due to the scope of this project, we did not evaluate these time- As can be seen for the results of the PRNU, visual artifact,
consuming methods. Instead, we focused on approaches that and frequency domain analysis, none returned results that can
did not make use of machine learning primarily. Specifically, we accurately indicate Deepfake presence or absence with a high
focused on the usage of PRNU patterns and logistic regression probability of success. Throughout our experiments, we at-
analysis of visual artifacts and frequency domain, which have tempted multiple approaches to improve these results. Regard-
been used for Deepfake detection in other research papers. We ing PRNU analysis, we considered that the stabilization of a
only used neural networks to create Deepfakes ourselves and camera when creating a Deepfake might influence the com-

Catherine de Weever, ing. Sebastian Wilczek page 12 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

puted results. We also attempted to compute PRNU cross- that such media may not hold up in court, or the public may
correlation on extracts of the entire pattern, such as the part dismiss it because it is marked as not trustworthy.
of the image containing a face. For visual artifact analysis, we Additionally, signing media may violate the privacy of the
tried to repair the code that trains the model. However, none of source. If all media were to be signed automatically for the
the improvements we attempted resulted in considerably more sake of provenance, the source becomes easily identifiable. This
positive results. mechanism may help in criminal cases, but it could also en-
In attempting to improve the detection methods, we uncov- danger sources that rely on anonymity, such as corporate and
ered some possibilities to evade Deepfake detection. Moving governmental whistleblowers.
the camera when creating a Deepfake lowers cross-correlation Both the absence and presence of a method to identify orig-
scores during PRNU analysis, making comparisons of original inal media or Deepfakes carry both technological and ethical
and fake media less successful. Because the visual artifact anal- problems that are yet to be addressed. Further research is war-
ysis and the frequency domain analysis method use a logistic ranted in all the mentioned topics. Please see 10 Future Work
regression model, an adversarial attack using FGSM can be for further information.
performed. Overall, all the mentioned approaches further dis- To summarize, plenty of approaches for Deepfake detection ex-
prove the possibilities of using the considered methodology for ist, making use of both neural nets and other means of eval-
Deepfake detection. Aside from minor technical improvements uation. However, the approaches considered in this research
and streamlining the analysis process through automation, we do not exhibit results that can lead to the detection of Deep-
did not find a noticeable way to improve existing detection fakes and original videos with sufficient probability. Other ap-
methods. proaches may be able to do so, but their existence may also
As previously mentioned, we were unable to pinpoint any re- improve the creation of Deepfakes. Enhancing them or creat-
liable method to detect or even indicate Deepfake presence. ing even better approaches could be considered reprehensible.
Please note that this research focuses on methods that do not Therefore, alternative solutions, such as authentication of orig-
use neural networks. Therefore, we can not claim that a reli- inal media, are required. However, new alternatives carry their
able method does not exist. own set of issues and challenges, warranting future research.
However, even if a reliable detection method would exist, its
applicability would be of limited potential. As mentioned in
4 Background, Deepfakes make use of GAN. Therefore, a dis- 9 Discussion
criminator is used to evaluate the generated results. Any sys-
In this section, we discuss the progress and results of this re-
tem that could rate the likelihood of Deepfake presence may
search critically. We evaluate which parts of this research one
also be used as part of a GAN discriminator. Such a detection
could have improved upon and how one can improve them in
method would actively support the improvement of Deepfake
the future.
creation since the discriminator could remove imagery that may
To start, we did not have fully sufficient data sets to perform
be susceptible to detection.
PRNU analysis. As previously mentioned, our experiments re-
Therefore, we do believe that alternative approaches are re- quire the presence of a check video, recorded with the same
quired. We detailed one such approach before. Authentication camera. The FaceForensics++ and Celeb-DF data sets do not
of original media may be a step to introducing provenance to include such videos. Therefore, we had to create our own,
digital imagery. Being able to claim authenticity is becoming together with the accompanying Deepfakes. The process of
ever more critical, especially for sources such as news outlets creating Deepfakes requires time and resources, both of which
and governmental institutions. were limited in this project.
Given the existing prototype implementation of AMP, it might One can say the same about the experiments in general. More
be possible to deploy such a media authentication service on measurements, perhaps taking more data sets into account,
a large scale. However, more research is required beforehand. could have been taken in longer project time. Perhaps the
This research is twofold. First, the design and implementation experiments could have been improved with a degree of au-
of such a system need to be detailed. If a system is to be avail- tomation. In the present setups of our experiments, PRNU
able to authenticate and sign original media, ideally worldwide, patterns have to be extracted and compared one-by-one, due
much careful thought has to be put into the system design. to the software used. Similarly, one could have streamlined
Secondly, deploying such a system carries a heavy load of eth- the evaluation of logistic regression and visual artifacts using a
ical concerns. One of them is the question as to who should more automated approach. We can not claim that this would
be responsible for such an authentication system. Should it be have led to more measurements since we can not estimate how
governmental organizations or private ones? Furthermore, who long the creation of such automation would have taken.
should be able to sign media, and who is empowering them to For the visual artifact analysis, better data sets could have
do so? been used, as a considerable amount of the frames extracted
Assuming such a system was in place, Deepfakes could be dis- from these videos were classified as invalid by this method. Ad-
tinguished from original media, because they would not be ditionally, the modified code for the training model can also be
signed by the person or responsible organization displayed. considered faulty.
Anyhow, only trusting signed media carries its own set of prob- As mentioned before, the frequency domain analysis did not
lems and concerns. Especially in a legal environment, only work for the Celeb-DF data set. We found out that a class im-
trusting certain types of media may be reprehensible. For balance occurred by plotting the spectra of the videos. When
example, the recently emerging riots in the United States of this phenomenon occurs, the accuracy metric breaks down.
America were caused by a video taken by a bystander of crime Class-imbalanced problems are one of the critical flaws of ac-
committed by a police officer. Such a video would not be signed curacy. So we can conclude that the metric accuracy is an
in this case, or at least not signed by a known, trusted source. unreliable metric to use for this method. As mentioned before,
Considering the implications for the future, it may be possible a class imbalance occurred because no spectrum was found on

Catherine de Weever, ing. Sebastian Wilczek page 13 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

the fake images. A reason for this is that these frames may As previously mentioned, this research paper considers
have been compressed to a large extend, being videos retrieved methodology that uses neural networks only marginally. We
from YouTube. The creators of this method also stated that have found the usage of neural networks as part of Deepfake
their approach might not work on videos or images that have detection to be quite popular in our analysis of related work.
been compressed to a large extend. [9] Two solutions for these Given the machine learning nature of Deepfakes, this connec-
problems could be to use the metric ”precision and recall” in- tion does make sense and perhaps warrants future research.
stead, or, if accuracy is preferred, to use another data set that While the approaches detailed in this paper and other method-
is not compressed. ologies not using neural networks may be presently less able to
Furthermore, some parts of this research are only taking a the- detect Deepfakes, neural-network-based approaches might be
oretical, abstract approach. We only checked the evasion of more successful.
Deepfake detection in relation to related work. Experiments Since the discriminator of a GAN can also use any detection
about the success of evasion using the mentioned techniques method that can score video input, it might be possible to look
could have provided valuable information. Similarly, the con- into alternatives that are unusable by a GAN. Given the con-
cept of original media authentication and the surrounding sys- tinuous improvement of Deepfakes using the GAN approach,
tems is only detailed based on previous work. It would be this may also be the only possible approach to be taken for
possible to create a prototype of such a system or evaluate Deepfake detection in the future. However, designing a detec-
existing approaches, such as AMP. However, given the scope tion mechanism that can not be used by a discriminator may
of this project, analyzing, designing, and implementing such be complicated, if not outright impossible.
a system should be considered future work, as shown in the Instead, as an alternative, further research into the topic of
following section. original media authentication is warranted. Systems like AMP
are a start to designing and implementing a system that one
could use to sign and identify original media where required.
10 Future Work More research is needed into the technical details and limi-
One can expand upon this research in a multitude of ways. tations of deploying such a system. Such research would en-
First, one can extend the measurements taken during this compass not only the underlying signature mechanism but also
project. Due to our limited time scope, we were only able infrastructural questions, such as wide-scale deployment or se-
to consider a subset of the data sets used in our experiments. curity of the system itself.
One can redo the experiments with all original and Deepfake Given the potential of such a system, one should also consider
videos from the FaceForensics++ and Celeb-DF data sets and how it should be handled in an organizational context. Ques-
any other data set containing both the original and at least one tions like who is responsible for the system, both in terms of
Deepfake version of any given video in the original set. This management and legal operations, have to be discussed.
applies to both PRNU and visual artifact analysis. Lastly, the potential of Deepfake technology also warrants fur-
Regarding PRNU analysis, one can also extend the experiments ther ethical discussion. If Deepfakes ever get so advanced that
detailed here by including more custom original and Deep- they are virtually indistinguishable from original media, rules
fake videos. Since our experiments required the existence of and regulations, both legal and moral, need to be present. The
check videos, recorded on the same camera in the same set- same applies to the concept of authenticating media. If only
ting, more Deepfakes and original videos could be created in signed media is perceived as trustworthy, a new dynamic of
the same manner to provide more comparable input to the consuming digital information is created. Furthermore, the
PRNU-related experiments. Especially the check videos, which question who signs which media also creates more ethical con-
are not present in any public data set known to us, could pro- cerns that one needs to address in the future. In the same way,
vide valuable insight into the cross-correlation of Deepfake and the technology and its advancements need to be monitored,
original videos. discussed, and researched.

References
[1] Adobe Inc. Introducing the Content Authenticity Initiative.
Available at: https://theblog.adobe.com/content- authenticity- initiative/ [Accessed 19 Jun. 2020].
2019.
[2] Afchar, D. et al. “MesoNet: a Compact Facial Video Forgery Detection Network”. In: 2018 IEEE International
Workshop on Information Forensics and Security (WIFS). 2018, pp. 1–7.
[3] Barbier, G. et al. “Provenance Data in Social Media”. In: Synthesis Lectures on Data Mining and Knowledge
Discovery. 2013.
[4] Blade Group. Shadow - Your gaming PC powered by Cloud Gaming.
Available at: https://shadow.tech [Accessed 11 Jun. 2020]. 2020.
[5] Chen, Y. and Thing, V. L. L. A study on the photo response non-uniformity noise pattern based image forensics
in real-world applications. Singapore: Institute for Infocomm Research, 2012.
[6] Chesney, B. and Citron, D. “Deep fakes: a looming challenge for privacy, democracy, and national security”. In:
Calif. L. Rev. 107 (2019), p. 1753.

Catherine de Weever, ing. Sebastian Wilczek page 14 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

[7] Cole, S. We Are Truly Fucked: Everyone Is Making AI-Generated Fake Porn Now.
Available at: https : / / www . vice . com / en _ us / article / bjye8a / reddit - fake - porn - app - daisy - ridley
[Accessed 02 Jun. 2020]. New York, United States of America, 2018.
[8] DataCadamia. Machine Learning - Logistic regression (Classification Algorithm).
Available at: https://datacadamia.com/data_mining/logistic_regression [Accessed 30 Jun. 2020].
[9] Durall Lopez, R. et al. Unmasking DeepFakes with simple Features.
Available at: https://arxiv.org/abs/1911.00686v3.pdf [Accessed 30 Jun. 2020]. 2020.
[10] England, P. et al. AMP: Authentication of Media via Provenance.
Available at: https://www.microsoft.com/en-us/research/publication/amp-authentication-of-media-
via-provenance/ [Accessed 30 Jun. 2020]. 2020.
[11] Faceswap. Welcome - Faceswap.
Available at: https://faceswap.dev/ [Accessed 03 Jun. 2020]. 2020.
[12] Fagan, K. A viral video that appeared to show Obama calling Trump a ’dips—’ shows a disturbing new trend
called ’deepfakes’.
Available at: https : / / www . businessinsider . com / obama - deepfake - video - insulting - trump - 2018 - 4
[Accessed 02 Jun. 2020]. 2018.
[13] Fridrich, J. Digital Image Forensics Using Sensor Noise. New York, United States of America: Binghamton
University, 2008.
[14] Goodfellow, I. J. et al. Generative Adversarial Nets. Montreal, Quebec, Canada: University of Montreal, 2014.
[15] Google. Logistic Regression: Calculating a Probability.
Available at: https://developers.google.com/machine-learning/crash-course/logistic-regression/
calculating-a-probability [Accessed 30 Jun. 2020]. 2020.
[16] Greenfield, Y. Intentionally Misleading DeepFakes & Keanu Reeves.
Available at: https://medium.com/@ubershmekel/intentionally-misleading-deepfakes-keanu-reeves-
b023a3be522a [Accessed 03 Jun. 2020]. 2019.
[17] Güera, D. and Delp, E. J. “Deepfake Video Detection Using Recurrent Neural Networks”. In: 2018 15th IEEE
International Conference on Advanced Video and Signal Based Surveillance (AVSS). 2018, pp. 1–6.
[18] Ken Tsui. Perhaps the Simplest Introduction of Adversarial Examples Ever.
Available at: https://towardsdatascience.com/perhaps-the-simplest-introduction-of-adversarial-
examples-ever-c0839a759b8d [Accessed 23 Jun. 2020]. 2018.
[19] Koopman, M., Rodriguez, A. M., and Geradts, Z. “Detection of deepfake video manipulation”. In: The 20th
Irish Machine Vision and Image Processing Conference (IMVIP). 2018, pp. 133–136.
[20] Leblanc, F. autocrop.
Available at: https://github.com/leblancfg/autocrop [Accessed 26 Jun. 2020]. 2019.
[21] Leetaru, K. Why Social Media Provenance Is More Important Than Ever In An AI-Falsified World.
Available at: https://www.forbes.com/sites/kalevleetaru/2018/06/19/why-social-media-provenance-
is-more-important-than-ever-in-an-ai-falsified-world/ [Accessed 19 Jun. 2020]. 2018.
[22] Li, Y., Chang, M., and Lyu, S. “In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking”.
In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS). 2018, pp. 1–7.
[23] Li, Y. and Lyu, S. “Exposing DeepFake Videos By Detecting Face Warping Artifacts”. In: computer vision and
pattern recognition (2019).
[24] Li, Y. et al. “Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics”. In: IEEE Conference on
Computer Vision and Patten Recognition (CVPR). 2020.
[25] Lyu, S. “Deepfake detection: Current challenges and next steps”. In: 2020 IEEE International Conference on
Multimedia & Expo Workshops (ICMEW). IEEE. 2020, pp. 1–6.
[26] Matern, F., Riess, C., and Stamminger, M. “Exploiting Visual Artifacts to Expose Deepfakes and Face Ma-
nipulations”. In: 2019 IEEE Winter Applications of Computer Vision Workshops (WACVW). 2019, pp. 83–
92.

Catherine de Weever, ing. Sebastian Wilczek page 15 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

[27] Netherlands Forensic Institute. Finding the link between camera and image - Camera individualisation with
PRNU Compare Professional from the Netherlands Forensic Institute.
Available at: https://www.forensicinstitute.nl/binaries/forensicinstitute/documents/publications/
2017/03/06/brochure-prnu-compare-professional/brochure-nfi-prnu-compare-professional_tcm36-
21580.pdf [Accessed 12 Jun. 2020]. 2017.
[28] Nguyen, H., Yamagishi, J., and Echizen, I. “Use of a Capsule Network to Detect Fake Images and Videos”. In:
(Oct. 2019).
[29] Perov, I. et al. DeepFaceLab: A simple, flexible and extensible face swapping framework. Location Unknown,
2020.
[30] Rössler, A. FaceForensics++: Learning to Detect Manipulated Facial Images.
Available at: https://github.com/ondyari/FaceForensics/ [Accessed 03 Jun. 2020]. 2019.
[31] TensorFlow. Adversarial example using FGSM.
Available at: https://www.tensorflow.org/tutorials/generative/adversarial_fgsm [Accessed 26 Jun.
2020].
[32] The New York Times Company. The News Provenance Project.
Available at: https://www.newsprovenanceproject.com [Accessed 19 Jun. 2020]. 2019.
[33] Yang, X., Li, Y., and Lyu, S. “Exposing Deep Fakes Using Inconsistent Head Poses”. In: ICASSP 2019 - 2019
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2019, pp. 8261–8265.
[34] Zhang, M. This Website Generates AI Portraits of People Who Don’t Exist.
Available at: https://petapixel.com/2019/02/19/this-website-generates-ai-portraits-of-people-
who-dont-exist/ [Accessed 02 Jun. 2020]. 2019.
[35] Zhou, P. et al. “Two-Stream Neural Networks for Tampered Face Detection”. In: 2017 IEEE Conference on
Computer Vision and Pattern Recognition Workshops (CVPRW). 2017, pp. 1831–1839.

Catherine de Weever, ing. Sebastian Wilczek page 16 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

Appendices
A Ground-truth labels parser
import o s
from s h u t i l import c o p y f i l e
import pandas a s pd

path= ’ /mnt/ c / U s e r s / c a t d e / OneDrive / Documents /SNE/RP2/ r e s u l t s /non−t r a i n e d / ’

def g e t r e s u l t s ( path ) :
f o r dir in o s . l i s t d i r ( path ) :
f o r f in o s . l i s t d i r ( path + dir ) :
i f f == ’ s c o r e s . c s v ’ :
p a r s e l a b e l ( path + dir + ’ / ’ + f , path , dir )

def p a r s e l a b e l ( f i l e , path , dir ) :

f=pd . r e a d c s v ( f i l e , s e p= ’ , ’ )
k e e p c o l = [ ’ Filename ’ , ’ V a l i d ’ ]
new f = f [ k e e p c o l ]
n e w f . t o c s v ( path + dir + ’ / l a b e l s . c s v ’ , i n d e x=F a l s e )

def g e t l a b e l s ( path ) :
f o r dir in o s . l i s t d i r ( path ) :
f o r f in o s . l i s t d i r ( path + dir ) :
i f f == ’ l a b e l s . c s v ’ :
r e n a m e h e a d e r ( path + dir + ’ / ’ + f )

def r e n a m e h e a d e r ( f i l e ) :
f=pd . r e a d c s v ( f i l e )
f . rename ( columns={” V a l i d ” : ” L a b e l ” } ) . t o c s v ( f i l e , i n d e x=F a l s e )

f o r d a t a s e t in o s . l i s t d i r ( path ) :
g e t r e s u l t s ( path + d a t a s e t + ’ / ’ )
g e t l a b e l s ( path + d a t a s e t + ’ / ’ )

Catherine de Weever, ing. Sebastian Wilczek page 17 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

B Modification of original code to create new classifiers

import os
import argparse
import numpy a s np
import pandas a s pd

from s k l e a r n . l i n e a r m o d e l import L o g i s t i c R e g r e s s i o n
from s k l e a r n . n e u r a l n e t w o r k import M L P C l a s s i f i e r
from s k l e a r n . e x t e r n a l s import j o b l i b

CLF NAMES = [ ”mlp” , ” l o g r e g ” ]

CLFS = [
MLPClassifier (
a l p h a =0.1 ,
h i d d e n l a y e r s i z e s =(64 , 6 4 , 6 4 ) ,
l e a r n i n g r a t e i n i t =0.001 ,
m a x i t e r =300
),
LogisticRegression () ,
]

def p a r s e a r g s ( ) :
””” P a r s e s i n p u t arguments . ”””
p a r s e r = a r g p a r s e . ArgumentParser ( )
p a r s e r . add argument ( ’−f ’ , ’−−f e a t u r e s ’ , d e s t= ’ f e a t u r e s ’ ,
help= ’ Path t o f e a t u r e s saved a s . npy . ’ )
p a r s e r . add argument ( ’−s ’ , ’−−s c o r e s ’ , d e s t= ’ s c o r e s ’ ,
help= ’ Path t o s c o r e s saved a s . c s v . ’ )
p a r s e r . add argument ( ’− l ’ , ’−−l a b e l s ’ , d e s t= ’ l a b e l s ’ , help= ’ Path t o l a b e l s saved a s . c s v . ’ )
p a r s e r . add argument ( ’−o ’ , ’−−output ’ , d e s t= ’ output ’ ,
help= ’ Path t o s a v e c l a s s i f e r s . ’ ,
d e f a u l t= ’ . / output ’ )
args = parser . parse args ()
return a r g s

def main ( i n p u t f e a t u r e s , i n p u t s c o r e , i n p u t l a b e l s , o u t p u t p a t h ) :
””” This s c r i p t f i t s t h e mlp and l o g r e g c l a s s i f e r s t o new d a t a .

P r o c e s s e s t h e f e a t u r e v e c t o r s and s c o r e s as s a v e d by p r o c e s s d a t a . py
and a . c s v f i l e c o n t a i n i n g t h e f i l e n a m e s w i t h a c c o r d i n g l a b e l s .
The l a b e l s . c s v f i l e i s e x p e c t e d t o have a column ’ Filename ’ and ’ L a b e l ’ .
The s c r i p t p r o v i d e s a b a s i c i m p l e m e n t a t i o n t o f i t t h e
mlp and l o g r e g c l a s s i f i e r s t o new d a t a .

Args :
i n p u t f e a t u r e s : Path t o f e a t u r e v e c t o r s as s a v e d by p r o c e s s d a t a . py .
i n p u t s c o r e : Path t o s c o r e s as s a v e d by p r o c e s s d a t a . py .
i n p u t l a b e l s : Path t o . c s v w i t h ’ Filename ’ and ’ L a b e l ’ column
output path : Directory to save c l a s s i f i e r s .
”””
# read input f i l e s
s c o r e s d f = pd . r e a d c s v ( i n p u t s c o r e , s e p= ’ , ’ )
l a b e l s d f = pd . r e a d c s v ( i n p u t l a b e l s , s e p= ’ , ’ )
f e a t u r e v e c s = np . l o a d ( i n p u t f e a t u r e s )

# f i l t e r i n v a l i d samples
v a l i d i d x s = s c o r e s d f . V a l i d . v a l u e s == 1
valid features = feature vecs [ valid idxs ]
v a l i d f i l e n a m e s = s c o r e s d f . Filename . v a l u e s [ v a l i d i d x s ]

Catherine de Weever, ing. Sebastian Wilczek page 18 of 19

 Deepfake detection through PRNU and logistic regression analyses
Project Paper

#l o g r e g s a m p l e s
f e a t u r e v e c s = np . nan to num ( f e a t u r e v e c s )
f i l e n a m e s l o g = s c o r e s d f . Filename . v a l u e s
np . nan to num ( f i l e n a m e s l o g )

# g e t l a b e l s f o r v a l i d samples
labels = [ ]
f o r f i l e n a m e in v a l i d f i l e n a m e s :
l a b e l s r o w = l a b e l s d f . l o c [ l a b e l s d f [ ’ Filename ’ ] == f i l e n a m e ]
i f l a b e l s r o w . s i z e == 0 :
print ” M i s s i n g l a b e l f o r : ” , f i l e n a m e
e x i t ( −1)
l a b e l s . append ( l a b e l s r o w [ ’ L a b e l ’ ] . v a l u e s [ 0 ] )

# g e t l a b e l s f o r a l l samples
labels log = []
f o r f i l e n a m e in f i l e n a m e s l o g :
l a b e l s r o w = l a b e l s d f . l o c [ l a b e l s d f [ ’ Filename ’ ] == f i l e n a m e ]
i f l a b e l s r o w . s i z e == 0 :
print ” M i s s i n g l a b e l f o r : ” , f i l e n a m e
e x i t ( −1)
l a b e l s l o g . append ( l a b e l s r o w [ ’ L a b e l ’ ] . v a l u e s [ 0 ] )

# create save f o l d e r
i f not o s . path . e x i s t s ( o u t p u t p a t h ) :
o s . m a k edirs ( o u t p u t p a t h )

f o r name , c l f in zip (CLF NAMES, CLFS ) :

i f name == ” l o g r e g ” :
c l f . f i t ( feature vecs , labels log )
else :
clf . f i t ( valid features , labels )
j o b l i b . dump( c l f , o s . path . j o i n ( o u t p u t p a t h , name + ’ . p k l ’ ) )

if name == ’ m a i n ’ :
args = parse args ()
main ( a r g s . f e a t u r e s , a r g s . s c o r e s , a r g s . l a b e l s , a r g s . output )

Catherine de Weever, ing. Sebastian Wilczek page 19 of 19