LEVEL UP THE DEEPFAKE DETECTION: A METHOD TO EFFECTIVELY DISCRIMINATE

IMAGES GENERATED BY GAN ARCHITECTURES AND DIFFUSION MODELS

Luca Guarnera⋆, Oliver Giudice† , Sebastiano Battiato⋆

⋆
Department of Mathematics and Computer Science, University of Catania, Italy
†
Applied Research Team, IT dept., Banca d’Italia, Rome, Italy
arXiv:2303.00608v1 [cs.CV] 1 Mar 2023

{luca.guarnera, sebastiano.battiato}@unict.it, [email protected]

ABSTRACT data. Stable Diffusion [6] and DALL-E 2 [7] are the most fa-
The image deepfake detection task has been greatly addressed mous state of the art DMs, based on the text-to-image trans-
by the scientific community to discriminate real images from lation operation. As demonstrated in [8], DMs are able to
those generated by Artificial Intelligence (AI) models: a bi- produce even better realistic images than GANs, since GANs
nary classification task. In this work, the deepfake detection generate high-quality samples but are demonstrated to fail in
and recognition task was investigated by collecting a dedi- covering the entire training data distribution.
cated dataset of pristine images and fake ones generated by
9 different Generative Adversarial Network (GAN) architec- To effectively counteract the illicit use of synthetic data
tures and by 4 additional Diffusion Models (DM). generated by GANs and DMs, new deepfake detection and
A hierarchical multi-level approach was then introduced recognition algorithms are needed. As far as image deep-
to solve three different deepfake detection and recognition fake detection methods in state of the art are concerned, they
tasks: (i) Real Vs AI generated; (ii) GANs Vs DMs; (iii) mostly focus on binary detection (Real Vs. AI generated [9,
AI specific architecture recognition. Experimental results 10]) . Interesting methods in state of the art already demon-
demonstrated, in each case, more than 97% classification strated to effectively discriminate between different GAN ar-
accuracy, outperforming state-of-the-art methods. chitectures [11, 12, 13]. Methods to detect DMs and recog-
nize them have been proposed just recently [14, 15].
Index Terms— Deepfake Detection, Generative Adver-
sarial Nets, Diffusion Models, Multimedia Forensics
In order to level up the deepfake detection and recognition
task, the objective of this paper and the main contribution is
1. INTRODUCTION
to classify an image among 14 different classes: 9 GAN ar-
The term deepfake refers to all those multimedia contents chitectures, 4 DMs engines and 3 pristine datasets (labeled
generated an AI model. The most common deepfake creation as belonging to the same “real” class). At first, a dedicated
solutions are those based on GANs [1] which are effectively dataset of images was collected. Then, a novel multi-level
able to create from scratch or manipulate a multimedia data. hierarchical approach exploiting ResNET models was devel-
In a nutshell, GANs are composed by two neural networks: oped and trained. The proposed approach consists of 3 lev-
the Generator (G) and the Discriminator (D). G creates new els of classification: (Level 1) Real Vs AI-generated images;
data samples that resemble the training data, while D evalu- (Level 2) GANs Vs DMs; (Level 3) recognition of specific AI
ates whether a sample is real (belonging to the training set) (GAN/DM) architectures among those represented in the col-
or fake (generated by the G). A GAN must be trained until D lected dataset. Experimental results demonstrated the effec-
is no longer able to detect samples generated by G, in other tiveness of the proposed solution, achieving more than 97%
words, when D starts to be fooled by G. Several surveys on accuracy on average for each task, exceeding the state of the
methods dealing with GAN-based approaches for the creation art. Moreover, the hierarchical approach can be used to ana-
and detection of deepfakes, have been proposed in [2, 3]. lyze multimedia data in depth to reconstruct its history (foren-
Recently, DMs [4, 5] are arousing interest thanks to their sic ballistics) [16], a task poorly addressed by the scientific
photo-realism and also to a wide choice in output control community on synthetic data.
given to the user. In contrast to GANs, DMs are a class
of probabilistic generative models that aims to model com- This paper is organized as follows: Section 2 and Section
plex data distributions by iteratively adding noise to a random 3 describe the dataset and the proposed approach built upon
noise vector input for the generation of new realistic samples it respectively. Experimental results and comparison are pre-
and, using them as basis, proceed to reconstruct the original sented in Section 4. Finally, Section 5 concludes the paper.
 Fig. 1. Examples of images collected from different datasets and images generated by different GANs and DMs.

2. DATASET DETAILS Classification Train Val Test

Task 50% 20% 30%
The dataset employed in this study is a dedicated collec- Total Images 28,000 4.200 7.000
14-classes
tion of images: real/pristine images collected from CelebA #Img ∀ class 2,000 300 500
Total Images 26,000 3,900 6,500
[17], FFHQ1 , and ImageNet [18] datasets and synthetic data 13-classes
#Img ∀ class 2,000 300 500
generated by 9 different GAN engines (AttGAN [19], Cy-
Total Images 46,480 11,620 24,900
cleGAN [20], GDWCT [21], IMLE [22], ProGAN [23], L1
#Img ∀ class 23,240 5,810 12,450
StarGAN [24], StarGAN-v2 [25], StyleGAN [26], Style- Total Images 23,800 5,950 12,750
GAN2 [27]) and 4 text-to-image DM architectures (DALL-E L2
#Img ∀ class 11,900 2,975 6,375
2 [7], GLIDE [28], Latent Diffusion [6]2 . For each consid- Total Images 12.600 3,150 6,750
ered GAN, 2, 500 images (a total of 22, 500) were generated GANs
#Img ∀ class 1,400 350 750
L3
while for the DMs, 5, 000 images were created for each ar- Total Images 11,200 2,800 6,000
DMs
chitecture employing more than 800 random sentences, for #Img ∀ class 2,800 700 1,500
a total of 20, 000 images. Overall, the total number of syn-
thetic data consists of 42, 500 images. Finally, for each real Table 1. Overview of the images employed for training,
dataset (CelebA, FFHQ and ImageNet) 13, 500 images were validation and test sets (last three columns with indication
considered, for a total of 40, 500. of % of samples). The first column denotes the classifica-
Table 1 summarizes the numbers of the obtained dataset tion task (e.g., 14 − classes is the flat classification task
with respect to each level and to division of training, vali- with 14 classes; L1 refers to the Level 1 of hierarchy. The
dation and test sets employed for the experiments. Figure 1 T otalImages rows indicate the total number of images em-
shows several examples of the obtained dataset. ployed for training, validation, and testing phases. The
#Img∀class represents the number of samples considered
for each class.
3. MULTI-LEVEL DEEPFAKE DETECTION AND
Thus this level is implemented as a binary ResNET-34 classi-
RECOGNITION
fier. Given that an image was previously classified as gener-
ated by an AI, Level 2 furtherly analyzes images to discrimi-
The dataset collected and described in the previous section
nate between those generated by a GAN from those generated
was preliminarly investigated as a “flat” classification task.
by a DM. Thus this level is implemented as another binary
This was carried out by employing a single ResNET-34 en-
ResNET-34 classifier. Finally, given that an image was previ-
coder with 14 classes as output layer. In addition, a further
ously classified as generated by a GAN or by a DM, the last
test was carried out by removing every image belonging to the
level solves the task of recognizing the specific architecture
real class (13 classes as output layer). The trained ResNET-
between those considered in the dataset described in Section
34 model showed that in this last case it is able to achieve
2. To do this, Level 3 is divided into two sub-modules: “Level
greater accuracy score. This gave the idea that a hierarchical
3-GANs” to discriminate the specific GAN, implemented as
approach could lead to even better results also giving a bit of
a 9-classes ResNET-34 classifier; and “Level 3-DMs” imple-
explainability on the analyzed image.
mented as 4-classes ResNET-34 Diffusion Model classifier.
The proposed multi-level deepfake detection and recogni- Figure 2 summarizes the overall approach.
tion approach consists of 3 levels. Level 1 has the objective
to detect real data from those created AI architectures (so all
synthetic data were labeled as belonging to the same class). 4. EXPERIMENTAL RESULTS AND COMPARISON

1 https://github.com/NVlabs/ffhq-dataset Experiments with ResNET architecture were performed con-

2 a.k.a. Stable Diffusion: https://github.com/CompVis/stable-diffusion sidering the following parameters for training: batchsize =
Fig. 2. Execution flow of the proposed hierarchical approach. Level 1 classifies images as real or AI-generated. Level 2 defines
whether the input images were created by GAN or DM technologies. Level 3, composed of two sub-modules, solves the AI
architecture recognition task. The dashed arrows represent an optional flow (e.g., in the case the input image is real, it will not
be analyzed by the next levels).

30, learningrate = 0.00001, optimizer Stochastic Gradient Level 1 Level 2 Level 3

Descent (SGD) with momentum = 0.9 and Cross Entropy Real Vs AI GANs Vs DMs GANs DMs
Loss. All images were resized to a resolution of 256x256. ResNET-18 95,24 97,68 95,54 97.29
ResNET-34 97,63 98,01 97,77 98,02
As regards 14-classes, 13-classes and Level 1 models were
trained for 150 epochs. The remaining three models (one for Table 2. Comparison of classification accuracy value (%)
Level 2 and two for Level 3 classification) were trained for obtained between ResNET-18 and ResNET-34 architectures
100 epochs. This difference is due to the different amount of with respect to the proposed hierarchical approach.
images at corresponding level (see Table 1).
Four different instances of the ResNET encoder were em- cal approach described in Section 3 was developed. The best
ployed for the proposed multi-level deepfake detection and results obtained are the following:
recognition task. Each ResNET model was properly trained • Level 1: classification accuracy of 97, 63%;
with a corresponding sub-dataset composed as shown in Ta- • Level 2: classification accuracy of 98, 01%;
ble 1. In particular, the Pytorch implementation of ResNET • Level 3: an accuracy of 97, 77% was obtained for the
and each training started from the weights pre-trained on Im- GAN recognition task and an accuracy of 98, 02% for
ageNet3 . For each model, a fully connected layer with an the DM one.
output size equal to the number of classes of the correspond-
ing classification level followed by a SoftMax was added to Figure 3 shows the accuracy and error trends obtained in the
the last layer of ResNET encoders. training and test phases for each epoch and for each model
As far as model selection is concerned, ResNET archi- of the hierarchical approach. Table 2 shows that ResNET-34
tecture was chosen among other as already demonstrated to achieves the best results compared with ResNET-18.
be effective in higher level deepfake recognition tasks [29]. The final solution was compared with various state-of-the-
Given the presented solution, 4 models have to be run simul- art works ([9, 10, 13, 11, 12, 14]) demonstrating to achieve
taneously on a single GPU. This limited the dimension of the best results in each task. In Table 3, the Level 3 DMs col-
useable models. Thus only ResNET-18 and ResNET-34 were umn does not show some values given that corresponding ap-
took into account. The ResNET-34 architecture was selected proaches does not cover the corresponding task. Therefore,
as final solution for achieving the best results (see Table 2). most of the methods do not seem to be able to effectively dis-
At first, the “flat” 14-classes classification task obtained tinguish between different DMs. This is because DMs leave
an overall accuracy of only 94, 85% in the test-set. In or- different traces on synthetic images than those generated by
der to slightly improve this result, first class (pristine images) GAN engines. This claim is further empirically demonstrated
was removed and the model retrained. With the remaining by the results reported in Table 3 on the “Real Vs AI” col-
13 classes (only synthetic images generated by all considered umn, where, due to the presence of DMs-generated images,
GAN and DM engines) a bit better overall accuracy of 95, 4% the classification accuracy values of the various methods are
was obtained. Taking advantage of this results, the hierarchi- sensibly lower than the 97, 63% obtained by the proposed ap-
proach. For this reason, a column regarding Level 2 task was
3 https://pytorch.org/vision/stable/models.html not added, as the classification results of the various methods
Fig. 3. Trend of accuracy and error values obtained in training and test phases for each epoch of the three levels of the proposed
hierarchical approach. Each column represents a classification level.

Level 1 Level 3 6. REFERENCES

Real Vs AI GANs DMs
AutoGAN [9] 68.5 80,3 — [1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu,
Fakespotter [10] 74,22 95,32 — D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio,
EM [13] 86.57 95,02 — “Generative Adversarial Nets,” in Advances in Neural
DCT[11] 87,20 95,89 — Information Processing Systems, 2014, pp. 2672–2680.
Wang et al. [12] 78.54 97.32 —
DE-FAKE [14] 90,52 – 93,45 [2] M. Masood, M. Nawaz, K. M. Malik, A. Javed, A. Ir-
Our 97,63 97,77 98,02 taza, and H. Malik, “Deepfakes Generation and Detec-
Table 3. Comparison with state of the art approaches. Classi- tion: State-of-the-art, Open Challenges, Countermea-
fication accuracy value is reported (%). — are reported where sures, and Way Forward,” Applied Intelligence, pp. 1–
the method can not handle the corresponding task. 53, 2022.
turn out to be significantly lower (almost random classifiers) [3] E. Nowroozi, A. Dehghantanha, R. M Parizi, and
than the classification accuracy of 98, 01% obtained by the K. Choo, “A Survey of Machine Learning Techniques
proposed approach. Finally, all state of the art methods, in- in Adversarial Image Forensics,” Computers & Security,
cluding the proposed one are able to generalize well in terms vol. 100, pp. 102092, 2021.
of distinguishing between GAN architectures that created the
synthetic data (GANs column). [4] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and
S. Ganguli, “Deep Unsupervised Learning Using
Nonequilibrium Thermodynamics,” in International
5. CONCLUSIONS Conference on Machine Learning. PMLR, 2015, pp.
2256–2265.
In this paper, a deepfake detection and recognition solution is
presented. The proposed solution is able to recognize if an [5] J. Ho, A. Jain, and P. Abbeel, “Denoising Diffusion
image was generated among 9 different GAN engines and 4 Probabilistic Models,” Advances in Neural Information
DM models. However, the proposed solution demonstrated to Processing Systems, vol. 33, pp. 6840–6851, 2020.
outperform state of the art in all tasks.
Future experiments could evaluate the robustness of the [6] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and
proposed method in real-world contexts (JPEG compression, B. Ommer, “High-Resolution Image Synthesis With
scaling, etc.). Also, the possibility to identify some analytical Latent Diffusion Models,” in Proceedings of the
traces [15, 11, 13] will be investigated just to exploit unusual IEEE/CVF Conference on Computer Vision and Pattern
statistics embedded on images generated by DMs. Recognition, 2022, pp. 10684–10695.
 [7] A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and [19] Z. He, W. Zuo, M. Kan, S. Shan, and X. Chen,
M. Chen, “Hierarchical Text-Conditional Image “AttGAN: Facial Attribute Editing by Only Changing
Generation with Clip Latents,” arXiv preprint What You Want,” IEEE Transactions on Image Pro-
arXiv:2204.06125, 2022. cessing, vol. 28, no. 11, pp. 5464–5478, 2019.
[8] P. Dhariwal and A. Nichol, “Diffusion Models Beat [20] J. Zhu, T. Park, P. Isola, and A. Efros, “Unpaired Image-
GANs on Image Synthesis,” Advances in Neural In- to-Image Translation Using Cycle-Consistent Adversar-
formation Processing Systems, vol. 34, pp. 8780–8794, ial Networks,” in Proceedings of the IEEE International
2021. Conference on Computer Vision, 2017, pp. 2223–2232.
[9] X. Zhang, S. Karaman, and S. Chang, “Detecting and [21] W. Cho, S. Choi, D. K. Park, I. Shin, and J. Choo,
Simulating Artifacts in GAN Fake Images,” in 2019 “Image-to-Image Translation via Group-Wise Deep
IEEE international workshop on information forensics Whitening-and-Coloring Transformation,” in Proceed-
and security (WIFS). IEEE, 2019, pp. 1–6. ings of the IEEE/CVF Conference on Computer Vision
and Pattern Recognition, 2019, pp. 10639–10647.
[10] R. Wang, F. Juefei-Xu, L. Ma, X. Xie, Y. Huang,
J. Wang, and Y. Liu, “Fakespotter: A Simple Yet Ro- [22] K. Li, T. Zhang, and J. Malik, “Diverse Image Synthe-
bust Baseline for Spotting AI-Synthesized Fake Faces,” sis from Semantic Layouts via Conditional IMLE,” in
arXiv preprint arXiv:1909.06122, 2019. Proceedings of the IEEE International Conference on
Computer Vision, 2019, pp. 4220–4229.
[11] O. Giudice, L. Guarnera, and S. Battiato, “Fighting
Deepfakes by Detecting GAN DCT Anomalies,” Jour- [23] T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progres-
nal of Imaging, vol. 7, no. 8, pp. 128, 2021. sive Growing of GANs for Improved Quality, Stability,
and Variation,” arXiv preprint arXiv:1710.10196, 2017.
[12] S. Wang, O. Wang, R. Zhang, A. Owens, and A. Efros,
“CNN-Generated Images are Surprisingly Easy to [24] Y. Choi, M. Choi, M. Kim, J. Ha, S. Kim, and J. Choo,
Spot... for Now,” in Proceedings of the IEEE/CVF Con- “StarGAN: Unified Generative Adversarial Networks
ference on Computer Vision and Pattern Recognition, for Multi-Domain Image-to-image Translation,” in Pro-
2020, pp. 8695–8704. ceedings of the IEEE Conference on Computer Vision
and Pattern Recognition, 2018, pp. 8789–8797.
[13] L. Guarnera, O. Giudice, and S. Battiato, “Fighting
Deepfake by Exposing the Convolutional Traces on Im- [25] Y. Choi, Y. Uh, J. Yoo, and J. Ha, “StarGAN v2: Diverse
ages,” IEEE Access, vol. 8, pp. 165085–165098, 2020. Image Synthesis for Multiple Domains,” in Proceedings
of the IEEE/CVF Conference on Computer Vision and
[14] Z. Sha, Z. Li, N. Yu, and Y. Zhang, “DE-FAKE:
Pattern Recognition, 2020, pp. 8188–8197.
Detection and Attribution of Fake Images Generated
by Text-to-Image Diffusion Models,” arXiv preprint [26] T. Karras, S. Laine, and T. Aila, “A Style-based Genera-
arXiv:2210.06998, 2022. tor Architecture for Generative Adversarial Networks,”
in Proceedings of the IEEE/CVF Conference on Com-
[15] R. Corvi, D. Cozzolino, G. Zingarini, G. Poggi,
puter Vision and Pattern Recognition, 2019, pp. 4401–
K. Nagano, and L. Verdoliva, “On the Detection of Syn-
4410.
thetic Images Generated by Diffusion Models,” arXiv
preprint arXiv:2211.00680, 2022. [27] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehtinen,
and T. Aila, “Analyzing and Improving the Image Qual-
[16] L. Guarnera, O. Giudice, and S. Battiato, “Deep-
ity of StyleGAN,” in Proceedings of the IEEE/CVF
fake Style Transfer Mixture: A First Forensic Ballistics
Conference on Computer Vision and Pattern Recogni-
Study on Synthetic Images,” in International Confer-
tion, 2020, pp. 8110–8119.
ence on Image Analysis and Processing. Springer, 2022,
pp. 151–163. [28] A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam,
P. Mishkin, B. McGrew, I. Sutskever, and M. Chen,
[17] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep Learning
“Glide: Towards Photorealistic Image Generation and
Face Attributes in the Wild,” in Proceedings of the IEEE
Editing with Text-Guided Diffusion Models,” arXiv
International Conference on Computer Vision, 2015, pp.
preprint arXiv:2112.10741, 2021.
3730–3738.
[29] L. Guarnera, O. Giudice, M. Nießner, and S. Battiato,
[18] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh,
“On the Exploitation of Deepfake Model Recognition,”
S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein,
et al., “ImageNet Large Scale Visual Recognition Chal- in Proceedings of the IEEE/CVF Conference on Com-
lenge,” International Journal of Computer Vision, vol. puter Vision and Pattern Recognition, 2022, pp. 61–70.
115, pp. 211–252, 2015.