Deepfake Detection Using Machine Learning

A PROJECT REPORT

21CSC305P –MACHINE LEARNING

(2021 Regulation)
III Year/ V Semester
Academic Year: 2024 -2025

Submitted by
O. Pradeep-RA2211026010032
K. J. Tilak Reddy-RA2211026010059
P. Bharadwaj-RA2211026010061
N. Umesh Karthik-RA2211026010064

Under the Guidance of

Dr. C. Sherin Shibi
Designation
Department of Computational Intelligence

in partial fulfillment of the requirementsfor the degree of

BACHELOR OF TECHNOLOGY
in
COMPUTER SCIENCE ENGINEERING
with specialization in
ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING

SCHOOL OF COMPUTING
COLLEGE OF ENGINEERING AND TECHNOLOGY
SRM INSTITUTE OF SCIENCE ANDTECHNOLOGY
KATTANKULATHUR- 603 203
NOVEMBER 2024

1
 SRM INSTITUTE OF SCIENCE AND
TECHNOLOGY
KATTANKULATHUR – 603 203
BONAFIDE CERTIFICATE

Certified that 21CSC305P - MACHINE LEARNING project

report titled “Deepfake Detection Using Machine Learning”
is the bonafide work of “O. Pradeep-RA2211026010032
K. J. Tilak Reddy-RA2211026010059, P. Bharaddwaj-
RA2211026010061, N. Umesh Karthik-RA2211026010064”
who carried out the task of completing the project within the
allotted time.

SIGNATURE SIGNATURE
Dr. C. Sherin Shibi Dr. R. Annie Uthra
Assistant Professor Head of the Department
Department of Computational Intelligence Professor
SRM Institute of Science and Technology Department of Computational Intelligence
Kattankulathur SRM Institute of Science and Technology
Kattankulathur

2
 ABSTRACT

In recent years, advancements in artificial intelligence have enabled the creation of hyper-
realistic yet fraudulent media content, commonly known as deep fakes. These synthetic videos
and images pose a significant threat to information integrity, privacy, and public trust. This
project focuses on developing a robust machine learning model capable of detecting deep fakes
with high accuracy. Leveraging convolutional neural networks (CNNs), our model is trained
on publicly available datasets such as Face Forensics++ and the Deep Fake Detection
Challenge (DFDC) dataset, which contain both authentic and manipulated media. The detection
model incorporates advanced preprocessing techniques, such as face detection and alignment,
to improve feature extraction and enhance model performance.

Performance metrics such as accuracy, precision, recall, and F1-score were used to evaluate
the model’s effectiveness. Experimental results indicate that our approach achieves a high
detection rate, showcasing the potential of machine learning algorithms in identifying deep
fake content. This work contributes to the ongoing efforts to combat misinformation and
proposes a scalable solution for real-world applications in media and security. Future
enhancements could include exploring alternative model architectures and larger datasets to
further improve detection accuracy.

3
 TABLE OF CONTENTS

ABSTRACT 3

LIST OF FIGURES 5
ABBREVIATIONS 6

1 INTRODUCTION 7
1.1 Programming Language
1.2 Libraries and Frameworks
1.3 Development Environment
1.4 Dataset
1.5 Operating System
2 LITERATURE SURVEY 11
2.1 Deep Fake Generation Techniques
2.2 Existing Deep Fake Detection Techniques
2.3 Public Datasets for Deep Fake Detection
2.4 Evaluation Metrics
2.5 Challenges in Deep Fake Detection
3 METHODOLOGY OF [Proposed System Name] 13
3.1 Data Collection and Preprocessing
3.2 Model Architecture
3.3 Training Procedure
3.4 Evaluation Metrics
4 RESULTS AND DISCUSSIONS 16
4.1 Model Performance
4.2 Analysis of Results
4.3 Comparison with Existing Methods
4.4 Limitations
5 CONCLUSION AND FUTURE ENHANCEMENT 18
REFERENCES 20
APPENDIX 21

4
 LIST OF FIGURES

Figure No Title of the Figure Page No

1 How Deep Fakes are created 7

2 Block Diagram 8

3 Data Set Exploration 9

4 Pre-Processing 10

5 Prediction Workflow 11

6 Evaluation Metrices 12

7 Data Processing 13

8 Model Architecture 14

9 Training Workflow 14

10 Training and Validating Loss 15

11 Training and Validating Accuracy 15

12 Deep Fake results 16

13 Result Accuracy 17

5
 ABBREVIATIONS

1. AI - Artificial Intelligence
2. ML - Machine Learning
3. DL - Deep Learning
4. GAN - Generative Adversarial Network
5. CNN - Convolutional Neural Network
6. RNN - Recurrent Neural Network
7. LSTM - Long Short-Term Memory (a type of RNN)
8. DFDC - DeepFake Detection Challenge
9. AUC - Area Under the Curve
10. ROC - Receiver Operating Characteristic
11. TP - True Positive
12. FP - False Positive
13. TN - True Negative
14. FN - False Negative
15. F1-score - F1 Score (Harmonic mean of Precision and Recall)
16. ReLU - Rectified Linear Unit (an activation function)
17. API - Application Programming Interface
18. FPS - Frames Per Second (useful in real-time processing)
19. GPU - Graphics Processing Unit
20. CPU - Central Processing Unit
21. MSE - Mean Squared Error (a common loss function)
22. IoU - Intersection over Union (used in object detection tasks)
23. SVM - Support Vector Machine
24. NLP - Natural Language Processing
25. PCA - Principal Component Analysis
26. RGB - Red, Green, Blue (color channels in images)
27. dB - Decibel (useful in audio-related fake detection)

6
 CHAPTER 1

INTRODUCTION
Deep fakes, which refer to media content (videos, images, or audio) manipulated using artificial
intelligence to convincingly impersonate real people, have emerged as a major threat in the
digital era. Created through advanced techniques like generative adversarial networks (GANs),
deep fakes can be used maliciously to spread misinformation, create fake news, or impersonate
individuals in a damaging way. The rapid evolution and accessibility of such technologies have
intensified concerns about their misuse and the potential harm they can cause in areas like
politics, business, and personal privacy.

This project focuses on developing a machine learning-based deep fake detection system. By
leveraging convolutional neural networks (CNNs), the goal is to create a model capable of
distinguishing real content from AI-generated deep fakes with high accuracy. The project
explores various preprocessing techniques and machine learning architectures to enhance
detection accuracy. The outcomes of this project can provide essential insights into the
limitations and strengths of current deep fake detection approaches, contributing to more
effective media verification tools in digital security.

Fig 1: How Deep Fakes are created

Software Requirements
To implement a deep fake detection project, you will need specific software tools and libraries.
Below is a list of typical software requirements:

7
1. Programming Language: Python (version 3.7 or later)
• Python is widely used in machine learning due to its extensive library support
and ease of use.
2. Libraries and Frameworks:
• TensorFlow or PyTorch: For building and training the deep learning models.
TensorFlow (with Keras) and PyTorch are both widely used for computer vision
and deep learning applications.
• OpenCV: For image processing tasks like face detection, cropping, resizing,
and alignment.
• NumPy and Pandas: For data manipulation and analysis.
• Scikit-Learn: For metrics evaluation and additional preprocessing tasks.
• Matplotlib and Seaborn: For visualization of results and metrics.
• Dlib (optional): Can be used for facial landmark detection if you want to
preprocess faces for more accurate alignment.

3. Development Environment:
• Jupyter Notebook: For interactive coding and experimentation. Alternatively,
IDEs like PyCharm, VS Code, or Google Colab for cloud-based development.
• CUDA (for GPU support): If using an NVIDIA GPU, CUDA enables faster
model training, which is especially helpful for deep learning.

Fig 2: Block Diagram

8
4. Dataset:
• FaceForensics++, DeepFake Detection Challenge (DFDC), or Celeb-DF:
Publicly available datasets containing real and fake videos/images for training
and testing.

Fig 3: Data Set Exploration

5. Version Control:
• Git: For tracking code changes, collaboration, and maintaining a history of
project versions.

9
6. Operating System:
• The project can be implemented on any OS, but Linux or macOS often have
better support for machine learning libraries, particularly with GPU
acceleration. Windows also works with Python and the required libraries.

Fig 4: Pre-Processing

10
 CHAPTER 2

LITERATURE SURVEY
The literature survey section provides an overview of existing research and techniques in the
field of deep fake detection. Key points to cover include:
1. Deep Fake Generation Techniques:
• Discuss common methods used to create deep fakes, such as Generative
Adversarial Networks (GANs), Autoencoders, and Variational Autoencoders
(VAEs).
• Explain how GANs, particularly, enable realistic deep fake generation by
training two networks—the generator and the discriminator—in opposition.

Fig 5: Prediction Workflow

2. Existing Deep Fake Detection Techniques:
• Summarize previous approaches, such as Convolutional Neural Networks
(CNNs), Recurrent Neural Networks (RNNs), and transformer-based models,
which have been applied to detect subtle differences in image and video frames.
• Highlight advanced methods like spatial and temporal analysis, which consider
both the visual (spatial) features and motion (temporal) features in videos.

11
 • Mention common preprocessing techniques such as face detection, cropping,
and alignment, which help focus the model on key facial features.
3. Public Datasets for Deep Fake Detection:
• List widely used datasets like FaceForensics++, DeepFake Detection Challenge
(DFDC), and Celeb-DF. These datasets provide both real and manipulated
images or videos, which are essential for training and evaluating detection
models.
• Discuss the advantages and limitations of these datasets, such as their size,
diversity, and quality.
4. Evaluation Metrics:
• Commonly used metrics include accuracy, precision, recall, F1-score, and AUC
(Area Under the Curve). These metrics help assess the model’s effectiveness in
distinguishing real from fake content.

Fig 6: Evaluation Metrices

5. Challenges in Deep Fake Detection:
• Mention the challenges, such as the evolving realism of deep fakes, the diversity
of potential manipulations, and generalization across different deep fake types.

12
 CHAPTER 3

METHODOLOGY

In the methodology section, describe the overall approach, data preparation, and model
development steps.

1. Data Collection and Preprocessing:

• Dataset: Specify the dataset(s) used, such as FaceForensics++ or DFDC. Detail
how you divided the data into training, validation, and test sets.
• Preprocessing: Describe steps such as face detection, cropping, resizing, and
normalization. You may also apply data augmentation techniques (e.g., rotation,
flipping, brightness adjustment) to enhance model robustness.

Fig 7: Data Processing

2. Model Architecture:
• Model Selection: Outline the deep learning model you chose, such as a CNN,
LSTM-CNN hybrid, or transformer-based model. Explain why this model is
suitable for detecting deep fakes.
• Architecture Details: Provide a high-level description of the model layers,
including convolutional layers, activation functions (e.g., ReLU), pooling
layers, and fully connected layers.

13
 • Regularization: Mention any regularization techniques, such as dropout, used
to prevent overfitting.

Fig 8: Model Architecture

3. Training Procedure:
• Hyperparameters: Document key hyperparameters, such as learning rate,
batch size, number of epochs, and optimizer used.
• Training Process: Describe the training and validation process, including how
you monitored metrics like loss and accuracy during training.

Fig 9: Training Workflow

14
4. Evaluation Metrics:
• Define the evaluation metrics used to measure the model’s performance,
including accuracy, precision, recall, F1-score, and AUC.

Fig 10: Training and Validating Loss

Fig 11: Training and Validating Accuracy

15
 CHAPTER 4

RESULTS

The results section presents and interprets the outcomes of your model.

1. Model Performance:
• Provide a summary of the final results based on the selected metrics (accuracy,
precision, recall, F1-score, AUC).
• Present results visually, using confusion matrices, ROC curves, and precision-
recall curves if applicable.
2. Analysis of Results:
• Compare the model’s performance on the training and test sets, discussing any
observed discrepancies.
• Highlight any specific challenges encountered, such as difficulty distinguishing
certain types of fakes or performance drops on unseen data.

Fig 12: Deep Fake results

3. Comparison with Existing Methods:
• If relevant, compare your model’s performance with existing deep fake
detection methods in the literature. Discuss if your model shows any
improvement or unique strengths.

16
4. Limitations:
• Acknowledge any limitations, such as dataset bias, limited generalizability
across different types of deep fakes, or computational constraints.

Fig 13: Result Accuracy

17
 CHAPTER 5

CONCLUSION

The conclusion summarizes the project’s findings and suggests potential directions for future
work.
1. Key Findings:
• Recap the main results, emphasizing the effectiveness of the chosen model and
methodology for deep fake detection.
2. Contributions:
• Highlight the contributions of your project to the field, such as improvements
in detection accuracy, robustness to different types of deep fakes, or innovative
model architectures.
3. Limitations and Future Work:
• Address any limitations that emerged, such as dataset constraints or
computational challenges.
• Suggest areas for future research, such as exploring larger or more diverse
datasets, testing different model architectures, or enhancing model
generalizability.
4. Significance:
• Conclude with a statement on the importance of this work in addressing the
challenges of misinformation and digital security, reinforcing the need for
ongoing research in deep fake detection.

FUTURE ENHANCEMENT
In this section, outline potential improvements that could make your deep fake detection model
more robust and effective:
1. Expanding the Dataset:
• Collect or integrate larger and more diverse datasets with various types of deep
fakes (e.g., different generative models, styles, and levels of realism). This could
improve the model’s generalizability to new, unseen types of deep fakes.
2. Improving Model Architecture:
• Explore advanced architectures like transformer-based models (e.g., Vision

18
 Transformers) that have shown success in image classification. Multi-stream
architectures combining CNNs for spatial features and RNNs or LSTMs for
temporal features (in video) could also enhance accuracy.
3. Integrating Explainable AI Techniques:
• Implement explainable AI techniques, such as Grad-CAM or LIME, to provide
insights into the model’s decision-making process. This could make the model
more interpretable and trustworthy, helping users understand why an image is
classified as real or fake.
4. Optimizing for Real-Time Detection:
• Work on reducing model complexity or using lightweight architectures (e.g.,
MobileNet, EfficientNet) to enable real-time detection. This could allow for
deep fake detection on devices with limited computational resources, such as
mobile phones.
5. Cross-Domain Generalization:
• Investigate transfer learning or domain adaptation techniques to enhance model
performance across different types of deep fakes. Training on multiple datasets
with diverse sources may help the model generalize better across platforms and
video formats.
6. Adversarial Defense Mechanisms:
• Study and implement adversarial training or robust feature extraction
techniques to defend against adversarial attacks, which can trick deep fake
detectors by subtly modifying input data.
7. Developing Ethical Guidelines:
• Propose guidelines for ethical use of deep fake detection, emphasizing privacy
considerations, false positive/negative implications, and responsible handling
of data.

19
 REFERENCES

The References section should list all sources we consulted, such as research papers, articles,
datasets, and any software documentation used. Here’s an outline for organizing references in
APA format:

1. Research Papers and Articles:

• Include references to scholarly articles on deep fake detection, GANs, CNNs,
and any specific techniques you used. For example:
• Karras, T., Laine, S., & Aila, T. (2019). A style-based generator
architecture for generative adversarial networks. Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition, 4401-
4410.

2. Datasets:
• Cite datasets like FaceForensics++, DFDC, or Celeb-DF. For example:
• Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., &
Nießner, M. (2019). FaceForensics++: Learning to detect manipulated
facial images. Proceedings of the IEEE International Conference on
Computer Vision, 1-11.

3. Software and Libraries:

• List libraries such as TensorFlow, PyTorch, OpenCV, and any other tools used.
For example:
• Abadi, M., Barham, P., Chen, J., et al. (2016). TensorFlow: A system
for large-scale machine learning. 12th USENIX Symposium on
Operating Systems Design and Implementation, 265-283.

4. Web Articles:
• For any online resources or articles, include the URL and date accessed if
necessary.

20
 APPENDIX

The Appendix section is for supplementary materials that add value to your report but aren’t
essential to the main body. This may include:
1. Code Snippets:
• Add important code snippets for model architecture, preprocessing, or
evaluation. This helps readers understand specific implementation details
without overwhelming the main report.
2. Detailed Hyperparameter Configurations:
• List the hyperparameters used in model training, such as learning rate, batch
size, and number of layers. This allows others to replicate your results or
experiment with similar settings.
3. Additional Visualizations:
• Include extra figures or charts, such as training and validation loss curves,
confusion matrices for each class, or sample images with model predictions.
This helps illustrate the model’s performance more comprehensively.
4. Dataset Description:
• Provide detailed information on the dataset used, such as sample counts, class
distribution, and any specific preprocessing techniques applied.
5. Glossary of Terms:
• If the report includes technical terms or abbreviations, list them in a glossary.
This makes the report more accessible to readers unfamiliar with the
terminology.
6. Experiment Logs or Configuration Files:
• If applicable, include experiment logs, configuration files (e.g., YAML or JSON
files for model parameters), or environment details (e.g., OS version, GPU
type). This can be useful for reproducibility.

21