COMBINING COMPUTER VISION TECHNIQUES AND

INTRAFRAME NOISE METHODS TO DETECT A

DEEPFAKE
Maya P Shelke Ajinkya Kharade Shubham Arakh
Department of Information Technology, Department of Information Technology, Department of Information Technology,
JSPM’s RSCOE, Tathawade, India. JSPM’s RSCOE, Tathawade, India. JSPM’s RSCOE, Tathawade, India.
[email protected] [email protected] [email protected]

Nihar Ranjan Pranav Gaikwad Analp Kalore

Department of Information Technology, Department of Information Technology, Department of Information Technology,
JSPM’s RSCOE, Tathawade, India. JSPM’s RSCOE, Tathawade, India. JSPM’s RSCOE, Tathawade, India.
[email protected] [email protected] [email protected]

ABSTRACT: This paper presents an implementation approach for techniques, we seek to create a model that can reliably detect
a DeepFake detection system aimed at addressing the escalating Deepfake videos with high accuracy and efficiency. This research
concern of manipulated media. DeepFake videos, fueled by represents a critical step towards combating the spread of
advancements in artificial intelligence, pose significant threats to misinformation and preserving the integrity of digital media in the
the integrity of digital content and the spread of misinformation. face of evolving AI technologies.
Proposed implementation leverages a convolutional neural
network (CNN) architecture for accurate detection of DeepFake Background: The proliferation of Deepfake videos in recent years
videos. Key components of the implementation include data pre- has raised significant concerns regarding the spread of
processing, model training, design, and evaluation. Real and fake misinformation and manipulation of public opinion.
video datasets are collected and processed to extract frames,
forming the basis for model training. A CNN model architecture is
designed and trained using a diverse dataset, with careful
consideration given to data augmentation and hyperparameter
optimization. The implementation employs multiprocessing
techniques for efficient video processing and utilizes Keras for
model development and evaluation. The efficiency of the suggested
strategy is demonstrated by the experimental findings, showcasing
superior performance compared to existing methods in terms of
accuracy and robustness. Overall, the implementation offers a
comprehensive framework for DeepFake detection, contributing
to the ongoing efforts to combat the proliferation of manipulated
media and safeguard the integrity of digital content. Fig 1. Growth of deepfake in recent years

KEYWORDS: DeepFake detection, convolutional neural These videos, generated using sophisticated artificial intelligence
networks, misinformation, public opinion manipulation, Data techniques, have the capability to convincingly depict individuals
Preprocessing, Model Training, Evaluation Metrics, Data saying or doing things they never actually did. As a result, they pose
Augmentation, Keras Framework. a serious threat to the integrity of digital media and have prompted
widespread alarm among researchers, policymakers, and the public.
I. INTRODUCTION:
Problem Statement: The rapid advancement of AI technology has
The proliferation of Deepfake videos in recent years has raised enabled the creation of Deepfake videos with unprecedented
significant concerns regarding the spread of misinformation and realism, making them increasingly difficult to detect. This poses a
manipulation of public opinion. These videos, generated using significant challenge for platforms, media outlets, and individuals
sophisticated artificial intelligence techniques, have the capability to who rely on digital content for information and entertainment.
convincingly depict individuals saying or doing things they never Existing Deepfake detection methods often fall short in accurately
actually did. As a result, they pose a serious threat to the integrity of identifying manipulated videos, leading to concerns about the
digital media and have prompted widespread alarm among unchecked spread of misinformation.
researchers, policymakers, and the public.
Objective: Therefore, there is a pressing need for robust detection
The rapid advancement of AI technology has enabled the creation of systems capable of effectively distinguishing between real and fake
DeepFake videos with unprecedented realism, making them videos. In response to this need, this implementation research aims
increasingly difficult to detect. This poses a significant challenge for to develop an accurate and efficient Deepfake detection system
platforms, media outlets, and individuals who rely on digital content using convolutional neural networks (CNNs). By leveraging deep
for information and entertainment. Existing DeepFake detection learning techniques, we seek to create a model that can reliably
methods often fall short in accurately identifying manipulated detect Deepfake videos with high accuracy and efficiency.
videos, leading to concerns about the unchecked spread of
Contributions: This research represents a critical step towards
misinformation.
combating the spread of misinformation and preserving the integrity
Therefore, there is a pressing need for robust detection systems of digital media in the face of evolving AI technologies. The
capable of effectively distinguishing between real and fake videos. proposed DeepFake detection system has the potential to
In response to this need, this implementation research aims to significantly enhance the capabilities of existing detection methods,
develop an accurate and efficient Deepfake detection system using thereby mitigating the harmful effects of manipulated media on
convolutional neural networks (CNNs). By leveraging deep learning society.
 II. LITERATURE REVIEW: Traditional methods, while less data-intensive, often struggle with
the nuanced features of DeepFake videos and may not generalize
In recent years, proliferation of DeepFake technology has raised well to unseen manipulations. Additionally, many existing
significant concerns regarding the authenticity and integrity of approaches focus primarily on visual artifacts, overlooking other
digital media. As a result, researchers and practitioners have devoted modalities such as audio, which can also be manipulated to create
considerable attention to the development of detection methods convincing DeepFakes.
capable of discerning between genuine and manipulated content. A
comprehensive review of existing literature reveals a diverse array Justification: Despite the challenges, the urgency of developing
of approaches, ranging from traditional image processing techniques robust DeepFake detection methods cannot be overstated. With the
to state-of-the-art deep learning models. proliferation of DeepFake technology and its potential to deceive
and manipulate, there is a critical need for reliable detection systems
Traditional methods for DeepFake detection often rely on capable of discerning between authentic and manipulated content.
handcrafted features and heuristic rules to identify anomalies By building upon the strengths of deep learning while addressing its
indicative of manipulation. These approaches, while effective in limitations, we aim to contribute to the advancement of DeepFake
certain contexts, are limited by their reliance on predefined features detection technology.
and may struggle to generalize to new types of manipulation. In
contrast, deep learning-based methods have emerged as a promising Our justification for focusing on deep learning stems from its ability
alternative, leveraging neural networks to automatically learn to learn complex patterns and representations directly from data,
discriminative features from data. Convolutional neural networks potentially capturing subtle cues indicative of manipulation.
(CNNs) in particular have shown remarkable success in capturing Additionally, we recognize the importance of multi-modal
intricate patterns in images and videos, enabling more nuanced approaches that consider not only visual but also auditory and
detection of DeepFake content. contextual cues. Through our research, we aim to develop a
comprehensive DeepFake detection framework that combines the
However, while deep learning approaches offer considerable strengths of deep learning with insights from traditional methods to
potential, they are not without their challenges. One significant create a robust and versatile solution.
hurdle is the need for large labeled datasets for training, which can
be difficult and expensive to acquire. Additionally, deep learning III. METHODOLOGY:
models are susceptible to adversarial attacks, where subtle
perturbations to input data can lead to misclassification. Despite The methodology employed in this study encompasses a systematic
these challenges, the rapid advancement of deep learning techniques approach to developing an effective DeepFake detection system.
presents an opportunity to develop more robust and reliable Central to this methodology is the integration of state-of-the-art
DeepFake detection systems. techniques in data collection, model architecture design, training
procedures, evaluation metrics, and experimental setup. Each
Review of Existing Methods: The landscape of DeepFake detection component is carefully designed to ensure the robustness, accuracy,
methods is multifaceted, comprising traditional techniques and and generalization capabilities of the proposed detection system.
modern deep learning approaches. Traditional methods often rely on
handcrafted features such as facial landmarks, texture analysis, and
motion inconsistencies to identify manipulated content. These
methods have been foundational in the field but are limited in their
ability to handle the complexity and diversity of DeepFake videos.

Fig 2. Existing technologies architecture

A viable substitute that makes use of neural networks' ability to

automatically extract discriminative characteristics from data is deep
learning-based methods. Especially effective are convolutional Fig 3. Over system architecture
neural networks (CNNs), which can recognize complex patterns in
Data Collection: A diverse and comprehensive dataset comprising
pictures and movies. CNNs may be trained on vast datasets of real
both real and fake videos is essential for training and evaluating the
and modified information to identify minute variations that point to
DeepFake detection model. The data collection process involves
manipulation.
sourcing videos from various sources, including online repositories,
Comparison and Critique: While deep learning techniques have social media platforms, and curated datasets. Special attention is
shown to be rather effective in identifying DeepFake videos, they paid to ensuring the representativeness and quality of the dataset,
are not without limitations. One challenge is the need for extensive with efforts made to cover a wide range of scenarios and
labeled datasets for training, which can be costly and time- manipulations. Data preprocessing techniques may be applied to
consuming to acquire. Furthermore, in adversarial attacks against standardize the format, resolution, and quality of the videos, thereby
deep learning systems, small modifications to the input data might enhancing the model's ability to learn discriminative features.
result in misclassification.
 Fig 7. Define Training

Fig 8. Run 100 Epoch

Evaluation Metrics: The performance of the DeepFake detection

Fir 4. Framing formation from image dataset
model is evaluated using standard evaluation metrics to assess its
effectiveness in distinguishing between real and fake videos. Metrics
such as accuracy, precision, recall, and F1-score provide quantitative
measures of the model’s performance across different classes and
thresholds. Additionally, visualizations such as confusion matrices,
ROC curves, and precision-recall curves may be employed to gain
insights into the model’s behavior and its ability to handle varying
levels of uncertainty.

Fig 5. Data preprocessing

Model Architecture: The architecture of the DeepFake detection

model plays a crucial role in its performance and effectiveness. In
this study, a deep learning-based approach is adopted, leveraging
convolutional neural networks (CNNs) due to their ability to
automatically learn hierarchical representations from raw input data.
Multiple convolutional and pooling layers make up the model
architecture, which is followed by fully linked layers for
classification. The design choices are guided by principles of feature Fig 9. Training Accuracy
extraction, spatial hierarchies, and parameter optimization to
maximize the model’s discriminative power.

Fig 6. Formation of CNN model

Training Procedure: The training procedure involves optimizing

the parameters of the DeepFake detection model using the collected
dataset. Data augmentation techniques such as rotation, flipping, and Fig 10. Training Loss
scaling may be applied to increase the diversity and robustness of Experimental Setup: The experimental setup encompasses the
the training data. Hyperparameter tuning is conducted to optimize configuration and execution of experiments to validate the
the model’s performance metrics, such as accuracy, precision, recall, performance of the DeepFake detection model. Parameters such as
and F1-score. The training process may be further accelerated using batch size, learning rate, and optimization algorithms are carefully
parallel computing techniques or distributed training frameworks to selected to balance computational efficiency and model
leverage the computational resources efficiently. convergence. The experiments are conducted on benchmark datasets
and real-world scenarios to assess the model’s generalization
capabilities and robustness to different types of manipulations.
Special considerations may be given to address ethical concerns and
ensure the responsible deployment of the detection system in
practice.

Fig 11. Experimental System of testing

IV. RESULTS:
The results section presents the outcomes of the experiments
conducted to evaluate the performance of the DeepFake detection
system. It provides insights into the effectiveness, accuracy, and Fig 14. Accuracy and Confusion Matrix
robustness of the proposed approach in identifying manipulated
videos. Comparison with Existing Methods: For assess a superiority and
improvements of the suggested DeepFake detection system, its
Performance Evaluation: The performance evaluation entails performance is contrasted with current state-of-the-art techniques.
quantitative analysis of the DeepFake detection model’s Traditional image processing methods, benchmarking against
performance metrics, including accuracy, precision, recall, and F1- baseline models, and other deep learning-based strategies can all be
score. These metrics are calculated on both the training and used in comparative analysis. Important performance metrics
validation datasets to assess the model’s ability to generalize to including scalability, computing efficiency, and detection accuracy
unseen data. Additionally, statistical tests may be performed to are taken into account to show the advantages and disadvantages of
determine the significance of the observed differences and validate the suggested approach.
the reliability of the results.
Visualizations: Visualizations such as confusion matrices, ROC
curves, precision-recall curves, and heatmaps are employed to
provide intuitive insights into the model's behavior and performance
characteristics. These visual aids help in understanding the model's
ability to differentiate between real and fake videos, identify
misclassifications, and assess its sensitivity to different thresholds.
Furthermore, visualizations aid in communicating the experimental
findings effectively to a broader audience, facilitating interpretation
and decision-making.

Fig 12. ROC Curve

Fig 15. Actual output of project

V. DISCUSSION:
The discussion section delves into a interpretation of the results
obtained from the experimentation phase, highlighting the
significance of the findings and their implications for the field of
Fig 13. P-Recall Curve DeepFake detection.

Interpretation of Results: To interpret the outcomes, examine the

performance measures and contrast them with the methodology's
stated goals. Finding patterns, trends, and correlations in the data is
one way to comprehend the fundamental elements affecting the
effectiveness of the DeepFake detection model. The understanding
acquired from the interpretation serves as a foundation for deriving
significant findings and validates the efficacy of the suggested
strategy.
Limitations: Addressing the limitations of the study is crucial for The IEEE ICASSP stands for IEEE International
acknowledging potential constraints and areas for improvement. Conference on Acoustics, Speech, and Signal Processing.
Limitations may stem from dataset biases, model assumptions, [4] Rossler, A., Thies, J., Nießner, M., Riess, C., Verdoliva,
computational resources, or methodological constraints. Discussing L., and Cozzolino, D. (2019). "FaceForensics++: Learning
these limitations transparently helps in contextualizing the results to Detect Manipulated Facial Images." The IEEE/CVF
and informing future research directions. Additionally, it fosters a Conference on Pattern Recognition and Computer Vision
critical evaluation of the proposed DeepFake detection system's (CVPR).
applicability in real-world scenarios. [5] Dang, T., Yamagishi, J., and Truong, T.A. (2020).
"Improved Detection of Deepfake Videos Using Temporal
Future Directions: Proposing future research directions is essential Dependencies." IEEE Information Forensics and Security
for advancing the field of DeepFake detection and addressing International Workshop (WIF
emerging challenges. This involves suggesting innovative [6] In 2019, Agarwal, A., Li, Y., Gavas, E., and Lyu, S.
approaches, methodologies, or techniques to overcome existing "Protecting World Leaders Against Deep Fakes." Preprint
limitations & enhance the performance of DeepFake detection arXiv:1910.08759, arXiv..
systems. Prospective research avenues might involve investigating [7] Qian, Y., Hua, G., Chen, Y., Dong, X., and Sun, J. (2020).
innovative methods for data augmentation., integrating multimodal "Thinking Inside the Box: Using Appearance Models and
information sources, enhancing model interpretability, or Context Based Encoding Method for Deepfake
developing robustness against adversarial attacks. By outlining Detection." IEEE Information Forensics and Security
potential avenues for further investigation, the discussion International Workshop (WIFS).
contributes to the continuous evolution of DeepFake detection [8] P. Korshunov and S. Marcel (2018). "A New Danger to
technology. Facial Recognition with DeepFakes? Evaluation and
Identification." Biometrics Theory, Applications, and
VI. CONCLUSION: Systems Conference, IEEE (BTAS).
In conclusion, this paper presents a comprehensive exploration of [9] Yamagishi, J., Echizen, I., Nozick, V., and Afchar, D.
DeepFake detection, aiming to address the pressing challenges (2018). "Mesonet: a Compact Facial Video Forgery
posed by the proliferation of manipulated media. Through the Detection Network." The IEEE ICASSP stands for IEEE
implementation of a novel DeepFake detection system, leveraging International Conference on Acoustics, Speech, and
advanced deep learning techniques, significant strides have been Signal Processing.
made in accurately identifying manipulated videos. [10] In 2020, Chen X., Chen C., Zhou Y., Guan Z., and Han J.
published a paper. "Detecting DeepFake Videos from the
The paper has provided an in-depth analysis of the DeepFake Clue of Heartbeat." 29, 7910-7921, IEEE Transactions on
detection problem, highlighting the importance of robust detection Image Processing.
systems in combating misinformation and preserving integrity of [11] Thies, J., Nießner, M. (2016), Theobalt, C., Stamminger,
digital media. By reviewing existing literature, proposing a novel M., and Zollhofer, M. "Face2Face: Real-Time Face
methodology, and conducting rigorous experimentation, the paper Capture and Reenactment of RGB Videos." IEEE
has demonstrated the efficacy of proposed approach in achieving Conference on Pattern Recognition and Computer Vision
superior performance compared to state-of-the-art methods. The (CVPR).
evaluation metrics and visualizations presented in the results section [12] Marra, F., Verdoliva, L., and Gragnaniello, D. (2019).
underscore the effectiveness and reliability of the DeepFake "Detection of GAN-Generated Fake Images Over Social
detection model. Networks." IEEE Information Forensics and Security
International Workshop (WIFS).
Beyond the scope of the paper, the full project represents a [13] Yamagishi, J., and Dang, T. (2019). "Deep Learning for
culmination of efforts to develop an end-to-end DeepFake detection Deepfakes Creation and Detection." IEEE Information
solution. Every stage of the research, from gathering data and Forensics and Security International Workshop (WIFS).
preparing it to training and assessing the model, has advanced the [14] In 2016, Bayar and Stamm published a paper. "A Deep
state-of-the-art in DeepFake detection technology. The project's Learning Approach to Universal Image Manipulation
contributions extend to the broader research community by Detection Using a New Convolutional Layer." IEEE
providing insights, methodologies, and benchmarks for future Information Forensics and Security Transactions, 12(4),
research endeavors in the field. 1160–1171.
[15] Yamagishi, J., and Dang, T. (2020). "A Patch-Based
In essence, a project underscores the importance of interdisciplinary
Method for Deepfake Detection in Videos." IEEE
collaboration, technological innovation, and ethical considerations
Information Forensics and Security International
in addressing the complex challenges posed by DeepFake media. By
Workshop (WIFS).
fostering a multidisciplinary approach and harnessing the power of
[16] Chang, S.F., Hsu, S., Liu, Y., Guera, D. (2020).
advanced technologies, we can strive towards a more trustworthy
"ShallowNet: Towards Effective Pixel-Wise Deepfakes
and resilient digital ecosystem.
Detection." Workshop on Information Forensics and
Security, IEEE International (WIFS).
VII. REFERENCES:
[17] Cornelius, C., Samek, W., Thies, J., Nießner, M., Rossler,
[1] Zhou (2017), Han (2017), Morariu V.I., Davis L.S. "Two-
A., and Rohrbach, M. (2019). "On Detecting Adversarial
Stream Neural Networks for Tampered Face Detection."
Perturbations." preprint arXiv:1906.04392; arXiv.
12(11), 2562–2575, IEEE Transactions on Information
[18] Theobalt, C., Valgaerts, L., Zollhofer, M., Nießner, M.,
Forensics and Security.
and Thies, J. (2015). "Real-Time Expression Transfer for
[2] Y. Li, M.C. Chang, and S. Lyu (2018). "Incorporating
Facial Reenactment." 34(6), 183, ACM Transactions on
Facial Regions into Deep Fake Image Detection." IEEE
Graphics (TOG).
Information Forensics and Security International
[19] Hussain, A., and H. Erdogan (2020). "A Deep Learning
Workshop (WIFS).
Approach for Deepfake Video Detection and Analysis."
[3] Yamagishi, J., and H.M. Nguyen (2020). "Use of
Access IEEE, 8, 54013–54025.
Temporal Information and Batch Normalization in
Convolutional Neural Networks for Deepfake Detection."