Deep Fake Today

Download as pdf or txt
Download as pdf or txt
You are on page 1of 7

COMBINING COMPUTER VISION TECHNIQUES AND INTRAFRAME NOISE

METHODS TO DETECT A DEEPFAKE

Ajinkya Kharade1, Pranav Gaikwad1, Shubham Arakh1, Analp Kalore1, Maya P Shelke1, Nihar Ranjan1

1. Department of Information Technology, JSPM’s Rajarshi Shahu College Of Engineering, Pune, Maharashtra, India.
Email: [email protected], [email protected], [email protected],
[email protected], [email protected], [email protected]

Abstract – Deep fakes are synthetic videos created by deep face of the targeted person in a video with the source footage
learning algorithms that can convincingly depict individuals while maintaining the same facial expressions. Lip Syncing
saying or doing things they never did. With the proliferation of isolates the lip region and syncs it with audio that looks to
deepfake videos in social media and the potential for them to pronounce something that the individual does not actually
cause significant harm, deep fake detection has become a
pressing issue. In this study, we offer a unique method for
say. [8]. In order to create Deep Fake the Deep neural
detecting deepfakes by combining computer vision methods networks are used. Various recent technological
with intraframe noise. The proposed approach involves advancements with deep learning techniques including auto
extracting features from the video frames, including texture, encoders and GAN [12][13](Generative Adversarial
color, and edges, and then adding a layer of intraframe noise to Networks) are used to create fake faces which are applied
the video frames. We tested the proposed method on a variety of mainly in the computer vision. Deep Fake detection method
benchmark datasets and found that it achieved high accuracy in using the Haar wavelet transforms. The method aims to
detecting deepfake videos. distinguish between original videos and Deep Fake videos by
exploiting the limitations and artifacts introduced during the
Keywords: Deep Fake, intraframe noise, extracting features,
high accuracy.
Deep Fake generation process.

The suggested method takes advantage of the fact that Deep


false algorithms have specific constraints, such as generating
false faces of a specific size and resolution. An affine
1. INTRODUCTION
transformation and blur function are applied to the
synthesised faces to match the source face to the target in the
Deep Fake is a technology that can superimpose the face of a
original video.
person with another person’s face, alter facial expressions or
gestures, appearing as in the original video using artificial
intelligence concepts. It manipulates the whole activity of the
targeted person which synthesizes audio and visual aspects.
Deepfakes are often created with the intention to deceive or
mislead viewers into believing that the manipulated content
is genuine. Some social media users posted a doctored video
to promote the myth that After meeting with President
Donald Trump, House Speaker Nancy Pelosi fumbled over
her speech [1]. The deep fake technology has created and
developed videos that cannot be identified by the normal
human being eyes [2]. It can detect the real and fake images
by using advanced network architecture.

To identify the forged videos and images created by using


deep learning methods can be achieved [3].
The fake images can be present in three forms [7].
Any AI-generated mimicking videos are now referred to as
"Deep Fake," which is a more general term. Basically, there
are three major types of Deep Fake videos. Making a movie Figure: 1[17]. Original vs Fake
of a person's complete head and upper shoulders is what head
puppetry entails. Face exchanging entails exchanging the
The detection method utilizes the Haar wavelet transform to R. Saravana Ram, M. Vinoth Kumar, Tareq M. Al-shami,
analyze the blur inconsistency and detect Deep Fake forgery. Mehedi Masud, Hanan Aljuaid and Mohamed Abouhawwash
Using the Haar wavelet transform function, it compares the in [8] suggested extracting features from the input deepfake
blurred synthesised ROI with the surrounding environment. image using fuzzy clustering.
By analyzing edge types and sharpness, the method
determines whether a face image has been blurred and to Kandasamy V, Hubálovsk, and Trojovsk [9] revealed the
what extent [10]. To combat the growing number of Deep learning approach with two levels for detecting
deepfakes on the internet, major tech companies are actively deepfake photos and videos. To extract features from face
investigating methods to detect deepfakes [6]. Also, Google images, the recommended SAE technique is employed.
has released a free dataset for the public as a contribution to
the deepfake detection. The Deepfake Detection Challenge Abdulqader M. Almars [10] gave a thorough explanation of
initiated by major tech companies and the release of a free the architecture, tools, and performance of the existing
dataset by Google is mentioned as examples of industry deepfake approaches. It also emphasised the publicly
efforts in addressing the deepfake issue [7], combines deep available datasets used by the scientific community and
learning techniques like recurrent neural networks (RNN), sorted them by source, technique, and dataset.
convolutional neural networks (CNN), and long short-term
memory to provide a thorough research for deepfake The proposed technique by Aarti Karandikar, Vedita
identification. [10] [11] [16]. Deshpande, Sanjana Singh, Sayali Nagbhidkar, and Saurabh
Agrawal [11] focuses on facial modification for forgery
detection and leverages transfer learning on the VGG-16
2. LITERATURE SURVEY model to train the dataset.

Saravana Balaji Balasu bramanian, Jagadeesh Kannan,


G. Lee and M. Kim. [2] proposed a technique for Using a Prabu, Venkata chalam and Pavel Trojovský [12] better
DNN to calculate the rate of change of computer vision accuracy.
features based on the difference between a given amount of
frames and frames. Suganthi ST. Ayoobkhan MUA, VKK, BacaninN
,KV,ŠtěpánH,PavelT [14] the fisherface Linear binary
The method is the first real-time facial reproduction system pattern histogram using the DBN classifier (FF-LBPH DBN)
that only needs monocular RGB input, according to J. Thies, technique was implemented as a detection technique for
M. Zollhofer, M. Stamminger, C. Theobalt, and M. Nießner deepfake images. The planned work was carried out much
[3]. quickly, and it was quite good at distinguishing between
phoney and real images.
According to S. Lyu's article[4] Predict that a number of
upcoming technology advancements will enhance the phoney Author has seen, [16] An very accurate bidirectional
videos' production effectiveness and visual quality. recurrent neural network model may identify bogus news.

Mohammed Akram Younus and Taha Mohammed Hasan. [6] 3. PROPOSED SYSTEM
DeepFakes, a novel method for spotting false faces created
artificially, have been proposed. Given that DeepFake can Deepfake detection is used in various real-life applications
only generate face photos in set sizes and low resolutions, such as social media platforms, news verification, law
which must then be blurred and modified to match the faces enforcement, entertainment industry, political campaigns,
that will be replaced in the original film, In the generated online identity verification, and online reputation
DeepFake movies, These additive blur and ROI changes management. It helps identify and flag manipulated content,
produce distinct artefacts that may be efficiently recorded by verify the authenticity of videos and images, analyze
spotting discrepancies. evidence in legal proceedings, enforce copyright and protect
intellectual property, combat political misinformation, verify
Dolhansky, Brian and Howes, Russ and Pflaum, Ben and online identities, and manage online reputation. Deepfake
Baram, Nicole and Ferrer, Cristian [7] introduced a preview detection is a dynamic field that requires continuous research
of the DFDC dataset that will be made available later this and updates to stay ahead of evolving deepfake techniques.
year with the goal of encouraging researchers to familiarise
themselves with the data, providing preliminary findings, and
comparing those findings to suggested baselines.
unsupervised learning, or semi-supervised learning
can be used, the availability of labeled data.

3. Detection Algorithm: The trained model is then


used as the core detection algorithm. It takes the
extracted features of an unknown sample as input
and calculates a confidence score or probability
indicating the likelihood of it being a deepfake. The
algorithm should be designed to detect both known
and unknown types of deepfakes.

4. Multimodal Analysis: To enhance the accuracy and


reliability of the detection system, multimodal
analysis can be employed. This involves analyzing
multiple modalities such as video, audio, and
metadata simultaneously. Inconsistencies or
discrepancies across different modalities can
indicate the presence of a deepfake.

5. Post-processing and Fusion: The outputs from


different detection algorithms or modalities can be
fused or combined to generate a final decision.
Fusion techniques can include weighted averaging,
Figure: 2[8]. Preprocessing System for Deepfake
ensemble methods, or decision-level fusion. This
step helps improve the overall accuracy and
reliability of the system.
deepfake detection can consist of several components and
techniques working together. A high-level description of a 6. Real-time Implementation: To detect deepfakes in
hypothetical system is given below: real-time scenarios, the system should be optimized
for efficiency and speed. Techniques like parallel
Preprocessing: The system starts by FIG.3 preprocessing the
computing, hardware acceleration (e.g., GPUs), or
input data, which could be a video, image, or audio file. This
model compression can be utilized to achieve real-
step may involve resizing, normalizing, or filtering the data
time performance.
to ensure consistency and compatibility with the subsequent
analysis steps. 7. Continuous Monitoring and Updates: Deepfake
techniques are evolving rapidly, so the system
1. Feature Extraction: In this stage, relevant features
should be regularly updated with new training data
are extracted from the input data. These features can
and models. Continuous monito to stay effective in
include visual cues, audio characteristics, and
detecting the latest threats.
metadata. Deep neural networks and other advanced
approaches or convolutional neural networks 8. User Interface and Reporting: The system can
(CNNs) can be employed to extract discriminative provide a user-friendly interface for users to interact
features. with and analyze the results. It can generate detailed
reports highlighting the detected deepfake
2. Training and Model Development: A machine
indicators, confidence scores, and any additional
learning model is trained on a large and diverse
information that aids in further investigation.
dataset that contains both real and deepfake
samples. The model learns to differentiate between 9. ring and research on emerging deepfake techniques
genuine and manipulated media by analyzing the will enable the system to stay effective in detecting
extracted features. Various techniques like the latest threats.
Depending on the situation, supervised learning,
10. User Interface and Reporting: The system can learning frameworks like TensorFlow or PyTorch,
provide a user-friendly interface for users to interact which provide GPU support. Ensure that the
with and analyze the results. It can generate detailed hardware infrastructure is properly set up and
reports highlighting the detected deepfake configured for efficient parallel processing.
indicators, confidence scores, and any additional
5. Model Optimization: Apply model optimization
information that aids in further investigation.
methods for reducing computational load and
memory requirements. This may include model
quantization to reduce the precision of model
weights and activations, or model compression
4. IMPLEMENTATION DETAILS
techniques to reduce the number of parameters.
Implementing deepfake detection using computer vision with Utilize optimized libraries or frameworks like
improved quality and speed involves several key steps and TensorFlow Lite or TensorRT to improve the speed
considerations. Consider the following implementation of execution.
details: 6. Real-time Implementation: Optimize the system for
Dataset Collection and Preparation: Gather a diverse and real-time performance by minimizing latency. Use
high-quality dataset consisting of both real and deepfake efficient data structures and algorithms to optimize
samples. Ensure that the dataset covers various deepfake the processing of intermediate results. Employ
generation techniques, resolutions, lighting conditions, and techniques like batch processing, multi-threading, or
camera angles. Preprocess the data by resizing, normalizing, asynchronous processing to parallelize computations
and augmenting the images or videos as needed. and reduce latency.

1. Model Selection and Training: Choose a suitable 7. Continuous Improvement: Stay updated with the
deep learning architecture for deepfake detection, latest research advancements in deepfake generation
such as a CNN or a combination of CNN and RNNs and detection. Continuously evaluate and refine the
for temporal analysis. Consider using pre-trained deepfake detection system by incorporating new
models like ResNet, Inception, or EfficientNet and techniques, datasets, and model updates. Regularly
fine-tune them on the deepfake detection task. monitor performance metrics and incorporate
Employ transfer learning to leverage features feedback to enhance the quality and speed of
learned from large-scale datasets like ImageNet. detection.

2. Feature Extraction: Extract meaningful features 8. Evaluation and Testing: Evaluate the performance
from the input data using computer vision of the deepfake detection system using appropriate
techniques. This may involve face detection, facial parameters including F1 score, recall, accuracy, and
landmark detection, texture analysis, or motion precision. Conduct extensive testing on diverse
analysis. Select features that capture relevant datasets, including unseen deepfake variations, to
information for differentiating between real and assess the robustness and generalization capability
deepfake content. of the system.

3. Ensemble Methods: Train multiple deepfake 9. Deployment and Integration: Integrate the deepfake
detection models with different architectures or detection system into the desired application or
trained on different subsets of the dataset. Combine platform. Ensure that the system is scalable, reliable,
their predictions through ensemble methods like and user-friendly. Consider factors like system
majority voting or averaging to improve detection requirements, user interfaces, and deployment
accuracy. Experiment with different ensemble options (cloud-based, on-premise, or edge
techniques to find the optimal combination. computing) based on the specific use case and
deployment environment.
4. Hardware Acceleration: Utilize GPUs or other
hardware accelerators to accelerate the execution of It's important to note that the implementation details can vary
deep learning models. This can be done using deep depending on the specific deepfake detection approach,
available resources, and the chosen technologies. It is important to note that the specific implementation and
Experimentation, fine-tuning, and adapting the mathematical expressions may vary depending on the chosen
implementation to suit the specific requirements and algorithms and techniques used for feature extraction and
classification. The above representation provides a general
constraints of the system are necessary to achieve optimal
framework for understanding the process of deepfake
results. detection using computer vision and algorithms.
A mathematical expression representing the process of
deepfake detection using computer vision and algorithms:
5. RESULT
Let:
𝑋 be the input data (preprocessed images/videos).
𝑌 be the ground truth labels (0 for real content, 1 for The result of the deepfake detection system can be a binary
deepfake content). classification indicating whether the input media (image or
video) is classified as a deepfake or genuine. This result is
𝑓(𝑥) be the feature extraction function that maps the input based on the analysis and prediction made by the trained
data to a set of features. deep learning model or ensemble of models.

The system may provide a confidence score or probability


𝜃 be the parameters of the detection model. along with the classification result, indicating the level of
certainty in the prediction. A higher confidence score
The deepfake detection model can be represented as a suggests a higher likelihood of the input being classified
function ℎ(𝑋; 𝜃) that takes the preprocessed data X and the correctly.
model parameters 𝜃 as inputs and outputs the probability of
Additionally, the system may generate a detailed report
the input being a deepfake:
highlighting the detected deepfake indicators or features that
contributed to the classification. This information can be
ℎ(𝑋; 𝜃) = 𝑃(𝑌 = 1|𝑋; 𝜃) useful for further investigation or analysis.

The overall result of the system can be evaluated using


To train the model, we use a labeled dataset accuracy, precision, recall, and F1 score are all performance
𝐷 = { (𝑋1, 𝑌1), (𝑋2, 𝑌2) … … . . (𝑋𝑛, 𝑌𝑛) } indicators. These metrics assess the efficiency and reliability
of the deepfake detection system in correctly identifying
deepfakes while minimizing false positives or false
negatives.
The purpose is to discover the best parameters 𝜃 * for
minimising classification error. This is accomplished by It is important to note that the accuracy and performance of
minimising the loss function. 𝐿(𝜃)with respect to 𝜃: the system can vary depending on the training dataset's
quality and variety, the chosen detection techniques, and the
implementation details. Regular monitoring, evaluation, and
𝜃 ∗ = 𝑎𝑟𝑔𝑚𝑖𝑛𝜃 𝐿(𝜃)
continuous improvement are crucial to ensure the system's
effectiveness in detecting evolving deepfake techniques.
The choice of the loss function depends on the specific
approach and can vary. Commonly used loss functions for
binary classification include cross-entropy loss or binary
6. CONCLUSION
logistic loss.
Choosing a suitable deep learning architecture, such as CNN
Once the model is trained, it can be used to classify new,
or a hybrid of CNN and RNN, is important for deepfake
unseen data by calculating the probability of being a
detection. Leveraging pre-trained models like ResNet,
deepfake using the learned parameters:
Inception, or EfficientNet through transfer learning can
𝑃(𝑌 = 1|𝑋; 𝜃 ∗) provide a good starting point. Feature extraction techniques,
The threshold for classifying an input as a deepfake can be including face detection, facial landmark detection, texture
adjusted based on the desired balance between false positives analysis, and motion analysis, help capture relevant
and false negatives. information for differentiation.
Ensemble methods, such as combining predictions from [2] Lee, G. and Kim, M., 2021. Deepfake detection using the
multiple models through majority voting or averaging, can rate of change between frames based on computer vision.
enhance detection accuracy. Leveraging GPUs or other Sensors, 21(21), p.7367.
hardware accelerators, along with model optimization [3] Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C.
techniques like quantization and compression, improves the and Nießner, M., 2016. Face2face: Real-time face capture
speed of execution. Real-time implementation can be and reenactment of rgb videos. In Proceedings of the IEEE
achieved by minimizing latency and utilizing efficient conference on computer vision and pattern recognition (pp.
algorithms. 2387-2395).
[4] Lyu, S., 2020, July. Deepfake detection: Current
Continuously updating the system with the latest research challenges and next steps. In 2020 IEEE international
advancements, evaluating performance metrics, and conference on multimedia & expo workshops (ICMEW) (pp.
incorporating feedback are crucial for continuous 1-6). IEEE.
improvement. Evaluating the system using appropriate [5] Younus, M.A. and Hasan, T.M., 2020, April. DeepFake
metrics and extensive testing on diverse datasets ensures Detection Method Based on Haar Wavelet Transform
robustness and generalization capability. Effective and Fast. In 2020 International Conference on
Computer Science and Software Engineering (CSASE) (pp.
Integrating the deepfake detection system into the desired 186-190). IEEE.
application or platform requires scalability, reliability, and [6] Khalil, H.A. and Maged, S.A., 2021, May. Deepfakes
user-friendliness. Consider system requirements, user creation and detection using deep learning. In 2021
interfaces, and deployment options based on the specific use International Mobile, Intelligent, and Ubiquitous Computing
case and environment. Conference (MIUCC) (pp. 1-4). IEEE.
[7] Dolhansky, B., Howes, R., Pflaum, B., Baram, N. and
Remember that the implementation details provided here Ferrer, C.C., 2019. The deepfake detection challenge (dfdc)
serve as a general guide. Adjustments and experimentation preview dataset. arXiv preprint arXiv:1910.08854.
may be necessary to optimize the deepfake detection system [8] Saravana Ram, R., Vinoth Kumar, M., Al-shami, T.M.,
based on your specific needs and available resources. Stay Masud, M., Aljuaid, H. and Abouhawwash, M., 2023. Deep
informed about the latest developments in deepfake Fake Detection Using Computer Vision-Based Deep Neural
generation and detection to ensure your system remains Network with Pairwise Learning. Intelligent Automation &
effective against evolving deepfake techniques. Soft Computing, 35(2).
[9] Kandasamy, V., Hubálovský, Š. and Trojovský, P., 2022.
Deep fake detection using a sparse auto encoder with a graph
7. FUTURE SCOPE capsule dual graph CNN. PeerJ Computer Science, 8, p.e953.
[10] Almars, A.M., 2021. Deepfakes detection techniques
The future scope for deepfake detection involves the using deep learning: a survey. Journal of Computer and
development of advanced algorithms and techniques to Communications, 9(5), pp.20-35.
detect increasingly sophisticated deepfake videos, images, [11] Karandikar, A., Deshpande, V., Singh, S., Nagbhidkar,
and audio. This includes improving detection algorithms S. and Agrawal, S., 2020. Deepfake video detection using
through machine learning and AI, incorporating multimodal convolutional neural network. International Journal of
analysis across different sources, creating benchmark Advanced Trends in Computer Science and Engineering,
datasets for evaluation, focusing on explainable AI to 9(2), pp.1311-1315.
provide transparency, promoting collaboration and standards, [12] Balasubramanian, S.B., Prabu, P., Venkatachalam, K.
working on real-time detection systems, countering anti- and Trojovský, P., 2022. Deep fake detection using cascaded
forensic techniques, raising awareness through education, deep sparse auto-encoder for effective feature selection.
and staying ahead of deepfake technology advancements. PeerJ Computer Science, 8, p.e1040.
Although detecting deepfakes completely may become more [13] Shad, H.S., Rizvee, M., Roza, N.T., Hoq, S.M.,
challenging, ongoing research and technological Monirujjaman Khan, M., Singh, A., Zaguia, A. and
advancements will help mitigate the risks associated with Bourouis, S., 2021. Comparative analysis of deepfake image
deepfake technology. detection method using convolutional neural network.
Computational Intelligence and Neuroscience, 2021.
[14] Suganthi, S.T., Ayoobkhan, M.U.A., Bacanin, N.,
8. REFERENCES Venkatachalam, K., Štěpán, H. and Pavel, T., 2022. Deep
learning model for deep fake face recognition and detection.
[1] https://edition.cnn.com/2019/05/23/politics/doctored- PeerJ Computer Science, 8, p.e881.
video-pelosi/index.html. [15] Rafique, R., Gantassi, R., Amin, R., Frnda, J.,
Mustapha, A. and Alshehri, A.H., 2023. Deep fake detection
and classification using error-level analysis and deep
learning. Scientific Reports, 13(1), p.7422.
[16] Jiang, T., Li, J.P., Haq, A.U. and Saboor, A., 2020,
December. Fake news detection using deep recurrent neural
networks. In 2020 17th International Computer Conference
on Wavelet Active Media Technology and Information
Processing (ICCWAMTIP) (pp. 205-208). IEEE.
[17]https://www.google.com/url?sa=i&url=https%3A%2F%
2Fwww.trymaverick.com%2Fblog-posts%2Fare-deep-fakes-
all-evil-when-can-they-be-used-for-
good&psig=AOvVaw2iw8xCwMUh9D92KIFneNPs&ust=1
685099255505000&source=images&cd=vfe&ved=0CBQQ3
YkBahcKEwiY5pSJqpD_AhUAAAAAHQAAAAAQBA

You might also like