0% found this document useful (0 votes)
8 views19 pages

Deepfake Image Detection

This document details a study on utilizing Sensor Pattern Noise (SPN) for camera identification through a machine learning model based on DenseNet121 architecture. The model achieved an overall accuracy of 86.67% and highlighted challenges such as dataset imbalance and computational complexity. Future work will focus on expanding the dataset, exploring advanced architectures, and enabling real-time deployment for practical applications in digital forensics.

Uploaded by

msaudk21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
8 views19 pages

Deepfake Image Detection

This document details a study on utilizing Sensor Pattern Noise (SPN) for camera identification through a machine learning model based on DenseNet121 architecture. The model achieved an overall accuracy of 86.67% and highlighted challenges such as dataset imbalance and computational complexity. Future work will focus on expanding the dataset, exploring advanced architectures, and enabling real-time deployment for practical applications in digital forensics.

Uploaded by

msaudk21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 19

Introduction

Background Objectives
➢Sensor Pattern Noise (SPN) is an intrinsic ➢The primary goal of this work is to develop a
artifact resulting from imperfections in machine learning model that utilizes SPN for
camera sensors. It is unique to each camera, camera identification. By harnessing SPN's
effectively functioning as a digital fingerprint.
unique properties, the study aims to showcase
This characteristic makes SPN invaluable for
tracing the origin of digital images, its practical applications in digital forensics
particularly in fields such as forensics and for source tracing and authenticity
ensuring media integrity. verification to detect potential tampering in
multimedia content.
Literature Review

SPN as a Digital Fingerprint Deep Learning Challenges


➢Sensor Pattern Noise ➢Advanced deep learning ➢ Dataset Imbalance:
(SPN) has been architectures, including Uneven representation of
extensively validated as a Dense Net, ResNet, and SPN patterns across
reliable method for Efficient Net, have camera models
camera identification. Its demonstrated exceptional complicates training.
unique, device-specific performance in
properties make it a classification tasks. These
➢ SPN Extraction
cornerstone in digital Complexity: Accurate
models excel in feature
forensics and source extraction of SPN from
extraction, making them
tracing. images requires
well-suited for identifying
sophisticated
SPN patterns.
preprocessing, which adds
to the computational
overhead.
Dataset
❑ Data Sources
The dataset used in this study combines publicly available Kaggle datasets and a
custom dataset created from images captured by five different smartphones,
ensuring a diverse range of SPN patterns.

❑ Preprocessing
➢ Resizing: All images were resized to 128x128 pixels to maintain uniformity and
optimize computational efficiency.
➢ SPN Extraction: Sensor Pattern Noise (SPN) was extracted using high-pass
filtering, isolating the unique noise patterns from image content.
➢ Format Conversion: Images were standardized into RGB format to align with
the input requirements of the deep learning model.

❑ Dataset Splits
The dataset was divided into the following proportions for model training and
evaluation:
➢70% Training: Used for model learning.
➢15% Validation: For tuning and preventing overfitting.
➢15% Testing: For assessing model performance.
Methodology - SPN Extraction
The Sensor Pattern Noise (SPN) extraction process involves the following steps:
❖ Convert to Grayscale
Images are converted to grayscale to eliminate color information and focus on
the luminance channel, which contains the SPN.
❖ Apply Gaussian Blur
A Gaussian blur is applied to smooth the image and suppress high-frequency
content unrelated to SPN.
❖ Perform High-Pass Filtering
High-pass filtering is used to isolate the SPN by removing low-frequency
components, such as general image content and lighting variations.
❖ Expand to RGB Format
The extracted SPN is expanded back into an RGB format, ensuring
compatibility with the input requirements of the DenseNet121 deep learning
model.
Model Architecture

Base Model Custom Layers


➢The foundation of the To adapt DenseNet121 for SPN classification,
model is DenseNet121, a custom layers were added:
deep convolutional neural 1. Global Average Pooling: Reduces spatial
network pretrained on the dimensions, focusing on the most relevant
ImageNet dataset. Its features while minimizing overfitting.
dense connections 2. Dense Layer: A fully connected layer with
enhance feature 512 neurons and ReLU activation for
propagation and reuse, capturing complex SPN patterns.
making it ideal for 3. Dropout (40%): Introduced for
classification tasks. regularization to reduce the risk of
overfitting.
4. SoftMax Output Layer: Produces
probabilities across 5 classes
corresponding to the camera sources.
Training Process
Data Augmentation Hyperparameters Early Stopping
➢To enhance model ➢Optimizer: Adam ➢ An early stopping
robustness and optimizer with a learning mechanism was
prevent overfitting, rate of 0.0001 for efficient implemented to
data augmentation gradient updates. monitor validation
techniques such as ➢Loss Function: Sparse performance and
rotation, zoom, Categorical Cross entropy, halt training if no
shear, and flipping tailored for multi-class improvement was
were applied. This classification tasks. observed, effectively
diversifies the training ➢Batch Size: 32 samples preventing
dataset and helps the per batch for balanced overfitting.
model generalize memory usage and
better. performance.
➢Epochs: 55 iterations to
ensure sufficient learning.
CODE SNIPPETS
# Enhanced SPN Feature Extraction Function with High-Pass Filter
def extract_spn_highpass(img):
# Extracting Sensor Pattern Noise (SPN) from the image using a high-pass filter.
gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Convert to grayscale
# Apply Gaussian Blur to create the low-pass filtered image
low_pass = cv2.GaussianBlur(gray_img, (21, 21), 0)
# High-pass filter by subtracting the low-pass image from the original
high_pass = gray_img - low_pass
# High-frequency components (edges and noise)
return high_pass
# Converting SPN from grayscale to RGB (3 channels)
def convert_to_rgb(spn):
#Converting grayscale SPN to RGB by repeating the single channel.
spn_rgb = np.repeat(spn[..., np.newaxis], 3, axis=-1) # Convert (128, 128) -> (128, 128, 3)
return spn_rgb
# Load and Process Dataset Using SPN (with High-Pass Filtering)
base_dir = r"C:\Users\Rishi Singh\P" #The actual dataset directory
phones = ['Phone 1', 'Phone 2', 'Phone 3', 'Phone 4', 'Phone 5’]
# Folder names for each phone
data = []
labels = []
# Iterate through each phone's folder
for label, phone in enumerate(phones):
phone_path = os.path.join(base_dir, phone)
print(f"Loading data from {phone}...") # Print loading status
for img_name in os.listdir(phone_path):
img_path = os.path.join(phone_path, img_name)
img = cv2.imread(img_path)
if img is not None:
img = cv2.resize(img, (128, 128)) # Ensure image is resized to 128x128
spn = extract_spn_highpass(img) # Use high-pass filtering to extract SPN
spn_rgb = convert_to_rgb(spn) # Convert SPN to RGB format
data.append(spn_rgb)
labels.append(label)
print(f"Data from {phone} loaded successfully.")
# Split Dataset with 70-15-15 split
X_train, X_temp, y_train, y_temp = train_test_split(data, labels, test_size=0.3, random_state=42)
X_val, X_test, y_val, y_test = train_test_split(X_temp, y_temp, test_size=0.5, random_state=42)

# Data Augmentation (to avoid overfitting)


datagen = ImageDataGenerator(
rotation_range=20,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest’
)
# Add Custom Layers on Top
model = Sequential([
base_model,
Flatten(),
Dense(512, activation='relu', kernel_regularizer=regularizers.l2(0.01)), # L2 regularization
Dropout(0.3), # Adjusted Dropout Rate (0.3 for less regularization)
Dense(len(phones), activation='softmax’)
])
# Compile the Model
model.compile(optimizer=Adam(learning_rate=0.0001), loss='sparse_categorical_crossentropy’,
metrics=['accuracy’])
# Early Stopping Callback
early_stopping = EarlyStopping(
monitor='val_loss', # Monitors validation loss
patience=5, # Stops training if no improvement
restore_best_weights=True # Restores weights from the best epoch
)
# Train the Model with Batch Size 32 and Epochs 50
history = model.fit(
datagen.flow(X_train, y_train, batch_size=32), # Using data augmentation here
validation_data=(X_val, y_val),
epochs=50, # Number of epochs (more epochs for better fine-tuning)
callbacks=[early_stopping] # Early stopping
)

# Evaluate the Model


test_loss, test_accuracy = model.evaluate(X_test, y_test)
print(f"Test Accuracy: {test_accuracy * 100:.2f}%")
# Predict the Source Camera
def predict_source(image_path):
# Predict the source camera for a given image.
img = cv2.imread(image_path)
img = cv2.resize(img, (128, 128)) # Resize to 128x128
spn = extract_spn_highpass(img) # Extract SPN using high-pass filter
spn_rgb = convert_to_rgb(spn) # Convert SPN to RGB
spn_rgb = spn_rgb / 255.0 # Normalize
spn_rgb = np.expand_dims(spn_rgb, axis=0) # Add batch dimension
pred = model.predict(spn_rgb)
phone_index = np.argmax(pred)
return phones[phone_index]
# Test with a new image
test_image = r"C:\Users\Rishi Singh\P\Phone 2\IMG20201213150205.jpg" # Test image path
print("Predicted Source Camera:", predict_source(test_image))
1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 77ms/step
Predicted Source Camera: Phone 2
Results and Analysis
Overall Accuracy Individual Camera
Confusion Matrix
Accuracies
➢The proposed ❑ Camera 1: 80% ➢ A confusion matrix was
model achieved an ❑ Camera 2: 69.23% utilized to analyze
86.67% overall
❑ Camera 3: 100% classification
accuracy,
showcasing its ❑ Camera 4: 85.71% performance in detail,
effectiveness in ❑ Camera 5: 100% illustrating areas of
SPN-based camera These results highlight the strength and
identification. model's strong performance, identifying
with notable precision for misclassifications
Camera 3 and perfect between cameras. This
classification for Camera 5. analysis provides
valuable insights for
refining the model and
addressing
weaknesses.
precision recall f1-score support

Phone 1 0.94 0.80 0.86 20


Phone 2 0.82 0.69 0.75 13
Phone 3 0.71 1.00 0.83 10
Phone 4 0.92 0.86 0.89 14
Phone 5 0.90 1.00 0.95 18

accuracy 0.87 75
macro avg 0.86 0.87 0.86 75
weighted avg 0.88 0.87 0.87 75

Accuracy of Phone 1:
Precision: 0.94
Recall: 0.80
F1 Score: 0.86

Accuracy of Phone 2:
Precision: 0.82
Recall: 0.69
F1 Score: 0.75
Accuracy of Phone 3:
Precision: 0.71
Recall: 1.00
F1 Score: 0.83

Accuracy of Phone 4:
Precision: 0.92
Recall: 0.86
F1 Score: 0.89

Accuracy of Phone 5:
Precision: 0.90
Recall: 1.00
F1 Score: 0.95

Accuracy of Phone 1: 80.00%


Accuracy of Phone 2: 69.23%
Accuracy of Phone 3: 100.00%
Accuracy of Phone 4: 85.71%
Accuracy of Phone 5: 100.00%
Challenges
➢ Dataset Imbalance
The dataset contained an imbalance, with certain classes (cameras) being
more represented than others. This caused a bias toward the dominant
classes, affecting the model’s ability to generalize well across all camera
types.
➢ Computational Complexity
SPN extraction involves high-pass filtering and preprocessing steps that
are computationally intensive. Additionally, fine-tuning the DenseNet121
model with custom layers required significant processing power and time.
➢ Model Tuning
Striking the right balance between transfer learning and overfitting was
challenging. While pretrained weights helped boost performance, there
was a risk of overfitting to the training data, necessitating careful model
tuning and regularization strategies to achieve optimal results.
Future Work

➢ Expand Dataset
Future efforts will focus on expanding the dataset by incorporating more
camera models to address dataset imbalance and enhance the model's
generalization across diverse devices.
➢ Explore Advanced Architectures
Exploration of advanced architectures, such as Efficient Net or Vision
Transformers, could improve accuracy and efficiency by leveraging state-
of-the-art techniques in image classification.
➢ Real-time Deployment
The goal is to develop a real-time deployment of the model, enabling its
use in practical scenarios such as digital forensics and social media
platforms, where rapid camera source identification and authenticity
checks are critical.
Conclusion
➢ SPN and DenseNet121
The combination of Sensor Pattern Noise (SPN) and the DenseNet121
architecture has proven to be an effective approach for camera identification,
achieving a high level of accuracy in distinguishing devices based on unique
sensor noise patterns.
➢ Practical Applications
This method holds significant promise for digital forensics and ensuring
media authenticity, providing a reliable way to trace the origins of images and
detect potential manipulation in various contexts.
➢ Future Improvements
While the current model demonstrates strong performance, further
improvements can focus on enhancing generalization to diverse camera
types and increasing efficiency for real-time deployment in practical, large-
scale applications.
References
➢ Lukás, J., Fridrich, J., & Goljan, M. (2006). "Digital camera identification
from sensor pattern noise." IEEE Transactions on Information Forensics
and Security, 1(2), 205-214.
➢ Zhang, Y., & Yu, H. (2019). "Camera model identification using
convolutional neural networks." Journal of Visual Communication and
Image Representation, 62, 120-126.
➢ Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017). "Densely
connected convolutional networks." Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition (CVPR).
➢ Kaggle. (n.d.). "Camera model identification dataset." Retrieved from
[https://www.kaggle.com/].
➢ Goodfellow, I., Bengio, Y., & Courville, A. (2016). "Deep Learning." MIT Press.
➢ Conference on Computer Vision and Pattern Recognition (CVPR).

You might also like