Ultimate Deepfake Detection Using Python: Master Deep Learning Techniques like CNNs, GANs, and Transformers to Detect Deepfakes in Images, Audio, and Videos Using Python (English Edition)

CHAPTER 1

Introduction to Generative AI and Deepfake Technology

Introduction

In this introductory chapter, the reader will be introduced to the concepts of generative AI models and their remarkable ability to create highly realistic data. We will also understand the meaning of deepfakes and their potential harms. This opening chapter lays the foundation for your exploration of the fascinating and complex world of generative AI and deepfake technology. We will delve into the core concepts, dive into their capabilities, and unveil the potential benefits and challenges they present.

Structure

In this chapter, we will discuss the following topics:

Generative AI

Core Principles

Working

Types

Applications

Deepfakes

History

Types

Impact

Ethical Considerations

Generative AI

Generative artificial intelligence (Gen AI) is a subset of deep learning (DL) that concerns itself with the generation of new data or content in the form of text, images, videos, 3D designs, and more. That is, while the conventional machine learning (ML) focuses on predicting values of data by learning patterns from existing data, generative AI enables you to create new data.

Figure 1.1: Relationship between various terms in AI domain

Artificial Intelligence

Artificial intelligence (AI), as defined by Mitchel (1997), is a field in computer science that aims at building machines having human-like intelligent abilities. As humans, we have the natural intelligence which gives us an ability to learn, reason, solve problems, make decisions and act according to the new information. To enable the machines to have these abilities, we need to build intelligence into them. Since this intelligence is not natural to the machines, it is known as Artificial Intelligence (AI).

A prominent and most successful technique for building artificial intelligence is machine learning (ML) and its subsets - deep learning (DL), natural language processing (NLP), and reinforcement learning (RL). Machine learning uses data to learn patterns in it and then uses these patterns for prediction. However, there are alternative approaches to build AI. These approaches include Symbolic systems (Expert systems), Evolutionary algorithms, Logic-based systems, and their hybrids.

Symbolic systems or expert systems encode the human knowledge and reasoning rules about an application domain into a software or hardware system’s knowledge base. Thus, the knowledge base of a symbolic system contains the facts about a domain and their relationships. These are useful if the domain knowledge and the rules are clear and well known.

Evolutionary algorithms are inspired by the process of biological evolution and natural selection (selection of the fittest). Biological evolution is characterized by a population of members of a species and their environmental challenges. The members who have the ability to adapt to the environmental challenges and survive them are the fittest ones. These members get to procreate and carry forward their genes into the next generation. Procreation allows the genes to mix, there may also be spontaneous mutations in the genes. Those who can’t adapt, perish. Based on this paradigm of natural selection, evolutionary algorithms begin with a population of all possible solutions (known as the candidate solution), a task and a performance metric. The candidate solutions who perform well on the task are iteratively selected to form the next generations of candidate solutions. These algorithms allow for candidate solutions to have mutations (random changes to portions of it) to enhance the performance.

Logic-based algorithms use the principles of formal logic for the knowledge representation and reasoning to reach a conclusion. It uses reasoning based on existing knowledge. However, it is often challenging to represent real-world problems, with changing scenarios and requirements, using formal logic. The problems which involve learning from experience also are not suitable to be solved using logic-based algorithms.

Machine learning builds intelligence into the machines by allowing them to learn from data without explicitly being programmed. Unlike all the methods we saw earlier, in ML we don’t need to have a rule base, or an algorithm to solve a problem. Instead, the rules and patterns in data are learned by an ML algorithm from the data itself. This makes ML applicable to almost any domain where sufficient data is available to learn from. ML is a data driven approach that can be used for the tasks of predictive analytics, data classification, and informed decision making.

Machine Learning

Machine learning (Mitchel, 1997) encompasses a variety of algorithms and techniques which are applicable to a diverse range of problems. ML techniques are broadly classified into following four categories:

Supervised Learning

Unsupervised Learning

Semi-supervised Learning

Reinforcement Learning

Figure 1.2: Types of Machine Learning

Supervised Learning

Data is at the heart of all types of ML. This data is often organized in the form of a table consisting of rows and columns of data. Supervised learning is learning from labeled data.

Labeled Data (Mitchel, 1997) comprises a collection of data points and their associated labels. For example, a set of images of various animals where each image is labeled with the name of the animal it contains (for example, cat or dog), or a set of text tweets where each tweet is associated with the label capturing the sentiment of the tweet (positive, negative, neutral). Table 1.1 shows a labeled tweets dataset:

Table 1.1: A Labeled Dataset for the Classification Task

The labels in the dataset act as the ground truth or the correct answer for each row of data (a datapoint) in the dataset. This implies they tell a supervised learning algorithm what output it should produce for that row. If the output predicted by a learning algorithm does not match the label given in the dataset, then it counts as an error made by the learning algorithm. The total error made by an algorithm is used to update the configuration parameters (hyperparameters) of the learning algorithm until its predicted output closely matches the actual label given in the dataset. This update is often carried out using an optimization algorithm like Gradient Descent or one of its variants.

Thus, supervised learning algorithms learn a pattern/model/mathematical relationship between data points and their labels and then use the learned pattern to predict the labels for yet unseen data points.

On the other hand, unlabeled data consists of data points without an associated label. For example, a set of satellite images from across the world, a set of uncategorized tweets. Unlabeled data is used by unsupervised learning algorithms that can learn from the inherent properties of the data and discover the relationships in it.

Supervised learning is applied to two wide categories of tasks: classification and regression.

Classification is the task of categorizing data points into two or more predefined classes or labels. For example, the task of categorizing emails as spam or no-spam, diagnosing MRI images as showing a cancerous tumor or a benign tumor, classifying apples as of good or bad quality, and more. The dataset shown in Table 1.1 is an example of a classification task.

Regression is the task of predicting a continuous valued target variable. For example, based on past performance of a stock, predicting its value in future, predicting salary based on age and experience of candidates. The dataset shown in Table 1.2 is an example of a task which can be solved using regression.

Table 1.2: A Labeled Dataset for the Regression Task

Unsupervised Learning

Unsupervised learning aims at learning the inherent structure of the data and using this learning to group similar data together or to simplify complex data by extracting the most important features of the data. Unsupervised learning does not require a labeled dataset, it uses self-learning to discover patterns in unlabeled data.

Suppose the ecommerce retailer ABC.com plans to launch a new product in the selected states of the USA. Before launching the product, the marketing head of ABC.com wants to know the customer profile of its buyers in the area. She requests the data science team to categorize the buyers by their purchase history, spending patterns, product category interest, loyalty, and more. This categorization helps the marketing team to show targeted recommendations for the new product to the customers for whom the probability of a purchase is higher for this product. The data science team must use unsupervised learning to learn similarities among customer profiles to cluster or segment them into groups.

Clustering, dimensionality reduction and mining association rules are three prominent tasks in unsupervised learning.

Clustering is the technique of discovering the inherent similarities among the uncategorized data points and then grouping similar data points together in a conceptual group defined by the similarity. This is useful for tasks such as customer segmentation, outlier detection, anomaly analysis, document clustering, and more.

Figure 1.3: Unsupervised Clustering of data based on similarity

Dimensionality reduction techniques are used in machine learning to reduce the number of features in input data. It is an important task due to the following advantages:

Helps overcome the curse of dimensionality. With more features, an ML model needs exponentially more quantities of data to train well, reducing the number of features reduces this need for more training data.

The reduced dimensionality also reduces the model complexity and hence the computational costs of the model.

Association rule mining is a task in data mining that discovers frequently occurring relationships or associations between data items in a dataset. This is achieved by identifying groups of items (itemsets) that appear together frequently in transactions involving the data, formulating rules to express the association among items and finally evaluating these rules based on certain metrics like support and confidence.

Semi-Supervised Learning

Semi-supervised learning involves the characteristics of both supervised learning and unsupervised learning. It uses some amount of labeled data to learn insights that are then used to guide the learning from a larger unlabeled dataset.

In semi-supervised learning, the labeled data is used to train a base-learner using supervised learning. The base learner can be any one of the supervised learning algorithms like decision tree, support vector machine, neural network, and more. The model learned by the base learner captures the patterns and relationships in the labeled data. The unlabeled data is used to improve the base learner by self-training. The base learner is used to predict the labels on unlabeled data and the high confidence predictions are added to the labeled dataset. Then the base learner is trained again on this new enhanced labeled dataset. Inherent relationships or similarities in unlabeled data guide the labeling of these datapoints.

Semi-supervised learning is an important learning method for tasks where the amount of labeled data is very limited and it is too expensive to obtain enough labeled data, but the unlabeled data is readily available.

Semi-supervised learning improves the performance of tasks like anomaly detection and reduces the human effort required for labelling data.

Reinforcement Learning

Reinforcement learning (RL) is a type of machine learning which involves learning from feedback. It is like how a pet dog learns to follow commands of its parents. The parent will reinforce the desired behavior of the pet with a reward in the form of a snack or petting and will penalize an undesired behavior with a rebuke or denial of a snack.

In the case of reinforcement learning, there is an agent (software, or a combination of software and hardware) that learns to achieve a goal within a specific environment. Achieving a goal often involves making a sequence of decisions. For each decision taken by the agent, the environment generates a feedback signal (reward).

For the agent, the goal is to maximize the cumulative reward earned from the environment. In the RL, the agent discovers the best or optimal sequence of decisions (those that help it optimize its long-term goal of maximizing the cumulative reward.)

Reinforcement learning models the problem to be solved in the form of a Markov decision process. The Markov decision process assumes that each state of the agent’s environment is dependent only on its current state, not on the past ones (Markov property). As the agent takes decisions and gets rewards (or penalties) for its decisions, it eventually learns the best sequence of actions in each of the states of its environment. This sequence of actions is known as a policy.

Thus, to learn the optimal policy (one that maximizes the cumulative reward), the agent has to try new decisions (explore the decision space) and once it has learned good decisions, it has to apply those decisions (exploitation) to earn rewards. The agent must again use a trade-off of exploration (trying new decisions) and exploitation (applying the learned decisions) to optimize its reward.

Reinforcement learning is very useful in the tasks where the data (both labeled and unlabeled) is not available, and the rules are very complex. These are the characteristics of problems like game playing, recommendation systems, robot training, financial markets, and predictions. And in the age of large language models like Gemini and ChatGPT, the reinforcement learning from human feedback is helping make these models more accurate and aligned with human values.

Figure 1.4: Important Elements of Reinforcement Learning Framework

Deep Learning

Deep learning is a subset of machine learning algorithms that use deep artificial neural networks (ANNs) for prediction. ANNs are inspired by how the human brain processes information and learns from input signals. Deep learning can be applied to supervised learning as well as unsupervised learning tasks.

Basic unit of processing in the human brain is called a neuron (a biological neuron). The neurons are connected to each other through connections known as the dendrites and the axons. This neuron receives inputs through its dendrites in the form of electrochemical signals from sensory organs or from other neurons in the brain. The neuron combines all the inputs it receives, if the combined signal exceeds a certain threshold signal level, the neuron transmits a signal to the next neuron through its axons.

An artificial neuron is a unit of computation modeled after the working of a biological neuron. It receives inputs which are numerical values from the user or from other neurons, it associates a weight (a numerical value) with each input. Every artificial neuron has an additional input called the bias (with a fixed value of -1) and a weight associated with it. This neuron computes the sum of weighted inputs and applies an activation function to this weighted sum. The activation function is akin to the threshold used by a biological neuron to decide whether to transmit the signal further or not. Activation function transforms the weighted sum into an output value and decides whether the output will be transmitted as the final output or to the next neuron.

An artificial neural network (ANN) (Goodfellow, Bengio, and Courville, 2016) is an interconnection of several artificial neurons. The neurons are arranged in the form of layers. The first layer is the input layer which consists of just the input values from data, the final layer is the output layer which consists of the final outputs from the ANN. Between the input layer and the output layer there are one or more intermediate layers of neurons known as the hidden layers.

Each layer of neurons transforms the inputs it receives from the previous layer. An ANN with more than one hidden layer of neurons is known as a deep neural network (DNN). Deep learning is the type of machine learning done by using deep neural networks. The ability of an ANN to learn complex relationships in input data increases with every additional layer of neurons in its architecture. This has allowed deep learning to be successfully applied for tasks like image classification, language translation, and more.

Figure 1.5: Deep Neural Network

Training Deep Neural Networks

DNNs require large amounts of data to learn from. Before a model is trained, this data needs to be cleaned and pre-processed. The data pre-processing is a critical step in building successful AI models.

As we saw, every neuron in a DNN has weights associated with it. During the initialization of the DNN these weights are assigned with random values and the learning involves learning the correct values of these weights such that the error in predicting the target values in the dataset is minimized. An optimization algorithm like Gradient Descent is used to calculate the amount of change to be applied to the weights based on the gradient of the error function.

An error function is a mathematical function that measures the error rate in the predictions made by the DNN. A learning algorithm for example, back propagation, is used to iteratively update all the weights in all the layers of a DNN starting from the output layer backwards to the first hidden layer.

Deep Neural Network Architectures

Deep neural networks come in multiple topologies based on the type of data and the problem they are required to solve. Deep neural networks can be used for supervised as well as unsupervised learning tasks. Table 1.3 shows a classification of DNN architectures for these two types of tasks:

Table 1.3: Deep Neural Network Architectures

Deep Learning Architectures for Generative AI

Machine learning models are classified as generative models and discriminative models. Discriminative models learn the statistical relationship between the features of input data and the corresponding target values. Given a dataset X and the class labels Y, these models learn the conditional probability p(Y|X), that is, what is the probability of the target labels being Y given that the dataset is X. These models learn to discriminate between data points based on their features and the target values. Common discriminative models for example, support vector machines, decision trees, convolutional neural networks, recurrent neural networks are used for classification and regression tasks.

Generative models on the other hand, learn the joint distribution over all the variables in data, that is, they learn the joint probability p(X,Y) in case of labeled data and p(X) in case of unlabeled data. They learn the probability of finding data that is given in the dataset. This allows these models to simulate the data generation processes in the real world. This modeling of the probability distribution of the data itself allows generative models to sample new points of similar probability distribution. These new points are the newly generated data points. It is this probability modeling of natural languages like English, Hindi, German, and more, that allows the large language models like Gemini and GPT-4 to predict the next word in a sequence of words and generate a plausible response for a user prompt.

The year 2022 ushered us into the era of generative AI (GenAI) models. Generative AI models specialize in the generation of new content in the form of text, images, video, and audio. The Gen AI models learn the joint probability distribution of the input data and the target labels. Then, to the model samples from this learned distribution to create new data points.

Large language models (LLMs) like Google’s Gemini and OpenAI’s GPT 3.5/4 are powerful text generation models that can generate human quality text. There are many open source LLMs like LLaMa2, Mistral, Gemma that work exceptionally well too. Text to image generation models like Stable Diffusion can generate photo-realistic images based on a text description input to the model. Video generation models like SoRA from OpenAI can generate almost realistic videos.

Important components in Gen AI models are:

A deep neural network architecture like generative adversarial network, transformer, and variational autoencoder.

Data for training in the form of images, text, audio, code, and more. The gen AI models