DEEP LEARNING III AIML SEM-II 1

Unit-I: Fundamentals of Deep Learning and Machine Learning

Fundamentals of Deep Learning: Artificial Intelligence, History of Machine learning:

Probabilistic Modeling, Early Neural Networks, Kernel Methods, Decision Trees, Random
forests and Gradient Boosting Machines, Fundamentals of Machine Learning: Four Branches
of Machine Learning, Evaluating Machine learning Models, Overfitting and Underfitting.

1.What is Artificial Intelligence (AI)?

Artificial Intelligence (AI) is the field of computer science that focuses on creating
machines or systems that can perform tasks that usually require human intelligence. These tasks
include things like recognizing speech, understanding language, solving problems, making
decisions, and recognizing objects or images. In simple terms, AI is when machines are able to
think, learn, and act like humans.
AI systems work by processing large amounts of data and finding patterns to make decisions
or predictions. Some AI systems can also improve their performance over time by learning
from experience, which is known as machine learning.
Today Real-Time Examples of AI
1. Self-Driving Cars (Autonomous Vehicles)
o Example: Companies like Tesla and Waymo are developing self-driving cars
that use AI to navigate streets, detect obstacles, and make decisions about
stopping, turning, or speeding up. These cars use sensors and cameras to
understand their environment, and AI helps them make safe driving decisions.
2. Voice Assistants (Smart Speakers and Phones)
o Example: Amazon Alexa, Apple Siri, and Google Assistant are AI-powered
voice assistants that can answer questions, set reminders, play music, and
control smart home devices. They understand spoken language and respond
based on their programming.
3. Recommendation Systems
o Example: Netflix, YouTube, and Spotify use AI to recommend movies, TV
shows, and music based on what you’ve watched or listened to in the past. AI
analyzes your behavior and preferences to suggest content that you are most
likely to enjoy.
4. Chatbots in Customer Service
o Example: Many companies use AI chatbots on their websites and apps to assist
customers. These bots can answer basic questions, help with bookings, and
solve common issues without human intervention. Examples include Bank of
America’s Erica and H&M’s chatbot.

U n i t - 1 P a g e 1 | 40
 DEEP LEARNING III AIML SEM-II 2

5. Facial Recognition
o Example: FaceID on iPhones uses AI to recognize your face and unlock your
phone. Similarly, airports use AI-powered facial recognition to identify
passengers quickly and improve security. AI is also used in social media
platforms like Facebook to automatically tag people in photos.
6. AI in Healthcare
o Example: AI is used to help doctors diagnose diseases. For example, AI systems
can analyze X-rays, MRIs, or CT scans and find patterns that indicate diseases
like cancer or heart disease. AI is also used to help predict a patient’s risk for
certain conditions based on medical data.
7. Smart Home Devices
o Example: Devices like Nest Thermostat use AI to learn your temperature
preferences and adjust accordingly to save energy. Ring Doorbell uses AI for
facial recognition and motion detection, alerting homeowners when someone is
at the door or when motion is detected outside.
8. AI in Finance
o Example: Banks use AI for fraud detection by analyzing transaction patterns
and identifying unusual activity. Robo-advisors, like those offered by
Betterment and Wealthfront, use AI to help manage investment portfolios and
make financial recommendations.
9. Autonomous Drones
o Example: Amazon Prime Air and other companies are using AI-powered
drones to deliver packages. The drones navigate through the air using AI to
avoid obstacles and find the most efficient delivery routes.
10. AI in Retail
• Example: Amazon Go stores use AI to allow customers to shop without going to a
checkout counter. Sensors and cameras track what customers pick up and automatically
charge them when they leave the store.
11. AI in Education
• Example: AI-powered tutoring systems, like Knewton and Socratic, help students
learn by providing personalized learning experiences. AI can track a student's progress
and offer custom lessons based on their strengths and weaknesses.
12. AI in Agriculture
• Example: AI-powered drones and sensors are used to monitor crop health, predict
harvest times, and detect pests. John Deere uses AI in its machinery to optimize
planting, fertilization, and harvesting.

U n i t - 1 P a g e 2 | 40
 DEEP LEARNING III AIML SEM-II 3

13. AI in Sports
• Example: In professional sports, AI is used to track player performance, analyze
strategies, and predict game outcomes. For example, AI can analyze soccer players
movements and help coaches understand how to improve their team’s tactics.
14. AI in Entertainment and Media
• Example: AI is used to create realistic visual effects in movies and TV shows.
Companies like Disney use AI for animation, motion capture, and to generate
computer-generated characters. AI also helps in creating deepfake videos, where an
actor’s face can be swapped with another person’s using AI technology.

2.The Evolution of Artificial Intelligence (AI)

The history of Artificial Intelligence (AI) is fascinating because it has evolved over many
decades, from simple ideas to the advanced technologies we see today. Let's break down its
evolution in simple terms:

1. The Beginning of AI (1950s - 1960s)

AI started as a new idea in the 1950s. People began to wonder: Could machines think like
humans?
• Alan Turing (a British mathematician) proposed a way to test if a machine could think.
This test is called the Turing Test. It asks whether a machine can behave in a way that
makes it seem like a human. If a person can’t tell the difference between a machine and
a person, the machine is said to have human-like intelligence.
• In 1956, John McCarthy (an American computer scientist) officially coined the term
Artificial Intelligence. This is the moment AI was recognized as its own field of study.
Key Idea: Early AI researchers were focused on creating machines that could simulate human
thinking.

2. Early AI Programs (1960s - 1970s)

In the 1960s and 1970s, AI research continued to grow. Early programs were based on rule-
based systems, which were simple sets of rules that machines followed to make decisions.
• Example: In 1966, ELIZA, an early AI program, was created. It acted like a chatbot
and could hold a simple conversation with people. However, it was not truly intelligent;
it just followed patterns of words.
Key Idea: During this time, AI focused on creating systems that could mimic human
reasoning using rules.

U n i t - 1 P a g e 3 | 40
 DEEP LEARNING III AIML SEM-II 4

3. AI Winter (1970s - 1990s)

After the initial excitement about AI, there was a period called the AI Winter (1970s-1990s).
This is when AI progress slowed down because:
• Early AI systems couldn’t do much beyond basic tasks.
• Researchers realized that AI programs were too limited and had major flaws. They
couldn't handle complex problems or real-world situations.
• Funding for AI research was reduced, and many scientists became skeptical.
Key Idea: AI faced challenges because the technology wasn’t advanced enough to solve
difficult problems, and researchers started losing confidence in its potential.

4. The Rise of Machine Learning (1990s - 2000s)

By the late 1990s, AI research began to shift towards Machine Learning (ML), which is a
new approach to AI. Instead of programming a machine with every rule, scientists started
teaching machines to learn from data and improve over time.
• In 1997, an important milestone was reached when IBM’s Deep Blue (a
supercomputer) beat the world chess champion, Garry Kasparov. This was the first
time a computer had beaten a human champion in chess, showing that AI could solve
complex problems with powerful computing.
• The success of Machine Learning and improved algorithms (the steps that AI follows
to make decisions) allowed machines to learn more efficiently and perform better.
Key Idea: AI evolved to focus on teaching machines to learn from data, rather than just
following fixed rules.

5. The Deep Learning Revolution (2010s - Present)

From the 2010s onwards, AI saw a major breakthrough with the rise of Deep Learning. Deep
learning is a type of Machine Learning that uses neural networks, which are systems
designed to work like the human brain.
• Neural Networks are made up of layers of artificial "neurons" that process information
in stages. These networks can learn from massive amounts of data and perform tasks
that were impossible before.
• Example: In 2012, a deep learning system called AlexNet won the ImageNet
competition by accurately classifying images, something that was previously difficult
for AI. This was a game-changer for image recognition.
In the past decade, AI has rapidly advanced with:
• Self-driving cars that use AI to navigate and avoid obstacles.
• Voice assistants like Siri, Alexa, and Google Assistant, which use AI to understand
and respond to human speech.
U n i t - 1 P a g e 4 | 40
 DEEP LEARNING III AIML SEM-II 5

• AI in healthcare, where AI can help doctors diagnose diseases from medical images,
analyze health data, and predict patient outcomes.
Key Idea: Deep Learning has allowed AI to perform complex tasks like recognizing images,
understanding speech, and even making decisions in real-time.

6. Current and Future AI (2020s and Beyond)

AI is now a huge part of our everyday lives, and it continues to evolve. We’re seeing AI being
used in many areas:
• Smart homes with AI-powered devices like smart thermostats, lights, and fridges.
• AI in business to improve customer service, manage inventories, and personalize
shopping experiences.
• Artificial General Intelligence (AGI): The next big step is creating AI that can think
and learn as well as humans. This kind of AI could do any task a human can do, but we
are still far from achieving it.
Researchers are also working on AI ethics to make sure that AI is used fairly and safely, without
harming people or society.
Key Idea: AI is getting smarter and more widespread, and in the future, it could solve even
more complex problems, possibly even becoming as intelligent as humans.

Summary of AI Evolution
• 1950s - 1960s: The idea of AI was born, and early experiments tried to simulate human
thinking.
• 1960s - 1970s: AI grew with rule-based systems, but progress was slow.
• 1970s - 1990s: AI faced a slowdown due to limitations, known as the AI Winter.
• 1990s - 2000s: AI shifted to Machine Learning, allowing computers to learn from data.
• 2010s - Present: AI exploded with Deep Learning, improving tasks like image
recognition and voice processing.
• Future: AI may become even smarter, with the potential for Artificial General
Intelligence (AGI).
The evolution of AI shows how far we've come, from simple ideas to powerful technologies,
and there is still much more to come!

U n i t - 1 P a g e 5 | 40
 DEEP LEARNING III AIML SEM-II 6

3.Why Do We Need AI?

Artificial Intelligence (AI) is becoming an essential part of our lives because it helps us in many
ways. But why exactly do we need AI? Let’s break it down in simple terms.

1. AI Helps Us Handle Large Amounts of Data

Why is this important?
In today's world, there is more information (data) than ever before. Whether it's medical
records, business reports, social media posts, or online shopping behaviors, there's just too
much data for humans to process on their own.
• AI can analyze big data quickly and find patterns or trends that humans might miss.
• For example, AI can look at thousands of medical records and help doctors identify
patterns that might point to a disease. Without AI, this would take years of manual work
and might not catch everything.
Example: In healthcare, AI can examine X-rays, MRIs, and CT scans, finding signs of diseases
like cancer or heart disease much faster than a human doctor could.

2. AI Saves Time and Automates Tasks

Why is this important?
People have many tasks in their daily lives and work, some of which are repetitive or boring.
By using AI, these tasks can be automated, which means machines do the work while
humans focus on more important things.
• AI can take care of repetitive tasks, like checking emails, sorting data, or even helping
manage inventories in stores or warehouses.
• This saves a lot of time and lets people focus on creative or complex tasks that require
human judgment.
Example: In factories, robots with AI can assemble products, sort parts, or manage stock
automatically, allowing human workers to handle more interesting or difficult jobs.

3. AI Improves Decision-Making
Why is this important?
Humans can make decisions based on experience, intuition, or available information.
However, AI can make decisions based on large amounts of data much faster than a human
can. This helps people make better, more informed choices.
• AI systems can analyze data and give recommendations that might be too complex
for humans to calculate easily.

U n i t - 1 P a g e 6 | 40
 DEEP LEARNING III AIML SEM-II 7

• AI can also predict outcomes. For example, AI can predict the weather, the stock
market, or even a person's likelihood of developing a health problem.
Example: In business, AI helps companies decide what products to sell, how to price them,
and where to advertise by analyzing past sales data and trends. This can lead to better profits
and smarter strategies.

4. AI Can Work 24/7 Without Getting Tired

Why is this important?
Humans need sleep, breaks, and rest. But AI systems can work non-stop, which means they
can help with tasks anytime, anywhere, without needing rest.
• AI can perform tasks continuously and consistently, like monitoring security
cameras, answering customer queries, or managing online services.
Example: AI-powered customer service chatbots work 24/7 to answer questions and solve
problems for customers. This helps businesses provide service at any time, even during nights
or holidays when human staff may not be available.

5. AI Can Solve Complex Problems

Why is this important?
Some problems are so complex that they are hard for humans to understand or solve by
themselves. AI can take large amounts of information and use advanced algorithms to find
solutions that would be impossible for a human to figure out on their own.
• AI can solve difficult problems like designing new medicines, optimizing traffic
systems, or predicting future events based on historical data.
Example: In drug discovery, AI is used to find new treatments for diseases by analyzing
millions of molecules and predicting which ones might work best. This speeds up the process
of finding cures and saves lives.

6. AI Helps Us Personalize Experiences

Why is this important?
AI can help create experiences that are personalized to each person. This makes interactions
more relevant, enjoyable, and efficient.
• AI systems learn about you over time and can recommend things like movies, music,
or even products based on your interests.
• This personalized approach makes services more convenient and user-friendly.
Example: When you use Netflix, Spotify, or Amazon, the AI behind these services remembers
your preferences and suggests movies, songs, or products you might like. This saves you time
looking for things you enjoy.

U n i t - 1 P a g e 7 | 40
 DEEP LEARNING III AIML SEM-II 8

7. AI Enhances Safety and Security

Why is this important?
AI can be used to monitor situations and identify risks or dangers faster than a human
could. This can help keep people safe in many ways.
• AI can detect unusual activity or threats in real-time, which is very useful in places
like airports, banks, or online systems.
• It can also help prevent accidents by predicting problems before they happen.
Example: In self-driving cars, AI continuously monitors the environment to avoid accidents.
It can detect other cars, pedestrians, or obstacles much faster than a human driver could.

8. AI Helps with Innovation and New Ideas

Why is this important?
AI can help discover new things and generate creative solutions that we might not think of
on our own. By analyzing lots of data and patterns, AI can suggest new ideas or even create
new inventions.
• AI is used in research to come up with new solutions to old problems, whether it’s in
technology, medicine, or space exploration.
• It can help invent new products, design new technology, or improve existing processes.
Example: AI is used in artificial intelligence art creation where machines can help generate
music, paintings, or even new recipes, showing the creative side of AI.

Conclusion: Why Do We Need AI?

In short, we need AI because:
1. It helps us handle large amounts of data and make sense of complex information.
2. It saves time by automating repetitive tasks.
3. It improves decision-making by providing better insights from data.
4. It works continuously without needing breaks, offering help 24/7.
5. It solves difficult problems that are beyond human abilities.
6. It personalizes experiences, making services better and more relevant to us.
7. It enhances safety and security, protecting people and systems.
8. It drives innovation, helping us discover new ideas and solutions.
As AI continues to grow and evolve, it will become an even bigger part of our lives, solving
problems and making tasks easier, faster, and more efficient!

U n i t - 1 P a g e 8 | 40
 DEEP LEARNING III AIML SEM-II 9

4.Fundamentals of Machine Learning?

Machine Learning (ML) is a type of Artificial Intelligence (AI) that allows computers to
learn from experience and improve their performance without being specifically programmed.
In simple terms, it’s when a computer or machine can learn from data, make decisions, and
improve over time without human help.

How Does Machine Learning Work?

Imagine you want to teach a computer to recognize pictures of cats and dogs. Instead of telling
the computer exactly what a cat or dog looks like, you give it a lot of pictures of cats and dogs.
The computer then looks at these pictures and learns from them. Over time, the more pictures
it sees, the better it becomes at identifying whether a new picture shows a cat or a dog.
Key Steps in Machine Learning:
1. Data: The computer is given examples of data (like pictures, numbers, or text).
2. Learning: The computer looks for patterns in the data to understand the differences or
similarities.
3. Prediction or Decision: Once the computer has learned from the data, it can make
predictions or decisions about new, unseen data.
4. Improvement: The computer gets better over time as it is given more examples or data.

Types of Machine Learning

There are three main types of Machine Learning:
1. Supervised Learning
2. Unsupervised Learning
3. Semi-Supervised Learning
4. Reinforcement Learning
Let’s explain each of them in simple terms:

1. Supervised Learning
In supervised learning, the machine is given labeled data to learn from. Labeled data means
that for each piece of data, we already know the correct answer (like knowing that a picture
shows a cat or a dog).

U n i t - 1 P a g e 9 | 40
 DEEP LEARNING III AIML SEM-II 10

How it works:
• You provide the machine with lots of data and the correct answers (labels).
• The machine looks at these examples, learns the patterns, and uses those patterns to
predict the answer for new data.
Example: If you want to teach a computer to identify fruits, you give it lots of pictures of fruits
along with their names (apple, banana, orange, etc.). Over time, the computer learns to
recognize fruits based on the patterns in the images.
Real-life example: Email spam filters: Supervised learning is used to train spam filters. You
give the computer emails that are marked as "spam" or "not spam." The computer learns from
these examples and then can predict whether a new email is spam.

2. Unsupervised Learning
In unsupervised learning, the machine is given unlabeled data, meaning the machine doesn't
know the correct answer. The goal is for the machine to find patterns or group the data on its
own.
How it works:
• You give the machine a large amount of data without any labels or answers.
• The machine looks for patterns and tries to group similar things together.
Example: If you give the machine a collection of pictures without telling it what’s in the
pictures, the machine might group similar pictures together, such as all the images of animals
in one group and all the images of buildings in another.
Real-life example: Customer segmentation: Unsupervised learning is often used in
marketing. A company might have customer data but not know how to categorize them. The
machine can group customers based on their behaviors (like shopping habits or interests) to
help the company target their marketing efforts.

3. Semi-Supervised Learning
This combines labeled and unlabeled data. It’s helpful when labeling data is expensive or
time-consuming.
• Example:
A dataset has 1000 images of cats and dogs, but only 100 are labeled. The model uses
labeled images to learn basic patterns and then applies this knowledge to classify the
unlabeled images.
• Real-Time Example:
o Medical Diagnosis: Doctors may label a few medical images with diseases, and
the model uses these to learn and classify other unlabeled images.

U n i t - 1 P a g e 10 | 40
 DEEP LEARNING III AIML SEM-II 11

4.Reinforcement Learning
In reinforcement learning, the machine learns by trial and error, just like how a child learns
to ride a bicycle. It tries things, makes mistakes, and gets feedback (rewards or punishments),
which helps it improve its actions over time.
How it works:
• The machine is given a goal or task but doesn’t know how to achieve it at first.
• It takes actions and receives feedback (positive or negative) based on how well it did.
• Over time, the machine learns which actions lead to good results (rewards) and which
lead to bad results (punishments).
Example: If you teach a computer to play a game like chess or Go, the machine makes moves,
gets points for good moves, and loses points for bad moves. After playing many games, it learns
the best strategy.
Real-life example: Self-driving cars use reinforcement learning to make decisions. The car
"learns" by driving in different situations (like in traffic or during bad weather) and adjusts its
behavior to improve over time.

Applications of Machine Learning

Machine learning is used in many areas of our lives. Here are a few examples:
1. Voice Assistants: Voice assistants like Siri, Alexa, and Google Assistant use machine
learning to understand and respond to your voice commands. They learn over time to
become better at understanding your accent, tone, and requests.
2. Recommendation Systems: When you use Netflix, Spotify, or Amazon, machine
learning helps these services recommend movies, music, or products based on your past
behavior and preferences.
3. Image Recognition: Facebook and Instagram use machine learning to automatically
tag people in photos. The machine learns to recognize faces and compare them to others
in its database.
4. Healthcare: Machine learning is used in medical diagnostics. For example, AI can
analyze medical images (like X-rays or MRIs) to detect diseases such as cancer.
5. Autonomous Vehicles: Self-driving cars use machine learning to understand their
environment and make decisions like when to stop, turn, or speed up based on their
sensors and data from the road.

In Short What is Machine Learning?

Machine learning is a powerful technology that allows computers to learn from data
and improve over time without needing to be programmed for every single task. It’s used in
many everyday technologies, such as voice assistants, recommendation systems, and self-
U n i t - 1 P a g e 11 | 40
 DEEP LEARNING III AIML SEM-II 12

driving cars. Machine learning can be divided into three main types: supervised learning,
unsupervised learning, and reinforcement learning, each with different ways of helping
computers learn and solve problems. As machine learning continues to grow, it will play an
even bigger role in shaping the future of technology.

5. History of Machine Learning

Machine Learning (ML) is a subset of AI that focuses on enabling computers to learn
from data rather than being explicitly programmed. Its development can be categorized into
several key areas:

1. Probabilistic Modeling: Probabilistic modeling uses mathematics to handle uncertainty in

data. These models are based on probability theory and help in making predictions when the
outcome is not certain.
• Example:
Suppose you're predicting if it will rain tomorrow. You consider factors like today's
temperature, humidity, and wind speed. A probabilistic model will assign probabilities
to the outcomes, like:
o 70% chance of rain.
o 30% chance of no rain.
• Real-Time Example:
o Spam Email Detection: Email services use probabilistic models to decide
whether an email is spam or not. Based on words like "free money" or "lottery,"
the model calculates the probability of the email being spam.

2. Early Neural Networks

Neural networks are inspired by the human brain and mimic how neurons work. Early neural
networks were simple but effective models for basic tasks.
• Example:
o The Perceptron: An early neural network that can classify data into two
categories. For example, if you provide it with data about shapes (like circles
and squares), it learns to classify a new shape as either a circle or a square.
• Real-Time Example:
o Handwriting Recognition: Early neural networks were used to recognize
handwritten digits, like in postal services to read zip codes.

U n i t - 1 P a g e 12 | 40
 DEEP LEARNING III AIML SEM-II 13

3. Kernel Methods
Kernel methods transform data into a higher-dimensional space to make it easier to find
patterns or separate groups. The most famous kernel method is the Support Vector Machine
(SVM).
• Example:
Imagine you have a set of points in a 2D space that cannot be separated by a straight
line. A kernel method transforms this 2D data into a 3D space where a plane can
separate the points.
• Real-Time Example:
o Face Detection: Kernel methods are used in computer vision tasks like
identifying whether a face is present in an image.

4. Decision Trees
A decision tree is like a flowchart that splits data based on conditions to make decisions. Each
node in the tree represents a decision based on a feature.
• Example:
Suppose you're building a model to decide if someone is eligible for a loan:
o Is the income > $50,000?
▪ Yes → Check credit score.
▪ No → Not eligible.
o Is the credit score > 700?
▪ Yes → Eligible.
▪ No → Not eligible.
• Real-Time Example:
o Loan Approval: Banks use decision trees to evaluate whether a customer
qualifies for a loan based on income, credit score, and repayment history.

U n i t - 1 P a g e 13 | 40
 DEEP LEARNING III AIML SEM-II 14

5. Random Forests
A random forest is a collection of decision trees where each tree gives a prediction, and the
majority vote is taken as the final result. It reduces errors and improves accuracy.
• Example:
If you have three decision trees predicting whether a fruit is an apple or orange:
o Tree 1: Apple.
o Tree 2: Orange.
o Tree 3: Apple.
The random forest takes the majority vote, so the prediction is "Apple."
• Real-Time Example:
o Fraud Detection: Credit card companies use random forests to detect
fraudulent transactions by analyzing patterns in spending behavior.

6. Gradient Boosting Machines

Gradient Boosting Machines (GBMs) build decision trees sequentially, where each new tree
corrects the errors of the previous one. This method leads to high accuracy.
• Example:
Imagine predicting house prices. The first tree predicts $200,000 for a house but makes
an error of $20,000. The second tree focuses on reducing that $20,000 error, and so on.
• Real-Time Example:
o Recommendation Systems: Online platforms like Amazon use GBMs to
recommend products by analyzing past purchases and user preferences.
By understanding these historical milestones, you can appreciate how machine learning
evolved to solve real-world problems efficiently.

U n i t - 1 P a g e 14 | 40
 DEEP LEARNING III AIML SEM-II 15

6.Evaluating Machine Learning Models

Evaluating a machine learning model means testing how well it works. This is important
because we want the model to perform well not just on the training data (the data it learned
from) but also on new, unseen data.

Key Metrics for Evaluation

1. Accuracy

2. Precision

U n i t - 1 P a g e 15 | 40
 DEEP LEARNING III AIML SEM-II 16

3. Recall (Sensitivity)

4. F1-Score

U n i t - 1 P a g e 16 | 40
 DEEP LEARNING III AIML SEM-II 17

5. Confusion Matrix
A confusion matrix is a table that shows the model’s predictions compared to the
actual results. It includes:
o True Positives (TP): Correctly predicted positive cases.
o True Negatives (TN): Correctly predicted negative cases.
o False Positives (FP): Incorrectly predicted positive cases.
o False Negatives (FN): Missed positive cases.
o Example:
For a medical test predicting disease:
▪ TP: The test correctly predicts the patient has the disease.
▪ TN: The test correctly predicts the patient doesn’t have the disease.
▪ FP: The test falsely predicts the patient has the disease.
▪ FN: The test falsely predicts the patient doesn’t have the disease.
o Real-Time Example:
▪ Medical Diagnosis: Doctors analyze the confusion matrix to understand
how often a test misses cases or gives false alarms.

Other Evaluation Techniques

1. Train-Test Split
o Divide the data into two parts: training data (to teach the model) and test data
(to evaluate it).
o Real-Time Example:
▪ Weather Forecasting: Train the model on past weather data and test it
on recent data to check accuracy.
2. Cross-Validation
o The data is split into multiple parts, and the model is tested on each part to get
an average performance score.
o Real-Time Example:
▪ Stock Market Predictions: Cross-validation ensures the model works
well on different time periods of stock data.

By evaluating models carefully, we ensure they perform well on real-world tasks and avoid
errors that could lead to incorrect predictions or decisions.

U n i t - 1 P a g e 17 | 40
 DEEP LEARNING III AIML SEM-II 18

7.Overfitting and Underfitting in Machine Learning

When building a machine learning model, we want it to work well for both the data it learned
from (training data) and new, unseen data (test data). Two common problems can occur:
overfitting and underfitting.

1. Overfitting
Overfitting happens when the model learns too much from the training data, including noise
and irrelevant details. It becomes overly complex and memorizes the training data instead of
understanding general patterns. This means it performs very well on training data but poorly
on new data.
• Why It Happens:
o The model is too complex (e.g., too many features or parameters).
o Training for too long on the same dataset.
• Example:
Imagine you're studying for a math test by memorizing every question in the practice
book. On the practice test, you do great because you've memorized the answers. But in
the real exam, where the questions are different, you fail because you didn’t understand
the concepts.

Real-Time Examples:
• Chatbot Training: If a chatbot is trained only on customer conversations from one
company, it might respond well to similar phrases but fail with new or diverse customer
questions.
• Face Recognition: A face recognition model trained on a small dataset of specific faces
may fail to recognize new faces because it memorized details of the training faces.

2. Underfitting
Underfitting occurs when the model is too simple and doesn’t learn enough from the training
data. It fails to capture important patterns and performs poorly on both training and test data.
• Why It Happens:
o The model is too simple (e.g., not enough features or too few parameters).
o The training process is stopped too early.
o The data provided is insufficient or not well-preprocessed.
• Example:
Imagine you study for a math test using only the first page of your textbook. You don’t

U n i t - 1 P a g e 18 | 40
 DEEP LEARNING III AIML SEM-II 19

learn enough, so you fail both the practice test and the real exam because you didn’t
study most of the material.
• Real-Time Examples:
o Weather Prediction: A model that only uses temperature to predict rain might
underfit because it ignores other factors like humidity and wind speed.
o Stock Market Predictions: A model that considers only one or two variables, like
past prices, may fail to capture the complexity of market trends and give inaccurate
predictions.

Key Differences Between Overfitting and Underfitting

Aspect Overfitting Underfitting

Model Too complex (memorizes noise and Too simple (misses important
Complexity irrelevant data) patterns)

Excellent on training data, poor on

Performance Poor on both training and new data
new data

Training too long or using an overly Using a model that's too basic or not
Cause
detailed model trained enough

Skimming through only a few

Example Memorizing practice questions
concepts

How to Handle Overfitting and Underfitting

1. Fixing Overfitting:
o Simplify the Model: Use fewer features or parameters.
o Regularization: Add a penalty to overly complex models to prevent them from
focusing on irrelevant details.
o Early Stopping: Stop training when the model starts overfitting.
o More Data: Train the model on a larger and more diverse dataset.
o Real-Time Example:
▪ If a face recognition model is overfitting, adding more photos with
different lighting, angles, and expressions can improve generalization.
2. Fixing Underfitting:
o Increase Model Complexity: Use more features, layers, or parameters.
o Train for Longer: Allow the model more time to learn patterns.
U n i t - 1 P a g e 19 | 40
 DEEP LEARNING III AIML SEM-II 20

o Better Features: Provide higher-quality data or preprocess data better.

o Real-Time Example:
▪ If a weather prediction model underfits, including additional variables
like pressure, wind, and season might improve its performance.

By avoiding overfitting and underfitting, we create models that not only perform well on
training data but also make accurate predictions on new, unseen data. This balance is called
generalization, and it's the ultimate goal in machine learning.

8. More Explaination about Each Machine Learning Models

1.Probabilistic Modeling in Machine Learning

Probabilistic modeling in machine learning is a way of predicting outcomes when
there is uncertainty in the data. Instead of giving a definite answer like "yes" or "no," it
calculates the likelihood (probability) of different outcomes happening.
It’s useful for making decisions when things aren’t certain.

Key Concept of Probabilistic Modeling

1. Handling Uncertainty:
Real-world data often has noise or missing information. Probabilistic models accept
this uncertainty and make the best possible prediction based on probabilities.
o Example:
If you’re guessing the weather for tomorrow:
▪ 70% chance of rain.
▪ 30% chance of no rain.
2. Probability Distribution:
Probabilistic models use probability distributions to describe all possible outcomes.
o Example: Rolling a dice:
▪ Each number (1-6) has an equal probability of 1/6

How Probabilistic Modeling Works ?

1. Input Data: The model takes data with some uncertainty.
2. Training: The model learns patterns from the data to estimate probabilities.
3. Prediction: The model predicts outcomes along with their likelihoods.
U n i t - 1 P a g e 20 | 40
 DEEP LEARNING III AIML SEM-II 21

Types of Probabilistic Models

1. Naive Bayes Classifier
A simple probabilistic model used for classification tasks.
o Example: Email Spam Detection
▪ The model looks at words in the email (like "free," "money") and
calculates the probability of the email being spam vs. not spam.
▪ Prediction: 90% spam, 10% not spam → It labels the email as spam.
2. Bayesian Networks
A graphical model that shows relationships between variables and predicts based on
those relationships.
o Example: Medical Diagnosis
▪ Variables: Symptoms, diseases, test results.
▪ If a person has a fever and cough, the model calculates the likelihood of
flu vs. common cold.
3. Hidden Markov Models (HMM)
Used to predict sequences or patterns over time.
o Example: Speech Recognition
▪ The model predicts the sequence of words spoken based on probabilities
of sounds and grammar.

Real-Life Examples of Probabilistic Modeling

1. Weather Forecasting
o Probabilistic models predict the likelihood of different weather conditions.
o Example:
▪ 70% chance of rain.
▪ 20% chance of cloudy weather.
▪ 10% chance of sunny weather.
▪ You decide to carry an umbrella because rain is most likely.

2. Spam Detection
o Email services like Gmail use probabilistic models to classify emails as spam
or not spam.

U n i t - 1 P a g e 21 | 40
 DEEP LEARNING III AIML SEM-II 22

o Example:
▪ Words like "win" and "prize" increase the probability of an email being
spam.
▪ If the model predicts 85% chance spam, the email goes to the spam
folder.

3. Medical Diagnosis
o Doctors use probabilistic models to predict diseases based on symptoms and test
results.
o Example:
▪ A patient has symptoms of headache and fever.
▪ Model prediction:
▪ 80% chance of flu.
▪ 15% chance of dengue.
▪ 5% chance of something else.
▪ The doctor treats for flu since it’s most likely.

4. Fraud Detection in Banking

o Banks use probabilistic models to identify potentially fraudulent transactions.
o Example:
▪ Unusual spending at odd hours might have an 85% chance of being
fraud.
▪ The bank blocks the transaction and alerts the customer.

5. Product Recommendations
o Platforms like Amazon or Netflix use probabilistic models to suggest products
or movies.
o Example:
▪ Based on your past viewing, the model predicts:
▪ 70% chance you’ll like "Inception."
▪ 50% chance you’ll like "Interstellar."
▪ It recommends "Inception" first.

U n i t - 1 P a g e 22 | 40
 DEEP LEARNING III AIML SEM-II 23

Advantages of Probabilistic Modeling

1. Handles Uncertainty: Works well when the data is incomplete or noisy.
2. Realistic Predictions: Instead of giving a hard "yes" or "no," it provides probabilities,
which mimic real-world decision-making.
3. Wide Applications: Useful in fields like healthcare, finance, natural language
processing, and more.

2.What are Early Neural Networks?

Early neural networks are computer systems inspired by how our brains work. They try to
mimic the way humans think and learn by using something called "neurons." These artificial
neurons are mathematical functions that work together to solve problems. Early neural
networks were simpler than modern ones, but they set the foundation for today's advanced
systems.

How Do Early Neural Networks Work?

1. Input Layer: Receives data (like numbers, images, or words).

o Example: A picture of a handwritten number like "5."
2. Hidden Layers: Process the data using connected "neurons."
o Neurons are like tiny decision-makers, deciding if a feature is important.
3. Output Layer: Gives the result.
o Example: Predicts the number in the image is "5."

U n i t - 1 P a g e 23 | 40
 DEEP LEARNING III AIML SEM-II 24

Important Features of Early Neural Networks

1. Perceptron:
o One of the first neural network models (introduced in the 1950s).
o Could solve simple problems like distinguishing between two categories.
o Example: Classifying whether an email is spam or not based on a few rules.
2. Feedforward Networks:
o Data flows in one direction, from input to output.
o Used for basic tasks like recognizing patterns.
3. Backpropagation:
o A method developed later to train networks.
o Helps the network improve by correcting errors after each attempt.

Examples of Early Neural Networks

1. Handwritten Digit Recognition
• Example: Recognizing handwritten digits (like 0-9) using simple neural networks.
• Real-Time Example: Postal systems used early neural networks to read handwritten
ZIP codes on letters.
2. Predicting Weather
• Example: Basic neural networks used past weather data to predict future conditions.
• Real-Time Example: Early weather forecasting systems used neural networks for basic
temperature predictions.
3. Speech Recognition
• Example: Early systems used simple networks to recognize spoken words like "yes" or
"no."
• Real-Time Example: Voice-controlled toys in the 1990s used early neural networks
for limited vocabulary.

Limitations of Early Neural Networks

1. Could Solve Only Simple Problems:
o They couldn't handle complex tasks like recognizing faces or translating
languages.

U n i t - 1 P a g e 24 | 40
 DEEP LEARNING III AIML SEM-II 25

2. Required a Lot of Data and Time:

o Training them was slow, and they needed many examples to learn.
3. Limited Computing Power:
o Early computers weren't powerful enough to train large networks.

How They Paved the Way for Modern AI

• Early neural networks introduced the idea of mimicking the human brain.
• They inspired today's advanced systems like deep learning, which can drive cars,
recognize faces, and even chat like humans.

Real-Life Impact
While early neural networks couldn't do much compared to today's AI, they laid the foundation
for big developments in:
• Banking: Detecting fraudulent transactions.
• Healthcare: Diagnosing diseases using patient data.
• Technology: Improving early speech and handwriting recognition systems.
These networks were simple but were a major step toward the complex AI systems we use
today.

3.What is Kernel Methods Explained

Kernel methods are techniques in machine learning that help find patterns in data. They are
used to solve problems where the data is not easily separated into categories or groups. These
methods work by transforming the data into a higher dimension where it's easier to classify or
analyze.

How Do Kernel Methods Work?

1. Raw Data: Start with data that might be hard to separate into categories.
o Example: Imagine data points scattered in a circle where one category is inside
the circle and another is outside.
2. Transformation: Kernel methods transform the data into a new space where it becomes
easier to separate.

U n i t - 1 P a g e 25 | 40
 DEEP LEARNING III AIML SEM-II 26

o This transformation is done without explicitly moving the data, saving time and
computation.
3. Learning Patterns: After transformation, a machine learning model like a Support
Vector Machine (SVM) can find the boundary between different categories.

Key Idea: The Kernel Function

The kernel function helps calculate the relationship between data points in the transformed
space. Common kernel functions include:
• Linear Kernel: For simple, straight-line separations.
• Polynomial Kernel: For curved boundaries.
• Radial Basis Function (RBF) Kernel: For very complex, non-linear relationships.

Examples of Kernel Methods

1. Handwriting Recognition
• Problem: Recognize handwritten letters like "A" and "B."
• How Kernel Methods Help: The shapes of "A" and "B" might overlap in raw data. A
kernel function transforms the data so the system can clearly separate the letters.
• Real-Time Example: Postal systems that read addresses on envelopes.
2. Medical Diagnosis
• Problem: Classify patients as having a disease or not based on symptoms.
• How Kernel Methods Help: Symptoms might not have a clear pattern in raw data, but
kernel methods make it easier to find a boundary.
• Real-Time Example: Predicting whether a patient has diabetes based on test results.
3. Image Classification
• Problem: Identify whether a photo contains a cat or a dog.
• How Kernel Methods Help: Transform the pixel data to identify patterns like fur
texture or ear shape.
• Real-Time Example: Mobile apps that can classify photos in your gallery.

Advantages of Kernel Methods

1. Handles Non-Linearity: Can solve problems where data is not clearly separated in its
original form.
o Example: Separating circular or spiral patterns.
U n i t - 1 P a g e 26 | 40
 DEEP LEARNING III AIML SEM-II 27

2. Versatile: Works well with many types of data, including images, text, and numbers.
o Example: Analyzing customer reviews to classify them as positive or negative.
3. Powerful with Small Data: Performs well even with limited data, unlike deep learning,
which often requires large datasets.

Limitations of Kernel Methods

1. Computationally Expensive: When dealing with very large datasets, kernel methods
can be slow.
o Example: Processing millions of user transactions in an e-commerce system.
2. Hard to Choose the Right Kernel: Selecting the best kernel function for a problem
requires trial and error.
3. Not Ideal for High-Dimensional Data: Struggles with data that has too many features
or dimensions.

Real-Time Applications of Kernel Methods

1. Banking: Fraud Detection
o Detect unusual transactions by classifying patterns in user behavior.
2. Social Media: Sentiment Analysis
o Analyze tweets or comments to determine if they are positive, negative, or
neutral.
3. Healthcare: Tumor Detection
o Classify medical images to identify whether a tumor is benign or malignant.
4. Face Recognition:
o Transform facial features into a space where different faces are easier to
distinguish.

Kernel methods are like special lenses that help machine learning models see patterns in data
that are otherwise hard to detect. They are still widely used in areas where complex patterns
need to be identified efficiently.

U n i t - 1 P a g e 27 | 40
 DEEP LEARNING III AIML SEM-II 28

4.What is Decision Trees Explained.

A Decision Tree is a machine learning model that helps in making decisions based on certain
criteria. It is called a "tree" because it looks like a flowchart, where each decision splits into
more options based on specific conditions or features.
In simpler terms, a decision tree helps answer questions by asking a series of "Yes/No" or
"True/False" questions at each step, like a branching path. At the end of the tree, it gives you a
final answer or prediction.

How Does a Decision Tree Work?

1. Start with a Question (Root):
The tree starts with a simple question or decision based on the most important feature
of the data.
o Example: "Is the weather sunny?"
2. Branching (Splitting):
Based on the answer (Yes or No), the tree splits into different branches that represent
further questions or decisions.
o If "Yes": Go to another question, like "Is the temperature above 30°C?"
o If "No": Move to a different question, like "Is it raining?"
3. Leaves (Final Decision):
Eventually, you reach the leaves of the tree, which are the final decisions or
predictions.
o If the leaf says "Yes, go outside," it means the conditions are good for outdoor
activities.
o If it says "No, stay inside," it means the weather isn’t suitable.

Example of a Simple Decision Tree

Imagine you want to decide whether to go out for a walk based on the weather:
• Root Question: "Is it sunny?"
o Yes: Go to the next question.
o No: Stay inside.
• Next Question (if it's sunny): "Is it hot (above 30°C)?"
o Yes: Stay inside, it's too hot for a walk.
o No: Go for a walk, it's a good day for it.

U n i t - 1 P a g e 28 | 40
 DEEP LEARNING III AIML SEM-II 29

• Next Question (if it's not sunny): "Is it raining?"

o Yes: Stay inside, it's wet outside.
o No: Go for a walk, it's clear.
This simple decision tree helps you decide when to go for a walk based on the weather
conditions.

Real-Time Examples of Decision Trees

1. Customer Support (Problem Resolution)
o Example: When you call customer service, they might use a decision tree to
help troubleshoot your problem.
o How It Works:
▪ "Is your device turning on?"
▪ Yes → "Is the screen working?"
▪ No → "Check the power cable."
▪ The tree guides the support representative to ask the right questions and
offer a solution.
2. Medical Diagnosis
o Example: A doctor may use a decision tree to diagnose a disease based on
symptoms.
o How It Works:
▪ "Do you have a fever?"
▪ Yes → "Do you have a cough?"
▪ No → "Check for other symptoms."
▪ The tree helps the doctor decide on the next step based on the answers.
3. Loan Approval in Banks
o Example: A bank might use a decision tree to decide whether to approve a loan
application.
o How It Works:
▪ "Is the applicant's credit score above 700?"
▪ Yes → "Is the income above $50,000?"
▪ No → "Decline the loan."
▪ The decision tree helps the bank make quick and consistent decisions.

U n i t - 1 P a g e 29 | 40
 DEEP LEARNING III AIML SEM-II 30

4. Movie Recommendations
o Example: Streaming services like Netflix use decision trees to recommend
movies.
o How It Works:
▪ "Do you like action movies?"
▪ Yes → "Do you like superheroes?"
▪ No → "Do you prefer romantic comedies?"
▪ Based on answers, the service recommends movies you’re most likely
to enjoy.

Advantages of Decision Trees

1. Easy to Understand: Decision trees are simple and resemble human decision-making,
so they're easy to interpret.
o Example: You can easily explain a decision tree to someone without technical
knowledge.
2. Can Handle Both Numeric and Categorical Data: They can work with all types of
data, including numbers and categories like colors, types of animals, etc.
3. No Need for Data Normalization: Unlike some other algorithms, decision trees don’t
require you to normalize or scale data.

Limitations of Decision Trees

1. Overfitting: Decision trees can become too complex and specific to the training data,
making them less accurate on new data.
o Example: A tree that is too detailed may perform well on past weather data but
fail to predict future weather correctly.
2. Instability: A small change in the data can result in a very different tree.
o Example: If new customers are added with different characteristics, the tree may
change dramatically.
3. Bias Toward Features with More Levels: Decision trees may favor features with more
categories or levels.
o Example: If you're classifying animals, a feature like "color" (with many
possible values) might become more important than "size," even though "size"
might be a better indicator of the animal type.

U n i t - 1 P a g e 30 | 40
 DEEP LEARNING III AIML SEM-II 31

Improvement of Decision Trees

To overcome these issues, decision trees are often combined with other algorithms to form
stronger models, such as:
• Random Forest: Combines many decision trees to improve accuracy.
• Gradient Boosting Machines: Builds trees sequentially, each improving the
performance of the previous one.

Conclusion
In summary, decision trees are powerful tools in machine learning for making decisions based
on a series of questions. They are used in many areas like customer service, healthcare,
banking, and entertainment to make quick, interpretable decisions based on data.

5.What are Random Forests Explained?

A Random Forest is a machine learning algorithm that uses many decision trees to make
predictions. Imagine a forest where each tree gives its opinion, and the Random Forest
combines all those opinions to make a better decision. The main idea behind Random Forests
is that by using many trees, it can create a more accurate and stable prediction compared to
using just one decision tree.

How Does Random Forest Work?

1. Multiple Decision Trees:
Random Forest builds many decision trees. Each tree is trained on a random sample
of the data. The idea is that each tree will make slightly different mistakes, but when
you combine them, they’ll work together to improve accuracy.
2. Random Sampling:
Instead of using the entire dataset to build each tree, Random Forest uses random
subsets of data and random subsets of features (variables) for each tree. This
randomness ensures that no single tree dominates the decision process.
3. Voting or Averaging:
After all the trees in the forest have made a prediction, the Random Forest algorithm
either:
o Votes for classification tasks (like deciding whether an email is spam or not)
o Averages the predictions for regression tasks (like predicting the price of a
house).

U n i t - 1 P a g e 31 | 40
 DEEP LEARNING III AIML SEM-II 32

Steps Involved in Random Forest

1. Create Multiple Random Subsets:
Random Forest randomly selects subsets of data (called bootstrapping) and features
(called feature bagging) to build each tree.
2. Build Decision Trees:
For each subset, a decision tree is built based on the selected features. Each tree is
trained on different data and might make different decisions.
3. Make Predictions:
After the trees are trained, when new data comes in, all the trees make their
predictions, and the Random Forest algorithm decides the final prediction based on
majority voting (for classification) or averaging (for regression).

Example of Random Forest in Action

1. Email Spam Detection
• Problem: Classify whether an email is spam or not.
• How Random Forest Helps:
o Each tree in the forest looks at different features of the email, such as the sender,
subject, and specific words.
o Some trees might focus more on words like "free" or "urgent," while others
might focus on the sender's email address.
o All trees vote, and if most trees classify the email as "spam," the final prediction
is spam.
2. Predicting Loan Approval
• Problem: Decide whether to approve or reject a loan based on customer information
(income, credit score, etc.).
• How Random Forest Helps:
o Random Forest builds multiple decision trees, each using different sets of data
features (like income, credit score, etc.).
o Each tree makes a prediction on whether the loan should be approved or
rejected.
o The Random Forest combines all the predictions and makes a final decision,
increasing the chance of accurate loan approval predictions.

U n i t - 1 P a g e 32 | 40
 DEEP LEARNING III AIML SEM-II 33

Real-Time Examples of Random Forests

1. Healthcare (Disease Diagnosis)
o Example: A Random Forest can predict whether a patient has a particular
disease based on various factors like age, gender, symptoms, and test results.
o How It Works: Multiple decision trees are trained on different features like age,
test results, and medical history, and then they "vote" on whether the patient has
the disease or not.
2. E-commerce (Customer Segmentation)
o Example: An online store might use a Random Forest to segment customers
based on their buying behavior (age, past purchases, browsing history).
o How It Works: Multiple decision trees look at different aspects of customer
behavior, and then the Random Forest classifies customers into segments like
"frequent shoppers," "bargain hunters," or "occasional buyers."
3. Stock Market Predictions
o Example: A Random Forest could predict whether the stock market will go up
or down based on factors like economic indicators, past performance, and global
events.
o How It Works: Different trees use different economic indicators, past stock
prices, and global news to make their predictions. The Random Forest combines
all predictions to make a final decision.

Advantages of Random Forest

1. Accuracy:
Random Forest generally gives more accurate results than individual decision trees
because it uses multiple trees to make decisions. The combined result is more reliable.
2. Handles Overfitting:
Overfitting happens when a model is too complex and works well on training data but
fails on new, unseen data. Since Random Forest uses multiple trees and random
sampling, it reduces the risk of overfitting.
3. Works with Missing Data:
Random Forest can handle datasets with missing values. It can still make predictions
by using available data from other trees.
4. Versatile:
Random Forest can be used for both classification tasks (like spam detection) and
regression tasks (like predicting house prices).

U n i t - 1 P a g e 33 | 40
 DEEP LEARNING III AIML SEM-II 34

Limitations of Random Forest

1. Computationally Expensive:
Random Forest needs a lot of processing power, especially when there are many trees
and features in the dataset. It can be slower compared to simpler models.
2. Hard to Interpret:
While decision trees are easy to interpret, a Random Forest with many trees can
become difficult to understand. It’s harder to explain why a decision was made since
it involves many trees.
3. Memory Intensive:
Storing multiple trees requires a lot of memory, especially when working with large
datasets.

Improvement of Random Forests

To improve Random Forests, several techniques can be applied:
• Feature Importance: Random Forests can give you the importance of each feature
(e.g., which factors influence the decision the most), helping in model optimization.
• Hyperparameter Tuning: Adjusting parameters like the number of trees or depth of
trees can improve performance.

Conclusion
In summary, a Random Forest is a powerful machine learning technique that combines
multiple decision trees to make more accurate predictions. It’s widely used in various real-
world applications like email spam detection, disease diagnosis, and customer behavior
analysis due to its ability to handle complex data and reduce overfitting.

6.What is Gradient Boosting Machines (GBM)

Gradient Boosting Machines (GBM) are a popular machine learning technique used for both
classification (like predicting whether an email is spam) and regression (like predicting house
prices). The main idea behind GBM is to combine the predictions of several simpler models
(usually decision trees) to create a powerful, accurate model.
Think of it like a team of workers. Each worker makes a mistake, but when they work together
and correct each other's mistakes, they end up doing a much better job. GBM builds multiple
decision trees sequentially, each time focusing on the mistakes made by the previous tree.

U n i t - 1 P a g e 34 | 40
 DEEP LEARNING III AIML SEM-II 35

How Does Gradient Boosting Work?

1. Start with a Simple Model:
The process begins by creating a very simple model (usually a shallow decision tree)
that makes an initial prediction on the data.
2. Calculate the Errors:
After making the initial predictions, the algorithm looks at the errors made by the first
tree (the difference between predicted values and actual values). These errors are also
called residuals.
3. Build the Next Tree:
A new decision tree is built to predict the errors (residuals) of the first tree. This new
tree focuses on correcting the mistakes made by the previous tree.
4. Combine the Trees:
The predictions of the second tree are added to the predictions of the first tree. This
combined prediction is more accurate.
5. Repeat the Process:
This process continues for multiple trees. Each new tree corrects the mistakes of the
combined model from previous trees. The model gradually improves as more trees are
added.
6. Final Prediction:
The final prediction is made by combining all the trees, usually by adding the
predictions from each tree together. For classification tasks, it can be done using a
majority vote, and for regression tasks, the predictions are averaged.

Example of Gradient Boosting in Action

1. Predicting House Prices
• Problem: You want to predict the price of a house based on features like size, number
of rooms, location, etc.
• How GBM Helps:
o Step 1: The first decision tree makes an initial guess about the house price.
o Step 2: The second tree looks at the errors made by the first tree and tries to
improve by correcting those mistakes.
o Step 3: The third tree improves further, and this process continues for several
trees.
o Result: The final prediction of the house price is more accurate because each
tree fixes the errors made by the previous ones.

U n i t - 1 P a g e 35 | 40
 DEEP LEARNING III AIML SEM-II 36

2. Customer Churn Prediction

• Problem: A telecom company wants to predict whether a customer will leave the
service (churn) based on features like usage patterns, contract type, and customer
service interactions.
• How GBM Helps:
o Step 1: The first tree predicts the likelihood of churn based on the available
features.
o Step 2: The second tree looks at the mistakes made by the first tree and focuses
on customers who were misclassified.
o Step 3: The trees continue correcting each other's mistakes, gradually
improving the prediction.
o Result: The final model provides a better prediction of whether a customer will
leave or stay.

Real-Time Examples of Gradient Boosting Machines

1. Fraud Detection in Banking
o Example: Banks use GBM to detect fraudulent transactions. The model checks
patterns like unusual spending behavior or login attempts from strange
locations.
o How It Works: Each tree focuses on correcting previous errors, gradually
improving the detection of fraud.
2. Search Engine Rankings
o Example: Google uses machine learning models like GBM to rank web pages
based on factors like relevance, keywords, and page quality.
o How It Works: GBM combines multiple decision trees, each focusing on
different aspects of page ranking to provide more accurate search results.
3. Recommendation Systems
o Example: Streaming platforms like Netflix use GBM to recommend movies
based on your past viewing history and preferences.
o How It Works: GBM helps predict which movies you might like, based on
previous mistakes and corrections made by the algorithm.
4. Stock Market Predictions
o Example: GBM can be used to predict stock prices based on various features
like past performance, economic indicators, and global news.
o How It Works: The model builds multiple trees that improve its predictions by
focusing on the residuals (errors) of previous predictions.
U n i t - 1 P a g e 36 | 40
 DEEP LEARNING III AIML SEM-II 37

Advantages of Gradient Boosting Machines

1. High Accuracy:
GBM is known for delivering high predictive accuracy because it builds multiple
models that improve over time.
o Example: It can achieve better results compared to single decision trees or linear
models.
2. Handles Complex Data:
GBM works well with both simple and complex data, including numerical,
categorical, and text data.
o Example: Predicting customer behavior based on a mix of data types like
transaction history and customer demographics.
3. Feature Importance:
GBM can provide insights into which features are most important for predictions.
o Example: In predicting house prices, the model might show that "location" and
"size" are more important than "number of rooms."

Limitations of Gradient Boosting Machines

1. Computationally Intensive:
GBM can be slow to train, especially with large datasets, because it builds multiple
trees sequentially.
o Example: Training a model with millions of data points may take a long time,
making it unsuitable for real-time predictions in some cases.
2. Sensitive to Noise:
GBM can overfit the data if there is a lot of noise or irrelevant features, making the
model too complex and less generalizable.
o Example: If there are many irrelevant features in a dataset, GBM might
"memorize" the noise, leading to poor performance on new data.
3. Hard to Interpret:
Like other ensemble methods, GBM can be difficult to interpret, especially when
dealing with many trees.
o Example: Explaining why the model made a certain prediction might be
challenging, as it involves combining multiple decision trees.

U n i t - 1 P a g e 37 | 40
 DEEP LEARNING III AIML SEM-II 38

Improvement Techniques for Gradient Boosting

• Early Stopping:
To prevent overfitting, training can be stopped early if the model's performance on
validation data stops improving.
• Regularization:
Regularization techniques like limiting the depth of trees or controlling the learning rate
can help reduce overfitting.
• Tuning Hyperparameters:
Adjusting hyperparameters such as the number of trees, learning rate, and tree depth
can improve the model's performance.

Conclusion
In summary, Gradient Boosting Machines (GBM) are a powerful machine learning technique
that combines multiple decision trees to make more accurate predictions. By focusing on
correcting the errors of previous models, GBM creates a strong, reliable model that can be used
in many real-world applications, from fraud detection to customer churn prediction. Despite its
strengths, GBM requires careful tuning and can be computationally intensive.

U n i t - 1 P a g e 38 | 40
 DEEP LEARNING III AIML SEM-II 39

Important Questions

1. Fundamentals of Deep Learning: Artificial Intelligence

1. Discuss the relationship between Deep Learning and Artificial Intelligence.
2. Explain the concept of Artificial Intelligence (AI) and its applications in various
industries. (Discuss its significance in modern technology.)
3. Define Artificial Intelligence (AI) and explain its significance in modern
technology.

2. History of Machine Learning: Probabilistic Modeling

1. Describe the history of Machine Learning, focusing on early methods like

probabilistic modeling and neural networks. (Highlight their evolution and
importance.)
2. Explain the concept of probabilistic modeling in Machine Learning. (Provide
examples of its applications in decision-making systems.)
3. What is Probabilistic Modeling in Machine Learning?
4. Explain the significance of Probabilistic Modeling in the evolution of Machine
Learning.

3. Early Neural Networks

Illustrate the structure and working of early neural networks. (Discuss their role in the
development of deep learning.)
Or
What are Early Neural Networks, and how do they function?
Or
Discuss the limitations of Early Neural Networks and their impact on the field.
Or
Describe the architecture of early neural networks and analyze their limitations in
comparison to modern deep learning models.

4. Kernel Methods

Analyze the role of kernel methods in Machine Learning. (Explain their importance in
handling non-linear data and give examples.)
Or
What are Kernel Methods in Machine Learning?
Or
Explain how Kernel Methods enhance the performance of Machine Learning
algorithms

U n i t - 1 P a g e 39 | 40
 DEEP LEARNING III AIML SEM-II 40

5. Decision Trees

Explain decision trees and their use in classification and regression tasks. (Provide a real-
world example to illustrate their application.)
Or
What is a Decision Tree, and how is it constructed?
Or
Discuss the advantages and disadvantages of using Decision Trees in Machine
Learning.
Or
Discuss the construction of decision trees and evaluate their advantages and
disadvantages in classification tasks.

6. Random Forests and Gradient Boosting Machines

Differentiate between Random Forests and Gradient Boosting Machines. (Highlight their
strengths, limitations, and use cases with examples.)
Or
What is a Random Forest, and how does it improve upon individual Decision Trees?
Or
Explain the concept of Gradient Boosting Machines and their advantages over other
ensemble methods.

7. Fundamentals of Machine Learning: Four Branches of Machine Learning

1. Discuss the four branches of Machine Learning: supervised, unsupervised,
semi-supervised, and reinforcement learning. (Provide examples of tasks suited for
each branch.)
2. Describe the four branches of Machine Learning and provide examples of each
3. OIdentify and explain the four branches of machine learning, providing
examples for each.

8. Evaluating Machine Learning Models

1. Evaluate the techniques used for assessing Machine Learning models. (Compare
metrics like accuracy, precision, recall, and F1-score with examples.)
2. Explain the concepts of overfitting and underfitting in Machine Learning.
(Describe methods to address these challenges, such as regularization and cross-validation.)
3. Describe the key metrics used in evaluating machine learning models and
analyze the challenges associated with overfitting and underfitting

U n i t - 1 P a g e 40 | 40