MACHINE LEARNING

Machine learning (ML) is a subdomain of artificial intelligence (AI) that focuses on developing
systems that learn—or improve performance—based on the data they ingest. Artificial intelligence is a
broad word that refers to systems or machines that resemble human intelligence. Machine learning and
AI are frequently discussed together, and the terms are occasionally used interchangeably, although
they do not signify the same thing. A crucial distinction is that, while all machine learning is AI, not all
AI is machine learning.

Data scientists use exploratory data analysis (EDA) to analyze and

investigate data sets and summarize their main characteristics, often
employing data visualization methods.
What is Machine Learning?
Machine Learning is the field of study that allows computers to learn without being explicitly
programmed. ML is one of the most exciting technologies that one would have ever come across. As
it is evident from the name, it gives the computer that makes it more similar to humans: The ability to
learn. Machine learning is actively being used today, perhaps in many more places than expected.
In the real world, we are surrounded by humans who can learn everything from their
experiences with their learning capability, and we have computers or machines which work
on our instructions. But can a machine also learn from experiences or past data like a human
does? So here comes the role of Machine Learning.

How does Machine Learning work

A machine learning system builds prediction models, learns from previous data, and predicts the
output of new data whenever it receives it. The amount of data helps to build a better model that
accurately predicts the output, which in turn affects the accuracy of the predicted output.

Let's say we have a complex problem in which we need to make predictions. Instead of writing
code, we just need to feed the data to generic algorithms, which build the logic based on the data
and predict the output. Our perspective on the issue has changed as a result of machine learning.
The Machine Learning algorithm's operation is depicted in the following block diagram:
Features of Machine learning Machine learning is data driven technology. Large amount of data
generated by organizations on daily bases. So, by notable relationships in data, organizations makes
better decisions.
 Machine can learn itself from past data and automatically improve.
 From the given dataset it detects various patterns on data.
 For the big organizations branding is important and it will become easier to target relatable
customer base.
 It is similar to data mining because it is also deals with the huge amount of data.

Need for Machine Learning

The demand for machine learning is steadily rising. Because it is able to perform tasks that are too
complex for a person to directly implement, machine learning is required. Humans are constrained
by our inability to manually access vast amounts of data; as a result, we require computer systems,
which is where machine learning comes in to simplify our lives.

By providing them with a large amount of data and allowing them to automatically explore the
data, build models, and predict the required output, we can train machine learning algorithms. The
cost function can be used to determine the amount of data and the machine learning algorithm's
performance. We can save both time and money by using machine learning.

The significance of AI can be handily perceived by its utilization's cases, Presently, AI is utilized in
self-driving vehicles, digital misrepresentation identification, face acknowledgment, and
companion idea by Facebook, and so on. Different top organizations, for example, Netflix and
Amazon have constructed AI models that are utilizing an immense measure of information to
examine the client interest and suggest item likewise.

Following are some key points which show the importance of Machine Learning:

 Rapid increment in the production of data

 Solving complex problems, which are difficult for a human
 Decision making in various sector including finance
 Finding hidden patterns and extracting useful information from data.

Applications of Machine learning

Machine learning is a buzzword for today's technology, and it is growing very rapidly day by day.
We are using machine learning in our daily life even without knowing it such as Google Maps,
Google assistant, Alexa, etc. Below are some most trending real-world applications of Machine
Learning:
1. Image Recognition:

Image recognition is one of the most common applications of machine learning. It is used to
identify objects, persons, places, digital images, etc. The popular use case of image recognition and
face detection is, Automatic friend tagging suggestion:

Facebook provides us a feature of auto friend tagging suggestion. Whenever we upload a photo
with our Facebook friends, then we automatically get a tagging suggestion with name, and the
technology behind this is machine learning's face detection and recognition algorithm.

It is based on the Facebook project named "Deep Face," which is responsible for face recognition
and person identification in the picture.

2. Speech Recognition

While using Google, we get an option of "Search by voice," it comes under speech recognition,
and it's a popular application of machine learning.

Speech recognition is a process of converting voice instructions into text, and it is also known as
"Speech to text", or "Computer speech recognition." At present, machine learning algorithms are
widely used by various applications of speech recognition. Google assistant, Siri, Cortana, and
Alexa are using speech recognition technology to follow the voice instructions.

3. Traffic prediction:

If we want to visit a new place, we take help of Google Maps, which shows us the correct path with
the shortest route and predicts the traffic conditions.

It predicts the traffic conditions such as whether traffic is cleared, slow-moving, or heavily
congested with the help of two ways:

 Real Time location of the vehicle form Google Map app and sensors
 Average time has taken on past days at the same time.
Everyone who is using Google Map is helping this app to make it better. It takes information from
the user and sends back to its database to improve the performance.

4. Product recommendations:

Machine learning is widely used by various e-commerce and entertainment companies such as
Amazon, Netflix, etc., for product recommendation to the user. Whenever we search for some
product on Amazon, then we started getting an advertisement for the same product while internet
surfing on the same browser and this is because of machine learning.

Google understands the user interest using various machine learning algorithms and suggests the
product as per customer interest.

As similar, when we use Netflix, we find some recommendations for entertainment series, movies,
etc., and this is also done with the help of machine learning.

5. Self-driving cars:

One of the most exciting applications of machine learning is self-driving cars. Machine learning
plays a significant role in self-driving cars. Tesla, the most popular car manufacturing company is
working on self-driving car. It is using unsupervised learning method to train the car models to
detect people and objects while driving.

6. Email Spam and Malware Filtering:

Whenever we receive a new email, it is filtered automatically as important, normal, and spam. We
always receive an important mail in our inbox with the important symbol and spam emails in our
spam box, and the technology behind this is Machine learning. Below are some spam filters used
by Gmail:

 Content Filter
 Header filter
 General blacklists filter
 Rules-based filters
 Permission filters

Some machine learning algorithms such as Multi-Layer Perceptron, Decision tree, and Naïve
Bayes classifier are used for email spam filtering and malware detection.

7. Virtual Personal Assistant:

We have various virtual personal assistants such as Google assistant, Alexa, Cortana, Siri. As the
name suggests, they help us in finding the information using our voice instruction. These assistants
can help us in various ways just by our voice instructions such as Play music, call someone, Open
an email, Scheduling an appointment, etc.

These virtual assistants use machine learning algorithms as an important part.

These assistant record our voice instructions, send it over the server on a cloud, and decode it using
ML algorithms and act accordingly.
8. Online Fraud Detection:

Machine learning is making our online transaction safe and secure by detecting fraud transaction.
Whenever we perform some online transaction, there may be various ways that a fraudulent
transaction can take place such as fake accounts, fake ids, and steal money in the middle of a
transaction. So to detect this, Feed Forward Neural network helps us by checking whether it is a
genuine transaction or a fraud transaction.

For each genuine transaction, the output is converted into some hash values, and these values
become the input for the next round. For each genuine transaction, there is a specific pattern which
gets change for the fraud transaction hence, it detects it and makes our online transactions more
secure.

9. Stock Market trading:

Machine learning is widely used in stock market trading. In the stock market, there is always a risk
of up and downs in shares, so for this machine learning's long short term memory neural
network is used for the prediction of stock market trends.

10. Medical Diagnosis:

In medical science, machine learning is used for diseases diagnoses. With this, medical technology
is growing very fast and able to build 3D models that can predict the exact position of lesions in the
brain.

It helps in finding brain tumors and other brain-related diseases easily.

11. Automatic Language Translation:

Nowadays, if we visit a new place and we are not aware of the language then it is not a problem at
all, as for this also machine learning helps us by converting the text into our known languages.
Google's GNMT (Google Neural Machine Translation) provide this feature, which is a Neural
Machine Learning that translates the text into our familiar language, and it called as automatic
translation.

The technology behind the automatic translation is a sequence to sequence learning algorithm,
which is used with image recognition and translates the text from one language to another
language.

Getting started with Machine Learning

From translation apps to autonomous vehicles, all powers with Machine Learning. It offers a way to
solve problems and answer complex questions. It is basically a process of training a piece of
software called an algorithm or model, to make useful predictions from data. This article discusses
the categories of machine learning problems, and terminologies used in the field of machine
learning.
Types of machine learning problems

There are various ways to classify machine learning problems. Here, we discuss the most obvious
ones.
 1. On the basis of the nature of the learning “signal” or “feedback” available to a learning
system
 Supervised learning: The model or algorithm is presented with example inputs and their
desired outputs and then finds patterns and connections between the input and the output. The
goal is to learn a general rule that maps inputs to outputs. The training process continues until
the model achieves the desired level of accuracy on the training data. Some real-life examples
are:

 Image Classification: You train with images/labels. Then in the future, you give a
new image expecting that the computer will recognize the new object.

 Market Prediction/Regression: You train the computer with historical market

data and ask the computer to predict the new price in the future.

Examples:

 Email Spam Detection: The model is trained on emails labeled as "spam" or "not spam."
Features such as keywords and metadata are used to classify new emails.
 Credit Card Fraud Detection: The model uses labeled transaction data to predict whether
new transactions are fraudulent.

Elaboration: In supervised learning, the process involves:

1. Data Collection: Gather labeled data.

2. Feature Extraction: Identify relevant features.
3. Model Training: Use algorithms like linear regression, decision trees, or neural networks to
train the model.
4. Evaluation: Test the model on unseen data to assess accuracy.

 Unsupervised learning: No labels are given to the learning algorithm, leaving it on its
own to find structure in its input. It is used for clustering populations in different groups.
Unsupervised learning can be a goal in itself (discovering hidden patterns in data).

 Clustering: You ask the computer to separate similar data into clusters, this is
essential in research and science.

 High-Dimension Visualization: Use the computer to help us visualize high-

dimension data.

 Generative Models: After a model captures the probability distribution of your

input data, it will be able to generate more data. This can be very useful to make
your classifier more robust.
 Customer Segmentation: Group customers into segments based on purchasing behavior
without predefined labels.
 Anomaly Detection: Identify unusual data points in datasets, such as detecting unusual
network traffic.

Elaboration: In unsupervised learning, the process involves:

 1. Data Collection: Gather unlabeled data.
2. Feature Extraction: Identify relevant features.
3. Model Training: Use algorithms like k-means clustering, hierarchical clustering, or
principal component analysis (PCA) to find patterns.
4. Evaluation: Assess the model's performance based on its ability to find meaningful
patterns.

A simple diagram that clears the concept of supervised and unsupervised learning is shown
below:

As you can see clearly, the data in supervised learning is labeled, whereas data in unsupervised
learning is unlabelled.
 Semi-supervised learning: Problems where you have a large amount of input data and
only some of the data is labelled, are called semi-supervised learning problems. These
problems sit in between both supervised and unsupervised learning. For example, a photo
archive where only some of the images are labelled, (e.g. dog, cat, person) and the majority
are unlabelled.

 Reinforcement learning: A computer program interacts with a dynamic environment in

which it must perform a certain goal (such as driving a vehicle or playing a game against an
opponent). The program is provided feedback in terms of rewards and punishments as it
navigates its problem space.

Examples:

 Game Playing: Training agents to play games like chess or Go, where the agent learns
optimal strategies through trial and error.
 Robotics: Training robots to navigate and manipulate objects in an environment.

Elaboration: In reinforcement learning, the process involves:

1. Environment Setup: Define the environment where the agent operates.

2. Reward System: Define rewards for desirable actions and penalties for undesirable actions.
3. Policy Learning: Use algorithms like Q-learning or deep reinforcement learning to learn
the best policy.
4. Evaluation: Test the agent in the environment to assess its performance.
2. Two most common use cases of supervised learning are:

 Classification: Inputs are divided into two or more classes, and the learner must produce
a model that assigns unseen inputs to one or more (multi-label classification) of these classes
and predicts whether or not something belongs to a particular class. This is typically tackled
in a supervised way. Classification models can be categorized in two groups: Binary
classification and Multiclass Classification. Spam filtering is an example of binary
classification, where the inputs are email (or other) messages and the classes are “spam” and
“not spam”.

In this case, an email service provider uses a supervised learning algorithm to classify incoming
emails as "spam" or "not spam." Here's how it works:

1. Data Collection: The system is trained using a large dataset of emails that have been
labeled as spam or not spam by human users.
2. Feature Extraction: The algorithm extracts features from these emails, such as the
presence of certain keywords, the frequency of those keywords, the sender's email address,
and other metadata.
3. Training: The labeled dataset is used to train a machine learning model. The model learns
to associate certain features with spam emails and others with non-spam emails.
4. Model Application: Once trained, the model can analyze new incoming emails in real time,
using the learned features to predict whether each email is spam or not.
5. Feedback Loop: Users can mark emails as spam or not spam, and this feedback is used to
continually improve the model.

For example, when you receive an email, the spam detection system quickly evaluates the content
and metadata of the email and determines whether it should be placed in your inbox or the spam
folder. If you find a spam email in your inbox and mark it as spam, this information helps the
system to improve its future predictions.

 Regression: It is also a supervised learning problem, that predicts a numeric value and
outputs are continuous rather than discrete. For example, predicting stock prices using
historical data.
Problem: Predicting the price of a house based on features such as size, number of bedrooms,
location, age, and other characteristics.

Steps Involved:

1. Data Collection: Gather historical data on houses, including their features and actual sale
prices.
2. Feature Selection: Identify relevant features that influence house prices, such as:
o Square footage
o Number of bedrooms and bathrooms
o Location (e.g., distance to city center, neighborhood quality)
o Age of the house
o Amenities (e.g., swimming pool, garage)
3. Model Training: Use a regression algorithm (e.g., linear regression, polynomial regression,
or more advanced techniques like gradient boosting regression) to learn the relationship
between the features and the house prices.
4. Model Evaluation: Assess the model's accuracy using metrics like Mean Absolute Error
(MAE), Mean Squared Error (MSE), or R-squared on a validation dataset.
5. Real-Time Prediction: Apply the trained model to predict the prices of new houses coming
into the market based on their features.

Example:

Suppose a real estate company wants to use machine learning to predict house prices for their new
listings. They have historical data on thousands of houses:

 Size: 2000 square feet

 Number of bedrooms: 3
 Number of bathrooms: 2
 Location: Suburban area, close to good schools
 Age: 10 years
 Amenities: Swimming pool, two-car garage

The company uses this data to train a regression model. Once trained, the model can take the
features of a new house and predict its price in real-time. For example, if a new house comes on the
market with the following features:

 Size: 2500 square feet

 Number of bedrooms: 4
 Number of bathrooms: 3
 Location: Urban area, close to public transportation
 Age: 5 years
 Amenities: No swimming pool, one-car garage

The model can predict a price, say $350,000, based on the learned relationships between features
and house prices.

An example of classification and regression on two different datasets is shown below:

3. Most common Unsupervised learning are:

 Clustering: Here, a set of inputs is to be divided into groups. Unlike in classification, the
groups are not known beforehand, making this typically an unsupervised task. Density
estimation: The task is to find the distribution of inputs in some space.
 Dimensionality reduction: It simplifies inputs by mapping them into a lower-
dimensional space. Topic modeling is a related problem, where a program is given a list of
human language documents and is tasked to find out which documents cover similar topics.

Problem: Identifying distinct customer groups within a large customer base to tailor marketing
strategies and offers.

Steps Involved:

1. Data Collection: Gather data on customer behavior and demographics, including:

o Purchase history
o Browsing behavior
o Age
o Location
o Income
o Product preferences
2. Feature Selection: Identify relevant features that can be used to segment customers.
3. Model Training: Use an unsupervised learning algorithm, such as k-means clustering or
hierarchical clustering, to group customers into clusters based on their similarities in the
feature space.
4. Model Evaluation: Analyze the clusters to ensure they make business sense and identify
distinct customer segments.
5. Real-Time Application: Continuously update the customer segments as new data comes in
and use these segments to tailor marketing efforts.

Example:

An e-commerce company wants to improve its marketing strategy by segmenting its customers into
different groups based on their shopping behavior. They have data on thousands of customers,
including:
  Purchase history: Total amount spent, frequency of purchases
 Browsing behavior: Pages visited, products viewed
 Demographics: Age, location, income
 Product preferences: Categories of products frequently bought

The company applies a k-means clustering algorithm to this data. The algorithm groups the
customers into clusters such as:

 Cluster 1: High spenders, frequent purchasers, primarily young adults

 Cluster 2: Occasional buyers, middle-aged, high-income individuals
 Cluster 3: Frequent browsers but infrequent buyers, young professionals
 Cluster 4: Bargain hunters, primarily interested in discounts and deals

Real-Time Application:

With these segments identified, the company can implement real-time targeted marketing
campaigns. For example:

 Cluster 1: Send personalized recommendations and exclusive early access to new products.
 Cluster 2: Offer premium services and high-end product promotions.
 Cluster 3: Provide incentives like discounts on their first purchase to convert browsing into
buying.
 Cluster 4: Send notifications about sales and special offers.

As customers interact with the website and make purchases, their behavior data is continuously fed
into the clustering model, allowing for dynamic updating of customer segments. This ensures that
the marketing strategies remain relevant and effective over time.

On the basis of these machine learning tasks/problems, we have a number of

algorithms that are used to accomplish these tasks. Some commonly used machine
learning algorithms are Linear Regression, Logistic Regression, Decision Tree,
SVM(Support vector machines), Naive Bayes, KNN(K nearest neighbors), K-
Means, Random Forest, etc. Note: All these algorithms will be covered in
upcoming articles.
Ensemble learning is a machine learning technique that aggregates two
or more learners (e.g. regression models, neural networks) in order to
produce better predictions. In other words, an ensemble model
combines several individual models to produce more accurate
predictions than a single model alone.
https://www.geeksforgeeks.org/a-comprehensive-guide-to-ensemble-learning/

Ensemble means ‘a collection of things’ and in Machine Learning

terminology, Ensemble learning refers to the approach of combining multiple
ML models to produce a more accurate and robust prediction compared to any
individual model. It implements an ensemble of fast algorithms (classifiers)
such as decision trees for learning and allows them to vote.
Table of Content
 What is ensemble learning with examples?
 Ensemble Learning Techniques
 Algorithm based on Bagging and Boosting
 How to stack estimators for a Classification Problem?
 Uses of Ensemble Learning
 Conclusion:

What is Ensemble Learning with examples?

 Ensemble learning is a machine learning technique that combines the
predictions from multiple individual models to obtain a better predictive
performance than any single model. The basic idea behind ensemble
learning is to leverage the wisdom of the crowd by aggregating the
predictions of multiple models, each of which may have its own strengths
and weaknesses. This can lead to improved performance and
generalization.
 Ensemble learning can be thought of as compensation for poor learning
algorithms that are computationally more expensive than a single model.
But they are more efficient than a single non-ensemble model that has
passed through a lot of learning. In this article, we will have a
comprehensive overview of the importance of ensemble learning and how it
works, different types of ensemble classifiers, advanced ensemble learning
techniques, and some algorithms (such as random forest, xgboost) for
better clarification of the common ensemble classifiers and finally their uses
in the technical world.
 Several individual base models (experts) are fitted to learn from the
same data and produce an aggregation of output based on which a final
decision is taken. These base models can be machine learning algorithms
such as decision trees (mostly used), linear models, support vector
machines (SVM), neural networks, or any other model that is capable of
making predictions.
 Most commonly used ensembles include techniques such as Bagging-
used to generate Random Forest algorithms and Boosting- to generate
algorithms such as Adaboost, Xgboost etc.
Ensemble Learning Techniques
 Gradient Boosting Machines (GBM): Gradient Boosting is a popular
ensemble learning technique that sequentially builds a group of decision
trees and corrects the residual errors made by previous trees, enhancing its
predictive accuracy. It trains each new weak learner to fit the residuals of
the previous ensemble’s predictions thus making it less sensitive to
individual data points or outliers in the data.
 Extreme Gradient Boosting (XGBoost): XGBoost features tree
pruning, regularization, and parallel processing, which makes it a preferred
choice for data scientists seeking robust and accurate predictive models.
 CatBoost: It is designed to handle features categorically that eliminates
the need for extensive pre-processing.CatBoost is known for its high
predictive accuracy, fast training, and automatic handling of overfitting.
 Stacking: It combines the output of multiple base models by training a
combiner(an algorithm that takes predictions of base models) and generate
more accurate prediction. Stacking allows for more flexibility in combining
diverse models, and the combiner can be any machine learning algorithm.
 Random Subspace Method (Random Subspace Ensembles): It is an
ensemble learning approach that improves the predictive accuracy by
training base models on random subsets of input features. It mitigates
overfitting and improves the generalization by introducing diversity in the
model space.
 Random Forest Variants: They introduce variations in tree construction,
feature selection, or model optimization to enhance performance.
Selecting the right advanced ensemble technique depends on the nature of the
data, the specific problem trying to be solved, and the computational resources
available. It often requires experimentation and changes to achieve the best
results.
Algorithm based on Bagging and Boosting
Bagging Algorithm
Bagging is a supervised learning technique that can be used for both
regression and classification tasks. Here is an overview of the steps including
Bagging classifier algorithm:
 Bootstrap Sampling: Divides the original training data into ‘N’ subsets
and randomly selects a subset with replacement in some rows from other
subsets. This step ensures that the base models are trained on diverse
subsets of the data and there is no class imbalance.
 Base Model Training: For each bootstrapped sample, train a base
model independently on that subset of data. These weak models are trained
in parallel to increase computational efficiency and reduce time
consumption.
 Prediction Aggregation: To make a prediction on testing data combine
the predictions of all base models. For classification tasks, it can include
majority voting or weighted majority while for regression, it involves
averaging the predictions.
 Out-of-Bag (OOB) Evaluation: Some samples are excluded from the
training subset of particular base models during the bootstrapping method.
These “out-of-bag” samples can be used to estimate the model’s
performance without the need for cross-validation.
 Final Prediction: After aggregating the predictions from all the base
models, Bagging produces a final prediction for each instance.

Boosting Algorithm
Boosting is an ensemble technique that combines multiple weak learners to
create a strong learner. The ensemble of weak models are trained in series
such that each model that comes next, tries to correct errors of the previous
model until the entire training dataset is predicted correctly. One of the most
well-known boosting algorithms is AdaBoost (Adaptive Boosting).
Here are few popular boosting algorithm frameworks:
 AdaBoost (Adaptive Boosting): AdaBoost assigns different weights to
data points, focusing on challenging examples in each iteration. It combines
weighted weak classifiers to make predictions.
 Gradient Boosting: Gradient Boosting, including algorithms like
Gradient Boosting Machines (GBM), XGBoost, and LightGBM, optimizes a
loss function by training a sequence of weak learners to minimize the
residuals between predictions and actual values, producing strong
predictive models.

Uses of Ensemble Learning

Ensemble learning is a versatile approach that can be applied to a wide range
of machine learning problems such as:-
 Classification and Regression: Ensemble techniques make problems
like classification and regression versatile in various domains, including
finance, healthcare, marketing, and more.
 Anomaly Detection: Ensembles can be used to detect anomalies in
datasets by combining multiple anomaly detection algorithms, thus making it
more robust.
 Portfolio Optimization: Ensembles can be employed to optimize
investment portfolios by collecting predictions from various models to make
better investment decisions.
 Customer Churn Prediction: In business and marketing analytics, by
combining the results of various models capturing different aspects of
customer behaviour, ensembles can be used to predict customer churn.
Churn is the measure of how many customers stop using a product. This can be
measured based on actual usage or failure to renew (when the product is sold using a
subscription model). Often evaluated for a specific period of time, there can be a
monthly, quarterly, or annual churn rate.
 Medical Diagnostics: In healthcare, ensembles can be used to make
more accurate predictions of diseases based on various medical data
sources and diagnostic models.
 Credit Scoring: Ensembles can be used to improve the accuracy of
credit scoring models by combining the outputs of various credit risk
assessment models.
 Climate Prediction: Ensembles of climate models help in making more
accurate and reliable predictions for weather forecasting, climate change
projections, and related environmental studies.
 Time Series Forecasting: Ensemble learning combines multiple time
series forecasting models to enhance accuracy and reliability, adapting to
changing temporal patterns.
 Conclusion
In conclusion ensemble learning is an method that harnesses the strengths
and diversity of multiple models to enhance prediction accuracy in various
machine learning applications. This technique is widely applicable, in areas
such as classification, regression, time series forecasting and other domains
where reliable and precise predictions are crucial. It also aids in mitigating
overfitting issues.

https://www.geeksforgeeks.org/underfitting-and-overfitting-in-machine-learning/

ML | Underfitting and Overfitting


When we talk about the Machine Learning model, we actually talk about how well it
performs and its accuracy which is known as prediction errors. Let us consider that we
are designing a machine learning model. A model is said to be a good machine learning
model if it generalizes any new input data from the problem domain in a proper way.
This helps us to make predictions about future data, that the data model has never seen.
Now, suppose we want to check how well our machine learning model learns and
generalizes to the new data. For that, we have overfitting and underfitting, which are
majorly responsible for the poor performances of the machine learning algorithms.

Bias and Variance in Machine Learning

 Bias: Bias refers to the error due to overly simplistic assumptions in the
learning algorithm. These assumptions make the model easier to
comprehend and learn but might not capture the underlying complexities of
the data. It is the error due to the model’s inability to represent the true
relationship between input and output accurately. When a model has poor
performance both on the training and testing data means high bias because
of the simple model, indicating underfitting.
 Variance: Variance, on the other hand, is the error due to the model’s
sensitivity to fluctuations in the training data. It’s the variability of the
model’s predictions for different instances of training data. High variance
occurs when a model learns the training data’s noise and random
fluctuations rather than the underlying pattern. As a result, the model
performs well on the training data but poorly on the testing data, indicating
overfitting.

Underfitting in Machine Learning

A statistical model or a machine learning algorithm is said to have underfitting
when a model is too simple to capture data complexities. It represents the
inability of the model to learn the training data effectively result in poor
performance both on the training and testing data. In simple terms, an underfit
model’s are inaccurate, especially when applied to new, unseen examples. It
mainly happens when we uses very simple model with overly simplified
assumptions. To address underfitting problem of the model, we need to use
more complex models, with enhanced feature representation, and less
regularization.
Note: The underfitting model has High bias and low variance.
Reasons for Underfitting
1. The model is too simple, So it may be not capable to represent the
complexities in the data.
2. The input features which is used to train the model is not the adequate
representations of underlying factors influencing the target variable.
3. The size of the training dataset used is not enough.
4. Excessive regularization are used to prevent the overfitting, which
constraint the model to capture the data well.
5. Features are not scaled.
Techniques to Reduce Underfitting
1. Increase model complexity.
2. Increase the number of features, performing feature engineering.
3. Remove noise from the data.
4. Increase the number of epochs or increase the duration of training to get
better results.
Overfitting in Machine Learning
A statistical model is said to be overfitted when the model does not make
accurate predictions on testing data. When a model gets trained with so much
data, it starts learning from the noise and inaccurate data entries in our data
set. And when testing with test data results in High variance. Then the model
does not categorize the data correctly, because of too many details and noise.
The causes of overfitting are the non-parametric and non-linear methods
because these types of machine learning algorithms have more freedom in
building the model based on the dataset and therefore they can really build
unrealistic models. A solution to avoid overfitting is using a linear algorithm if
we have linear data or using the parameters like the maximal depth if we are
using decision trees.
In a nutshell, Overfitting is a problem where the evaluation of machine learning
algorithms on training data is different from unseen data.
Reasons for Overfitting:
1. High variance and low bias.
2. The model is too complex.
3. The size of the training data.
Techniques to Reduce Overfitting
1. Improving the quality of training data reduces overfitting by focusing on
meaningful patterns, mitigate the risk of fitting the noise or irrelevant
features.
2. Increase the training data can improve the model’s ability to generalize
to unseen data and reduce the likelihood of overfitting.
3. Reduce model complexity.
4. Early stopping during the training phase (have an eye over the loss over
the training period as soon as loss begins to increase stop training).
5. Ridge Regularization and Lasso Regularization.
6. Use dropout for neural networks to tackle overfitting.

Good Fit in a Statistical Model

Ideally, the case when the model makes the predictions with 0 error, is said to
have a good fit on the data. This situation is achievable at a spot between
overfitting and underfitting. In order to understand it, we will have to look at the
performance of our model with the passage of time, while it is learning from the
training dataset.
With the passage of time, our model will keep on learning, and thus the error
for the model on the training and testing data will keep on decreasing. If it will
learn for too long, the model will become more prone to overfitting due to the
presence of noise and less useful details. Hence the performance of our model
will decrease. In order to get a good fit, we will stop at a point just before
where the error starts increasing. At this point, the model is said to have good
skills in training datasets as well as our unseen testing dataset.

https://www.geeksforgeeks.org/underfitting-and-overfitting-in-machine-learning/