Introduction to Linear Algebra for

Machine Learning

PRESENTATION BY UPLATZ
https://training.uplatz.com
Email: [email protected]
Phone: +44 7836 212635
 CONTENT
➢ Basic introduction to linear algebra and Mathematics
equations for Machine
➢ Learning
➢ Matrices
➢ Vectors in linear algebra.
 What is Linear Algebra?

➢ Linear algebra is a field of mathematics that is universally agreed

to be a prerequisite to a deeper understanding of machine learning.
➢ In Linear Algebra, data is represented by linear equations, which
are presented in the form of matrices and vectors.
➢ When you have the right libraries, like Numpy, at your disposal,
you can compute complex matrix multiplication very easily with
just a few lines of code.
 Mathematical Objects

Scalar
A scalar is simply a single number. For example 24.
Vector
A Vector is an ordered array of numbers and can be in a row or a column.
Matrix
A Matrix is an ordered 2D array of numbers and it has two indices. The first one
points to the row and the second one to the column.
Matrix
Matrix is a way of writing similar things together to handle and manipulate them as per our requirements easily. In Data Science, it
is generally used to store information like weights in an Artificial Neural Network while training various algorithms. You will be able
to understand my point by the end of this article.
Technically, a matrix is a 2-D array of numbers (as far as Data Science is concerned). For example look at the matrix A below.
Generally, rows are denoted by ‘i’ and column are denoted by ‘j’. The elements are indexed by ‘i’th row and ‘j’th column.We
denote the matrix by some alphabet e.g. A and its elements by A(ij).
In above matrix
A12 = 2 1 2 3
To reach to the result, go along first row and reach to second column.

Terms related to Matrix:

4 5 6

Order of matrix
7 8 9
Square matrix
Diagonal matrix
Upper triangular matrix
Lower triangular matrix
Scalar matrix
Identity matrix

Matrix
Column matrix
Row matrix
Trace
 What is an identity
matrix?

An identity matrix is a given square matrix of any order which

contains on its main diagonal elements with value of one, while the
rest of the matrix elements are equal to zero.
 Identity matrix
properties
An identity matrix is always an square matrix:
As seen in equations 1 and 2, the order of an identity matrix is always n, which
refers to the dimensions nxn (meaning there is always the same amount of rows and
columns in the matrix).
An identity matrix is capable of multiplying any matrix with any order (dimensions)
as long as it follows the next rules:
If in the multiplication, the identity matrix is the first factor, then the identity
matrix must have dimensions with as many columns as the matrix it is
multiplying has rows.

Equation 3: Multiplying an identity matrix times a non-unit matrix

If the identity matrix is the second factor in the multiplication, the
identity matrix must have the same amount of rows as the matrix
that is multiplying it has column

Equation 4: Multiplying a non-unit matrix times an identity matrix

Identity matrix example
problems
 Tensor
You can think of a Tensor as an array of numbers, arranged on a
regular grid, with a variable number of axes. A Tensor has three
indices, where the first one points to the row, the second to the
column and the third one to the axis. For example, T232 points to the
second row, the third column, and the second axis. This refers to the
value 0 in the right Tensor in the graphic below:
1. Matrix-Scalar Operations
If you multiply, divide, subtract, or add a Scalar to a Matrix, you do
so with every element of the Matrix. The image below illustrates this
perfectly for multiplication:
Matrix-Vector Multiplication
Multiplying a Matrix by a Vector can be thought of as
multiplying each row of the Matrix by the column of the
Vector. The output will be a Vector that has the same
number of rows as the Matrix. The image below shows
how this works:
 Matrix-Matrix Addition and
Subtraction
Matrix-Matrix Addition and Subtraction is fairly easy and
straightforward. The requirement is that the matrices have the same
dimensions and the result is a Matrix that has also the same
dimensions. You just add or subtract each value of the first Matrix
with its corresponding value in the second Matrix. See below:
Matrix-Matrix Multiplication
Multiplying two Matrices together isn’t that hard either if you know how to
multiply a Matrix by a Vector. Note that you can only multiply Matrices together if
the number of the first Matrix’s columns matches the number of the second
Matrix’s rows. The result will be a Matrix with the same number of rows as the
first Matrix and the same number of columns as the second Matrix.
Matrix Multiplication Properties

1. Not Commutative: Scalar Multiplication is commutative but

Matrix Multiplication is not. This means that when we are
multiplying Scalars, 7*3 is the same as 3*7. But when we
multiply Matrices by each other, A*B isn’t the same as B*A.
2. Associative: Scalar and Matrix Multiplication are both
associative. This means that the Scalar multiplication 3(5*3) is
the same as (3*5)3 and that the Matrix multiplication A(B*C) is
the same as (A*B)C.
3. Distributive: Scalar and Matrix Multiplication are also both
distributive. This means that 3(5 + 3) is the same as 3*5 + 3*3
and that A(B+C) is the same as A*B + A*C.
 Identity Matrix
The Identity Matrix is a special kind of Matrix but first, we need to define
what an Identity is. The number 1 is an Identity because everything you
multiply with 1 is equal to itself. Therefore every Matrix that is multiplied
by an Identity Matrix is equal to itself. For example, Matrix A times its
Identity-Matrix is equal to A.
You can spot an Identity Matrix by the fact that it has ones along its
diagonals and that every other value is zero. It is also a “squared matrix,”
meaning that its number of rows matches its number of columns.
Inverse and Transpose
The Matrix inverse and the Matrix transpose are two special kinds of Matrix
properties. Again, we will start by discussing how these properties relate to real
numbers and then how they relate to Matrices.
1. Inverse: First of all, what is an inverse? A number that is multiplied by its
inverse is equal to 1. Note that every number except 0 has an inverse. If you
multiply a Matrix by its inverse, the result is its Identity Matrix. The example
below shows what the inverse of Scalars looks like:
 2x2 Matrix
OK, how do we calculate the inverse?
Well, for a 2x2 matrix the inverse is:
In other words: swap the positions of a and d, put negatives in front of b
and c, and divide everything by the determinant (ad-bc).
Let us try an example:
How do we know this is the right answer?
Remember it must be true that: A × A-1 = I
So, let us check to see what happens when we multiply the matrix by its inverse:
The image below shows a Matrix multiplied by its inverse, which
results in a 2-by-2 identity Matrix.
Transpose
MACHINE LEARNING TUTORIAL

PRESENTATION BY UPLATZ
https://training.uplatz.com
Email: [email protected]
Phone: +44 7836 212635
Introduction to Machine Learning
Machine learning uses various algorithms for building
mathematical models and making predictions using historical
data or information.
What is Machine Learning?
Field of study that gives computers the capability to learn without
being explicitly programmed.
How is it different from traditional
programming?
In traditional programming, we would feed the input data and a well written and
tested program into a machine to generate output. When it comes to machine
learning, input data along with the output associated with the data is fed into the
machine during the learning phase, and it works out a program for itself.
Why do we need Machine
Learning?
➢ Machine Learning can automate many tasks, especially the ones
that only humans can perform with their innate intelligence.
➢ With the help of Machine Learning, businesses can automate
routine tasks. It also helps in automating and quickly create
models for data analysis.
➢ Image recognition, text generation, and many other use-cases are
finding applications in the real world.
Features of Machine Learning
➢ Automation: Nowadays in your Gmail account, there is a spam folder that
contains all the spam emails. You might be wondering how does Gmail know
that all these emails are spam?
➢ Improved customer experience: For any business, one of the most crucial
ways to drive engagement, promote brand loyalty and establish long-lasting
customer relationships is by providing a customized experience and providing
better services.
➢ Automated data visualization: With the help of user-friendly automated data
visualization platforms such as AutoViz, businesses can obtain a wealth of new
insights in an effort to increase productivity in their processes.
➢ Business intelligence.
Transpose
What is Machine Learning
Machine Learning is said as a subset of artificial intelligence that is mainly
concerned with the development of algorithms which allow a computer to learn
from the data and past experiences on their own. The term machine learning was
first introduced by Arthur Samuel in 1959.

A machine has the ability to learn if it can improve its performance by gaining
more data.

How does Machine Learning work

A Machine Learning system learns from historical data, builds the prediction
models, and whenever it receives new data, predicts the output for it.
Features of Machine Learning:
➢ Machine learning uses data to detect various patterns in a given
dataset.
➢ It can learn from past data and improve automatically.
➢ It is a data-driven technology.
➢ Machine learning is much similar to data mining as it also deals
with the huge amount of the data.
Need for Machine Learning:
The importance of machine learning can be easily understood by its uses cases,
Currently, machine learning is used in self-driving cars, cyber fraud
detection, face recognition, and friend suggestion by Facebook, etc. Various top
companies such as Netflix and Amazon have build machine learning models that
are using a vast amount of data to analyze the user interest and recommend product
accordingly.
Types of Machine Learning
Machine learning has been broadly categorized into three
categories
➢ Supervised Learning
➢ Unsupervised Learning
➢ Reinforcement Learning
What is Supervised Learning?
➢ Supervised learning is a type of machine learning method in
which we provide sample labelled data to the machine learning
system in order to train it, and on that basis, it predicts the output.
➢ The goal of supervised learning is to map input data with the
output data. The supervised learning is based on supervision, and
it is the same as when a student learns things in the supervision of
the teacher. The example of supervised learning is spam filtering.
➢ Supervised learning can be grouped further in two categories of
algorithms:
➢ Classification
➢ Regression
What is Supervised Learning?
Let us start with an easy example, say you are teaching a kid to
differentiate dogs from cats. How would you do it?
What is Supervised Learning?

Number of rooms Price

1 $100
3 $300
5 $500

Supervised learning model has a set of input variables (x), and an

output variable (y). An algorithm identifies the mapping function
between the input and output variables. The relationship is y = f(x).
Regression problems – Used to predict future values and the model
is trained with the historical data. E.g., Predicting the future price of
a house.

Classification problems – Various labels train the algorithm to

identify items within a specific category. E.g., Dog or cat( as
mentioned in the above example), Apple or an orange, Beer or wine
or water.
What is Unsupervised Learning?
This approach is the one where we have no target variables, and we
have only the input variable (features) at hand. The algorithm learns
by itself and discovers an impressive structure in the data.
The goal is to decipher the underlying distribution in the data to gain
more knowledge about the data.
We can group the unsupervised learning problems as:
Clustering: This means bundling the input variables with the same
characteristics together. E.g., grouping users based on search history
Association: Here, we discover the rules that govern meaningful
associations among the data set. E.g., People who watch ‘X’ will also
watch ‘Y’.
What is Unsupervised Learning?
What is Reinforcement Learning?
In this approach, machine learning models are trained to make a
series of decisions based on the rewards and feedback they receive
for their actions. The machine learns to achieve a goal in complex
and uncertain situations and is rewarded each time it achieves it
during the learning period.
History of Machine Learning
Machine learning is making our day to day life easy from self-
driving cars to Amazon virtual assistant "Alexa".

➢ The early history of Machine Learning (Pre-1940):

➢ The era of stored program computers:
➢ Computer machinery and intelligence:
➢ Machine intelligence in Games:
➢ The first "AI" winter:
➢ Machine Learning from theory to reality
➢ Machine Learning at 21st century
➢ Machine Learning at present:
Prerequisites
Before learning machine learning, you must have the basic
knowledge of following so that you can easily understand the
concepts of machine learning:

➢ Fundamental knowledge of probability and linear algebra.

➢ The ability to code in any computer language, especially in
Python language.
➢ Knowledge of Calculus, especially derivatives of single variable
and multivariate functions.
Applications of Machine learning
1. Image Recognition:
Image recognition is one of the most common applications of
machine learning. It is used to identify objects, persons, places,
digital images, etc. The popular use case of image recognition and
face detection is, Automatic friend tagging suggestion:
2. Speech Recognition
3. Traffic prediction:

4. Product recommendations:
5. Self-driving cars:

6. Email Spam and Malware Filtering:

7. Virtual Personal Assistant:
8. Online Fraud Detection:
9. Stock Market trading:
10. Medical Diagnosis:

11. Automatic Language Translation:

Machine learning Life cycle
➢ Gathering Data
➢ Data preparation
➢ Data Wrangling
➢ Analyse Data
➢ Train the model
➢ Test the model
➢ Deployment
Gathering Data
Here are the top six data collection methods:
Interviews
Questionnaires and surveys
Observations
Documents and records
Focus groups
Oral histories
Data preparation
Good data preparation allows for efficient analysis, limits errors and
inaccuracies that can occur to data during processing, and makes all
processed data more accessible to users. It’s also gotten easier with
new tools that enable any user to cleanse and qualify data on their
own.

Data preparation is the process of cleaning and transforming raw

data prior to processing and analysis. It is an important step prior to
processing and often involves reformatting data, making corrections
to data and the combining of data sets to enrich data.
Data Wrangling
Data Analysis
Selection of analytical techniques
Building models
Review the result

The aim of this step is to build a machine learning model to analyze

the data using various analytical techniques and review the outcome.
It starts with the determination of the type of the problems, where we
select the machine learning techniques such
as Classification, Regression, Cluster analysis, Association, etc.
then build the model using prepared data, and evaluate the model.
Train Model
What is training data?
In machine learning, training data is the data you use to train a
machine learning algorithm or model. Training data requires some
human involvement to analyze or process the data for machine
learning use. How people are involved depends on the type of
machine learning algorithms you are using and the type of problem
that they are intended to solve.
Test the model
➢ The test set is a set of observations used to evaluate the
performance of the model using some performance metric. It is
important that no observations from the training set are included
in the test set.
➢ A program that generalizes well will be able to effectively
perform a task with new data.
➢ Memorizing the training set is called over-fitting. A program that
memorizes its observations may not perform its task well, as it
could memorize relations and structures that are noise or
coincidence.
➢ Regularization may be applied to many models to reduce over-
fitting.
Performance Measures − Bias
and Variance
There are two fundamental causes of prediction error for a model -
bias and variance.
Ideally, a model will have both low bias and variance, but efforts to
decrease one will frequently increase the other. This is known as
the bias-variance trade-off.
Accuracy, Precision and Recall
ACC = (TP + TN)/(TP + TN + FP + FN)
Deployment
The last step of machine learning life cycle is deployment, where we
deploy the model in the real-world system.

Difference between Artificial intelligence and Machine learning

Artificial intelligence and machine learning are the part of computer

science that are correlated with each other. These two technologies
are the most trending technologies which are used for creating
intelligent systems.
 How to get datasets for Machine
Learning
A dataset is a collection of data in which data is arranged in some
order. A dataset can contain any data from a series of an array to a
database table. Below table shows an example of the dataset.

Country Age Salary Purchased

A 38 48000 No

B 43 45000 Yes

C 30 54000 No

D 48 65000 No

E 40 Yes

F 35 58000 Yes
Need of Dataset
During the development of the ML project, the developers
completely rely on the datasets. In building ML applications, datasets
are divided into two parts:
Training dataset:
Test Dataset
Below is the list of datasets which are freely available for the public
to work on it:
1. Kaggle Datasets
2. UCI Machine Learning Repository
3. Datasets via AWS
4. Google's Dataset Search Engine
 Data Pre-processing in Machine
learning
Data pre-processing is a process of preparing the raw data and
making it suitable for a machine learning model. It is the first and
crucial step while creating a machine learning model.
It involves below steps:
Getting the dataset
Importing libraries
Importing datasets
Finding Missing Data
Encoding Categorical Data
Splitting dataset into training and test set
Feature scaling
1) Get the Dataset
To create a machine learning model, the first thing we required is a
dataset as a machine learning model completely works on data. The
collected data for a particular problem in a proper format is known as
the dataset.
2) Importing Libraries
In order to perform data pre-processing using Python, we need to
import some predefined Python libraries. These libraries are used to
perform some specific jobs. There are three specific libraries that we
will use for data pre-processing, which are:
3) Importing the Datasets
Now we need to import the datasets which we have collected for our
machine learning project. But before importing a dataset, we need to
set the current directory as a working directory.
Extracting dependent and
independent variables:
Extracting independent variable:
To extract an independent variable, we will use iloc[ ] method of Pandas
library. It is used to extract the required rows and columns from the dataset.
x= data_set.iloc[:,:-1].values
In the above code, the first colon(:) is used to take all the rows, and the
second colon(:) is for all the columns. Here we have used :-1, because we
don't want to take the last column as it contains the dependent variable. So
by doing this, we will get the matrix of features.
By executing the above code, we will get output as:
[['India' 38.0 68000.0] ['France' 43.0 45000.0] ['Germany' 30.0 54000.0]
['France' 48.0 65000.0] ['Germany' 40.0 nan] ['India' 35.0 58000.0]
['Germany' nan 53000.0] ['France' 49.0 79000.0] ['India' 50.0 88000.0]
['France' 37.0 77000.0]]
Extracting dependent variable:

To extract dependent variables, again, we will use Pandas .iloc[]

method.
y= data_set.iloc[:,3].values
Here we have taken all the rows with the last column only. It will
give the array of dependent variables.
By executing the above code, we will get output as:
Output:
array(['No', 'Yes', 'No', 'No', 'Yes', 'Yes', 'No', 'Yes', 'No', 'Yes'],
dtype=object)
4) Handling Missing data:
The next step of data preprocessing is to handle missing data in the
datasets. If our dataset contains some missing data, then it may
create a huge problem for our machine learning model. Hence it is
necessary to handle missing values present in the dataset.
5) Encoding Categorical data:
Categorical data is data which has some categories such as, in our
dataset; there are two categorical variable, Country, and Purchased.
6) Splitting the Dataset into the Training set and Test set:
In machine learning data preprocessing, we divide our dataset into a
training set and test set. This is one of the crucial steps of data
preprocessing as by doing this, we can enhance the performance of
our machine learning model.
7) Feature Scaling:
Feature scaling is the final step of data preprocessing in machine
learning. It is a technique to standardize the independent variables of
the dataset in a specific range. In feature scaling, we put our
variables in the same range and in the same scale so that no any
variable dominate the other variable.
Supervised Machine Learning
Supervised learning is the types of machine learning in which
machines are trained using well "labelled" training data, and on basis
of that data, machines predict the output. The labelled data means
some input data is already tagged with the correct output.
Steps Involved in Supervised
Learning:
➢ Determine the type of training dataset.
➢ Collect/Gather the labelled training data.
➢ Split the training dataset into training dataset, test dataset, and
validation dataset.
➢ Determine the input features of the training dataset.
➢ Determine the suitable algorithm for the model.
➢ Execute the algorithm on the training dataset.
➢ Evaluate the accuracy of the model by providing the test set.
Regression
Regression algorithms are used if there is a relationship between the
input variable and the output variable. It is used for the prediction of
continuous variables, such as Weather forecasting, Market Trends,
etc. Below are some popular Regression algorithms which come
under supervised learning:
➢ Linear Regression
➢ Regression Trees
➢ Non-Linear Regression
➢ Bayesian Linear Regression
➢ Polynomial Regression
Classification
Classification algorithms are used when the output variable is
categorical, which means there are two classes such as Yes-No,
Male-Female, True-false, etc.

➢ Spam Filtering,
➢ Random Forest
➢ Decision Trees
➢ Logistic Regression
➢ Support vector Machines
Unsupervised Machine Learning
As the name suggests, unsupervised learning is a machine learning
technique in which models are not supervised using training dataset.
Instead, models itself find the hidden patterns and insights from the
given data. It can be compared to learning which takes place in the
human brain while learning new things. It can be defined as:

Unsupervised learning is a type of machine learning in which

models are trained using unlabelled dataset and are allowed to act
on that data without any supervision.
Why use Unsupervised Learning?
Clustering: Clustering is a method of grouping the objects into
clusters such that objects with most similarities remains into a group
and has less or no similarities with the objects of another group.

Association: An association rule is an unsupervised learning method

which is used for finding the relationships between variables in the
large database. It determines the set of items that occurs together in
the dataset.
Popular unsupervised learning
algorithms
K-means clustering
KNN (k-nearest neighbours)
Hierarchal clustering
Anomaly detection
Neural Networks
Principle Component Analysis
Independent Component Analysis
Apriori algorithm
Singular value decomposition
Supervised Learning Unsupervised Learning

Supervised learning algorithms are trained using labelled data. Unsupervised learning algorithms are trained using unlabelled data.

Supervised learning model takes direct feedback to check if it is predicting correct output or not. Unsupervised learning model does not take any feedback.

Supervised learning model predicts the output. Unsupervised learning model finds the hidden patterns in data.

In supervised learning, input data is provided to the model along with the output. In unsupervised learning, only input data is provided to the model.

The goal of supervised learning is to train the model so that it can predict the output when it is given new data. The goal of unsupervised learning is to find the hidden patterns and useful insights from the unknown dataset.

Supervised learning needs supervision to train the model. Unsupervised learning does not need any supervision to train the model.

Supervised learning can be categorized in Classification and Regression problems. Unsupervised Learning can be classified in Clustering and Associations problems.

Supervised learning can be used for those cases where we know the input as well as corresponding outputs. Unsupervised learning can be used for those cases where we have only input data and no corresponding output data.

Supervised learning model produces an accurate result. Unsupervised learning model may give less accurate result as compared to supervised learning.

Supervised learning is not close to true Artificial intelligence as in this, we first train the model for each data, and then only Unsupervised learning is more close to the true Artificial Intelligence as it learns similarly as a child learns daily routine things by his

it can predict the correct output. experiences.

It includes various algorithms such as Linear Regression, Logistic Regression, Support Vector Machine, Multi-class It includes various algorithms such as Clustering, KNN, and Apriori algorithm.

Classification, Decision tree, Bayesian Logic, etc.

What is the difference between data
mining and predictive analytics?
Data mining is basically the process of analysing large sets of data to
find patterns, relationships, and trends that otherwise might be
missed through more traditional analysis methods. It is used to
uncover shared similarities or groupings in web data that help gain
insights for business decisions.

Predictive analytics uses mathematical algorithms and machine

learning to identify how likelihood something will occur in the future
based on patterns of previous data. The goal of predictive analytics is
to use past knowledge of what has happened to provide a better idea
of what to expect in the future.
MODULE 4:
Types of Machine Learning

➢ Supervised Learning,
➢ Unsupervised Learning,
➢ Reinforcement Learning.
➢ Machine Learning Lifecycle.
Supervised Learning
Supervised learning as the name indicates the presence of a
supervisor as a teacher. Basically supervised learning is a learning in
which we teach or train the machine using data which is well
labelled that means some data is already tagged with the correct
answer.
After that, the machine is provided with a new set of examples(data)
so that supervised learning algorithm analyses the training data(set of
training examples) and produces a correct outcome from labelled
data.
Applications of Supervised
Learning
Sentiment Analysis
Recommendations
Spam Filtration
BioInformatics
Speech Recognition
Object-Recognition for Vision
How Supervised Learning Works
For example, you want to train a machine to help you predict how
long it will take you to drive home from your workplace. Here, you
start by creating a set of labelled data. This data includes:

Weather conditions
Time of the day
Holidays
Types of Supervised Machine Learning Algorithms
Regression
Logistic Regression
Classification
Naïve Bayes Classifiers
Decision Trees
Support Vector Machine
Challenges in Supervised
machine learning
➢ Irrelevant input feature present training data could give inaccurate
results.
➢ Data preparation and pre-processing is always a challenge.
➢ Accuracy suffers when impossible, unlikely, and incomplete
values have been inputted as training data.
➢ If the concerned expert is not available, then the other approach is
"brute-force." It means you need to think that the right features
(input variables) to train the machine on. It could be inaccurate.
Advantages of Supervised Learning

Disadvantages of Supervised Learning

Best practices for Supervised
Learning
➢ Before doing anything else, you need to decide what kind of data is to be used
as a training set.
➢ You need to decide the structure of the learned function and learning algorithm.
➢ Gather corresponding outputs either from human experts or from measurements

Summary
➢ In Supervised learning, you train the machine using data which is well
"labelled."
➢ You want to train a machine which helps you predict how long it will take you
to drive home from your workplace is an example of supervised learning
➢ Regression and Classification are two types of supervised machine learning
techniques.
➢ Supervised learning is a simpler method while Unsupervised learning is a
complex method.
In order to solve a given problem of supervised learning,
one has to perform the following steps:

➢ Determine the type of training examples.

➢ Gather a training set.
➢ Determine the input feature representation of the learned function.
➢ Determine the structure of the learned function and corresponding
learning algorithm.
➢ Complete the design.
➢ Evaluate the accuracy of the learned function.
 Regression Analysis in Machine
learning
Regression analysis is a statistical method to model the relationship between a dependent
(target) and independent (predictor) variables with one or more independent variables. More
specifically, Regression analysis helps us to understand how the value of the dependent
variable is changing corresponding to an independent variable when other independent
variables are held fixed. It predicts continuous/real values such as temperature, age,
salary, price, etc.

Now, the company wants to do the advertisement of

$200 in the year 2019 and wants to know the
prediction about the sales for this year. So to
solve such type of prediction problems in machine
learning, we need regression analysis.
Some examples of regression can be as

Prediction of rain using temperature and other factors

Determining Market trends
Prediction of road accidents due to rash driving.

Terminologies Related to the Regression Analysis:

➢ Dependent Variable
➢ Independent Variable
➢ Outliers
➢ Underfitting and Overfitting
 Why do we use Regression
Analysis?
➢ Regression estimates the relationship between the target and the independent
variable.
➢ It is used to find the trends in data.
➢ It helps to predict real/continuous values.
➢ By performing the regression, we can confidently determine the most
important factor, the least important factor, and how each factor is
affecting the other factors.

➢ Types of Regression:
Linear Regression:
➢ Linear regression is a statistical regression method which is used for
predictive analysis.
➢ It is one of the very simple and easy algorithms which works on
regression and shows the relationship between the continuous variables.
➢ It is used for solving the regression problem in machine learning.
➢ Linear regression shows the linear relationship between the independent
variable (X-axis) and the dependent variable (Y-axis), hence called
linear regression.
➢ If there is only one input variable (x), then such linear regression is
called simple linear regression. And if there is more than one input
variable, then such linear regression is called multiple linear
regression.
➢ The relationship between variables in the linear regression model can be
explained using the below image. Here we are predicting the salary of
an employee on the basis of the year of experience.
Y= aX+b
Here, Y = dependent variables (target variables),
X= Independent variables (predictor variables),
a and b are the linear coefficients
Some popular applications of linear regression are:
Analyzing trends and sales estimates
Salary forecasting
Real estate prediction
Arriving at ETAs in traffic.
Logistic Regression:
➢ Used to solve the classification problems.
➢ Logistic regression algorithm works with the categorical variable
such as 0 or 1, Yes or No, True or False, Spam or not spam, etc.
➢ It is a predictive analysis algorithm which works on the concept of
probability.
➢ Logistic regression uses sigmoid function or logistic function
which is a complex cost function. This sigmoid function is used to
model the data in logistic regression. The function can be
represented as:

f(x)= Output between the 0 and 1 value.

x= input to the function
e= base of natural logarithm.
There are three types of logistic regression:
➢ Binary(0/1, pass/fail)
➢ Multi(cats, dogs, lions)
➢ Ordinal(low, medium, high)
Polynomial Regression:
It is similar to multiple linear regression, but it fits a non-linear curve between the
value of x and corresponding conditional values of y.
Suppose there is a dataset which consists of datapoints which are present in a non-
linear fashion, so for such case, linear regression will not best fit to those
datapoints.
➢ In Polynomial regression, the original features are
transformed into polynomial features of given degree and
then modeled using a linear model. Which means the datapoints
are best fitted using a polynomial line.
➢ The equation for polynomial regression also derived from linear
regression equation that means Linear regression equation Y= b0+
b1x, is transformed into Polynomial regression equation Y=
b0+b1x+ b2x2+ b3x3+.....+ bnxn.
➢ Here Y is the predicted/target output, b0, b1,... bn are the
regression coefficients. x is our independent/input variable.
➢ The model is still linear as the coefficients are still linear with
quadratic.
Support Vector Regression:
It can be used for regression as well as classification problems.
Support Vector Regression is a regression algorithm which works for
continuous variables.
Kernel: It is a function used to map a lower-dimensional data into
higher dimensional data.
Hyperplane: In general SVM, it is a separation line between two
classes, but in SVR, it is a line which helps to predict the continuous
variables and cover most of the datapoints.
Boundary line: Boundary lines are the two lines apart from
hyperplane, which creates a margin for datapoints.
Support vectors: Support vectors are the datapoints which are
nearest to the hyperplane and opposite class.
Here, the blue line is called hyperplane, and the other two lines are known
as boundary lines.
Decision Tree Regression
➢ Decision Tree is a supervised learning algorithm which can be
used for solving both classification and regression problems.
➢ Decision Tree regression builds a tree-like structure in which each
internal node represents the "test" for an attribute, each branch
represent the result of the test, and each leaf node represents the
final decision or result.
➢ A decision tree is constructed starting from the root node/parent
node (dataset), which splits into left and right child nodes (subsets
of dataset).
 Example
Let's illustrate this with help of an example. Let's assume we want to
play badminton on a particular day — say Saturday — how will you
decide whether to play or not. Let's say you go out and check if it's
hot or cold, check the speed of the wind and humidity, how the
weather is, i.e. is it sunny, cloudy, or rainy. You take all these factors
into account to decide if you
Weathe Temper Humidi
want to play or not. Day
r ature ty
Wind Play?

1 Sunny Hot High Weak No

2 Cloudy Hot High Weak Yes
3 Sunny Mild Normal Strong Yes
4 Cloudy Mild High Strong Yes
5 Rainy Mild High Strong No
6 Rainy Cool Normal Strong No
7 Rainy Mild High Weak Yes
8 Sunny Hot High Strong No
9 Cloudy Hot Normal Weak Yes
10 Rainy Mild High Strong No
How To Implement Classification In
Machine Learning?
Classification is a process of categorizing a given set of data into classes, It can be
performed on both structured or unstructured data. The process starts with
predicting the class of given data points. The classes are often referred to as target,
label or categories.

Classification Terminologies In Machine Learning

Classifier
Classification Model
Feature
Binary Classification
Multi-Class Classification
Multi-label Classification
Initialize
Train the Classifier
Predict the Target
Evaluate
K-Nearest Neighbour(KNN)
➢ K-Nearest Neighbour is one of the simplest Machine Learning algorithms
based on Supervised Learning technique.
➢ K-NN algorithm assumes the similarity between the new case/data and
available cases and put the new case into the category that is most similar to the
available categories.
➢ K-NN algorithm stores all the available data and classifies a new data point
based on the similarity. This means when new data appears then it can be easily
classified into a well suite category by using K- NN algorithm.
➢ K-NN algorithm can be used for Regression as well as for Classification but
mostly it is used for the Classification problems.
Why do we need a K-NN
How does K-NN work?
Step-1: Select the number K of the neighbors
Step-2: Calculate the Euclidean distance of K number of neighbors
Step-3: Take the K nearest neighbors as per the calculated Euclidean
distance.
Step-4: Among these k neighbors, count the number of the data
points in each category.
Step-5: Assign the new data points to that category for which the
number of the neighbor is maximum.
Transpose
Transpose
Transpose
Transpose
Transpose
Transpose
Transpose