Supervised Machine Learning

How Supervised Learning Works?

• Supervised learning is the types of machine learning in which machines are • In supervised learning, models are trained using labelled dataset, where the model learns
trained using well "labelled" training data, and on basis of that data, machines about each type of data. Once the training process is completed, the model is tested on the
predict the output. The labelled data means some input data is already tagged basis of test data (a subset of the training set), and then it predicts the output.
with the correct output. • The working of Supervised learning can be easily understood by the below example and
• In supervised learning, the training data provided to the machines work as the diagram:
supervisor that teaches the machines to predict the output correctly. It applies
the same concept as a student learns in the supervision of the teacher.
• Supervised learning is a process of providing input data as well as correct
output data to the machine learning model. The aim of a supervised learning
algorithm is to find a mapping function to map the input variable(x) with the
output variable(y).
• In the real-world, supervised learning can be used for Risk Assessment,
Image classification, Fraud Detection, spam filtering, etc.

Suppose we have a dataset of different types of shapes which includes square,

rectangle, triangle, and Polygon. Now the first step is that we need to train the For example, a labeled dataset of images of Elephant, Camel and Cow would have
model for each shape. each image tagged with either “Elephant” , “Camel”or “Cow.”

•If the given shape has four sides, and all the sides are equal, then it will be
labelled as a Square.
•If the given shape has three sides, then it will be labelled as a triangle.
•If the given shape has six equal sides then it will be labelled as hexagon.

Now, after training, we test our model using the test set, and the task of the model
is to identify the shape.
The machine is already trained on all types of shapes, and when it finds a new
shape, it classifies the shape on the bases of a number of sides, and predicts the
output.
 For instance, suppose you are given a basket filled with different kinds of fruits. Now Types of supervised Machine learning Algorithms:
the first step is to train the machine with all the different fruits one by one like this:
Supervised learning can be further divided into two types of problems:
•If the shape of the object is rounded and has a depression at the top, is red in color,
then it will be labeled as –Apple.
•If the shape of the object is a long curving cylinder having Green-Yellow color, then
it will be labeled as –Banana.

Now suppose after training the data, you have given a new separate fruit, say Banana
from the basket, and asked to identify it.

Since the machine has already learned the things from previous data and this time
has to use it wisely. It will first classify the fruit with its shape and color and would
confirm the fruit name as BANANA and put it in the Banana category. Thus the
machine learns the things from training data(basket containing fruits) and then
applies the knowledge to test data(new fruit).

1. Regression Applications of Supervised learning

Regression algorithms are used if there is a relationship between the input variable
and the output variable. It is used for the prediction of continuous variables, such as Supervised learning can be used to solve a wide variety of problems, including:
Weather forecasting, Market Trends, etc. Below are some popular Regression
algorithms which come under supervised learning: •Spam filtering: Supervised learning algorithms can be trained to identify and classify
spam emails based on their content, helping users avoid unwanted messages.
•Linear Regression •Image classification: Supervised learning can automatically classify images into
•Regression Trees different categories, such as animals, objects, or scenes, facilitating tasks like image
•Non-Linear Regression search, content moderation, and image-based product recommendations.
•Bayesian Linear Regression •Medical diagnosis: Supervised learning can assist in medical diagnosis by analyzing
•Polynomial Regression patient data, such as medical images, test results, and patient history, to identify patterns
that suggest specific diseases or conditions.
2. Classification •Fraud detection: Supervised learning models can analyze financial transactions and
Classification algorithms are used when the output variable is categorical, which identify patterns that indicate fraudulent activity, helping financial institutions prevent
means there are two classes such as Yes-No, Male-Female, True-false, etc. fraud and protect their customers.
•Natural language processing (NLP): Supervised learning plays a crucial role in NLP
Spam Filtering, tasks, including sentiment analysis, machine translation, and text summarization,
•Random Forest enabling machines to understand and process human language effectively.
•Decision Trees
•Logistic Regression
•Support vector Machines
Backpropagation
Backpropagation is one of the important concepts of a neural network. Our task is How Backpropagation Algorithm Works
to classify our data best. For this, we have to update the weights of parameter and
bias, but how can we do that in a deep neural network? In the linear regression
model, we use gradient descent to optimize the parameter. Similarly here we also
use gradient descent algorithm using Backpropagation.
For a single training example, Backpropagation algorithm calculates the gradient of
the error function. Backpropagation can be written as a function of the neural
network.
Features of Backpropagation:

1.it is the gradient descent method as used in the case of simple perceptron network
with the differentiable unit.

1.it is different from other networks in respect to the process by which the weights are
calculated during the learning period of the network.
2.training is done in the three stages :

1. the feed-forward of input training pattern

2. the calculation and backpropagation of the error
3. updation of the weight

Neural networks use supervised learning to generate output vectors from input
vectors that the network operates on. It Compares generated output to the desired
output and generates an error report if the result does not match the generated
output vector. Then it adjusts the weights according to the bug report to get your Parameters :
desired output. •x = inputs training vector x=(x1,x2,…………xn).
•t = target vector t=(t1,t2……………tn).
•δk = error at output unit.
Backpropagation Algorithm: •δj = error at hidden layer.
Step 1: Inputs X, arrive through the preconnected path. •α = learning rate.
Step 2: The input is modeled using true weights W. Weights are usually chosen •V0j = bias of hidden unit j.
randomly.
Step 3: Calculate the output of each neuron from the input layer to the hidden
layer to the output layer.
Step 4: Calculate the error in the outputs
Backpropagation Error= Actual Output – Desired Output

Step 5: From the output layer, go back to the hidden layer to adjust the weights to
reduce the error.
Step 6: Repeat the process until the desired output is achieved.
Training Algorithm :
Step 1: Initialize weight to small random values. Backpropagation Error :
Step 2: While the stopping condition is to be false do step 3 to 10. Step 6: Each output unit yk (k=1 to n) receives a target pattern
Step 3: For each training pair do step 4 to 9 (Feed-Forward). corresponding to an input pattern then error is calculated as:
Step 4: Each input unit receives the signal unit and transmits the signal xi signal to all δk = ( tk – yk ) + yink
the units. Step 7: Each hidden unit Zj (j=1 to a) sums its input from all units in the
Step 5 : Each hidden unit Zj (z=1 to a) sums its weighted input signal to calculate its layer above
net input δinj = Σ δj wjk
zinj = v0j + Σxivij ( i=1 to n) The error information term is calculated as :
Applying activation function zj = f(zinj) and sends this signals to all units in the layer δj = δinj + zinj
about i.e output units
For each output l=unit yk = (k=1 to m) sums its weighted input signals.
Updation of weight and bias :
yink = w0k + Σ ziwjk (j=1 to a)

and applies its activation function to calculate the output signals.

yk = f(yink)

Step 8: Each output unit yk (k=1 to m) updates its bias and weight (j=1 to Input values
a). The weight correction term is given by : X1=0.05
Δ wjk = α δk zj X2=0.10
and the bias correction term is given by Δwk = α δk. Initial weight
therefore wjk(new) = wjk(old) + Δ wjk W1=0.15 w5=0.40
w0k(new) = wok(old) + Δ wok W2=0.20 w6=0.45
for each hidden unit zj (j=1 to a) update its bias and weights (i=0 W3=0.25 w7=0.50
to n) the weight connection term W4=0.30 w
Δ vij = α δj xi 8=0.55
and the bias connection on term Bias Values
Δ v0j = α δj b1=0.35 b2=0.60
Therefore vij(new) = vij(old) + Δvij Target Values
v0j(new) = v0j(old) + Δv0j T1=0.01
T2=0.99

Step 9: Test the stopping condition. The stopping condition can be the
minimization of error, number of epochs.
 Unsupervised Learning
Advantages:
As the name suggests, unsupervised learning is a machine learning technique in which
models are not supervised using training dataset. Instead, models itself find the hidden
•It is simple, fast, and easy to program.
patterns and insights from the given data. It can be compared to learning which takes place
•Only numbers of the input are tuned, not any other parameter.
in the human brain while learning new things.
•It is Flexible and efficient.
•No need for users to learn any special functions.
The goal of unsupervised learning is to find the underlying structure of dataset, group
Disadvantages:
that data according to similarities, and represent that dataset in a compressed format.
•It is sensitive to noisy data and irregularities. Noisy data can lead to inaccurate results.
Example: Suppose the unsupervised learning algorithm is given an input dataset containing
•Performance is highly dependent on input data.
images of different types of cats and dogs. The algorithm is never trained upon the given
•Spending too much time training.
dataset, which means it does not have any idea about the features of the dataset. The task
•The matrix-based approach is preferred over a mini-batch.
of the unsupervised learning algorithm is to identify the image features on their own.
Unsupervised learning algorithm will perform this task by clustering the image dataset into
the groups according to similarities between images.

Why use Unsupervised Learning? How does unsupervised learning work?

•Unsupervised learning is helpful for finding useful insights from the data. Here, we have taken an unlabeled input data, which means it is not categorized and
corresponding outputs are also not given. Now, this unlabeled input data is fed to the
•Unsupervised learning is much similar as a human learns to think by their own machine learning model in order to train it. Firstly, it will interpret the raw data to find the
experiences, which makes it closer to the real AI. hidden patterns from the data and then will apply suitable algorithms such as k-means
clustering, Decision tree, etc.
•Unsupervised learning works on unlabeled and uncategorized data which make
unsupervised learning more important. Once it applies the suitable algorithm, the algorithm divides the data objects into
groups according to the similarities and difference between the objects.
•In real-world, we do not always have input data with the corresponding output so to solve
such cases, we need unsupervised learning.
Types of Unsupervised Learning Algorithm:
Advantages of Unsupervised learning

•No labeled data required: Unlike supervised learning, unsupervised learning does not
require labeled data, which can be expensive and time-consuming to collect.

•Can uncover hidden patterns: Unsupervised learning algorithms can identify patterns
and relationships in data that may not be obvious to humans.

•Can be used for a variety of tasks: Unsupervised learning can be used for a variety of
•Clustering: Clustering is a method of grouping the objects into clusters such that
tasks, such as clustering, dimensionality reduction, and anomaly detection.
objects with most similarities remains into a group and has less or no similarities
with the objects of another group. Cluster analysis finds the commonalities
•Can be used to explore new data: Unsupervised learning can be used to explore new
between the data objects and categorizes them as per the presence and absence
data and gain insights that may not be possible with other methods.
of those commonalities.

•Association: An association rule is an unsupervised learning method which is

used for finding the relationships between variables in the large database. It
determines the set of items that occurs together in the dataset. Association rule
makes marketing strategy more effective. Such as people who buy X item
(suppose a bread) are also tend to purchase Y (Butter/Jam) item. A typical
example of Association rule is Market Basket Analysis.

Disadvantages of Unsupervised learning Applications of Unsupervised learning

•Difficult to evaluate: It can be difficult to evaluate the performance of •Customer segmentation: Unsupervised learning can be used to segment customers
unsupervised learning algorithms, as there are no predefined labels or into groups based on their demographics, behavior, or preferences. This can help
categories against which to compare results. businesses to better understand their customers and target them with more relevant
marketing campaigns.
•Can be difficult to interpret: It can be difficult to understand the decision- •Fraud detection: Unsupervised learning can be used to detect fraud in financial data
making process of unsupervised learning models. by identifying transactions that deviate from the expected patterns. This can help to
prevent fraud by flagging these transactions for further investigation.
•Can be sensitive to the quality of the data: Unsupervised learning •Recommendation systems: Unsupervised learning can be used to recommend items
algorithms can be sensitive to the quality of the input data. Noisy or to users based on their past behavior or preferences. For example, a recommendation
incomplete data can lead to misleading or inaccurate results. system might use unsupervised learning to identify users who have similar taste in
movies, and then recommend movies that those users have enjoyed.
•Can be computationally expensive: Some unsupervised learning •Natural language processing (NLP): Unsupervised learning is used in a variety of
algorithms, particularly those dealing with high-dimensional data or large NLP tasks, including topic modeling, document clustering, and part-of-speech
datasets, can be computationally expensive tagging.
•Image analysis: Unsupervised learning is used in a variety of image analysis
tasks, including image segmentation, object detection, and image pattern recognition.
Hebbian Learning • Hebbian Learning Rule, also known as Hebb Learning Rule, was proposed by Donald O Hebb. It
is one of the first and also easiest learning rules in the neural network.
This rule, one of the oldest and simplest, was introduced by Donald Hebb in • It is used for pattern classification. It is a single layer neural network, i.e. it has one input layer
his book The Organization of Behavior in 1949. It is a kind of feed-forward, and one output layer. The input layer can have many units, say n.
unsupervised learning. • The output layer only has one unit. Hebbian rule works by updating the weights between
neurons in the neural network for each training sample.
Basic Concept − This rule is based on a proposal given by Hebb, who wrote −
“When an axon of cell A is near enough to excite a cell B and repeatedly or
Hebbian Learning Rule Algorithm :
persistently takes part in firing it, some growth process or metabolic change
takes place in one or both cells such that A’s efficiency, as one of the cells
1.Set all weights to zero, wi = 0 for i=1 to n, and bias to zero.
firing B, is increased.”
2.For each input vector, S(input vector) : t(target output pair), repeat steps 3-5.
From the above postulate, we can conclude that the connections between two
3.Set activations for input units with the input vector Xi = Si for i = 1 to n.
neurons might be strengthened if the neurons fire at the same time and
4.Set the corresponding output value to the output neuron, i.e. y = t.
might weaken if they fire at different times.
5.Update weight and bias by applying Hebb rule for all i = 1 to n:

Implementing AND Gate :

Competitive Learning
• This network is just like a single layer feed-forward network having feedback
• Artificial neural networks often utilize competitive learning models to classify input without the connection between the outputs. The connections between the outputs are
use of labeled data. inhibitory type, which is shown by dotted lines, which means the competitors
• The process begins with an input vector (often a data set). This input is then presented to a never support themselves.
network of artificial neurons, each of which has its own set of weights, which act like filters. Each
neuron computes a score based on its weight and the input vector, typically through a dot
product operation (a way of multiplying the input information with the filter and adding the
results together).
• After the computation, the neuron that has the highest score (the "winner") is updated, usually by
shifting its weights closer to the input vector. This process is often referred to as the "Winner-
Takes-All" strategy. Over time, neurons become specialized as they get updated toward input
vectors they can best match.
• Here all the output nodes try to compete with each other to represent the input pattern and the
winner is declared according to the node having the most outputs and is given the output 1
while the rest are given 0.
• There are a set of neurons with arbitrarily distributed weights and the activation function is
applied to a subset of neurons. Only one neuron is active at a time. Only the winner has updated
weights, the rest remain unchanged.
Basic Concept of Competitive Learning Rule
• As said earlier, there would be competition among the output nodes so the main
concept is - during training, the output unit that has the highest activation to a given
input pattern, will be declared the winner. This rule is also called Winner-takes-all
because only the winning neuron is updated and the rest of the neurons are left
unchanged.

• To illustrate how one can use competitive learning, imagine an eCommerce business
wants to segment its customer base for targeted marketing, but they have no prior
labels or segmentation. By feeding customer data (purchase history, browsing
pattern, demographics, etc.) to a competitive learning model, they could
automatically find distinct clusters (like high spenders, frequent buyers, discount
lovers) and tailor marketing strategies accordingly.

The Competitive Learning Process: A Step-by-Step Example

For this simple illustration, let's assume we have a dataset composed of 1-dimensional input
vectors ranging from 1 to 10 and a competitive learning network with two neurons.
Step 1: Initialization

We start by initializing the weights of the two neurons to random values. Let's assume:
•Neuron 1 weight: 2
•Neuron 2 weight: 8

Step 2: Presenting the input vector

Now, we present an input vector to the network. Let's say our input vector is '5’.

Step 3: Calculating distance

We calculate the distance between the input vector and the weights of the two neurons. The
neuron with the weight closest to the input vector 'wins.' This could be calculated using any
distance metric, for example, the absolute difference:
•Neuron 1 distance: |5-2| = 3
•Neuron 2 distance: |5-8| = 3
Since both distances are equal, we can choose the winner randomly. Let's say Neuron 1 is the
winner.
Step 4: Updating weights
Some common competitive learning algorithms include:
We adjust the winning neuron's weight to bring it closer to the input vector. If our learning
rate (a tuning parameter in an optimization algorithm that determines the step size at each •Winner-take-all competitive learning. In this simple algorithm, the neuron
iteration) is 0.5, the weight update would be: with the highest activation "wins" and has its weights adjusted to move
•Neuron 1 weight: 2 + 0.5*(5-2) = 3.5 closer to the input. The other neurons do not update.
•Neuron 2 weight: 8 (unchanged)
•Self-organizing map (SOM). Maps high-dimensional input data onto a
Step 5: Iteration lower-dimensional grid of neurons and adjusts the weights of nearby
We repeat the process with all the other input vectors in the dataset, updating the weights neurons to be more similar to each input.
after each presentation.
•Neural gas. Similar to SOM, but forms clusters more flexibly without a rigid
Step 6: Convergence topology. The weights of neurons near the input are adjusted to be more
After several iterations (also known as epochs), the neurons' weights will start to converge to similar.
the centers of their corresponding input clusters. In this case, with 1-dimensional data
ranging from 1 to 10, we could expect one neuron to converge around the lower range (1 to •Learning vector quantization (LVQ). Builds on ideas from SOM but uses
5) and the other around the higher range (6 to 10). explicit class labels to guide competitive learning, resulting in prototypes that
cluster inputs by class.

Self Organizing Maps – Kohonen Maps

The architecture of the Self Organizing Map with two clusters and n input features of
any sample is given below:
Self organizing maps (SOM) were introduced in the 1980s by the Finnish computer scientist
Teuvo Kalevi Kohonen, also known as Kohonen’s Map after him.

It follows an unsupervised learning approach and trained its network through a competitive
learning algorithm. SOM is used for clustering and mapping (or dimensionality reduction)
techniques to map multidimensional data onto lower-dimensional which allows people to
reduce complex problems for easy interpretation.

SOM has two layers, one is the Input layer and the other one is the Output layer.
Trying SOM algorithm for a particular data

Initial weights be w1 = (0.45,0.89) , w2 = (0.55,0.83) , w3 = (0.95,0.32) and w4 = (0.62,0.78).

And let the four Neurons(N=4) N1, N2, N3 and N4 be located at the grid in Cartesian plane
positions (0,0), (1,0) , (1,0) and (1,1) respectively. And suppose the network take 2-dimensional
(A=2) input vector (x1,x2). Let w_i= (w1_i,w2_i) be the weight for the neuron i . nr = 0.6. Input
vector (3,1). Assume learning rate = 0.5 and it is constant throughout. Assume initial weights as
noted in the diagram.
7. update their weights using Updating weight for N4:

w(t+1) = w(t) + Q(t) * ර(t) (x-w(t))

Updating weight for N3:

Bidirectional Associative Memory (BAM)

Bidirectional associative memory (BAM), first proposed by Bart Kosko in the year 1988.
The BAM network performs forward and backward associative searches for stored
stimulus responses. The BAM is a recurrent hetero associative pattern-marching
nerwork that encodes binary or bipolar patterns using Hebbian learning rule. It
associates patterns, say from set A to patterns from set B and vice versa is also
performed. BAM neural nets can respond to input from either layers (input layer and
output layer).
Bidirectional Associative Memory Architecture

The architecture of BAM network consists of two layers of neurons which are connected
by directed weighted pare interconnecrions. The network dynamics involve two layers of
interaction. The BAM network iterates by sending the signals back and forth between the
two layers until all the neurons reach equilibrium. The weights associated with the Figure shows a BAM network consisting of n units in X layer and m units in Y layer. The layers can
network are bidirectional. Thus, BAM can respond to the inputs in either layer. be connected in both directions(bidirectional) with the result the weight matrix sent from the X layer
to the Y layer is W and the weight matrix for signals sent from the Y layer to the X layer is WT. Thus,
the Weight matrix is calculated in both directions.
 3. Healthcare:
Applications of Artificial Neural Networks
• Artificial Neural Networks are used in Oncology to train algorithms that can identify cancerous
1.Social Media: tissue at the microscopic level at the same accuracy as trained physicians. Various rare diseases
• Artificial Neural Networks are used heavily in Social Media. For example, let’s take the ‘People may manifest in physical characteristics and can be identified in their premature stages by
you may know’ feature on Facebook that suggests people that you might know in real life so using Facial Analysis on the patient photos.
that you can send them friend requests. Well, this magical effect is achieved by using Artificial • So the full-scale implementation of Artificial Neural Networks in the healthcare environment
Neural Networks that analyze your profile, your interests, your current friends, and also their can only enhance the diagnostic abilities of medical experts and ultimately lead to the overall
friends and various other factors to calculate the people you might potentially know. improvement in the quality of medical care all over the world.
• Another common application of Machine Learning in social media is facial recognition. This is • Convolutional Neural Networks are actively employed in the healthcare industry for X ray
done by finding around 100 reference points on the person’s face and then matching them with detection, CT Scan and ultrasound.
those already available in the database using convolutional neural networks.
2. Marketing and Sales: 4. Personal Assistants:

• When you log onto E-commerce sites like Amazon and Flipkart, they will recommend your • I am sure you all have heard of Siri, Alexa, Cortana, etc., and also heard them based on the
products to buy based on your previous browsing history. Similarly, suppose you love Pasta, then phones you have!!! These are personal assistants and an example of speech recognition that
Zomato, Swiggy, etc. will show you restaurant recommendations based on your tastes and uses Natural Language Processing to interact with the users and formulate a response
previous order history. This is true across all new-age marketing segments like Book sites, Movie accordingly.
services, Hospitality sites, etc. and it is done by implementing personalized marketing. This uses
Artificial Neural Networks to identify the customer likes, dislikes, previous shopping history, etc., • Natural Language Processing uses artificial neural networks that are made to handle many tasks
and then tailor the marketing campaigns accordingly. of these personal assistants such as managing the language syntax, semantics, correct speech,
the conversation that is going on, etc.

7. Weather Forecasting
5. Defence
• Multilayer Perceptron (MLP), Convolutional Neural Network (CNN) and Recurrent
• Defence is the backbone of every country. Every country’s state in the international Neural Networks (RNN) are used for weather forecasting. Traditional ANN multilayer
domain is assessed by its military operations. Neural Networks also shape the defence models can also be used to predict climatic conditions 15 days in advance. A
operations of technologically advanced countries. The United States of America, combination of different types of neural network architecture can be used to predict air
Britain, and Japan are some countries that use artificial neural networks for developing temperatures.
an active defence strategy. • Various inputs like air temperature, relative humidity, wind speed and solar radiations were
• Neural networks are used in logistics, armed attack analysis, and for object location. considered for training neural network based models. Combination models (MLP+CNN),
They are also used in air patrols, maritime patrol, and for controlling automated (CNN+RNN) usually works better in the case of weather forecasting.
drones.

6. Signature Verification and Handwriting Analysis

8. Stock Market Prediction
• Signature Verification , as the self explanatory term goes, is used for verifying an
individual’s signature. Banks, and other financial institutions use signature verification • To make a successful stock prediction in real time a Multilayer Perceptron MLP (class of
to cross check the identity of an individual. feedforward artificial intelligence algorithm) is employed. MLP comprises multiple layers
• Artificial Neural Networks are used for verifying the signatures. ANN are trained of nodes, each of these layers is fully connected to the succeeding nodes. Stock’s past
to recognize the difference between real and forged signatures. ANNs can be used for performances, annual returns, and non profit ratios are considered for building the MLP
the verification of both offline and online signatures. model.
• Convolutional Neural Networks (CNN) are used for handwriting analysis and handwriting
verification.