Day 3 - Machine Learning

Continuation
 Contents

1. Unsupervised Learning and Algorithms

2. Reinforcement Learning Introduction
3. Interactive simulation demonstration.
4. Q&A and reflection on learning applications.

2
 Unsupervised Learning
What is Unsupervised Learning?
Unsupervised learning is a machine learning technique, where we do not need to
supervise the model,Instead of that we need to encourage the model to work on
its own to discover knowledge. It mainly deals with the unlabelled data.

3
 Unsupervised Learning
● Dataset contains no labels:
x1,...xn
● Goal: to find interesting structures in the data

4
 Unsupervised Learning Algorithms
● Unsupervised Learning Algorithms allow users to
perform more complex processing tasks compared to
supervised learning.
● Although, unsupervised learning can be more
unpredictable compared with other natural learning
methods.
● Unsupervised learning algorithms include clustering,
anomaly detection, neural networks, etc.

5
 Example of Unsupervised Learning
● Let’s, take an example of Unsupervised Learning for a baby and her family dog.

● She knows and identifies this dog. Few weeks later a family friend brings
along a dog and tries to play with the baby.
● Baby has not seen this dog earlier. But it recognizes many features (2
ears, eyes, walking on 4 legs) are like her pet dog. She identifies the new
animal as a dog. This is unsupervised learning, where you are not taught
but you learn from the data (in this case data about a dog.)

6
 Why is it important?

● They find patterns which are previously unknown.

● Patterns help you to find features which can be useful
for categorization.
● They can detect anomalies and defects in the data.
● They work on unlabeled data which makes our work
easier.

7
 Need for Unsupervised Learning

● Annotating large datasets is very costly and time

consuming. Example: Speech Recognition, Medical
Image Analysis, etc.
● There may be cases where we don’t know how
many/what classes is the data divided into. Example:
Data Mining, Sentimental Analysis.
● We may want to use clustering to gain some insight
into the structure of the data before designing a
classifier

8
 Types of Unsupervised Learning:
Clustering Association

● Clustering is the task of dividing the ● Association rules allow us to

population or data points into a establish associations amongst data
number of groups such that data objects inside large databases. This
points in the same groups are more unsupervised technique includes
similar to other data points and discovering important relationships
dissimilar to the data points in the between variables in large
other groups. databases.

9
 Clustering in Machine Learning

● Clustering or cluster analysis is a machine learning technique, which groups

the unlabelled dataset.
● It does it by finding some similar patterns in the unlabelled dataset such as
shape, size, color, behavior, etc., and divides them as per the presence and
absence of those similar patterns.
● It is an unsupervised learning method, hence no supervision is provided to
the algorithm, and it deals with the unlabeled dataset.
● After applying this clustering technique, each cluster or group is provided
with a cluster-ID. ML system can use this id to simplify the processing of large
and complex datasets.
● The clustering technique is commonly used for statistical data analysis.

10
 Clustering in Machine Learning

● working of the clustering algorithm

11
 Types of Clustering Methods

● Main clustering methods used in Machine learning:

○ Partitioning Clustering
○ Density-Based Clustering
○ Distribution Model-Based Clustering
○ Hierarchical Clustering
○ Fuzzy Clustering

12
 Types of Clustering Methods

Partitioning Clustering

It is a type of clustering that divides the data into non-hierarchical groups. It is

also known as the centroid-based method. The most common example of
partitioning clustering is the K-Means Clustering algorithm.

13
 Types of Clustering Methods

Density-Based Clustering

The density-based clustering method connects the highly-dense areas into

clusters, and the arbitrarily shaped distributions are formed as long as the dense
region can be connected. This algorithm does it by identifying different clusters
in the dataset and connects the areas of high densities into clusters. The dense
areas in data space are divided from each other by sparser areas.

14
 Types of Clustering Methods

Distribution Model-Based Clustering

In the distribution model-based clustering method, the data is divided based on

the probability of how a dataset belongs to a particular distribution. The
grouping is done by assuming some distributions commonly Gaussian
Distribution.

15
 Types of Clustering Methods

Hierarchical Clustering

Hierarchical clustering can be used as an alternative for the partitioned

clustering as there is no requirement of pre-specifying the number of clusters to
be created. In this technique, the dataset is divided into clusters to create a
tree-like structure, which is also called a dendrogram. The observations or any
number of clusters can be selected by cutting the tree at the correct level. The
most common example of this method is the Agglomerative Hierarchical
algorithm.

16
 Types of Clustering Methods

Fuzzy Clustering

Fuzzy clustering is a type of soft method in which a data object may belong to
more than one group or cluster. Each dataset has a set of membership
coefficients, which depend on the degree of membership to be in a cluster.
Fuzzy C-means algorithm is the example of this type of clustering; it is
sometimes also known as the Fuzzy k-means algorithm.

17
 Clustering Algorithms

K-Means algorithm: The k-means algorithm is one of the most popular

clustering algorithms. It classifies the dataset by dividing the samples into
different clusters of equal variances. The number of clusters must be specified in
this algorithm. It is fast with fewer computations required, with the linear
complexity of O(n).

Mean-shift algorithm: Mean-shift algorithm tries to find the dense areas in

the smooth density of data points. It is an example of a centroid-based model,
that works on updating the candidates for centroid to be the center of the points
within a given region.

DBSCAN Algorithm: It stands for Density-Based Spatial Clustering of

Applications with Noise. It is an example of a density-based model similar to the
mean-shift, but with some remarkable advantages. In this algorithm, the areas of
high density are separated by the areas of low density. Because of this, the
clusters can be found in any arbitrary shape.
18
 Clustering Algorithms

Expectation-Maximization Clustering using GMM: This algorithm can

be used as an alternative for the k-means algorithm or for those cases where K-
means can be failed. In GMM, it is assumed that the data points are Gaussian
distributed.

Agglomerative Hierarchical algorithm: The Agglomerative hierarchical

algorithm performs the bottom-up hierarchical clustering. In this, each data point
is treated as a single cluster at the outset and then successively merged. The
cluster hierarchy can be represented as a tree-structure.

Affinity Propagation: It is different from other clustering algorithms as it does

not require to specify the number of clusters. In this, each data point sends a
message between the pair of data points until convergence. It has O(N2T) time
complexity, which is the main drawback of this algorithm.

19
 Clustering Algorithms

K-Means Algorithm
● It will groups the unlabeled dataset into different clusters.
● Here K defines the number of predefined clusters that need to be created in
the process, as if K=2, there will be two clusters, and for K=3, there will be
three clusters, and so on.
● It allows us to cluster the data into different groups and a convenient way
to discover the categories of groups in the unlabeled dataset on its own
without the need for any training.
● It is a centroid-based algorithm, where each cluster is associated with a
centroid. The main aim of this algorithm is to minimize the sum of distances
between the data point and their corresponding clusters.
● The algorithm takes the unlabeled dataset as input, divides the dataset into
k-number of clusters, and repeats the process until it does not find the best
clusters. The value of k should be predetermined in this algorithm.
20
 Clustering Algorithms

K-Means Algorithm

● The k-means clustering algorithm mainly performs two tasks:

○ Determines the best value for K center points or centroids by an
iterative process.
○ Assigns each data point to its closest k-center. Those data points which
are near to the particular k-center, create a cluster.
working of the K-means Clustering Algorithm:

21
 Clustering Algorithms
The working of the K-Means algorithm is explained in the below
steps:
● Step-1: Select the number K to decide the number of clusters.
● Step-2: Select random K points or centroids. (It can be other from the input
dataset).
● Step-3: Assign each data point to their closest centroid, which will form the
predefined K clusters.
● Step-4: Calculate the variance and place a new centroid of each cluster.
● Step-5: Repeat the third steps, which means reassign each datapoint to the
new closest centroid of each cluster.
● Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.
● Step-7: The model is ready.
22
 Clustering Algorithms

K-Means Algorithm

23
 Clustering Algorithms

Agglomerative Hierarchical clustering

● To group the datasets into clusters, it follows the bottom-up approach.
● This algorithm considers each dataset as a single cluster at the beginning,
and then start combining the closest pair of clusters together. It does this
until all the clusters are merged into a single cluster that contains all the
datasets.

This hierarchy of clusters is represented in the form of the dendrogram.

24
 Clustering Algorithms

How the Agglomerative Hierarchical clustering Work?

25
 Applications of Clustering Algorithms

● In Identification of Cancer Cells

● In Search Engines

● Customer Segmentation

● In Biology

● In Land Use

26
 Association Rule Learning

● Association rule learning is a type of unsupervised learning technique that

checks for the dependency of one data item on another data item and maps
accordingly so that it can be more profitable.
● It tries to find some interesting relations or associations among the variables
of dataset.
● It is based on different rules to discover the interesting relations between
variables in the database.
● it is employed in Market Basket analysis, Web usage mining, continuous
production, etc.

27
 Association Rule Learning Algorithms
● Apriori Algorithm
○ This algorithm uses frequent datasets to generate association rules. It is
designed to work on the databases that contain transactions. This
algorithm uses a breadth-first search and Hash Tree to calculate the
itemset efficiently.
○ It is mainly used for market basket analysis and helps to understand the
products that can be bought together. It can also be used in the
healthcare field to find drug reactions for patients.
● Eclat Algorithm
○ Eclat algorithm stands for Equivalence Class Transformation. This
algorithm uses a depth-first search technique to find frequent itemsets in
a transaction database. It performs faster execution than Apriori
Algorithm.
● F-P Growth Algorithm
○ The F-P growth algorithm stands for Frequent Pattern, and it is the
improved version of the Apriori Algorithm. It represents the database in
the form of a tree structure that is known as a frequent pattern or tree.
The purpose of this frequent tree is to extract the most frequent 28
patterns.
 Association Rule Learning Algorithms
Apriori Algorithm
The Apriori algorithm uses frequent itemsets to generate association rules, and it
is designed to work on the databases that contain transactions. With the help of
these association rule, it determines how strongly or how weakly two objects are
connected. This algorithm uses a breadth-first search and Hash Tree to calculate
the itemset associations efficiently. It is the iterative process for finding the
frequent itemsets from the large dataset.

Steps for Apriori Algorithm

● Step-1: Determine the support of itemsets in the transactional database, and
select the minimum support and confidence.
● Step-2: Take all supports in the transaction with higher support value than
the minimum or selected support value.
● Step-3: Find all the rules of these subsets that have higher confidence value
than the threshold or minimum confidence.
● Step-4: Sort the rules as the decreasing order of lift.

29
 Association Rule Learning Algorithms
FP-Growth Algorithm
It is an alternative way to find frequent item sets without using candidate
generations, thus improving performance. For so much, it uses a divide-and-
conquer strategy. The core of this method is the usage of a special data structure
named frequent-pattern tree (FP-tree), which retains the item set association
information.

Steps of FP-Growth Algorithm

● First, it compresses the input database creating an FP-tree instance to
represent frequent items.
● After this first step, it divides the compressed database into a set of
conditional databases, each associated with one frequent pattern.
● Finally, each such database is mined separately.

30
 Applications of Association Rule Learning

● Market Basket Analysis

● Medical Diagnosis

● Protein Sequence

● It is also used for the Catalog Design and Loss-leader Analysis and many more

other applications.

31
 Disadvantages of Unsupervised Learning

● Unsupervised Learning is harder as compared to Supervised Learning.

Since, quantifying the inference made is difficult, since the ground-
truth is missing.
● How do we know if results are meaningful since it has unlabelled
data?
○ External evaluation- Expert analysis.
○ Internal evaluation- Objective function.

32
 Reinforcement Algorithms

● A reinforcement learning algorithm, or agent, learns by

interacting with its environment. The agent receives rewards
by performing correctly and penalties for performing
incorrectly.

33
Terminologies used in Reinforcement Learning

34
 Terminologies used in Reinforcement Learning
● Agent — the learner and the decision maker.
● Environment — where the agent learns and decides what actions to
perform.
● Action — a set of actions which the agent can perform.
● State — the state of the agent in the environment.
● Reward — for each action selected by the agent the environment provides a
reward. Usually a scalar value.
● Policy — the decision-making function (control strategy) of the agent, which
represents a mapping from situations to actions.
● Value function — mapping from states to real numbers, where the value of
a state represents the long-term reward achieved starting from that state,
and executing a particular policy.
● Function approximator — refers to the problem of inducing a function from
training examples. Standard approximators include decision trees, neural
networks, and nearest-neighbor methods

35
 Terminologies used in Reinforcement Learning
● Markov decision process (MDP) — A probabilistic model of a sequential
decision problem, where states can be perceived exactly, and the current
state and action selected determine a probability distribution on future
states. Essentially, the outcome of applying an action to a state depends only
on the current action and state (and not on preceding actions or states).
● Dynamic programming (DP) — is a class of solution methods for solving
sequential decision problems with a compositional cost structure. Richard
Bellman was one of the principal founders of this approach.
● Monte Carlo methods — A class of methods for learning of value functions,
which estimates the value of a state by running many trials starting at that
state, then averages the total rewards received on those trials.
● Temporal Difference (TD) algorithms — A class of learning methods, based
on the idea of comparing temporally successive predictions. Possibly the
single most fundamental idea in all of reinforcement learning.
● Model — The agent’s view of the environment, which maps state-action
pairs to probability distributions over states. Note that not every
reinforcement learning agent uses a model of its environment.

36
 Reinforcement Learning Workflow

● Create the Environment

● Define the reward
● Create the agent
● Train and validate the agent
● Deploy the policy

37
 Need for Reinforcement Learning

● Reinforcement learning can be used to solve very complex problems that

cannot be solved by conventional techniques.
● In the absence of a training dataset, it is bound to learn from its experience.
• Reinforcement learning models can outperform humans in many tasks and
learning process is similar to human learning.
● DeepMind’s AlphaGo program, a reinforcement learning model, beat the
world champion Lee Sedol at the game of Go in March 2016.

38
 Characteristics of Reinforcement Learning
● No supervision, only a real value or reward signal
● Decision making is sequential
● Time plays a major role in reinforcement problems
● Feedback isn’t prompt but delayed
● The following data it receives is determined by the agent’s actions

39
 Reinforcement Learning Algorithms

Value-Based – The main goal of this method is to maximize a value function.

Here, an agent through a policy expects a long-term return of the current
states.
Policy-Based – In policy-based, you enable to come up with a strategy that
helps to gain maximum rewards in the future through possible actions
performed in each state. Two types of policy-based methods are
deterministic and stochastic.
Model-Based – In this method, we need to create a virtual model for the
agent to help in learning to perform in each specific environment

40
Reinforcement Learning Algorithm

Reinforcement Learning taxonomy as defined by OpenAI [Source]

41
Types of Reinforcement Learning

42
 Types of Reinforcement Learning
Positive Reinforcement Negative Reinforcement
Positive reinforcement is defined as Negative Reinforcement is
when an event, occurs due to specific represented as the strengthening of a
behavior, increases the strength and behavior. In other ways, when a
frequency of the behavior. It has a negative condition is barred or
positive impact on behavior. avoided, it tries to stop this action in
the future.
Advantages
– Maximizes the performance of an Advantages
action – Maximized behavior
– Sustain change for a longer period – Provide a decent to minimum
standard of performance
Disadvantage
– Excess reinforcement can lead to an Disadvantage
overload of states which would – It just limits itself enough to meet up
minimize the results. a minimum behavior

43
 Applications of Reinforcement Learning

● Gaming
Reinforcement learning has been applied to various games,
from classic board games like chess and Go to modern video
games like Dota 2 and StarCraft II. In these games, an agent
learns to make decisions and compete against human players
or other agents.
● Robotics
Reinforcement learning has been used in robotics to enable
robots to learn to navigate environments, manipulate objects,
and perform complex tasks. In these applications, an agent
learns from feedback received through sensors and actuators.

44
 Applications of Reinforcement Learning

● Finance
Reinforcement learning has found applications in finance,
including algorithmic trading, portfolio optimization, and fraud
detection. In these applications, an agent learns to make
decisions based on market data and financial indicators.

● Healthcare
Reinforcement learning has been used in healthcare to
optimize treatment plans and drug dosages. In these
applications, an agent learns from patient data and medical
records to make decisions that maximize patient outcomes.

45
 Practical Applications of reinforcement learning
● Robotics for Industrial Automation
● Text summarization engines, dialogue agents (text, speech),
gameplays
● Autonomous Self Driving Cars
● Machine Learning and Data Processing
● Training system which would issue custom instructions and
materials with respect to the requirements of students
● AI Toolkits, Manufacturing, Automotive, Healthcare, and Bots
● Aircraft Control and Robot Motion Control
● Building artificial intelligence for computer games

46
QUIZ Time:
https://forms.gle/mX8DJMBzMmcB9d4L8

47
 Demonstration.

https://colab.research.google.com/drive/17l6NxZN-lz-
oc2A598iqYU1mn7aUbws6?usp=sharing

48
 References
I. https://www.guru99.com/unsupervised-machine-learning.html
II. https://societyofai.medium.com/unsupervised-learning-a9f7264a0f2d
III. https://www.analyticsvidhya.com/blog/2021/02/introduction-to-
reinforcement-learning-for-beginners/
IV. https://theintactone.com/2019/08/26/ob-u3-topic-7-reinforcement-theory/
V. https://www.youtube.com/watch?v=2NEu9dbM4A8&t=800s
VI. https://smartlabai.medium.com/reinforcement-learning-algorithms-an-
intuitive-overview-904e2dff5bbc
VII.https://arjun-sarkar786.medium.com/reinforcement-learning-for-beginners-
introduction-concepts-algorithms-and-applications-3f805cbd7f92
VIII.https://www.javatpoint.com/clustering-in-machine-learning
IX. https://www.geeksforgeeks.org/frequent-pattern-growth-algorithm/

49
THANKS