Scribd
Upload
26 views22 pages

Lecture 33 35 Fuzzy C Means Clustering

Fuzzy C-means clustering, developed by Jim Bezdek in 1981, allows data points to belong to multiple clusters with varying degrees of membership, unlike the binary assignment of K-means clustering. The algorithm involves initialization, iteration, and termination processes similar to K-means, focusing on calculating centroids and updating membership values until convergence. Additionally, fuzzy controllers utilize linguistic variables and rules to make decisions based on fuzzy logic, enhancing control systems with uncertainty.

Uploaded by

rjain5be21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
26 views22 pages

Lecture 33 35 Fuzzy C Means Clustering

Fuzzy C-means clustering, developed by Jim Bezdek in 1981, allows data points to belong to multiple clusters with varying degrees of membership, unlike the binary assignment of K-means clustering. The algorithm involves initialization, iteration, and termination processes similar to K-means, focusing on calculating centroids and updating membership values until convergence. Additionally, fuzzy controllers utilize linguistic variables and rules to make decisions based on fuzzy logic, enhancing control systems with uncertainty.

Uploaded by

rjain5be21
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 22

Fuzzy C – Means Clustering

Dr. Saurabh Bhardwaj


Fuzzy c-means Clustering

• Developed by Jim BEZDEK in 1981


• K-means clustering algorithm gives the values of any point lying in some particular cluster to be
either as 0 or 1
• In Fuzzy c-means clustering each point has a probability (degree) of belonging to each cluster,
rather than completely belonging to just one cluster
• Processes of initialization, iteration, and termination are the same as used in k-means
• Find out the centroid of the data points and then calculate the distance of each data point from
the given centroids until the clusters formed becomes constant
• K-means is a special case of fuzzy c-means when the probability function used is simply 1 if the
data point is closest to a centroid and 0 otherwise
Fuzzy Sets

A set of ordered pairs given by A = {(x, 𝜇A(𝑥)) : 𝑥 ∈ 𝑋},where first element is an element of set A and
second element is a value in the interval [0,1].

• boundaries of the fuzzy sets are vague and ambiguous


Fuzzy Controller : Steps
(1) Inputs & Outputs (2) Linguistic Descriptions: (3) Rule Base
“error is poslarge” Pendulum is at a
Linguistic variables
significant angle to the left of vertical.
“error”, “change-in-error”, “force”
describes e(t), d/dt( e(t)), u(t)
“error is zero” represents situation
where pendulum is near to vertical pos.
Linguistic values
“neglarge”, “negsmall”, “zero”, If error is neglarge and change-in-error
“possmall”, “poslarge” is neglarge Then force is poslarge
-2 to represent “neglarge”
-1 to represent “negsmall”
0 to represent “zero”
1 to represent “possmall”
2 to represent “poslarge”

𝑒 𝑡 = 𝑟 𝑡 − 𝑦(𝑡)

𝑑𝑒 𝑡
𝑑𝑡
Fuzzy Controller : Steps
(4) Membership Functions
Function µ quantifies the certainty that e(t) can be classified
linguistically as “possmall”

If e(t) = −π/2 then µ(−π/2) = 0,


We are certain that e(t) = −π/2 is not “possmall”

If e(t) = π/8 then µ(π/8) = 0.5,


We are halfway certain that e(t) = π/8 is “possmall”

If e(t)= π/4 then µ(π/4) = 1.0,


Absolutely certain that e(t) = π/4 is “possmall”

Set of values described by µ as being “positive


small” is called a “fuzzy set”

Vertical axis : Certainty


Horizontal axis : Universe of discourse
Fuzzy Controller : Steps
(5) Determining Which Rules to Use

Premise Quantification via Fuzzy Logic :


Premises of all the rules are compared with inputs to determine which rules apply to the current
situation.

Determining Which Rules Are On:


Conclusions are determined using the rules that have been determined to apply at the current
time.
Fuzzy Controller : Steps
For inference, Quantify each of the rules
(7) Premise Quantification Using Fuzzy Logic
! #$(&) !
Let 𝑒 𝑡 = and =
" #( )*

Quantify “and”

The certainty of statement µpremise


“error is zero and change-in-error is possmall”

Minimum:
µpremise = min{0.5,0.25}=0.25

Product:
µpremise =(0.5)(0.25)= 0.125
Fuzzy Controller : Steps

A rule is on if its premise membership


function

Now, inference mechanism will combine


the recommendations of all the rules to
come up with a single conclusion.

1. If error is zero and change-in-error is zero Then force is zero


2. If error is zero and change-in-error is possmall Then force is negsmall
Fuzzy Controller : Steps
(7) Inference Step: Determining Conclusions
Recommendation from Rule-2
Recommendations from Rule-1:
If error is zero and change-in-error is zero If error is zero and change-in-error is possmall
Then force is zero Then force is negsmall
µpremise(1) =min {0.25,1}=0.25 µpremise(2) =min{0.75,1}=0.75
Certainty of rule : 0.25 Certainty of rule : 0.75
“implied fuzzy set”= “implied fuzzy set”= µ(2)(u)=min{0.75,µnegsmall(u)}
µ(1)(u)=min{0.25,µzero(u)}

Can be no more certain about consequent


than premise
Fuzzy Controller : Steps
(7) Converting Decisions into Actions
Fuzzy c-means clustering
a) Normal c-partition {(𝑐 ≥ 2), c is an integer } of a set 𝑠 = {𝑥! , 𝑥" , … , 𝑥# } ⊆ ℝ$ represented as

𝑝(𝐴! , 𝐴" , … , 𝐴% )
Definition:

%
𝐴& ≠ ∅ ∀𝑖 = 1,2, … , 𝑐 ; 𝐴& ∩ 𝐴' = ∅ ∀𝑖 ≠ j ; = 𝐴& = 𝑠
!
Objective function of normal c-means
𝑣& = 𝑚𝑒𝑎𝑛 𝑜𝑓𝐴& ; ∀𝑖 = 1,2, … , 𝑐

%
"
E E 𝑥 − 𝑣&
&(! )∈+!
The best partition is the minimum within cluster distance.
Fuzzy c-means clustering
b) Fuzzy c-partition of a set 𝑠 = {𝑥! , 𝑥" , … , 𝑥# } ⊆ ℝ$ represented by 𝑈, 𝑠 where 𝑈 is an 𝑛×𝑐 matrix

𝑈 = 𝑢&'
#×%

𝑢&' denotes the membership value of the ith point to the jth fuzzy set 1 ≤ 𝑖 ≤ 𝑛; 1 ≤ 𝑗 ≤ 𝑐 satisfying
the properties stated as :
0 ≤ 𝑢&' ≤ 1 ∀𝑖, 𝑗

%
E 𝑢&' = 1 ∀𝑖 = 1,2, … , 𝑛
'(!

#
0 < E 𝑢&' < 𝑛 ∀𝑗 = 1,2, … , 𝑐
&(!
Fuzzy c-means clustering
Let 𝑟 > 1

∑#&(! 𝑥& 𝑢&'


-
𝑣' = ; 𝑗 = 1,2, … , 𝑐
∑#&(! 𝑢&'
-

The objective function is

# %
-
𝐽- 𝑈, 𝑠, 𝐴 = E E 𝑢&' 𝑥& − 𝑣' ′𝐴$.# 𝑥& − 𝑣'
&(! '(!

Where A is a positive definite matrix


If A is an identity matrix and 𝑢&' are 0 or 1
𝐽- 𝑈, 𝑆, 𝐴 is to be minimized for a given A, s, and r we need to find the optimal U.
Fuzzy c-means clustering Algorithm
1. We are given s, A, r and the number of clusters c
2. Start with a fuzzy c-partition U of S
3. Compute v’j j=1,2,…,c

% 1!
# &'%
0!" 1≤𝑖≤𝑛
4. Update membership values: 𝑢&' = ∑%/(! #
0!$ 1≤𝑗≤𝑐

*
𝑑+, = 𝑥+ − 𝑣, ′𝐴-./ 𝑥+ − 𝑣,

5. Go to 3 if convergence criteria is not satisfied


Fuzzy c-means clustering
Step 1: Given the data points based on the number of clusters required initialize the membership
table with random values.

• Suppose the given data points are {(1, 3), (2, 5), (4, 8), (7, 9)}

Cluster (1, 3) (2, 5) (4, 8) (7, 9)


1 0.8 0.7 0.2 0.1
2 0.2 0.3 0.8 0.9
Fuzzy c-means clustering
Step 2: Find out the centroid.

Centroids are:
(1.568, 4.051) and (5.35, 8.215)
Fuzzy c-means clustering
Step 4: Update membership values

Centroids are:
(1.568, 4.051) and (5.35, 8.215)
Fuzzy c-means clustering
Step 4: Update membership values

𝑢!! =

𝑢!" =

𝑢"! =

𝑢"" =
Fuzzy K-Means Algorithm Example
• Assumption : Two Clusters K = 2
• Initialization of memberships of data points randomly
• Computation of Centroid of clusters

This process is repeated until the termination criteria is achieved


Fuzzy K-Means Algorithm Example

• Find similarity measure


(Euclidian distance)
• Computation of memberships

• Computation of centroid

This process is repeated until the termination criteria is achieved


Fuzzy K-Means Algorithm Example

This process is repeated until the termination criteria is achieved


EIED
Thank You!!

+91-175-2393055

NAAC

[email protected]

You might also like