Scribd
Upload
7 views

SML unit 3

The document covers unsupervised learning techniques, focusing on K-means clustering and autoencoders. K-means clustering groups unlabeled data into predefined clusters by iteratively determining centroids and assigning data points. Autoencoders, which consist of an encoder, code, and decoder layers, compress and reconstruct data, with deep autoencoders featuring multiple layers for enhanced encoding and decoding capabilities.

Uploaded by

anrish.555
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
7 views

SML unit 3

The document covers unsupervised learning techniques, focusing on K-means clustering and autoencoders. K-means clustering groups unlabeled data into predefined clusters by iteratively determining centroids and assigning data points. Autoencoders, which consist of an encoder, code, and decoder layers, compress and reconstruct data, with deep autoencoders featuring multiple layers for enhanced encoding and decoding capabilities.

Uploaded by

anrish.555
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 38

Unit-3

Unsupervised Learning

K-means Clustering, Principal Component Analysis (PCA), Singular Value decomposition


(SVD), Deep auto encoders, Deep auto encoders applied on handwritten digits using
Keras

Dept of Networking & Communications 1


K-Means Clustering Algorithm
K-Means Clustering is an Unsupervised Learning algorithm, which groups the unlabeled dataset
into different clusters.
Here K defines the number of pre-defined clusters that need to be created in the process, as if
K=2, there will be two clusters, and for K=3, there will be three clusters, and so on.

The k-means clustering algorithm mainly performs two tasks:


•Determines the best value for K center points or centroids by an iterative process.
•Assigns each data point to its closest k-center. Those data points which are near to the particular
k-center, create a cluster.

Dept of Networking & Communications 2


How does the K-Means Algorithm Work?

The working of the K-Means algorithm is explained in the below steps:


Step-1: Specify the number k of Cluster to assign.
Step-2: Randomly initialize K Centroid.
Step-3: Repeat
Step-4:Expectation : Assign each point to its closest centroid.
Step-5: maximization : Compute the new centroid(mean) of each cluster.
Step-6: Until the centroid position do not change

Dept of Networking & Communications 3


Dept of Networking & Communications 4
Dept of Networking & Communications 5
Dept of Networking & Communications 6
Dept of Networking & Communications 7
Dept of Networking & Communications 8
Dept of Networking & Communications 9
Dept of Networking & Communications 10
Dept of Networking & Communications 11
Dept of Networking & Communications 12
K – means clustering
K – means clustering
K – means clustering
K – means clustering
Dept of Networking & Communications 17
Dept of Networking & Communications 18
Dept of Networking & Communications 19
Dept of Networking & Communications 20
Dept of Networking & Communications 21
Dept of Networking & Communications 22
Dept of Networking & Communications 23
Dept of Networking & Communications 24
Dept of Networking & Communications 25
Dept of Networking & Communications 26
Dept of Networking & Communications 27
Dept of Networking & Communications 28
Dept of Networking & Communications 29
Dept of Networking & Communications 30
Dept of Networking & Communications 31
Dept of Networking & Communications 32
Dept of Networking & Communications 33
Autoencoders

What Are Autoencoders?


Autoencoders are very useful in the field of unsupervised machine learning. You can use
them to compress the data and reduce its dimensionality.

The main difference between Autoencoders and Principle Component Analysis (PCA) is that
while PCA finds the directions along which you can project the data with maximum
variance, Autoencoders reconstruct our original input given just a compressed version of it.
If anyone needs the original data can reconstruct it from the compressed data using an
autoencoder

Dept of Networking & Communications 34


Architecture
An Autoencoder is a type of neural network that can learn to reconstruct images, text, and other data
from compressed versions of themselves.
An Autoencoder consists of three layers:
1.Encoder
2.Code
3.Decoder
• The Encoder layer compresses the input image into a latent space representation. It encodes the input
image as a compressed representation in a reduced dimension.
• The compressed image is a distorted version of the original image.
• The Code layer represents the compressed input fed to the decoder layer.
• The decoder layer decodes the encoded image back to the original dimension. The decoded image is
reconstructed from latent space representation, and it is reconstructed from the latent space
representation and is a lossy reconstruction of the original image.

Dept of Networking & Communications 35


Deep Autoencoders
A deep autoencoder is composed of two, symmetrical deep-belief networks that typically have four or
five shallow layers representing the encoding half of the net, and second set of four or five layers that
make up the decoding half.

Encoding Input Data


Let’s sketch out an example encoder:
784 (input) ----> 1000 ----> 500 ----> 250 ----> 100 -----> 30

the input fed to the network is 784 pixels (the square of the 28x28 pixel images in
the MNIST dataset), then the first layer of the deep autoencoder should have 1000
parameters; i.e. slightly larger.
This may seem counterintuitive, because having more parameters than input is a
good way to overfit a neural network.
In this case, expanding the parameters, and in a sense expanding the features of the
input itself, will make the eventual decoding of the autoencoded data possible.
The layers will be 1000, 500, 250, 100 nodes wide, respectively, until the end,
where the net produces a vector 30 numbers long. This 30-number vector is the
last layer of the first half of the deep autoencoder, the pretraining half, and it is the
product of a normal RBM, Dept of Networking & Communications 36
Decoding Representations

Those 30 numbers are an encoded version of the 28x28 pixel image. The second half of a deep
autoencoder actually learns how to decode the condensed vector, which becomes the input as it makes its
way back.
The decoding half of a deep autoencoder is a feed-forward net with layers 100, 250, 500 and 1000 nodes
wide, respectively. Layer weights are initialized randomly.
784 (output) <---- 1000 <---- 500 <---- 250 <---- 30
The decoding half of a deep autoencoder is the part that learns to reconstruct the image. It does so with a
second feed-forward net which also conducts back propagation. The back propagation happens through
reconstruction entropy.

Dept of Networking & Communications 37


Dept of Networking & Communications 38

You might also like