Scribd
Upload
267 views21 pages

Autoencoders: Presented By: 2019220013 Balde Lansana (

An autoencoder is a type of neural network that is trained to copy its input to its output. It consists of an encoder that compresses the input into a hidden representation, and a decoder that tries to reconstruct the output from the hidden representation. Autoencoders are used for tasks like dimensionality reduction, feature extraction, image coloring, and data denoising. There are different types of autoencoders like convolutional autoencoders, sparse autoencoders, and deep autoencoders. Autoencoders are trained by minimizing the difference between the input and output using backpropagation. They can learn efficient data encodings in an unsupervised manner.

Uploaded by

BALDE
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
267 views21 pages

Autoencoders: Presented By: 2019220013 Balde Lansana (

An autoencoder is a type of neural network that is trained to copy its input to its output. It consists of an encoder that compresses the input into a hidden representation, and a decoder that tries to reconstruct the output from the hidden representation. Autoencoders are used for tasks like dimensionality reduction, feature extraction, image coloring, and data denoising. There are different types of autoencoders like convolutional autoencoders, sparse autoencoders, and deep autoencoders. Autoencoders are trained by minimizing the difference between the input and output using backpropagation. They can learn efficient data encodings in an unsupervised manner.

Uploaded by

BALDE
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 21

Autoencoders

Presented by:
2019220013
BALDE LANSANA( 兰撒那 )
Introduction

 An autoencoder is a neural network that is trained to


attempt to copy its input to its output.
 Internally, it has a hidden layer h that describes a code
used to represent the input.
 The network may be viewed as consisting of two parts: an
encoder function h=f(x) and a decoder that produces a
reconstruction r=g(h).
What are autoencoders?

 "Autoencoding" is a data compression algorithm where the


compression and decompression functions are:
 data-specific,
 lossy,
  learned automatically from examples rather than
engineered by a human.
What are autoencoders?
Hyperparameters of Autoencoders:
There are 4 hyperparameters that we need to set before training an
autoencoder:
Application of autoencoders

Autoencoders are used for tasks that involve :


Features extraction
Image Coloring
Feature variation
Data Compression
Learning generative model for data
data denoising  
dimensionality reduction for data
visualization.
Application of autoencoders
Image Coloring

Autoencoders are used for converting any black and white picture into a
colored image. Depending on what is in the picture, it is possible to tell
what the color should be.
Application of autoencoders
Dimensionality Reduction

The reconstructed image is the same as our input but with reduced
dimensions. It helps in providing the similar image with a reduced pixel
value.
Application of autoencoders
Denoising Image

The input seen by the autoencoder is not the raw input but a
stochastically corrupted version. A denoising autoencoder is thus
trained to reconstruct the original input from the noisy version.
Application of autoencoders
Watermark Removal
It is also used for removing watermarks from images or to remove
any object while filming a video or a movie.
Application of autoencoders
Feature variation

It extracts only the required features of an image and generates


the output by removing any noise or unnecessary interruption.
Architecture of Autoencoders
An Autoencoder consist of three layers:
1.Encoder
2.Code
3.Decoder
Types of Autoencoders
Convolution Autoencoders

Autoencoders in their traditional formulation does not take into account


the fact that a signal can be seen as a sum of other signals.
Convolutional Autoencoders use the convolution operator to exploit this
observation. They learn to encode the input in a set of simple signals and
then try to reconstruct the input from them, modify the geometry or the
reflectance of the image.
Types of Autoencoders
Sparse Autoencoders
Sparse autoencoders offer us an alternative method for
introducing an information bottleneck without requiring
a reduction in the number of nodes at our hidden
layers. Instead, we’ll construct our loss function such that
we penalize activations within a layer.
Types of Autoencoders
Deep Autoencoders
A deep autoencoder is composed of two, symmetrical deep-belief
networks-
Types of Autoencoders
Contractive Autoencoders

A contractive autoencoder is an unsupervised deep


learning technique that helps a neural network encode unlabeled
training data.
This is accomplished by constructing a loss term which penalizes
large derivatives of our hidden layer activations with respect to the
input training examples, essentially penalizing instances where a
small change in the input leads to a large change in the encoding
space.
How to train the AutoEncoder

・Starting with random


weights in the two networks
・They are trained by
minimizing the discrepancy
between the original data
and its reconstruction.

・ Gradients are obtained by


the chain rule to back-
propagate error from the
decoder network to encoder
network.
Comparing PCA to an autoencoder

The MNIST database holds 60,000 samples of images for training the system, and
another 10,000 for testing the system.
Comparing PCA to an autoencoder
possible research ideas

 Combine an Auto-encoder with any other


machine learning algorithm:
 Classification: LDA or SVM
 Regression technique.

 Use more than two data sets to compare the


performances of an Auto-Encoder and PCA
 Find a data set of image and use an
autoencoder as a generator model
 Use the Auto-encoders as data compressor
technique for IoT
References

[1] https://www.edureka.co/blog/autoencoders-tutorial/
[2] G. Hinton, R. R. Salakhutdinov, Science 313, 504-507 (2006)
[3] G. E. Hinton. A practical guide to training restricted Boltzmann
machines. Technical Report 2010-003, University of Toronto, 2010
[4] http://cl.naist.jp/~kevinduh/a/deep2014/
[5] L. Deng, "The MNIST Database of Handwritten Digit Images for Machine
Learning Research," in IEEE Signal Processing Magazine, vol. 29, no. 6, pp.
141-142, Nov. 2012.
[6] A. Kane and N. Shiri, "Selecting the Top-k Discriminative Features Using
Principal Component Analysis," 2016 IEEE 16th International Conference on
Data Mining Workshops (ICDMW), Barcelona, Spain, 2016, pp. 639-646.
[7] C. Hu, X. Hou and Y. Lu, "Improving the Architecture of an Autoencoder
for Dimension Reduction," 2014 IEEE 11th Intl Conf on Ubiquitous
Intelligence and Computing and 2014 IEEE
Thank You

Pattern Recognition

You might also like