Scribd
Upload
10 views6 pages

Alexnet and Data Augmentation

AlexNet is a convolutional neural network architecture developed by Alex Krizhevsky and others, notable for being the first to utilize GPU for training efficiency. It comprises 5 convolutional layers, 3 max-pooling layers, and employs techniques like ReLU activation, dropout, and data augmentation to enhance performance and reduce overfitting. The model was trained on a GTX 580 GPU and features over 60 million parameters, with input images typically sized at 227x227x3.

Uploaded by

beebird234
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
10 views6 pages

Alexnet and Data Augmentation

AlexNet is a convolutional neural network architecture developed by Alex Krizhevsky and others, notable for being the first to utilize GPU for training efficiency. It comprises 5 convolutional layers, 3 max-pooling layers, and employs techniques like ReLU activation, dropout, and data augmentation to enhance performance and reduce overfitting. The model was trained on a GTX 580 GPU and features over 60 million parameters, with input images typically sized at 227x227x3.

Uploaded by

beebird234
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 6

Alexnet

OR

1. The convolutional neural network (CNN) architecture known


as AlexNet was created by Alex Krizhevsky, Ilya Sutskever, and
Geoffrey Hinton, who served as Krizhevsky’s PhD advisor.
2. This was the first architecture that used GPU to boost the
training performance.
3. AlexNet consists of 5 convolution layers, 3 max-pooling layers,
2 Normalized layers, 2 fully connected layers and 1 SoftMax
layer.
4. Each convolution layer consists of a convolution filter and a
non-linear activation function called “ReLU”. The pooling
layers are used to perform the max-pooling function and the
input size is fixed due to the presence of fully connected layers.
5. The input size is mentioned at most of the places as 224x224x3
but due to some padding which happens it works out to be
227x227x3. Above all this AlexNet has over 60 million
parameters.

Key Features:

 ‘ReLU’ is used as an activation function rather than ‘tanh’

 Batch size of 128

 SGD Momentum is used as a learning algorithm

 Data Augmentation is been carried out like flipping, jittering,


cropping, colour normalization, etc.

AlexNet was trained on a GTX 580 GPU with only 3 GB of memory


which couldn’t fit the entire network. So the network was split
across 2 GPUs, with half of the neurons(feature maps) on each
GPU.
Data Augmentation

Overfitting can be avoided by showing Neural Net various iterations


of the same image. Additionally, it assists in producing more data
and compels the Neural Net to memorise the main qualities.

Data augmentation- artificially increase the size of the training set-


create a batch of "new" data from existing data by means of
translation, flipping, noise

 Augmentation by Mirroring

Consider that our training set contains a picture of a cat. A cat can
also be seen as its mirror image. This indicates that by just flipping
the image above the vertical axis, we may double the size of the
training datasets.

Data Augmentation by Mirroring


 Augmentation by Random Cropping of Images

Randomly cropping the original image will also produce additional


data that is simply the original data shifted.

For the network’s inputs, the creators of AlexNet selected random


crops with dimensions of 227 by 227 from within the 256 by 256
image boundary. They multiplied the size of the data by 2048 using
this technique.

Data Augmentation by Random Cropping

Dropout

A neuron is removed from the neural network during dropout with a


probability of 0.5. A neuron that is dropped does not make any
contribution to either forward or backward propagation. As seen in
the graphic below, each input is processed by a separate Neural
Network design. The acquired weight parameters are therefore more
reliable and less prone to overfitting.

AlexNet Summary

You might also like