Scribd
Upload
68 views20 pages

Artificial Intelligence: Alexnet

The document describes AlexNet, which was the winner of the ImageNet LSVRC-2012 competition for image classification. AlexNet used a neural network architecture with 5 convolutional layers and 3 fully-connected layers. It achieved top-5 and top-1 error rates on the ImageNet test set that were significantly lower than other models at the time, demonstrating the effectiveness of deep learning for image classification tasks.

Uploaded by

Thành Cao Đức
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
68 views20 pages

Artificial Intelligence: Alexnet

The document describes AlexNet, which was the winner of the ImageNet LSVRC-2012 competition for image classification. AlexNet used a neural network architecture with 5 convolutional layers and 3 fully-connected layers. It achieved top-5 and top-1 error rates on the ImageNet test set that were significantly lower than other models at the time, demonstrating the effectiveness of deep learning for image classification tasks.

Uploaded by

Thành Cao Đức
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 20

Artificial Intelligence

AlexNet

Pham Viet Cuong


Dept. Control Engineering & Automation, FEEE
Ho Chi Minh City University of Technology
AlexNet

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 2
AlexNet
✓ Winner of ImageNet LSVRC-2012
✓ Image classification problem
❖ 1000 classes
❖ 650K neurals
❖ 62M parameters
❖ Training: 1.2M images
❖ Validation: 50K images
❖ Test: 150K images

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 3
Architecture
✓ 5 convolutional layers
✓ 3 fully-connected layers
✓ Softmax

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 4
Architecture
✓ Training on Multiple GPUs
❖ Smaller error

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 5
Architecture
✓ Pooling
❖ Overlapping pooling
❖ Window: 3x3
❖ Stride: 2

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 6
Architecture
✓ Activation function
❖ ReLU
❖ ReLU vs tanh

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 7
Overfitting
✓ Data augmentation
❖ Without data augmentation: substantial overfitting
❖ Image translation & horizontal reflection
➢ 256x256 image → 224x224 patches
➢ 2048 times
❖ Altering intensities of RGB channels
✓ Test time: 1 images → 5 patches → 10 patches (reflection)

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 8
Overfitting
✓ Dropout
❖ Without dropout: substantial overfitting
❖ Used in the first two fully-connected layers
❖ p = 0.5
❖ Roughly doubles # iterations

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 9
Training
✓ Stochastic gradient descent
✓ Actually: mini-batch gradient descent
✓ With momentum
✓ Batch size: 128
❖ Momentum: 0.9
❖ Weight decay: 0.0005 (for regularization?)

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 10
Results
✓ Example:

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 11
Training

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 12
Results
✓ Evaluation criteria
❖ Top-1 & top-5 error rate
✓ ILSVRC-2010

✓ ILSVRC-2012

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 13
Results
✓ ImageNet 2009
❖ 10.184 classes
❖ 8.9M images
❖ 6th convolutional layer added
Model Top 1 error rate (%) Top 5 error rate (%)
AlexNet 67.4 40.9
Other 78.1 60.9

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 14
Results

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 15
Results
✓ Kernel: 96, 11x11x3

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 16
Results

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 17
Results

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 18
Results

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 19
Results

Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 20

You might also like