Scribd
Upload
59 views14 pages

CS230: Lecture 3: The Mathematics of Deep Learning

This document summarizes a lecture on the mathematics of deep learning, covering three topics: backpropagation, initializations, and regularization. Backpropagation is explained for logistic regression with one training example and a batch of examples. Different initializations are discussed along with proving that the goal is for the mean of the activations to be zero in each layer. Regularization helps prevent overfitting.

Uploaded by

Sarah Eharot
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
59 views14 pages

CS230: Lecture 3: The Mathematics of Deep Learning

This document summarizes a lecture on the mathematics of deep learning, covering three topics: backpropagation, initializations, and regularization. Backpropagation is explained for logistic regression with one training example and a batch of examples. Different initializations are discussed along with proving that the goal is for the mean of the activations to be zero in each layer. Regularization helps prevent overfitting.

Uploaded by

Sarah Eharot
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

CS230:

Lecture 3
The mathematics of deep learning
Backpropagation, Initializations, Regularization

Kian Katanforoosh
I – Backpropagation
II – Initializations
III – Regularization
I - Backpropagation
Problem statement
A – Logistic Regression backpropagation for one training example
B – Logistic Regression backpropagation for a batch of m examples
Question: You have trained an animal classifier. Can you tell what part of
the input led to this prediction?
II - Initializations
Problem statement
In class, you’ve seen that:
The goal:
[%]
1
Let’s prove that: !"# " [%&']
= !"# " [%]
→ !"# + =
-[%&']
[%]
1
Let’s prove that: !"# " [%&']
= !"# " [%]
→ !"# + =
-[%&']

2 345
% %&'
Checkpoint: !"# "[%] = !"# . [%] = !"# / +',1 "1
16'
We’ve shown that for every layer l: !"# "[%] = -[%&'] !"# + [%] !"# "[%&']

You might also like