Scribd
Upload
95 views

Computational Infrastructure: Sweta Agrawal

This document discusses computational infrastructure for deep learning. It introduces frameworks like PyTorch and TensorFlow that allow for quick development and testing of ideas using automatic gradient computation and GPU acceleration without deep knowledge of deep learning. Computational graphs are used to represent operations as nodes and flows, enabling forward and backward propagation. PyTorch is highlighted as a framework that uses tensors and computational graphs to perform the forward pass, backward pass, and update weights in deep learning models. Visualization tools like Visdom and Tensorboard are also mentioned.

Uploaded by

Zu Ki
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
95 views

Computational Infrastructure: Sweta Agrawal

This document discusses computational infrastructure for deep learning. It introduces frameworks like PyTorch and TensorFlow that allow for quick development and testing of ideas using automatic gradient computation and GPU acceleration without deep knowledge of deep learning. Computational graphs are used to represent operations as nodes and flows, enabling forward and backward propagation. PyTorch is highlighted as a framework that uses tensors and computational graphs to perform the forward pass, backward pass, and update weights in deep learning models. Visualization tools like Visdom and Tensorboard are also mentioned.

Uploaded by

Zu Ki
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 18

Computational Infrastructure

Sweta Agrawal
Slide Credits: https://xkcd.com/1838/
Frameworks!
● Quick to develop and test new ideas
● Automatically compute gradients
● Use GPUs to speed up computation
● Don’t need to know deep learning???

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Computational graphs
● Graph construction
● Forward propagation
○ Loop over nodes in topological order
○ Compute the value of the node given its inputs
○ Given my inputs, make a prediction (i.e. “error” vs. “target output”)
● Backward propagation
○ Loop over the nodes in reverse topological order, starting with goal node
○ Compute derivatives of final goal node value wrt each edge’s tail node
○ How does the output change with small change to inputs?
Computational graphs
import numpy as np
np.random.seed(0)
N, D = 3, 4

x = np.random.randn(N, D)
y = np.random.randn(N, D)
z = np.random.randn(N, D)

a=x*y
b=a+z
c = np.sum(b)

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Computational graphs
import torch

N, D = 3, 4

x = torch.randn(N, D)
y = torch.randn(N, D)
z = torch.randn(N, D)

a=x*y
b=a+z
c = torch.sum(b)

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Computational graphs
Computational graphs

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Pytorch
Create random tensors for
data and weights

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Pytorch

Forward pass

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Pytorch

Backward pass

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Pytorch

Forward pass

Backward pass

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Pytorch

Update weights

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Visualization Tools

Visdom Tensorboard
Pytorch Vs Tensorflow

Slide Credits: Fei-Fei Li & Justin Johnson & Serena Yeung


Colab

https://github.com/Hvass-Labs/TensorFlow-Tutorials
Google Cloud Credits
No office hour today!

You might also like