CS294-158 Deep Unsupervised Learning

Lecture 1 Intro: Logistics and Motivation

Pieter Abbeel, Xi (Peter) Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan
UC Berkeley
 Instructor Team

Pieter Abbeel Xi (Peter) Chen Jonathan Ho Aravind Srinivas Alex Li Wilson Yan

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 2
 Communication
● Website: https://sites.google.com/view/berkeley-cs294-158-sp20/home

● Announcements
○ Piazza -- sign up today!

● Questions
○ Piazza (preferred!)
○ [email protected]

● Office hours: [all starting next week]

○ Pieter: Thu 5-6pm -- 242 Sutardja Dai Hall
○ Alex: Mon 5-6pm, Tue 11-noon -- 326 Soda Hall
○ Wilson: Wed noon-1pm, Fri 2-3pm -- 347 Soda Hall
For homework, TA office hours are the best venue.
For other questions (lecture, final project, research, etc.) any office hours should be great fits

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 3
 Admission into the Course
■ Application: see website

■ We’ll review applications before the end of the weekend!

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 4
 Syllabus
Week 1 (1/22) Intro
Week 2 (1/29) Autoregressive Models
Week 3 (2/5) Flow Models
Week 4 (2/12) Latent Variable Models
Week 5 (2/19) Implicit Models / Generative Adversarial Networks
Week 6 (2/26) Implicit Models / Generative Adversarial Networks (ctd) + Final Project Discussion
Week 7 (3/4) Self-Supervised Learning / Non-Generative Representation Learning
Week 8 (3/11) Self-Supervised Learning / Non-Generative Representation Learning
Week 9 (3/18) Strengths and Weaknesses of Unsupervised Learning Methods
Spring Break Week (no lecture)
Week 10 (4/1) Semi-Supervised Learning; Unsupervised Distribution Alignment
Week 11 (4/8) Compression
Week 12 (4/15) Language Models
Week 13 (4/22) Midterm
Week 14 (4/29) Representation Learning in Reinforcement Learning
Week 15 (5/6) RRR Week (no lecture)
Week 16 (5/13) Final Project Presentations + Final Project Reports due

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 5
 Homework
■ HW1: Autoregressive Models (out 1/29, due 2/11)

■ HW2: Flow Models (out 2/12, due 2/25)

■ HW3: Latent Variable Models (out 2/26, due 3/10)
■ HW4: Implicit Models / GANs (out 3/11, due 3/31)

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 6
 Homework Policy
■ Collaboration: Students may discuss assignments. However, each student must code up and write up their
solutions independently.

■ Late assignments: Recognizing that students may face unusual circumstances and require some flexibility in the
course of the semester, each student will have a total of 7 free late (calendar) days to use as s/he sees fit, but no
more than 4 late days can be used on any single assignment. Late days are counted at the granularity of days:
e.g., 3 hours late is one late day.

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 7
 Midterm
■ Date: 4/22 (during lecture slot)
■ Topics: everything covered through (and including) 4/15
■ Format: we will provide a document with questions and answers ahead of time (~20)
■ Rationale: opportunity to force yourself to fully internalize key derivations and concepts

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 8
 Final Project
■ SCOPE:
■ Goal: explore and push the boundaries in unsupervised learning.
■ E.g. proposal+evaluation of new algorithms / architectures, investigation of an application of unsupervised learning,
benchmarking unsupervised learning, compression, studying synergies between unsupervised learning and other types
of learning, etc.
■ Ideally, the project covers interesting new ground and might be the foundation for a future conference paper
submission or product.

■ PROJECT TOPICS / STAFF INPUT:

■ We encourage trying to come up with your own project idea. We are also happy to make suggestions and/or
brainstorm ideas together.
■ One of the main reasons we are so excited to teach this class is to see more Deep Unsupervised Learning projects
happen. We are very excited to advise on your projects, please don't hesitate to come to office hours to discuss project
ideas, project progress, ideas for next steps, etc.

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 9
 Final Project -- Timeline
- March 2nd Project Proposals Due: 1 page description of project + goals for milestone. --
Submission through google doc shared with instructors, so we can give feedback/suggestions most
easily.

- March 9th Approved Project Proposals Due: by this time your proposals should have incorporated
instructor feedback, at this stage it should be assured that your proposal is of right fit and scope

- April 13th 3-Page Milestone Due: This is to make sure you are indeed making progress on the
project and an opportunity to get feedback on your progress thus far, as well as on any revisions you
might want to propose to your project goals. Expectation is that you report on some initial experimental
findings (or if you are doing something purely theoretical, some initial progress on that front). --
Submission through google doc shared with instructors, so we can give feedback/suggestions most
easily.

- May 13th Project Presentations: 250 SDH, 5-8pm (same as lecture slot)

- May 15th 6-Page Final Project Reports Due

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 10
 Grading Logistics
■ 60% Homework
■ 10% Midterm
■ 30% Final Project

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 11
 Do we need to attend class?
■ No hard requirement
■ BUT: very highly recommended
■ Great opportunity to get to know other students at Berkeley embarking
on Deep Unsupervised Learning
■ Pizza!

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 12
 WARNING
Second offering of this course

There will be some rough edges, please bear with us

+ give feedback!

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 13
 What is Deep Unsupervised Learning?
■ Capturing rich patterns in raw data with deep networks in a
label-free way
■ Generative Models: recreate raw data distribution
■ Self-supervised Learning: “puzzle” tasks that require semantic understanding
■ But why do we care?

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 14
 Geoffrey Hinton
(in his 2014 AMA on Reddit)

“The brain has about 10^14 synapses and we only live for about 10^9
seconds. So we have a lot more parameters than data. This motivates
the idea that we must do a lot of unsupervised learning since the
perceptual input (including proprioception) is the only place we can get
10^5 dimensions of constraint per second.”

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 15
 LeCake

Yann LeCun

Need tremendous
amount of
information to build
machines that have
common sense and
generalize
[LeCun-20161205-NeurIPS-keynote]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 16
 “Ideal Intelligence”
“Ideal Intelligence” is all about compression (finding all patterns)
■ Finding all patterns = short description of raw data (low Kolmogorov Complexity)
■ Shortest code-length = optimal inference (Solomonoff Induction)
■ Extensible to optimal action making agents (AIXI)

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 17
 Aside from theoretical interests
■ Deep Unsupervised Learning has many powerful applications
■ Generate novel data
■ Conditional Synthesis Technology (WaveNet, GAN-pix2pix)
■ Compression
■ Improve any downstream task with un(self)supervised pre-training
■ Production level impact: Google Search powered by BERT
■ Flexible building blocks

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 18
 Generate Images

[Deep Belief Nets, Hinton, Osindero, Teh, 2006]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 19
 Generate Images

[VAE, Kingma and Welling, 2013]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 20
 Generate Images

[GAN, Goodfellow et al. 2014]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 21
 Generate Images

[DCGAN, Radford, Metz, Chintala 2015]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 22
 Generate Images

[DCGAN, Radford, Metz, Chintala 2015]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 23
 Generate Images

[Ledig, Theis, Huszar et al, 2017]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 24
 Generate Images

[CycleGAN: Zhu, Park, Isola & Efros, 2017]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 25
 Generate Images

[BigGAN, Brock, Donahue, Simonyan, 2018]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 26
 Generate Images

[StyleGAN, Karras, Laine, Aila, 2018]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 27
 Generate Audio

Parametric WaveNet

[WaveNet, Oord et al., 2018]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 28
 Generate Video

DVD-GAN: Adversarial Video Generation on Complex Datasets, Clark, Donahue, Simonyan, 2019

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 29
 Generate Text
PANDARUS:
Alas, I think he shall be come approached and the day
When little srain would be attain'd into being never fed,
And who is but a chain and subjects of his death,
I should not sleep.

Second Senator:
They are away this miseries, produced upon my soul,
Breaking and strongly should be buried, when I perish
The earth and thoughts of many states.

DUKE VINCENTIO:
Well, your wit is in the care of side and that.

[Char-rnn, karpathy, 2015]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 30
 Generate Math
\begin{proof}
We may assume that $\mathcal{I}$ is an abelian sheaf on
$\mathcal{C}$.
\item Given a morphism $\Delta : \mathcal{F} \to
\mathcal{I}$
is an injective and let $\mathfrak q$ be an abelian sheaf on
$X$.
Let $\mathcal{F}$ be a fibered complex. Let $\mathcal{F}$
be a category.
\begin{enumerate}
\item \hyperref[setain-construction-phantom]{Lemma}
\label{lemma-characterize-quasi-finite}
Let $\mathcal{F}$ be an abelian quasi-coherent sheaf on
$\mathcal{C}$.
Let $\mathcal{F}$ be a coherent $\mathcal{O}_X$-module.
Then
$\mathcal{F}$ is an abelian catenary over $\mathcal{C}$.
\item The following are equivalent
\begin{enumerate}
\item $\mathcal{F}$ is an $\mathcal{O}_X$-module.
\end{lemma}

[Char-rnn, karpathy, 2015]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 31
 Generate Text
Prompt [human written]: A train carriage containing controlled nuclear materials was stolen in
Cincinnati today. Its whereabouts are unknown.

GPT-2: The incident occurred on the downtown train line, which runs from Covington and Ashland stations.
In an email to Ohio news outlets, the U.S. Department of Energy said it is working with the Federal
Railroad Administration to find the thief.

‘The theft of this nuclear material will have significant negative consequences on public and
environmental health, our workforce and the economy of our nation,’ said Tom Hicks, the U.S. Energy
Secretary, in a statement. ‘Our top priority is to secure the theft and ensure it doesn’t happen again.’

The stolen material was taken from the University of Cincinnati’s Research Triangle Park nuclear research
site, according to a news release from Department officials.

The Nuclear Regulatory Commission did not immediately release any information. According to the release,
the U.S. Department of Energy’s Office of Nuclear Material Safety and Security is leading that team’s
investigation.

‘The safety of people, the environment and the nation’s nuclear stockpile is our highest priority,’ Hicks
said. ‘We will get to the bottom of this and make no excuses.’

[OpenAI’s GPT2 -- Radford, Wu, Child, Luan, Amodei, Sutskever, 2019]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 32
 Generate Text
Prompt (human written): Recycling is good for the world, no, you could not be
more wrong.

GPT-2: Recycling is NOT good for the world. It is bad for the environment, it is
bad for our health, and it is bad for our economy. I’m not kidding. Recycling is
not good for the environment. It is destructive to the earth and it is a major
contributor to global warming. Recycling is not good for our health. It
contributes to obesity and diseases like heart disease and cancer. Recycling is
bad for our economy. It increases the cost of a product, and in turn, the price
of everything that is made with that product. Recycling is not good for our
nation. We pay a tremendous price for the privilege of having the world’s most
advanced and efficient recycling system. Recycling is a huge, colossal waste of
time, energy, money, and resources.

[OpenAI’s GPT2 -- Radford, Wu, Child, Luan, Amodei, Sutskever, 2019]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 33
 Try it yourself

https://talktotransformer.com/

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 34
 Compression - Lossless

Generative models provide better

bit-rates than distribution-unaware
compression methods like JPEG, etc.

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 35
 Compression - Lossy

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 36
Downstream Task - Sentiment Detection

[Radford et al., 2017]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 37
Downstream Tasks - NLP (BERT Revolution)

[https://gluebenchmark.com/leaderboard]

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 38
Downstream Tasks - Vision (Contrastive)

"If, by the first day of autumn (Sept 23) of

2015, a method will exist that can match or
beat the performance of R-CNN on Pascal
VOC detection, without the use of any extra,
human annotations (e.g. ImageNet) as
pre-training, Mr. Malik promises to buy Mr.
Efros one (1) gelato (2 scoops: one chocolate,
one vanilla)."

Table: Data-Efficient Image Recognition using CPC

(Henaff, Srinivas, et al)
UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 39
 Summary
■ Unsupervised Learning: Rapidly advancing field thanks to compute; deep learning
engineering practices; datasets; lot of people working on it.
■ Not just an academic interest topic. Production level impact [example: BERT is in
use for Google Search and Assistant].
■ What is true now may not be true even a year from now [example:
self-supervised pre-training was way worse than supervised in computer vision
tasks like detection/segmentation last year. Now it is better].
■ Language Modeling (GPT), Image Generation (conditional GANs), Language
pre-training (BERT), vision pre-training (CPC / MoCo) starting to work really well.
Good time to learn these well and make very impactful contributions.
■ Autoregressive Density Modeling, Flows, VAEs, UL for RL, etc have huge room for
improvement. Great time to work on them.

UC Berkeley -- Spring 2020 -- Deep Unsupervised Learning -- Pieter Abbeel, Peter Chen, Jonathan Ho, Aravind Srinivas, Alex Li, Wilson Yan -- L1 Intro 40