Python Deep

Learning
Introducción práctica con
Keras y TensorFlow 2
PATC Courses | Barcelona - February 2020

Jordi TORRES.AI
1
Transparencias para
impartir docencia con
el libro #PythonDL

https://torres.ai/python-deep-learning/

2
Acerca de estas transparencias:

● Versión: 0.8 (Barcelona, 31/01/2020)

○ Borrador actual de las transparencias del libro

«Pyhon Deep Learning».
○ Algunas transparencias contienen texto en inglés. Con el tiempo
iremos «puliendo» las transparencias (pero creemos que incluso
así pueden ser usadas y por ello ya las compartimos)

3
Contenido del curso
Course content
PART 1: INTRODUCCIÓN PART 3: TÉCNICAS DEL DEEP LEARNING
PART 1: INTRODUCTION PART 3: DEEP LEARNING TECHNIQUES
1. ¿Qué es el Deep Learning? 9. Etapas de un proyecto Deep Learning
1. What is Deep Learning? 9. Stages of a Deep Learning project
2. Entorno de trabajo 10. Datos para entrenar redes neuronales
2. Work environment 10. Data to train neural networks
3. Python y sus librerías 11. Data Augmentation y Transfer Learning
3. Python and its libraries 11. Data Augmentation and Transfer Learning
12. Arquitecturas avanzadas de redes neuronales
12. Advanced neural network architectures
PART 2: FUNDAMENTOS DEL DEEP LEARNING
PART 2: FUNDAMENTALS OF DEEP LEARNING
4. Redes neuronales densamente conectadas PART 4: DEEP LEARNING GENERATIVO
4. Densely connected neural networks. PART 4: GENERATIVE DEEP LEARNING
5. Redes neuronales en Keras
5. Neural networks in Keras
13. Redes neuronales recurrentes
6. Cómo se entrena una red neuronal 13. Recurrent neural networks
6. How a neural network is trained 14. Generative Adversarial Networks
7. Parámetros e hiperparámetros en redes neuronales 14. Generative Adversarial Networks
7. Parameters and hyperparameters in neural networks
8. Redes neuronales convolucionales
8. Convolutional neural networks.

4
Recursos del libro
● Página web del libro:
https://JordiTorres.ai/python-deep-learning

● Github del libro:

https://github.com/JordiTorresBCN/python-deep-learning

● Material adicional del libro para descargar:

https://marketing.marcombo.com + código promocional del libro

5
PARTE 1: INTRODUCCIÓN
PART 1: INTRODUCTION

1. ¿Qué es el Deep Learning?

1. What is Deep Learning?

2. Entorno de trabajo
2. Work environment

3. Python y sus librerías

3. Python and its libraries

6
PARTE 1: INTRODUCCIÓN
PART 1: INTRODUCTION

1. ¿Qué es el Deep Learning?

1. What is Deep Learning?

2. Entorno de trabajo
2. Work environment

3. Python y sus librerías

3. Python and its libraries

7
La inteligencia artificial está cambiando
nuestras vidas

● ¿Estamos ante una revolución tecnológica?

8
Self-driving Tech

Foto: Jordi Torres

Foto: Jordi Torres Foto: NVIDIA

Jordi TORRES.AI 9
Foto: Jordi Torres

Jordi TORRES.AI 10
La inteligencia artificial está cambiando
nuestras vidas

● ¡Estamos ante una revolución tecnológica!

● ¿Y quizás social también? el trabajo manual está siendo

reemplazado

11
Driverless
Taxis

Source: WAYMO.COM
12
 13

Source: https://www.businessinsider.de/report-amazon-wants-3000-amazon-go-stores-by-2021-2018-9
Foto: Jordi
14 Torres
La inteligencia artificial está cambiando
nuestras vidas

● ¡Estamos ante una revolución tecnológica!

● ¿Y quizás social también? el trabajo manual está siendo

reemplazado

● ¿Puede la inteligencia artificial no solo reemplazar el

trabajo manual sino también el trabajo mental?

15
AlphaGo versus Lee Sedol
DeepMind Challenge Match - March 2016

Jordi TORRES.AI image: https://gogameguru.com/alphago-defeats-lee-sedol-game-1

16
17
AlphaStar (2019):
real-time &
hidden information

StarCraft

Jordi TORRES.AI
18
19
https://www.youtube.com/watch?time_continue=55&v=6eiErYh_FeY

20
Clases de inteligencia artificial
Artificial
Intelligence
E.g. Smart speaker, self-driving car,
ANI
web search, …
Artificial Narrow
Intelligence

AGI Do anything a human can do

Artificial General
Intelligence
21
Clases de inteligencia artificial
Artificial
Intelligence
E.g. Smart speaker, self-driving car,
ANI
web search, …
Artificial Narrow We are still
Intelligence very far from AGI

AGI Do anything a human can do

Artificial General
Intelligence
22
 Natural
Speech Language Computer
Recognition Processing Vision

23
 Speech
Recognition

Jordi TORRES.AI Credits: https://www.yahoo.com/tech/battle-of-the-voice-assistants-siri-cortana-211625975.html

24
 Natural
Language
Processing

25
 Computer
Vision

Jordi TORRES.AI FOTO: NVIDIA 26

IA + ML + DL Any technique which
enables computers to
mimic human behavior

AI techniques that give

computers the ability to
learn without being
explicitly programmed to
do so

A subset of ML which make

the computation of multi-
layer neural networks
feasible
27
Machine Learning
Machine Learning es un amplio campo con una compleja
taxonomía de algoritmos que se agrupan, en general, en tres
grandes categorías:

○ aprendizaje supervisado (supervised learning),

○ aprendizaje no supervisado (unsupervised learning) y

○ aprendizaje por refuerzo (reinforcement learning).

28
¿Qué es el Deep Learning?
● Neural networks?

Dibujo: Jordi Torres

29
Redes neuronales artificiales

30
Redes neuronales básicas
● Perceptrón multicapa (MLP)
Multi-layer perceptron: capítulo 4.

● Redes neuronales convolucionales (CNN)

Convolutional Neural Networks: capítulo 8.

● Redes neuronales recurrentes (RNN)

Recurrent Neural Networks: capítulo 13.

31
Deep Learning: ¿Cómo aprende?

Faces

32
Deep Learning: ¿Cómo aprende?

Not Faces

33
Foto: Jordi Torres
34
Foto: Jordi Torres

35
La inteligencia artificial y las redes
neuronales, ¡no son conceptos nuevos!

36
¿Por qué ahora?

COMPUTING POWER
is the real enabler!

37
El «entrenamiento» de redes neuronales tiene
una insaciable demanda de computación

source: cs231n.stanford.edu/slides/2017/cs231n_2017_lecture15.pdf 38
Crecimiento exponencial de la capacidad
de computación

Fuente: https://blog.openai.com/ai-and-compute/ [Consultado: 21/11/2019]. 39

Thanks to advances in Computer Architecture,
nowadays we can solve problems that would
have been intractable some years ago.

40 40
 “my” computer

Credits: http://www.ithistory.org/sites/default/files/hardware/facom230-50.jpg
1982
FACOM 230 – Fujitsu
few Mips * (M = 1.000.000)
1 Processor

Jordi TORRES.AI 41
 “my” computer

2012
MARENOSTRUM III - IBM
1.000.000.000 MFlops
6046 processors (48448 cores)

Jordi TORRES.AI 42
 “my” computer

2012
MARENOSTRUM III - IBM
1.000.000.000 MFlops
6046 processors (48448 cores)

only 1.000.000.000 times faster

Jordi TORRES.AI 43
2012

44
2018 2012
CLUSTER-CTE
MARENOSTRUM IV

Jordi TORRES.AI 45
¡Necesito comprarme el Marenostrum!
● ¿Qué pasa si no tengo este hardware?

Now we are entering into an era of

computation democratization for
companies !

46
 Democratization of the
computation: Cloud

Jordi TORRES.AI 47
Source: http://www.google.com/about/datacenters/gallery/images
 big
Data Centers

28.000 m2

Jordi TORRES.AI 48
Credits: http://datacenterfrontier.com/server-farms-writ-large-super-sizing-the-cloud-campus/
 28.000 m2?
Foto: Google

Jordi TORRES.AI
Foto: Google
49
Maps
 Foto: Google

28.000 m2
Jordi TORRES.AI 50
 Foto: Google
Foto: Google

28.000 m2
Jordi TORRES.AI 51
 Telefonica Madrid: 65.700 metros cuadrados

Foto: https://www.businessinsider.es/nadie-quiere-tener-centro-datos-propio-espana-403167
52
 Foto: Google

Jordi TORRES.AI
53
54
And Google ...

55
AIaaS

Microsoft Azure Cogeco Peer 1

IBM Cloud Penguin Computing
Aliyun RapidSwitch
Cirrascale Rescale
NIMBIX SkyScale
Outscale SoftLayer
... ...
56
¿Por qué ahora?
INFORMATICS

ALGORITHMS

DATA
COMPUTERS

57
¿Por qué ahora?

The triumvirate
BIG
fueling AI BIG DATA
ALGORITHMS

BIG
COMPUTERS

58
The triumvirate
fueling AI

BIG
BIG DATA
ALGORITHMS

BIG
COMPUTERS

59
One of the key drivers:
The data deluge

Thanks to the advent of Big Data

AI models can be “trained” by
exposing them to large data sets
that were previously unavailable.

60 60
Source:http://www.economist.com/node/15579717
Cada día se generan …
● 500 million tweets are sent

● 294 billion emails are sent

● 4 petabytes of data are created on Facebook

● 4 terabytes of data are created from each connected car

● 65 billion messages are sent on WhatsApp

● 5 billion searches are made

Source: https://www.visualcapitalist.com/how-much-data-is-generated-each-day/

61
1 Gigabyte (GB) = 1.000.000.000 byte
1 Terabyte (TB) = 1.000 Gigabyte (GB)
1 Petabyte (PB) = 1.000.000 Gigabyte (GB)
1 Exabyte (EB) = 1.000.000.000 Gigabyte (GB)
1 Zettabyte (ZB) = 1.000.000.000.000 (GB)

62
Open Data

63
Open Data

64
The triumvirate
fueling AI

BIG
BIG DATA
ALGORITHMS

BIG
COMPUTERS

65
An open-source world for the Deep
Learning community

Few years ago, doing Deep Learning

required significant C++ and CUDA
expertise. Nowadays, basic Python skills
are enough.

66
An open-source world for the Deep
Learning community:

67
“open-publication ethics”:

68
Clausura:
● Despite all the manifest progress in certain areas, such as
speech recognition, language translation, and voice
recognition. Deep learning still isn’t any kind of universal
solvent. Nor does it have much to do with general
intelligence, of the sort we need for open systems

● In particular, it faces some core problems:

○ Deep Learning is gready
○ Deep Learning is opaque

69
PARTE 1: INTRODUCCIÓN
PART 1: INTRODUCTION

1. ¿Qué es el Deep Learning?

1. What is Deep Learning?

2. Entorno de trabajo
2. Work environment

3. Python y sus librerías

3. Python and its libraries

70
Entorno de trabajo

71
72
73
74
75
76
77
78
TensorFlow is the world’s most popular
open source machine learning library.

79
Since its initial release in 2015, the Google
Brain product has been downloaded

80
TensorFlow 2.0
● During 2019 TensorFlow Dev Summit, Google announced
a major upgrade on the framework
à the TensorFlow 2.0 Alpha version.

● TensorFlow 2.0 focuses on simplicity and ease of use:

○ with updates like eager execution
○ flexible model building on any platform
○ intuitive higher-level APIs

81
• was developed and
maintained by François
Chollet

• https://keras.io

• Python library on top of

TensorFlow, Theano or CNTK.

82
• Good for
beginners

• Austerity and
simplicity

• Keras model is a
combination of
discrete elements

83
TensorFlow
Ecosystem

TensorFlow provides a
collection of workflows to
develop and train models
using Python, JavaScript,
or Swift, and to easily
deploy in the cloud, on-
prem, in the browser, or
on-device no matter what
language you use.

84
TensorFlow Ecosystem

85
TensorFlow Ecosystem

86
TensorFlow en Colab

87
Keras en Colab

88
89
90
Contexto de ejecución de Colab

91
92
PARTE 1: INTRODUCCIÓN
PART 1: INTRODUCTION

1. ¿Qué es el Deep Learning?

1. What is Deep Learning?

2. Entorno de trabajo
2. Work environment

3. Python y sus librerías

3. Python and its libraries

93
94