Deeplearning - Ai Deeplearning - Ai
Deeplearning - Ai Deeplearning - Ai
DeepLearning.AI makes these slides available for educational purposes. You may not use or distribute
these slides for commercial purposes. You may make copies of these slides and use or distribute them for
educational purposes as long as you cite DeepLearning.AI as the source of the slides.
(4,3)
Vectors
4 (4,3)
3
Vectors
4 (4,3)
3
Vectors
4 (4,3)
(4,3,1)
3
Vectors
4 (4,3)
(4,3,1)
3 3
4
1
How to get from point A to point B?
How to get from point A to point B?
4

How to get from point A to point B?
4


How to get from point A to point B?
4


How to get from point A to point B?
4



How to get from point A to point B?
4


How to get from point A to point B?
4


How to get from point A to point B?
4



How to get from point A to point B?
4


How to get from point A to point B?
4


How to get from point A to point B?
Helicopter distance = 32 + 42 = 5
3
4



How to get from point A to point B?
2
b
2 +
a b
Helicopter distance = 32 + 42 = 5
3 a
Pythagorean Theorem
4



Norms
(a,b)
Norms
(a,b)
L1-norm = | (a, b) |1 = | a | + | b |

Norms
(a,b)
(4,3)
u
Norm of a vector
4 (4,3)
u 3
Norm of a vector
4 (4,3)
u 3
42 + 32 = 25 = 5
Direction of a vector
(4,3)
u 3
4
Direction of a vector
(4,3)
u 3
θ
4
Direction of a vector
(4,3)
3
u tan(θ) =
3 4
θ
4
Direction of a vector
(4,3)
3
u tan(θ) =
3 4
θ θ = arctan(3/4) = 0.64
4
Direction of a vector
(4,3)
3
u tan(θ) =
3 4
θ θ = arctan(3/4) = 0.64 = 36.87∘
4
Vectors and Linear Transformations
u (4,1)
Sum of vectors
(1,3)
v
u (4,1)
Sum of vectors
(1,3)
v
u (4,1) u + v = (4 + 1,1 + 3) = (5,4)
Sum of vectors
(5,4)
(1,3) v
v u+
u (4,1) u + v = (4 + 1,1 + 3) = (5,4)
Sum of vectors
(5,4)
(1,3) v
v u+
u (4,1) u + v = (4 + 1,1 + 3) = (5,4)
Difference of vectors
(1,3)
v
u (4,1)
Difference of vectors
(1,3)
v
u (4,1) u − v = (4 − 1,1 − 3) = (3, − 2)
Difference of vectors
(1,3)
v
u (4,1) u − v = (4 − 1,1 − 3) = (3, − 2)
Difference of vectors
(1,3)
v
u (4,1) u − v = (4 − 1,1 − 3) = (3, − 2)
Difference of vectors
(1,3)
v
u (4,1) u − v = (4 − 1,1 − 3) = (3, − 2)
(3,-2)
Vectors and Linear Transformations
v
u (6,2)
Distances
(1,5)
u−
v=
(5,
v −3
)
u (6,2)
Distances
(1,5)
u−
v= | u − v |1 = | 5 | + | − 3 | = 8
(5, L1-distance
v −3
)
u (6,2)
Distances
(1,5)
u−
v= | u − v |1 = | 5 | + | − 3 | = 8
(5, L1-distance
v −3
)
u (6,2)
| u − v |2 = 52 + 32 = 5.83
L2-distance
Distances
(1,5)
| u − v |1 = | 5 | + | − 3 | = 8
L1-distance
v
u (6,2)
| u − v |2 = 52 + 32 = 5.83
L2-distance
Distances
(1,5)
| u − v |1 = | 5 | + | − 3 | = 8
L1-distance
v
u (6,2)
| u − v |2 = 52 + 32 = 5.83
L2-distance
Distances
(1,5)
| u − v |1 = | 5 | + | − 3 | = 8
L1-distance
v
θ u (6,2)
| u − v |2 = 52 + 32 = 5.83
L2-distance
Distances
(1,5)
| u − v |1 = | 5 | + | − 3 | = 8
L1-distance
v
θ u (6,2)
| u − v |2 = 52 + 32 = 5.83
L2-distance
cos(θ)
Cosine distance

Vectors and Linear Transformations
Multiplying a vector by a
scalar
Multiplying a vector by a scalar
Multiplying a vector by a scalar
u (1,2)
Multiplying a vector by a scalar
u = (1,2)
u (1,2)
Multiplying a vector by a scalar
u = (1,2)
λ=3
u (1,2)
Multiplying a vector by a scalar
u = (1,2)
λ=3
u (1,2)
λu = (3,6)
Multiplying a vector by a scalar
3u (3,6)
u = (1,2)
λ=3
u (1,2)
λu = (3,6)
If the scalar is negative
If the scalar is negative
u = (1,2)
If the scalar is negative
u = (1,2)
λ =−2
If the scalar is negative
u = (1,2)
λ =−2
λu = (−2, − 4)
If the scalar is negative
u = (1,2)
λ =−2
λu = (−2, − 4)
−2u
(-2,-4)
If the scalar is negative
u = (1,2)
λ =−2
u (1,2)
λu = (−2, − 4)
−2u
(-2,-4)
Vectors and Linear Transformations
6 + 20 + 2 = 28




A shortcut for linear operations
Quantities Prices
2 apples apples: $3 Total price
4 bananas bananas: $5 $28
1 cherry cherries: $2
2 x $ 3 = 2×3=6
4 x $ 5 = 4 × 5 = 20
1 x $ 2 = 1×2=2
6 + 20 + 2 = 28




The dot product
2 $ 3
4 . $ 5 = $28
1 $ 2
The dot product
2 $ 3
4 . $ 5 = $28
1 $ 2
2 ⋅ 3 + 4 ⋅ 5 + 1 ⋅ 2 = 28

The dot product
$ 3
2 4 1 . $ 5 = $28
28
$ 2
2 ⋅ 3 + 4 ⋅ 5 + 1 ⋅ 2 = 28

The dot product
3
2 4 1 . 5 = 28
2
2 ⋅ 3 + 4 ⋅ 5 + 1 ⋅ 2 = 28

Norm of a vector using dot product
(4,3)
u
Norm of a vector using dot product
4 (4,3)
u 3
Norm of a vector using dot product
42 + 32 = 25 = 5
4 (4,3)
u 3
Norm of a vector using dot product
42 + 32 = 25 = 5
4 (4,3)
u 3 4 3
4
= 25
3
Norm of a vector using dot product
42 + 32 = 25 = 5
4 (4,3)
u 3 4 3
4
= 25
3
L2 − norm = dot0product(u, u)
Norm of a vector using dot product
42 + 32 = 25 = 5
4 (4,3)
u 3 4 3
4
= 25
3
L2 − norm = dot0product(u, u)
| u |2 = ⟨u, u⟩
Vectors and Linear Transformations
(-1,3)
(6,2)
u v
Orthogonal vectors have dot product 0
(-1,3)
(6,2)
u v 6 2
Orthogonal vectors have dot product 0
(-1,3)
(6,2) -1
u v 6 2
3
Orthogonal vectors have dot product 0
(-1,3)
(6,2) -1
= 0
u v 6 2
3
Orthogonal vectors have dot product 0
(-1,3)
(6,2) -1
= 0
u v 6 2
3
⟨u, v⟩ = 0
The dot product
The dot product
u
The dot product
u
⟨u, u⟩ = | u |2
The dot product
u
⟨u, u⟩ = | u |2
v u
The dot product
u
⟨u, u⟩ = | u |2
v u ⟨u, v⟩ = 0
The dot product
u
⟨u, u⟩ = | u |2
v u ⟨u, v⟩ = 0
v u
The dot product
u
⟨u, u⟩ = | u |2
v u ⟨u, v⟩ = 0
v u
⟨u, v⟩ = ?
The dot product
The dot product
u
The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
u
The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v
u
The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v ⟨u, v⟩ = | u | ⋅ | v |
u
The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v ⟨u, v⟩ = | u | ⋅ | v |
u
u
The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v ⟨u, v⟩ = | u | ⋅ | v |
u
u v
The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v ⟨u, v⟩ = | u | ⋅ | v |
u
u v
u′

The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v ⟨u, v⟩ = | u | ⋅ | v |
u
u v ⟨u, v⟩ = | u′| ⋅ | v |
u′


The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v ⟨u, v⟩ = | u | ⋅ | v |
u
u v
θ ⟨u, v⟩ = | u′| ⋅ | v |
u′


The dot product
u ⟨u, u⟩ = | u |2 = | u | ⋅ | u |
v ⟨u, v⟩ = | u | ⋅ | v |
u
u v
θ ⟨u, v⟩ = | u′| ⋅ | v |
u′ = | u | | v | cos(θ)


Geometric dot product
Geometric dot product
(6,2)
Geometric dot product
(-1,3)
(6,2)
-1
6 2
3
= 0
Geometric dot product
2
(2,4) 6 2
4
= 20
(-1,3)
(6,2)
-1
6 2
3
= 0
Geometric dot product
2
(2,4) 6 2
4
= 20
(-1,3)
(6,2)
-1
(-4,1) 6 2 = 0
3
-4
6 2
1
= -22
Geometric dot product
2
(2,4) 6 2
4
= 20 Positive
(-1,3)
(6,2)
-1
(-4,1) 6 2 = 0
3
-4
6 2
1
= -22
Geometric dot product
2
(2,4) 6 2
4
= 20 Positive
(-1,3)
(6,2)
-1
(-4,1) 6 2 = 0
+ 3
-4
6 2
1
= -22
Geometric dot product
2
(2,4) 6 2
4
= 20 Positive
(-1,3)
(6,2)
-1
(-4,1) 6 2 = 0
+ 3
-4
6 2
1
= -22 Negative
Geometric dot product
2
(2,4) 6 2
4
= 20 Positive
(-1,3)
(6,2)
-1
(-4,1) 6 2 = 0
+ 3
-
-4
6 2
1
= -22 Negative
Geometric dot product
u

Geometric dot product
u

Geometric dot product
u
⟨u, v⟩ = 0
⟨u, v⟩ > 0
u
⟨u, v⟩ = 0
⟨u, v⟩ > 0
u
⟨u, v⟩ = 0
⟨u, v⟩ < 0
Multiplying a matrix by a
vector
Equations as dot product
2a + 4b + c = 28
$ a
2 4 1 . $ b = $ 28
$ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
1 1 1
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a
1 1 1 .$ b
$ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a
1 1 1 .$ b =$ 10
$ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a
1 1 1 .$ b =$ 10 1 2 1
$ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a $ a
1 1 1 .$ b =$ 10 1 2 1 .$ b
$ c $ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a $ a
1 1 1 .$ b =$ 10 1 2 1 .$ b =$ 15
$ c $ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a $ a
1 1 1 .$ b =$ 10 1 2 1 .$ b =$ 15 1 1 2
$ c $ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a $ a $ a
1 1 1 .$ b =$ 10 1 2 1 .$ b =$ 15 1 1 2 .$ b
$ c $ c $ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a $ a $ a
1 1 1 .$ b =$ 10 1 2 1 .$ b =$ 15 1 1 2 .$ b =$ 12
$ c $ c $ c
Equations as dot product
a + b + c = 10 a + 2b + c = 15 a + b + 2c = 12
$ a $ a $ a
1 1 1 .$ b =$ 10 1 2 1 .$ b =$ 15 1 1 2 .$ b =$ 12
$ c $ c $ c
Equations as dot product
$ a
1 1 1 .$ b =$ 10
$ c
a + b + c = 10
$ a
a + 2b + c = 15 1 2 1 .$ b =$ 15
$ c
a + b + 2c = 12
$ a
1 1 2 .$ b =$ 12
$ c
Equations as dot product
$ a
1 1 1 .$ b =$ 10
$ c
a + b + c = 10
$ a
a + 2b + c = 15 1 2 1 .$ b =$ 15
$ c
a + b + 2c = 12
$ a
1 1 2 .$ b =$ 12
$ c
Equations as dot product
System of equations Matrix product
a + b + c = 10
1 1 1 $ a 10
a + 2b + c = 15 1 2 1 .$ b =$ 15
1 1 2 $ c 12
a + b + 2c = 12
Equations as dot product
System of equations Matrix product
a + b + c = 10
1 1 1 a 10
a + 2b + c = 15 1 2 1 b = 15
1 1 2 c 12
a + b + 2c = 12
Vectors and Linear Transformations
Matrices as linear
transformations
Matrices as linear transformations
3 1
1 2
Matrices as linear transformations
b
3 1
1 2
a
Matrices as linear transformations
b
3 1
1 2
a
Matrices as linear transformations
b
3 1
1 2
a
Matrices as linear transformations
b
3 1 0 0
=
1 2 0 0
(0,0) (0,0)
a
Matrices as linear transformations
b
3 1 0 0
=
1 2 0 0
(0,0) (0,0)
a
Matrices as linear transformations
b
3 1 0 0
=
1 2 0 0
(0,0) (0,0)
a
Matrices as linear transformations
b
3 1 0
1 0
3
=
1 2 0 0
1
(0,0) (0,0)
a (1,0) (3,1)
Matrices as linear transformations
b
3 1 0
1 0
3
=
1 2 0 0
1
(0,0) (0,0)
a (1,0) (3,1)
Matrices as linear transformations
b
3 1 0
1 0
3
=
1 2 0 0
1
(0,0) (0,0)
a (1,0) (3,1)
Matrices as linear transformations
b
3 1 1
0 0
3
1
=
1 2 0
1 0
1
2
(0,0) (0,0)
a (1,0) (3,1)
(0,1) (1,2)
Matrices as linear transformations
b
3 1 1
0 0
3
1
=
1 2 0
1 0
1
2
(0,0) (0,0)
a (1,0) (3,1)
(0,1) (1,2)
Matrices as linear transformations
b
3 1 1
0 0
3
1
=
1 2 0
1 0
1
2
(0,0) (0,0)
a (1,0) (3,1)
(0,1) (1,2)
Matrices as linear transformations
b
3 1 0
1 0
3
1
4
=
1 2 0
1 0
1
2
3
(0,0) (0,0)
a (1,0) (3,1)
(0,1) (1,2)
(1,1) (4,3)
Matrices as linear transformations
b
3 1 0
1 0
3
1
4
=
1 2 0
1 0
1
2
3
(0,0) (0,0)
a (1,0) (3,1)
(0,1) (1,2)
(1,1) (4,3)
Matrices as linear transformations
b
3 1 0
1 0
3
1
4
=
1 2 0
1 0
1
2
3
(0,0) (0,0)
a (1,0) (3,1)
(0,1) (1,2)
(1,1) (4,3)
Matrices as linear transformations
b
3 1 0
1 0
3
1
4
=
1 2 0
1 0
1
2
3
(0,0) (0,0)
a (1,0) (3,1)
(0,1) (1,2)
(1,1) (4,3)
Matrices as linear transformations
b
3 1 -2 -3
=
1 2 3 4
a
Matrices as linear transformations
b
3 1 -2 -3
=
1 2 3 4
a
Matrices as linear transformations
b
3 1 -2 -3
=
1 2 3 4
a
Matrices as linear transformations
b
3 1 -2 -3
=
1 2 3 4
a
Matrices as linear transformations
b
3 1 -2 -3
=
1 2 3 4
a
Matrices as linear transformations
b
3 1 -2 -3
=
1 2 3 4
a
Systems of equations as linear transformations
b
a
Systems of equations as linear transformations
b
First day: 3a + b
Second day: a + 2b
a
Systems of equations as linear transformations
b
3 1 a Day 1
=
1 2 b Day 2
First day: 3a + b
Second day: a + 2b
a
Systems of equations as linear transformations
b
3 1 1 4
=
1 2 1 3
First day: 3a + b
Second day: a + 2b
a
Systems of equations as linear transformations
b
3 1 1 4
=
1 2 1 3
First day: 3a + b
Second day: a + 2b
a
(1,1)
(a,b) (4,3)
Systems of equations as linear transformations
b
3 1 1 4
=
1 2 1 3
First day: 3a + b
Second day: a + 2b
(1,1)
a
(1,1)
(a,b) (4,3)
Systems of equations as linear transformations
b
3 1 1 4
=
1 2 1 3
First day: 3a + b
(4,3)
Second day: a + 2b
(1,1)
a
(1,1)
(a,b) (4,3)
Systems of equations as linear transformations
b
3 1 1 4
=
1 2 1 3
First day: 3a + b
(4,3)
Second day: a + 2b
(1,1)
a First day
(1,1)
(a,b) (4,3)
Systems of equations as linear transformations
b Second day
3 1 1 4
=
1 2 1 3
First day: 3a + b
(4,3)
Second day: a + 2b
(1,1)
a First day
(1,1)
(a,b) (4,3)
Vectors and Linear Transformations
Linear transformations as
matrices
Linear transformations as matrices
b b
? ?
? ?
a a
Linear transformations as matrices
b b
? ?
? ?
a a
Linear transformations as matrices
b b
? ?
? ?
a a
Linear transformations as matrices
b b
? ?
? ?
a a
(0,0) (0,0)
Linear transformations as matrices
b b
? ?
? ?
a a
(0,0) (0,0)
(1,0) (3,-1)
Linear transformations as matrices
b b
? ?
? ?
a a
(0,0) (0,0)
(1,0) (3,-1)
(0,1) (2,3)
Linear transformations as matrices
b b
? ?
? ?
a a
(0,0) (0,0)
(1,0) (3,-1)
(0,1) (2,3)
(1,1) (5,2)
Linear transformations as matrices
b
? ? 0 2
=
? ? 1 3
? ? 1 3
=
? ? 0 -1
a
(1,0) (3,-1)
(0,1) (2,3)
Linear transformations as matrices
b
? ? 0 2
=
? ? 1 3
? ? 1 3
=
? ? 0 -1
a
1 (1,0) (3,-1) 3
0 (0,1) (2,3) -1
Linear transformations as matrices
b
? ? 0 2
=
? ? 1 3
2
0 ? ? 1 3 3
=
1 ? ? 0 -1
a
1 (1,0) (3,-1) 3
0 (0,1) (2,3) -1
Linear transformations as matrices
b
? ? 0 2
=
? ? 1 3
2
0 ? ? 1 3 3
=
1 ? ? 0 -1
a
1 (1,0) (3,-1) 3
0 (0,1) (2,3) -1
Linear transformations as matrices
b
? 2? 0 2
=
? 3? 1 3
2
0 ? ?
2 1 3 3
=
1 ? ?
3 0 -1
a
1 (1,0) (3,-1) 3
0 (0,1) (2,3) -1
Linear transformations as matrices
b
? 2? 0 2
=
? 3? 1 3
2
0 ? ?
2 1 3 3
=
1 ? ?
3 0 -1
a
1 (1,0) (3,-1) 3
0 (0,1) (2,3) -1
Linear transformations as matrices
b
3
? 2? 0 2
=
-1
? 3? 1 3
2
0 3
? ?
2 1 3 3
=
1 -1
? ?
3 0 -1
a
1 (1,0) (3,-1) 3
0 (0,1) (2,3) -1
Vectors and Linear Transformations
Matrix multiplication
Combining linear transformations
Combining linear transformations
3 1
1 2
Combining linear transformations
3 1 1 3
=
1 2 0 1
Combining linear transformations
3 1 1 3
=
1 2 0 1
Combining linear transformations
3 1 1 3
=
1 2 0 1
3 1 0 1
=
1 2 1 2
Combining linear transformations
3 1 1 3
=
1 2 0 1
3 1 0 1
=
1 2 1 2
Combining linear transformations
3 1 1 3
=
1 2 0 1
3 1 0 1
=
1 2 1 2
Combining linear transformations
3 1 1 3
=
1 2 0 1
3 1 0 1
=
1 2 1 2
Combining linear transformations
3 1 1 3
=
1 2 0 1
3 1 0 1
=
1 2 1 2
Combining linear transformations
Combining linear transformations
2 -1
0 2
Combining linear transformations
2 -1 3 5
=
0 2 1 2
Combining linear transformations
2 -1 3 5
=
0 2 1 2
Combining linear transformations
2 -1 3 5
=
0 2 1 2
Combining linear transformations
2 -1 3 5
=
0 2 1 2
2 -1 1 0
=
0 2 2 4
Combining linear transformations
2 -1 3 5
=
0 2 1 2
2 -1 1 0
=
0 2 2 4
Combining linear transformations
2 -1 3 5
=
0 2 1 2
2 -1 1 0
=
0 2 2 4
Combining linear transformations
2 -1 3 5
=
0 2 1 2
2 -1 1 0
=
0 2 2 4
Combining linear transformations
2 -1 3 5
=
0 2 1 2
2 -1 1 0
=
0 2 2 4
Combining linear transformations
2 -1 3 5
=
0 2 1 2
2 -1 1 0
=
0 2 2 4
Combining linear transformations
Combining linear transformations
3 1
1 2
Combining linear transformations
3 1 2 -1
1 2 0 2
Combining linear transformations
3 1 2 -1
1 2 0 2
? ?
? ?
Combining linear transformations
3 1 2 -1
1 2 0 2
? ?
? ?
Combining linear transformations
3 1 2 -1
1 2 0 2
1
0
? ?
? ?
Combining linear transformations
3 1 2 -1
1 2 0 2
5
2
1
0
? ?
? ?
Combining linear transformations
3 1 2 -1
1 2 0 2
5
2
1
0
?
5 ?
?
2 ?
Combining linear transformations
3 1 2 -1
1 2 0 2
5
0 2
1
1
0
?
5 ?
?
2 ?
Combining linear transformations
3 1 2 -1
1 2 0 2
0
4
5
0 2
1
1
0
?
5 ?
?
2 ?
Combining linear transformations
3 1 2 -1
1 2 0 2
0
4
5
0 2
1
1
0
?
5 ?0
?
2 ?4
Combining linear transformations
2 -1 3 1 5 0
. =
0 2 1 2 2 4
Combining linear transformations
First
2 -1 3 1 5 0
. =
0 2 1 2 2 4
Combining linear transformations
Second First
2 -1 3 1 5 0
. =
0 2 1 2 2 4
Multiplying matrices
2 -1 5 0
. =
0 2 0 4
Multiplying matrices
3 1
2 -1 2 -1
1 2
2 -1 3 1
. =
0 2 1 2
3 1
0 2 0 2
1 2
Multiplying matrices
2 -1 3 1
2 5
-1
3
1
2 -1
1
2
. =
0 2 1 2
3 1
0 2 0 2
1 2
Multiplying matrices
2 -1 3 1
2 5
-1
3
1
2 0
-1
1
2
. =
0 2 1 2
3 1
0 2 0 2
1 2
Multiplying matrices
2 -1 3 1
2 5
-1
3
1
2 0
-1
1
2
. =
0 2 1 2
2
3 1
0 2 0 2
1 2
Multiplying matrices
2 -1 3 1
2 5
-1
3
1
2 0
-1
1
2
. =
0 2 1 2
2 4
3 1
0 2 0 2
1 2
Vectors and Linear Transformations
1 0 0 0 0
0 1 0 0 0
0 0 1 0 0
0 0 0 1 0
0 0 0 0 1
The identity matrix
1 0 0 0 0 a
0 1 0 0 0 b
0 0 1 0 0 c
0 0 0 1 0 d
0 0 0 0 1 e
The identity matrix
1 0 0 0 0 a a
0 1 0 0 0 b b
0 0 1 0 0 c = c
0 0 0 1 0 d d
0 0 0 0 1 e e
The identity matrix
a
b
1 0 0 0 0 c = a
d
e
1 0 0 0 0 a a
0 1 0 0 0 b b
0 0 1 0 0 c = c
0 0 0 1 0 d d
0 0 0 0 1 e e
The identity matrix
b
?
1 ?
0 x x
=
?
0 ?
1 y y
a
The identity matrix
b
?
1 ?
0 x x
=
?
0 ?
1 y y
(0,0) (0,0)
a (1,0) (1,0)
(0,1) (0,1)
(1,1) (1,1)
Vectors and Linear Transformations
Matrix inverse
Matrix inverses
Matrix inverses
Matrix inverses
3 1
1 2
Matrix inverses
3 1
1 2
Matrix inverses
3 1 ? ?
1 2 ? ?
Matrix inverses
3 1 ? ?
1 2 ? ?
Matrix inverses
3 1 ? ?
1 2 ? ?
1 0
0 1
Multiplying matrices
Multiplying matrices
a b
c d
Multiplying matrices
a b 3 1
.
c d 1 2
Multiplying matrices
a b 3 1 1 0
. =
c d 1 2 0 1
Multiplying matrices
a b 3 1 1 0
. =
c d 1 2 0 1
-1
3 1
1 2
Multiplying matrices
a b 3 1 1 0
. =
c d 1 2 0 1
-1
3 1 2/5 -1/5
=
1 2 -1/5 3/5
How to find an inverse?
a b 3 1 1 0
. =
c d 1 2 0 1
How to find an inverse?
a b 3 1 1 0
. =
c d 1 2 0 1
3
a b = 1
1
1
a b = 0
2
3
c d = 0
1
1
c d = 1
2
How to find an inverse?
a b 3 1 1 0
. =
c d 1 2 0 1
3
a b
1
= 1 3a + 1b = 1
1
a b
2
= 0 1a + 2b = 0
3
c d
1
= 0 3c + 1d = 0
1
c d
2
= 1 1c + 2d = 1




How to find an inverse?
a b 3 1 1 0
. =
c d 1 2 0 1
3 2
a b
1
= 1 3a + 1b = 1 a=
5
1 1
a b = 0 1a + 2b = 0 b=−
2 5
3 1
c d = 0 3c + 1d = 0 c=−
1 5
1 3
c d = 1 1c + 2d = 1 d=
2 5








Quiz
● Find the inverse of the following matrix. If you find that the task is
impossible, feel free to click on “I couldn’t find it”
5 2
1 2
Solution
● By solving the corresponding system of linear equations, we get the
following. 5 2 a b 1 0
. =
1 2 c d 0 1
a
5 2 = 1
c
b
5 2 = 0
d
Solution
● By solving the corresponding system of linear equations, we get the
following. 5 2 a b 1 0
. =
1 2 c d 0 1
a
5 2 = 1
c
b
5 2 = 0
d
a
1 2 = 0
c
b
1 2 = 1
d
Solution
● By solving the corresponding system of linear equations, we get the
following. 5 2 a b 1 0
. =
1 2 c d 0 1
a
5 2 = 1 • 5a + 2c = 1
c
b
5 2 = 0 • 5b + 2d = 0
d
a
1 2 = 0 • a + 2c = 0
c
b
1 2 = 1 • b + 2d = 1
d
Solution
● By solving the corresponding system of linear equations, we get the
following. 5 2 a b 1 0
. =
1 2 c d 0 1
a
5 2 = 1 • 5a + 2c = 1 • a = 1/4
c
b
5 2 = 0 • 5b + 2d = 0
d
a
1 2 = 0 • a + 2c = 0
c
b
1 2 = 1 • b + 2d = 1
d
Solution
● By solving the corresponding system of linear equations, we get the
following. 5 2 a b 1 0
. =
1 2 c d 0 1
a
5 2 = 1 • 5a + 2c = 1 • a = 1/4
c
b
5 2 = 0 • 5b + 2d = 0 • b = -1/4
d
a
1 2 = 0 • a + 2c = 0
c
b
1 2 = 1 • b + 2d = 1
d
Solution
● By solving the corresponding system of linear equations, we get the
following. 5 2 a b 1 0
. =
1 2 c d 0 1
a
5 2 = 1 • 5a + 2c = 1 • a = 1/4
c
b
5 2 = 0 • 5b + 2d = 0 • b = -1/4
d
a
1 2 = 0 • a + 2c = 0 • c = -1/8
c
b
1 2 = 1 • b + 2d = 1
d
Solution
● By solving the corresponding system of linear equations, we get the
following. 5 2 a b 1 0
. =
1 2 c d 0 1
a
5 2 = 1 • 5a + 2c = 1 • a = 1/4
c
b
5 2 = 0 • 5b + 2d = 0 • b = -1/4
d
a
1 2 = 0 • a + 2c = 0 • c = -1/8
c
b
1 2 = 1 • b + 2d = 1 • d = 5/8
d
Quiz
● Find the inverse of the following matrix. If you find that the task is
impossible, feel free to click on “I’m reaching a dead end”
1 1
2 2
Solutions
● The inverse doesn’t exist!
1 1 a b 1 0
=
2 2 c d 0 1
a+c=1
2b + 2d = 1
2a + 2c = 0
b+d=0
This is clearly a contradiction, since equation 1 says a+c=1, and equation 3 says
2a+2c=0.
Vectors and Linear Transformations
5−1 = 0.2
Which matrices have inverses?
-1
3 1 0.4 -0.2
=
1 2 -0.2 0.6
Which matrices have inverses?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25
= =
1 2 -0.2 0.6 1 2 -0.125 0.625
Which matrices have inverses?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
Which matrices have inverses?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
Non-singular matrix
Which matrices have inverses?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
Det = 5
Which matrices have inverses?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
Det = 5 Det = 8
Which matrices have inverses?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
Non-zero determinants
Which matrices have inverses?
-1 -1
3 1 0.4 -0.2 5 2 0.25 -0.25 1 1 ? ?
= = =
1 2 -0.2 0.6 1 2 -0.125 0.625 2 2 ? ?
No 0 0 0 No
Yes 0 2 2 Yes
No 0 1 1 No
No 1 0 1 No
Yes 2 2 4 Yes
Yes 2 0 2 Yes
Yes 1 2 3 Yes
Graphical natural language processing
Spam Lottery Win
Yes 1 1
Yes 2 1
No 0 0
Yes 0 2
No 0 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Graphical natural language processing
Spam Lottery Win Win
Yes 1 1
Yes 2 1
No 0 0
Yes 0 2
No 0 1
No 1 0
Yes 2 2
Yes 2 0
Lottery
Yes 1 2
Graphical natural language processing
Spam Lottery Win Win
Yes 1 1
Yes 2 1
No 0 0
Yes 0 2
No 0 1
No 1 0
Yes 2 2
Yes 2 0
Lottery
Yes 1 2
Graphical natural language processing
Spam Lottery Win Win
Yes 1 1
Yes 2 1 Line:
1 ⋅ Win + 1 ⋅ Lottery = 1.5
No 0 0
Yes 0 2
No 0 1
No 1 0
Yes 2 2
Yes 2 0
Lottery
Yes 1 2



Graphical natural language processing
Spam Lottery Win Win
Yes 1 1 Positive zone:
Yes 2 1 Line: 1 ⋅ Win + 1 ⋅ Lottery > 1.5
1 ⋅ Win + 1 ⋅ Lottery = 1.5
No 0 0
Yes 0 2
No 0 1
No 1 0
Yes 2 2
Yes 2 0
Lottery
Yes 1 2






Graphical natural language processing
Spam Lottery Win Win
Yes 1 1 Positive zone:
Yes 2 1 Line: 1 ⋅ Win + 1 ⋅ Lottery > 1.5
1 ⋅ Win + 1 ⋅ Lottery = 1.5
No 0 0
Yes 0 2
No 0 1
No 1 0
Yes 2 2
Yes 2 0 Negative zone:
Lottery
Yes 1 2 1 ⋅ Win + 1 ⋅ Lottery < 1.5









Graphical natural language processing
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Graphical natural language processing
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
2 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Graphical natural language processing
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
2 1 = 3
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Graphical natural language processing
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
2 1 = 3
No 0 1 1
Spam
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Dot product between vectors
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Dot product between vectors
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
0 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Dot product between vectors
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
0 1 = 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Dot product between vectors
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
0 1 = 1
No 0 1 1
Not spam
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Matrix multiplication
Spam Lottery Win
Yes 1 1
Yes 2 1
No 0 0 Model
Yes 0 2 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Matrix multiplication
Spam Lottery Win Prod
Yes 1 1 2
Yes 2 1 3
No 0 0 Model 0
Yes 0 2 1 = 2
No 0 1 1 1
No 1 0 1
Yes 2 2 4
Yes 2 0 2
Yes 1 2 3
Matrix multiplication
Spam Lottery Win Prod
Yes 1 1 2
Yes 2 1 3
Check: >1.5?
No 0 0 Model 0
Yes 0 2 1 = 2
No 0 1 1 1
No 1 0 1
Yes 2 2 4
Yes 2 0 2
Yes 1 2 3
Matrix multiplication
Spam Lottery Win Prod Check
Yes 1 1 2 Yes
Yes 2 1 3 Yes
Check: >1.5?
No 0 0 Model 0 No
Yes 0 2 1 = 2 Yes
No 0 1 1 1 No
No 1 0 1 No
Yes 2 2 4 Yes
Yes 2 0 2 Yes
Yes 1 2 3 Yes
Perceptrons
Spam Word1 Word2 … WordN Prod Check
Yes Yes
Yes Model Yes
Check:
No No
Yes Yes
=
No … No
No No
Yes Yes
Yes Yes
Yes Yes
Threshold and bias
Spam Lottery Win
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2
Threshold and bias
Threshold
Check
Spam Lottery Win 1 ⋅ Win + 1 ⋅ Lottery −1.5 >0
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2



Threshold and bias
Threshold
Check
Spam Lottery Win 1 ⋅ Win + 1 ⋅ Lottery −1.5 >0
1 ⋅ Win + 1 ⋅ Lottery −1.5 > 0 Bias
Yes 1 1
Yes 2 1
Model Check: > 1.5?
No 0 0
Yes 0 2 1
No 0 1 1
No 1 0
Yes 2 2
Yes 2 0
Yes 1 2






Threshold and bias
Threshold
Check
Spam Lottery Win Bias 1 ⋅ Win + 1 ⋅ Lottery −1.5 >0
1 ⋅ Win + 1 ⋅ Lottery −1.5 > 0 Bias
Yes 1 1 1
Yes 2 1 1
Model Check: > 1.5?
No 0 0 1
Yes 0 2 1 1
No 0 1 1 1
No 1 0 1
Yes 2 2 1
Yes 2 0 1
Yes 1 2 1






Threshold and bias
Threshold
Check
Spam Lottery Win Bias 1 ⋅ Win + 1 ⋅ Lottery −1.5 >0
1 ⋅ Win + 1 ⋅ Lottery −1.5 > 0 Bias
Yes 1 1 1
Yes 2 1 1
Model Check: > 1.5?
No 0 0 1
Yes 0 2 1 1
No 0 1 1 1
No 1 0 1 -1.5
Yes 2 2 1
Yes 2 0 1 Bias
Yes 1 2 1






Threshold and bias
Threshold
Check
Spam Lottery Win Bias 1 ⋅ Win + 1 ⋅ Lottery −1.5 >0
1 ⋅ Win + 1 ⋅ Lottery −1.5 > 0 Bias
Yes 1 1 1
Yes 2 1 1
Model Check: > 0?
1.5?
No 0 0 1
Yes 0 2 1 1
No 0 1 1 1
No 1 0 1 -1.5
Yes 2 2 1
Yes 2 0 1 Bias
Yes 1 2 1






The AND operator
AND x y
No 0 0
No 1 0
No 0 1
Yes 1 1
The AND operator
AND x y
No 0 0 Model
No 1 0 1
No 0 1 1
Yes 1 1
The AND operator
AND x y
No 0 0
No 1 0
No 0 1
Yes 1 1
x
The AND operator
AND x y
No 0 0
No 1 0
No 0 1
Yes 1 1
x
1 ⋅ x + 1 ⋅ y = 1.5



The perceptron
x 1
Yes
1 ⋅ x + 1 ⋅ y −1.5
1
No
y
-1.5



The perceptron
x 1
Yes
1 ⋅ x + 1 ⋅ y −1.5
1
No
y
-1.5



Vectors and Linear Transformations
Conclusion