lecture5
lecture5
Pekka Marttinen
Aalto University
February, 2025
1 , 0, . . . , 0)T
zn = (0, . . . , 0, |{z}
k th elem.
De…ne
K K
p (zn ) = ∏ πkz nk
and p (xn jzn ) = ∏ N (xn jµk , Σk )z nk
k =1 k =1
Then the marginal distribution p (xn ) is a GMM:
K
p (xn ) = ∑ πk N (xn jµk , Σk )
k =1
Pekka Marttinen (Aalto University) Probabilistic Machine Learning February, 2025 3 / 16
GMM: responsibilities, complete data
πk N (xn jµk , Σk )
γ(znk ) p (znk = 1jxn ) =
∑ j = 1 π j N ( xn j µ j , Σ j )
K
Let X denote the observed data, and θ model parameters. The goal
b
in maximum likelihood is to …nd θ:
log p (X , Z jθ )
is easy to maximize.
Problem: Z is not observed
Solution: maximize
Set θ0 θ new
4 Repeat E and M steps until convergence
simple_example.pdf
from Wikipedia