Scribd
Upload
97 views7 pages

Mixture Models: Sargur Srihari Srihari@cedar - Buffalo.edu

This document discusses mixture models and the Expectation-Maximization (EM) algorithm. It explains that mixture models use latent variables to model complex distributions as combinations of simpler distributions. The Gaussian mixture model represents a distribution as a weighted sum of Gaussian components. Mixture models can be used for unsupervised learning and clustering by modeling sub-populations within a data set. The EM algorithm is introduced as a general technique for finding maximum likelihood estimators in latent variable models.

Uploaded by

asdfasdffdsa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
97 views7 pages

Mixture Models: Sargur Srihari Srihari@cedar - Buffalo.edu

This document discusses mixture models and the Expectation-Maximization (EM) algorithm. It explains that mixture models use latent variables to model complex distributions as combinations of simpler distributions. The Gaussian mixture model represents a distribution as a weighted sum of Gaussian components. Mixture models can be used for unsupervised learning and clustering by modeling sub-populations within a data set. The EM algorithm is introduced as a general technique for finding maximum likelihood estimators in latent variable models.

Uploaded by

asdfasdffdsa
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 7

Machine Learning Srihari

Mixture Models

Sargur Srihari
[email protected]

1
Machine Learning Srihari

Topics in Mixture Models and EM


•  Modeling complex distributions
•  K-means algorithm for finding clusters in a data set
•  Latent variable view of mixture distributions
•  General technique for finding m.l. estimators in
latent variable models
•  EM Algorithm
•  Infinite Mixture Models

2
Machine Learning Srihari

Modeling complex distributions


•  Complex distribution p(x) of observed variable x

p(x)

–  Can be expressed in terms of a more tractable joint


distribution over observed and latent variables p(x,z)
•  Latent variable z with three values can model this distribution
–  Distribution of x alone obtained by marginalization p(x) = ∑ p(x,z)
z

•  Latent variables allow complicated distributions


to be formed from simpler components
–  Gaussian mixtures have latent variables z that are discrete
–  Also called Finite Mixture Models
3
Machine Learning Srihari

Gaussian Mixture Model (GMM)


•  Linear superposition of Gaussian components
–  Two Gaussians Three Gaussians
z has 2 values z has 3 values (data: petal width in Iris)

Since p(x) = ∑ p(x,z) = ∑ p(z)p(x | z)


z z

We can write (for the mixture of two Gaussians): p(x)=p(z=1)p(x|z=1)+p(z=2)p(x|z=2)

4
Machine Learning Srihari

Mixture Model As Unsupervised Learning

•  Probabilistic model representing sub-


populations within a population
–  Without requiring that the sub-population of the data
items be identified (supervised)
•  Constructing such models is called
unsupervised learning or clustering

5
Machine Learning Srihari

Bernoulli Mixture Model


•  Handwritten Digit Data (560×420 pixels)
–  Mixture Model for digits 0-4 with K= 12
•  Identifies three 0s, two 1s,two 3s, and three 4s

Superimposed data of 12 components

6
Machine Learning Srihari

Role of Mixture Models


•  Mixture models provide:
1.  Framework for building complex probability
distributions
•  Complex distribution expressed in terms of tractable joint
distribution of observed and latent variables
–  Distribution of observed variables: by marginalization
2.  A method for clustering data
•  Unsupervised learning

You might also like