Scribd
Upload
2 views

ExpectationMaximization Algorithm (1)

The Expectation-Maximization (EM) algorithm is used in machine learning to estimate missing data and optimize parameters when dealing with latent variables. It consists of an Expectation step to estimate missing values and a Maximization step to update parameters, repeating until convergence. The algorithm has applications in filling missing data, unsupervised clustering, and estimating parameters of models like Hidden Markov Models, but it has slow convergence and may only reach local optima.

Uploaded by

Pranav Nanda
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
2 views

ExpectationMaximization Algorithm (1)

The Expectation-Maximization (EM) algorithm is used in machine learning to estimate missing data and optimize parameters when dealing with latent variables. It consists of an Expectation step to estimate missing values and a Maximization step to update parameters, repeating until convergence. The algorithm has applications in filling missing data, unsupervised clustering, and estimating parameters of models like Hidden Markov Models, but it has slow convergence and may only reach local optima.

Uploaded by

Pranav Nanda
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 7

GIBBS ALGORITHM

Expectation-Maximization
Algorithm
• In the real-world applications of machine learning, it is very common
that there are many relevant features available for learning but only
a small subset of them are observable.
• Expectation-Maximization algorithm can be used for the latent
variables (variables that are not directly observable and are actually
inferred from the values of the other observed variables) too in
order to predict their values with the condition that the general form
of probability distribution governing those latent variables is known
to us.
• This algorithm is actually at the base of many unsupervised
clustering algorithms in the field of machine learning.
• It is used to find the local maximum likelihood parameters of a
statistical model in the cases where latent variables are involved and
the data is missing or incomplete.
Algorithm
• Given a set of incomplete data, consider a set of starting parameters.
• Expectation step (E – step): Using the observed available data of the
dataset, estimate (guess) the values of the missing data.
• Maximization step (M – step): Complete data generated after the
expectation (E) step is used in order to update the parameters.
• Repeat step 2 and step 3 until convergence.
• The essence of Expectation-Maximization algorithm is to use
the available observed data of the dataset to estimate the
missing data and then using that data to update the values of
the parameters.
• Let us understand the EM algorithm in detail.
• Initially, a set of initial values of the parameters are considered. A set of
incomplete observed data is given to the system with the assumption that the
observed data comes from a specific model.
• The next step is known as “Expectation” – step or E-step. In this step, we use
the observed data in order to estimate or guess the values of the missing or
incomplete data. It is basically used to update the variables.
• The next step is known as “Maximization”-step or M-step. In this step, we use
the complete data generated in the preceding “Expectation” – step in order to
update the values of the parameters. It is basically used to update the
hypothesis
• Now, in the fourth step, it is checked whether the values are converging or not,
if yes, then stop otherwise repeat step-2 and step-3 i.e. “Expectation” – step
and “Maximization” – step until the convergence occurs.
Usage of EM algorithm
• It can be used to fill the missing data in a sample.
• It can be used as the basis of unsupervised learning of clusters.
• It can be used for the purpose of estimating the parameters of Hidden Markov
Model (HMM).
• It can be used for discovering the values of latent variables.
Advantages of EM algorithm
• It is always guaranteed that likelihood will increase with each iteration.
• The E-step and M-step are often pretty easy for many problems in terms of
implementation.
• Solutions to the M-steps often exist in the closed form.
Disadvantages of EM algorithm
• It has slow convergence.
• It makes convergence to the local optima only.
• It requires both the probabilities, forward and backward (numerical optimization
requires only forward probability).

You might also like