718CST02 MACHINE LEARNING TECHNIQUES

Unit V - GRAPHICAL MODELS

• Markov Chain Monte Carlo Methods
• Sampling
• Proposal Distribution
• Markov Chain Monte Carlo
• Graphical Models
• Bayesian Networks
• Markov Random Fields
• Hidden Markov Models
• Tracking Methods
Markov Chain Monte Carlo Methods

• Sampling

• Guassian Random Number

• Box–Muller scheme, which uses a pair of uniformly randomly distributed
numbers in order to make two independent Gaussian-distributed numbers
with zero mean and unit variance.
Markov Chain Monte Carlo Methods
 Markov Chain Monte Carlo Methods

• Proposal Distribution

We make the decision of whether or not to accept the sample by picking a

uniformly distributed random number u between 0 and Mq(x ∗). If this random
number is less than p˜(x ∗), then we accept x ∗, otherwise we reject it. The
reason why this works is known as the envelop e principle: the pair (x ∗, u) is
uniformly distributed under Mq(x ∗), and the rejection part throws away
samples that don’t match the uniform distribution on p(x ∗), so Mq(x) forms an
envelope on p(x).
Markov Chain Monte Carlo Methods

• Proposal Distribution
Markov Chain Monte Carlo Methods

• Proposal Distribution
Markov Chain Monte Carlo Methods

• The most popular method for sampling from high-dimensional

distributions is Markov chain Monte Carlo or MCMC.
• Markov chain is a systematic method for generating a sequence of
random variables where the current value is probabilistically
dependent on the value of the prior variable. Specifically, selecting
the next variable is only dependent upon the last variable in the
chain.
• A Markov chain is a chain with the Markov property, i.e., the
probability at time t depends only on the state at t − 1,
• Combining these two methods, Markov Chain and Monte Carlo,
allows random sampling of high-dimensional probability distributions
that honors the probabilistic dependence between samples by
constructing a Markov Chain that comprise the Monte Carlo sample.
Markov Chain Monte Carlo Methods

• The chain also has to be ergodic, which means that we will revisit every
state, so that the probability of visiting any particular state in the future
never goes to zero
• The distribution p(x) to be invariant to the Markov chain, which means
that the transition probabilities don’t change the distribution:

• This is known as Markov Chain Monte Carlo (MCMC) sampling, and

the most popular algorithm that is used for MCMC is the Metropolis–
Hastings algorithm
Markov Chain Monte Carlo Methods
Markov Chain Monte Carlo Methods
 Markov Chain Monte Carlo Methods

Sampling –Random Number

• Gaussian Random Number
• The usual method of doing this is the Box–Muller scheme, which uses a pair
of uniformly randomly distributed numbers in order to make two
independent Gaussian-distributed numbers with zero mean and unit variance

https://www.publichealth.columbia.edu/research/population-health-methods/markov-chain-monte-carlo
 Graphical Models

• Bayesian Networks
• DAGs: directed, acyclic graphs

If we have a set of observations that can be

used to predict an unknown outcome, then
we are doing top-down inference or
prediction, whereas if the outcome is known,
but the causes are hidden, then we are doing
bottom-up inference or diagnosis.
Graphical Models

• In order to compute the probability of being scared, we need to

compute P(b, r, a, s),
Graphical Models
Graphical Models
Graphical Models - Markov Random Fields

The Markov Random Field model is a model which use an undirected

graph. Undirected graphical models edge represents the potential
between two variables, syntactically, Factorization distribution
probabilities between variable.
A Markov Random Field is a graph whose nodes model random
variables, and whose edges model desired local influences among pairs
of them. Local influences propagate globally, leveraging the
connectivity of the graph.
Two nodes in a Markov Random Field (MRF) are conditionally
independent of each other, given a third node, if there is no path
between the two nodes that doesn’t pass through the third node. This
is actually a variation on the Markov property,
Graphical Models - Markov Random Fields

P(A, B, C, D, E) α Ф(A,B) Ф (B,C) Ф (B,D) Ф (C,E) Ф (D,E)

 Graphical Models - Markov Random Fields

P(A, B, C, D, E) α Ф(A,B) Ф (B,C) Ф (B,D) Ф (C,E) Ф (D,E)

It is some different if we join D,C and B,E clique over here then it is also change its probability.
Graphical Models - Markov Random Fields

P(A, B, C, D, E) α Ф(A,B) Ф (B,C) Ф (B,D) Ф (C,E) Ф (D,E)

Independence properties such as Markov properties:

Any 2 subsets if variables are conditionally independent given a separating subset.

If we take 'A' as a subset and 'C' as one subset then there is wore between them. So, there is no way to go between
'A' and 'C' without getting threw the subset. So, we are using (A, B) than B, C, D, E.
Therefore, A and C are separating subsets

Any 2 subset of variable are conditionally independent given a separating subset.

{B,D}, {B,E} and {B,D,E} are separating subsets.
Graphical Models - Markov Random Fields
Graphical Models - Hidden Markov Models

• Hidden Markov Models (HMMs) are a type of probabilistic model that are commonly used
in machine learning for tasks such as speech recognition, natural language processing,
and bioinformatics.
• It has been used in data science to make efficient use of observations for successful predictions or
decision-making processes.
• The hidden Markov model (HMM) is another type of Markov model where there are few states
which are hidden.
• The hidden state is the term given to the next possible variable which cannot be directly observed
but can be inferred by observing one or more states according to Markov’s assumption.
• Markov assumption is the assumption that a hidden variable is dependent only on the previous
hidden state. Mathematically, the probability of being in a state at a time t depends only on the
state at the time (t-1).
• Another Markov assumption states that the conditional distribution over the next state, given
the current state, doesn’t change over time. This is also termed a stationary process assumption.
Graphical Models - Hidden Markov Models

A hidden Markov model consists of five important components:

Initial probability distribution: An initial probability distribution over states, πi is the probability that the Markov
chain will start in state i. Some states j may have πj = 0, meaning that they cannot be initial states. The initialization
distribution defines each hidden variable in its initial condition at time t=0 (the initial hidden state).
One or more hidden states
Transition probability distribution: A transition probability matrix where each [latex]a_{ij}[/latex] represents the
probability of moving from state i to state j. The transition matrix is used to show the hidden state to hidden state
transition probabilities.
A sequence of observations
Emission probabilities: A sequence of observation likelihoods, also called emission probabilities, each expressing
the probability of an observation [latex]o_{i}[/latex] being generated from a state I. The emission probability is used
to define the hidden variable in terms of its next hidden state. It represents the conditional distribution over an
observable output for each hidden state at time t=0.
Graphical Models - Hidden Markov Models

The hidden Markov model in the above diagram

represents the process of predicting whether
someone will be found to be walking, shopping,
or cleaning on a particular day depending upon
whether the day is rainy or sunny. The following
represents five components of the hidden
Markov model in the above diagram:
Graphical Models - Hidden Markov Models
• Let’s notice some of the following in the above picture:
• There are two hidden states such as rainy and sunny. These states are hidden because
what is observed as the process output is whether the person is shopping, walking, or
cleaning.
• The sequence of observations is shop, walk, and clean.
• An initial probability distribution is represented by start probability
• Transition probability represents the transition of one state (rainy or sunny) to
another state given the current state
• Emission probability represents the probability of observing the output, shop, clean
and walk given the states, rainy or sunny.
• The Hidden Markov model is a special type of Bayesian network that has hidden
variables which are discrete random variables.
Graphical Models - Hidden Markov Models
• Real-world examples of Hidden Markov Models (HMM)
Retail scenario: Now if you go to the grocery store once per week, it is
relatively easy for a computer program to predict exactly when your shopping
trip will take more time. The hidden Markov model calculates which day of
visiting takes longer compared with other days and then uses that information
in order to determine why some visits are taking long while others do not seem
too problematic for shoppers like yourself. Another example from e-commerce
where hidden Markov models are used is the recommendation engine. The
hidden Markov models try to predict the next item that you would like to buy.
Travel scenario: By using hidden Markov models, airlines can predict how long
it will take a person to finish checking out from an airport. This allows them to
know when they should start boarding passengers!
Graphical Models - Hidden Markov Models
• Real-world examples of Hidden Markov Models (HMM)
Medical Scenario: The hidden Markov models are used in various medical
applications, where it tries to find out the hidden states of a human body system
or organ. For example, cancer detection can be done by analyzing certain
sequences and determining how dangerous they might pose for the patient.
Another example where hidden Markov models get used is for evaluating
biological data such as RNA-Seq, ChIP-Seq, etc., that help researchers understand
gene regulation. Using the hidden Markov model, doctors can predict the life
expectancy of people based on their age, weight, height, and body type.
Marketing scenario: As marketers utilize a hidden Markov model, they can
understand at what stage of their marketing funnel users are dropping off and
how to improve user conversion rates.
Graphical Models - Hidden Markov Models
• Libraries which can be used for training hidden Markov models

• One of the popular hidden Markov model libraries is PyTorch-HMM, which

can also be used to train hidden Markov models. The library is written in
Python and it can be installed using PIP.
• HMMlearn: Hidden Markov models in Python
• PyHMM: PyHMM is a hidden Markov model library for Python.
• DeepHMM: A PyTorch implementation of a Deep Hidden Markov Model
Graphical Models - Hidden Markov Models
• Forward Algorithm
Graphical Models - Hidden Markov Models
• Forward Algorithm
Graphical Models - Hidden Markov Models
• Forward Algorithm
Graphical Models – Tracking Method
• Two Methods
• Kalman Filter - is a recursive estimator. It makes an estimate of the next step,
then computes an error term based on the value that was actually produced in
the next step, and tries to correct it. It then uses both of those to make the next
prediction, and iterates this procedure.
• Particle Filter
Graphical Models – Tracking Method