What is the necessity of HMM ?

Atul Surwase
• Limitation:- We study that problem of
estimating the parameter in class conditional
densities needed to make a single decision.

• Solution:- The problem of making a sequence

of decision.

• 1) Markov model
• 2) Hidden Markov model

Atul Surwase
 HMM Uses
• Uses
– Speech recognition
• Recognizing spoken words and phrases

– Text processing
• Parsing raw records into structured records

– Bioinformatics
• Protein sequence prediction

– Financial
• Stock market forecasts (price pattern prediction)
• Comparison shopping services
Atul Surwase
 Speech Production Model

• Each phoneme in a word corresponds to number of model states .

• Each model state represents a distinct sound with its own acoustic
spectrum.
– For each state, we store the mean value and variance for each of F features.
• When saying a word, the speaker stays in each state for one or
more frames and then goes on to the next state.
– The time in each state will vary according to how fast he/she is speaking.
– Some speech sounds last longer than others
 Hidden Markov Model

• To calculate the probability density (pd) that observation matches a particular

word with a given alignment, we multiply together:
– the probability of the alignment
– the output pds of each frame
• Try this for every possible alignment of every possible word sequence and choose
the one with highest probability.
• Hidden Markov Model => the correct alignment is hidden: we can’t observe it
directly.
• We talk of the probability density of the model “generating” the observed
frames sequence.
• Body movement:- Normal or Abnormal :-
Security purpose.
In this to recognize such temporal signal
sequence is important in that states.

Atul Surwase
• The essential difference between a Markov
chain and a hidden Markov model is that for a
hidden Markov model there is not a one-to-
one correspondence between the states and
the symbols (Why Hidden?).
• It is no longer possible to tell what state the
model was in when xi was generated just by
looking at xi.

Atul Surwase
 First order Hidden Markov model

• In simpler Markov models (like a Markov chain

), the state is directly visible to the observer,
and therefore the state transition probabilities
are the only parameters.

Atul Surwase
Observable Markov Model

Atul Surwase
• We consider a sequence of states at successive
times; the state at any time t is denoted ω(t).
• A particular sequence of length T is denoted by
ωT = {ω(1), ω(2), ..., ω(T)}
• for instance we might have ω6 = {ω1, ω4, ω2,
ω2, ω1, ω4}. Note that the system can revisit a
state at different steps, and not every state
need be visited.

Atul Surwase
• we have been discussing a Markov model, or
technically speaking, a first-order discrete time
Markov model, since the probability at t+1
depends only on the states at t.
• For instance, in a Markov model for the
production of spoken words, we might have states
representing phonemes, and a Markov model for
the production of a spoken work might have
states representing phonemes.
• Such a Markov model for the word “cat” would
have states for /k/, /a/ and /t/, with transitions
from /k/ to /a/; transitions from /a/ to /t/; and
transitions from /t/ to a final silent state.
Atul Surwase
Markov Chain Example
• Based on the weather today what will it be
tomorrow?
• Assuming only four possible weather states
° Sunny
° Cloudy
° Rainy
° Snowing

Atul Surwase
Markov Chain Structure

(Sunny)
State S1 State S2 (Cloudy)

(Rainy) (Snowing)
State S3 State S4

• Each state is an observable event

• At each time interval the state changes to
another or same state (qt  {S1, S2, S3, S4})
Atul Surwase
Markov Chain Structure
Sunny Cloudy

Rainy Snowy

• Not necessarily all changes are possible

° Can’t change between sunny and snowing
° Can’t snow after it has rained
Atul Surwase
Markov Chain Assumptions
• The Markov assumption
° Next state only dependents on current state
° Tomorrows weather depends only on today's
• The stationarity assumption
° Transition probabilities are independent of the
time the transition takes place

Atul Surwase
 Markov Chain Transition
Probabilities
• Transition probability matrix:
Time t + 1
State S1 S2 S3 S4 Total
S1 a11 a12 a13 a14 1
S2 a21 a22 a23 a24 1
Time t
S3 a31 a32 a33 a34 1
S4 a41 a42 a43 a44 1
• aij = P(qt + 1 = Sj | qt = Si)

Atul Surwase
 Markov Chain Transition
Probabilities
• Probabilities for tomorrow’s weather based on
today’s weather

Time t + 1
State Sunny Cloudy Rainy Snowing
Sunny 0.6 0.3 0.1 0.0
Cloudy 0.2 0.4 0.3 0.1
Time t
Rainy 0.1 0.2 0.5 0.2
Snowing 0.0 0.3 0.2 0.5

Atul Surwase
 Ex:- Coke vs. Pepsi
Given that a person’s last cola purchase was Coke ™,
there is a 90% chance that her next cola purchase will
also be Coke ™.
If that person’s last cola purchase was Pepsi™, there
is an 80% chance that her next cola purchase will also
be Pepsi™.
0.9 0.1
0.8

coke pepsi

0.2
Atul Surwase
 Coke vs. Pepsi
Given that a person is currently a Pepsi purchaser,
what is the probability that she will purchase Coke
two purchases from now?
The transition matrices
are: (corresponding to
0.9 0.1
P  one purchase ahead)
0.2 0.8

0.9 0.1 0.9 0.1 0.83 0.17

P 
2
    
0.2 0.8 0.2 0.8 0.34 0.66
Atul Surwase
 Coke vs. Pepsi
Given that a person is currently a Coke
drinker, what is the probability that she will
purchase Pepsi three purchases from now?

0.9 0.1 0.83 0.17  0.781 0.219

P 
3
    
0.2 0.8 0.34 0.66 0.438 0.562

Atul Surwase
Hidden Markov Models

Atul Surwase
• A hidden Markov model (HMM) is a statistical
Markov model in which the system being
modeled is assumed to be a Markov process
with unobserved (hidden) states.

Atul Surwase
HMM Assumptions: dependence
• The Markov Assumption
° Next state only dependent on current state
• The stationary assumption
° Transition probabilities independent of the time
the transition takes place
• The output independence assumption
° Observations independent of previous
observations

Atul Surwase
 Markov HMM seen and unseen
sequences
• Markov has an observed state sequence
° S1, S2, S3, S4, S5, S6, …
• HMM has an unseen state sequence
° S1, S2, S3, S4, S5, S6, …
• And an observed event sequence
° O1, O2, O3, O4, O5, O6, …
• HMM unseen state sequence can only be
implied from the observed event sequence

Atul Surwase
 Hidden Markov Models
(probabilistic finite state automata)
Often we face scenarios where states cannot be
directly observed.
We need an extension: Hidden Markov Models
a11 a22 a33 a44

a12 a23 a34

b11 b14
b13
b12
4
1
2 3
aij are state transition probabilities.
Observed bik are observation (output) probabilities.
phenomenon b11 + b12 + b13 + b14 = 1,
Atul Surwase b21 + b22 + b23 + b24 = 1, etc.
First-order hidden Markov models
• We continue to assume that at every time step
t the system is in a state ω(t) but now we also
assume that it emits some (visible).
• A particular sequence of such visible states as
VT = {v(1), v(2), ..., v(T)} and thus we might
have V6 = {v5, v1, v1, v5, v2, v3}.

Atul Surwase
 HMM Weather Example
• Predicting weather based on today’s
• BUT visible weather determined by unseen
meteorological conditions
• Classified as:
° Good
° Variable
° Bad

Atul Surwase
HMM Model – Markov States

Variable

Good Bad

• aij = P(qt + 1 = Sj | qt = Si)

° qt is the current state
Atul Surwase
HMM Model – Markov States

Variable

Good Bad

• States hidden
° e.g. stuck in a windowless room
Atul Surwase
HMM Model – Linked Events

Sunny, Cloudy,
Rainy, Snowy

Sunny, Cloudy, Sunny, Cloudy,

Rainy, Snowy Rainy, Snowy

• bj(k) = P(vt = Ok | qt = Sj)

° vt is the current observed event
° bj(k) = P(get observation vt given that state is Sj)
Atul Surwase
 Hidden Markov Model Example
• Coin toss:
– Heads, tails sequence with 2 coins
– You are in a room, with a wall
– Person behind wall flips coin, tells result
• Coin selection and toss is hidden
• Cannot observe events, only output (heads, tails) from
events

– Problem is then to build a model to explain

observed sequence of heads and tails
Atul Surwase
 Common HMM Types
• Ergodic (fully connected):
– Every state of model can be reached
in a single step from every other state
of the model

• Bakis (left-right):
– As time increases, states proceed
from left to right

Atul Surwase
 HMM Components

• A set of states (x’s)

• A set of possible output symbols (y’s)
• A state transition matrix (a’s)
– probability of making transition from one
state to the next

• Output emission matrix (b’s)

– probability of a emitting/observing a
symbol at a particular state

• Initial probability vector

– probability of starting at a particular state
– Not shown, sometimes assumed to be 1
Atul Surwase
Atul Surwase
Atul Surwase
 HMM Core Problems
• Three problems must be solved for HMMs to
be useful in real-world applications

1) Evaluation

2) Decoding

3) Learning
Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
(HMM Backward)

Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
Atul Surwase
 Example: The dishonest casino

A casino has two dice:

• Fair die
P(1) = P(2) = P(3) = P(4) = P(5) = P(6) = 1/6
• Loaded die
P(1) = P(2) = P(3) = P(4) = P(5) = 1/10
P(6) = 1/2
Casino player switches between fair and loaded die
with probability 1/20 at each turn

Game:
1. You bet $1
2. You roll (always with a fair die)
3. Casino player rolls (maybe with fair die, maybe with
loaded die)
4. Highest number wins $2
 The dishonest casino model
0.05
0.95 0.95

FAIR LOADED

P(1|F) = 1/6 P(1|L) = 1/10

P(2|F) = 1/6 P(2|L) = 1/10
P(3|F) = 1/6 0.05 P(3|L) = 1/10
P(4|F) = 1/6 P(4|L) = 1/10
P(5|F) = 1/6 P(5|L) = 1/10
P(6|F) = 1/6 P(6|L) = 1/2

Atul Surwase
 Question # 1 – Decoding
GIVEN

A sequence of rolls by the casino player

1245526462146146136136661664661636616366163616515615115146123562344

FAIR LOADED FAIR

QUESTION

What portion of the sequence was generated with the fair die, and what portion
with the loaded die?

This is the DECODING question in HMMs

Atul Surwase
 Question # 2 – Evaluation
GIVEN

A sequence of rolls by the casino player

1245526462146146136136661664661636616366163616515615115146123562344

Prob = 1.3 x 10-35

QUESTION

How likely is this sequence, given our model of how the casino works?

This is the EVALUATION problem in HMMs

Atul Surwase
 Question # 3 – Learning
GIVEN

A sequence of rolls by the casino player

1245526462146146136136661664661636616366163616515615115146123562344

Prob(6) = 64%

QUESTION

How “loaded” is the loaded die? How “fair” is the fair die? How often
does the casino player change from fair to loaded, and back?

This is the LEARNING question in HMMs

Atul Surwase