Scribd
Upload
27 views

Chapter 4 Multinomial Learning: Presented To

This document discusses multinomial learning, which is a generalization of Bernoulli learning where the outcome of a random event can be one of K mutually exclusive states instead of just two. It defines the probability of each state pi and the indicator variables xi. It explains that the sample X follows a parametric distribution p(x|q) and the goal is to estimate the parameters q using maximum likelihood estimation. Specifically, it provides the formulas for the likelihood of q given the sample, the log likelihood, and the maximum likelihood estimator, drawing comparisons to the Bernoulli case with two states.

Uploaded by

adnan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
27 views

Chapter 4 Multinomial Learning: Presented To

This document discusses multinomial learning, which is a generalization of Bernoulli learning where the outcome of a random event can be one of K mutually exclusive states instead of just two. It defines the probability of each state pi and the indicator variables xi. It explains that the sample X follows a parametric distribution p(x|q) and the goal is to estimate the parameters q using maximum likelihood estimation. Specifically, it provides the formulas for the likelihood of q given the sample, the log likelihood, and the maximum likelihood estimator, drawing comparisons to the Bernoulli case with two states.

Uploaded by

adnan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

Chapter 4 Multinomial learning

Assignment 6

Presented to
Meritorious. Professor .Dr.Aqil Burni
Head of Actuarial Sciences
Institute of Business Management

Machine Learning Adnan Alam Khan [email protected]

Page 1

Multinomial Learning
Consider the generalization of Bernoulli where instead of two states, the outcome of a random
event is one of K mutually exclusive and exhaustive states, for example, classes, each of which has a
probability of occurring pi with
Summation I to K from pi=1 pi = 1. Let x1, x2, . . . , xK are the indicator variables where xi is 1 if the
outcome is state i and 0 otherwise.
X = { xt }t where xt ~ p (x)
Parametric estimation:
Assume a form for p (x |q ) and estimate q , its sufficient statistics, using X
e.g., N ( , 2) where q = { , 2}
Likelihood of q given the sample X
l (|X) = p (X |) = t p (xt|)
Log likelihood
L(|X) = log l (|X) = t log p (xt|)
Maximum likelihood estimator (MLE)
* = argmax L(|X)
Bernoulli: Two states, failure/success, x in {0,1}
P (x) = pox (1 po ) (1 x)
L (po|X) = log t poxt (1 po ) (1 xt)
MLE: po = t xt / N
Multinomial: K>2 states, xi in {0,1}
P (x1,x2,...,xK) = i pixi
L(p1,p2,...,pK|X) = log t i pixit
t
MLE: pi = t xi / N

Machine Learning Adnan Alam Khan [email protected]

Page 2

You might also like