Scribd
Upload
8 views26 pages

Probability Distributions

Uploaded by

f20212242
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
8 views26 pages

Probability Distributions

Uploaded by

f20212242
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPSX, PDF, TXT or read online on Scribd
You are on page 1/ 26

Foundations of Data Science (CS F320)

Dr. Bharat Richhariya


Assistant Professor, Dept. of CSIS
BITS Pilani BITS Pilani
Pilani Campus
Probability distributions

BITS Pilani
Pilani Campus
Common probability distributions

Exponential
family

Not in exponential family

Exponential
family

3
Exponential distribution

4
Estimating probability distribution of a random variable

• Given a random variable , and a finite set of observations


• Modelling the probability distribution is called density estimation.
• We shall assume that the data points are independent and identically
distributed.

5
Estimating probability distribution of a random variable
Prior distribution

Posterior distribution

6
Types of probability distributions

7
Bernoulli distribution
Reason for belonging to exponential
family

1
𝐿𝑒𝑡 𝑝= −η
=σ (η)
1+𝑒

−η𝑥
Bernoulli ( x ; p)=σ ( − η ) 𝑒

• MLE for the parameter is sample mean.


8
Multinoulli distribution

9
Multinomial distribution

𝑁!
𝑥1! 𝑥1! … 𝑥𝑘!
10
Poisson distribution

11
Binomial distribution

12
Uniform distribution

13
Beta distribution
Normalizer

Conjugacy property:
The posterior distribution, which is
proportional to the product of the
prior and the likelihood function, have
the same form as the prior.

14
Beta distribution

Plots of the beta distribution ) as a function of for various values of the hyperparameters and

15
Gamma distribution

16
Dirichlet distribution

17
Univariate Gaussian distribution

18
Multivariate Gaussian distribution

Determinant

19
Multivariate Gaussian distribution

20
Properties of Gaussian distribution
• The marginals and conditionals of Gaussians are Gaussians.

21
The marginals and conditionals of Gaussians are Gaussians.

22
Linear transformation with Gaussians

23
Central Limit Theorem (CLT)

Subject to certain mild conditions,


the sum of a set of random variables,
which is also a random variable,
has a distribution that becomes increasingly Gaussian as the number of
terms in the sum increases.

24
BinomialGaussian (as N->
• Consider N variables x1,x2 . . . , xN each of which has a uniform distribution over the interval [0, 1]

and then considering the distribution of the mean (x1+x2· · ·+xN)/N.


• Histogram plots of the mean of N uniformly distributed numbers for various values of N.
• Observe that as N increases, the distribution tends towards a Gaussian.
• Binomial distribution is a distribution of sum of N observations of the random binary variable p,
where p is the probability of success, and thus leads to Gaussian for large values of N, as per CLT.

25
Sources
• https://www.cse.iitk.ac.in/users/piyush/courses/pml_fall17/material/
probabilty_tutorial.pdf
• https://people.eecs.berkeley.edu/~jordan/courses/260-spring10/other-
readings/chapter9.pdf
• https://www.cs.princeton.edu/courses/archive/spr09/cos513/scribe/
lecture11.pdf
• Bishop (Pattern Recognition and Machine Learning) textbook.

26

You might also like