Machine Learning for Signal Processing

EHB 328E

Ibraheem Shayea
Electronics and Communication Engineering Department,
Faculty of Electrical and Electronics Engineering
İstanbul Teknik Üniversitesi - İTÜ
 Syllabus 2
Distribution Functions
(1) Distribution Functions (Introduction)
(2) Probability Density Functions(PDF)
(3) Probability Mass Functions (PMF)
(4) Cumulative Distribution Function(CDF)
 Syllabus 2
Distribution Functions

(3)

Probability Mass Function (PMF)

Lecture Objectives
What you should Learn from this Lecture ?

❑ Understanding :

o Concept of Probability Mass Function (PMF)

o Mathematical Expression

o Properties of PMF
Outline

❑ Probability Mass Function (PMF)

❑ Mathematical Formulation
❑ Properties of PMF
❑ Examples

5
Probability Mass Function (PMF)

6
Probability Mass Function (PMF)
Probability Mass Function

Random Variables
❑ PMF is a probability function used to describe the probability
distribution of a discrete random variable.

7
Probability Mass Function (PMF)
Differences between MPF and PDF ?

❑ PDF is used for a continuous random variable.

❑ PMF is used for a discrete random variable.

Figure: PMF (for discrete outcomes) or a PDF (for continuous

outcomes) 8
Mathematical Formulation

9
Mathematical Formulation
Probability Mass Function (PMF)

❑ Probability Mass Function of a discrete random variable 𝑋

expresses the complete probability model of a random experiment.

❑ PMF is a probability measure that gives us probabilities of the

possible values for a RandomVariable.

10
Mathematical Formulation
Probability Mass Function (PMF)
A discrete random variable assumes values from :

❑ A countably infinite set 𝑆𝑋 = {𝑥1 , 𝑥2 , 𝑥3 , … } or

❑ A finite set 𝑆𝑋 = {𝑥1 , 𝑥2 , 𝑥3 , … , 𝑥𝑛 } ,

o where n is a positive integer.

❑ 𝑥1 , 𝑥2 , 𝑥3 ,... are possible values of the random variable X.

o 𝑋 is the discrete random variable

11
Mathematical Formulation
Probability Mass Function (PMF)
Let 𝑋 be a discrete random variable with range:
𝑅𝑋 = {𝑥1 , 𝑥2 , 𝑥3 , … } (finite or countably infinite).

The Probability Mass Function (PMF) of 𝑋can be represented by:

𝑝𝑋 𝑥𝑖 = 𝑃 𝑋 = 𝑥𝑖 , 𝑓𝑜𝑟 𝑖 = 1, 2, 3, … , 𝑥𝑖 ∈ 𝑆𝑋

o The above notation is the standard notation for the PMF of 𝑋.

12
Mathematical Formulation
Probability Mass Function (PMF)

❖ 𝒑𝑿 (𝒙): The notation for PMF is 𝑝𝑋 (𝑥). where we use the lowercase
letter 𝑝, with a subscript corresponding to the name of the random
variable 𝑋, as a function of a possible value of the random variable,
denoted by the corresponding lowercase letter 𝑥.

❖ The subscript 𝑋 here indicates that this is the PMF of the random
variable 𝑋.

❖ For example, 𝑝𝑋 (1) shows the probability that 𝑋 = 1.

13
Mathematical Formulation
Probability Mass Function (PMF)

❑ 𝒑𝑿 (𝒙) is a function ranging over real numbers 𝑥.

❑ 𝒑𝑿 (𝒙) can be non-zero only at the values 𝒙 ∈ 𝑺𝑿 .

❑ For any value of 𝑥, the function 𝑝𝑋 (𝑥) is the probability of the event
{𝑋 = 𝑥}.

❑ The PMF 𝑝𝑋 (𝑥) contains all the information required to calculate

probability for any event involving the discrete random variable
X.

14
Mathematical Formulation
Probability Mass Function (PMF)
Although the PMF is usually defined for values in the range, it is
sometimes convenient to extend the PMF of 𝑋 to all real numbers.

If 𝑥 ∉ 𝑅𝑋 , we can simply write:

𝑝𝑋 (𝑥) = 𝑃(𝑋 = 𝑥) = 0.

Thus, in general we can write:

𝑷 𝑿=𝒙 𝒊𝒇 𝒙 𝒊𝒔 𝒊𝒏 𝑹𝑿
𝒑𝑿 (𝒙) =
𝟎 𝒐𝒕𝒉𝒆𝒓𝒘𝒊𝒔𝒆
15
Mathematical Formulation
CDF and Probability Mass Function (PMF)

??

16
Mathematical Formulation
CDF and Probability Mass Function (PMF)

𝐹𝑋 (𝑥) = ෍ 𝑝𝑋 𝑥𝑘
𝑘

𝑝𝑋 𝑥𝑖 = 𝑃 𝑋 = 𝑥𝑖 , 𝑓𝑜𝑟 𝑖 = 1, 2, 3, … , 𝑥𝑖 ∈ 𝑆𝑋

17
Properties of PMF

18
Properties of PMF
Property 1
For a discrete random variable X with PMF 𝑝𝑋 (𝑥) :
0 ≤ 𝑝𝑋 𝑥𝑘 ≤1 𝑘 = 1, 2, …
𝑝 𝑥𝑖 ≥ 0, for all 𝑥𝑖
❑ The discrete probability model assigns a number between 0 and 1 to
each outcome in a sample space.

𝑝𝑋 𝑥𝑘 𝑝𝑋 𝑥𝑘 𝑝𝑋 𝑥𝑘

𝑘 𝑘 𝑘 𝑘
19
Properties of PMF
Property 2
For a discrete random variable X with PMF 𝑝𝑋 (𝑥) :

𝑝𝑋 𝑥 = 0 𝑖𝑓 𝑥 ≠ 𝑥𝑘 (𝑘 = 1, 2, … )

o 𝒑𝑿 (𝒙) can be non-zero only at the values 𝑥 ∈ 𝑆𝑋 .

𝑝𝑋 𝑥𝑘 𝑝𝑋 𝑥𝑘 𝑝𝑋 𝑥𝑘

𝑘 𝑘 𝑘 𝑘
20
Properties of PMF
Property 3

෍ 𝑝𝑋 𝑥𝑘 = 𝑝𝑋 𝑥1 + 𝑝𝑋 𝑥2 + 𝑝𝑋 𝑥3 + … … = 1
𝑘

𝑝𝑋 𝑥𝑘 𝑝𝑋 𝑥𝑘 𝑝𝑋 𝑥𝑘

𝑘 𝑘 𝑘 𝑘
21
Examples

22
Examples
Examples

23
Examples
Example 1
Toss a fair coin twice, and let 𝑋 be defined as the number of heads we
observe. Find
a) The range of 𝑿, 𝑹𝑿 ,
b) Probability Mass Function 𝒑𝑿 (𝒙) ?

Solution

??

24
Examples
Example 1
a) The range of 𝑿, 𝑹𝑿 ,

Solution
2 1 0
𝑆 = {𝐻𝐻, 𝐻𝑇, 𝑇𝐻, 𝑇𝑇}.

The number of heads will be 0, 1 or 2.

Thus

𝑅𝑋 = {0, 1, 2}.

25
Examples
Example 1
b) Probability Mass Function 𝒑𝑿 (𝒙) ?

Solution

The PMF is defined as:

𝑃𝑋 𝑘 = 𝑃 𝑋 = 𝑘 𝑓𝑜𝑟 𝑘 = 0, 1, 2

1
• 𝑃𝑋 0 = 𝑃 𝑋 = 0 = 𝑃 𝑇𝑇 = ,
4
1 1 1
• 𝑃𝑋 (1) = 𝑃(𝑋 = 1) = 𝑃({𝐻𝑇, 𝑇𝐻}) = + = ,
4 4 2
1
• 𝑃𝑋 (2) = 𝑃(𝑋 = 2) = 𝑃(𝐻𝐻) = .
4

26
Examples
Example 1
b) Probability Mass Function 𝒑𝑿 (𝒙) ?
Solution
1
o 𝑃𝑋 0 = 𝑃 𝑋 = 0 = 𝑃 𝑇𝑇 = ,
4
1 1 1
o 𝑃𝑋 (1) = 𝑃(𝑋 = 1) = 𝑃({𝐻𝑇, 𝑇𝐻}) = + =
4 4 2
1
o 𝑃𝑋 (2) = 𝑃(𝑋 = 2) = 𝑃(𝐻𝐻) = .
4

1
𝑥=0
4

1
𝑥=1
𝑝𝑋 (𝑥) = 2

1
𝑥=2
4

0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒 27
Examples
Example 1
b) Probability Mass Function 𝒑𝑿 (𝒙) ?

Solution
1
𝑥=0
4

1
𝑥=1
𝑝𝑋 (𝑥) = 2

1
𝑥=2
4

0 𝑜𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

28
The End

Thank You

29