5CS037-Concepts and Technologies of AI.

Week04, Lecture04
Discrete Random Variable
and
Their Probability Distributions.

Siman Giri

12/11/2023 Lecture03: What, Why and How of Proabability. 1

A. Revision and Preliminary Concepts.

12/11/2023 Random Variable and Discrete Distribution 2

A.1 From Last Week: Probabilistic Space.

Elements of Probabilistic Space

Probabilistic Experiment. Sample Space. Events. Probability Measure.

12/11/2023 Random Variable and Discrete Distribution 3

A.2 Preliminary Concepts: Statistical Experiment.

• A statistical experiment involves the observation of a sample selected

from a larger body of data, existing or conceptual, called a population. The
measurements in the sample, viewed as observations, are then employed
to make an inference about the characteristics of the target population.
• How are these inferences made?
• The probability of the observed sample plays a major role in making an
inference and evaluating the credibility of the inference.
• On the other hand, it is also clear that the method of sampling will affect
the probability of a particular sample outcome.
• Let’s review an example:

12/11/2023 Random Variable and Discrete Distribution 4

A.2 Preliminary Concepts: Statistical Experiment.

• For example: sampling without and with replacement,

• Thus to keep thing simple for our subsequent discussions of the

probabilities associated with (random variables) we introduce simplest
sampling procedure known as simple random sampling and random
samples.

12/11/2023 Random Variable and Discrete Distribution 5

A.3 Preliminary Concepts: Random Samples
• Simple Random Sampling and Random Samples:

12/11/2023 Random Variable and Discrete Distribution 6

 1. Random Variables

12/11/2023 Random Variable and Discrete Distribution 7

1.1Random Variables: Basic Definition.
• Remember the variables from programming?
• Random variables are like programming variables, but with uncertainty.
• Random Variable is a numeric value assigned to each outcome of a chance/random
experiment.
• For each element (outcomes) in a sample space, the random variable can take on exactly
one value i.e. random variable must be measurable.
• Basic Definition:

12/11/2023 Random Variable and Discrete Distribution 8

1.2Random Variables: Example.
• Experiment: Flip a three fair coins.
• Observations of interest{Y}: Number of “heads” on the three coins.
• Y is an Random Variable, which maps observation of interest to real
numbers.
• What numbers and How?

12/11/2023 Random Variable and Discrete Distribution 9

1.2Random Variables: Example.
• Experiment: Flip a three fair coins.
• Observations of interest{Y}: Number of “heads” on the three coins.
• Y is an Random Variable, which maps observation of interest to real
numbers.
• What numbers and How?
• Which in general is answer to the question: How many possible
combinations of outcome for our observation of interest is
possible?
• In above experiment how many possible outcomes are
possible?
• Let us find out:

12/11/2023 Random Variable and Discrete Distribution 10

1.2Random Variables: Example.
• Experiment: Flip a three fair coins.
• Observations of interest{Y}: Number of “heads” on the three coins.
• Y is an Random Variable, which maps observation of interest to real
numbers.
• What numbers and How?
• Which in general is answer to the question: How many possible
combinations of outcome for our observation of interest is
possible?
• In above experiment how many possible outcomes are
possible?
• Let us find out:

12/11/2023 Random Variable and Discrete Distribution 11

 Example:

Thus our Random Variable takes value:

Y = {0, 1, 2, 3}
12/11/2023 Random Variable and Discrete Distribution 12
Random Variables Vs. Events
• Even though we use the same notation for random variables and for
events (both use capital letters) they are distinct concepts.
• An event is a scenario, a random variable is an object.
• The scenario where a random variable takes on a particular value (or range
of values) is an event.

12/11/2023 Random Variable and Discrete Distribution 13

Random Variables: Types.
• Discrete Random Variable: • Continuous Random Variable:
• A random variable Y is said to be • A random variable Y is said to be
discrete if it can assume only a finite Continuous if it can assume any value
or countably infinite number of defined in-between interval or range.
distinct values. • The variable is continuous if its set of
• A random variable is discrete if its set possible values includes an entire
of possible values is a collection of interval on the number line.
isolated points on the number line.

12/11/2023 Random Variable and Discrete Distribution 14

 2. The Probability Distribution
for a
Discrete Random Variable.

12/11/2023 Random Variable and Discrete Distribution 15

2.1 Probability Distribution.
• The probability of random variable (Discrete or Continuous) is a list of
probabilities associated with each of its possible values.
• Probability Distribution for Discrete Random Variables are also known as
Probability Mass Function.
• Topic for this week: Probability Mass Function.

12/11/2023 Random Variable and Discrete Distribution 16

2.2 Probability of Discrete Random Variable.
• Notationally, we will use an uppercase letter, such as Y , to denote a
random variable and a lowercase letter, such as y, to denote a particular
value that a random variable may assume.

• The expression (Y = y) can be read, the set of all points in S assigned the
value y by the random variable Y .
• The probability that Y takes on the value y, P(Y = y), is defined as the sum
of the probabilities of all sample points in S that are assigned the value y.
We will sometimes denote P(Y = y) by p(y).

12/11/2023 Random Variable and Discrete Distribution 17

2.3 Probability Mass Function.
• The Probability mass function is probability distribution for a discrete
variable Y is a representation of all the possible probabilities a random
variable Y can take in the form of : a formula, a table, or a graph.
• The probability distribution for a discrete random variable assigns
nonzero probabilities to only a countable number of distinct y values.
• Any value y not explicitly assigned a positive probability is understood to
be such that p(y) = 0.
• Let’s illustrate these ideas with an example.

12/11/2023 Random Variable and Discrete Distribution 18

2.4 PMF: Example 1.
• PMF for Y: the value of a six sided die roll.
• By the definition:
• P(Y=y) can take values of
• P(Y =1); P(Y =2); P(Y =3); P(Y =4); P(Y =5); P(Y =6)
• As per definition of Probability Distribution,
we can arrange this in table, graph or formula.

12/11/2023 Random Variable and Discrete Distribution 19

2.4 PMF: Example 2.
• A supervisor in a manufacturing plant has three men and three women
working for him. He wants to choose two workers for a special job. Not
wishing to show any biases in his selection, he decides to select the two
workers at random. Let Y denote the number of women in his selection.
Find the probability distribution for Y .
• Solution:

12/11/2023 Random Variable and Discrete Distribution 20

2.4 PMF: Example 2.
• Solution:
• Total possible outcomes = Total workers supervisor can select = 62 = 15.
• If Y = number of women in selection , then possible values for Y that can
have non zero probability are {0, 1, 2}
• Case1: zero women were selected:
𝟑𝐂𝟎,𝟑𝐂𝟐 𝟏
• P(Y = 0) = = .
𝟏𝟓 𝟓
• Case2: One Woman and One Man are Selected:
𝟑𝐂𝟏,𝟑𝐂𝟏 𝟑
• P(Y = 1) = = .
𝟏𝟓 𝟓
• Case3: Both the Worker are Women:
𝟑𝐂𝟐,𝟑𝐂𝟎 𝟏
• P(Y = 2) = = .
𝟏𝟓 𝟓
• To Find the Probability Distribution we have to represent afore
probability either in Equation or Table or Graph.

12/11/2023 Random Variable and Discrete Distribution 21

2.4 PMF: Example 2.
• Represent in the form of formula or equation:
𝟑𝑪𝒚,𝟑𝑪𝟐−𝒚
• P(Y=y) = ; for y = 0, 1,2
𝟔𝑪𝟐

12/11/2023 Random Variable and Discrete Distribution 22

2.5 Properties of PMF.
• For any discrete probability distribution, the following must be true:
• 0 ≤ p(y) ≤ 1 for all y.
• 𝒚𝒑 𝒚 = 𝟏
• {where the summation is over all values of y with nonzero probability.}

12/11/2023 Random Variable and Discrete Distribution 23

 3.The Expected Value
of a
Discrete Random Variable

12/11/2023 Random Variable and Discrete Distribution 24

3.1 Discrete Random Variable: Expectation.
• A random variable is fully represented by its probability mass function
(PMF), which represents each of the values the random variable can take
on, and the corresponding probabilities.
• Expectation can be thought as a Summary statistics of a Random Variable!

12/11/2023 Random Variable and Discrete Distribution 25

3.2 Properties of Expectations.

12/11/2023 Random Variable and Discrete Distribution 26

3.2 Properties of Expectations.

12/11/2023 Random Variable and Discrete Distribution 27

3.1 Discrete Random Variable: Variance.
• In the last slide we showed that Expectation was a useful summary of a
random variable
• it calculates the "weighted average" of the random variable.
• One of the next most important properties of random variables to
understand is variance: the measure of spread.

12/11/2023 Random Variable and Discrete Distribution 28

 4. Special Distributions
Bernoulli Distribution

12/11/2023 Random Variable and Discrete Distribution 29

4.1 What are Special Distributions?
• There are some specific distributions that are used over and over in
practice, thus they have been given special names.
• Behind such distribution are a Random Experiments which
• models a lot of real life phenomenon and
• are frequently used in different applications including Machine
Learning,
• thus we devote a section to study them.
• For the discussed distribution (Discrete and Continuous):
• We will try to understand the random experiment,
• And provide a PMFs

12/11/2023 Random Variable and Discrete Distribution 30

4.3 Bernoulli Distribution: Trial
• Bernoulli Trial:
• Bernoulli trial is an experiment with only two possible outcomes.
• The two possible outcomes are labelled:
• Success(S) and Failure (F).
• The probability of success is
• P(S) = p
• and the probability of failure is P(f) = q = 1-p
• Examples:
• Tossing a Coin
• {Success = H, Failure = T and p = P(S) = P(H)}
• Inspecting an item for defects
• {Success = defective, Failure = non-defective, and p = P(S) = P(defective)}

12/11/2023 Random Variable and Discrete Distribution 31

4.4 Bernoulli Distribution: Experiment
• The Bernoulli (Experiment) Process:
• Bernoulli Process is an Random Experiment that must satisfy the
following properties:
• The Experiment consists of n repeated Bernoulli trials.
• The probability of success,{P(S) = p} remains constant from trial to
trial.
• The repeated trials are independent, i.e. the outcome on one trial
has no effect on the outcome of any other trial.

12/11/2023 Random Variable and Discrete Distribution 32

 4.5 Bernoulli Distribution: PMF

PMF of a 𝑩𝒆𝒓𝒏𝒐𝒖𝒍𝒍𝒊(𝒑) random variable.

12/11/2023 Random Variable and Discrete Distribution 33
4.6 Bernoulli Distribution: Expectation
• Expectation of Bernoulli:
• If Y is a Bernoulli with parameter p, Y ~ Bernoulli(p), then Expectation
of Bernoulli Random Variable Y is :
•𝑬𝒀 =𝒑
• Proof(Optional):

12/11/2023 Random Variable and Discrete Distribution 34

4.6 Bernoulli Distribution: Variance
• Variance of Bernoulli:
• If Y is a Bernoulli with parameter p, Y ~ Bernoulli(p), with
Expectation𝐸 𝑌 = 𝑝 then variance of Bernoulli Random Variable Y is :
• Var(𝒀) = 𝒑 𝟏 − 𝒑 .
• Proof (Optional):

12/11/2023 Random Variable and Discrete Distribution 35

 5. Special Distributions
Binomial Distribution

12/11/2023 Random Variable and Discrete Distribution 36

5.1 Binomial Distribution: Experiment.
• Consider n independent (Bernoulli) trials of an experiment where each
trial is a "success" with probability p.
• Example:
• For 5 tosses of coin, what is the probability you flip exactly 3 heads. Let
X be the random variable, Where X ={0, 1, 2, 3, 4, 5}. Find a PMF.

12/11/2023 Random Variable and Discrete Distribution 37

5.1 Binomial Distribution: Experiment.
• Consider n independent (Bernoulli) trials of an experiment where each trial is a "success" with
probability p.
• Example:
• For 5 tosses of coin, what is the probability you flip exactly 3 heads. Let X be the random
variable, Where X ={0, 1, 2, 3, 4, 5}. Find a PMF.

12/11/2023 Random Variable and Discrete Distribution 38

5.2 Binomial Distribution: PMF.

12/11/2023 Random Variable and Discrete Distribution 39

5.2 Binomial and Bernoulli Distribution .
• Relationship is defined by:
• If 𝑌1 , 𝑌2 … , 𝑌𝑛 are independent Bernoulli(p) random variables, then the
random variable 𝑋 defined by 𝑋 = 𝑌1 + 𝑌2 + ⋯ + 𝑌𝑛 has a Binomial (n,
p) distribution.
• 𝑿~𝑩𝒊𝒏 𝒏, 𝒑 = 𝒏𝒊=𝟏 𝒀𝒊 ~𝑩𝒆𝒓𝒏 𝒑

12/11/2023 Random Variable and Discrete Distribution 40

5.3 Binomial Distribution: E[Y] and Var(Y)
• Expectation:
• 𝐸 𝑌 = 𝑛. 𝑝
• Variance:
• 𝑉𝑎𝑟 𝑌 = 𝑛. 𝑝. (1 − 𝑝)

• Home Task:
• Try proving above relationship as we did for Bernoulli Distribution.

12/11/2023 Random Variable and Discrete Distribution 41

5.3 Binomial Distribution: Example.
• You are performing a cohort study. If the probability of developing disease
in the exposed group is .05 for the study duration. Then if you
sample(randomly) 500 exposed people, how many do you expect to
develop the disease?

12/11/2023 Random Variable and Discrete Distribution 42

5.3 Binomial Distribution: Example.
• You are performing a cohort study. If the probability of developing disease
in the exposed group is .05 for the study duration. Then if you
sample(randomly) 500 exposed people, how many do you expect to
develop the disease?
• In this scenario, you can model the problem as a binomial distribution,
• where each individual in the exposed group either develops the
disease (success) with a probability of 0.05
• or does not develop the disease (failure) with a probability 1−0.05=0.95.
• Expectation ~ Bin(n, p):
• 𝐸 𝑌 = 𝑛. 𝑝 = 500 × 0.05 = 25

12/11/2023 Random Variable and Discrete Distribution 43

 6. Special Distributions
Poisson Distribution.

12/11/2023 Random Variable and Discrete Distribution 44

6.1 Poisson Distribution: Intuition.
• The Poisson distribution is one of the most widely used probability distributions.
• It is usually used in scenarios where we are counting the occurrences of certain events in
an interval of time or space.
• In practice, it is often an approximation of a real-life random variable.
• Suppose that we are counting the number of customers who visit a certain store from
1pm to 2 pm.
• Based on data from previous days, we know that on average 𝜆 = 15 customers visit
the store.
• Here, we may model the random variable
• Y: {showing the number customers} as a Poisson random variable (𝜆)

12/11/2023 Random Variable and Discrete Distribution 45

6.1 Poisson Distribution: PMF.

12/11/2023 Random Variable and Discrete Distribution 46