Module I : Probability Distributions

Topic 1: Review of basic probability theory. Random

variables, Probability mass and density functions.
Mathematical expectation, Mean and Variance.

Dr. P. Rajendra, Professor, Department of mathematics,

CMRIT, Bengaluru.
Basic Probability Theory
▶ Probability is a measure of the likelihood of an event
occurring.
▶ For any event A, the probability of A is denoted by P(A),
where 0 ≤ P(A) ≤ 1.
▶ The sum of probabilities of all possible outcomes in a sample
space is 1.
▶ Conditional Probability: The probability of event A given
that event B has occurred is P(A | B) = P(A∩B)
P(B) , assuming
P(B) > 0.
▶ Independence: Two events A and B are independent if
P(A ∩ B) = P(A)P(B).

Example
Consider a spam detection model where A is the event ”email
contains the word ’free’” and B is the event ”email is spam”.
Understanding P(B | A) helps in improving the accuracy of the
model.
Random Variables

▶ A Random Variable is a function that assigns a numerical

value to each outcome in a sample space.
▶ Random variables can be discrete or continuous.
▶ Discrete Random Variables take on a countable number of
distinct values.
▶ Continuous Random Variables take on an uncountable
number of values within an interval.

Example
In a recommendation system, let X be a discrete random variable
representing the number of clicks on recommended items. A
continuous random variable Y might represent the time a user
spends on a page.
Mathematical Expectation (Expected Value)
▶ The Expected Value or Expectation of a random variable is
the long-run average value of repetitions of the experiment it
represents.
▶ For a discrete random variable X with PMF P(X = xi ) = pi ,
the expectation is:
X
E [X ] = xi · pi
i

▶ For a continuous random variable X with PDF f (x), the

expectation is: Z ∞
E [X ] = x · f (x) dx
−∞

Example
In an e-commerce setting, X could represent the revenue generated
per user. The expected revenue, E [X ], is a crucial metric for
decision-making.
Mean
▶ The Mean of a random variable is another term for its
expected value and provides a measure of the central
tendency of the distribution.
▶ For a discrete random variable X :
X
µ = E [X ] = x i · pi
i

▶ For a continuous random variable X :

Z ∞
µ = E [X ] = x · f (x) dx
−∞

Example
In a user engagement model, the mean number of interactions per
session can inform interface design and content placement.
Variance
▶ The Variance of a random variable is a measure of how much
the values of the random variable differ from the mean.
▶ For a discrete random variable X :
X
Var(X ) = E [(X − µ)2 ] = (xi − µ)2 · pi
i

▶ For a continuous random variable X :

Z ∞
2
Var(X ) = E [(X − µ) ] = (x − µ)2 · f (x) dx
−∞

▶ The square root of the variance is the standard deviation,

which gives a measure of spread in the same units as the
random variable.
Example
In a predictive model, understanding the variance of predictions
(e.g., predicted sales) helps in assessing the reliability and
confidence in those predictions.
Probability Mass Function (PMF)
The Probability Mass Function(PMF) is the probability distribution
of a discrete random variable and provides the possible values and
their associated probabilities.
1. P(xi ) ≥ 0
Pn
2. i=1 P(X = xi ) = 1
3. 0 ≤ P(x) ≤ 1
4. Mean(µ):
n
X
µ = E [X ] = xi P(xi )
i=1

5. Variance(σ 2 ):
n
X
σ2 = xi2 P(xi ) − µ2 or σ 2 = E [X 2 ] − (E [X ])2
i=1
Probability Density Function (PDF)
The Probability Density Function (PDF) is the probability
distribution of a continuous random variable and provides the
possible values and their associated probabilities infinitely.
1. P(xi ) ≥ 0 or f (x) ≥ 0
R∞
2. −∞ f (x) dx = 1
3. Mean (µ): Z ∞
µ= xf (x) dx
−∞
4. Variance (σ 2 ):
Z ∞
2
σ = x 2 f (x) dx − µ2
−∞

5. Probability over an interval:

P(a ≤ x ≤ b) = P(a < x ≤ b)
Z b
= P(a ≤ x < b) = P(a < x < b) = f (x) dx
a
Problem 1

Show that the following probabilities satisfy the properties of a

discrete random variable. Hence, find the mean and variance.
x 0 1 2 3
1 3 3 1
P(x) 8 8 8 8

Solution: To verify that P(x) satisfies the properties of a discrete

random variable, we need to check two conditions:
(i) P(xi ) ≥ 0 for all i.
P
(ii) i P(xi ) = 1.
(i). Non-negativity:
 
1 3 3 1
P(x) = , , , ≥0
8 8 8 8
(ii). Sum of probabilities equals 1:
X 1 3 3 1 8
P(x) = + + + = =1
x
8 8 8 8 8
Since both conditions are satisfied, P(x) represents a valid discrete
probability distribution.
The mean (µ) of a discrete random variable is given by:
X
µ = E [X ] = x · P(x)
x
Calculating the mean:
1 3 3 1
µ=0· +1· +2· +3·
8 8 8 8
3 6 3 12
µ=0+ + + = = 1.5
8 8 8 8
The variance (σ 2 ) is given by:

σ 2 = E [X 2 ] − (E [X ])2
First, find E [X 2 ]:
X
E [X 2 ] = x 2 · P(x)
x

Calculating E [X 2 ]:
1 3 3 1
E [X 2 ] = 02 · + 12 · + 2 2 · + 32 ·
8 8 8 8
3 12 9 24
E [X 2 ] = 0 + + + = =3
8 8 8 8
Now, calculate the variance:

σ 2 = E [X 2 ] − (E [X ])2 = 3 − (1.5)2 = 3 − 2.25 = 0.75

Thus, the mean is µ = 1.5 , the variance is σ 2 = 0.75 and S.D is
σ = 0.86.
Problem 2
Find the value of k such that the following distribution represents
a discrete probability distribution. Hence, find the Mean, Standard
Deviation, P(x ≤ 1), P(x > 1), and P(−1 < x ≤ 2).

x -3 -2 -1 0 1 2 3
P(x) k 2k 3k 4k 3k 2k k

Solution:
The sum of all probabilities must equal 1 for it to be a valid
probability distribution:

k + 2k + 3k + 4k + 3k + 2k + k = 1

16k = 1
1
k=
16
The mean µ is given by:
X
µ = E [X ] = xi · P(xi )
i

Substituting the values:

1 2 3 4 3 2 1
µ = (−3) · + (−2) · + (−1) · + 0 · + 1 · + 2 · + 3 ·
16 16 16 16 16 16 16
−3 − 4 − 3 + 0 + 3 + 4 + 3 0
µ= = =0
16 16
The variance σ 2 is given by:

σ 2 = E [X 2 ] − (E [X ])2

First, compute E [X 2 ]:
X
E [X 2 ] = xi2 · P(xi )
i
 1 2 3
E [X 2 ] = (−3)2 · + (−2)2 · + (−1)2 · + 02
16 16 16
4 3 2 1
· + 12 · + 22 · + 32 ·
16 16 16 16
9 + 8 + 3 + 0 + 3 + 8 + 9 40
E [X 2 ] = = = 2.5
16 16
Since µ = 0:
σ 2 = 2.5 − 02 = 2.5
√
σ = 2.5 ≈ 1.58
P(x ≤ 1) = P(−3) + P(−2) + P(−1) + P(0) + P(1)
1 13
∴ P(x ≤ 1) = k + 2k + 3k + 4k + 3k = 13k = 13 × =
16 16
1 3
P(x > 1) = P(2) + P(3) = 2k + k = 3k = 3 × =
16 16
1 9
= P(0) + P(1) + P(2) = 4k + 3k + 2k = 9k = 9 × =
16 16
Problem 3

A random variable X has a probability function for various values

of x as given below:

x 0 1 2 3 4 5 6 7
P(x) 0 k 2k 2k 3k k2 2k2 7k2+k

Find:
(i) k
(ii) P(x < 6)
(iii) P(x ≥ 6)
(iv) P(3 < x ≤ 6)
Solution: To find the value of k, we use the fact that the total
probability must equal 1:

0 + k + 2k + 2k + 3k + k 2 + 2k 2 + (7k 2 + k) = 1
Simplifying this, we have:

10k + 10k 2 − k = 1
This simplifies to:

10k 2 + 9k − 1 = 0
√
2
b −4ac
Using the quadratic formula k = −b± 2a , where a = 10,
b = 9, and c = −1, we find:
p √
−9 ± 92 − 4 · 10 · (−1) −9 ± 81 + 40
k= =
2 · 10 20
 √
−9 ± 121
k=
20
−9 ± 11
k=
20
So, the two possible values for k are:

1
k= = 0.1(we have k = -1 also, but k must be positive)
10
Now, calculate the probabilities:
ii) P(x < 6) = P(0) + P(1) + P(2) + P(3) + P(4) + P(5) =
0 + k + 2k + 2k + 3k + k 2 = 8k + k 2
iii) P(x ≥ 6) = P(6) + P(7) = 2k 2 + (7k 2 + k) = 9k 2 + k
iv) P(3 < x ≤ 6) = P(4)+P(5)+P(6) = 3k +k 2 +2k 2 = 3k +3k 2
Using k = 0.1:

P(x < 6) = 8(0.1) + (0.1)2 = 0.8 + 0.01 = 0.81

P(x ≥ 6) = 9(0.1)2 + 0.1 = 0.09 + 0.1 = 0.19
P(3 < x ≤ 6) = 3(0.1) + 3(0.1)2 = 0.3 + 0.03 = 0.33
Problem 4

Show that the function

(
e −x , x ≥ 0
f (x) =
0, x <0
is a probability density function (PDF). Hence, find
P(1.5 < x < 2.5).

Solution:
To show that f (x) is a probability density function, it must satisfy
two conditions:
1. f (x) ≥ 0 for all x.
2. The total integral of f (x) over all x must equal 1.
(i). Non-negativity:

f (x) = e −x ≥ 0 for all x ≥0

(ii). Integral equals 1:
Z ∞ Z ∞
f (x) dx = e −x dx
−∞ 0
Z ∞ ∞
e −x dx = −e −x 0 = (0 − (−1)) = 1

∴
0
Since both conditions are satisfied, f (x) is a valid probability
density function.
 Z 2.5
P(1.5 < x < 2.5) = e −x dx
1.5
Calculate the integral:
Z 2.5 2.5
e −x dx = −e −x 1.5

1.5

= −e −2.5 + e −1.5

= e −1.5 − e −2.5

1 1
= −
e 1.5 e 2.5
Thus,
1 1
P(1.5 < x < 2.5) = −
e 1.5 e 2.5
Problem 5

A random variable X has a probability density function (PDF)

given by:
(
kx 2 , 0 ≤ x ≤ 3
f (x) =
0, otherwise
Evaluate:
(i) k
(ii) P(x ≤ 1)
(iii) P(x > 1)
(iv) P(1 ≤ x ≤ 2)
(v) P(x ≤ 2)
(vi) P(x ≥ 2)
Solution:
The total probability must be equal to 1, so:
Z ∞ Z 3
f (x) dx = kx 2 dx = 1
−∞ 0

3 3
x3
Z   
27
∴k x 2 dx = k =k −0 = 9k
0 3 0 3
So,
1
9k = 1 =⇒ k =
9
ii) P(x ≤ 1):
Z 1 Z 1
1 2
P(x ≤ 1) = f (x) dx = x dx
0 0 9
 1
1 x3 1 1 1
∴ = · =
9 3 0 9 3 27
iii) P(x > 1):
1 26
P(x > 1) = 1 − P(x ≤ 1) = 1 − =
27 27
iv) P(1 ≤ x ≤ 2):
Z 2 Z 2
1 2
P(1 ≤ x ≤ 2) = f (x) dx = x dx
1 1 9
3 2
   
1 x 1 8 1 1 7 7
∴ = − = · =
9 3 1 9 3 3 9 3 27
v) P(x ≤ 2):
Z 2 Z 2
1 2
P(x ≤ 2) = f (x) dx = x dx
0 0 9
3 2
 
1 x 1 8 8
∴ = · =
9 3 0 9 3 27
vi) P(x ≥ 2):
8 19
P(x ≥ 2) = 1 − P(x ≤ 2) = 1 − =
27 27
Problem 6
The diameter of an electric cable is assumed to be a continuous
random variable with a probability density function (PDF) given by:
(
kx(1 − x), 0 ≤ x ≤ 1
f (x) =
0, otherwise
Find:
(i) The value of k.
(ii) The Mean and Variance.

Solution:
The total probability must be equal to 1:
Z ∞ Z 1
f (x) dx = kx(1 − x) dx = 1
−∞ 0
 Z 1 Z 1 Z 1 
2
∴k x(1 − x) dx = k x dx − x dx
0 0 0
1 1 !
x2 x3
   
1 1
k − =k −
2 0 3 0 2 3
   
3 2 1
k − =k
6 6 6
k
= 1 =⇒ k = 6
6
The mean µ is given by:
Z ∞ Z 1
µ = E [X ] = xf (x) dx = x · 6x(1 − x) dx
−∞ 0
 Z 1 Z 1 Z 1 
2 2 3
∴6 x (1 − x) dx = 6 x dx − x dx
0 0 0
1 !
1
x4
x3
   
1 1
6 − =6 −
0 34 0 3 4
   
4 3 1 6 1
6 − =6 = =
12 12 12 12 2
Thus, the mean µ = 12 .
The variance σ 2 is given by:

σ 2 = E [X 2 ] − (E [X ])2
 Z ∞ Z 1
E [X 2 ] = x 2 f (x) dx = x 2 · 6x(1 − x) dx
−∞ 0
Z 1 Z 1 Z 1 
3 3 4
6 x (1 − x) dx = 6 x dx − x dx
0 0 0
1 !
1
x4x5
  
1 1
6 − =6 −
0 4 5 0 4 5
   
5 4 1 6 3
6 − =6 = =
20 20 20 20 10
3
So, E [X 2 ] = 10 .
 2
2 3 1 3 1 3 2.5 0.5 1
σ = − = − = − = =
10 2 10 4 10 10 10 20
1
Thus, the variance σ 2 = 20 .
Assignment Problems (DRV)
1. Find the value of k such that the following distribution
represents a discrete probability distribution. Hence, find the
Mean, S.D, P(x ≥ 5), and P(3 < x ≤ 6). [Ans: (i) k=1/49,
Mean = 4.14, S.D = 1.64, (ii) 0.48 (iii) 0.67]
x 0 1 2 3 4 5 6
P(x) k 3k 5k 7k 9k 11k 13k
2. A random variable has the following probability function for
the various values of X = x. Find i) the value of k, ii)
P(x < 1), iii) P(x ≥ 1). [Ans: (i) k=0.1 (ii) 0.4 (iii) 0.9]
x -2 -1 0 1 2 3
P(x) 0.1 k 0.2 2k 0.3 k
3. A random variable has the following probability function for
the various values of X = x. Find i) the value of k, ii)
P(x ≤ 1), iii) P(0 ≤ x < 3). [Ans: (i) k=1/32 (ii) 0.18 (iii)
0.5]
x 0 1 2 3 4 5
P(x) k 5k 10k 10k 5k k
Assignment Problems (CRV)
1. A random variable X has the probability density function
(PDF):
(
kx 2 , −3 ≤ x ≤ 3
f (x) =
0, otherwise
Find:
i) k [Ans:1/18]
ii) P(x ≤ 2) [Ans:35/54]
iii) P(x ≥ 2) [Ans:19/54]
iv) P(x > 1) [Ans:26/54]
v) P(1 ≤ x ≤ 2) [Ans:7/54]
2. Find the constant k such that
(
kxe −x , 0 < x < 1
f (x) =
0, otherwise
is a valid probability density function (PDF). Find the mean.
[Ans: (i)k=e/(e-2) (ii)=(2e-5)/(e-2)]