Lecture Slides#3

Chapter#4

O ne R andom Var iable

 O ver view
Generalized random variables (Continuous, Discrete, Mixed Type)
 The Cumulative Distribution Function
 The Probability Density Function
 The Expected Value of X
 Important Continuous Random Variables
 Functions of a Random Variables
 4.1 T he cdf
If the domain of a random variable is real numbers, the probability at a
number, P [X = b], is usually (but not always) zero. The pmf, defined in
terms of events of the form {X = b}, is useless in this case.
The cumulative distribution function (cdf) is an alternative approach to
define the probability of a random variable, which uses events of the form
{X ≤ b}
Definition of cdf: the probability that the random variable X takes on a
value in the set (−∞, x],

FX (x) = P [X ≤ x] for −∞<x<∞

cdf is not limited to discrete RVs and applies to all types of RVs.
It can be used to calculate the probabilities of event of interests.
The events of interest can be intervals of the real line, and their
complements, unions, and intersections.
 Example: 4.1, Three Coin Tosses
Toss the Coin Three Times, Let ‘X’ be the random variable (discrete) which
denotes the number of heads

SX = {0, 1, 2, 3} P=1/2= P[H]=P[T]

pmf: P[X=0] = 1/8, P[X=1] = 3/8, P[X=2] = 3/8, P[X=3] = 1/8

How to build CDF : 𝐹𝑋 𝑥 = 𝑃[𝑋 ≤ 𝑥]

Cumulative Operation
Interval#1: x < 0, 𝐹𝑋 𝑥 = 0

1
Interval#2: 0 ≤ x < 1 𝐹𝑋 𝑥 = , Probability of ‘0’ head
8

1 3 4 1 Probability of ‘0’ head + ‘1’ head

Interval#3: 1 ≤ x<2, 𝐹𝑋 𝑥 = + = =
8 8 8 2

1 3 3 7
Interval#4: 2 ≤ x<3, 𝐹𝑋 𝑥 = + + = Probability of ‘0’ head + ‘1’ head+’2’ head
8 8 8 8
1 3 3 1
Interval#5: x≥3 , 𝐹𝑋 𝑥 = + + + = 1
8 8 8 8
Probability of ‘0’ head + ‘1’ head+’2’ head+’3’ head
 E x 4.1 T hr ee C oin Tosses
Figure below for cdf

where the unit step function:

1
7/8

1/2

1/8

Example of Discrete CDF, ‘X’ is discrete random variable

Pr oper ties of cdf

Amount of jump, if continuous then zero

 Pr oper ties: Only Discrete cdf
𝑃 𝑎 < 𝑋 ≤ 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 (𝑎)

𝑃 𝑎 ≤ 𝑋 ≤ 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 (𝑎− )

𝑃 𝑎 < 𝑋 < 𝑏 = 𝐹𝑋 𝑏 − − 𝐹𝑋 (𝑎)

𝑃 𝑎 ≤ 𝑋 < 𝑏 = 𝐹𝑋 𝑏 − − 𝐹𝑋 (𝑎−)

𝑃 𝑋 = 𝑎 = 𝐹𝑋 𝑎 − 𝐹𝑋 (𝑎− ) Amount of jump at ‘a’

 D is crete -cdf
1 1 1
𝐹𝑋 1 = , 𝐹𝑋 1− = , 𝐹𝑋 1+ = P[X=1]= 3/8 (amount of jump at x=1)
2 8 2

7 −
1 +
7
𝐹𝑋 2 = , 𝐹𝑋 2 = , 𝐹𝑋 2 = , 𝑃 𝑋 = 2 = 3/8
8 2 8

P[X=0.5]=0, P[X=1.1]=0 , P[X=2.5]= 0

1
7/8

1/2

1/8

Example of Discrete CDF, ‘X’ is discrete random variable

 Pr oper ties: Continuous cdf
𝑃 𝑎 < 𝑋 ≤ 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 (𝑎)

𝑃 𝑎 ≤ 𝑋 ≤ 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 (𝑎)
All Same
𝑃 𝑎 < 𝑋 < 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 (𝑎)

𝑃 𝑎 ≤ 𝑋 < 𝑏 = 𝐹𝑋 𝑏 − 𝐹𝑋 (𝑎)

𝑃 𝑋=𝑎 =0
 E xample 4.4, Discrete cdf
Let X be the number of heads in three tosses of fair coins. Use the cdf to find the
probabilities of different events
 Continuous cdf
A continuous random variable X is uniform between 𝑎 < 𝑥 ≤ 𝑏, then its CDF is
given below:

0 𝑥<𝑎
𝑥−𝑎
𝐹𝑋 𝑥 = 𝑎<𝑥≤𝑏
𝑏−𝑎
1 𝑥>𝑏
 Example:4.5, Continuous cdf
Let X be the uniform random in [0, 1] (a=0,b=1), Use the cdf to
calculate the probabilities:
0 𝑥<0
𝐹𝑋 𝑥 = 𝑥 0<𝑥≤1
1 𝑥>1

𝑃 −0.5 < 𝑋 < 0.25 = 𝐹𝑋 0.25 − 𝐹𝑋 −0.5 = 0.25 − 0 = 0.25

𝑃 0.3 < 𝑋 < 0.65 = 𝐹𝑋 0.65 − 𝐹𝑋 0.3 = 0.65 − 0.3 = 0.35

𝑃 𝑋 = 0.5 = 0, 𝑃 𝑋 = 1.5 = 0 𝑃 𝑋 > 1.5 = 1 − 𝑃 𝑋 ≤ 1.5 = 1 − 𝐹𝑋 1.5 = 1 − 1 = 0

𝑃 𝑋 − 0.4 > 0.2 = 1 − 𝑃 𝑋 − 0.4 ≤ 2 = 1 − 𝑃[−0.2 ≤ 𝑋 − 0.4 ≤ 0.2]]

= 1 − 𝑃 0.2 ≤ 𝑋 ≤ 0.6 = 1 − 𝐹𝑋 0.6 − 𝐹𝑋 0.2 = 1 − 0.6 − 0.2 = 0.6

 4.1.1 T he T hr ee Types of RVs
Discrete random variable (generalized definition of a discrete RV) The cdf
is a right-continuous, staircase function of x with jumps at a countable
set of points x1, x2, . . ..

where the pmf : pX (xk ) = P [X = xk ] gives the magnitude of the jump.

Continuous random variable: The cdf FX (x) is continuous everywhere.
P [X = x] = 0 for all x

Random variable of mixed type: The cdf has jumps on a countable set of points and
also increases continuously over at least one interval of values pf x
FX (x) = pF1(x) + (1 − p)F2(x)

where 0 < p < 1, F1(x) is the cdf of a discrete RV, and F2(x) is the cdf of a
continuous RV.
 4.2 T he pdf
Continuous RV The probability of a continuous random variable at a point is
zero, so the behavior of the probability change is useful instead.
pdf (probability density function) by definition, the derivative of cdf
dFX (x)
f X (x) =
dx
Meaning of pdf

FX (x + h) − FX (x) P [x < X ≤ x + h]
f X (x) = lim = lim
h→0 h h→0 h
FX (x + h) − FX (x)
P [x < X ≤ x + h] = h.
h

So, for a very short interval h,

P [x < X ≤ x + h] ≈ f X (x)h

⇒ the pdf f X (x) represents the “density” of probability at the point x

Pr oper ties of pdf

Condition for valid pdf

Pr oper ties of pdf
 E xamples: Uniform R.V
4.6 The pdf and cdf of a uniform random variable: 𝑋~𝑈[𝑎, 𝑏]
 E xamples: Exponential R.V
4.7 The pdf and cdf of an exponential random variable, 𝑋~𝐸𝑥𝑝(𝜆)

= 0, 𝑥 < 0
 E x 4.8 L aplacian R andom Var iable
The pdf of the samples of the amplitude of speech waveforms is found to
decay exponentially at a rate α,

f X (x) = ce−α|x| − ∞ < x < ∞

Find the constant c, and then find the probability P [|X| < v].
 4.2.1 pdf of Discr ete RVs
The derivative of the cdf does not exist when it is not continuous
⇒ pdf of discrete RVs cannot be obtained by direct differentiation.
Several special functions bridge the continuous and discrete RVs.

𝑑𝑢(𝑥)
𝛿 𝑥 =
𝑑𝑥
 pdf of a Discr ete RV
The cdf of a discrete RV:

The pdf for a discrete RV is then

The generalized definition of pdf places a delta function of weight

P [X = xk ] at the points xk where the cdf is discontinuous.
At discrete jumps
 E xample 4.9
Let X be the number of heads in three coin tosses. Find the
pdf of X. Sketch the cdf and pdf.
 Example: Mixed Type R.V.

Consider a mixed type pdf sketched in Figure below. Determine

 The constant ‘K’ K=0.05
 The cdf

f X (x)
0.25 ( x  5) 0.25 ( x  10)

x
0
5 10

10K + 0.25 + 0.25 = 1

K = 0.5/10 = 0.05
 ‘x’ will be used to determine different intervals

𝑥 < 0, 𝐹𝑋 𝑥 = 0

𝑥
0 ≤ 𝑥 < 5, 𝐹𝑋 𝑥 = 0
𝐾𝑑𝑥 = 𝐾𝑥 , 𝑥 = 0, 𝐹𝑋 𝑥 = 0 x = 5, 𝐹𝑋 (𝑥) = 0.25
𝑥
𝑥 = 5, 𝐹𝑋 𝑥 = 0.25𝛿 𝑥 − 5 𝑑𝑥 = 0.25, 𝑑𝑖𝑠𝑐𝑒𝑟𝑒𝑡 𝑗𝑢𝑚𝑝 𝑜𝑓 0.25 𝑎𝑡 𝑥 = 5
0
5 5 𝑥
5 ≤ 𝑥 < 10, 𝐹𝑋 𝑥 = 𝐾𝑑𝑥 + 0.25𝛿 𝑥 − 5 𝑑𝑥 + 𝐾𝑑𝑥
0 0 5

= 5𝐾 + 0.25 + 𝐾𝑥 − 5𝐾 = 𝐾𝑥 + 0.25
x = 5, 𝐹𝑋 (𝑥) = 0.5 x = 10 𝐹𝑋 (𝑥) = 0.75

𝑥
𝑥 = 10, 𝐹𝑋 𝑥 = 0.25𝛿 𝑥 − 10 𝑑𝑥 = 0.25, 𝑑𝑖𝑠𝑐𝑒𝑟𝑒𝑡 𝑗𝑢𝑚𝑝 𝑜𝑓 0.25 𝑎𝑡 𝑥 = 10
0
 Example: Mixed Type R.V.
FX (x)

1.0

0.75

0.5 Slope:K

0.25
Slope:K

0 5 10 x
 Mixed R.V: Probability Calculation
Using pdf

𝑃 0 ≤ 𝑋 < 5 = 5𝐾 = 0.25
𝑃 0 ≤ 𝑋 ≤ 5 = 5𝐾 + 0.25 = 0.5

𝑃 3 ≤ 𝑋 ≤ 7 = 4𝐾 + 0.25 = 0.45

Using CDF

𝑃 0 ≤ 𝑋 < 5 = 𝐹𝑋 5− − 𝐹𝑋 0− = 𝐾5 − 0 = 0.25

𝑃 0 ≤ 𝑋 ≤ 5 = 𝐹𝑋 5 − 𝐹𝑋 0− = 0.5 − 0 = 0.5

𝑃 3 ≤ 𝑋 ≤ 7 = 𝐹𝑋 7 − 𝐹𝑋 3− = 0.6 − 0.15 = 0.45

𝐹𝑋 7 = 𝐾7 + 0.25 = 0.6

𝐹𝑋 3− = 𝐾𝑥 = 3𝐾 = 0.15
 4.3 T he E xpected Value of X
 Calculation of sample mean of independent observations continuous R.V
 For continuous R.V and any specific values of ‘x’, P[X=x]=0
 Divide the real line into small intervals and count the number of times 𝑁𝑘 (𝑛) ,
the observations fall in the interval {𝑥𝑘 < 𝑋 < 𝑥𝑘 + ∆}
𝑁 (𝑛)
 As ‘n’ becomes large, relative frequency, 𝑓𝑘 𝑛 = 𝑘  𝑓𝑋 (𝑥𝑘 )∆ ,
𝑛
probability of interval
 Calculate the sample mean in terms of relative frequencies and let 𝑛 → ∞

𝑋 𝑛 = 𝑥𝑘 𝑓𝑘 (𝑛) → 𝑥𝑘 𝑓𝑋 (𝑥𝑘 )∆ As ∆ goes to zero, the term become an integral

𝑘 𝑘

+∞
𝐸𝑋 = 𝑥𝑓𝑋 𝑥 𝑑𝑥
−∞

If pdf is viewed as the distribution of mass on the real line, then E[X] represent the center
of mass of this distribution
 Example: 4.12
Example 4.12 Mean of a uniform random variable
E x 4.14 M ean of an E xponential RV
4. 3.1 :E xpected Value of Y= g(X )
E x 4.15 E xpected Value of a Sinusoid
Let Y = acos(ωt + Θ) where a, ω, and t are constants, and Θ
is a uniform random variable in the interval (0, 2π). Find E[Y ]
and E[Y 2]
4.3.2 : Var iance of X
 E xamples
4.18 Variance of a uniform random variable
Find the variance of the random variable X that is uniformly
distributed in the interval : [ a, b ].
 4.4 Im por tant C ontinuous RVs
1. Uniform Random Variable
2. Exponential Random Variable
3. Gaussian (Normal) Random Variable
4. Gamma Random Variable
5. Laplacian Random Variable
6. Rayleigh Random Variable
7. Cauchy Random Variable
8. Pareto Random Variable
9. Beta Random Variable
 T he Unifor m R andom Var iable
X ∈ [a, b]: all values in an interval of the real line are equally likely to occur
 T he E xponential RV
Modeling of the time between occurrence of events (e.g., the time
between customer demands for call connections), and in the
modeling of the lifetime of devices and systems
 T he G aussian (Nor m al) RV
Most popular: This random variable appears so often in problem involving randomness
that it has come to be known as the “normal” random variable

pdf cdf
 Standar dized G aussian RV
 The cdf of Gaussian R.V is given by:

 Difficult to calculate this integral for probability calculation. Using change of

variables to form the Standard Gaussian R.V and then use specialized
functions given for these Standard Gaussian R.V for probability calculation
 Change of variable: 𝑡 = (𝑥 ′ − 𝑚)/𝜎

 𝜑 𝑥 is the cdf of Standard Gaussian R.V with m=0, 𝜎 = 1, 𝑋~𝑁(0,1)

 Standar dized G aussian RV
 Any probability involving an arbitrary Gaussian R.V can be
expressed in terms of 𝜑 𝑥
 𝜑 𝑥 values are given in terms of tables and can also be
calculated using Matlab built-in functions
 In Electrical engineering it is customary to work with Q-function

 Q(x) is simply the probability of the “tail” of the pdf.

 Because of the symmetry of the pdf Q(0)= ½, Q(-x)= 1 – Q(x)
 Q(x) are listed in Tables 4.2 and 4.3.
Appr oximating the G aussian RV
Probability Calculation for Normal R.V
𝑋~𝑁(0,1)

𝑃 𝑋 ≤ 𝑥 = 𝜑 𝑥 = 1 − 𝑄(𝑥)

𝑃 𝑎 < 𝑋 ≤ 𝑏 = 𝜑 𝑏 − 𝜑 𝑎 = 𝑄 𝑎 − 𝑄(𝑏) Assume b > a

𝑃 𝑋 > 𝑥 = 1 − 𝑃 𝑋 ≤ 𝑥 = 1 − 𝜑 𝑥 = 𝑄(𝑥)

𝑋~𝑁(𝑚, 𝜎 2)

𝑥−𝑚 𝑥−𝑚
𝑃 𝑋≤𝑥 =𝜑 = 1 − 𝑄( )
𝜎 𝜎
𝑏−𝑚 𝑎−𝑚 𝑎−𝑚 𝑏−𝑚
𝑃 𝑎<𝑋≤𝑏 =𝜑 −𝜑 =𝑄 − 𝑄( )
𝜎 𝜎 𝜎 𝜎

𝑥−𝑚 𝑥−𝑚
𝑃 𝑋 >𝑥 =1−𝑃 𝑋 ≤𝑥 =1−𝜑 = 𝑄( )
𝜎 𝜎
 E xample 4.22
The Gaussian random variable plays a very important role in communication systems.
Transmission signals are corrupted by noise voltages resulting from the thermal motion of
electrons. These voltages usually have a Gaussian pdf.
A communication system accepts a positive voltage V as input and
outputs a voltage Y = αV + N , where α = 10−2 and N ∼ N (0, 2). Find the
value of V giving P [Y< 0] = 10−6.
 4.5: Functions of a Random Variable

Let X be a r.v defined on the and suppose g(x) is a function of

the variable x. Define
Y  g ( X ).

Is Y necessarily a r.v? If so what is its cdf FY ( y), pdf fY ( y) ?

Clearly if Y is a r.v, then for every set B, the set of  for which
Y ( )  B . In that case if X is a r.v, so is Y, and for every set B
P(Y  B)  P( X  g 1 ( B )).
In particular
 
FY ( y )  P(Y ( )  y )  Pg ( X ( ))  y   P X ( )  g 1 ( y ] .

Thus the distribution function as well of the density function

of Y can be determined in terms of that of X. To obtain the
distribution function of Y, we must determine the set on the
x-axis such that X ( )  g 1
( y ) for every given y, and the
probability of that set.
 Example:4.31 Y  aX  b a  0.

 y b  y b
FY ( y )  P Y ( )  y   P aX ( )  b  y   P X ( )    FX  .
 a   a 

1  y b
fY ( y )  f X  .
a  a 

On the other hand if a  0, then

 y b
FY ( y )  PY ( )  y   PaX ( )  b  y   P X ( )  
 a 
 y b
 1  FX   ,
 a 
and hence:
1  y b
fY ( y )   f X  .
a  a 
 1  y b
From both, we obtain (for all a) fY ( y )  fX  .
|a |  a 
 E xample 4.32
Let X be a random variable with a Gaussian pdf with mean m and σ.

Let Y = aX + b, then find the pdf of Y :

A linear function of a Gaussian RV is also a Gaussian RV.

𝑌~𝑁(𝑎𝑚 + 𝑏, 𝑎 𝜎)
 E xample 4.33
Let Y = X2 where X is a continuous random variable. Find the cdf and pdf of Y :

𝐹𝑌 𝑦 = 𝑃[𝑌 ≤ 𝑦]

𝐹𝑌 𝑦 = 𝑃[𝑋 2 ≤ 𝑦]

X = ±√𝑦

𝐹𝑌 𝑦 = 𝑃[− 𝑦 ≤ 𝑋 ≤ 𝑦]
 E xample 4.34
1 −𝑥 2
𝑋~𝑁(0,1) 𝑓𝑋 𝑥 = 𝑒 2
2𝜋

𝑌 = 𝑋2 𝑋=± 𝑦

𝑓𝑌 𝑦

−𝑦 −𝑦
1 1 1
= [ 𝑒 2 + 𝑒 2 ]
2 𝑦 2𝜋 2𝜋
−𝑦
2 1
= [
2 𝑦 2𝜋
𝑒 2 ]

1 1 −𝑦
= 𝑒2 , y>0
𝑦 2𝜋

Chi-Square random variable