Adama Science and Technology University

School of Electrical Engineering and Computing

Department of Electronics and Communication Engineering

Probability and Random Process (ECEg3103)

Chapter 4: Expectation
By
Gemechu D.
Outline

Expected Value, Variance and Moments

Some Special Distributions

Functions of One Random Variable

2
Expected Value, Variance and Moments
i. Expected Value (Mean)
 The expected value (mean) of a continuous random variable X,
denoted by μX or E(X), is defined as:

 X  E ( X )   xf X ( x)dx


 Similarly, the expected value of a discrete random variable X is

given by:

 X  E ( X )   xk PX ( xk )
k

 Mean represents the average value of the random variable in a

very large number of trials.
3
Expected Value, Variance and Moments Cont’d…..
ii. Variance
 The variance of a continuous random variable X, denoted by
σ2X or VAR(X), is defined as:

 2 X  Var ( X )  E[( X   X ) 2 ]

 2
X  Var ( X )   ( x   X ) 2 f X ( x)dx


 Expanding (x-μX )2 in the above equation and simplifying the

resulting equation, we will get:

 2 X  Var( X )  E( X 2 )  [ E( X )]2

4
 Expected Value, Variance and Moments Cont’d…..
 The variance of a discrete random variable X is given by:

 2
X  Var ( X )   ( xk   X ) PX ( xk )
2

 The standard deviation of a random variable X, denoted by σX, is

simply the square root of the variance, i.e.,

 X  E ( X   X ) 2  Var ( X )

iii.Moments
 The nth moment of a continuous random variable X is defined as:

E ( X )   x n f X ( x)dx ,
n
n 1

5
Expected Value, Variance and Moments Cont’d…..

 Similarly, the nth moment of a discrete random variable X is

given by:

E ( X )   xk PX ( xk ) , n 1
n

 Mean of X is the first moment of the random variable X.

6
Some Special Distributions

i. Continuous Probability Distributions

1. Normal (Gaussian) Distribution
 The random variable X is said to be normal or Gaussian random
variable if its pdf is given by:
1  ( x   ) 2 / 2 2
f X ( x)  e .
2 2
 The corresponding distribution function is given by:
x 1  x 
FX ( x)   e ( y   ) 2 / 2 2
dy  G 

2 2
  
x 1 y /2
where G ( x)   e dy
2


2
7
Some Special Distributions Cont’d……

 The normal or Gaussian distribution is the most common

continuous probability distribution.
f X (x)

x

Fig. Normal or Gaussian Distribution

8
 Some Special Distributions Cont’d……

2. Uniform Distribution f X (x)

1
 1 ba
 , a xb
f X ( x)   b  a a b
x

 0, otherwise. Fig. Uniform Distribution

3. Exponential Distribution
f X (x )

 1
 e  x /  , x  0,
f X ( x)    x

 0, otherwise. Fig. Exponential Distribution

9
Some Special Distributions Cont’d……

4. Rayleigh Distribution

 x  x 2 / 2 2
 2e , x  0,
f X ( x )  

 0, otherwise.

5. Cauchy Distribution
 /
f X ( x)  2 ,    x  .
  (x  ) 2

6. Laplace Distribution
1 |x|/ 
f X ( x)  e ,    x  .
2
10
Some Special Distributions Cont’d….

i. Discrete Probability Distributions

1. Bernoulli Distribution
P( X  0)  q, P( X  1)  p.
2. Binomial Distribution
 n  k n k
P( X  k )  
k p q , k  0,1,2,, n.
 
3. Poisson Distribution
k
P( X  k )  e   , k  0,1,2, , .
k!

11
Some Special Distributions Cont’d….

4. Hypergeometric Distribution
m  N m 
   
k   n k 
P( X  k )    
N 

, max(0, m  n  N )  k  min( m, n )
 
n 
 

5. Geometric Distribution

P( X  k )  pqk , k  0,1,2,, , q  1  p.

6. Negative Binomial Distribution

 k  1 r k  r
P( X  k )    p q , k  r, r  1, .
 r 1
12
Random Variable Examples

Example-1:
The pdf of a continuous random variable is given by:
kx , 0  x 1
f X ( x)  
0 , otherwise

where k is a constant.

a. Determine the value of k .

b. Find the corresponding cdf of X .

c. Find P(1 / 4  X  1)

d . Evaluate the mean and variance of X .

13
Random Variable Examples Cont’d……
Solution:
 1
a. 
f X ( x ) dx  1  
0
kxdx  1
 x2 1
 k    1
 2 0
k
 1
2
k  2

2 x, 0  x 1
 f X ( x)  
0, otherwise
14
Random Variable Examples Cont’d……
Solution:
b. The cdf of X is given by :
x
FX ( x)   
f X (u ) du
Case 1 : for x  0
FX ( x)  0, since f X ( x )  0, for x  0
Case 2 : for 0  x  1
x x x
FX ( x )   f X (u ) du   2udu  u  x2
2
0 0 0

15
Random Variable Examples Cont’d……
Solution:

Case 3 : for x  1
1 1 1
FX ( x)   f X (u ) du   2udu  u 1
2
0 0 0
 The cdf is given by
0, x0
 2
FX ( x)   x , 0  x 1
1, x 1


16
Random Variable Examples Cont’d……
Solution:
c. P (1 / 4  X  1)
i. Using the pdf
1 1
P (1 / 4  X  1)   f X ( x) dx   2 xdx
1/ 4 1/ 4

1
 P (1 / 4  X  1)  x 2
 15 / 16
1/ 4
 P (1 / 4  X  1)  15 / 16
ii. Using the cdf
P (1 / 4  X  1)  FX (1)  FX (1 / 4)
 P (1 / 4  X  1)  1  (1 / 4) 2  15 / 16
 P (1 / 4  X  1)  15 / 16
17
Random Variable Examples Cont’d……
Solution:
d. Mean and Variance
i. Mean
1 1
 X  E ( X )   xf X ( x)dx   2 x 2 dx
0 0

2 x3 1
 X   2/3
3 0
ii. Variance
 X 2  Var ( X )  E ( X 2 )  [ E ( X )]2
1 1
E ( X )   x f X ( x)dx   2 x 3 dx  1 / 2
2 2
0 0

  X  Var ( x)  1 / 2  (2 / 3) 2  1 / 18
2

18
Random Variable Examples Cont’d……..

Example-2:
Consider a discrete random variable X whose pmf is given by:

1 / 3 , xk  1, 0, 1
PX ( xk )  
0 , otherwise

Find the mean and variance of X .

19
Random Variable Examples Cont’d……
Solution:
i. Mean
1
 X  E( X )  x
k  1
k PX ( xk )  1 / 3(1  0  1)  0

ii. Variance
 X 2  Var ( X )  E ( X 2 )  [ E ( X )]2
1
E( X )   k X k     ]  2/3
2 2 2 2 2
x P ( x ) 1 / 3[( 1) ( 0) (1)
k  1

  X  Var ( x)  2 / 3  (0) 2  2 / 3
2

20
 Functions of One Random Variable
 Let X be a continuous random variable with pdf fX(x) and suppose
g(x) is a function of the random variable X defined as:
Y  g (X )

 We can determine the cdf and pdf of Y in terms of that of X.

 Consider some of the following functions.
aX  b
sin X X2

1
Y  g( X ) |X |
X
X
log X
eX | X | U ( x)

21
 Functions of a Random Variable Cont’d…..
 Steps to determine fY(y) from fX(x):
Method I:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. Determine the cdf of Y using the following basic approach.

FY ( y )  P( g ( X )  y)  P(Y  y )

3. Obtain fY(y) from FY(y) by using direct differentiation, i.e.,

dFY ( y )
fY ( y ) 
dy

22
 Functions of a Random Variable Cont’d…..
Method II:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. If Y=g(X) is one to one function and has an inverse
transformation x=g-1(y)=h(y), then the pdf of Y is given by:
dx dh( y )
fY ( y )  f X ( x)  f X [h( y )]
dy dy
3. Obtain Y=g(x) is not one-to-one function, then the pdf of Y can
be obtained as follows.
i. Find the real roots of the function Y=g(x) and denote them by xi

23
 Functions of a Random Variable Cont’d…..

ii. Determine the derivative of function g(xi ) at every real root xi ,

i.e. ,
dxi
g ( xi ) 
dy
iii. Find the pdf of Y by using the following formula.

dxi
f Y ( y)   f X ( xi )   g ( xi ) f X ( xi )
i dy i

24
Examples on Functions of One Random Variable

Examples:
a. Let Y  aX  b. Find f Y ( y ).

b. Let Y  X 2 . Find f Y ( y ).

1
c. Let Y  . Find f Y ( y ).
X
 
d . The random variable X is uniform in the interval [ , ].
2 2
If Y  tan X , determine the pdf of Y .

25
Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b

i. Using Method  I

Suppose that a  0

 y b
Fy ( y )  P (Y  y )  P (aX  b  y )  P X  
 a 

 y b
FY ( y )  FX  
 a 
dFY ( y ) 1  y b
 f Y ( y)   fX   (i )
dy a  a 

26
Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b

i. Using Method  I

On the other if a  0, then

 y b
Fy ( y )  P (Y  y )  P(aX  b  y )  P X  
 a 

 y b
FY ( y )  1  FX  
 a 
dFY ( y ) 1  y b
 f Y ( y)    fX   (ii)
dy a  a 
27
Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b

i. Using Method  I

From equations (i ) and (ii) , we obtain :

1  y b
f Y ( y)  fX  , for all a
a  a 

28
Examples on Functions of One Random Variable…..
Solutions:

a. Y  aX  b
ii. Using Method  II
The function Y  aX  b is one - to - one and the range
space of Y is R
y b
For any y, x   h( y ) is the principal solution
a
dx dh( y ) 1 dx 1
   
dy dy a dy a
dh y   y b
f X h( y )   fY ( y ) 
dx 1
fY ( y )  f X ( x)  fX  
dy dy a  a 

29
Examples on Functions of One Random Variable…..
Solutions:

b. The function Y  X 2 is not one - to - one and the range

space of Y is y  0
For each y  0, there are two solutions given by
x1   y and x 2  y

30
Examples on Functions of One Random Variable…..
Solutions:
dx1 1 dx1 1
b.    and
dy 2 y dy 2 y

dx 2 1 dx 2 1
  
dy 2 y dy 2 y

dxi dx1 dx 2
f Y ( y)   f X ( xi )  f Y ( y )  f X ( x1 )  f X ( x2 )
i dy dy dy

 1
2 y f X  y   f  y ,
X y0
 f Y ( y)  

 0, otherwise
31
Examples on Functions of One Random Variable…..
Solutions:
1
c. The function Y  is one - to - one and the range
X
space of Y is IR /0

1
For any y, x   h( y ) is the principal solution
y
dx dh( y ) 1
  2
dy dy y

dh y  1
f X h( y )   f Y ( y )  2 f X  
dx 1
f Y ( y)  f X ( x) 
dy dy y  y

1
IR /0
1
 f Y ( y)  2
f X 
  ,
y  y
32
Examples on Functions of One Random Variable…..
Solutions:
d . The function Y  tan X is one - to - one and the range

space of Y is (, )

For any y, x  tan 1 y  h( y ) is the principal solution

dx dh( y ) 1
 
dy dy 1 y2

dh y  1/ 
f X h( y )   f Y ( y ) 
dx
f Y ( y)  f X ( x) 
dy dy 1 y2

1
 f Y ( y)  ,   y  
 (1  y )
2

33
Examples on Functions of One Random Variable…..
Solutions:

34
...Z end of chapter 4...

35