Addis Ababa Science & Technology University
Department of Electrical & Computer Engineering
Probability and Random Process (EEEg-2114)
Chapter 4:
Functions of Random
Variables
Outline
• Introduction
• Functions of One Random Variable
• Functions of Two Random
Variables
• Functions of n Random Variables
• Expectation
• Transforms
moment Generating Functions
characteristic Functions
2
Introduction
Most modern systems today are composed of many
subsystems in
which the output of one system serves as the input to
another.
another
When the input to a system or a subsystem is random,
so is the
The first step in this process is to find the cdf of the
output.
output To evaluate system performance,
system performance it is
necessary
output to take
if we know the pmf or density of the random input.
into account this randomness.
We consider systems modeled by real-valued functions
g(x).
The system input is a random variable X, and the
system output is the random variable Y = g(X). To find
FY (y), observe that
3
Scenario 1: Consider an electrical circuit whose
random input voltage X is first amplified by a gain μ > 0
and then added to a constant offset voltage β . Then the
output voltage is Y = μX +β . If the input voltage is a
continuous random variable X, find the density of the
output voltage Y.
Solution. Although the question asks for the density
of Y, it is more advantageous to find the cdf first and
then differentiate to obtain the density. Write
If X has density fX
4
Scenario 2: In wireless communications systems,
fading is sometimes modeled by lognormal random
variables. We say that a positive random variable Y
is lognormal if lnY is a normal random variable. Find
the density of
Solution. PutY if
X lnY ∼ N(m,σ2).
= lnY so that Y = eX, where X ∼
N(m,σ2). Although the question asks for the density of
Y, it is more advantageous to find the cdf first and then
differentiate to obtain the density. To begin, note that
since Y = eX is positive, if y ≤ 0, FY (y) = P(Y ≤y) = 0. For
y > 0, write
By the chain rule
Using the fact that X ∼ N(m,σ2),
5
Scenario 3: Amplitude
Amplitude modulation
modulation in certain
communication systems can be accomplished using various
nonlinear devices such as a semiconductor diode. Suppose
we model the nonlinear device by the function Y = X2. If the
input X is a continuous random variable, find the density of
the output Y = X2.
Solution. Although the question asks for the density
of Y, it is more advantageous to find the cdf first and
then differentiate to obtain the density. To begin, note that
since Y =X2 is nonnegative, for y < 0, FY (y) = P(Y ≤ y) =
0. For nonnegative y, write
Then, the density is
6
When the diode input voltage X of the preceding
example is N(0,1), it turns out that Y is chi-
squared with one degree of freedom .
If X is N(m,1) with m = 0, then Y is noncentral
chi-squared with one degree of freedom .
These results are frequently used in the analysis
of digital communication systems.
7
Scenario 4: Find the cdf and density of Y = g(X),
if X ∼ uniform[0,4], and
Solution.
We begin by sketching g as shown in Figure below(a).
Since 0 ≤ g(x) ≤ 1, we can never have Y = g(X) < 0, and
we always have Y = g(X) ≤ 1. Hence, we immediately
have
8
To deal with 0 ≤ y < 1, draw a horizontal line at level y as
shown in Figure 5.9(b). Also drop vertical lines where the
level crosses the curve g(x). In Figure 5.9(b) the vertical
lines intersect the x-axis at u and v. Observe also that g(x)
≤ y if and only if x ≤ u or x ≥ v.
Hence, for 0 ≤ y < 1,
Since X ∼ uniform[0,4],
It remains to find u and v. From Figure 5.9(b), we see
that g(u) = y, and since 0 ≤ u < 1, the formula for g(u) is
g(u) = u. Hence, g(u) = y implies u = y. Similarly, since
g(v) = y and 2 ≤ v < 3, the formula for g(v) is g(v) = 3−v.
Solving 3−v = y yields v = 3−y. We can now simplify
The complete formula for FY
(y) is
9
Examination of this formula shows that there are jump
discontinuities at y = 0 and y = 1. Both jumps are of
height 1/4. See Figure 5.10(a). Jumps in the cdf mean
there are corresponding impulses in the density. The
complete density formula is
10
(1). Functions of One Random Variable
Let X be a continuous random variable with pdf fX(x) and suppose g(x) is a function of
the random variable X defined as:
Y g ( X )
We can determine the cdf and pdf of Y in terms of that of X.
Consider some of the following functions.
aX b
sin X X2
1 |X |
Y g ( X )
X
X
log X
eX | X | U ( x)
11
Functions of a Random Variable Cont’d…..
Steps to determine fY(y) from fX(x):
Method I:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. Determine the cdf of Y using the following basic approach.
3. Obtain fY(y) from FY(y) by using direct differentiation, i.e.,
FY ( y ) P ( g ( X ) y ) P (Y y )
dFY ( y )
fY ( y )
dy
12
Functions of a Random Variable Cont’d…..
Method II:
1. Sketch the graph of Y=g(X) and determine the range space of Y.
2. If Y=g(X) is one to one function and has an inverse transformation x=g-1(y)=h(y), then the pdf of Y is given by:
3. If Y=g(x) is not one-to-one function,
function then the pdf of Y can be obtained as follows.
i. Find the real roots of the function Y=g(x) and denote them by xi
dx dh( y )
fY ( y ) f X ( x) f X [h( y )]
dy dy
13
Functions of a Random Variable Cont’d…..
ii. Determine the derivative of function g(xi ) at every real root xi , i.e. ,
dx
g ( xi )
iii. Find the pdf of Y by using the followingi formula.
dy
dxi
f Y ( y ) f X ( x i ) g ( x i ) f X ( x i )
i dy i
14
Examples on Functions of One Random Variable
Examples:
a. Let Y aX b. Find f Y ( y ).
b. Let Y X 2 . Find f Y ( y ).
1
c. Let Y . Find f Y ( y ).
X
d . The random variable X is uniform in the interval [ , ].
2 2
If Y tan X , determine the pdf of Y .
15
Examples on Functions of One Random Variable…..
Solutions:
a. Y aX b
i. Using Method I
Suppose that a 0
y b
Fy ( y ) P (Y y ) P (aX b y ) P X
a
y b
FY ( y ) FX
a
dFY ( y ) 1 y b
f Y ( y) fX (i )
dy a a
16
Examples on Functions of One Random Variable…..
Solutions:
a. Y aX b
i. Using Method I
On the other if a 0, then
y b
Fy ( y ) P (Y y ) P (aX b y ) P X
a
y b
FY ( y ) 1 FX
a
dFY ( y ) 1 y b
f Y ( y) f X (ii )
dy a a
17
Examples on Functions of One Random Variable…..
Solutions:
a. Y aX b
i. Using Method I
From equations (i ) and (ii ) , we obtain :
1 y b
f Y ( y) f X , for all a
a a
18
Examples on Functions of One Random Variable…..
Solutions:
a. Y aX b
ii. Using Method II
The function Y aX b is one - to - one and the range
space of Y is IR
y b
For any y, x h( y ) is the principal solution
a
dx dh( y ) 1 dx 1
dy dy a dy a
dx dhy 1 y b
f Y ( y) f X ( x) f X h( y ) f Y ( y ) f X
dy dy a a
19
Examples on Functions of One Random Variable…..
Solutions:
b. The function Y X 2 is not one - to - one and the range
space of Y is y 0
For each y 0, there are two solutions given by
x1 y and x 2 y
20
Examples on Functions of One Random Variable…..
Solutions:
dx1 1 dx1 1
b. and
dy 2 y dy 2 y
dx2 1 dx2 1
dy 2 y dy 2 y
dxi dx1 dx2
f Y ( y ) f X ( xi ) f Y ( y ) f X ( x1 ) f X ( x2 )
i dy dy dy
1
2 y f X y f X
y , y 0
f Y ( y )
0, otherwise
21
Examples on Functions of One Random Variable…..
Solutions:
1
c. The function Y is one - to - one and the range
X
space of Y is IR /0
1
For any y , x h( y ) is the principal solution
y
dx dh( y ) 1
2
dy dy y
dx dhy 1 1
f Y ( y) f X ( x) f X h( y ) f Y ( y ) 2 f X
dy dy y y
1 1
f Y ( y) 2
f X
, IR /0
y y
22
Examples on Functions of One Random Variable…..
Solutions:
d . The function Y tan X is one - to - one and the range
space of Y is ( , )
For any y, x tan 1 y h( y ) is the principal solution
dx dh( y ) 1
dy dy 1 y2
dx dhy 1/
f Y ( y) f X ( x) f X h( y ) f Y ( y )
dy dy 1 y2
1
f Y ( y) , y
(1 y )
2
23
Examples on Functions of One Random Variable…..
Solutions:
24
ii. Functions of Two Random
Variables
Consider the following functions of
two random variables X and Y,
25
For example
(a). In a telephone channel the signal X is corrupted
by additive noise Y.
(b). In a wireless channel, the signal X is corrupted
by fading (multiplicative noise Y).
(c). If X and Y are the traffic rates at two
different routers of an Internet service provider, it
is desirable to have these rates less than the
router capacity, say u; i.e., we want max(X,Y) ≤
u.
(d). If X and Y are sensor voltages, we may want to
trigger an alarm if at least one of the sensor
voltages falls below a threshold v; e.g., if min(X,Y)
≤ v.v 26
Figure 3.1. Systems represented by operations on
random variables
We now show that the cdfs of these four functions of
X and Y can be expressed in the form P((X,Y) ∈ A) for
various sets A ⊂ IR2.
We then argue that such probabilities can be computed
in terms of the joint cumulative distribution27
function to be defined later in the section.
Example 3.1 (signal in additive noise).
A random signal X is transmitted over a channel
subject to additive noise Y. The received signal is Z
= X +Y. Express the cdf of Z in the form P((X,Y) ∈
Az) for some set Az.
Solution. Write FZ(z) = P(Z ≤ z) = P(X +Y ≤ z) = P((X,Y) ∈
Az),
Where Az = {(x,y) : x+y ≤ z}.
Figure 3.3. The shaded region is Az = {(x,y)=
x+y ≤ z}. 28
The equation of the diagonal line is y = −x+z.
Example 3.2 (signal in multiplicative noise).
noise A
random signal X is transmitted over a channel subject to
multiplicative noise Y. The received signal is Z = XY.
Express the cdf of Z in the form P((X,Y) ∈ Az) for some set
Az.
Solution. Write
FZ(z) = P(Z ≤ z) = P(XY ≤ z) = P((X,Y)
∈ Az),
Where Az = {(x,y) : xy ≤
z}.
To see how to sketch this set, we are tempted to write
Az = {(x,y) : y ≤ z/x}, but this would be wrong
because if x < 0 we need to reverse the inequality.
To get around this problem, it is convenient to partition Az
into two disjoint regions,
29
Since we know the sign of x in each set. Hence, it is
correct to write
Figure 3.4. The curve is y = z/x. The shaded region to the
left of the vertical axis is A −z = {(x,y) : y ≥ z/x, x < 0},
and the shaded region to the right of the vertical axis is
A+z = {(x,y) : y ≤ z/x, x > 0}. The sketch is for the case z
> 0.
How would the sketch need to change if z = 0 or if z30
< 0?
Example 3.3. Express the cdf of U = max(X,Y) in the form
P((X,Y) ∈ Au) for some set Au.
Solution. To find the cdf of U, begin with
FU(u) = P(U ≤ u) = P(max(X,Y) ≤
u).
Since the larger of X and Y is less than or
equal to u
ifwhere
and only
Au =if {(x,y)
X ≤ u and
: x ≤Yu≤and
u, y ≤ u} is the shaded
region
Figure 3.5. (a) left {(x,y) : x ≤ u and y ≤ u}.
31
(b) right {(x,y) : x ≤ v or y ≤ v}.
Example 3.4. Express the cdf of V = min(X,Y) in the
form P((X,Y) ∈ Av) for some set Av.
Solution. To find the cdf of V, begin with
FV (v) = P(V ≤ v) = P(min(X,Y) ≤ v).
Since the smaller of X and Y is less than or equal to v
if and only either X ≤ v or Y ≤ v,
P(min(X,Y) ≤ v) = P(X ≤ v or Y ≤ v) = P((X,Y) ∈ Av),
where Av = {(x,y) : x ≤ v or y ≤ v} is
the shaded region shown in Figure
3.5(b).
32
In general, we analyze problems of the ff
type:
X Y
max( X , Y ) XY
min( X , Y ) Z g ( X ,Y ) XY
X 2 Y 2 X /Y
tan 1 ( X / Y )
The cdf of Z is given by:
FZ ( z ) PZ ( ) z Pg ( X , Y ) z
f XY ( x, y )dxdy,
x, y
33
Example -1: Let Z = X + Y. Find
Solution: f Z (z ).
z y
FZ ( z ) P X Y z f XY ( x, y )dxdy ,
Since the required regiony in x
the xy plane where
is the shaded area shown below to the x yleft
z of
the line x y z.
Integrating over the horizontal strip along
the x-axis first (inner integral) followed by
sliding that strip
along
the y-axis from to
(outer integral)
y we cover the entire
shaded area.
x z y
34
We can find by differentiating
directly. f Z (z ) FZ (z )
In this context, it is useful to recall the
differentiation rule by Leibnitz.
b( z ) Suppose
H ( z ) h ( x, z )dx.
a( z)
Then,
dH ( z ) db( z ) da ( z ) b ( z ) h ( x , z )
h b( z ), z h a ( z ), z dx.
dz dz dz a( z) z
Using the above two equations, we get
z y z y f
XY ( x , y )
f Z ( z ) f XY ( x, y )dx dy f XY ( z y , y ) 0 dy
z z
f XY ( z y , y )dy. (i)
Alternatively, the above integration can be
carried out first along the y-axis followed by 35
In the second case
z x y
FZ ( z ) f XY ( x, y )dxdy ,
x y
and differentiating the above
equation gives y z x
dFZ ( z ) z x
f Z ( z) f XY ( x, y )dy dx x
dz x
z y
x
(ii)
f XY ( x, z x )dx.
If X and Y are independent, then
f XY ( x, y ) f X ( x ) fY ( y ) (iii)
and inserting equation (iii) into equations (i) and
(ii), we get
f Z ( z ) f X ( z y ) fY ( y )dy f X ( x ) fY ( z x )dx. (iv)
y x
36
exponential r.vs with common parameter , and let
Z = X + Y. Determine
Solution: We have
f Z (z ). and we
x y
can make use of (13) eobtain
f X ( x ) to U ( x ), the e ofU Z
fY ( y )pdf ( y ),= X +
Y.
z z
f Z ( z ) e
2 x
e ( z x )
dx e
2 z
dx z2 z
e U ( z ).
0 0
As the next example shows, care should be taken in
using the convolution formula for r.vs with finite
range.
Example-2: X and Y are independent f Z (z ), uniform r.vs
Z X Y (0,1).
in the common interval 0 z Determine
2
where Z = X + Y. Solution: Clearly,
here, and as following figure shows there
are two cases of z for which the shaded areas
37
are quite different in shape and they should
y y
x z y
x z y
x x
(a ) 0 z 1 ( b) 1 z 2
For 0 z 1,
z z y z z2
FZ ( z ) 1 dxdy ( z y )dy , 0 z 1.
y 0 x 0 y 0 2
For 1 z 2, notice that it is easy to deal with
the unshaded region. In that case
1 1
FZ ( z ) 1 P Z z 1 1 dxdy
y z 1 x z y
1 (2 z )2
1 (1 z y )dy 1 , 1 z 2. 38
y z 1 2
Finally, differentiating the cdf, we obtain
dFZ ( z ) z 0 z 1,
f Z ( z)
dz 2 z, 1 z 2.
By direct convolution of
f X ( x) fand we
Y ( y ),
obtain the same result as above. In fact, for
0 z 1
[Figure (a)]
z
f Z ( z ) f X ( z x ) fY ( x )dx 1 dx z.
0
1 z 2
and for [Figure (b)]
1
f Z ( z ) 1 dx 2 z.
z 1
f Z (z )
Figure (c) shows which agrees with the
convolution of two rectangular waveforms as well.
39
fY (x ) f X ( z x) f X ( z x ) fY ( x )
x x x
1 z 1 z z
( a ) 0 z 1
fY (x ) f X ( z x) f X ( z x ) fY ( x )
x x x
1 z 1 z
z 1 1
(b) 1 z 2
f Z (z )
z
0 1 2
(c) 40
Example-3: Let Determine its
Z X Y . f Z (z ).
pdf
Solution: z y
FZ ( z ) P X Y z f XY ( x, y )dxdy
y x
and hence
dF ( z ) z y
fZ ( z) Z
f XY ( x, y )dx dy f XY ( y z , y )dy.
dz y
z x
If X and Y are independent, then the above
formula reduces
f Z ( z ) f Xto
( z y ) fY ( y )dy f X ( z ) f Y ( y ),
f X ( z) fY (z ).
which represents the
y convolution of with
y x y z
x y z
x
41
Example-4: Given Z = X / Y, obtain its density
function.
Solution: We have FZ ( z ) P X / Y z .
X / Y z The inequality X Yz 0,
Ycan
be rewritten
X Yz Y as
0. if and if
P X / Y z P X / Y z, Y 0 P X / Y z, Y 0
P X Yz , Y 0 P X Yz , Y 0 .
Figure (a) shows the area corresponding to the
first term, and Figure (b) shows that
corresponding to the second term in the above
equation. y y
x yz
x yz x
x
(a) (b)
42
Integrating over these two regions, we get
yz 0
FZ ( z ) f XY ( x, y )dxdy f XY ( x, y )dxdy.
y 0 x y x yz
Differentiation with respect to z gives
0
f Z ( z ) yf XY ( yz, y )dy ( y ) f XY ( yz, y )dy
0
| y | f XY ( yz, y )dy, z .
Note that if X and Y are nonnegative random
variables, then the area of integration reduces
y
to that shown below
x yz
43
Jacobian method of
finding pdf of functions of
multiple varibles- one –to-
one & that has inverse
transformation
44
45
46
47
48
Exercise
The joint pdf of two random variables X and Y is given by:
x y , 0 x 1, 0 y 1
f XY ( x, y )
0 , otherwise
Find the pdf of Z if :
a. Z X Y c. Z XY
X
b. Z d. Z X Y
Y
49
Assignment-III
1. Suppose that two continuous random variables X and Y
have joint pdf given by:
k ( 2 x y ) , 2 x 6, 0 y 5
f XY ( x, y )
0 , otherwise
where k is a constant.
a. Find the constant k .
b. Find the joint cdf of X and Y .
c. Find the marginal cdf and pdf of X and Y .
d . Are X and Y independen t?
e. Find the conditional pdf of X and Y .
e. Evaluate the following probabilities.
i. P (3 X 4, Y 2) iii. P ( X Y 4)
ii. P ( X 3) iv. P (0 Y 2) 50
Assignment-III Cont’d……
2. Let the joint pmf of two discrete random variables X and
Y is given by:
k ( xi y j ) , xi 1, 2, 3; y 1, 2
PXY ( xi , y j )
0 , otherwise
where k is a constant.
a. Find the value of k .
b. Find the marginal pmf of X and Y .
c. Are X and Y independent?
51
Assignment-III Cont’d…..
3. Suppose that two continuous random variables X and Y
are independent and uniform in the interval (0, 4).
Find the pdf of Z if :
a. Z X Y
b. Z Y X
c. Z X Y
d . Z XY
Y
e. Z
X
52
Transform methods
We will see two transforms that help us recover
probability mass functions and densities and that
greatly simplify finding expectations, variances &
moments of
o random
1. Moment variables. functions
generating
The moment generating function (mgf) of a real-valued
random variable X is defined by
To see why MX is called the moment generating
function, we differentiate it with respect to s and
obtain Taking s = 0, we
have
Differentiating k times
and
then setting s = 0 yield NB: The variable s here is matter of
choice; it can be t, v, w, etc 53
Example: If X is an exponential random variable
with parameter λ > 0, find its moment generating
function
Exercise: Let X be a Bernoulli r.v.
(a) Find the moment generating function
of X.
(b) Find the mean and variance of X.
Exercise: Let X be a binomial r.v. with parameters
(n, p).
Find the moment generating function of X.
Find the mean and variance of X.
54
2. Characteristic functions
the moment generating function was finite only for Res <
λ.
It is possible to have random variables for which MX (s) is
defined only for Re s = 0; i.e.,MX (s) is only defined for
imaginary s.
For example, if X is a Cauchy random variable, then it is
easy to see that MX (s) = ∞ for all real s ≠ 0.
In order to develop transform methods that always work
for any random variable X, we introduce the
characteristic function of X, defined by
If X is a continuous random variable with density f ,
then
which is just the Fourier transform of f . Using the
Fourier inversion formula,
55
If X is an integer-valued random variable,
then
When the moment generating function is not finite in
a neighborhood of the origin, the moments of X
cannot be obtained . However, the moments can
sometimes be obtained from the characteristic
function. For example, if we differentiate the above
with respect to ν, we obtain
56