26-Dec-1

Random Processes
Addis Ababa Science & Technology University
Outline
Department of Electrical & Electronics Engineering
Introduction

Definition of a Random Process

Characterization of Random Processes
Probability and Random Process (EEEg-2114)

Mean, Correlation, and Covariance Functions

Classification of Random Processes

Power Spectral Densities of Random Processes
Chapter 4: Random Processes
Response of Linear Systems to Random Inputs

Semester-II, 2013/14 By Welelaw Y. 2

Introduction Definition of a Random Process

The theory of random processes was first developed in
A random process is a family of random variables {X(t), tϵT}
connection with the study of fluctuations and noise in physical defined on a given probability space, indexed by the parameter
systems. t, where t varies over an index set T.

A random process is the mathematical model of an empirical
In a random process {X(t), tϵT}, the index set T is called the
process whose development is governed by probability laws. parameter set of the random process.

Random processes provides useful models for the studies of
The values assumed by X(t) are called states, and the set of all
such diverse fields as statistical physics, communication and possible values forms the state space E of the random process.
control, time series analysis, population growth, and
If the index set T of a random process is discrete, then the
management sciences. process is called a discrete-time random process.
Semester-II, 2013/14 By Welelaw Y. 3 Semester-II, 2013/14 By Welelaw Y. 4
 26-Dec-1

Definition of a Random Process Cont’d….. Definition of a Random Process Cont’d…..

A discrete-time random process is also called a random
On the other hand, for a fixed sample point ωi ϵΩ, X(t, ωi) =
sequence and is denoted by {Xn , n = 1, 2, 3, . . .). Xi(t) is a single function of time t, called a sample function or a

If T is continuous, then we have a continuous-time random realization of the process.

process.
The totality of all sample functions is called an ensemble.

In fact, a random process {X(t), tϵT} is a function of two
Of course if both ω and t are fixed, X(tk , ωi) is simply a real
arguments {X(t, ω), tϵT, ωϵΩ}. number.

For a fixed time t=tk, X(tk, ω) = Xk(ω) is a random variable
In the following discussion, we use the notation X(t) to
denoted by X(tk), as ω varies over the sample space Ω. represent X(t, ω).

Semester-II, 2013/14 By Welelaw Y. 5 Semester-II, 2013/14 By Welelaw Y. 6

Characterization of Random Processes Characterization of Random Processes Cont’d…..

If X(t) is a random process, then for fixed t=t1, X1=X(t1)
For t = t1 and t = t2, X(t) represents two different random
represents a random variable. variables X1 = X(t1) and X2 = X(t2) respectively.

Its distribution function is given by:
Their joint distribution is given by:
FX ( x1 , t1 ) = P{ X (t ) ≤ x1 } FX ( x1 , x2 , t1 , t2 ) = P{ X (t1 ) ≤ x1 , X (t2 ) ≤ x2 }

Notice that FX(x, t) depends on t, since for a different t, we
The second-order probability density function of the random
obtain a different random variable. process X(t) is:

The first-order probability density function of the process X(t) ∂ 2 FX ( x1 , x2 , t1 , t2 )
f X ( x1 , x2 , t1 , t2 ) =
is defined as: ∂x1 ∂x2
dFX ( x1 , t1 )
Similarly f X ( x1 , x2 ,L xn , t1 , t2 L , tn ) represents the nth order
f X ( x1 , t1 ) =
dx1
density function of the process X(t).

Semester-II, 2013/14 By Welelaw Y. 7 Semester-II, 2013/14 By Welelaw Y. 8

 26-Dec-1

Mean, Correlation, and Covariance Functions Mean, Correlation, and Covariance Functions …...

As in the case of random variables, random processes are often
A measure of dependence among the random variables of X(t)
described by using statistical averages. is provided by its autocorrelation function, defined by:

The mean of X(t) is defined by: R XX (t1 , t 2 ) = E [X (t1 ) X ( 2 )]

µ X (t ) = E [ X (t ) ]
Note that:

where X(t) is treated as a random variable for a fixed value of t. [

R XX (t1 , t 2 ) = R XX (t 2 , t1 ) and R XX (t , t ) = E X 2 (t ) ]

In general, µX(t) is a function of time, and it is often called the
The autocovariance function of X(t) is defined by:
ensemble average of X(t). C XX (t1 , t 2 ) = Cov[X (t1 ) , X (t 2 )] = E{[X (t1 ) − µ X (t1 )][ X (t 2 ) − µ X (t 2 )]}

= R XX (t1 , t 2 ) − µ X (t1 ) µ X (t 2 )

Semester-II, 2013/14 By Welelaw Y. 9 Semester-II, 2013/14 By Welelaw Y. 10

Mean, Correlation, and Covariance Functions …... Classification of Random Processes

It is clear that if the mean of X(t) is zero, then: i. Stationary Processes

C XX (t1 , t 2 ) = R XX (t1 , t 2 )
A random process {X(t), tϵT} is said to be stationary or strict-

sense stationary (SSS) if, for all n and for every set of time

Note that the variance of X(t) is given by:
instants (ti ϵT, i = 1,2, . . . , n),
{
σ X 2 (t ) = Var [X (t )] = E [X (t ) − µ X (t )]2 } FX ( x1 ,........, x n , t1 , ....., t n ) = FX ( x1 , ........, x n , t1 + τ , ......, t n + τ )

If X(t) is a complex random process, then its autocorrelation
Hence, the distribution of a stationary process will be
function RXX(t1, t2) and autocovariance function CXX(t1, t2) are unaffected by a shift in the time origin, and X(t) and X(t+τ)
will have the same distributions for any τ.
defined, respectively, by:
[ ]
R XX (t1 , t 2 ) = E X (t1 ) X * (t 2 ) and

Nonstationary processes are characterized by distributions
depending on the points t1, t2, . . . , tn.
{
C XX (t1 , t 2 ) = E [X (t1 ) − µ X (t1 )][X (t 2 ) − µ X (t 2 )]
*
}
Semester-II, 2013/14 By Welelaw Y. 11 Semester-II, 2013/14 By Welelaw Y. 12
 26-Dec-1

Classification of Random Processes Cont’d…… Classification of Random Processes Cont’d……

ii. Wide-Sense Stationary Processes
Other types of random processes include:

A random process X(t) is wide-sense stationary (WSS) if:  Independent Processes
 Markov Processes
1 . E [ X (t ) ] = µ X (constant )
 Normal Processes
2. R XX (t1 , t 2 ) = E [X (t1 ) X (t 2 ) ] = R XX ( t 2 − t1 )
 Ergodic Processes

Note that a strict-sense stationary process is also a WSS  Poisson Processes
process, but, in general, the converse is not true.

Semester-II, 2013/14 By Welelaw Y. 13 Semester-II, 2013/14 By Welelaw Y. 14

Power Spectral Densities of Random Processes Power Spectral Densities of Random Processes……

The autocorrelation function of a continuous-time random
In case of a discrete-time random process X(n), the
process X(t) is defined as: autocorrelation function of X(n) is defined by:

R XX (τ ) = E [X (t ) X (t + τ )] R XX ( k ) = E [X (n) X ( n + k ) ]

Properties of RXX(τ):
Properties of RX(k):

1. R XX (−τ ) = R XX (τ ) 1. R XX ( −k ) = R XX (k )

2. R XX (τ ) ≤ R XX (0) 2. R XX ( k ) ≤ R XX (0)

[
3. R XX (0) = E X 2 (t ) ≥ 0 ] [
3. R XX (0) = E X 2 (n) ≥ 0 ]
Semester-II, 2013/14 By Welelaw Y. 15 Semester-II, 2013/14 By Welelaw Y. 16
 26-Dec-1

Power Spectral Densities of Random Processes…… Power Spectral Densities of Random Processes……

Two processes X(t) and Y(t) are called (mutually) orthogonal if:
The power spectral density (or power spectrum) SXX(ω) of a
continuous-time random process X(t) is defined as the Fourier
R XY (τ ) = 0 , for all τ transform of RXX(τ), i.e. ,
∞
S XX (ω ) = ∫ R XX (τ )e − jωτ dτ

Similarly, the cross-correlation function of two discrete-time
−∞
jointly WSS random processes X(n) and Y(n) is defined by:

Thus, taking the inverse Fourier transform of SX(ω), we obtain:
R XY (k ) = E [X (n)Y (n + k ) ] 1 ∞
R XX (τ ) = ∫ S XX (ω )e jωτ dω

The various properties of RXY(k) similar to those of RXY(τ) can 2π −∞

be obtained by replacing τ by k in the above equations.
The above equations are known as the Wiener-Khinchin
relations.
Semester-II, 2013/14 By Welelaw Y. 17 Semester-II, 2013/14 By Welelaw Y. 18

Power Spectral Densities of Random Processes…… Power Spectral Densities of Random Processes……

Properties of SXX(ω):
Thus, taking the inverse Fourier transform of SXX(Ω), we obtain:

1 π
1. S XX (ω ) is real and S XX (ω ) ≥ 0 R XX (k ) = ∫πS (Ω)e jΩk dΩ
2π
XX
−

2. S XX (−ω ) = S XX (ω )

Properties of SXX(Ω):

[ ]
3. E X 2 (t ) = R XX (0) =
1
2π ∫
∞

−∞
S XX (ω )dω 1. S XX (Ω + 2π ) = S XX (Ω)

2. S XX (Ω) is real and S XX (Ω) ≥ 0

Similarly, the power spectral density SXX(Ω) of a discrete-time
3. S XX (−Ω) = S XX (Ω)
random process X(n) is defined as the Fourier transform of
RXX(k): ∞ [ ]
3. E X 2 (n) = R XX (0) =
1
2π
π
∫πS
−
XX (Ω)dΩ
S XX (Ω ) = ∑R XX ( k )e − jΩk

k = −∞
Semester-II, 2013/14 By Welelaw Y. 19 Semester-II, 2013/14 By Welelaw Y. 20
 26-Dec-1

Power Spectral Densities of Random Processes…… Power Spectral Densities of Random Processes……

The cross power spectral density (or cross power spectrum)
Properties of SXY(ω):
SXY(ω) of two continuous-time random processes X(t) and Y(t)
Unlike SXX(ω), which is a real-valued function of ω, SXY(ω), in
is defined as the Fourier transform of RXY(τ): general, is a complex-valued function.
∞
S XY (ω ) = ∫ R XY (τ )e − jωτ dτ 1. S XY (ω ) = S YX (−ω )
−∞

2. S XY (−ω ) = S XY (ω )
*

Thus, taking the inverse Fourier transform of SXY(ω), we get:
1 ∞
R XY (τ ) = ∫ S XY (ω )e jωτ dω
Similarly, the cross power spectral density SXY(Ω) of two
2π −∞
discrete-time random processes X(n) and Y(n) is defined:
∞
S XY (Ω) = ∑R
k = −∞
XY (k )e − jΩk

Semester-II, 2013/14 By Welelaw Y. 21 Semester-II, 2013/14 By Welelaw Y. 22

Power Spectral Densities of Random Processes…… Example on Random Processes

Example:

Taking the inverse Fourier transform of SXY(Ω), we get: Consider a random process X(t) defined by
1 π X (t ) = A cos(ω 0 t + φ )
R XY (k ) =
2π ∫πS −
XY (Ω)e jΩk dΩ
where ω 0 and A are constants and φ is a uniform

Properties of SXY(ω): random variable over the interval (0, 2π )

Unlike SXX(Ω), which is a real-valued function of Ω, SXY(Ω), in
a. Find the mean µ X (t ).
general, is a complex-valued function.
b. Find the autocorrelation function R XX (t1 , t 2 ).
1. S XY (Ω + 2π ) = S XY (Ω)
c. Find the autocovariance function C XX (t1 , t 2 ).
2. S XY (Ω) = S YX (−Ω)
d . Determine whether X (t ) is WSS random process or not.
3. S XY (−Ω) = S XY (Ω)
*
e. Find the power spectral density of X (t ).
Semester-II, 2013/14 By Welelaw Y. 23 Semester-II, 2013/14 By Welelaw Y. 24
 26-Dec-1

Example on Random Processes Cont’d…… Example on Random Processes Cont’d……

Solution: Solution:
a. µ X (t ) = E [X (t ) ] = E [A cos( ω 0 t + φ ) ] = AE [cos( ω 0 t + φ ) ] b. R XX (t 1 , t 2 ) = E [X (t1 ) X (t 2 ) ]
But , cos( ω 0 t + φ ) = cos( ω 0 t)cos φ - sin( ω 0 t)sin φ = E [A cos( ω 0 t 1 + φ ) A cos( ω 0 t 2 + φ ) ]
⇒ µ X (t ) = E [X (t ) ] = AE [cos( ω 0 t)cos φ - sin( ω 0 t)sin φ ] = A 2 E [cos( ω 0 t 1 + φ ) cos( ω 0 t 2 + φ ) ]
= A cos( ω 0 t) E [cos φ ] − A sin( ω 0 t) E [sin φ ] A2
= E [cos ω 0 (t 2 − t 1 ) + cos( ω 0 (t1 + t 2 ) + 2φ ) ]
1 2π 2
E [cos φ ] = ∫ cos φd φ = 0
2π 0
But , E [cos ω 0 (t 2 − t 1 ) ] = cos ω 0 (t 2 − t 1 ) and

Similarly , E [sin φ ] =
1 2π
E [cos( ω 0 (t1 + t 2 ) + 2φ ) ] = 0
2π ∫0
sin φd φ = 0
A2
∴ µ X (t ) = E [X (t ) ] = 0 ∴ R XX (t 1 , t 2 ) = cos ω 0 (t 2 − t 1 )
2
Semester-II, 2013/14 By Welelaw Y. 25 Semester-II, 2013/14 By Welelaw Y. 26

Example on Random Processes Cont’d…… Example on Random Processes Cont’d……

Solution: Solution:
e. Since X (t ) is a WSS random process, the autocorrel ation
c. C XX (t1 , t 2 ) = R XX (t1 , t 2 ) − µ X (t1 ) µ X (t 2 )
function can be simply wri tten as :
A2
= cos ω 0 (t 2 − t1 ) − 0 A2
2 R XX (τ ) = cos(ω 0τ )
2
A2 The power spectral density of X (t ) is given by :
∴ C XX (t1 , t 2 ) = cos ω 0 (t 2 − t1 )
2 ∞
S XX (ω ) = ∫−∞
R XX (τ )e − jωτ dτ
d . Since the mean is constant and the autocorrelation function
But from Fourier tr ansform pair table , we have :
depends on time difference only, X (t ) is a WSS random process.
FT {cos(ω 0 t )} = πδ (ω − ω 0 ) + πδ (ω + ω 0 )

A 2π A 2π
∴ S XX (τ ) = δ (ω − ω 0 ) + δ (ω + ω 0 )
2 2
Semester-II, 2013/14 By Welelaw Y. 27 Semester-II, 2013/14 By Welelaw Y. 28
 26-Dec-1

Response of Linear Systems to Random Inputs Response of Linear Systems to Random Inputs…..

If a WSS random process X(t) with autocorrelation function
Using properties of Fourier transform, we get:
RXX(τ) is applied to a linear system with impulse response h(t), f (t ) ←→
FT
F (ω ) and g (t ) ←→
FT
G (ω )
then the cross correlation function RXY(τ) and the output
⇒ f (t ) * g (t ) ←→
FT
F (ω )G (ω )
autocorrelation function RYY(τ) are given as follows.

Then using the above property, the cross and output power
X(t) h(t) Y(t) spectral densities can be evaluated as:

{ }
S XY (ω ) = FT R XX (τ ) * h * (−τ ) = S XX (ω ) H * (ω )
R XY (τ ) = R XX (τ ) * h * ( −τ )
And ,
And ,
S YY (ω ) = FT {RYY (τ )} = FT {R XY (τ ) * h(τ )}
RYY (τ ) = R XY (τ ) * h(τ ) 2
= S XY (ω ) H (ω ) = S XX (ω ) H (ω )
= R XX (τ ) * h (−τ ) * h(τ )
*

Semester-II, 2013/14 By Welelaw Y. 29 Semester-II, 2013/14 By Welelaw Y. 30

Example on Response of Linear Systems Example on Response of Linear Systems Cont’d……

Example: Solution:
Consider a WSS random process X(t) with autocorrelation
The frequency response H (ω ) of the system is :
function given by:
1
−a τ
H (ω ) = FT {h (t )} =
R XX (τ ) = e , where a is a real positive constant jω + b
The power spectral density of X (t ) is :
Let the random process X(t) be applied to the input of an LTI
system with impulse response given by: 2a
S XX (ω ) = FT {R XX (τ )} =
− bt ω 2 + a2
h (t ) = e u (t ) , where b is a real positive constant
Then, the power spectral density of Y (t ) is given by :
Find the autocorrelation function of the output Y(t) of the
2  1  2 a 
system. S YY (ω ) = S XX (ω ) H (ω ) = 2 2  2 
 ω + b  ω + a 
2

Semester-II, 2013/14 By Welelaw Y. 31 Semester-II, 2013/14 By Welelaw Y. 32

 26-Dec-1

Example on Response of Linear Systems Cont’d…… Exercise on Random Processes

1. Consider a random process X(t) defined by
Solution:
a  2b  b  2a  X (t ) = A cos( ω 0 t + φ )
⇒ S YY (ω ) =  −  
( ) ( )
a − b2 b  ω 2 + b2  a2 − b2 b  ω 2 + b2 
2
where ω 0 and φ are constants and A is a uniform
Taking the inverse Fourier tr ansform of both sides of the above random variable over the interval ( 0, 2)
a. Find the mean µ X (t ).
equation, we obtain :
b. Find the autocorrel ation function R XX (t1 , t 2 ).
RYY (τ ) = 2
1
(
a − b2 b
ae
−b τ

)
(
− be
−a τ
) c. Find the autocovari ance function C XX (t1 , t 2 ).
d . Determine whether X (t ) is WSS random process or not.
e. Find the power spectral density of X (t ).

Semester-II, 2013/14 By Welelaw Y. 33 Semester-II, 2013/14 By Welelaw Y. 34

Exercise on Random Processes Exercise on Random Processes Cont’d……

2. Consider a random process X(t) defined by:
3. Two random processes X(t) and Y(t) are given by:
X (t ) = A sin(ω0 t + φ )
where A and φ are independent random variables which are X (t ) = A cos(ω0t + φ ) and Y (t ) = A sin(ω 0t + φ )
 π π where A and ω0 are constants and φ is a uniform random
uniformly distributed over the intervals[0, 1] and − , 
 2 2 variable over the interval (0, 2π ).
respectively and ω0 is a constant. a. Find the cross correlation function of X (t ) and Y (t ).
a. Find the mean µ X (t ).
b. Verify that R XY (-τ ) = R XY (τ )
b. Find the autocorrelation function R XX (t1 , t 2 ).
c. Find the autocovariance function C XX (t1 , t 2 ).
d . Determine whether X (t ) is WSS random process or not.
e. Find the power spectral density of X (t ).

Semester-II, 2013/14 By Welelaw Y. 35 Semester-II, 2013/14 By Welelaw Y. 36

 26-Dec-1

Exercise on Random Processes Cont’d……

4. Consider a discrete-time WSS random process X(n) with

autocorrelation function given by:

R XX (k ) = 2e −|k |

Find the power spectral density of X(n).

Semester-II, 2013/14 By Welelaw Y. 37