RANDOM PROCESS– TEMPORAL & SPECTRAL CHARACTERISTIC

Random Process – Concept, Classification

Distribution and Density Function, Stationarity and

Statistical Independence

Time Averages and Ergodicity, Auto Correlation, Cross

Correlation - Properties

Covariance Functions, Gaussian Random Processes,

Poisson Random Process

The Power Density Spectrum and its Properties

Relationship between Power Spectrum and

Autocorrelation Function

The Cross-Power Density Spectrum and its Properties

Relationship between Cross-Power Spectrum and Cross-

Correlation Function
RCEW, Pasupula (V), Nandikotkur Road,
Near Venkayapalli, KURNOOL
 RANDOM PROCESS CONCEPT
• A random process is a function of both sample space and time variables. It is
represented as 𝑋(𝑡, 𝑠)or simply 𝑋(𝑡). The random processes are also called as
stochastic processes which deal with randomly varying time wave forms such as any
message signals and noise.
 The concept of random process can
be extend to include time and the
outcome will be random functions of
time as shown beside 𝑥(𝑡, 𝑠), Where
s is the outcome of anexperiment.
 Thefunctions
… 𝑥𝑛+2 𝑡 , 𝑥𝑛+1 (𝑡),𝑥𝑛 (𝑡) , 𝑥𝑛−1 𝑡 … ar
e one realizations of many of the
random process 𝑋(𝑡).

 A random process also represents a

RCEW, Pasupula (V), Nandikotkur Road,
random variable when time is Near
fixed Venkayapalli, KURNOOL
 CLASSIFICATION OF RANDOM PROCESS

 Continuous Random Process

 Discrete Random Process

 Continuous Random Sequence

 Discrete Random Sequence

Continuous RandomProcess:
A random process is said to be
continuous if both the random
variable 𝑋 and time 𝑡 are
continuous.
Discrete Random Process: In discrete
random process, the random variable 𝑋
has only discrete values while time, 𝑡 is
continuous. RCEW, Pasupula (V), Nandikotkur Road,
Near Venkayapalli, KURNOOL
 CLASSIFICATION OF RANDOM PROCESS
Continuous Random Sequence: A random process for
which the random variable 𝑋 is continuous but 𝑡 has
discrete values is called continuous random sequence. A
continuous random signal is defined only at discrete
(sample) time intervals. It is also called as a discrete
time random process and can be represented as a set of
random variables {𝑋(𝑡)} for samples 𝑡𝑘 , 𝑘 = 0, 1, 2, … . .

Discrete Random Sequence: In discrete random

sequence both random variable 𝑋 and time 𝑡 are
discrete. It can be obtained by sampling and
quantizing a random signal. This is called the
random process and is mostly used in digital
signal processing applications. The amplitude of
RCEW,
the sequence can be quantized Pasupula
into two levels(V),
or Nandikotkur Road,
Near Venkayapalli, KURNOOL
multi levels
 DETERMINISTIC AND NON-DETERMINISTIC PROCESSES
• Deterministic processes:A process is called as deterministic random process if future
values of any sample function can be predicted from its past values.

• For example, 𝑋(𝑡) = 𝐴 sin(𝜔0 𝑡 + ϴ), where the parameters 𝐴, 𝜔0 and ϴ may be
random variables, is deterministic random process because the future values of the
sample function can be detected from its known shape.

• Non-Determinist processes: If future values of a sample function cannot be detected

from observed past values, the process is called non-deterministic process.

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 DISTRIBUTION AND DENSITY FUNCTION
• If 𝑋(𝑡) is a stochastic process, then for fixed 𝑡, 𝑋(𝑡) represents a random variable. Its
distribution function is given 𝐹
by𝑋 (𝑥, 𝑡) = 𝑃{𝑋(𝑡) ≤ 𝑥}

• Notice that depends on 𝑡, since for a different 𝑡, we obtain a different random

𝐹𝑋 (𝑥, 𝑡)
variable. Further
𝑑𝐹𝑋 (𝑥, 𝑡)
𝑓𝑋 (𝑥, 𝑡) =
• represents the first-order probability
𝑑𝑥 density function of the process 𝑋(𝑡).
• For t = t1 and t = t2, X(t) represents two different random variables X1 = X(t1) and X2 =
X(t2) respectively. Their joint distribution is given by

• and
𝐹𝑋 (𝑥1 , 𝑥2 , 𝑡1 , 𝑡2 ) = 𝑃{𝑋(𝑡1 ) ≤ 𝑥1 , 𝑋(𝑡2 ) ≤ 𝑥2 }

𝜕 2 𝐹𝑋 (𝑥1 , 𝑥2 , 𝑡1 , 𝑡2 )
• represents
𝑓𝑋 (𝑥1the
, 𝑥2second-order
, 𝑡1 , 𝑡2 ) = density function of the process X(t).
𝜕𝑥1 𝜕𝑥2

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 STATIONARITY AND STATISTICAL INDEPENDENCE
• Similarly
𝑓𝑋 (𝑥1 , 𝑥2 , ⋯ 𝑥𝑛 , 𝑡1 , 𝑡2 ⋯ , 𝑡𝑛 ) represents the nth order density function of
the process X(t). Complete specification of the stochastic process X(t) requires the
𝑓knowledge
𝑋 𝑥1 , 𝑥2 , ⋯of
𝑥𝑛 , 𝑡1 , 𝑡2 ⋯ , 𝑡𝑛 ∀𝑡𝑖 , 𝑖 = 1,2,3 …, and for all
n.
Stationary RandomProcess: all its statistical properties do not change withtime
Non Stationary Random Process: not stationary

• Statistical Independence: Two Processes 𝑋(𝑡)and 𝑌 𝑡 are statistically independent if

the random variable group 𝑋 𝑡1 , 𝑋(𝑡2 ) ,…, 𝑋(𝑡𝑁 ) is independent of the group
𝑌(𝑡1′ ),𝑌(𝑡2′ ),…,𝑌(𝑡𝑀
′
) for any choice of times. Independence requires that the joint
density be factorable by groups:

• 𝑓𝑋,𝑌 𝑥1 , 𝑥2 , ⋯ 𝑥𝑁 , 𝑦1 , 𝑦2 , ⋯ 𝑦𝑀 ; 𝑡1 , 𝑡2 ⋯ , 𝑡𝑛 , 𝑡1′ , 𝑡2′ , … , 𝑡M

′

= 𝑓𝑋 𝑥1 , 𝑥2 , ⋯ 𝑥𝑁 , ; 𝑡1 , 𝑡2 ⋯ , 𝑡𝑛 𝑓𝑌 𝑦1 , 𝑦2 , ⋯ 𝑦𝑀 ; 𝑡1′ , 𝑡2′ , … , 𝑡M
′

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 FIRST ORDER STATIONARY, SECOND ORDER STATIONARY
 A random process is classified as first-order stationary if its first-order probability density
function remains equal regardless of any shift in time to its timeorigin.
 In other words, 𝑓𝑋 𝑥1 , 𝑡1 = 𝑓𝑋 𝑥1 , 𝑡1 + ∆ must be true for any 𝑡1 and any real number ∆,
if 𝑋 𝑡 to be a first order stationary.
 Therefore the condition for a process to be a first order stationary random process is that
its mean value must be constant at any time instant. i.e. 𝐸[𝑋(𝑡)] = 𝑋ത = 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡.
 A random process is said to be stationary to order two or second order stationary if its
second order joint density function does not change with time or shift in time value i.e.
𝑓𝑋 (𝑥1 , 𝑥2 ; 𝑡1 , 𝑡2 ) = 𝑓𝑋 (𝑥1 , 𝑥2 ; 𝑡1 + ∆, 𝑡2 + ∆) for all 𝑡1 ,𝑡2 and ∆.
 It is a function of time difference (𝑡2 , , 𝑡1 ) and not absolute time 𝑡. Note that a second
order stationary process is also a first order stationary process. The condition for a
process to be a second order stationary is that its autocorrelation should depend only on
time differences and not on absolute time. i.e. If 𝑅𝑋𝑋 (𝑡1 , 𝑡2 ) = 𝐸[𝑋(𝑡1 ) 𝑋(𝑡2 )] is auto
correlation function and 𝜏 = 𝑡Pasupula
RCEW, 2 – 𝑡1 then (V),
𝑅𝑋𝑋Nandikotkur
𝑡1 , 𝑡1 + 𝜏 = 𝐸Road,
𝑋 𝑡1 𝑋 𝑡1 + 𝜏
Near Venkayapalli,
= 𝑅𝑋𝑋 (𝜏) . 𝑅𝑋𝑋 (𝜏)should be independent of time t. KURNOOL
 WIDE-SENSE STATIONARY (WSS)
A process that satisfies thefollowing:
The mean is a constant and the autocorrelation function depends only on the difference
between the timeindices
• 𝐸[𝑋(𝑡)] = 𝑋ത = 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
• 𝐸[𝑋 𝑡 𝑋 𝑡 + 𝜏 ] = 𝑅 𝑋𝑋 (𝜏)
• is a Wide-Sense Stationary(WSS)
• Second-orderstationary Wide-SenseStationary
• The converse is not true ingeneral.
 If they are jointly WSS, then the cross correlation function of 𝑋(𝑡) and 𝑌(𝑡) is a function
of time difference 𝜏 = 𝑡2 – 𝑡1 only and not absolute time. i.e. 𝑅𝑋𝑌 (𝑡1 , 𝑡2 )
= 𝐸[𝑋(𝑡1 ) 𝑌(𝑡2 )] .
 If 𝜏 = 𝑡2 – 𝑡1 , 𝑡1 = 𝑡then 𝑅𝑋𝑌 𝑡, 𝑡 + 𝜏 = 𝐸 𝑋 𝑡 𝑌 𝑡 + 𝜏 = 𝑅𝑋𝑌 (𝜏).
Therefore the conditions for a process to be joint wide sense stationary are
𝐸[𝑋(𝑡)] = 𝑋ത RCEW, Pasupula
= 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡, ഥNandikotkur
𝐸[𝑌(𝑡)] (V),
= 𝑌 = 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡andRoad,
𝐸[𝑋(𝑡) 𝑌(𝑡 + 𝜏)] Near Venkayapalli,
= 𝑅 (𝜏) is independentKURNOOL
𝑋𝑌 of time t.
 STRICT SENSE STATIONARY (SSS) PROCESSES
• In strict terms, the statistical properties are governed by the joint probability
density function. Hence a process is nth-order Strict-Sense Stationary (S.S.S) if

𝑓𝑋 (𝑥1 , 𝑥2 , ⋯ 𝑥𝑛 , 𝑡1 , 𝑡2 ⋯ , 𝑡𝑛 ) ≡ 𝑓𝑋 (𝑥1 , 𝑥2 , ⋯ 𝑥𝑛 , 𝑡1 + 𝑐, 𝑡2 + 𝑐 ⋯ , 𝑡𝑛 + 𝑐)
• for any c, where the left side represents the joint density function of the random
variables
𝑋1 = 𝑋 𝑡1 , 𝑋2 = 𝑋 𝑡2 , … , 𝑋𝑛 = 𝑋(𝑡𝑛 ) and the right side
corresponds to the joint density function of the random variables

𝑋1′ = 𝑋(𝑡1 + 𝑐), 𝑋2′ = 𝑋(𝑡2 + 𝑐), ⋯ , 𝑋𝑛′ = 𝑋(𝑡𝑛 + 𝑐).

• A process X(t) is said to be strict-sense stationary if the above equation is true for
all
𝑡𝑖 , 𝑖 = 1, 2, ⋯ , 𝑛; 𝑛 = 1, 2, ⋯ and 𝑎𝑛𝑦 𝑐.

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 TIME AVERAGES & ERGODICITY

x X  Jointly Ergodic => Ergodic X(t) and Y(t)

 Ergodic =>
xx ( )  RXX ()  xy ( )  RXY ()

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 AUTO CORRELATION

 Autocorrelation occurs in time-series studies when the errors associated with a given
time period carry over into future time periods.

 For example, if we are predicting the growth of stock dividends, an overestimate in one
year is likely to lead to overestimates in succeeding years.

 Times series data follow a natural ordering over time.

 It is likely that such data exhibit intercorrelation, especially if the time interval between
successive observations is short, such as weeks or days.
 We expect stock market prices to move or move down for several days in succession.

 We experience autocorrelation when E(uiu j ) 0

 Tintner defines autocorrelation as ‘lag correlation of a given series within itself, lagged
by a number of times units’ whereas serial correlation is the ‘lag correlation between
two different series’.
RCEW, Pasupula (V), Nandikotkur Road,
Near Venkayapalli, KURNOOL
 AUTO CORRELATION
The autocorrelation function of a random process X(t) is thecorrelation 𝐸 𝑋1 𝑋2 of
tworandom variables 𝑋1 = 𝑋 𝑡 1 𝑎𝑛𝑑 𝑋2 = 𝑋(𝑡 2) by the process at times 𝑡1 and 𝑡 2

• 𝑅 𝑋𝑋 𝑡1 , 𝑡 2 = 𝐸 [ 𝑋(𝑡1 )𝑋(𝑡 2 )]

• Assuming a second-order stationaryprocess

• 𝑅 𝑋𝑋 𝑡, 𝑡 + 𝜏 = 𝐸 𝑋 𝑡 𝑋 𝑡, 𝑡 + 𝜏 = 𝑅 𝑋𝑋 𝜏

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 PROPERTIES OF ACF

(1)RXX ( )  RXX (0)

(2) RXX ( )  RXX ( )

(3) RXX (0)  E[ X2 (t) ]

(4) X(t) is stationary &ergodic with no periodic components

(5) If X (t) is stationary and has a periodiccomponent

 RXX ( ) has a periodic component with the sameperiod.
(6) The autocorrelation function of random process RXX ( ) cannot
have any arbitrary shape.

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 Cross Correlation - Properties

• Properties of cross-correlation function of jointly WSS random processes

• RXY ()  E[X(t)Y(t  )]

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
AUTO COVARIANCE

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
CROSS COVARIANCE

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
GAUSSIAN & POISSON RANDOM PROCESS

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 RANDOM PROCESSES – SPECTRAL CHARACTERISTICS

• Consider a random process X (t). The amplitude of the random process, when it
varies randomly with time, does not satisfy Dirichlet’s conditions.

• Therefore it is not possible to apply the Fourier transform directly on the random
process for a frequency domain analysis. Thus the autocorrelation function of a WSS
random process is used to study spectral characteristics such as power density
spectrum or power spectral density (psd).

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 INTRODUCTION

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
POWER SPECTRAL DENSITY

power densityspectrum

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
PROPERTIES OF PSD

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 PROPERTIES OF PSD

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 BANDWIDTH

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
RELATIONSHIP BETWEEN PSD AND ACF

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
RELATIONSHIP BETWEEN PSD AND ACF

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
RELATIONSHIP BETWEEN PSD AND ACF

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 EXAMPLE

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
CROSS POWER SPECTRAL DENSITY

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
CROSS POWER SPECTRAL DENSITY

Parseval's theorem
RCEW, Pasupula (V), Nandikotkur Road,
Near Venkayapalli, KURNOOL
CROSS POWER SPECTRAL DENSITY

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
CROSS POWER SPECTRAL DENSITY

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
PROPERTIES OF CROSS POWER SPECTRAL DENSITY (CPSD)

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
PROPERTIES OF CROSS POWER SPECTRAL DENSITY (CPSD)

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 EXAMPLE

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
RELATIONSHIP BETWEEN CPSD AND CCF

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
RELATIONSHIP BETWEEN CPSD AND CCF

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
 EXAMPLE

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
DIGITAL RESOURCES
Lecture Notes - Lecture Notes

Video Lectures – Click here

E-Book - Click here

Model Papers - External Question Papers

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL
THANK YOU

RCEW, Pasupula (V), Nandikotkur Road,

Near Venkayapalli, KURNOOL