Probability Theory & Random Processes

Unit V
Random Processes
 Introduction
• In electrical systems, voltage or current waveforms are used as signals for
collecting, transmitting or processing information, as well as for controlling and
providing power to a variety of devices.
• These signals (voltage or current waveforms) are functions of time and are of
two classes—deterministic and random.
• Deterministic signals can be described by the usual mathematical functions with
time ‘t’ as the independent variable. But a random signal always has some
element of uncertainty associated with it and hence it is not possible to
determine its value exactly at any given point of time.
• However, we may be able to describe the random signal in terms of its average
properties such as the average power in the random signal, its spectral
distribution and the probability tat the signal amplitude exceeds a given value.
The probabilistic model used for characterizing a random signal is called a
random process or stochastic process.

2
4
Remember:
RV maps Events Constants
RP maps Events f(t)

5
Classification of Random Processes

6
Classification of Random Processes
Average values of Random Processes
Average values of Random Processes
In real-life applications, we are often interested in multiple observations of
random values over a period of time. For example, suppose that you are
observing the stock price of a company over the next few months. In
particular, let S(t) be the stock price at time t∈[0,∞). Here, we assume t = 0
refers to current time. Figure shows a possible outcome of this random
experiment from time t = 0 to time t = 1.

Figure - A possible realization of

values of a stock observed as a
function of time.
Here, S(t) is an example of a
random process.
Figure - Possible realization of the random process {Xn,n=1,2,3,⋯}
where Xn shows the temperature in New York City at noon on day n.
RECALL the definition of Random process:
Consider an experiment E with outcomes ω and sample space Ω.
To every outcome ω, assign a real-valued function X(t, ω) of time t ∈ T
Then Random Process is ensemble (collection) of time functions :

If ω is continuous, the random process can be denoted by

15
 Probability distribution of a random process
Example: A random process is identified by four sample functions

(a) Find marginal distribution of X(0) and X(2)

(b) Find the joint PMF of X(0) and X(2)
Solution: It is given that

(a) Marginal distributions of X(0) and X(2)

X(t) = X(t, ω) depends on two random variables. We can find marginal
distributions by fixing one variable.

16
 Here we have fixed t to obtain

Marginal distributions of X(0) are same as pmf of X(0) ie

X(0) = -1 -2 3 0
P(X =x) 0.25 0.25 0.25 0.25
Similarly
X(2) = -1 -2 3 2
P(X =x) 0.25 0.25 0.25 0.25
(b) The joint PMF of X(0) and X(2)

X(2) -1 -2 33 22
-1 -2
X(0)
-1 ✔ 0× ×0 ×0
¼
-2 0× ✔ ×0 ×0
¼
3 0× ✔ ×0
0× ¼
0 0× 0× ×0 ✔
¼

17
 Characterizing X(t)

• The following functions give crude but useful information about a random
process:

• Mean Function

• Auto correlation Function

• Auto-covariance Function

18
Definition: Mean Function of X(t)

The mean of a random process X(t) at any time t is the expected value of
X(t) at that time.

X(t)
X1(t)

X2(t)

t
t=a

19
 Example: A random process is identified by four sample functions

(a) Find marginal distribution of X(0) and X(2)

(b) Find the joint PMF of X(0) and X(2)
(c) Find mean of X(0) and X(2)
Solution: We have obtained

(a) Marginal distributions of X(0) and X(2)

X(0) = -1 -2 3 0
P(X =x) 0.25 0.25 0.25 0.25

X(2) = -1 -2 3 2
P(X =x) 0.25 0.25 0.25 0.25 20
 Definition: Auto Correlation Function (ACF)

Consider a random process X(t)

Generate two random variables X₁ = X(t₁) and X₂ = X(t₂)
Then auto correlation at time t is defined as

Auto Correlation Function measures the extent to which the random

process varies with time

21
 Example : Consider that a coin is tossed 3 times. We define two random variables:
X = 1 if Head appears on first toss, X = 0 if tail appears on first toss
Y = Number of heads in three tosses.
Then we have TTT TTH THT THH HTT HHT HTH HHH

RV 1 X 0 0 0 0 1 1 1 1
RV 2 Y 0 1 1 2 1 2 2 3
Sample space of random variable (X, Y) is
(X,Y) = { (0, 0), (0,1), (0,2), (03), (1,0), (1,1), (1,2), (1,3)}
Probability distribution and marginal probability distributions are
Marginal probability of X
Y 0 1 2 3 P(X=x)
X X P(X=x)
0 1/8 2/8 = ¼ 1/8 0 ½ 0 ½
1 0 1/8 2/8 = ¼ 1/8 ½ 1 ½
P(Y=y) 1/8 3/8 Marginal1/8
3/8 probability
1 of Y

Y 0 1 2 3
P(Y=y) 1/8 3/8 3/8 1/8
22
 Example: A random process is identified by four sample functions

(a) Find marginal distribution of X(0) and X(2)

(b) Find the joint PMF of X(0) and X(2)
(c) Find mean of X(0) and X(2)
(d) Auto-correlation of X(0) and X(2)
Solution: We have obtained

(a) Marginal distributions of X(0) and X(2)

X(0) = -1 -2 3 0
P(X =x) 0.25 0.25 0.25 0.25
X(2) = -1 -2 3 2
P(X =x) 0.25 0.25 0.25 0.25

23

23
(b) Joint PMF of X(0) and X(2)

X(2) -1 -2 3 2
X(0)
-1 ¼ 0 0 0
-2 0 ¼ 0 0
3 0 0 ¼ 0
0 0 0 0 ¼

= 3.5

24
 Definition: Auto-covariance of Random Process

Auto-covariance of random process is defined as

25
Example: A random process is identified by four sample functions

Find auto-covariance of X(0) and X(2)

Solution:
Auto-covariance of X(t₁) =X ₁ and X(t₂) = X₂ It is given by

26
X(0) = -1 -2 3 0
P(X =x) 0.25 0.25 0.25 0.25

X(2) = -1 -2 3 2
P(X =x) 0.25 0.25 0.25 0.25

X(2) -1 -2 3 2
X(0)
-1 ¼ 0 0 0
-2 0 ¼ 0 0
3 0 0 ¼ 0
0 0 0 0 ¼

27