Probability Theory and

Stochastic Processes

Probability Theory and Stochastic Processes

 Unit-II
OPERATIONS ON SINGLE
VARIABLE – EXPECTATIONS

Dr Hima Bindu Valiveti

Department of ECE, GRIET

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 2
 Unit-II Syllabus & Lesson Plan
OPERATIONS ON SINGLE VARIABLE – EXPECTATIONS
Lsn_1: Introduction, Expected Value of a Random Variable,
Function of a Random Variable
Lsn_2&3: Moments about the Origin, Central Moments, Variance
and Skew
Lsn_4&5: Characteristic Function, Moment Generating Function
Lsn_6&7:Transformations of a Random Variable: Monotonic
Transformations for a Continuous Random Variable
Lsn_8: Non-monotonic Transformations of Continuous Random
Variable
Lsn_9: Transformation of a Discrete Random Variable. Vector
Random Variables.
Lsn_10: Problems and Simulation of Moments in MATLAB
Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 3
 Introduction: Functions of Random
Variables
• Expectation of a rv: mean of the random variable
• Median: Middle or most central item in an ordered set of
numbers
• Mode = Max{elements of rv}
• Variance is the spread of the distribution of the random
variable about its mean value
• Standard Deviation is measure of spread or deviation from
the mean
• Moments : Mean is the first moment; moments about mean
and moments about origin
Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 4
 Functions of Random Signals (continuous)
• Expectation of discrete rv�with pmf �� (�),
�(�) = � = �� �� (�� )
�=1
• Expectation of a continuous rv with pdf f(x)
∞
�(�) = � = ��(�)
−∞
• The expected value of a random variable is also
called the mean of the random variable

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 5
 Expectations of CRV (contd.)
• Symmetric Random
Variables µ f ( x) E( X )  
– If � has a pdf �(�) that is
symmetric about a point
so that
f (  x)  f (  x)

– Then, E ( X )  
– the expectation of the RV is
equal to the point of  x
symmetry

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 6
 Exercise Problem
Find E(X) for the random variable X with table:
� 1 3 5
�� (�) 1/6 1/6 2/3
Let X be a Bernoulli rv. Find E(X).

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 7
 Exercise Problem
Let X = number of prior convictions for prisoners at a state
prison at which there are 500 prisoners. (x=0,1,2,3,4). Find
mean of X
� 0 1 2 3 4
No. of Prisoners 80 265 100 40 15

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 8
 Exercise Problem
Find the expected value of X for the density function
of the RV ‘X’,
1
�(�) = 2 � 0 < � < 2
0 ��ℎ������

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 9
 Expectation of a RV
Suppose that X is a rv and g is a function of X, i.e., a real
function, and � = �(�). Then

1. If X is discrete, with pmf �� (�),

�
Then � � = �=� �(�� )�� (�� ) = �(�)�(�)
( )

2. if X is continuous, with pdf �� (�),

∞
Then �(�) = −∞
�(�)�� (�) ��

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 10
 Exercise Problem
If X is a rv with the density function
1
� 0 < � < 2
�(�) = 2 , then what is the expected
0 ��ℎ������
value for 3�2 − 2�

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 11
 Expectation of a RV
Theorems on Expectation of rv
∞
�(�) = � = ��(�)
−∞
Th_1: If C is any constant, then
�(��) = ��(�)
Th_2: If X and Y are rv’s, then
�(� + �) = �(�) + �(�)
Th_3: If X and Y are independent rv’s, then
�(��) = �(�). �(�)

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 12
 Functions of RV
• Variance(�2 )
– A positive quantity that measures the spread of the
distribution of the random variable about its mean value
– Larger values of the variance indicate that the distribution
is more spread out

�
���(�) = �[ � − �(�)
= � (� − �)� = � �� − (�(�))�

• Standard Deviation (Denoted by �)

– The positive square root of the variance

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 13
 Functions of RV
Theorems on Variance of rv
Th_1: �� = �[(� − �)� = �(�� ) − ��
= �(�� ) − [�(�) �
Th_2: If C is any constant, then
���(��) = �� ���(�)
Th_3: The quantity �[(� − �)� is minimum when
� = � = �[�
Th_4:If X and Y are independent rv’s, then
���(� ± �) = ���(�) + ���(�)
Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 14
 Functions of RV (Variance)
V a r ( X )  E (( X  E ( X )) 2 )
2
 E(X  2 X E ( X )  ( E ( X )) 2 )
2
 E(X )  2 E ( X ) E ( X )  ( E ( X )) 2
2
 E(X )  ( E ( X )) 2
f ( x)

Two distribution with

identical mean values
but different variances

µ or E(X) x
Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 15
 Exercise Problem
Let X be a discrete rv with pmf as in table,
find a. E(X); b. E(2X+3); c. E(X^2); d. E[(2X+1)^2]
� -2 -1 0 1 2
p(x) 1/5 2/5 1/10 1/10 1/5

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 16
 Exercise Problem
If X is a rv with the density function
1
� 0<�<2
�(�) = 2 ,
0 ��ℎ������
then evaluate the moments (to order 4), mean, variance and standard deviation

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 17
 Exercise Problem
Find the mean and variance of Uniform Distributed rv

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 18
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 19
 Exercise Problem
Find the expectation and variance of discrete rv whose
PMF is given as ( 1 2 )� ; � = 1,2,3…. .

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 20
 Exercise Problem
Find the mean, mean square, variance and standard
deviation for a RV whose pdf is given as,
−2�
( )
� � = 2� � > 0
0 �≤0

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 21
 Exercise Problem
Find the expectation of sum of points in tossing a pair
of fair dice. (Expectation of two independent variables)

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 22
 Exercise Problem
Find variance and standard deviation of sum obtained
in tossing a pair of fair dice.

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 23
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 24
 Moments
Moments in statistics are quantitative measure that describes the
specific characteristics of a probability distribution. It helps to
understand the data set's shape, spread, and central tendency.

Moments of a Random distribution is defined as the expected values of the

powers of the related functions of the RV.

The ��ℎ moment of a rv is defined as,

� ( )
�
� � � �: ��������
� (� � ) =
�� �(�) �� �: ����������

Mean is the first moment E(X) where n=1

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 25
 Moments
1. Moments about Origin or Raw Moments or
Moments about Zero
� (�)
�
� � �: ��������
�� 1 =
�� �(�) �� �: ����������

2. Central moments or Moments about Mean (for n≥2)

� (�)
�
(� − �) � �: ��������
�� =
(� − �)� �(�) �� �: ����������

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 26
 Moments
Find the mean, mean square and variance of
simple coin flipping. �� �(�)

Mean, E(X) = � � �(�) 1 p

2 2 0 1-p
Mean Square, E(� ) = �
� �(�)
Var(X), E[(� − �)2 = �
(� − �)2 ( )
� �

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 27
 Moments
The second, third and fourth central moments can be expressed
in terms of the raw moments
′
as follows:
′
(Special Cases)
�1 = � & �0 = 1
′
�2 = �2 − �2
′ ′
�3 = �3 − 3�2 � + 2�3
′ ′ ′
�4 = �4 − 4�3 � + 6�2 �2 − 3�4

�� ′
′ � ′ � ′ �
= �� − ��−1 � + ��−2 �2 ± … + (−1)� ��−� ��
1 ′ 2 �
� �
+ … + (−1) �0 �

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 28
 Standardized Random Variables
Let X be a RV with mean (µ) , variance (σ²) and
standard deviation (σ>0). We define the associated
standardized rv (x*) in the following way
∗
�−�
� =
�
which is defined to have,
Mean, �(�∗ ) = 0 and Variance, ���(�∗ ) = 1

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 29
 Skewness & Kurtosis
The third and fourth moments about the mean,
called skewness and kurtosis, are occasionally used
in risk analysis as numerical descriptions of
symmetry and peaking respectively

�−� 3
( ) �(�) �: ��������
�−� 3 �
�[ = �
� �−� 3
( ) �(�) �� �: ����������
�

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 30
 Skewness
a) If a distribution has a negative
skewness (sometimes described as
left skewed), has a longer tail to
the left than to the right.

b) A positively skewed distribution

(right skewed) has a longer tail to
the right.

c) Zero skewed distributions

are usually symmetric.
Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 31
 Kurtosis
The kurtosis statistic refers to the peakedness of the
distribution
- the higher the kurtosis, the more peaked the distribution.
- A Normal distribution has a kurtosis of 3, so kurtosis
values for a distribution are often compared to 3.
- Eg: If a distribution has a kurtosis below 3 it is flatter than
a Normal distribution.

�−� 4
( ) �(�) �: ��������
�−� 4 �
�[ = �
� �−� 4
( ) �(�) �� �: ����������
�
Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 32
 Kurtosis

Laplace, normal, and uniform distributions with mean 0 and variance 1. Respectively,
their excess kurtosises are 3, 0, and −1.2.

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 33
 Exercise Problem
Find the first four moments
a) About the origin
b) About the mean
for a RV ‘X’ having the density function
4�(9 − �2 )
�(�) = 0≤�≤3
81
0 ��ℎ������

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 34
 Moment Generating Function
Represented as �� (�) = �[���

��� �(�) �: ��������

�� (�) = �

��� �� (�) �� �: ����������

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 35
 Moment Generating Function
�� (�) = �[���

�2 �2 �3 �3 �4 �4 �� ��
�� (�) = �[1 + �� + + + + ...
2! 3! 4! �!
�� � � �� � � �� � � �� � �
Note: Taylor series expansion of � =� + �� +
��
+ + + ... + .
�! �! �! �!

�2 �3 �4
�� (�) = �[1 + ��[� + �[�2 + �[�3 + �[�4 + . . . . .
2! 3! 4!
Note: From the theorems of Expectation E[X+Y]=E[X]+E[Y]; E[cX]=cE[X]

�2 ' �3 ' �4 ' �� '

�� (�) = 1 + �� + �2 + �3 + �4 + . . + ��
2! 3! 4! �!

Probability Theory and Stochastic Processes

11/9/2023 Dr Hima Bindu Valiveti, GRIET 36
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 37
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 38
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 39
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 40
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 41
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 42
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 43
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 44
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 45
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 46
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 47
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 48
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 49
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 50
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 51
 Probability Theory and Stochastic Processes
11/9/2023 Dr Hima Bindu Valiveti, GRIET 52