Introduction to Probability Theory and Stochastic Processes

Prof. S Dharmaraja
Department of Mathematics
Indian Institute of Technology, Delhi

Lecture – 40

In this lecture we have already discussed moments of functions of several random

variables, covariance variance matrix. In the lecture 1 and the lecture 2, we have
discussed the correlation coefficient, in this lecture we are going to discuss about the
condition expectation.

(Refer Slide Time: 00:30)

This is a very important concept in probability, because the way we have discussed the
conditional distribution conditional expectations also important; that means, after
something happen, what is the distribution of the future events, we are finding the
probability, then finding the distribution of the various random variable.

The same way we can go for computing the condition expectation, umm for that first we
should know what is the conditional distribution, then we can find out the expectation of
that is going to be call it as a conditional expectation; that means, first I should define
what is a conditional distribution, then followed I can go for finding the expectation of
that conditional distribution is a conditional expectation.
The provided condition of expectation in absolute sense it has been finite the same thing
play here, also to compute the condition expectation, provided the expectation in
absolute sense that is convergent or absolute sense it has a finite value.

So, let me give the definition of conditional expectation for the two dimensional discrete
type random variable first, then I will go for the definition of condition expectation of
two dimensional continuous type random variables so, definition.

First let me give the definition of conditional probability mass function, let X and Y be
two discrete type random variables, with joint probability mass function is P X comma Y,
then we have defined the conditional probability mass function of X given the other
random variable takes a value Y is given by probability X, given the other random
variable Y has a function of x given small y that is nothing, but the probability that X
takes a value x, given Y takes a value small y.

This we have already defined provided, provided the probability of Y takes a value small
y is greater than 0, this is the conditional probability mass function of the random
variable X given Y takes a value small y.

(Refer Slide Time: 04:07)

Now, I can go for defining the conditional expectation, conditional expectation of the
random variable X given the other random variable takes a value Y, that is defined as
expectation of X given Y takes a value small y. Whenever we use a word slash; that
means, it is a conditional or given that is same as that is same as summation x times the
conditional probability mass function of x given y.

So, the definition says the conditional expectation of X given Y takes a value small y
means a summation over X, X times the conditional probability mass function of X given
Y provided this right hand side summation in absolute sense, it is finite value. As long as
the right hand side in the absolute sense, it is a finite value without absolute sense that
summation quantity, is going to be call it as a conditional expectation of X given Y takes
a value small y.

There is another name for this the quantity that is expectation of E X given Y takes a
value small y, that is called the regression of the random variable X given Y, takes a
value small y we will study this in detailed the regression of X given Y takes a value
small y in the statistic course, but here we are connecting the conditional expectation of
X given Y that is nothing, but the regression of X given Y takes a value small y.

One more observation the conditional expectation X given Y is equal to small y is a

function of Y is a function of Y; that means, the expectation of X given capital Y is the
value taken for the values of the different Y therefore, expectation X given small y is the
value of the a random variable expectation E X given capital Y. Since this is going to be
a function of small y and capital Y is a random variables.

Therefore expectation of X given small y is the function of small y that is nothing, but
the value of the random variable expectation of X given capital Y. The quantity
expectation of X given capital Y takes value small y is called the regression of X on
capital Y takes a value small y, we will study this regression of X on capital Y takes a
value small y in detail in the statistic courses, but as far as this probability and stochastic
process course is concerned, you can consider this as the this quantity as a regression of
X on capital Y takes the value small y.

And the other observation is the conditional expectation of X given the capital Y takes a
value small y is the function of y. And since capital Y is a random variable it is taking a
different values of the small y therefore, expectation of X given small y is the value of
the random variable expectation X given capital Y. Note that expectation of a random
variable is a constant whereas, the conditional expectation of one random variable given
another random variable take some value, that is value of the random variable
expectation of X given the random variable. So, there is a different between expectation
and the conditional expectation.

We will study the some results over the conditional expectation, after I go for continuous
type random variable definition and some more examples.

(Refer Slide Time: 09:16)

We will go for example, for conditional expectation of two dimensional discrete type
random variables. Let X be Poisson distributed random variable with the parameter
lambda and let Y be a again Poisson distributed random variable with the parameter mu,
assume that X and Y are independent random variables, we have already proved the sum
of 2 independent Poisson distributed random variable, also going to be Poisson
distribution with the parameter is sum.

Also we have already proved the probability of X given X plus Y take some value that is
nothing, but the conditional distribution of X takes the value small x and X plus Y takes a
value some n. This we have already proved that follows a n c x lambda divided by
lambda plus mu power x, 1 minus lambda divided by lambda plus mu the whole power n
minus x, where x can take the value 0 1 and so, on till n this is a conditional distribution
of X given X plus Y.

Since it is a, the probability mass function the condition probability mass function of X
given X plus Y, that follows a binomial distribution.
(Refer Slide Time: 11:24)

We can easily write X given X plus Y follows binomial distribution with the parameters
n comma p p is lambda divided by lambda plus mu

Our interest is to find out the conditional expectation. So, the conditional expectation of
X given X plus Y takes a value n. So, here also I can write is equal to n that is same as.
Since it is a binomial distribution you know that the expectation of binomial distributed
random variable is product of the parameters. So, n into P since we know the mean of
binomial distribution exist therefore, we are directly writing the condition expectation of
X given X plus Y is equal to n that is n times lambda divided by lambda plus mu.

Like that for any two dimensional or n dimensional discrete type random variable one
can first find the conditional probability mass function, from there you can find out the
conditional expectation. Now, we will go for the conditional expectation for two
dimensional continuous type random variables.
(Refer Slide Time: 13:00)

Let X and Y be two continuous type random variables with joint probability density
function f X comma Y, then one can define the conditional probability density function
of X given Y takes a value small y is f, X slash Y as a function of x given y that is
nothing, but the joint probability density function divided by the marginal distribution of
Y at the point small y provided, provided f Y at the point y has to be strictly greater than
0. So, this is the condition probability density function of X given Y takes a value small
y.

(Refer Slide Time: 14:34)

From here one can define the conditional expectation of the random variable X given Y
takes a value small y, that is defined as expectation of X given other random variable
takes a value small y that is nothing, but minus infinity to infinity X times the conditional
probability density function of X given Y with respect to X provided the right hand side
integration in absolute sense is the finite quantity.

So, this is the conditional expectation of X given Y takes a value small y, when both the
random variables are of the continuous type we will go for the simple example.

(Refer Slide Time: 15:55)

The example is let X and Y be two continuous type random variables with the joint
probability density function that is given by 2, when y lies between 0 to x, x lies between
y to 1 otherwise 0, this example we have already discussed when I discussed the
conditional distribution.

For this problem we have already got the conditional, we have got the conditional
probability density function, that is 1 divided by 1 minus y, when x takes the value when
x takes a value y to 1 otherwise it is 0. So, this is the conditional probability density
function of x given y, here y as we did treated as a constant. So, this is a conditional
distribution of x given y.
(Refer Slide Time: 17:49)

One can find the conditional expectation of X given Y takes a value small y that is
nothing, but by definition minus infinity to infinity x times the conditional probability
density function of x given y, this is same as we know the conditional probability density
function of x given y takes a value 1 divided by 1 minus y between the interval y to 1
therefore, it is going to be y to 1, x 1 divided by 1 minus y integration with respect to x,
if you do the simplification answer is 1 plus y by 2.

So, the conditional probability density function is 1 divided by 1 minus y between the
interval y to 1; that means, it is a uniform distribution between the interval y to 1
therefore, the mean of mean of uniform distribution between the interval y to 1, that is 1
plus y divided by 2, that is same as the conditional expectation of X given Y takes a
value small y.

So, that is 1 plus y by 2 that is going to either you do by integration simplifying you will
get this answer, or by observing the conditional distribution is nothing, but the uniform
distribution.
(Refer Slide Time: 19:37)

Therefore the conditional expectation is same as the expectation of uniform distribution

during the interval y to 1. Similarly one can get the conditional probability density
function of Y given X, in the same problem you can get that is 1 divided by x between y
lies between 0 to x otherwise it is 0. So, this is a conditional probability density function
of Y given X.

So, from here you can get the conditional expectation of Y given X takes a value small x,
that is same as minus infinity to infinity y times, the condition probability density
function of y given x, by doing the simplification you will get 0 to x by substitution you
will get 0 to x y times 1 divided by x dy.

Again if you do the simplification you will get x by 2, either by simplifying this
integration you can get x by 2, or here also you can observe the again the conditional
distribution of y given x, that is uniform distribution between the interval 0 to x
therefore, conditional expectation of y given x that is same as expectation of uniform
distribution between the interval 0 to x that is x by 2. So, till now we have discussed the
conditional expectation for the two dimensional discrete type random variable with one
example, two dimensional continuous type random variables with one example.