Statistics for Data Science – II

Professor. Andrew Thangaraj

Department of Electrical Engineering
Indian Institute of Technology, Madras
Lecture No. 1.7
Conditioning with Multiple Discrete Random Variables

(Refer Slide Time: 00:13)

Hello and welcome to this lecture. We are going to discuss conditioning with multiple discrete
random variables. So, like I said marginalization is one key idea to sort of reduce the scale of the
problem look at a smaller problem, from a bigger problem; that is always very nice. But
conditioning is what connects the marginals to give you some picture of the whole in some
sense; so, that is important also. We have seen before how the joint PMF of two random
variables is the product of the marginal times the conditional. So, so you can see that the
conditional is sort of like a bridge between marginal and joint; so that is an important thing,
maybe principle to keep in mind.

So, when you have multiple random variables, let us say here the example. Here I am starting
with an example of 4 X1, X2, X3, X4; you can do all sorts of conditioning. So, I mean previously
we saw when you had 2 random variables x and y, you could do x given y, y given x; that is it.
So, now with multiple random variables you can do variety of conditioning; let me give you the
simplest example.
So, the simplest example we have seen, we have seen before. You take 2 of them X1 and X2, and
look at this random variable; the conditional random variable that I have been talking about.
( )
( | ) | ( ) ; so, that is how we write it. And we know that this is
( )

just the joint divided by the marginal. Conditional equals joint by marginal; so that is a very nice
formula. Remember joint equals conditional times marginal that is also a nice formula to
remember.

But, conditional equals joint by marginal, it is a very nice simple thing to remember; and notice
how the joint came. You took you took X1 X2 and simply joined it, put it on the numerator; and
the conditioning alone you kept in the denominator. So, this simple idea one can extend, not just
with 2 random variables even if you have more than 2. So, let us say I want to define a pair of
conditional random variables; in some sense X1 comma X2 given X3 equals t3.

So, remember see all these things are random variables on the same space, and I have events
defined using X3; that is what I am doing here. And once I define an event using X3, I can of
course condition the random variables X1, X2 conditioned on that event. I am looking at both of
them conditioned on that event; so, it is all valid and clear and very easy things to think of in
( )
some sense. So, I can think of the (X1, X2 | X3 = t3)= | ( ) . It is
( )

maybe a bit confusing, but you can see that the idea is very similar to the conditioning of X1 with
X2.

And the formula is also very very similar; it is the same principle joint by marginal joint by
marginal. So, what is the joint? I have X1, X2 given X3 equals t3 of t1, t2; I simply join everything,
put in the numerator ( ). You divide by whatever you are conditioning on; so
( ), simple enough. It is just simple extension and you can you can show that these are all
valid, they satisfy all the conditions of the PMF; thereby their values between 0 and 1 and they
add up to 1. All of these can be quite very reasonably shown; it is not a big problem to do that.

Let us look at the next one; this one is a little bit more interesting. So, I am going to condition on
a slightly more complex event what is the event? The event is that X2 = t2 and X3 = t3that is the
event. I am taking that event and I am conditioning the random variable X1 given that. So, X1
given this event X2 = t2 comma X3 = t3; how do I write the distribution for this? The notation is
again a little bit cumbersome. But, it is easy to write down | ( ),; that is simply
once again the joint by marginal nothing more to worry about. What is the joint? Combine
everything together ( ). And what is the marginal? What comes in the
denominator, whatever you are conditioning on, so ( ).

So, it is very easy to write down these conditional expressions. You can also do some more
flavor, more interesting variety here. So, here is the joint distribution of X1 and X3, joint PMF of
X1 and X3; conditioned on the event that X2 = t2, X4 = t4. Once again one can write a joint PMF,
joint conditional

( )
( | ) | ( ) ; so it is complex.
( )

Unveil these our expression, I am writing it like this; there are some simplifications here. But,
anything here is unwieldy; so we will just live with it. And once again the principle how do you
write the joint PMF? It is joint I mean I am sorry.

How do you write the conditional PMF? Is joint by marginal. So, you take ( )
divided by whatever you are conditioning on, simple. It is almost like it is the formula itself is so
simple that you can teach anybody. But, how how it connects to practice is where all the
difficulties will come; we will see conditioning with multiple discrete random variables. So, this
is done.

(Refer Slide Time: 05:38)

Let us see an example; this is slightly more complicated example. So you can see I thought I
should do one worked out example, try a few cases; and at least evaluate a few probabilities, few
joint PMF values to show you how this will work out in terms of flavor. You will see it is it is a
bit confusing sometimes, but it is worth it. So, let us start with let us start with what start at the
very beginning; so, we will start with X1| X2 = 0. So, first thing you have to do is sort of identity
the range; so let me remind you of when somebody gives you a big table of values and things
like that. The first is to look at every column, identify the range.

Make sure you understand what values it takes and all that; so that you can start doing working
on the problem. The first thing is I have identified here that * +; . And you
can see also that this is ; so, 3 of them have the same range and the range of X3 is 0, 1, 2.
Quick things to identify always start whenever you have a random variable problem start with
the range; range is easy to write down. So, let us just write that down first. And then I am going
to start looking at I am dealing with conditioning here; so let say I want to look at X1 given X2
equals 0.

So, maybe not let me not write down the PMF. So, let us look at X1 | X2 = 0. So, this guy so given
X2 is 0, I know I can only it is enough to look at the case, where X2 is 0. So, this guy and this
guy; that is it, I do not have to worry about anything else. So, I know the probability the PMF of
X1 given X2=0, evaluated at say 0 is the joint of X1, X2 of 0 comma 0 divided by the marginal X2
of 0. So, you look at how many cases you have X1 equals X2 equals 0; so if you look at this
distribution, I know it takes 2 values 0 and 1. So, given that X2 is 0, I am seeing that X1 takes
value 0 sometimes takes value 1 sometimes.

So, that is the first thing, so you identify the range of the conditional random variable also; you
identify the overall range that is ok. Then a particular conditioning if I ask you identify, the range
of that object. How do you identify the range? You just focus on what you have conditioned on.
This is X2=0, and then see what are the values that X1 takes for those particular conditioned
situation. So, when X2 is 0, you see that X1 can take value 0 and it can also take value 1, so 0 and
1.

The next step is to find the probabilities the conditional the actual (distrib) PMF; the conditional
PMF values for 0 and 1. For 0 what do you d? You look at 0 0; the cases where you have 0 0 for
both X1 X2 divided by the total probability of X2 equals 0. So, that you will have so that there are
7 cases; so in the denominator you will have 7/12; in the numerator, you will have 4/12. So, this
0 case is just 4/12 by 7/12; maybe I should write this down more cleanly 7/12. And what about
1? 1 is 3/12 by 7/12; so hopefully you see how I got that for the conditional random variable to
take the value 0. I have to look at the case, where X1 and X2 both take 0; that is 4 out of the 4 by
12, probability is 4/12.

And then I have to divide by the probability that X2 is 0; that is 7 out of 12. So, hopefully you
saw that same thing with 1, I will get 3 by 12 and 7 by 12; so, let us look at slightly more
confusing situation. So, let say X1| X3 = 0 and X4 = 1; look at this maybe I should write this. Let
me just write this in some other color; I will pick a blue color. X1 given X3 is 0, X4 is 1; what is X3
=0, X4= 1 that is here, that is here. Notice what is there is one more here, forgot this. So, this
number of cases is sharply decreasing even in this table; but anyway it is there. So, we see here
that this corresponds to 0; these two corresponds to 1, so the range is 0, 1 again.

And what about the probability? The 0, 1 case appeared three times that is like 3 out of 12; and
then 0 appeared and 1 out of 12; so that works out as 1 by 3, just cutting it down. So, once I have
1 by 3, there are only two values other thing should be 2 by 3. I do not need to I do not need to
work out that at all; once I get 1 by 3, I know the next one is 2 by 3. So, it is a valid PMF, it
needs to be valid PMF; so that is enough. So, that was one slightly more confusing situation; let
us look at something bit more interesting maybe; maybe maybe we will look at this case. Let us
look at X3, X4 equals 0, so this is something I am going to look at.

So, I will have a table of possibilities, so I should put some t4, t3 here and t4 here. Now, given X1
is 0 is all of these guys and if I want to look at X3, X4; I have X3 taking values 0, 1, 2. I have X4
taking values 0, 1; so X3 0, 1, 2 and X4 is 0, 1. And if I look at 0, 0 that is 1 out of the 7 cases; so
you can you can do it more carefully. You can do 7/12 in the denominator, 1/12 in the
numerator; we get the same answer. t3 0, t4 is 1 that is again 1/7, t3 is 1 and t4 is 0; that is again
1/7. 1, 1 again happens 2/7, so it happens twice; it happens once here, one is there and then 2, 0.
That happens twice again that is 2/7, and 2, 1 never happens.

So, you see here it works out to be a joint PMF in this way. So, so this an important simple basic
skill in conditioning and all that. Given a table or given the joint PMF marginalize; and find
whatever conditional you want, whatever marginal you want. Hopefully you this is easy to do in
some sense; it is just a question of carefully identifying and summing up the probabilities. In the
uniform case it is very easy, if it is not uniform and all that you have to add up; and if it is a more
complicated joint PMF, it is really really painful. But, it is easy enough to do and the principle is
important.

(Refer Slide Time: 13:48)

So, the next important thing is the conditioning the conditional PMF and the marginal PMFs are
related to the factoring of the joint PMF. So, what is factoring of anything? If you take a big
number, the number seems very big. But, if you keep factoring it, you can write it as a product of
various small numbers. So, even though like a number like 1024 seems very large; but actually it
is 2 into 2 into 2 into 2 ten times. If you do that you get 1024, it seems like if you if you say
description is 2 into 2 into 2 ten times; it seems a bit simpler than 1024 in some sense. So, so that
way if you look at a complicated big joint PMF, which we are not able to understand; or maybe it
is very confusing.

If you can factor it, if you can write it as a product of several terms; and each term maybe is okay
in some level. You can deal with it in some reasonable way, then there is some hope that you can
deal with the joint PMF. So, factoring the joint PMF is a hugely hugely important idea in
understanding a stochastic phenomena, with many different random variables. You do not want
to look at just the joint one, you want to see if you can factor, if you can think of the factors in
some way. So, for that this marginal and conditioning idea is very very crucial it turns out; you
can do factoring in so many different ways. And I will I will talk about some very important
special cases later.

But, where does the fundamental idea come from? It is actually very simple idea. We use the
marginals and conditioning that we saw so far, and we write it as a factor. So, let us look at a
simple example. So, here are 4 random variables X1, X2, X3, X4 and I have a joint PMF
( ); there is some range et-cetera, all of that is ok. I know that the joint PMF
( ) ( ); so, this is easy
enough to write down. So, now I will start putting these brackets, so notice this and is and; I
mean it is kind of everything together. So, it does not matter if I take the and of X2, X3, X4; those
things first and then do an and with this is the same.

So, everything has to be true, so it better be true in that way. So, now I have an and of two
things, and I can do conditioning; so notice how I do conditioning here. I can do probability of
X1 ( |( )) ( ). I
have achieved one factor, I have written the joint PMF as the product of two things, but next
what do I have is I can now repeat this. I have ( ); I can do
bracketing, conditioning and multiplication. ( ) (
|( )) ( ) and finally ( )
( |( )) ( )

( ) ( ); ( |(
)) ( |( )) ( |(
)) ( )

So, notice how conditioning is helping me out; I have written the joint PMF as a product of 4
factors. The first factor is X1 = t1 given all the rest, or actually maybe it is better to read it in
reverse. X4 equals t4 the marginal of X4 equals t4, then X3 equals t3 given X4 equals t4; and X2
equals t2 given X3 equals t3 t4. And then X1 = t1 given all of them; so it is a very nice and simple
thing. I can now write it in terms of my conditional PMFs that I know how to calculate; and I
have a product of 4 conditional PMFs; and 1 marginal and 3 conditional PMFs. So, this kind of
factoring is very very very useful to understand complicated joint PMFs.
You may have a lot of random variable in the joint PMF is very difficult to understand. So, you
maybe able to write down them write these things as product of margins; maybe the conditionals
are still difficult to understand. I am not saying now, but at least it gives you a sense of maybe
some approximation, maybe something you get an idea of. So, we will explore this idea more
later on; but let us focus on the factoring for now. The factoring itself seems to have enough
variety here.

(Refer Slide Time: 18:01)

So, let me show you some examples of this factoring, and show how particular cases it works out
like that. So, let us say let us look at the case, where we have I want to look at ( ). So,
I know I can write this as ( ) | ( ) | ( ); so, let us verify this. This we know
is ( ) 1/ 9, what is ( ) ? So, look at X3 equals 0 and that is this this and this.

There are 3 possible cases, ( ) ; and then I have | ( ). Given X3 is 0, X2 is 0 let us

change the color here; this is this guy and this guy is not it. So, given ( ) and

( ) ; so you get | ( ) . So, 2/9 /3/9 you get 2 by 3; so let us come to

this last case, maybe I will use my next color here. X2 = 0, X3 = 0 that is this, what I had in green
here, and then if you look at that case; that is the case when X1 is 0. So, that works out to be

| ( ) , and you can see ( ) | ( ) | ( )

( ).
So, it is easy enough to check this factoring; it has to be true, and I hope I convinced you that if
you do it in for for any other value. I did it for 000, you can do it for any other 011 002; you pick
your favorite value. And try and practice this calculation, you will see it what it will work; so it
is easy enough.
(Refer Slide Time: 20:19)

Now, here is what is something that is more interesting. I did it in one particular sequence, I did
t1, t2, t3, t4; and it turns out you can do it in any sequence. See the, and is the and in any other
sequence. So, instead of doing X1, X2, X3, X4 I can do X4 equals t4, and X3 equals t3; just reorder
the anding. I can and in any way, so it is and of everything. So, if I do that then repeat the same
thing as before; ( ) ( |(
)) ( |( )) ( |( )) ( ) Now, I get it in this
form, so I can first do marginal of X1 and then conditional of X2 given X1; then conditional of X3
given X1 X2; and then conditional of X4 given X1 X2 X3.

That is also possible; it is the same thing I have written down, I have done it in some other order.
You can do it in one more order; why you stick to just 1, 2, 3, 4 or 4, 3, 2, 1? Let me do 3, 2, 1, 4
why not. I am just adding, I can do it in any order I want; I can do ( ) ( |(
)) ( |( )) ( |( )). You
can go in any order that you like, and you will get the exact same answer.
(Refer Slide Time: 21:34)

So, let us try that; so previously I went 3, 2, 1, let me go 1, 2, 3. So, let me show you how that is
done. So, let say I want to look at the same evaluation, I did;
( ) ( ) | ( ) | ( ) I am going to do it as ( ). Just to be sure
let me check what I did before; I showed you X3, X2| X3 and then X1| X2 X3. So, I am going to do
it in reverse here. I will do X1 and then X2 given X1equals 0 of 0; the same calculation just to
convince you that you will get the same answer of 0. So, let us do a little multicolor presentation
once again. ( ) ; X2 given that one.
So, if you want to look at this guy | ( ), you will get you have to look at this possibility.

| ( ) ; you can divide carefully. If you want, you divide by 5/9, and then the numerator

( )
you will have ( ) . | ( ) ; and then let me pick my third color
( )

for this. X3| X1=0, X2 = 0 and that is just I think it is this one; I am sorry by this. This there are 3
cases here and then this is the favorite favorable case; there are 3 cases of X1=0 X2 = 0 and X3
being 0 is only one of them.

So, you will get | ( ) ; so you can repeat this calculation more carefully. So, if

you multiply it ( ) ; very well you get 1/ 9, which is what which is

what you know is the answer. So, you can check this in any other way, you can take anything
else do it in any other sequence; you will get the same answer, you will get the same. So, these
are identities which are always true. So, alright so I think, so that I would like to stop with the
conditional distributions with this with this lecture.

Hopefully, you saw the utility of it, so we will see soon enough from actual practical uses for
these kind of ideas. But, the factoring of a joint PMF into multiple factors as marginals and
conditionals is a very very powerful idea. It helps simplify a lot of calculations and know
formulations and modeling and all that. So, we will pick up from here in the next class. Thank
you.