Probability Reading Material for ISI Exam

PROBABILITY THEORY

 Probability is a measure of uncertainty.

 Example:

1. Drawing a number at random between 0 and 1 has probability 0. So for each number it has 0
probability and combinational form will give 0 probability as a whole.

2. P(Number drawn is ) between 0 and 1.

So, P(Number drawn is 1/2) .

Rules of Probability.

In an experiment let be the set of all outcomes. is called the Sample space.

 A subset A of is called an event.

 Event A occurs if any occurs.
 To define probability measures we need to specify probabilities of a collection of events, .
 @ is just a specified collection of subsets of the set .
 Only sets in will be assigned probabilities.

Q. What properties must have?

Answer : be closed under finite unions and finite intersections of set in as well as under
complementation

An induction argument shows that if are sets in then so are .

i.e., By, ……(i)

Also if …….(ii)

So from (i) & (ii)

 A non−empty collection satisfying (i) & (ii) is called a field of subsets of or

an algebra of subsets of .

1 Ctanujit Classes
 Probability Reading Material for ISI Exam

 However an algebra is not sufficient to define a probability measure.

 −field or −algebra: A non−empty collection of subsets of a set is called a −filed of

subsets of provided that the following twp properties hold.
(i) If A .
(ii) If is a sequence of sets in , then

both belong to .

Probability Measure:

A probability measure P on a −algebra or −field of subset of is a function such that,

i)
ii) P
iii) If is a sequence of mutually disjoint (exclusive) sets in , then

Let then

Power sets of

Properties of Probability Measures:

If , then

(i) For

(ii)
(iii) (By D’morgan’s Law)

2 Ctanujit Classes
 Probability Reading Material for ISI Exam

Can be extended to a larger number of events.

i.e.

For any event E.

For events A & B

……(1)

If . So, eq. (1) reduces to

Also,

Law of addition:

For any two events A & B.

Proof:

&

……(1)

Now,

3 Ctanujit Classes
 Probability Reading Material for ISI Exam

Since

……(2)

Using (1) & (2), we have

And consequently

Ex:

Solution :

Homework:

Can be proved by induction on A.

Exercise : 200 people attended a dinner party.

of them ate potato salad (among other things)

of them ate chicken casserole.

of them had potato salad but not chicken casserole.

4 Ctanujit Classes
 Probability Reading Material for ISI Exam

What proportions ate either potato salad or chicken casserole (or both)?

……(1)

Now,

So, using (i) & (ii)

Finite Probability : In a finite probability model, the sample space has a finite number of outcomes.
Say

Define simple events:

Ex: Toss 2 coins together

5 Ctanujit Classes
 Probability Reading Material for ISI Exam

{exactly one head shows up}

Facts :

i) Any two different simple events are disjoints.

ii) Any event E is the union of simple events corresponding to outcomes contained in E.

Probability of any event in a finite probability model is the sum of probabilities of simple events whose

(Discrete) Uniform Probability Model

In a finite probability model if all simple events have equal probability then it is called a uniform
probability model.

If and the probability model is uniform then

In such a probability model, if an event E has k outcomes then

Exercise: Roll a balanced die. Probability of any number between 1 & 6 is 1/6.

P(die comes up with an even number)

6 Ctanujit Classes
 Probability Reading Material for ISI Exam

Sample Random Sampling

Consider a population with N, .

We want to choose a random sample of n units from this population to study its characteristics. This is a
random experiments where the outcomes are samples of n units from the above population.

Q. A bowl has 100 marbles of which 50 are red, 30 blue and 20 green. Consider drawing 5 marbles at
random from this bowl. How are marbles drawn? How does the composition of the bowl change after
each draw?

Simple Random Sampling with replacement (SRSWR)

(i) Each marble in the bowl at the time of drawing has the same chance of being drawn at
each draw. After each draw, the color of the marble drawn is noted and the marble drawn
is returned to the bowl.
 In this sampling composition does not change.

Simple Random Sampling without Replacement (SRSWOR)

Each marble in the bowl at the time of drawing has the chance of being drawn at each draw.

After each draw, the color of the marble drawn is noted. But the marble drawn is not returned to the
bowl. * i.e., contents of the bowl changes.

Ordered Sample:

If a set S has m distinct points and another set T has n distinct points, then the numbers of pairs
that can be formed is mn.

If we take then no. of k−tuples is .

Where (S) .

SRSWR:

Population has N distinct units. A sample of size n is drawn using SRSWR.

All the outcomes are equally likely. Each simple event has probability is .

If an event E consist of k outcomes, then

7 Ctanujit Classes
 Probability Reading Material for ISI Exam

Exercise: Roll a pair of balanced die once. What is the probability that the sum of the dots which show
up is 7?

Solution:

Possible favorable outcome

 SRSWOR is a discrete uniform probability model also but the sample is an ordered samples
(n−tuples) where any unit can appear at most once.

Permutation: A permutation of n distinct objects is an arrangement of these in a particular order, say,

two objects.

A, B can be arranged as .

So, total i.e., 21.

Similarly, {A, B, C} can be arranged in 6 ways i.e, 31.

Number of distinguishable permutation of n object is

Consider a set S of N distinct elements. Select one object from it. Without replacing it to the set, draw
one from the remaining (N−1) objects. Continue until we have a sample of n objects. .

Outcome is an n−tuple

Total no. of such outcomes or samples is

Al outcomes are equally likely simple event has probability

Exercise: Consider n distinct boxes and n distinct balls. There are n! ways of throwing these balls into
boxes such that each box gets one ball. In such an experiment probability that ball numbered getting
placed into box numbered j is .

Arguments for this:

8 Ctanujit Classes
 Probability Reading Material for ISI Exam

Remove the th ball & th box then we are left with (n−1) distinct boxes and (n−1) distinct ball.
Probability of the distinct event is

P(K specified balls are in K specified boxes)

Unordered Samples or Combination

A committee of 7 people needs to be formed from a house of 180 people. Committee formed by (151, 2,
7, 92, 57, 63, 10) is the same as that (2, 7, 10, 57, 63, 92, 151). Each permutation of the combination
leads to the same committee.

A poker hand of 5 cards is drawn from a standard deck of cards.

The hand arranged in any order means the same.

Exercise: Consider a population S of m distinct objects. SRSWOR of n objects

m(m−1)(m−2)…….(m−n+1) different samples (ordered n−tuples);

If we take one of the examples, it can be permuted in n! different ways, all leading to the same
combination.

There will be a total of m(m−1) ………(m−n+1) different combination or unordered examples.

Number of ways of choosing n places out of m is

In a population of M objects, m1 are of one type, m2 are type 2, ……., are of type k,

No. of ways of choosing n objects from this population such that n1 are of type 1, n2 are of type 2, ………
are of type k is

Exercise: If there are 8 men & 4 women among 12 volunteers, the no. of ways of choosing a team
consisting of 3 men and 2 women is

9 Ctanujit Classes
 Probability Reading Material for ISI Exam

Q. What is the probability a team of 5 consist of 3 men & 2 women is

Conditional Probability

 Toss a fair 10 times. Probability of any event is ratio of the number of outcomes favouring the
event to 210 (total no. of outcomes).

Q. What is the probability that the last toss came up head?

Q. What is the probability that the last toss came up heads given that there was at least one head in
the experiment?

Now we know that the sample space include (TT….TT), but all others are exactly likely.

P(last toss came up heads given that the experiment resulted in at least one head)

Definition:

Let A & B be two events such that P(A) > 0. Then the conditional probability of B given A is defined as

Ex: Toss a fair coin 10 times. What is the probability of getting exactly 5 heads?

Solution: Total outcome

Possible favorable outcome

10 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: A bowl has m marbles of which m1, are red and the rest blue, n marbles are drawn at random
without replacing. What is the probability that there will be k red marbles in the sample?

Solution:

Ex: For any day in September suppose the following probability model holds.

Bright Sunshine

Partly Sunny

Totally cloud.

Rain

(a) Given that it is cloudy, what is the probability that will rain?

11 Ctanujit Classes
 Probability Reading Material for ISI Exam

(b) Given that it is partly sunny or has bright sunshine what is the probability that it will rain?

LAW OF MULTIPLICATION:

Generalizing to

Partition :

Events partition the sample space

12 Ctanujit Classes
 Probability Reading Material for ISI Exam

Each outcome in belongs one and only one of the partitioning events.

For any event B,

Consider E such that P(E) > 0 and take as the partition of . Then

Bayes Theorem: If P(A) > 0 and P (B)> 0. Then

Proof:

Generalizes to

Ex: In a particular community of the people smoke. For a smoker there is a chance of
getting cancer, whereas for a non−smoker it is 20 . What is the chance that a cancer patient chosen
at random is a smoker?

13 Ctanujit Classes
 Probability Reading Material for ISI Exam

Solution:

Ex: Toss a fair coin 3 times. Find

(a) P(2nd toss comes up head)

(b) P(2nd toss comes up head | 1st cost coming up head)

Solution:

(a)

(b)

A gives no information about the occurrence of B.

Definition: If then we say that A & B are independent events. (probabilistic/statistical).

Definition: Two events A & B are independent if

Ex: Consider SRSWR & SRSWOR of sample size from a set .

Sample {c, c} {(a, a)} {(a, b)} {(a, c)} {(b, a)} {(b, b)} {(b, c)} {(c, a)} {(c, b)}
Events
Probability 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9 1/9
(WR)
Probability 0 0 1/6 1/6 1/6 0 1/6 1/6 1/6
(WOR)

WR: {both draws produce ‘a’}

14 Ctanujit Classes
 Probability Reading Material for ISI Exam

{2nd draw produce ‘c’}

A & B are independent events.

SRSWOR:

Now, A & B are not independent.

In fact

Note: If A & B are independent events with the probability they cannot be disjoint.

Independence .

Pairwise Independence of Mutual independence.

A, B & C are pair wise independent of A & B are independent and A & C are independent & B & C are
independent.

i.e.,

A, B & C are mutually independent if in addition to pair wise independence we have

Ex: .

15 Ctanujit Classes
 Probability Reading Material for ISI Exam

A, B & C are pair wise independent.

 n events are said to be mutually independent if for all subcollections

of size .

1. Toss a coin 3 times.

2. Choose a sample of size 2 using SRSWR for a population of size 100.
3. SRSWOR –no independence for draws P

4. Roll a die. If you hit a six, roll it again. If no six stop.

Ex: (1): Toss a fair coin 3 times. It is reasonable to assume that outcome of the first toss does not
influence that of 2nd toss, or 3rd, outcome of 2nd does not influence that of 3rd, so mutual independence
of components of the experiment may be assured.

for each toss

Ex (2): Draw a sample of size 2 from population of 100 using SRSWR.

In Steps :

16 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex(3): Same with SRSWOR

Elements of Combinatorial Analysis

RULE –I : If there are two groups G1 & G2; consisting of n elements and

consisting of m elements then the no. of pairs formed by taking one

element .

If there are k groups , such that

Then the number ordered k−tuples formed by taking one element from each group is

Example : ‘Placing balls into the cells’ amounts to choose one cell for each ball. Let there are r balls and
n cells. For the 1st ball, we can choose any one of the n cells. Similarly, for each of the balls, we have n
choices, assuming the capacity of each cell is infinite or we can place more than one ball in each cell.
Hence the r balls can be placed in the n cells in ways.

Applications:

1. A die is rolled r times. Find the probability that –

i) No ace turns up. [ace −1]
ii) No ace turns up.

Solution:

i) The experiment of throwing a die r times has possible outcomes.

17 Ctanujit Classes
 Probability Reading Material for ISI Exam

Assume that all possible cases are equally likely. The no. of cases favorable to the event (A), ‘no ace
turns up’ is .

By Classical Definition,

ii) P[ an ace turns up]

Remark : The all possible outcomes of ‘r’ throw of a die correspond to the placing r balls into cells.

RULE –II:

Ordered Samples: Consider a population of n elements any order arrangement

of r elements is called an ordered sample of size r, drawn from the population. Two
procedure are possible –

i) Sampling with replacement: Here an element is selected from the population and the
selected element is returned to the population before the next selection is made. Each
selection is made from the entire population, so that the same element can be drawn more
than ones.
ii) Sampling without replacement : Here an element once chosen is removed from the
population, so that the sample becomes an arrangement without repetition.

 For a population with n elements and a prescribed sample size r, there are different ordered
samples with replacement and n(n−1)…..(n−r+1) different ordered samples
without replacement.

Remark:

1. is defined if and r is a non−negative integers. But

is defined if and r is non−negative integer. In the same
way if then

Example : 1) A random sample of size ‘r’ with replacement is taken from a population of n elements.
Find the probability that in the sample no element appear twice.

Solution: There are sample in all. As the samples are drawn randomly, all samples are equally likely.
The no. of the samples in which in which no element appears twice is the no. of samples drawn without
replacement.

18 Ctanujit Classes
 Probability Reading Material for ISI Exam

Favorable sample is

Hence, the probability is

Example: 2) If n balls are randomly placed into n cells, what is the probability that each cell will be
occupied.

Solution:

SOLVED EXAMPLES:

1. Find the probability that among five randomly selected digits, all digits are different.

Ans:

2. In a city seven accidents occur each week in a particular week there occurs one accidents per
day. Is it surprising?

Ans :

3. An elevator (lift) stands with 7 passengers and stops at 10th floor. What is the probability that
no two passengers leave at the same floor?

Solution:

19 Ctanujit Classes
 Probability Reading Material for ISI Exam

4. What is the probability that r individuals have different birthdays? Also show that the
probability is approximately equal to . How many people are required to make
the prob. of distinct birthdays less than ½ ?

Solution:

More than 23 people are required.

5. Six dice are thrown. What’s the prob. that every possible number will appear.

Hints:

6. There are four children in a family. Find the prob. that

(a) At least two of them have the same birthday?
(b) Only the oldest and the youngest have the same birthday?

20 Ctanujit Classes
 Probability Reading Material for ISI Exam

Hints: (a)

(b)

7. The number 1, 2,….n are arranging in a random order. Find the probability that digits (a) 1, 2 ,
(b) 1, 2, 3 appears as neighbours in the order named.

Hints: consider (1, 2) as a single digit then there are (n−1) entities which can be arranged in (n−1)! ways.

(a) Required prob. is .

(b) Required prob. is

8. (i) In sampling with replacement find the prob. that a fixed element be included at least once.
(ii) In sampling without replacement find the prob. that a fixed element of a population of n
elements to be included in a random sample of size r.

Hints:

(i) [ the fixed element is not included in the sample WOR]

(ii) [ a fixed element is not included in the sample WR]

9. There is 3 volume dictionary among 30 books is arranged in a shelf in random way. Find the
prob. of 3 volume standing in an increasing order from left to right? (The vols. are not
necessary side by side).

Solution: The order of the 3 vols. doesn’t depend on the arrangement of the remaining books. Here 3
vols. can be arranged in 3! ways of which only one case is favorable. Hence prob. is 1/3!.

10. Two fair dice are thrown 10 times. Find the prob. that the first 3 throws result in a sum of 7
and the last 7 throws in a sum of 8.

21 Ctanujit Classes
 Probability Reading Material for ISI Exam

Solution: , be the sample space of the kth throw of a pair of dice,

the sample space of the experiment is

Let, , the event of getting a sum of 7 in a throw of a pair of dice.

And , the event of getting a sum of 8 in a throw of a pair of dice.

Our event is

Favorable cases are

11.
(i) If n men, among whom A and B, stand in a row. What’s the prob. that there will be exactly r
men between A and B?
(ii) If they stand in a ring instead of in a row, show that the prob. is independent of ‘r’.

[In the circular arrangement, consider only that they are leading from A to B in the +ve direction.]

Solution:

(i) n persons can be arranged among themselves in n! ways. Since, the persons are randomly,
all possible cases are equally likely. For the favorable cases if A occupies a position to the left
of B, then A may choose any of the positions:

1st, 2nd,….. (n−r−1)th from the left, with r persons between A and B. The remaining (n−2) persons can
stand in (n−2) places in (n−2)! Ways. Similar thing for B on the left of A.

Hence, no. of favorable cases,

Required probability

(ii) If they form a ring, then the no. of possible arrangement is (n−1)! which is obtained by
keeping the place for any person fixed and arranging the remaining (n−1) persons.

22 Ctanujit Classes
 Probability Reading Material for ISI Exam

For the favorable cases, we fixed the places for A and B, with r individuals between them and then
remaining (n−2) persons can be arranged in (n−2)! ways.

RULE−III:

Subpopulations and Groups: Consider a subpopulation of size ‘r’ from a given population of size ‘n’, let
the no. of the groups of size r be x.

Now the r elements in a group can be arranged in r! ways. Hence x . r! ordered samples of size r.

Application :

1. Each of the 50 states has two senator. Find the prop. of the event that in a committee of 50
senators chosen randomly –
(a) A given state is represented.
(b) All states are represented.

Solution: We can choose a group of 50 senators in ways & since 50 senators are chosen randomly
50 all possible outcomes are equally likely.

(a) There are 100 senators and 98 not from the given state.

Required probability [the given state is not represented] C

(b) All states will be represented if one senators from each state is selected. A committee of 50 with
one senator from 50 states can be selected in ways.

2. If n balls are placed at random in n cells, find the probability that exactly one cell remains
empty.

23 Ctanujit Classes
 Probability Reading Material for ISI Exam

Solution:

Since k balls can be chosen in ways which are to be placed in the specified cells and the remaining
(r−k) balls can be placed in the remaining (n−1) cells in ways.

3. If n balls are placed at a random order in n cells, find the prob. that exactly one cell remains
empty.

Solution:

For the favorable cases, the empty cell can be chosen in n ways and the two balls to be kept in the same
cell can be chosen in ways.

Consider the two balls as a single ball or entity, then (n−1) entities can be arranged in (n−1) cells in
(n−1)! ways.

So, the required prob.

4. A closent contains n pairs of shoes. If 2r shoes chosen at random (2r < n). What is the prob.
that there will be:
(a) No complete pair
(b) Exactly one complete pair
(c) Exactly two complete pair among them.

Solution: (a)

(b)

24 Ctanujit Classes
 Probability Reading Material for ISI Exam

(c)

5. A car is parked among N cars in a row, not at either end. On the return the car owner finds
that exactly r of the N places are still occupied. What’s the prob. that both neighbouring
places are empty?

Solution:

RULE –IV:

The no. of ways in which a population of n elements can be divided into K−ordered parts of which 1st
contains , 2nd contains r2 elements and so on is

Application:

1. In a bridge table, calculate the prob. that

(a) Each of the 4 players has an ace
(b) One of the player receives all 13 spades.

Solution:

(a) In a bridge table 52 cards are partitioned into four equal groups and the no. of different hands is

For the favorable cases, 4 aces can be arranged in 4! ways and each arrangement represents one
possibility of given one ace to each player and the remaining 48 cards can be distributed equally among
the 4 players in

25 Ctanujit Classes
 Probability Reading Material for ISI Exam

(b)

2. In a bridge hand of cards consists of 13 cards drawn at random WOR from a deck of 52 cards.
Find the prob. that a hand of cards will contain
(a) clubs, spades, diamonds
(b) aces
(c) aces and kings.

Solution: (a)

(b)

(c)

3. 4 cards are drawn at random from a full deck of 52 cards. What’s the prob. that
(i) They are of different denominations?
(ii) They are of different suits?
(iii) Both?

Solution:

(i) In a deck of cards there are 13 denominations and 4 suits.

26 Ctanujit Classes
 Probability Reading Material for ISI Exam

For favorable cases select a group of 4 denominations from 13 and then choose one card from each of
the 4 denomination.

So , no. of favorable cases .

(ii)

(iii) For favorable cases, selecting 4 denominations from 13 and then taking one card from the
1st denomination in 4 ways from the 4 suits. Then taking 2nd from the 2nd denomination in 3
ways & so on.

4. From a deck of 52 cards are drawn successively until an ace appears. What is the prob. that
the 1st ace will appear
(a) At the nth draw,
(b) After the nth draw.

Solution:

(a) For the favourable cases, at the nth draw an ace can occur in 4 ways and the first (n−1) cards are
to be taken from 48 non−ace cards which can be done in ways.

(b) For the favorable cases, 1st n cards contain no ace.

5. (Spread of Rumours) In a town of (n+1) inhabitants, a person tells a rumour to a second

person, who in turn, repeats it to a third person, etc. At each step the receipt of the rumour is
chosen at random from n people available.
(i) Find the prob. that the rumour will be told r times without

27 Ctanujit Classes
 Probability Reading Material for ISI Exam

(a) Returning to the originator.

(b) Being repeated to any person.

(ii) Do the same problem when at each step the rumour is told by one person to a gathering
of N randomly chosen individuals.

Solution:

(i) Since any person can tell the rumour to any one of the n available persons in n ways, total
possible cases .
(a) The originator can tell the rumour to anyone of the remaining n persons in n ways & each of the
(r−1) receipts of the rumour can tell to anyone of the remaining (n−1) persons without returning
to the originator in (n−1) ways.

(b)

(ii)
(a)

(b)

28 Ctanujit Classes
 Probability Reading Material for ISI Exam

6. 5 cards are taken at random from a full deck. Find the probability that
(a) They are different denominations?
(b) 2 are of same denominations?
(c) One pair if of one denomination & other pair of a different denomination and one odd?
(d) There are of one denomination & two scattered?
(e) 2 are of one denomination and 3 of another?
(f) 4 are of one denomination and 1 of another?

Solution : (a)

(b)

(c)

(d)

(e)

(f)

29 Ctanujit Classes
 Probability Reading Material for ISI Exam

RULE−V:

 Occupancy Problem: In many situations it is necessary to treat the balls indistinguishable, e.g., in
statistical studies of the distribution of accidents among week days, here one is interested only
in the number of occurrences and not in the individual involved.

Such an example is completely described by its occupancy numbers ; where, denotes the
number of balls in the kth cell.

Here we are interested in number of possible distribution, i.e., the number of different n−tuples
such that .

 Theorem 1: The number of different distributions of ‘r’ indistinguishable balls in n cells, i.e., the
number of different solution of the above fact is

 Theorem 2: The number of different distribution of ‘r’ indistinguishable balls in the n cells in
which no cell remains empty is

Ex: r distinguishable balls are distributed into n cells and all possible distributions are equally likely.
Find the prob. that exactly m cells remain empty.

Solution: The m cells which are to be kept empty can be chosen from n cells in ways and r
indistinguishable balls can be distributed in the remaining (n−m) cells so that no cell remain empty is in

No. of favorable cases

 Application:
1. Show that r indistinguishable balls can be distributed in n cells i.e., the no. of different
solution such that is , where .

30 Ctanujit Classes
 Probability Reading Material for ISI Exam

Solution : Denoting the choices of i.e., 0, 1, ….., r in the indices, we get the factors
.

The no. of different solutions of

The coefficient of in

The coefficient of in

The coefficient of in the expression

The coefficient of in

2. Show that the no. of different distributions of r indistinguishable balls in n cells where no cell
remains empty is .

Hints:

The coefficients of in

The coefficient of in

The coefficient of

31 Ctanujit Classes
 Probability Reading Material for ISI Exam

Random Variables:

Ex: Suppose a machine produces bolts, of which are defective. Find the probability that a box of 20
bolts contains atmost one defective bolt.

Discrete Random Variables:

e.g. : Toss a fair coin 4 times – discrete uniform model

equally likely outcomes.

X of heads

Possible values are 0, 1, 2, 3, 4

if x is not 0, 1, 2, 3, 4.

Definition: The real−valued function f or p defined on R by is called the

probability mass function or probability discrete density function or probability density function of X.

e.g.:

32 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: random integer between 0 & 99 –discrete uniform

Ex: of tosses until a head shows up (in an experiment of tossing a fair coin).

 Properties of P.M.F
i)

ii) support of X is a finite or a countable finite set.

iii)

Examples of Standard discrete probability distribution

1. Bernoulli random Variable & Bernoulli distribution :

Let A be an event with

X is the indicator function of A.

33 Ctanujit Classes
 Probability Reading Material for ISI Exam

A occurs if 1

Ex: Toss a coin once. Let if head comes up, 0 otherwise. Let the coin have probability p of coming
up heads on any toss. Then X is Bernoulli (p) or Bernoulli (p).

For a fair coin, .

Ex: Roll a balanced die once. Let of six dots show up, 0 otherwise.

Bernoulli (1/6).

Let

Bernoulli (1/2).

Binomial Distribution:

Toss a fair coin n times. All outcomes are equally likely. Let of heads in these n tosses.

X can take values 0, 1, 2, 3,…, n.

34 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex 2: Roll a balanced die n times.

of 6’s. What is the probability f(x) ?

of accidents

of radioactive particles emitted during an interval.

Poisson distribution

Intensity rate :

Poisson ( ) if

This f(x) is probability mass function because

i)

ii)

It is convenient to define the function

35 Ctanujit Classes
 Probability Reading Material for ISI Exam

Is called (c.d.f) cumulative distribution function or distribution function (d. f)

For any f,

For any .

F is non−decreasing since

Ex: Toss a fair coin 4 times.

of heads

In this example, F is continuous everywhere except at and at these points just size is

36 Ctanujit Classes
 Probability Reading Material for ISI Exam

In this example, if F is given,

Jointly Distributed Random Variables:

Simultaneously distributed random variables.

Ex: Roll a pair of balanced dice.

of sixes

Sum of the numbers that show up.

A random vector is a set of simultaneously defined random variables.

is called p.m.f or p.d.f.

 Marginal Probability distribution

Ex: Roll a pair of balanced dice

of dots a first dice

of dots on 2nd dice.

Sum of the dots

37 Ctanujit Classes
 Probability Reading Material for ISI Exam

e.g.

& show

Roll a pair of balanced dice

dots on first die

dots on second die

38 Ctanujit Classes
 Probability Reading Material for ISI Exam

Joint p.m.f.

Marginal p.m.f of x,

sum of the dots on 1st &2nd

Given the joint p.m.f. of (X, Y), the marginal p.m.f. of x & y are uniquely determined. However, marginal
p.m.fs do not determine the joint p.m.f. in general.

39 Ctanujit Classes
 Probability Reading Material for ISI Exam

Independent Random Variables:

Consider a bowl with 100 marbles of which 20 are red. Go on drawing one at a time using SRSWR.
Define

are identically distributed.

Since this is simple Random sampling with replacement, outcome of the ith draw can have no influence
on the outcome of any other draw. .

Therefore,

The events are independent if that is

Ex. : Roll a balanced die twice.

of dots in the first roll

of dots in the second roll.

can be modeled to be independent.

Def: Let be r discrete random variables having p.m.f. respectively.

Then are said to be mutually independent if the joint p.m.f.

satisfies

Suppose X & Y are independent with joint p.m.f. & marginal p.m.f. . Then

40 Ctanujit Classes
 Probability Reading Material for ISI Exam

Generalizing if are mutually independent random variables, then

2 = 1 1. 2 2 .

Reason for this:

Ex: (1)

Are X & Y independent?

Ex: (2)

are integers

Are X & Y independent?

Examples: Consider a sequence of independent and identically distributed Bernoulli (p) random
variables . Then for any binary sequence of length n.

41 Ctanujit Classes
 Probability Reading Material for ISI Exam

number of success in a (S, F) sequence where

For 0, 1, …., n

If .

 It is possible to construct an infinite sequence of i.i.d. Bernoulli (p) random variables:

,… for any positive integer n, are mutually dependent.

Consider such a sequence if i.i.d. Bernoulli (p) random variable.

42 Ctanujit Classes
 Probability Reading Material for ISI Exam

Let of 0’s before the first 1 in the sequence.

geometric (p)

If k is a non negative integer,

Suppose the lifetime of a device (in whole units) is modeled using the geometric (p) distribution.

of units until the device fails.

What is implication of such a model?

For +ve integers K & , consider

the device will not fail until K+ units given that it has not failed by time K)

43 Ctanujit Classes
 Probability Reading Material for ISI Exam

Geometric (P) if 0, 1, 2,…

of failures before the 1st success.

(success)

Negative Binomial: Consider again an infinite sequence of i.i.d. Bernoulli (P) :

Let be any positive integer.

number of failures before there are r successes

of 0’s in the sequence until there are exactly r 1‘s for the first time.

Where is the number of 0’s after the th 1 and before the ith 1.

are iid geometric (p).

K 0’s are distributed in k+r−1 places in the binary sequence).

Negative Binomial geometric (p)

Hyper geometric distribution

Recall the example where we consider a bowl with m marbles of which mp are red in colour and m(1−p)
blue. n marbles are drawn at random without replacement.

of red marbles in the sample.

44 Ctanujit Classes
 Probability Reading Material for ISI Exam

Bernoulli (p), are Bernoulli (p)

But they are not independent.

What will happen if m is very large compared to n?

Then removing any fixed number of marbles upto n will not change the constitution of the bowl at any
draw significantly.

are approximately iid Bernoulli (P).

Hyper geometric distribution can be approximated by the Binomial distribution or SRSWOR SRSWR.

Multinomial Distribution

Ex Roll a balanced die 20 times and

Let of times dots showing

For

45 Ctanujit Classes
 Probability Reading Material for ISI Exam

Here n 20

In general, let an experiment have r possible outcomes.

Let ,

Repeat this experiment n times and define

of times jth outcome occurs,

( of sequences such that there are 1’s, 2’s ,….., r’s)

Define

46 Ctanujit Classes
 Probability Reading Material for ISI Exam

Then are iid Bernoulli (p1) since

Marginals of multinomial are Binomial.

Functions of Random Variable (discrete)

Ex: Bernoulli (p).

is also a Bernoulli i.e., Bernoulli (p).

Ex: Binomial (n, p)

Let X have p.m.f. and let for some function g.

Then
: ( )=

47 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex:

Sum of independent discrete random variables

Consider (X, Y) with joint p.m.f. .

Let X take values Y take values

Let , then

Suppose X & Y are independent

Then

48 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: Suppose Binomial (n1, p) , Binomial (n2, p) & X & Y are independent.

Show Binomial .

For

If sum is over all x such that (*) holds.

So,

Binomial (n1, p) of successes in trials where

Binomial (n2, p) of success in the next n2 trials

of successes in trials. .

49 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: . X & Y are independent. Show .

Expectation of Discrete Random Variables

Ex: In a population there are 3 grades of workers, A , B & C who collect fruits. The properties are 20 ,
50 . A grade workers collect 100 kg of fruits per day, B grade 60 kg & C grade 40 kg. What is the
average collection of a randomly chosen person (per day)?

Solution:

100 workers

Collects

So average kg

Collection of a Randomly chosen worker

X takes 3 values 100, 60 & 40.

Expectation or expected value or Mean of a discrete random variable.

 In general, if X is a discrete r.v with p.m.f. , then the expected value of X is defined as

50 Ctanujit Classes
 Probability Reading Material for ISI Exam

e.g. Suppose X is uniformly over the integers 0, 1, 2, ….., 100 i.e.

 In general, if X is uniform over

Ex: X Bernoulli (p). Then

Ex: Binomial (n, p). Then

Ex: X Geometric (p). Then

51 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: X Poisson

Then

Show

Ex:

If

52 Ctanujit Classes
 Probability Reading Material for ISI Exam

Result:

If X is a discrete r.v. with p.m.f. and g is a real−valued function such that

Then for

 If summation goes to we generally don’t define E(X).

Result: If is a discrete random vector having p.m.f.

Theorem:

Let X & Y be two random variables with joint p.m.f. and suppose E(X) and E(Y) exist.

i) If for some C,
ii) For any c constant, exists and

iii)

53 Ctanujit Classes
 Probability Reading Material for ISI Exam

iv) If P(X , then , with equality if .

Proof: ,

v) .

 , then

 Suppose X takes values,

Similarly

54 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: X Bernoulli (p), .

Result: If X & Y are independent with p.m.f. and they have finite expectations, then

Proof :

But converse may not be true (in general).

If , then X & Y need not be linearly independent.

E(X)

with probability 1

mean of X

Ex: X Binomial (n, p)

55 Ctanujit Classes
 Probability Reading Material for ISI Exam

i.i.d Bernoulli (p)

Ex: X NB (r, p)

Ex: A bowl has m marbles out of which mp are red, rest blue.

a) Draw n marbles at random using SRSWR from this

of red marbles in the sample

b) Draw n marbles at random using SRSWOR.

X of red marbles in the sample,

X HG (m, p ; n)

Bernoulli (p)

 Poisson approximation for the Binomial Bin (n, p) n is large, p small, no is moderate.

Consider the number of radioactive properties emitted by a source during a unit time interval. Suppose
the average rate is (we would model the count as Poisson ( )).

Total of particles emitted sum of counts in the subintervals

Sum of n Bernoulli trials

Binomial

is large enough so that there will be at most one particle emitted is any subinterval.

56 Ctanujit Classes
 Probability Reading Material for ISI Exam

What happens as ?

Probability that a Poisson takes value k

Ex: X Binomial (n, p),

MOMENTS

Let X be a discrete random variables with p.m.f .

If X has finite expectation we denote it by .

For any integer if has finite expectations, the rth moment (raw moment of X is defined as

57 Ctanujit Classes
 Probability Reading Material for ISI Exam

If exists, rth moment of is called the rth central moment of X.

Result: If rth moment exists, rth central moment exists, and vice versa.

Result: If rth moment exists, then all moments of order k, exist.

Result: If X & Y have moments of order r, so does X+Y.

Def: If X has finite second moment, then its second central moment is called its variance.

Def: Standard Deviation of X is defined as

Result: for some C.

If , then

Then,

If , then

58 Ctanujit Classes
 Probability Reading Material for ISI Exam

Suppose X has mean . Thus

Is called the standardized form of X.

Variation of the sum. Let X & Y be two random variables with finite second moment. Then X+Y has 2 nd
moment and

Covariance between X & Y :

59 Ctanujit Classes
 Probability Reading Material for ISI Exam

Suppose,

Then .

As X increases does Y typically increase?

Is there a positive associate between X & Y?

If X & Y are independent,

If are mutually independent.

iid or a random sample from a population with mean .

60 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: Y Bernoulli (p). Then

iid Bernoulli (p).

Each has mean p & variance p(1−p)

61 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: Show that if X Poisson .

−depends on the scaling or units used by X & Y.

i.e.

is a standardized measure derived from Cov (X, Y) which does not depend on scaling.

is called the correlation coefficient between X & Y.

Theorem :

for some constant a.

Proof: Hint

Cauchy−Schwarz inequality . If X & Y have second moments,

Then

Apply this to r.v. :

62 Ctanujit Classes
 Probability Reading Material for ISI Exam

Q. If X & Y have finite second moments then if in addition X & Y are independent.

is a measure of dependence between X & Y.

However, X & Y need not to be independent if .

, for some constant a.

So measures linear association between X & Y.

CONTINUOUS RANDOM VARIABLE

Many random variables are best treated as continuous, in the sense, any value in an interval of values is
a probability.

lifetime of a light bulb.

Probability of T being any fixed value is O, but the probability of T being in a subinterval is positive.

waiting time at a service counter.

A random number between 0 & 1

Point where the arrow in a spinning wheel stops.

Arrival time of radioactive particle

For a continuous Random Variable X

for any x

How do we describe its probability distribution?

We do so by describing probability such as

Def : A random variable X on a probability space is a real−valued function

Such that for all

With this distribution we can proceed by noting that the cdf of X,

is available

Then, for any

63 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: For of particles

Emitted by a radioactive substance in the time interval [0, f]. Suppose N

Waiting time for the first particle to arrive

No rounding off or approximation is involved.

Ex: Random number between 0 & 1.

Poisson (2)

How does one get the probability density function from its cdf?

Continuous Probability Distributions

If X is any random variable and F is its c.d.f. then

(i)

64 Ctanujit Classes
 Probability Reading Material for ISI Exam

(ii) F is non−decreasing
(iii) F(− ,F
(iv) F(x+) light continuity

Since

F has a jump discontinuity at x; otherwise if , F is continuous at x.

For a discrete random variable, its c.d.f. F has a countable no. of jump discontinuities

A random variable is continuous (i.e., ) if, its cdf is continuous at every x, i.e., from
(*),

Definition:

The probability density function (when it exists) of a continuous random variable x, with cdf F is a
non−negative function, F such that

Since,

(at all the points x where f is continuous)

Now, F is the area under the curve of . Since, area over any point is 0; f can be arbitrarily defined at
many points. Also, F must be differentiable almost everywhere.

If X is continuous with p.d.f. f,

65 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex:

random variable between 0 & 1

i.e., X is uniform over [0, 1]

Ex:

Thus

Exponential

is the p.d.f of a continuous random variable X.

What does this mean?

Suppose

For any

66 Ctanujit Classes
 Probability Reading Material for ISI Exam

lifetime of a light bulb

(Memory less property)

is called the parameter of the distribution

e.g. X

For

Put

67 Ctanujit Classes
 Probability Reading Material for ISI Exam

Now,

But

i)

So,

ii)

iii)

68 Ctanujit Classes
 Probability Reading Material for ISI Exam

Properties of Gamma Distribution

i) Gamma

ii) For

The gamma p.d.f. is unbounded near O.

Except for certain values of ,

is not available in closed form.

If is an integer, then

Repeating the lines

69 Ctanujit Classes
 Probability Reading Material for ISI Exam

If are iid Exp then,

Ex: X if

Where are parameters of distribution

i)

ii)

iii)

Unbounded near O

Ex: X

is called the standard normal distribution

Q (z)

Density of N (0, 1) : Q (z)

cdf of N(0, 1)

70 Ctanujit Classes
 Probability Reading Material for ISI Exam

71 Ctanujit Classes
 Probability Reading Material for ISI Exam

Functions of Random Variables

Ex: Suppose X

What is the probability distribution of Y?

Solution:

For

Find the probability distribution of

Ex: Suppose

Then

For y > 0

72 Ctanujit Classes
 Probability Reading Material for ISI Exam

Chi –square with k degrees of freedom

Change of Variable Formula

Find the p.d.f. of where the p.d.f. of X is given and g is I−1 real value function.

Theorem: Let g be a differentiable strictly monotone (strictly increase or strictly decreasing) function on
an interval I. Let g(I) denote the range of g and let denote the inverse of g. Let X be a continuous
random variable having p.d.f .

Then has p.d.f given by

Proof: Let denote the cdf of X & Y, respectively consider the case when g is strictly increasing.
Then is also strictly increasing.

73 Ctanujit Classes
 Probability Reading Material for ISI Exam

If &

Since

For

Show this result when g is decreasing.

Ex: X

Let

74 Ctanujit Classes
 Probability Reading Material for ISI Exam

For a < 0

Result: Let X be a continuous random variable with pdf . For constants a & b, a , let

Then

Ex: Let .

75 Ctanujit Classes
 Probability Reading Material for ISI Exam

What is the median income of an Indian family?

In [2011−12, 90500 per annum]

income of an Indian Family

continuous strictly increasing in an interval I

Given 0>p<1 there exists

Such that

Expectation of a Continuous Random Variable

If Y is a discrete with p.m.f f(y) and

If Y is continuous with p.d.f f then

76 Ctanujit Classes
 Probability Reading Material for ISI Exam

i.e., replace sum by integral

Def: Let X be a continuous r.v. with p.d.f .

Then we say that X has finite expectation (expected value) and define its expectation by

Ex:

Ex: If U

Ex: If X then

77 Ctanujit Classes
 Probability Reading Material for ISI Exam

[b−a length of (a, b)]

OR

Ex: X

By his result

If Y is a continuous r.v. with p.d.f. such that and if Y has finite expectation,
then .

Let

78 Ctanujit Classes
 Probability Reading Material for ISI Exam

So (1) reduces to using (2)

Ex: X

Show

More generally, if then

For a continuous random variable X with p.d.f. , and mean if X has finite 2nd moment then

Then the variance of X is given by

79 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: U

So

Ex: If X

Then

Then

Ex: X

Then ,

Let

80 Ctanujit Classes
 Probability Reading Material for ISI Exam

Ex: X

Continuous bivariate distribution

Let X & Y be jointly distributed with cdf F(x, y)

Marginal cdfs of X & Y are given by

Joint p.d.f. of (X, Y) is given by f(x, y)

Where

81 Ctanujit Classes
 Probability Reading Material for ISI Exam

Similarly

Ex: (X, Y) coordinates of a randomly chosen points inside the unit disk

Find

Sol. Since

82 Ctanujit Classes