Probability and Statistics
Probability and Statistics
Probability and Statistics
In our day to day life, we come across many uncertainty of events. We wake up in the morning
and check the weather report. The statement could be 'there is 60% chance of rain today'. This
statement infers that the chance of rain is more than that having a dry weather. We decide upon
our breakfast from a statement that "corn flakes might reduce cholesterol". What is the chance of
getting a flat tyre on the way to an important apartment? And so on.
ADVERTISEMENT
How probable an event is? We generally infer by repeated observation of such events in long
term patterns.
Probability is the branch of mathematics devoted to the study of such events.
People have always been interested in games of chance and gambling. The existence of games
such as dice is evident since 3000 BC. But such games were not treated mathematically till
fifteenth century. During this period, the calculation and theory of probability originated in Italy.
Later in the seventeenth century, French contributed to this Literature of study. The foundation
of modern probability theory is credited to the Russian mathematician, Kolmogorov. In 1993, he
proposed the axioms, at which the present subject of probability is based.
Random Experiment and Sample Space
An experiment repeated under essentially homogeneous and similar conditions results in an
outcome, which is unique or not unique but may be one of the several possible outcomes. When
the result is unique then the experiment is called a 'deterministic' experiment.
ADVERTISEMENT
Example:
While measuring the inner radius of an open tube, using slide callipers, we get the same result by
performing repeatedly the same experiment. Many scientific and Engineering experiments are
deterministic.
If the outcome is one of the several possible outcomes, then such an experiment is called a
"random experiment" or 'nondeterministic' experiment.
In other words, any experiment whose outcome cannot be predicted in advance, but is one of the
set of possible outcomes, is called a random experiment.
If we think an experiment as being performed repeatedly, each repetition is called a trial. We
observe an outcome for each trial.
Example:
An experiment consists of 'tossing a die and observing the number on the upper-most face'
In such cases, we talk of chance of probability, which numerically measures the degree of chance
of the occurrence of events.
Sample Space (S)
The set of all possible outcomes of a random experiment is called the sample space, associated
with the random experiment.
Note:
Each element of S denotes a possible outcome. Each element of S is known as sample point.
Any trial results in an outcome and corresponds to one and only one element of the set S.
e.g.,
1. In the experiment of tossing a coin,
S = {H, T}
2. In the experiment of tossing two coins simultaneously,
S = {HH, HT, TH, TT}
3. In the experiment of throwing a pair of dice,
S = {(1,1), (1,2), (1,3), (1,4), (1,5), (1,6), (2,1), (2,2),………. (6,1), (6,2), (6,3), (6,4), (6,5),
(6,6)}
Events
An event is the outcome or a combination of outcomes of an experiment. In other words, an
event is a subset of the sample space. e.g.,
{a head} in the experiment of tossing a coin is an event.
Types of Events S= {1,2,3,4,5,6}] If the event is set of elements less than 2, then E = {1} is a
simple event 1)
Simple Event:
If an event has one element of the sample space then it is called a simple or elementary event.
Example:
Consider the experiment of throwing a die.
Compound Event:
If an event has more than one sample points, the event is called a compound event . In the above
example, of throwing a die, {1, 4} is a compound event.
Null Event (φ ):
As null set is a subset of S, it is also an event called the null event or impossible event. 4) The
sample space S= {1, 2, 3, 4, 5, 6} in the above experiment is a subset of S. The event represented
by S occurs whenever the experiment is performed. Therefore, the event represented by S is
called a sure event or certain event.
Complement of an Event:
The complement of an event E with respect to S is the set of all the elements of S which are not
in E.
The complement of E is denoted by E' or EC. Note: In an experiment if E has not occurred then
E' has occurred. Algebra of Events In a random experiment, considering S(the sample space) as
the universal set, let A, B and C be the events of S. We can define union, intersection and
complement of events and their properties on S, which is similar to those in set theory.
Example: Consider the same experiment throwing a die, then S= {1, 2, 3, 4, 5} Let A = {1, 2, 3,
Example:
In throwing of a dice, there are 6 exhaustive cases, {1}, {2}, {3}, {4}, {5}, {6}.
In throwing of a pair of dice, there are 36 exhaustive cases. Example of an event which is
exhaustive, but not mutually exclusive. In throwing a die experiment, let E1 represent the
outcomes which are less than 4, E1= {1,2,3}.
Let E2 represent the outcomes which are greater than 2, E2= {3,4,5,6}.
Let E3 represent the outcomes which are greater than 4, E3 ={5, 6}. Clearly E1 and E3 are
mutually exclusive, but not exhaustive. Let E4 ={3, 4, 6} and E5 = {1, 2, 5} Then E4 and E5 are
mutually exclusive and exhaustive since.
Equally Likely Outcomes The outcomes of a random experiment are said to be equally likely, if
each one of them has equal chance of occurrence. Example: The outcomes of an unbiased coin
are equally likely.
Probability of an Event
So far, we have introduced the sample of an experiment and used it to describe events. In this
section, we introduce probabilities associated to the events. If a trial results in n-exhaustive,
mutually exclusive and equally likely cases and m of them are favourable to the occurrence of an
event A, then the probability of the happening of A, denoted by P(A), is given by
ADVERTISEMENT
Note 2:
If P(A) = 0 then A is called a null event, or impossible event.
Note 3:
If P(A) = 1 then A is called a sure event.
Note 4:
If m is the number of cases favourable to A. Then m - n is favourable to "non occurrence of A
Theorems of Probability
Theorem 1:(Addition Rule of Probability)
(Fr
ADVERTISEMENT
Note 1:
Note 2:
Example:
In tossing a fair die, what is the probability that the outcome is odd or grater than 4? Suggested
answer: Let E1 be the event that the outcomes are odd. E1 = {1,3,5} Let E2 be the event that the
∴ P(A) = 1 - P(AC)
Example:
In tossing a die experiment, what is the probability of getting at least 2. Suggested answer: Let
E be the event that the outcome is at least 2, then E = {2,3,4,5,6} EC= {1}
Theorem 3:
=1-1=0
Example:
In throwing a die experiment, what is the probability of occuring a number greater than 8 ?
Suggested answer: Let E be the event where the outcome is greater than 8. E = φ P(φ ) = 0
Multiplication Rule of Probability
We have already proved that if two events A and B from a sample S of a random experiment are
mutually exclusive, then
In this section, we examine whether such a rule exists, if ' ' is replaced by ' ' and '+' is replaced
by 'x' in the above addition rule. If it does exist, what are the particular conditions restricted on
the events A and B.
This leads us to understand the dependency and independency of the events.
ADVERTISEMENT
Example:
A bag contains 5 white and 8 black balls, 2 balls are drawn at random. Find
a) The probability of getting both the balls white, when the first ball drawn, is replaced.
b) The probability of getting both the balls white, when the first ball is not replaced.
Suggested answer:
a) The probability of drawing a white ball in the first draw is . Since the ball is replaced, the
b) The probability of drawing a white ball in the first draw is . If the first ball drawn is white
and if it is not replaced in the bag, then there are 4 white balls and 8 black balls. Therefore, the
Independent Events
Events are said to be independent if the occurrence of one event does not affect the occurrence of
others.
Observe in case(a) of above example,
The probability of getting a white ball in the second draw does not depend on the occurrence of
the event on the first draw.
However in case(b), the probability of getting a white ball in the second draw depends on the
occurrence and non - occurrence of the event in the first draw.
It can be verified by different example.
If A and B are two independent events, then
In case (b),
Here P(A B) P (A). P(B) since the events are not independent. Independent Experiment
Two random experiments are said to be independent if for every pair of events E and F, where E
is associated with the first experiment and F is associated with the second experiment, the
probability of simultaneous occurrence of E and F, when the two experiments are performed, is
the product of the probabilities P(E) and P(F), calculated separately on the basis of the two
experiments.
(Considering the experiments as independent, because A and B solve the problem independently)
Note:
If A and B are independent, then
i) Ac and Bc are independent
ii) Ac and B are independent
iii) A and Bc are independent
Random Variables and Probability Distributions
It is often very important to allocate a numerical value to an outcome of a random experiment.
For example, consider an experiment of tossing a coin twice and note the number of heads (x)
obtained. Outcome HH HT TH TT No. of heads (x) 2 1 1 0 x is called a random variable, which
can assume the values 0, 1 and 2.
ADVERTISEMENT
Thus, random variable is a function that associates a real number to each element in the sample
space. Random variable (r.v) Let S be a sample space associated with a given random
experiment.
A real valued function X which assigns to each ω i ∈ S, a unique real number, X(ω i) = xi is
called a random variable.
Note:
There can be several r.v's associated with an experiment. A random variable which can assume
only a finite number of values or countably infinite values is called a discrete random variable.
e.g., Consider a random experiment of tossing three coins simultaneously. Let X denote the
number of heads obtained. Then, X is a r.v which can take values 0, 1, 2, 3. Continuous random
variable A random variable which can assume all possible values between certain limits is called
a continuous random variable. Discrete probability distribution A discrete random variable
assumes each of its values with a certain probability.
Let X be a discrete random variable which takes values x1, x2, x3,…xn where pi = P{X = xi} Then
X : x1 x2 x3 …xn P(X) : p1 p2 p3 … pn is called the probability distribution of x, If in the probability
distribution of x,
Note 1 :
P{X = x} is called probability mass function.
Note 2:
Although the probability distribution of a continuous r.v cannot be presented in tabular forms, we
can have a formula in the form of a function represented by f(x) usually called the probability
density function.
Theorems Introduction
From our earlier chapter we know that, in statistical experiments, if the events A and B are
independent, then
But suppose the two events are not independent, that is the occurrence of one depends on the
Let B = {2, 3, 4, 5, 6}. If, after the die is thrown, we are given the information, that the event B
has occurred, then the probability of event A will no more be 1/2, because in this case, the
favourable cases are two and the total number of possible outcomes will be five and not six.
ADVERTISEMENT
The probability of event A, with the condition that event B has happened will 2/5. This
conditional probability is denoted as P(A/B). Let us define the concept of conditional probability
in a formal manner.
Let A and B be any two events associated with a
random experiment. The probability of occurrence of event A when the event B has already
occurred is called the conditional probability of A when B is given and is denoted as P(A/B). The
conditional probability P(A/B) is meaningful
Remark 1:
Remark 2:
If A and B are mutually exclusive events, then
∴ If A and B are mutually exclusive events, then A/B and B/A are impossible events.
For an illustration, let us consider the random experiment of throwing two coins.
∴ S = {HH, HT, TH, TT}
Let A = {HH, HT}, B = {HH, TH} and C = {HH, HT, TH}
A/B is the event of getting A with the condition that B has occurred.
Remark 3:
We know that for the events A and B,
If B = S then
= P (A)
Remark 4:
If A = B
Remark 5:
From the formula of conditional probabilities, we have
Equation (2) and equation (3) are known as multiplication rules of probability for any two events
A and B of the same sample space.
Remark 6:
We know that two events are independent if the occurrence of one does not effect the occurrence
of other. If A and B are independent events
P (A/B) = P (A) and P (B/A) = P (B)
Remark 7:
So far, we have assumed that the elementary events are equally likely and we have used the
corresponding definition of probability. However the same definition of conditional probability
can also be used when the elementary events are not equally likely. This will be clear from the
following example.
Suppose a die is tossed. Let B be the event of getting a perfect square.
The die is so constructed that the event numbers are twice as likely to occur as the odd numbers.
Let us find the probability of B given A, where A is the event getting a number greater than 3
while tossing the die.
S = {1, 2, 3, 4, 5, 6}
If probability of getting an odd number is x, the probability of getting an even number is 2x.
Since P (S) = 1
x + 2x + x + 2x + x + 2x = 1
9x = 1
A = {4, 5, 6}
Example 1:
A card is drawn from an ordinary deck and we are told that it is red, what is the probability that
the card is greater than 2 but less than 9.
Suggested answer:
Let A be the event of getting a card greater than 2 but less than 9.
B be the event of getting a red card. We have to find the probability of A given that B has
occurred. That is, we have to find P (A/B).
In a deck of cards, there are 26 red cards and 26 black cards.
∴ n(B) = 26
Among the red cards, the number of outcomes which are favourable
Example 2:
A pair of dice is thrown. If it is known that one die shows a 4, what is the probability that
a) the other die shows a 5
b) the total of both the die is greater than 7
Suggested answer:
Let A be the event that one die shows up 4. Then the outcomes which are favourable to A are
(4, 1), (4, 2), (4, 3), (4, 4), (4, 5), (4, 6) (1, 4), (2, 4), (3, 4),
(5, 4), (6, 4)
(a) Let B be the event of getting a 5 in one of the dies. Then the outcomes which are favourable
to both A and B are (4, 5), (5, 4)
(b) Let C be the event of getting a total of both the die greater than 7.
The out-comes which are favourable to both C and A.
(4, 4), (4, 5), (4, 6), (5, 4), (6, 4)
∴ n (C) = 5
Note that in the above example P (B) and P (B/A) are different.
ii)
Before we state and prove Baye's Theorem, we use the above two rules to state the law of total
probability. The law of total probability is useful in proving Baye's theorem and in solving
probability problems. Following is an example which explains this law.
ADVERTISEMENT
Example:
Let S is the sample space which is the population of adults in a small town who have completed
the requirement for a college degree. The population is categorized according to sex and
employment status as follows
∴ One of these individual is to be selected for a tour throughout the country. Knowing that the
individual chosen is employed, what is the probability that the individual is a man?
Suggested answer:
Let M be the event that a man is selected, E be the event that the individual selected, is
employed. Using the reduced sample space, we have
Also, we have
From the original sample space, we have
Suppose that we are now given the additional information that 36 of those employed and 12 of
these unemployed are the members of the rotary club. What is the probability of the event A that
the selected individual is a member of the rotary club?
Suggested answer:
A is the event that the selected individual is a member of the rotary club.
The generalization of the foregoing example, where the sample space is partitioned into n subsets
is known as Law of Total Probability. Theorem: (Law of Total Probability) If B1, B2, B3, ….., Bn
are mutually exclusive and exhaustive events of the sample space S, then for any event A of S.
Proof:
Proof:
P(A) P(A1) P(A/A1) +P(A2)P(A/A2)+….+P(An) P(A/An) ….(1)
Also,
Example:
In a bolt factory 25%, 35% and 40% of the total is manufactured by machines A, B and C, out of
which 5%, 4% and 2% are respectively defective. If the bolt drawn is found to be defective, what
is the probability it is manufactured by the machine A?
Suggested answer:
Given P (A) = 0.25, P (B) = 0.35 and P (C) = 0.4 Let D be the event of getting a defective bolt.
P (D/A) = 0.05 P (D/B) = 0.04 P (D/C) = 0.02
Random variable (r.v) Let S be a sample space associated with a given random
experiment. A real valued function X which assigns to each ω i ∈ S, a unique real
number.
Note:
There can be several r.v's associated with an experiment. A random variable which
can assume only a finite number of values or countably infinite values is called a
discrete random variable. e.g., Consider a random experiment of tossing three coins
simultaneously. Let X denote the number of heads then X is a random variable
which can take values 0, 1, 2, 3.
Note :
Example :
Two cards are drawn successively without replacement, from a well shuffled deck of cards. Find
the mean and standard deviation of the random variable X, where X is the number of aces.
Suggested answer:
X is the number of aces drawn while drawing two cards from a pack of cards. The total ways of
drawing two cards 52C2. Out of 52 cards these are 4 aces. The numbers of ways of not drawing an
Ace =48C2. The number of ways of drawing an ace is 4C1 x 48C1 and two aces 4C2. Therefore the
r.v. X can take the values 0, 1, 2.
= 0.1629 - 0.0236 = 0.13925 Let X
be a continuous random variable which can assume values in (a, b) and f(x) be the probability
x is given by
Binomial Distribution
A trial, which has only two outcomes i.e., "a success" or "a failure", is called a Bernoulli trial.
Let X be the number of successes in a Bernoulli trial, then X can take 0 or 1 and
P(X =1) = p = "probability of a success"
P(X = 0) = 1 - p = q = "probability of failure".
ADVERTISEMENT
Example:
5 cards are drawn successively with replacement from well shuffled deck of 52 cards. What is
the probability that
i) all the five cards are spades
ii) only 3 cards are spades
iii) none is a spade.
Suggested answer:
Let X be the random variable for the number of spade cards drawn.
p = probability of drawing a spade card
q=1-p
n=5
Recurrence Relation for the Binomial Distribution
We have
P(X = x + 1) = nCx+1 px+1 qn-x-1
= n2p2 - np2 + np
= n2p2 + np(1-p)
= n2p2+ npq
Now V(x) = E(x2) - [E(x)]2
= n2p2 + npq - n2 p2
= npq
Example:
If the mean and variance of a binomial distribution are respectively 9 and 6, find the
distribution.
Suggested answer:
Mean of a binomial distribution = np = 9
Variance of a binomial distribution = npq = 6
np = λ is finite.
Poisson Distribution as a Limiting Form of the Binomial Distribution We shall now deduce the
Poisson distribution from the binomial distribution by assuming that n → ∞ and p → 0 such that
the product np always remains finite, say λ .
We shall now use a very important result of limits in Calculus. We state this result without proof:
where e is a constant lying between the number 2 and 3 and ex is defined by
with x a real number. From equation (1), we observe that each of (r - 1) factors,
that
Thus
where λ is a finite number and is equal to np. The sum of the probabilities P(X = r) or simply
P(r) for r = 0, 1, 2, … is 1. This can be seen by putting r = 0, 1, 2, … in (4) and adding all the
probabilities.
Also, each of the probabilities is a non-negative fraction. This leads to the distribution defined
below:
A random variable X taking values 0, 1, 2, … is said to have a Poisson distribution with
parameter λ (finite), if its probability distribution is given by
There are many daily life situations where n is very large and p is very small. In such situations,
the Poisson distribution can be more conveniently used as an approximation to binomial
distriburtion which may prove cumbersome for large values of n. This is called Poisson
approximation to binomial distribution. The Poisson approximation to binomial distribution is
easier to compute directly and easier to tabulate than the binomial distribution, since the values
of e-λ for various values of λ are found in standard tables. Some examples of such situations are
i) telephone trunk lines with a large number of subscribers and the probability of telephone lines
being available is very small, ii) traffic problems with repeated occurrence of events such as
accidents whose probability is very small, iii) many industrial processes undergoing mass scale
production with probability of events as 'faults' or 'breakdowns' being very small, etc. The
probability mass function of the Poisson distribution given by
Note 1:
= e-λ eλ =
1
Example: A random variable X has a Poisson distribution with parameter λ such that P (X = 1)
= (0.2) P (X = 2). Find P (X = 0).
Suggested answer: For the Poisson distribution, the probability function is given by
Given P (x = 1) = (0.2) P (X = 2)
= λ e-λ eλ = λ
Example:
If the variance of the Poisson distribution is 2, find the probabilities for r = 1, 2, 3, 4
and 5 from the recurrence relation of the Poisson distribution.
Suggested answer:
The variance of the Poisson distribution
=λ =2
Recurrence relation is given by
Applications
After the elementary study in probability, let us see how we can utilise this basic knowledge in
solving problems of different areas.
ADVERTISEMENT
Example 1:
In a housing colony 70% of the houses are well planned and 60% of the houses are well planned
and well built. Find the probability that an arbitrarily chosen house in this colony is well built
given that it is well planned.
Suggested answer:
Let A be the event that the house is well planned. B be the event that the house is well built. P
(A) = 0.7 Probability that a house, selected is well built given that it is well
planned.
Example 2:
In a binary communication channel, A is the input and B is the
Suggested answer:
We know that
We know that
By Law of total probability
Example 3:
A manufacture ships his products in boxes of 10. He guarantees that not more than 2 out of 10
items are defective. If the probability that an item selected at random from his production line
will be defective is 0.1, what is the probability that the guarantee is satisfied.
Suggested answer:
Let X be the random variable which represents number of defective items selected which has a
binomial distribution with
n = 10, p = 0.1, q = 0.9
Probability that the guarantee is satisfied
=
Conclusion
In this chapter we have studied the method of evaluating probabilities of events relating to
independent events and conditional events. We have also studied about random variables and
their probability distributions, namely binomial distribution and Poisson distribution.
ADVERTISEMENT
The binomial distribution is defined on a random variable which takes finite discrete values
whereas the Poisson distribution is defined on a random variable which takes infinite discrete
values such as 0, 1, 2, 3, ….
There are few more discrete probability distribution which will be discussed in higher classes.
The most important continuous probability distribution in the entire field of statistics is normal
distribution.
This bell shaped probability distribution function is also an approximate to binomial distribution.
We will be learning the importance of these distribution function in later classes.