Probability

Probability

• Probability defines the likelihood of occurrence of an event.

• There are many real-life situations in which we may have to predict the outcome
of an event.
• We may be sure or not sure of the results of an event.
• In such cases, we say that there is a probability of this event to occur or not occur.
• Probability generally has great applications in games, in business to make
probability-based predictions, and also probability has extensive applications in
the new area of artificial intelligence.
• Formula for Probability
• The probability formula is defined as the possibility of an event to happen is
equal to the ratio of the number of favourable outcomes and the total number of
outcomes.
Probability of event to happen P(E) = Number of favourable outcomes/Total Number of outcomes
Terminology of Probability Theory

• Experiment: A trial or an operation conducted to produce an outcome is called an

experiment.
• Sample Space: All the possible outcomes of an experiment together constitute a
sample space. For example, when tossing a coin, the set of possible outcomes is
{heads, tails}. Or when rolling a single die, the set of possible outcomes is {1, 2, 3,
4, 5, 6}.
• Favorable Outcome: An event that has produced the desired result or expected
event is called a favorable outcome. For example, when we roll two dice, the
possible/favorable outcomes of getting the sum of numbers on the two dice as 4
are (1,3), (2,2), and (3,1).
• Trial: A trial denotes doing a random experiment.
• Random Experiment: An experiment that has a well-defined set of outcomes is
called a random experiment. For example, when we toss a coin, we know that we
would get ahead or tail, but we are not sure which one will appear.
Terminology of Probability Theory

• Event: The total number of outcomes of a random experiment is called an event.

• Types of events:
•Exhaustive Events
•Mutually Exclusive Events
•Equally Likely Events
•Complimentary events
•Independent and Dependent events

• Equally Likely Events:

Events that have the same chances or probability of occurring are called
equally likely events. The outcome of one event is independent of the other.
For example, when we toss a coin, there are equal chances of getting a head or
a tail.
Mutually exclusive events

• Mutually exclusive events In probability theory, two events are said to be

mutually exclusive if they cannot occur at the same time or simultaneously.
• For example, when a coin is tossed then the result will be either head or tail, but
we cannot get both the results.
• Such events are also called disjoint events since they do not happen
simultaneously.
• If A and B are mutually exclusive events then its probability is given by P(A Or B)
or P (A U B).
• If A and B are the two events, then the probability of disjoint of event A and B is
written by:
• Probability of Disjoint (or) Mutually Exclusive Event =
P ( A and B) = 0
Real-life Examples on Mutually Exclusive Events

• Some of the examples of the mutually exclusive events are:

• When tossing a coin, the event of getting head and tail are mutually exclusive.
Because the probability of getting head and tail simultaneously is 0.
• In a six-sided die, the events “2” and “5” are mutually exclusive. We cannot get
both the events 2 and 5 at the same time when we threw one die.
• In a deck of 52 cards, drawing a red card and drawing a club are mutually
exclusive events because all the clubs are black.
 Dependent and Independent Events

• Two events are said to be dependent if the occurrence of one event changes the
probability of another event.
• Two events are said to be independent events if the probability of one event does not
affect the probability of another event.
• If two events are mutually exclusive, they are not independent. Also, independent events
cannot be mutually exclusive.
• Independent Event: are those events whose occurrence is not dependent on any other
event. For example, if we flip a coin in the air and get the outcome as Head, then again if
we flip the coin but this time we get the outcome as Tail. In both cases, the occurrence of
both events is independent of each other.
• If the probability of occurrence of an event A is not affected by the occurrence of another
event B, then A and B are said to be independent events.
• Consider an example of rolling a die. If A is the event ‘the number appearing is odd’ and B
be the event ‘the number appearing is a multiple of 3’, then
• P(A)= 3/6 = 1/2 and P(B) = 2/6 = 1/3
 Complementary events
• Two events are said to be complementary when one event occurs if and only if the
other does not.
• The probabilities of two complimentary events add up to 1.
• For example, rolling a 5 or greater and rolling a 4 or less on a die are complementary
events, because a roll is 5 or greater if and only if it is not 4 or less.
• The probability of rolling a 5 or greater 2/6=1/3,
• and the probability of rolling a 4 or less is 4/6=2/3,
• Thus, the total of their probabilities is 1/3+2/3 =3/3 =1
 Exhaustive Events

• Exhaustive events are a set of events in a sample space such that one of them compulsorily occurs while
performing the experiment.
• In simple words, we can say that all the possible events in a sample space of an experiment constitute
exhaustive events. For example, while tossing an unbiased coin, there are two possible outcomes - heads or
tails. So, these two outcomes are exhaustive events as one of them will definitely occur while flipping the
coin.
• Let us consider the experiment of throwing a die.
• Sample space S = {1, 2, 3, 4, 5, 6}
• Assume that A, B and C are the events associated with this experiment. Also, let us define these events as:
• A be the event of getting a number greater than 3
• B be the event of getting a number greater than 2 but less than 5
• C be the event of getting a number less than 3
• We can write these events as:
• A = {4, 5, 6}; B = {3, 4} and C = {1, 2}
• We observe that
• A ⋃ B ⋃ C = {4, 5, 6} ⋃ {3, 4} ⋃ {1, 2} = {1, 2, 3, 4, 5, 6} = S
• Therefore, A, B, and C are called exhaustive events.
• However, the probability of exhaustive events is equal to 1.
Classical Approach or a priori Probability
• Classical Approach. . . If an experiment has n simple outcomes, this method
would assign a probability of 1/n to each outcome. In other words, each outcome
is assumed to have an equal probability of occurrence.
• Classical Probability Examples
• Example 1: The typical example of classical probability would be rolling of a fair
dice because it is equally probable that top face of die will be any of the 6
numbers on the die: 1, 2, 3, 4, 5, or 6.
• Example 2: Another example of classical probability would be tossing an unbiased
coin. There is an equal probability that your toss will yield either head or tail.
• Example 3: In selecting bingo balls, each numbered ball has an equal chance of
being chosen.
 Empirical Probability or Relative Frequency Probability Theory

• The Relative Frequency Probability Theory is as follows:

• We can define the probability of an event as the relative frequency with which it
occurs in an indefinitely large number of trials. Therefore, if an event occurs ‘a’ times
out of ‘n’, then its relative frequency is a/n
• Further, the value that a/n approaches when ‘n’ becomes infinity is the limit of the
relative frequency.
• Symbolically,
P(A) = limn→∞a/n
• However, in practice, we write the estimate of P(A) as follows: P(A) = a/n
• While the classical probability is normally encountered in problems dealing with
games of chance. On the other hand, the empirical probability is the probability
derived from past experience and is used in many practical problems.
 Subjective Approach

• In the subjective approach, we define probability as the degree of belief that we hold in
the occurrence of an event.
• Thus, judgment is used as the basis for assigning probabilities. Notice that the classical
approach of assigning equal probabilities to simple events is, in fact, also based on
judgment.
• What is somewhat different here is that the use of the subjective approach is usually
limited to experiments that are unrepeatable.
• Example 1: Horse Race Consider a horse race with 8 horses running. What is the
probability for a particular horse to win? Is it reasonable to assume that the probability is
1/8? Note that we can’t apply the relative-frequency approach.
• People regularly place bets on the outcomes of such “onetime” experiments based on
their judgment as to how likely it is for a particular horse to win.
• Indeed, having different judgments is what makes betting possible
Axiomatic Probability
• Axiomatic Probability - Axiomatic Probability is a theory of unifying probability
where there is an application of a set of rules made by Kolmogorov
• the three axioms are:
1.The probability of an event A is always greater or equal to zero but can never be
less than zero.
2.If S is a sample space then the probability of occurrence of sample space is always
1. That is, if the experiment is performed then it is sure to get one of the sample
spaces.
3.For mutually exclusive events, the probability of either of the events happening is
the sum of the probability of both the events happening.
Coin Toss Probability

• Tossing a Coin
• A single coin on tossing has two outcomes, a head, and a tail. The concept of probability
which is the ratio of favorable outcomes to the total number of outcomes can be used
to find the probability of getting the head and the probability of getting a tail.
• Total number of possible outcomes = 2; Sample Space = {H, T}; H: Head, T: Tail
• P(H) = Number of heads/Total outcomes = 1/2
• P(T)= Number of Tails/ Total outcomes = 1/2
• Tossing Two Coins
• In the process of tossing two coins, we have a total of four outcomes. The probability
formula can be used to find the probability of two heads, one head, no head, and a
similar probability can be calculated for the number of tails. The probability calculations
for the two heads are as follows.
• Total number of outcomes = 4; Sample Space = {(H, H), (H, T), (T, H), (T, T)}
• P(2H) = P(0 T) = Number of outcome with two heads/Total Outcomes = 1/4
• P(1H) = P(1T) = Number of outcomes with only one head/Total Outcomes = 2/4 = 1/2
• P(0H) = (2T) = Number of outcome with two heads/Total Outcomes = 1/4
Coin Toss Probability

• Tossing Three Coins

• The number of total outcomes on tossing three coins simultaneously is equal
to 23 = 8. For these outcomes, we can find the probability of getting one head,
two heads, three heads, and no head. A similar probability can also be
calculated for the number of tails.
• Total number of outcomes = 23 = 8 Sample Space = {(H, H, H), (H, H, T), (H, T,
H), (T, H, H), (T, T, H), (T, H, T), (H, T, T), (T, T, T)}
• P(0H) = P(3T) = Number of outcomes with no heads/Total Outcomes = 1/8
• P(1H) = P(2T) = Number of Outcomes with one head/Total Outcomes = 3/8
• P(2H) = P(1T) = Number of outcomes with two heads /Total Outcomes = 3/8
• P(3H) = P(0T) = Number of outcomes with three heads/Total Outcomes = 1/8
Dice Roll Probability

• Rolling One Dice

• The total number of outcomes on rolling a die is 6, and the
sample space is {1, 2, 3, 4, 5, 6}. Here we shall compute the
following few probabilities to help in better understanding the
concept of probability on rolling one dice.
• P(Even Number) = Number of even number outcomes/Total
Outcomes = 3/6 = 1/2
• P(Odd Number) = Number of odd number outcomes/Total
Outcomes = 3/6 = 1/2
• P(Prime Number) = Number of prime number outcomes/Total
Outcomes = 3/6 = 1/2
• Rolling Two Dice
• The total number of outcomes on rolling two dice is 62 = 36. The
following image shows the sample space of 36 outcomes on
rolling two dice.
• Let us check a few probabilities of the outcomes from
two dice. The probabilities are as follows.
• Probability of getting a doublet(Same number) = 6/36 =
1/6
• Probability of getting a number 3 on at least one dice =
11/36
• Probability of getting a sum of 7 = 6/36 = 1/6
• As we see, when we roll a single die, there are 6
possibilities. When we roll two dice, there are 36
possibilities. When we roll 3 dice we get 216
possibilities. So a general formula to represent the
n
Probability of Drawing Cards

• A deck containing 52 cards is grouped into four suits of

clubs, diamonds, hearts, and spades. Each of the clubs,
diamonds, hearts, and spades have 13 cards each,
which sum up to 52. Now let us discuss the probability
of drawing cards from a pack. The symbols on the cards
are shown below. Spades and clubs are black cards.
Hearts and diamonds are red cards.
The 13 cards in each suit are ace, 2, 3, 4, 5, 6, 7, 8, 9, 10, jack, queen, king. In these, the jack,
the queen, and the king are called face cards.
We can understand the card probability from the following examples.
•The probability of drawing a black card is P(Black card) = 26/52 = 1/2
•The probability of drawing a hearts card is P(Hearts) = 13/52 = 1/4
•The probability of drawing a face card is P(Face card) = 12/52 = 3/13
•The probability of drawing a card numbered 4 is P(4) = 4/52 = 1/13
•The probability of drawing a red card numbered 4 is P(4 Red) = 2/52 = 1/26
• Union/Addition
• Let A and B be two events.
• Then, P(A ∪ B) = P(A) + P(B)−P(A ∩ B) .
• The subtraction of P(A ∩ B) is necessary because A and B may
“overlap.”
• If A and B are mutually exclusive, i.e., A ∩ B = ∅,
• then P(A ∪ B) = P(A) + P(B) .
Addition Rule for Probabilities

• The addition rule for probabilities consists of two rules or formulas, with one that
accommodates two mutually-exclusive events and another that accommodates
two non-mutually exclusive events.
• Addition Rule for Disjoint Events
• Suppose A and B are disjoint, their intersection is empty. Then the probability of
their intersection is zero.
• In symbols: P(A∩B)=0P(A∩B)=0. The addition law then simplifies to:
• P(A∪B)=P(A)+P(B) when A∩B=∅
• The symbol ∅ represents the empty set, which indicates that in this
case A and B do not have any elements in common (they do not overlap).
• The addition law of probability (sometimes referred to as the addition rule or
sum rule), states that the probability that A or B will occur is the sum of the
probabilities that A will happen and that B will happen, minus the probability that
both A and B will happen. The addition rule is summarized by the formula:
• P(A∪B)=P(A)+P(B)−P(A∩B)
• Suppose a card is drawn from a deck of 52 playing
cards: what is the probability of getting a king or a
queen?
• Let A represent the event that a king is drawn
and B represent the event that a queen is drawn.
• These two events are disjoint, since there are no kings
that are also queens.
• Thus:
• P(A∪B)=P(A)+P(B)
• =4/52+4/52=8/52=2/13
• When drawing one card out of a deck of 52 playing cards,
what is the probability of getting heart or a face card (king,
queen, or jack)? Let H denote drawing a heart and F denote
drawing a face card.
• Since there are 13 hearts and a total of 12 face cards (3 of
each suit: spades, hearts, diamonds and clubs), but
only 3 face cards of hearts, we obtain:
• P(H)=13/52
• P(F)=12/52
• P(F∩H)=3/52
• Using the addition rule, we get:
• P(H∪F)=P(H)+P(F)−P(H∩F)=13/52+12/52−3/52
Multiplication Rule in Probability-independent events

• If A and B are independent

events then
• P(A ∩ B) = P(A) × P(B).
• In words ‘The probability of
independent events A and B
occurring is the product of the
probabilities of the events
occurring separately.’
 Multiplication Rule in Probability- Dependent events
• In probability theory, the Multiplication Rule states that the probability that A and B occur
is equal to the probability that A occurs times the conditional probability that B occurs,
given that we know A has already occurred. This rule can be written:
• P(A∩B)=P(B)⋅P(A|B)
• Switching the role of AA and BB, we can also write the rule as:
• P(A∩B)=P(A)⋅P(B|A)
• We obtain the general multiplication rule by multiplying both sides of the definition of
conditional probability by the denominator.
• That is, in the equation P(A|B)=P(A∩B)/P(B) if we multiply both sides by P(B), we obtain
the Multiplication Rule.
• The rule is useful when we know both P(B) and P(A|B), or both P(A) and P(B|A).
Conditional Probability
• Let A and B be two events.
• Then, the conditional probability of A given that B has occurred,
• P(A | B), is defined as:
• P(A | B) = P(A ∩ B) P(B) .
• You have a cowboy hat, a top hat, and an Indonesian hat called a songkok. You
also have four shirts: white, black, green, and pink. If you choose one hat and one
shirt at random, what is the probability that you choose the songkok and the
black shirt?
• The two events are independent events; the choice of hat has no effect on the
choice of shirt.
• There are three different hats, so the probability of choosing the songkok is 1/3,
There are four different shirts, so the probability of choosing the black shirt is ¼
• So, by the Multiplication Rule:
• P(songok and black shirt)= 1/3 X 1/ 4 = 1/12
• Suppose you take out two cards from a standard pack of cards one after another,
without replacing the first card. What is probability that the first card is the ace of
spades, and the second card is a heart?
• The two events are dependent events because the first card is not replaced.
• There is only one ace of spades in a deck of 52 cards, so
• P(1st card is the ace of spades)= 1/52
• If the ace of spaces is drawn first, then there are 51 ards left in the deck, of which
13 are hearts:
• P(2nd card is a heart | 1st cardis the ace of spades)= 13/51
• So, by the multiplication rule of probability, we have:
• P(ace of spades, then a heart)= 1/52 X 13/51
• A statistics course has seven male and three female students. The professor
wants to select two students at random to help her conduct a research project.
What is the probability that the two students chosen are female?
• A = the first student selected is female
• B = the second student selected is female
• A ∩ B = both chosen students are female
• P(A) = 3/10;
• P(B | A) = 2/9
• Hence, P(A ∩ B) = P(B | A)P(A) = 2/ 9 · 3/ 10 = 1 /15