Fundamental concepts of probability

Recently, you learned that probability uses math to quantify uncertainty, or to describe the
likelihood of something happening. For example, there might be an 80% chance of rain
tomorrow, or a 20% chance that a certain candidate wins an election.

In this reading, you’ll learn more about fundamental concepts of probability. We’ll discuss
the concept of a random experiment, how to represent and calculate the probability of an
event, and basic probability notation.

Probability fundamentals
Foundational concepts: Random experiment, outcome, event

Let’s begin with three concepts at the foundation of probability theory:

 Random experiment
 Outcome
 Event

Probability deals with what statisticians call random experiments, also known as statistical
experiments. A random experiment is a process whose outcome cannot be predicted with
certainty.

For example, before tossing a coin or rolling a die, you can’t know the result of the toss or the
roll. The result of the coin toss might be heads or tails. The result of the die roll might be 3 or
6.

All random experiments have three things in common:

 The experiment can have more than one possible outcome.

 You can represent each possible outcome in advance.
 The outcome of the experiment depends on chance.

In statistics, the result of a random experiment is called an outcome. For example, if you roll
a die, there are six possible outcomes: 1, 2, 3, 4, 5, 6.

An event is a set of one or more outcomes. Using the example of rolling a die, an event might
be rolling an even number. The event of rolling an even number consists of the outcomes 2,
4, 6. Or, the event of rolling an odd number consists of the outcomes 1, 3, 5.

In a random experiment, an event is assigned a probability. Let’s explore how to represent

and calculate the probability of a random event.

The probability of an event

The probability that an event will occur is expressed as a number between 0 and 1.
Probability can also be expressed as a percent.
  If the probability of an event equals 0, there is a 0% chance that the event will occur.
 If the probability of an event equals 1, there is a 100% chance that the event will
occur.

There are different degrees of probability between 0 and 1. If the probability of an event is
close to zero, say 0.05 or 5%, there is a small chance that the event will occur. If the
probability of an event is close to 1, say 0.95 or 95%, there is a strong chance that the event
will occur. If the probability of an event equals 0.5, there is a 50% chance that the event will
occur—or not occur.

Knowing the probability of an event can help you make informed decisions in situations of
uncertainty. For example, if the chance of rain tomorrow is 0.1 or 10%, you can feel
confident about your plans for an outdoor picnic. However, if the chance of rain is 0.9 or
90%, you may want to think about rescheduling your picnic for another day.

Calculate the probability of an event

To calculate the probability of an event in which all possible outcomes are equally likely, you
divide the number of desired outcomes by the total number of possible outcomes. You may
recall that this is also the formula for classical probability:

# of desired outcomes ÷ total # of possible outcomes

Let’s explore the coin toss and die roll examples to get a better idea of how to calculate the
probability of a single random event.

Example: Coin toss

Tossing a fair coin is a classic example of a random experiment:

 There is more than one possible outcome.

 You can represent each possible outcome in advance: heads or tails.
 The outcome depends on chance. The toss could turn up heads or tails.

Say you want to calculate the probability of getting heads on a single toss. For any given coin
toss, the probability of getting heads is one chance out of two. This is 1 ÷ 2 = 0.5, or 50%.

Now imagine that you were to toss a specially designed coin that had heads on both sides.
Every time you toss this coin it will turn up heads. In this case, the probability of getting
heads is 100%. The probability of getting tails is 0%.

Note that when you say the probability of getting heads is 50%, you aren’t claiming that any
actual sequence of coin tosses will result in exactly 50% heads. For example, if you toss a fair
coin ten times, you may get 4 heads and 6 tails, or 7 heads and 3 tails. However, if you
continue to toss the coin, you can expect the long-run frequency of heads to get closer and
closer to 50%.

Example: Die roll

Rolling a six-sided die is another classic example of a random experiment:

  There is more than one possible outcome.
 You can represent all possible outcomes in advance: 1, 2, 3, 4, 5, and 6.
 The outcome depends on chance. The roll could turn up any number 1–6.

Say you want to calculate the probability of rolling a 3. For any given die roll, the probability
of rolling a 3 is one chance out of six. This is 1 ÷ 6 = 0.1666, or about 16.7%.

Probability notation

It helps to be familiar with probability notation as it’s often used to symbolize concepts in
educational and technical contexts.

In notation, the letter P indicates the probability of an event. The letters A and B represent
individual events.

For example, if you’re dealing with two events, you can label one event A and the other event
B.

 The probability of event A is written as P(A).

 The probability of event B is written as P(B).
 For any event A, 0 ≤ P(A) ≤ 1. In other words, the probability of any event A is
always between 0 and 1.
 If P(A) > P(B), then event A has a higher chance of occurring than event B.
 If P(A) = P(B), then event A and event B are equally likely to occur.

Key takeaways
Data professionals use probability to help stakeholders make informed decisions about
uncertain events. Your knowledge of fundamental concepts of probability will be useful as a
building block for more complex calculations of probability.

Resources for more information

To learn more about fundamental concepts of probability, refer to the following resources:

 These lecture notes from Richland Community College

provide a useful summary of the fundamental concepts and basic rules of probability.
The probability of multiple events
So far, you’ve been learning about calculating the probability of single events. Many
situations, both in daily life and in data work, involve more than one event. As a future data
professional, you’ll often deal with probability for multiple events.

In this reading, you’ll learn more about multiple events. You’ll learn three basic rules of
probability: the complement rule, the addition rule, and the multiplication rule. These rules
help you better understand the probability of multiple events. First, we’ll discuss two
different types of events that these rules apply to: mutually exclusive and independent. Then,
you’ll learn how to calculate probability for both types of events.

Two types of events

The three basic rules of probability apply to different types of events. Both the complement
rule and the addition rule apply to events that are mutually exclusive. The multiplication rule
applies to independent events.

Mutually exclusive events

Two events are mutually exclusive if they cannot occur at the same time.

For example, you can’t be on the Earth and on the moon at the same time, or be sitting down
and standing up at the same time.

Or, take two classic examples of probability theory. If you toss a coin, you cannot get heads
and tails at the same time. If you roll a die, you cannot get a 2 and a 4 at the same time.

Independent events

Two events are independent if the occurrence of one event does not change the probability
of the other event. This means that one event does not affect the outcome of the other event.

For example, watching a movie in the morning does not affect the weather in the afternoon.
Listening to music on the radio does not affect the delivery of your new refrigerator. These
events are separate and independent.

Or, take two consecutive coin tosses or two consecutive die rolls. Getting heads on the first
toss does not affect the outcome of the second toss. For any given coin toss, the probability of
any outcome is always 1 out of 2, or 50%. Getting a 2 on the first roll does not affect the
outcome of the second roll. For any given die roll, the probability of any outcome is always 1
out of 6, or 16.7%.

Three basic rules

Now that you know more about the difference between mutually exclusive and independent
events, let’s review three basic rules of probability:
  Complement rule
 Addition rule
 Multiplication rule

Complement rule

The complement rule deals with mutually exclusive events. In statistics, the complement of
an event is the event not occurring. For example, either it snows or it does not snow. Either
your soccer team wins the championship or it does not win the championship. The
complement of snow is no snow. The complement of winning is not winning.

The probability of an event occuring and the probability of it not occuring must add up to 1.
Recall that a probability of 1 is the same as a 100%.

Another way to think about it is that there is a 100% chance of one event or the other event
occurring. There may be a 40% chance of snow tomorrow. However, there is a 100% chance
that it will either snow or not snow tomorrow.

The complement rule states that the probability that event A does not occur is 1 minus the
probability of A. In probability notation, you can write this as:

Complement rule

P(A’) = 1 - P(A)

Note: In probability notation, an apostrophe (‘) symbolizes negation. In other words, if you
want to indicate the probability of event A NOT occurring, add an apostrophe after the letter
A: P(A’). You can say this as “the probability of not A.”

So, if you know there is a 40% chance of snow tomorrow, or a probability of 0.4, you can use
the complement rule to calculate the probability that it does not snow tomorrow. The
probability of no snow equals one minus the probability of snow.

P(no snow) = 1 - P(snow) = 1 - 0.4 = 0.6.

So, the probability of no snow tomorrow is 0.6, or 60%.

Addition rule (for mutually exclusive events)

The addition rule states that if events A and B are mutually exclusive, then the probability of
A or B occuring is the sum of the probabilities of A and B. In probability notation, you can
write this as:

P(A or B) = P(A) + P(B)

Note that there is also an addition rule for mutually inclusive events. In this course, we focus
on the rule for mutually exclusive events.

Let’s explore our example of rolling a die.

Die roll (rolling either a 2 or a 4)

Say you want to find the probability of rolling either a 2 or a 4 on a single roll. These two
events are mutually exclusive. You can roll a 2 or a 4, but not both at the same time.

The addition rule says that to find the probability of either event occurring, you sum up their
probabilities. The odds of rolling any single number on a die are 1 out of 6, or 16.7%.

P(rolling 2 or rolling 4) = P(rolling 2) + P(rolling 4) = ⅙ + ⅙ = ⅓

So, the probability of rolling either a 2 or a 4 is one out of three, or 33%.

Multiplication rule (for independent events)

The multiplication rule states that if events A and B are independent, then the probability of
both A and B occuring is the probability of A multiplied by the probability of B. In
probability notation, you can write this as:

P(A and B) = P(A)×P(B)

Note that there is also a multiplication rule for dependent events. In this course, we focus on
the rule for independent events.

Let’s continue with our example of rolling a die.

Die roll (rolling a 1 and then rolling a 6)

Now imagine two consecutive die rolls. Say you want to know the probability of rolling a 1
and then rolling a 6. These are independent events as the first roll does not affect the outcome
of the second roll.

The probability of rolling a 1 and then a 6 is the probability of rolling a 1 multiplied by the
probability of rolling a 6. The probability of each event is ⅙, or 16.7%. You can write this
as:

P(rolling 1 on the first roll and rolling 6 on the second roll) = P(rolling 1 on the first
roll)×P(rolling 6 on the second roll) = ⅙×⅙ = 1/36

So, the probability of rolling a 1 and then a 6 is one out of thirty-six, or about 2.8%.

Key takeaways
The basic rules of probability help you describe events that are mutually exclusive or
independent. Understanding basic rules of probability is an essential foundation for more
complex analyses you will perform as a future data professional.

Resources for more information

To learn more about the basic rules of probability, refer to the following resources:
  This online textbook from the University of Florida

provides a detailed overview of the basic rules of probability from a more technical
perspective.

Objective versus subjective probability

Probability helps you measure and quantify uncertainty and make informed decisions about
uncertain outcomes. For example, you might use probability to decide what to wear on a
given day. Today's weather forecast says there’s a 70 percent chance of snow. Based on this
data, you decide to wear your hat, gloves, and snow boots. When the snow falls, you stay
warm and dry. Data professionals might use probability to predict the chances that a
company will sell a certain amount of product in a given time period, a financial investment
will have a positive return, a political candidate will win an election, or a medical test will
be accurate. In this video, we'll explore the two main types of probability: objective and
subjective.
Objective probability is based on statistics, experiments, and mathematical measurements.
Subjective probability is based on personal feelings, experience, or judgment. Let's start with
objective probability. Data professionals use objective probability to analyze and interpret
data.
There are two types of objective probability: classical and empirical.
Classical probability is based on formal reasoning about events with equally likely
outcomes. To calculate classical probability for an event, you divide the number of desired
outcomes by the total number of possible outcomes. For example, if you flip a coin, the
result will be either heads or tails. Heads and tails are terms commonly used to refer the two
sides of the coin. There are only two possible outcomes, and both outcomes are equally
likely. The chance that you get heads is one out of two or 50 percent, the same goes for tails.
Or take playing cards. There are 52 cards in a standard deck. Choosing a card gives you a
one-in-52 chance, or 1.9 percent of getting any card in the deck, whether it's the ace of hearts,
10 of clubs, or four of spades. But most events are more complex and do not have equally
likely outcomes. Usually, the weather isn’t a 50 percent chance of rain or snow; there might
be an 80 percent chance of rain tomorrow and a 20 percent chance of some other outcome.
While classical probability applies to events with equally likely outcomes, data professionals
use empirical probability to describe more complex events.
Empirical probability is based on experimental or historical data; it represents the likelihood
of an event occurring based on the previous results of an experiment or past events. To
calculate empirical probability, you divide the number of times a specific event occurs by the
total number of events. For example, say you conduct a taste test with 100 people to find out
whether they prefer strawberry or mint chip-flavored ice cream. You want to know the
probability that a person prefers strawberry ice cream. Your taste test reveals that 80 people
prefer strawberry ice cream. To calculate probability, you divide the number of times the
event of preferring strawberry ice cream occurs, 80, by the total number of events, 100. 80
divided by 100 equals 0.8 or 80 percent. So the probability that a person prefers strawberry
over mint chip is 80 percent.
Earlier, you learned about inferential statistics and how data professionals use sample
data to make inferences or predictions about larger populations.
Inferential statistics uses probability too. For instance, a retail company might survey a
representative sample of 100 customers to predict the shopping preferences of all their
customers. Data professionals rely on empirical probability to help them make accurate
predictions based on sample data. For example, in an A/B test of a website, you test a sample
of users to make a prediction about the future behavior of all users. Say the sample of users
prefer a green add-to-cart button over a blue one. You may infer from this data that the larger
population of future users will probably share their preference. An A/B test lets you make a
reasonable prediction about future users based on empirical probability. This probability can
help an online business make smarter decisions and increase sales.
In contrast, the results of subjective probability are based on personal feeling, experience, or
judgment. This type of probability does not involve formal calculations, statistical analysis,
or scientific experiments. For instance, you may have an overwhelming feeling that a certain
horse will win a horse race, or that your favorite team will win the championship game.
You may have good reasons for your belief, but your reasons are personal or subjective.
Your belief is not based on statistical analysis or scientific experiments. For this reason, the
subjective probability of an event may differ widely from person to person.
It's important to know the difference between subjective and objective probability when you
evaluate a prediction or make a decision. For example, the CEO of an auto company might
feel confident that using a new technology to manufacture their pickup truck will cut costs
and increase profits. But if their prediction is only based on personal feeling or subjective
probability, it may not be reliable. Data science based on statistical analysis or objective
probability can help accurately predict the potential impact of the new technology and help
the CEO make an informed, data-driven decision about adopting the technology.
The principles of probability
Recently, you learned that probability uses math to deal with uncertainty or to determine how
likely it is that an event will occur. In this video, you'll learn some fundamental concepts of
probability. We'll discuss the mathematical definition of probability and how to calculate
probability for single random events.
First, I want to give you some context about the types of examples we'll be using. In this part
of the course, we're going to continue to reference examples of events like flipping coins,
rolling dice, and drawing cards. There are a couple of reasons for this. One is historical. The
modern theory of probability originates in the analysis of games, of chance in the 16th and
17th centuries.
Second, and more importantly, these are events with clearly defined outcomes that most
people are familiar with. They're just super useful examples of basic probability concepts.
That's why they're used in stats classes around the world.
Later on in the course, we will explore probability of for more complex events like the ones
you'll encounter in your future work as a data professional. Let's talk about the fundamental
concepts of probability.
First, the probability that an event will occur is expressed as a number between 0 and 1. If the
probability of an event equals zero, there's a zero percent chance that the event will occur. If
the probability of an event equals one, there's a 100 percent chance that the event will occur.
There are lots of possibilities in between 0 and 1. If the probability of an event equals 0.5,
there is a 50 percent chance that the event will occur or not occur. If the probability of an
event is close to zero, there's a small chance that the event will occur. If the probability of an
event is close to one, there's a strong chance that the event will occur. For example, the
chance of a stock price going up this year is 0.05 or five percent, then you probably don't
want to buy it. If it's 0.95 or 95 percent, then it's probably a good investment. Probability
measures the likelihood of random events. The result of a random event cannot be predicted
with certainty. Before flipping a coin or rolling a die, you do not know the outcome. The coin
could turn up heads or tails, and a die could show any number one through six. These are
examples of what statisticians call a random experiment, also known as a statistical
experiment.
A random experiment is a process whose outcome cannot be predicted with certainty. All
random experiments have three things in common. The experiment can have more than one
possible outcome, you can represent each possible outcome in advance, and the outcome of
the experiment depends on chance. Let's take the example of flipping a coin. There's more
than one possible outcome. You can represent each possible outcome and advance heads or
tails, and the outcome depends on chance. Until you actually toss the coin, you can't know
whether it will be heads or tails, or think about rolling a six-sided die. There's more than one
possible outcome, and all outcomes can be represented in advance, 1, 2, 3, 4, 5, and 6. The
outcome of any roll depends on chance. Until you roll the die, you can't know which number
will turn up. To calculate the probability of a random experiment, you divide the number of
desired outcomes by the total number of possible outcomes. You may recall that this is also
the formula for classical probability. The probability of tossing a coin and getting heads is
one chance in two. This is 1 divided 2 equals 0.5 or 50 percent. The probability of rolling a
die and getting two, is one chance out of six, this is 1 divided 6 equals 0.166 repeating or
about 16.7 percent. Now, let's conduct a different random experiment. Imagine a jar contains
10 marbles, two marbles are red, three are green, and five are blue. You decide to take one
marble from the jar. You want to know the probability that the marble will be green. First,
count the number of possible outcomes. You have an equal chance of choosing any one of the
10 marbles. Next, figure out how many of these outcomes refer to what you want to know.
The chance of choosing a green marble. Of the 10 total marbles, three are green. Therefore,
the probability of choosing a green marble is 3 out of 10, or 0.3. In other words, you have a
30 percent chance of choosing a green marble. Now you know how to calculate the
probability of a single random event. This knowledge will be useful as a building block for
more complex calculations of probability.

The basic rules of probability and events

So far, we've been focusing on calculating the probability of single events. Many situations,
both in everyday life and in data analytics, involve more than one event. As a future data
professional, you'll often deal with probability for multiple events. In this video, we'll cover
three basic rules of probability:
the complement rule,
the addition rule,
and the multiplication rule.
These rules help you better understand the probability of multiple events. We'll also discuss
two different types of events: mutually exclusive events and independent events. Then you'll
learn how to calculate probability for each of them. First, let's discuss probability notation,
which is the standard way to symbolize probability concepts. As we go along, I'll share some
useful notations that will help us communicate more efficiently when it comes to basic
probability.
The letter P indicates the probability of an event. For example, if you're dealing with two
events, you can label one event A, and the other event B. The notation for the probability of
event A is the letter P followed by the letter A in parenthesis. For the probability of event B,
it's the letter P, followed by the letter B in parenthesis. If you want to talk about the
probability of event A not occurring, add an apostrophe after the letter A. You can also say
this is the probability of not A.
Now, let's check out our first basic rule, the complement rule. In stats, the complement of an
event is the event not occurring. For example, either it rains or it does not rain. Either you
win the lottery or you don't win the lottery. The complement of rain is no rain. The
complement of winning is not winning. The important thing to note is that the two
probabilities, the probability of an event happening and the probability of it not happening,
must add to one. Recall that a probability of one is the same as saying there's 100 percent
certainty of an event occurring. Another way to think about it is that there is a 100 percent
chance of one event or the other event happening. There may be a 30 percent chance of rain
tomorrow, but there is a 100 percent chance that it will either rain or not rain tomorrow.
The complement rule says that the probability that event A does not occur is 1 minus the
probability of event A. For example, if the weather forecast says there's a 30 percent chance
of rain tomorrow, there's a probability of 0.3. You can use the complement rule to calculate
the probability that it does not rain tomorrow. The probability of no rain equals 1 minus the
probability of rain. This is 1 minus 0.3 equals 0.7 or 70 percent.
But the complement rule and our next rule, the addition rule, applied two events that are
mutually exclusive. Two events are mutually exclusive if they cannot occur at the same time.
For example, you can't visit both Argentina and China at the same time, or turn left and right
at the same time.
The addition rule says that if the events A and B are mutually exclusive, then the probability
of A or B happening is the sum of the probabilities of A and B. Let's check out an example
using a six-sided die. Say you want to find out the probability of rolling either a two or a four
on a single roll of the die. These two events are mutually exclusive. You can roll a two or a
four, but not both at the same time. The addition rule says that to find the probability of either
event happening, you should sum up their probabilities. The odds of rolling any single
number on a six-sided die are 1/6. The probability of rolling a two is 1/6, and the probability
of rolling a four is 1/6. 1/6 plus 1/6 equals 1/3. The probability of rolling either a two or a
four is 1/3 or 33 percent. The addition rule applies to mutually exclusive events.
If you want to calculate probability for independent events, you can use the multiplication
rule. Two events are independent if the occurrence of one event does not change the
probability of the other event. This means that one event does not affect the outcome of the
other event. For example, checking out a book from your local library does not affect
tomorrow's weather. Drinking coffee in the morning does not affect the delivery of your mail
in the afternoon. These events are separate and independent. The multiplication rule says that
if the events A and B are independent, then the probability of both A and B happening is the
probability of A multiplied by the probability of B. For instance, imagine two consecutive
tosses. Say you want to know the probability of tails on the first toss and heads on the second
toss. First, figure out what events you're dealing with and then apply the appropriate rule.
Two coin tosses are independent events. The first toss does not affect the outcome of the
second toss. For any toss, the probability of getting either heads or tails always remains 1/2 or
50 percent. You would use the multiplication rule for this event. The probability of getting
tails and heads is the probability of getting tails, multiply it by the probability of getting
heads. The probability of each event is 0.5 or 50 percent. Now, plug in the numbers, 0.5 times
0.5 equals 0.25 or 25 percent. The probability of getting tails on the first toss and heads on
the second toss is 25 percent.
To recap, let's compare the addition and multiplication rules and list their differences. It will
be helpful to keep these differences in mind, so you know when to use the two rules.
The addition rule sums up the probabilities of events, and the multiplication rule multiplies
the probabilities.
The addition rule applies to events that are mutually exclusive.
The multiplication rule applies to events that are independent.
The basic rules of probability help you describe events that are mutually exclusive or
independent.