Viktor Mehandziyski

Probability
365 DATA SCENCE 2

Table of Contents
Abstract .................................................................................................................................... 3
Words of welcome.................................................................................................................. 5
1. The Basics of Probability ................................................................................................... 6
1.1 Expected Values .......................................................................................................... 7
1.2 Probability Frequency Distribution ........................................................................... 8
1.3 Complements ............................................................................................................... 9
2. Combinatorics .................................................................................................................. 10
2.1 Permutations ...............................................................................................................10
2.2 Factorials .....................................................................................................................11
2.3 Variations......................................................................................................................12
2.4 Combinations .............................................................................................................13
2.4.1 Combinations with separate sample spaces ...................................................13
2.4.2 Winning the Lottery ............................................................................................14
2.4.3 Combinations with Repetition ..........................................................................15
2.4.4 Applications of Combination with Repetition..................................................16
2.4.5 Pizza Example...............................,.......................................................................16
2.5 Methodology .............................................................................................................17
2.5.1 Pizza and Sequences ..............................................................................................18
2.5.2 Always Ending in 0 ................................................................................................. 19
2.5.3 Positions ..............................................................................................................20
2.5.4 The Final Step ......................................................................................................20
2.6 Symmetry of Combinations ......................................................................................21
3. Bayesian Notation...............................................................................................................22
3.1 Multiple Events ..............................................................................................................23
3.1.1 Intersection .......................................................................................................23
3.1.2 Union .................................................................................................................24
3.1.3 Mutually Exclusive Sets ....................................................................................24
3.2 Independent and Dependent Events ......................................................................25
3.3 Conditional Probability ..............................................................................................26
3.4 Law of Total Probability .............................................................................................27
365 DATA SCENCE 3

3.5 Additive Law ................................................................................................................28

3.6 The Multiplication Rule ..............................................................................................28
3.7 Bayes’ Law..................................................................................................................29
4. Discrete Distributions ................................................................................................30
4.1 Important Notation for Distribution .....................................................................30
4.2 Types of Distributions ............................................................................................ 31
4.2.1 Discrete Distributions .....................................................................................32
4.2.2 Uniform Distribution .......................................................................................33
4.2.3 Bernoulli Distribution......................................................................................34
4.2.4 Binomial Distribution .....................................................................................35
4.2.5 Poisson Distribution ........................................................................................36
5. Continuous Distribution ...............................................................................................37
5.1 Normal Distribution ................................................................................................38
5.2 Standardizing a Normal Distribution ..................................................................39
5.3 Student T’s Distribution ..........................................................................................40
5.4 Chi-Squared Distribution ......................................................................................40
5.5 Exponential Distribution ........................................................................................41
5.6 Logistic Distribution ...............................................................................................42
5.7 Poisson Distribution: Expected Value and Variance.......................................43
6. Setting Up Wolfram Alpha...........................................................................................46
7. Typing up the integral formula.......................................................................................47
8. Expressing Complex Formulas ..................................................................................48
8.1 Understanding the Solution .................................................................................49
9. Normal Distribution 𝑬 𝒀 , 𝑽𝒂𝒓(𝒀).................................................................................50
365 DATA SCENCE 4

Abstract

Adopting a probabilistic mindset is much more important than knowing “absolute

truths” when it comes to succeeding in data science. Probability is at the heart
of business-driven decisions and success. Therefore, any managers and business
professionals worth their salt will have to be able to understand and compute
probabilistic concepts, in this data-driven business world.

In these Probability Course Notes, we will go over:

• The Basics of Probability
• Combinatorics
• Bayesian Interference

Finally, we go over distributions which are the “heart” of probability applied in data
science. You may have heard of many of them, but this is the only place where you’ll
find detailed information about many of the most common distributions.
• Discrete Distributions : Uniform distribution, Bernoulli distribution, Binomial
distribution (that’s where you’ll see a lot of the combinatorics from the previous
parts), Poisson
• Continuous: Normal distribution, Standard normal distribution, Student’s T, Chi-
Squared, Exponential, Logistic

Keywords: probability, discrete distributions, continuous distributions, Bayesian

interference , combinatorics
365 DATA SCENCE 5

Words of welcome

Dataset
? ML Insight

You are here because you want to comprehend the basics of probability before you
can dive into the world of statistics and machine learning. Understanding the driving
forces behind key statistical features is crucial to reaching your goal of mastering data
science. This way you will be able to extract important insight when analysing data
through supervised machine learning methods like regressions, but also fathom the
outputs unsupervised or assisted ML give you.

Bayesian Inference is a key component heavily used in many fields of mathematics to

succinctly express complicated statements. Through Bayesian Notation we can convey
the relationships between elements, sets and events. Understanding these new
concepts will aid you in interpreting the mathematical intuition behind sophisticated
data analytics methods.

Distributions are the main way we lie to classify sets of data. If a dataset complies with
certain characteristics, we can usually attribute the likelihood of its values to a specific
distribution. Since many of these distributions have elegant relationships between
certain outcomes and their probabilities of occurring, knowing key features of our
data is extremely convenient and useful.
365 DATA SCENCE 6

1. The Basics of Probability

Probability is the likelihood of an event occurring. This event can be pretty much
anything – getting heads, rolling a 4 or even bench pressing 225lbs. We measure
probability with numeric values between 0 and 1, because we like to compare the
relative likelihood of events. Observe the general probability formula.

𝑃𝑟𝑒𝑓𝑒𝑟𝑟𝑒𝑑 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠
P(X)=
𝑆𝑎𝑚𝑝𝑙𝑒 𝑆𝑝𝑎𝑐𝑒

Probability Formula:
• The Probability of event X occurring equals the number of preferred outcomes
over the number of outcomes in the sample space.
• Preferred outcomes are the outcomes we want to occur or the outcomes we are
interested in. We also call refer to such outcomes as “Favorable”.
• Sample space refers to all possible outcomes that can occur. Its “size” indicates the
amount of elements in it.

If two events are independent:

The probability of them occurring simultaneously equals the product of them
occurring on their own.
365 DATA SCENCE 7

1.1 Expected Values

Trial – Observing an event occur and recording the outcome.

Experiment – A collection of one or multiple trials.

Experimental Probability – The probability we assign an event, based on an

experiment we conduct.

Expected value – the specific outcome we expect to occur when we run an

experiment.

Example: Trial Example: Experiment

Flipping a coin and recording the Flipping a coin 20 times and
outcome. recording the 20 individual outcomes.

In this instance, the experimental probability for getting heads would equal the
number of heads we record over the course of the 20 outcomes, over 20 (the total
number of trials).

The expected value can be numerical, Boolean, categorical or other, depending on

the type of the event we are interested in. For instance, the expected value of the trial
would be the more likely of the two outcomes, whereas the expected value of the
experiment will be the number of time we expect to get either heads or tails after the
20 trials.

Expected value for categorical variables. Expected value for numeric variables.
365 DATA SCENCE 8

1.2 Probability Frequency Distribution

What is a probability frequency

distribution?: Sum Frequency Probability
A collection of the probabilities for
2 1 1/36
each possible outcome of an event.
3 2 1/18
Why do we need frequency
distributions?: 4 3 1/12
We need the probability frequency
distribution to try and predict future 5 4 1/9
events when the expected value is
6 5 5/36
unattainable.
7 6 1/6
What is a frequency?:
Frequency is the number of times a 8 5 5/36
given value or outcome appears in
the sample space. 9 4 1/9

What is a frequency distribution

10 3 1/12
table?:
11 2 1/18
The frequency distribution table is a
table matching each distinct outcome 12 1 1/36
in the sample space to its associated
frequency.

How do we obtain the probability

frequency distribution from the
frequency distribution table?:
By dividing every frequency by the
size of the sample space. (Think about
the “favoured over all” formula.)
365 DATA SCENCE 9

1.3 Complements

The complement of an event is everything an event is not. We denote the

complement of an event with an apostrophe.

A’ = Not A

complement original event

opposite

Characteristics of complements:
• Can never occur simultaneously.
• Add up to the sample space. (A + A’ = Sample space)
• Their probabilities add up to 1. (P(A) + P(A’) = 1)
• The complement of a complement is the original event. (A’)’ = A)

Example:
• Assume event A represents drawing a spade, so P(A) = 0.25.
• Then, A’ represents not drawing a spade, so drawing a club, a diamond or a heart.
P(A’) = 1 – P(A), so P(A’) = 0.75.
365 DATA SCENCE 10

2. Combinatorics

2.1 Permutations

Permutations represent the number of different possible ways we can arrange a

number of elements.

𝑃(𝑛) =𝑛×(𝑛−1) × (𝑛−2) ×⋯×1

Options for who

we put last
Permutations
Options for who
we put first Options for who
we put second

Characteristics of Permutations:
• Arranging all elements within the sample space.
• No repetition.
• 𝑃 𝑛 =𝑛× 𝑛−1 × 𝑛−2 ×⋯×1=𝑛! (Called“nfactorial”)

Example:
• If we need to arrange 5 people, we would have P(5) = 120 ways of doing so.
365 DATA SCENCE 11

2.2 Factorials

Factorials express the product of all integers from 1 to n and we denote them with
the “!” symbol.

𝑛!=𝑛× (𝑛−1) × (𝑛−2) ×⋯×1

Key Values:
• 0!=1.
• If n < 0, n! does not exist.

Rules for factorial multiplication. (For n>0 and n>k)

Examples: n = 7, k = 4
365 DATA SCENCE 12

2.3 Variations

Variations represent the number of different possible ways we can pick and arrange a
number of elements.

Variations
with Number of elements
repetition we are arranging
Number of different
elements available

Intuition behind the formula. (With Repetition)

• We have n-many options for the first element.

• We still have n-many options for the second element because repetition is allowed.

• We have n-many options for each of the p- many elements.

Number of elements we
Variations are arranging
without
repetition
Number of different
elements available

Intuition behind the formula. (Without Repetition)

• We have n-many options for the first element.

• We only have (n-1)-many options for the second element because we cannot
repeat the value for we chose to start with.

• We have less options left for each additional element.

•
365 DATA SCENCE 13

2.4 Combinations

Combinations represent the number of different possible ways we can pick a number
of elements.

Combinations

Total number of elements Number of elements we

in the sample space need to select

Characteristics of Combinations:

• Takes into account double-counting. (Selecting Johny, Kate and Marie is the same
as selecting Marie, Kate and Johny)

• All the different permutations of a single combination are different variations.

• Combinations are symmetric, so , since selecting p elements is the

same as omitting n-p elements.

2.4.1 Combinations with Seperate Sample Spaces

Combinations represent the number of different possible ways we can pick a number
of elements.

𝐶 = 𝑛1 × 𝑛2 × ⋯ × 𝑛𝑝

Combinations Size of the last

sample space
Size of the first Size of the second
sample space sample space
Characteristics of Combinations with separate sample spaces:

• The option we choose for any element does not affect the number of options for
the other elements.

• The order in which we pick the individual elements is arbitrary.

• We need to know the size of the sample space for each individual element.
(𝑛1, 𝑛2 ... 𝑛𝑝)
365 DATA SCENCE 14

2.4.2 Winning the Lottery

To win the lottery, you need to satisfy two distinct independent events:

• Correctly guess the “Powerball” number. (From 1 to 26)

• Correctly guess the 5 regular numbers. (From 1 to 69)

69!
𝐶= ×26
64!5!
Total number of
Combinations
𝐶𝑃𝑜𝑤𝑒𝑟𝑏𝑎𝑙𝑙 𝑛𝑢𝑚𝑏𝑒𝑟
𝐶5 𝑛𝑢𝑚𝑏𝑒𝑟𝑠

Intuition behind the formula:

• We consider the two distinct events as a combination of two elements with

different sample sizes.

• One event has a sample size of 26, the other has a sample size of

• Using the “favoured over all” formula, we find the probability of any single ticket

winning equals
365 DATA SCENCE 15

2.4.3 Combinations With Repetition

Combinations represent the number of different possible ways we can pick a number
of elements. In special cases we can have repetition in combinations and for those we
use a different formula.

Combinations with
repetition Number of elements
we need to select

Total number of elements

in the sample space

Now that you know what the formula looks like, we are going to walk you through the
process of deriving this formula from the Combinations without repetition formula.
This way you will be able to fully understand the intuition behind and not have to
bother memorizing it.