22AMH32 – DATA ANALYTICS AND

DATA SCIENCE

UNIT I & PROBABILITY

DISTRIBUTIONS AND FITTING A
MODEL

1. PROBABILITY DISTRIBUTIONS AND FITTING A MODEL

A probability distribution is a mathematical function that describes the probability of
different possible values of a variable. Probability distributions are often depicted using
graphs or probability tables.
Example: Probability distribution
We can describe the probability distribution of one coin flip using a probability table:
Outcom Probability
e
Heads Tails
.5 .5
1.1 What is a probability distribution?
A probability distribution is an idealized frequency distribution.
A frequency distribution describes a specific sample or dataset. It’s the number of times each
possible value of a variable occurs in the dataset.
The number of times a value occurs in a sample is determined by its probability of
occurrence. Probability is a number between 0 and 1 that says how likely something is to
occur:
 0 means it’s impossible.
 1 means it’s certain.
The higher the probability of a value, the higher its frequency in a sample.
More specifically, the probability of a value is its relative frequency in an infinitely large
sample.
Infinitely large samples are impossible in real life, so probability distributions are theoretical.
They’re idealized versions of frequency distributions that aim to describe the population the
sample was drawn from.
Probability distributions are used to describe the populations of real-life variables, like coin
tosses or the weight of chicken eggs. They’re also used in hypothesis testing to determine p
values.

Example: Probability distributions are idealized frequency distributions

Imagine that an egg farmer wants to know the probability of an egg from her farm being a
certain size.
The farmer weighs 100 random eggs and describes their frequency distribution using a
histogram:

Figure 1: Frequency_distribution_example_egg_weight

She can get a rough idea of the probability of different egg sizes directly from this frequency
distribution. For example, she can see that there’s a high probability of an egg being around
1.9 oz., and there’s a low probability of an egg being bigger than 2.1 oz.
Suppose the farmer wants more precise probability estimates. One option is to improve her
estimates by weighing many more eggs.
A better option is to recognize that egg size appears to follow a common probability
distribution called a normal distribution. The farmer can make an idealized version of the egg
weight distribution by assuming the weights are normally distributed:

Figure 2: Normal_distribution_example_egg_weight
Since normal distributions are well understood by statisticians, the farmer can calculate
precise probability estimates, even with a relatively small sample size.
Variables that follow a probability distribution are called random variables. There’s special
notation you can use to say that a random variable follows a specific distribution:
 Random variables are usually denoted by X.
 The ~ (tilde) symbol means “follows the distribution.”
 The distribution is denoted by a capital letter (usually the first letter of the
distribution’s name), followed by brackets that contain the distribution’s parameters.
For example, the following notation means “the random variable X follows a normal
distribution with a mean of µ and a variance of σ2.”

There are two types of probability distributions:

 Discrete probability distributions

 Continuous probability distributions

2.1 Discrete probability distributions

A discrete probability distribution is a probability distribution of a categorical or discrete
variable.
Discrete probability distributions only include the probabilities of values that are possible. In
other words, a discrete probability distribution doesn’t include any values with a probability
of zero. For example, a probability distribution of dice rolls doesn’t include 2.5 since it’s not
a possible outcome of dice rolls.
The probability of all possible values in a discrete probability distribution add up to one. It’s
certain (i.e., a probability of one) that an observation will have one of the possible values.

2.1.1 Probability tables

A probability table represents the discrete probability distribution of a categorical variable.
Probability tables can also represent a discrete variable with only a few possible values or a
continuous variable that’s been grouped into class intervals.
A probability table is composed of two columns:

 The values or class intervals

 Their probabilities
A robot greets people using a random greeting. The probability distribution of the greetings is
described by the following probability table:
Greeting Probability

“Greetings, human!” .6

“Hi!” .1

“Salutations, organic life-form.” .2

“Howdy!” .1

Table 1: Probability table

Notice that all the probabilities are greater than zero and that they sum to one.
2.1.2 Probability mass functions
A probability mass function (PMF) is a mathematical function that describes
a discrete probability distribution. It gives the probability of every possible value of a
variable.
A probability mass function can be represented as an equation or as a graph.
Example: Probability mass function
Imagine that the number of sweaters owned per person in the United States follows a Poisson
distribution.
The probability mass function of the distribution is given by the formula:

Where:

 is the probability that a person has exactly sweaters

 is the mean number of sweaters per person ( , in this case)
 is Euler’s constant (approximately 2.718)
This probability mass function can also be represented as a graph:

Figure 3: Probability mass function

Notice that the variable can only have certain values, which are represented by closed circles.
You can have two sweaters or 10 sweaters, but you can’t have 3.8 sweaters.
The probability that a person owns zero sweaters is .05, the probability that they own one
sweater is .15, and so on. If you add together all the probabilities for every possible number
of sweaters a person can own, it will equal exactly 1.
Common discrete probability distributions
Distribution Description Example

Binomial Describes variables with two possible The number of times a

outcomes. It’s the probability distribution of the coin lands on heads
number of successes in n trials when you toss it five
with p probability of success. times
Discrete Describes events that have equal probabilities. The suit of a randomly
uniform drawn playing card

Poisson Describes count data. It gives the probability of The number of text
an event happening k number of times within a messages received per
given interval of time or space. day

Table 2: Common discrete probability distributions

2.2 Continuous probability distributions
A continuous probability distribution is the probability distribution of a continuous variable.
A continuous variable can have any value between its lowest and highest values. Therefore,
continuous probability distributions include every number in the variable’s range.
The probability that a continuous variable will have any specific value is so infinitesimally
small that it’s considered to have a probability of zero. However, the probability that a value
will fall within a certain interval of values within its range is greater than zero.
2.2.1Probability density functions
A probability density function (PDF) is a mathematical function that describes a continuous
probability distribution. It provides the probability density of each value of a variable, which
can be greater than one.
A probability density function can be represented as an equation or as a graph.
In graph form, a probability density function is a curve. You can determine the probability
that a value will fall within a certain interval by calculating the area under the curve within
that interval. You can use reference tables or software to calculate the area.
The area under the whole curve is always exactly one because it’s certain (i.e., a probability
of one) that an observation will fall somewhere in the variable’s range.
A cumulative distribution function is another type of function that describes a continuous
probability distribution.
Example: Probability density function
The probability density function of the normal distribution of egg weight is given by the
formula:

Where:
  is the probability density of egg weight
 is the mean egg weight in the population ( oz., in this case)
 is the standard deviation of egg weight in the population ( oz., in this
case)
The probability of an egg being exactly 2 oz. is zero. Although an egg can weigh very close
to 2 oz., it is extremely improbable that it will weigh exactly 2 oz. Even if a regular scale
measured an egg’s weight as being 2 oz., an infinitely precise scale would find a tiny
difference between the egg’s weight and 2 oz.
The probability that an egg is within a certain weight interval, such as 1.98 and 2.04 oz., is
greater than zero and can be represented in the graph of the probability density function as a
shaded region:

Figure 4: Probability density function

The shaded region has an area of .09, meaning that there’s a probability of .09 that an egg
will weigh between 1.98 and 2.04 oz. The area was calculated using statistical software.
Common continuous probability distributions
Distribution Description Example

Normal Describes data with values that become less probable SAT scores
distribution the farther they are from the mean, with a bell-shaped
probability density function.

Continuous Describes data for which equal-sized intervals have The amount of time
uniform equal probability. cars wait at a red light

Log-normal Describes right-skewed data. It’s the probability The average body
distribution of a random variable whose logarithm is weight of different
normally distributed. mammal species
Distribution Description Example

Exponential Describes data that has higher probabilities for small Time between
values than large values. It’s the probability earthquakes
distribution of time between independent events.

DISCUSSION QUESTIONS:
1.What are the advantages and limitations of different methods for fitting probability
distributions to empirical data?
2. How can understanding the characteristics and parameters of probability distributions
enhance the accuracy of predictive modeling?
3. In what ways do the properties of specific probability distributions influence their
suitability for modeling different types of data?