1/60

Statistics

Probability Theory Review

Shaheena Bashir

FALL, 2019
2/60
Outline

Statistics
Probability Theory
Probability
Random variable
Discrete Random variable
Probability Mass function pmf
Expectation of a Discrete Random variable
Continuous Random Variable
Continuous Probability Distribution pdf
Expectation of Continuous Random Variables
Binomial Distribution
Poisson Distribution
Normal Distribution
Properties
Standard Normal Distribution o
 3/60
Statistics

Why Study Statistics?

I Vitally important to the advancement of science

I Using the scientific method to answer questions and make
decisions
I A research question that involves use of a sample to draw a
conclusion about some larger population. We might need to
answer a question like ’is a particular drug effective in treating
· · · ?’.
I To answer such a question, to get the information from the
data efficiently, we need to understand Statistics, to present
the data in a meaningful way & to model the data to make
meaningful conclusions.

o
 4/60
Statistics

Scientific Research

To answer scientific questions

I Designing studies
I Collecting good data
I Describing the data with numbers and graphs
I Data analysis, model fitting, statistical tests, and then making
conclusions.

o
5/60
Statistics

o
 6/60
Probability Theory
Probability

Probability: Classical Definition

Let an event A consist of m out of n, equally likely elementary
events where the total number n is formed by a complete system of
mutually exclusive events. The probability of event A is then given
by the quotient
Number Of Ways Event A Can Occur
P(A) =
Total number Of Possible Outcomes
m
=
n

o
7/60
Probability Theory
Random variable

Many random processes produce numbers. These numbers are

called random variables.

o
 8/60
Probability Theory
Random variable

Random Variable: Background

I Summarize the outcome from a random experiment by a

simple number
I The particular outcome of the random experiment is not
known in advance
I The resulting value of the variable that results from an
experiment is not known in advance.

o
 9/60
Probability Theory
Random variable

Random Variable

I A random variable is a function from the sample space S to

the real numbers, i.e., X is a rule which assigns a number X(s)
for each outcome x ∈ S
I A random variable is denoted by an uppercase letter such as
X , Y , Z . After an experiment is conducted, the measured
value of the random variable is denoted by a lowercase letter
such as x, y , z

o
 10/60
Probability Theory
Random variable

Random Variable: Example

Toss a coin.

X : Number of heads from the result of a coin toss; X = {0, 1}

 1
P(X = x) = 2 for x = 0, 1;
0 otherwise.

o
 11/60
Probability Theory
Random variable

Random Variable: Example

Roll 2 dice, and let X denotes the sum of the numbers on the
upper faces that might appear.

o
 12/60
Probability Theory
Random variable

Random Variable: Example

An experiment consists of flipping a coin 4 times and observing the

sequence of heads and tails. The random variable X is the number
of heads in the observed sequence. Draw a table that shows the
possible values of X and the number of outcomes associated to
each value.

o
 13/60
Probability Theory
Random variable

Random Variable
I Discrete: possible values of the variable can be listed in
either a finite or an infinite list., e.g., number of defective
parts produced in manufacturing, number of people getting flu
in winter, etc..

0 1 2 3

I Continuous: assumes any values in an interval (either finite

or infinite) of real numbers for its range, e.g., height of
students, weight of new born babies, the temperature range
during a day, life length of some electric device, etc.

o
150 160 170 180 190
 14/60
Probability Theory
Discrete Random variable

Example

Let X = Number of times a Mobilink customer center receives a

phone call in 1 hour duration. Let the maximum number of calls
during a 1 hour period is 45. We might be interested in probability
of events such as finding that
I More than 1 call will be received during a minute
I At most 1 call in 1 minute
Written in terms of the random variable X , we would write these
probability statements as P(X > · · · ) and P(X · · · ), respectively

o
15/60
Probability Theory
Probability Mass function pmf

I Probability, a quantification of uncertainty for events in the

sample space.
I How to assign a probability to the values that the random
variable might assume.

o
 16/60
Probability Theory
Probability Mass function pmf

Discrete Probability Distribution

I The probability distribution of a random variable X is a

description of the probabilities associated with the possible
values of X .
I For a discrete random variable, the distribution is often
specified by just a list of the possible values along with the
probability of each, or
I In some cases, it is convenient to express the probability in
terms of a formula
I Probability Mass Function or pmf written as

P(X = x) = p(x)

o
 17/60
Probability Theory
Probability Mass function pmf

Probability Mass Function: Properties

1. 0 ≤ P(X = x) ≤ 1, ∀x ∈ S.
P
2. P(X = x) = 1

o
 18/60
Probability Theory
Probability Mass function pmf

Discrete Probability Distribution

I For a discrete random variable, its pmf or probability mass

function defines all that we need to know about the random
variable.
I A pmf for a discrete random variable is defined (with positive
probabilities) only for a finite or countably infinite set of
possible values - typically integers.

o
 19/60
Probability Theory
Probability Mass function pmf

Example
Toss a fair coin three times and let x denote the number of heads
observed. Construct the probability distribution of X & plot the
graph. 1/2
3/8
P(X=x)

1/4
1/8
0

0 1 2 3

Number of Heads in 3 tosses of a fair coin

o
 20/60
Probability Theory
Probability Mass function pmf

Exercise

Roll a die, and let X denote the value that appears on the upper
face. What is the probability mass function P(X ) for the random
variable X ? Plot the graph.

o
21/60
Probability Theory
Probability Mass function pmf

o
22/60
Probability Theory
Expectation of a Discrete Random variable

An expected value gives a quick insight into the behavior of a

random variable
o
23/60
Probability Theory
Expectation of a Discrete Random variable

Toss a coin 100 times and count the number of times heads might
turn up? How many heads do you expect?

o
 24/60
Probability Theory
Expectation of a Discrete Random variable

Expected value of a Discrete Random Variable

Let a discrete random variable X that assumes values x1 , x2 , . . . , xn
has expected value
Xn
E (X ) = xj px (xj )
j=1

If X takes on a countably infinite number of values x1 , x2 , . . . , then

X∞
E (X ) = xj px (xj )
j=1

o
 25/60
Probability Theory
Expectation of a Discrete Random variable

Example

A company manufactures electronic gadgets. 1 out of every 50

gadgets is faulty. However, the company does not know which
ones are faulty until they receive a complaint from the buyer. The
company makes a profit of Rs.30 on the sale of a working gadget
but suffers a loss of Rs.800 for every faulty gadget. Can the
company make a profit in the long term?

o
 26/60
Probability Theory
Continuous Random Variable

Continuous Random Variable

Continuous variable can take any value in an interval. There is the

beginning and end point, but there are infinitely many possibilities
of answers within that range such as an interval of the real line.
e.g.,
X = waiting time for a bus; 0 ≤ X ≤ ∞
we might want to calculate probabilities such as
I You have to wait less than 20 minutes, i.e., P(X < 20) or
I the event that the waiting time was between 5 to 10 minutes,
i.e., P(5 ≤ X ≤ 10)
Since X is continuous, the range of values of X covered by these
events is a continuous range of real numbers.
o
 27/60
Probability Theory
Continuous Random Variable

Examples Cont’d

Several practical life examples of continuous random variables,

I The speed of a car
I Weights of participants in a diet control program
I The life time of an electric device
I Rainfall during a given day in Lahore

o
 28/60
Probability Theory
Continuous Probability Distribution pdf

Probability Density Function pdf

I A continuous random variable has an analogous function
associated with it called its probability density function or pdf.
I A pdf for a continuous random variable is defined for all real
numbers in the range of the random variable
I The overriding concept here for a continuous random variable
is that AREA=PROBABILITY.
I More specifically, the area under the pdf curve between points
a and b is the same as the probability that the random
variable will have a value between a and b
Zb
P(a ≤ X ≤ b) = f (x)d(x)
a
o
 29/60
Probability Theory
Continuous Probability Distribution pdf

Properties of pdf

I f (x) ≥ 0, ∀x
R∞
I f (x)d(x) = 1
−∞
Ra
I P(X = a) = P(a ≤ X ≤ a) = f (x)d(x) = 0
a
Ra
I P(X < a) = P(X ≤ a) = F (a) = f (x)d(x)
−∞
I This has a very useful consequence in the continuous case:

P(a ≤ X ≤ b) = P(a < X ≤ b) = P(a ≤ X < b) = P(a < X < b) =

o
 30/60
Probability Theory
Continuous Probability Distribution pdf

Example

Suppose that a random variable X has density

 3
f (x) = 8 x(2 − x)(3 − x) 0 ≤ x ≤ 2,
0 otherwise

I Find P(X ≤ 1)

o
 31/60
Probability Theory
Expectation of Continuous Random Variables

Expectation
For a continuous random variable X with pdf f(x)
Z ∞
E (X ) = xf (x)d(x)
−∞
Z ∞
E (g (X )) = g (x)f (x)d(x)
−∞

o
 32/60
Binomial Distribution

Bernoulli distribution

I A Bernoulli random variable models random experiments that

have two possible outcomes, sometimes referred to as
’success’ and ’failure.’
I This random variable can only take two possible values,
usually 0 and 1.
I Taking a pass-fail exam; either pass (X=1) or fail (X=0).
I Toss a coin. The outcome is ether heads (X=1) or tails (X=0).
I A newborn child is either male or female.

o
33/60
Binomial Distribution

I n independent bernoulli trials

I Count the number of successes X that occur in the n trials,
then X is said to be a binomial random variable with
parameters (n, p)
I Bernoulli random variable is just a binomial random variable
with parameters (1, p) o
34/60
Binomial Distribution

1
I Tossing a coin, two possible outcomes with probability of 2
for each
I Count the number of times that head appears when tossing a
coin 3 times (denoted by X)
I X can take on any of the values X = 0, 1, 2, 3
I Some values of X are more likely to occur than others
I Quantify the associated chances of occurrence of different
values of X

o
35/60
Binomial Distribution

The possible outcomes in 3 tosses of a fair coin are:

o
35/60
Binomial Distribution

The possible outcomes in 3 tosses of a fair coin are:

S = {TTT , TTH, THT , THH, HTT , HTH, HHT , HHH}

o
35/60
Binomial Distribution

The possible outcomes in 3 tosses of a fair coin are:

S = {TTT , TTH, THT , THH, HTT , HTH, HHT , HHH}

o
35/60
Binomial Distribution

The possible outcomes in 3 tosses of a fair coin are:

S = {TTT , TTH, THT , THH, HTT , HTH, HHT , HHH}

1 1
P(THH) = × ( )2
2 2

o
35/60
Binomial Distribution

The possible outcomes in 3 tosses of a fair coin are:

S = {TTT , TTH, THT , THH, HTT , HTH, HHT , HHH}

1 1
P(THH) = × ( )2
2 2

P(2 H in 3 tosses) = · · ·

o
35/60
Binomial Distribution

The possible outcomes in 3 tosses of a fair coin are:

S = {TTT , TTH, THT , THH, HTT , HTH, HHT , HHH}

1 1
P(THH) = × ( )2
2 2

P(2 H in 3 tosses) = · · ·

What will be the probability of obtaining 2 H in 100 tosses?

o
 36/60
Binomial Distribution

Practical Life Examples of Binomial Distribution

I Each item from manufacturing production line can be either

defective or non-defective
I Chances of profit and loss in Stock Market
I Proportion of students who have a success in exam
I Proportion of voters who favored Liberals

o
37/60
Binomial Distribution

o
 38/60
Binomial Distribution

Binomial Experiment

I A random experiment with fixed number of trials, i.e., n

I Trials are independent & identical
I Each trial results in one of 2 possible outcomes, i.e.,
”success”, or a ”failure”.
I The probabilities of success p and of failure (q = 1 − p) are
constant across trials.

o
39/60
Binomial Distribution

The pmf  
n x
P(X = x) = p (1 − p)n−x
x

I n is the fixed number of trials

I x is the number of successes in n trials, i.e., x = 0, 1, . . . , n
I p is the probability of success on any given trial
n


x is the binomial coefficient
I

I n & p are the parameters of the binomial distribution

o
 40/60
Binomial Distribution

Example

Suppose that 80% of adults with allergies report symptomatic

relief with a specific medication. If the medication is given to 10
new patients with allergies, what is the probability that it is
effective in exactly seven?

o
 41/60
Binomial Distribution

Effect of n & p on the Shape

n=15, p=0.2 n=15, p=0.8

0.25

0.25
0.20

0.20
0.15

0.15
0.10

0.10
0.05

0.05
0.00

0.00
0 1 2 3 4 5 6 7 8 9 11 13 15 0 1 2 3 4 5 6 7 8 9 11 13 15

o
 42/60
Binomial Distribution

Effect of n & p on the Shape Cont’d

n=15, p=0.5 n=40, p=0.2

0.15
0.15

0.10
0.10

0.05
0.05
0.00

0.00
0 1 2 3 4 5 6 7 8 9 11 13 15 0 3 6 9 12 16 20 24 28 32 36 40

o
 43/60
Binomial Distribution

Expectation & Variance

n
X
E (X ) = xi P(X = xi )
i=1
n  
X n x
= xi p (1 − p)n−x
x
i=0
= np

Var (X ) = E (X 2 ) − [E (X )]2
= np(1 − p)

o
 44/60
Poisson Distribution

Poisson Distribution

Many experimental situations occur in which we observe the counts

of events within a set unit of time, area, volume, length, etc. e.g.,
I The number of airplanes that come into an airport in 2 hours
I The number of mutations in set sized regions of a
chromosome
I The number of particles emitted by a radioactive source in a
given time
I The number of customers arriving in 1 hour at a shop
I The number of defects in a certain size carpet

o
 45/60
Poisson Distribution

Conditions for the Poisson Distribution

A random variable X has a Poisson distribution if the following

conditions hold
I X counts the number of events within a specified time or
space, etc.
I The events occur independently of each other
I No 2 events can happen exactly at the same time

o
 46/60
Poisson Distribution

Poisson Distribution

I The Poisson distribution can be viewed as arising from the

binomial distribution or from the exponential density.
I The Poisson distribution applies particularly to rare events,
i.e., events which occur infrequently in time, space, volume or
any other dimension.
I The Poisson distribution is used as an approximation to the
binomial distribution when the parameter n is large, (i.e.,
n → ∞) while p is small (p → 0); (rule of thumb; when
np < 7)

o
 47/60
Poisson Distribution

Probability Mass Function

X ∼ Poisson(λ)

λx
P(X = x) = e −λ , x = 0, 1, 2, . . .
x!

I Here X = number of events that occur during the specified 1

unit of time
I λ = the average rate of events that occur during the specified
1 unit of time, space, volume, etc.
E (X ) = λ
Var (X ) = λ

o
 48/60
Poisson Distribution

Units in Poisson Probability

It is important to use consistent units in the calculation of

probabilities, means, and variances involving Poisson random
variables.
For example, if the
I average number of flaws per millimeter of wire is 3.4, then the
I average number of flaws in 10 millimeters of wire is 34, and
the
I average number of flaws in 100 millimeters of wire is 340.

o
 49/60
Poisson Distribution

Examples

I The number of people entering a movie theater averages one

every two minutes. Assuming that a Poisson distribution is
appropriate, what is the probability that no people enter
between 12:00 and 12:05?

o
50/60
Normal Distribution

o
51/60
Normal Distribution

I Continuous variable can take any value in an interval of real

numbers for its range, e.g., height of students, weight of new
born babies, the temperature range during a day, life length of
some electric device, etc.
I Such variables have distributions that are evenly distributed
about the mean; bell-shaped (approximately normal) also
called a Gaussian distribution.

o
 52/60
Normal Distribution

Normal Distribution

The probability density function (pdf) is:

1 (x−µ)2
f (x, µ, σ) = √ e − 2σ2 ; −∞ < x < ∞
σ 2π

I X ∼ N(µ, σ 2 ), where µ is the mean, & σ 2 is the variance

I µ & σ 2 are the parameters of the normal distribution
I π = 3.141593 & e = 2.71828
Rx −
(x−µ)2
I No closed form solution for CDF: √1 e 2σ 2 dx
σ 2π
−∞

o
 53/60
Normal Distribution

Mean & Variance

1.0
0.8
µ=0
µ=1
µ = −1

0.6
Density

0.4
0.2
0.0

−4 −2 0 2 4

x
1.0
0.8

σ=1
σ=2
σ = 0.5
0.6
Density

0.4
0.2
0.0

o
−4 −2 0 2 4
 54/60
Normal Distribution
Properties

Properties of Normal Distribution

I It is a bell shaped distribution

I The total area under the curve is equal to 1, useful to solve
practical application problems
I The mean, median & mode are equal & located at the center
of the distribution
I Maximum occurs at µ, 50% area lies to either side of the
mean µ
I The inflection points are located at µ − σ and µ + σ. (An
inflection point is a point on the curve where the sign of the
curvature changes.)

o
 55/60
Normal Distribution
Properties

Properties of Normal Distribution Cont’d

I This curve lies entirely above the horizontal axis, and x-axis is
an asymptote in both horizontal directions
I The area between the curve and the horizontal axis is exactly
1. Note that this is the area of a region that is infinitely wide,
since the curve never actually touches the axis.

o
 56/60
Normal Distribution
Standard Normal Distribution

Standard Normal Distribution

2
e −z /2
f (z) = √ , −∞ < z < ∞
2π
I It is a normal distribution with a mean of 0 & standard
deviation of 1, i.e., z ∼ N(0, 1)
I The standard normal distribution table gives the area under
the standard normal curve for any value of z from 0 − 3.4 or
so

o
 57/60
Normal Distribution
Standard Normal Distribution

Standardizing a Normal Random Variable

I All normally distributed variables can be transformed into the

standard normally distributed variables by the formula:

x −µ
z = ⇒ X = µ + σz
σ
value − mean
z =
standard deviation

Z ∼ N(0, 1) X ∼ N(µ, σ 2 )

o
58/60
Normal Distribution
Standard Normal Distribution

o
 59/60
Normal Distribution
Standard Normal Distribution

Finding Probabilities Using Table

I The table for the cumulative distribution of the normal is for

the STANDARD normal variable Z .
I The table gives values for nonnegative z. For negative values
of z, the area can be obtained from the symmetry property of
the curve
I Remember to convert the normal random variable X to Z
before using the table
I Convert Z back to X .

o
 60/60
Normal Distribution
Standard Normal Distribution

Finding Probabilities Using Table Cont’d

I Probability to the left of z value

I P(Z ≤ 1.67)
I P(Z ≤ −1.67)
I Probability to the right of z value
I P(Z ≥ 1.67)
I P(Z ≥ −1.67)
I Probability between 2 z values
I P(−1.67 ≤ Z ≤ 1.67)
I Probability between 2 x values; X ∼ N(µ = 8, σ = 4)
I P(4 ≤ X ≤ 16)