WEEK 02

Lecture 6: Introduction to Probability

Lecture objectives
• Comprehend the different ways of assigning probability
• Understand and apply marginal, union, joint, and conditional probabilities
• Solve problems using the laws of probability including the laws of addition, multiplication and
conditional probability
• Revise probabilities using Bayes’ rule
Probability
• Probability is the numerical measure of the likelihood that
an event will occur.
• The probability of any event must be between 0 and 1,
inclusively – 0 ≤ P(A) ≤ 1 for any event A.
• The sum of the probabilities of all mutually exclusive and
collectively exhaustive events is 1. – P(A) + P(B) + P(C) = 1 – A,
B, and C are mutually exclusive and collectively exhaustive
Methods of Assigning Probabilities
• Classical method of assigning probability (rules and laws)
• Relative frequency of occurrence (cumulated historical data)
• Subjective Probability (personal intuition or reasoning)
Classical Probability
• Number of outcomes leading to the event divided by the total
number of outcomes possible
• Each outcome is equally likely
• Determined a priori -- before performing the experiment
• Applicable to games of chance
• Objective -- everyone correctly using the method assigns an
identical probability

Relative Frequency Probability

• Based on historical data
• Computed after performing the experiment
• Number of times an event occurred divided by the number of
trials
• Objective -- everyone correctly using the method assigns an
identical probability

Subjective Probability
• Comes from a person’s intuition or reasoning
• Subjective
-- different individuals may (correctly) assign different numeric probabilities to the same event
• Degree of belief
• Useful for unique (single-trial) experiments
– New product introduction
– Initial public offering of common stock
– Site selection decisions
– Sporting events
Probability - Terminology • Experiment • Event • Elementary Events • Sample Space • Unions and
Intersections • Mutually Exclusive Events • Independent Events • Collectively Exhaustive Events
• Complementary Events
Experiment, Trial, Elementary Event, Event
• Experiment: a process that produces outcomes
– More than one possible outcome
– Only one outcome per trial
• Trial: one repetition of the process
• Elementary Event: cannot be decomposed or broken down into other events
• Event: an outcome of an experiment
– may be an elementary event, or
– may be an aggregate of elementary events
– usually represented by an uppercase letter, e.g., A, E1
An Example Experiment
• Experiment: randomly select, without replacement, two
families from the residents of Tiny Town
• Elementary Event: the sample includes families A and C
• Event: each family in the sample has children in the
household
• Event: the sample families own a total of four automobiles
Sample Space
• The set of all elementary events for an experiment
• Methods for describing a sample space
– roster or listing
– tree diagram
– set builder notation
– Venn diagram
Sample Space: Roster Example
• Experiment: randomly select, without
replacement, two families from the
residents of Tiny Town
• Each ordered pair in the sample space
is an elementary event, for example --
(D,C)

Sample Space: Set Notation

for Random Sample of Two
Families
• S = {(x,y) | x is the family
selected on the first draw, and y
is the family selected on the
second draw}
• Concise description of large
sample spaces
Independent Events
• Occurrence of one event does not affect the
occurrence or non-occurrence of the other event
• The conditional probability of X given Y is equal to
the marginal probability of X.
• The conditional probability of Y given X is equal to
the marginal probability of

𝑃(𝑋|𝑌) = 𝑃(𝑋) 𝐴𝑁𝐷 𝑃(𝑌|𝑋) = 𝑃(𝑌)

Counting the Possibilities

• mn Rule
• Sampling from a Population with
Replacement
• Combinations: Sampling from a
Population without Replacement
mn Rule
• If an operation can be done m ways and a second operation can be done n ways, then there are mn
ways for the two operations to occur in order.
• This rule is easily extend to k stages, with a number of ways equal to n1.n2.n3..nk
• Example: Toss two coins . The total umber of simple events is 2 x 2 =4
Sampling from a Population with Replacement
• A tray contains 1,000 individual tax returns. If 3 returns are randomly selected with replacement
from the tray, how many possible samples are there?
• (N)n = (1,000)3 = 1,000,000,000

Problem
• A company conducted a survey for the American Society of Interior Designers in which workers
were asked which changes in office design would increase productivity.
• Respondents were allowed to answer more than one type of design change.

Problem • If one of the survey respondents was randomly selected and asked what office design
changes would increase worker productivity, – what is the probability that this person would select
reducing noise or more storage space?
Solution
• Let N represent the event “reducing noise.”
• Let S represent the event “more storage/ filing space.”
• The probability of a person responding with N or S can be symbolized statistically as a union
probability by using the law of addition. 𝑃(𝑁 ∪ 𝑆)

Problem
• A company data reveal that 155 employees worked one of four types of positions.
• Shown here again is the raw values matrix (also called a contingency table) with the frequency
counts for each category and for subtotals and totals containing a breakdown of these employees by
type of position and by sex.
Problem
• Shown here are the raw values matrix and corresponding probability matrix for the results of a
national survey of 200 executives who were asked to identify the geographic locale of their company
and their company’s industry type.
• The executives were only allowed to select one locale and one industry type.

Lecture 7: Introduction to Probability-II

Problem
• A company data reveal that 155 employees worked one of four types of positions.
• Shown here again is the raw values matrix (also called a contingency table) with the frequency
counts for each category and for subtotals and totals containing a breakdown of these employees by
type of position and by sex.

Problem
• Shown here are the raw values matrix and corresponding probability matrix for the results of a
national survey of 200 executives who were asked to identify the geographic locale of their company
and their company’s industry type.
• The executives were only allowed to select one locale and one industry type.
 Questions
a. What is the probability that the
respondent is from the Midwest (F)?
b. What is the probability that the
respondent is from the communications
industry (C) or from the Northeast (D)?
c. What is the probability that the respondent
is from the Southeast (E) or from the finance
industry (A)?

Law of Multiplication

Problem • A company has 140 employees, of which 30 are supervisors.

• Eighty of the employees are married, and 20% of the married employees are supervisors.
• If a company employee is randomly selected, what is the probability that the employee is married
and is a supervisor?
Law of Conditional Probability
• The conditional probability of X given Y is the joint probability of X and Y divided by the marginal
probability of Y.

Computing Conditional
Probability
• Of the cars on a used car lot, 70%
have air conditioning (AC) and 40%
have a CD player (CD). 20% of the
cars have both.
• What is the probability that a car
has a CD player, given that it has
AC ?
• We want to find P(CD | AC).
Independent Events
• If X and Y are independent events, the occurrence of Y does not affect the probability of X
occurring.
• If X and Y are independent events, the occurrence of X does not affect the probability of Y
occurring.

Independent Events Demonstration Contd… Independent Events

Revision of Probabilities: Bayes’ Rule

• An extension to the conditional law of probabilities
• Enables revision of original probabilities with new information
Problem
• A particular type of printer ribbon is produced by only two companies, Alamo Ribbon Company
and South Jersey Products.
• Suppose Alamo produces 65% of the ribbons and that South Jersey produces 35%.
• Eight percent of the ribbons produced by Alamo are defective and 12% of the South Jersey ribbons
are defective
• A customer purchases a new ribbon. What is the probability that Alamo produced the ribbon?
What is the probability that South Jersey produced the ribbon?

Lecture 8: Probability Distributions

Lecture Objectives • Empirical Distribution • Discrete Distributions • Continuous Distributions
What is a distribution?
• Describes the ‘shape’ of a batch of numbers
• The characteristics of a distribution can sometimes be defined using a small number of numeric
descriptors called ‘parameters’
Why distribution?
• Can serve as a basis for standardized comparison of empirical distributions
• Can help us estimate confidence intervals for inferential statistics
• Form a basis for more advanced statistical methods – ‘fit’ between observed distributions and
certain theoretical distributions is an assumption of many statistical procedures
Random variable
• A variable which contains the outcomes of a chance experiment
• “Quantifying the outcomes”
• Example X= (1 = Head, 0 = Tails)
• A variable that can take on different values in the population according to some “random”
mechanism
• Discrete – Distinct values, countable – Year
• Continuous – Mass
Probability Distributions
• The probability distribution function or probability density function (PDF) of a random variable X
means the values taken by that random variable and their associated probabilities.
• PDF of a discrete r.v. (also known as PMF): Example 1: Let the r.v. X be the number of heads
obtained in two tosses of a coin. Sample Space: {HH, HT, TH, TT}
Cumulative Distribution Function
• The CDF of a random variable X (defined as F(X)) is a graph associating all possible values, or the
range of possible values with P(X  x).
• CDFs always lie between 0 and 1 i.e., 0  F(Xi)  1, Where F(Xi) is the CDF.
The Expected Value of X
Let X be a discrete rv with set of possible values D and pmf p(x).
The expected value or mean value of X, denoted
Properties of Expected Value
1. E (b) =b, b is constant
2. E (X+Y) = E(X) +E(Y)
𝑋 𝐸(𝑋)
3. 𝐸 ( ) ≠
𝑦 𝐸(𝑌)
4. 𝐸(𝑋𝑌) ≠ 𝐸(𝑋)𝐸(𝑌) unless they are independent
5. 𝐸(𝑎𝑋) = 𝑎𝐸(𝑋) a is Constant,
6. 𝐸(𝑎𝑋 + 𝑏) = 𝑎𝐸(𝑥) + 𝑏 , 𝑎 𝑎𝑛𝑑 𝑏 𝑎𝑟𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
Properties of Variance
1. Var(constant) = 0
2. If X and Y are two independent random variables,
then Var(X + Y) = Var(X) + Var (Y) and Var(X - Y) = Var(X) + Var (Y)
3. If b is a constant then Var(b+X) = Var(X)
4. If a is a constant then 𝑉𝑎𝑟(𝑎𝑋) = 𝑎2 𝑉𝑎𝑟(𝑋).
5. If a and b are constants then 𝑉𝑎𝑟(𝑎𝑥 + 𝑏) = 𝑎2 𝑉𝑎𝑟(𝑥).
6. If X and Y are two independent random variables and a and b are constants then
𝑉𝑎𝑟(𝑎𝑋 + 𝑏𝑌) = 𝑎2 𝑉𝑎𝑟(𝑋) + 𝑏 2 𝑉𝑎𝑟(𝑌)
Covariance
Covariance: For two discrete random variables X and Y with E(X) = x and E(Y) = y,
the covariance between X and Y is defined as Cov(XY) = xy = E(X - x) E(Y - y) = E(XY) - x y.
Covariance
• In general, the covariance between two random variables can be positive or negative.
• If two random variables move in the same direction, then the covariance will be positive, if they
move in the opposite direction the covariance will be negative.
Properties:
1.If X and Y are independent random variables, their covariance is zero. Since E(XY) = E(X)E(Y)
2. Cov(XX) = Var(X)
3. Cov(YY) = Var(Y)
Correlation Coefficient
• The covariance tells the sign but not the magnitude about how strongly the variables are positively
or negatively related. The correlation coefficient provides such measure of how strongly the variables
are related to each other.
• For two random variables X and Y with E(X) = x and E(Y) = y, the correlation coefficient is defined
as

Lecture 9: Probability Distributions-II

Some Special Distributions
• Discrete – Binomial – Poisson – Hyper geometric
• Continuous – Uniform – Exponential – Normal
Binomial Distribution
• Let us consider the purchase decisions of the next three customers who enter a store.
• On the basis of past experience, the store manager estimates the probability that any one
customer will make a purchase is .30.
• What is the probability that two of the next three customers will make a purchase?

Binomial Distribution- Assumtions

• Experiment involves n identical
trials
• Each trial has exactly two possible
outcomes: success and failure
• Each trial is independent of the
previous trials
• p is the probability of a success on
any one trial q = (1-p) is the
probability of a failure on any one trial
• p and q are constant throughout the
experiment
• X is the number of successes in the n
trials
Mean and Variance
• Suppose that for the next month the Clothing Store forecasts 1000 customers will enter
the store.
• What is the expected number of customers who will make a purchase?
• The answer is μ = np = (1000)(.3) = 300.
• For the next 1000 customers entering the store, the variance and standard deviation for
the number of customers who will make a purchase are

Poisson Distribution
• Describes discrete occurrences over a continuum or interval
• A discrete distribution
• Describes rare events
• Each occurrence is independent any other occurrences.
• The number of occurrences in each interval can vary from zero to infinity.
• The expected number of occurrences must hold constant throughout the experiment.
Poisson Distribution: Applications
• Arrivals at queuing systems
– airports -- people, airplanes, automobiles, baggage
– banks -- people, automobiles, loan applications
– computer file servers -- read and write operations
• Defects in manufactured goods
– number of defects per 1,000 feet of extruded copper wire
– number of blemishes per square foot of painted surface
– number of errors per typed page
The Hypergeometric Distribution
• The binomial distribution is applicable when
selecting from a finite population with replacement
or from an infinite population without replacement.
• The hypergeometric distribution is applicable when
selecting from a finite population without
replacement.
Hyper Geometric Distribution
• Sampling without replacement from a finite
population
• The number of objects in the population is denoted
N.
• Each trial has exactly two possible outcomes,
success and failure.
• Trials are not independent
• X is the number of successes in the n trials
• The binomial is an acceptable approximation,
if N/10 > n Otherwise it is not. 17
Continuous Probability Distributions
• A continuous random variable is a variable that can assume any value on a continuum (can assume
an uncountable number of values)
– thickness of an item
– time required to complete a task
– temperature of a solution
– height
• These can potentially take on any value, depending only on the ability to measure precisely and
accurately.
Continuous Distributions • Uniform • Normal • Exponential
The Uniform Distribution
• The uniform distribution is a probability distribution that has equal probabilities for all possible
outcomes of the random variable
• Because of its shape it is also called a rectangular distribution
 Example: Uniform Distribution
• Consider the random variable x representing the
flight time of an airplane traveling from Delhi to
Mumbai.
• Suppose the flight time can be any value in the
interval from 120 minutes to 140 minutes.
• Because the random variable x can assume any
value in that interval, x is a continuous rather than a
discrete random variable

Example : Uniform Distribution contd…. Uniform Probability Distribution for Fligh time
• Let us assume that sufficient actual flight
data are available to conclude that the
probability of a flight time within any 1-
minute interval is the same as the probability
of a flight time within any other 1-minute
interval contained in the larger interval from
120 to 140 minutes.
• With every 1-minute interval being equally
likely, the random variable x is said to have a
uniform probability distribution.

Exponential Probability Distribution

• Density Function
Exponential Probability Distribution
• Suppose that x represents the loading time for a truck at
loading dock and follows such a distribution.
• If the mean, or average, loading time is 15 minutes ( μ =
15), the appropriate probability density function for x is
 Exponential Probability Distribution
Cumulative Probabilities

Example: Exponential Probability Distribution

• The time between arrivals of cars at a Petrol pump follows an exponential probability distribution
with a mean time between arrivals of 3 minutes.
• The Petrol pump owner would like to know the probability that the time between two successive
arrivals will be 2 minutes or less.

Mean of Poisson and Mean of Exponential Distributions

• Because the average number of arrivals is 10 cars per hour,
the average time between cars arriving is
The Standardized Normal Distribution
• Any normal distribution (with any mean
and standard deviation combination) can
be transformed into the standardized
normal distribution (Z).
• Need to transform X units into Z units.
• The standardized normal distribution has
a mean of 0 and a standard deviation of 1.
The Standardized Normal Distribution
• Translate from X to the standardized normal
(the “Z” distribution) by subtracting the mean
𝑋−𝑢
𝑧=
dividing by its standard deviation:
𝜎
The Standardized Normal Distribution:
Density Function • The formula for the standardized normal probability density function is
Where, e = the mathematical constant approximated by 2.71828
π = the mathematical constant approximated by 3.14159
Z = any value of the standardized normal distribution

The Standardized Normal Distribution: Shape

• Also known as the “Z” distribution
• Mean is 0
• Standard Deviation is 1

Values above the mean have positive Z-values,

values below the mean have negative Z-values

The Standardized Normal Distribution: Example

• If X is distributed normally with mean of 100 and standard deviation of 50, the Z value for X = 200 is

This says that X = 200 is two standard

deviations (2 increments of 50 units)
above the mean of 100.

Normal Probability Tables

Example: P(Z < 2.00) = .9772
Finding Normal Probability Procedure

To find P(a < X < b) when X is distributed

normally:

• Draw the normal curve for the problem

in terms of X.

• Translate X-values to Z-values.

• Use the Standardized Normal Table.

 Given Normal Probability: Find
the X Value
• Let X represent the time it takes
(in seconds) to download an
image file from the internet.
• Suppose X is normal with mean
8.0 and standard deviation 5

Given Normal Probability,

Find the X Value
• Second, convert the Z value to X units using
the following formula.
X = = = μ 8 + . 0 3 . Zσ + 80 ( − 0 . 84 ) 5 . 0
So 20% of the download times from the
distribution with mean 8.0 and standard
deviation 5.0 are less than 3.80 seconds.
Assessing Normality
• It is important to evaluate how well the data set is approximated by a normal distribution.
• Normally distributed data should approximate the theoretical normal distribution:
– The normal distribution is bell shaped (symmetrical) where the mean is equal to the median.
– The empirical rule applies to the normal distribution.
– The interquartile range of a normal distribution is 1.33 standard deviations.
Assessing Normality
• Construct charts or graphs
– For small- or moderate-sized data sets, do stem-and-leaf display and box-and-whisker plot look
symmetric?
– For large data sets, does the histogram or polygon appear bell shaped?
• Compute descriptive summary measures
– Do the mean, median and mode have similar values?
– Is the interquartile range approximately 1.33 σ?
– Is the range approximately 6 σ?
Assessing Normality
• Observe the distribution of the data set
– Do approximately 2/3 of the observations lie within mean ± 1 standard deviation?
– Do approximately 80% of the observations lie within mean ± 1.28 standard deviations?
– Do approximately 95% of the observations lie within mean ± 2 standard deviations?