Probability and Statistics

Chapter 1
Probability and Distributions
 Introduction
• An (random or statistical) experiment : its
outcome cannot be predicted with certainty,
but can be repeated under the same
conditions.
• Sample space : 
the collection of every possible outcome
Ex1: Toss of a coin  ={H, T}
Ex2: Cast of one red die and one white die
 ={(1,1), …, (1,6), (2,1), …, (2,6), …, (6,6)}
 Introduction
• Event C: an element or a collection of elements of
sample space .
• If the outcome is in C, we shall say that the event
C has occurred.

Ex3: Toss a die

- A: an odd number
- B: a number > 2
 Set theory
• Subset
• Null set
• Union
• Intersection
• Complement
• Difference
• DeMorgan’s Laws
 Probability
 : sample space, B: a collection of subsets of  (
(-field)

• Define P: a real valued function defined on B satisfying the followings

1. P(C) ≥ 0, for all C  B
P() =1
2. P(
3. countable additivity
 Properties of Probability
c
 P(C) 1  P(C )
 P() 0
 C1  C 2  P(C1 ) P(C 2 )
 0 P(C) 1
 P(C1 C 2 ) P(C1 )  P(C 2 )  P(C1  C 2 )
If C1 and C 2 are disjoint(mutually exclusive), then
P(C1  C 2 ) P(C1 )  P(C 2 )
 Probability
• Probabilities can be found using
– Empirical studies
– Equally likely events.

Ex4:
–Toss a fair coin. P(H) = ½
–30% of the Slovenian people has blonde.
Select a person at random. Then
P(blonde hair) = 0.3
Ex5: Suppose that there were 120 students in the
classroom, and that they could be classified as
follows:
Brown Not Brown
A: brown hair Male 20 40
P(A) = 50/120
Female 30 30
B: female
P(B) = 60/120

P(AB)
P(AB) == P(A)
P(A) ++ P(B)
P(B) –– P(AB)
P(AB)
== 50/120
50/120 ++ 60/120
60/120 -- 30/120
30/120
== 80/120
80/120 == 2/3
2/3
 Counting and mn Rules
• If all c   are equally likely, you can calculate

• If an experiment is performed in two stages, with m ways to accomplish the first stage and n ways to accomplish the second stage, then there are mn ways to accomplish the experiment
• This rule is easily extended to k stages, with the number of ways equal to

nnAA number
number of
of simple
simple events
events in
in A
A
n1 n2 n3 … nk

P(A) 
P(A)  

NN total
totalnumber
numberof of simple
simpleevents
events
 Permutations and Combinations
• The number of ways you can arrange
n distinct objects, taking them r at a time is
n n!
P 
r
(n  r )!
where n!n(n  1)(n  2)...( 2)(1) and 0!1.

• The number of distinct combinations of n

distinct objects that can be formed, taking
them r at a time is n n!
Cr 
r!(n  r )!
 Permutations and Combinations
•Ex6: A lock consists of five parts and can be
assembled in any order. A quality control engineer
wants to test each order for efficiency of assembly.
How many orders are there?

•Ex7: Three members of a 5-person committee

must be chosen to form a subcommittee. How
many different subcommittees could be formed?
 Exercise 1
(Urn problem) A bowl contains 16 chips, of which
6 are red, 7 are white, and 3 are blue. If four
chips are taken at random and without
replacement, find the probability that
a. each of the 4 chips is red
b. none of the 4 chips is red
c. there is at least 1 chip of each color
 Exercise 2
(Urn problem) A bowl contains 16 chips, of which
6 are red, 7 are white, and 3 are blue. If four
chips are taken at random and with replacement,
find the probability that
a. each of the 4 chips is red
b. none of the 4 chips is red
c. there is at least 1 chip of each color
 Exercise 3
(Poker hands) Consider poker hands drawn from a
well shuffled deck.
a. one-pair
b. three of a kind
c. full house
 Exercise 4
(Hard) A secretary types three letters and three
corresponding envelopes. In a hurry she places
at random one letter in each envelope. What is
the probability that at least one letter is in the
correct envelope?
 Conditional Probabilities
• The probability that A occurs, given that event B
has occurred is called the conditional probability
of A given B and is defined as
PP((AA  BB))
PP((AA|| BB)) 
 ifif PP((BB)) 
00
PP((BB))

Two
Two events,
events, A A and
and B,
B, are
are said
said to
to be
be independent
independent ifif
and
and only
only ifif the
the probability
probability that
that event
event A A occurs
occurs does
does
not
not change,
change, depending
depending on
on whether
whether or or not
not event
event BB
has
has occurred.
occurred.
 Conditional Probabilities
Ex8: Toss a fair coin twice.
Define A: head on second toss, B: head on first toss
P(A|B)
P(A|B) == ½
½ A and B are
P(A|not independent!
P(A|not B)
B) == ½
½

Two
Two events
events A
A and
and BB are
are independent
independent ifif and
and only
only ifif
P(AB)
P(AB) == P(A)
P(A) or or P(B|A)
P(B|A) == P(B)
P(B)
or P(A B)
or P(A B) == P(A)
P(A) P(B)
P(B)
Otherwise,
Otherwise, they
they are
are dependent.
dependent.
Ex9: In a certain population, 10% of the people can be
classified as being high risk for a heart attack. Three
people are randomly selected from this population.
What is the probability that exactly one of the three are
high risk?
Define H: high risk N: not high risk

P(exactly
P(exactly one
one high
high risk)
risk) == P(HNN)
P(HNN) ++ P(NHN)
P(NHN) ++ P(NNH)
P(NNH)
== P(H)P(N)P(N)
P(H)P(N)P(N) ++ P(N)P(H)P(N)
P(N)P(H)P(N) ++ P(N)P(N)P(H)
P(N)P(N)P(H)
== (.1)(.9)(.9)
(.1)(.9)(.9) + (.9)(.1)(.9) + (.9)(.9)(.1)= 3(.1)(.9) == .243
+ (.9)(.1)(.9) + (.9)(.9)(.1)= 3(.1)(.9) 22
.243
 The Law of Total Probability
• Let S1 , S2 , S3 ,..., Sk be mutually exclusive and
exhaustive events (or a partition). Then the
probability of another event A can be written as

P(A) P(A 
P(A) == P(A P(A 
 SS11)) ++ P(A  SS22)) ++ … P(A 
… ++ P(A  SSkk))
== P(S
P(S11)P(A|S
)P(A|S11)) ++ P(S
P(S22)P(A|S
)P(A|S22)) ++ …
… ++ P(S
P(Skk)P(A|S
)P(A|Skk))
 The Law of Total Probability
S1

A Sk
A

A  S1
Sk
S2….

Let S1 , S2 , S3 ,..., Sk be mutually exclusive and exhaustive

events (or a partition) of the sample space.

P(A) P(A 
P(A) == P(A P(A 
 SS11)) ++ P(A  SS22)) ++ … P(A 
… ++ P(A  SSkk))
== P(S
P(S11)P(A|S
)P(A|S11)) ++ P(S
P(S22)P(A|S
)P(A|S22)) ++ …
… ++ P(S
P(Skk)P(A|S
)P(A|Skk))
 Bayes’ Rule
• Let S1 , S2 , S3 ,..., Sk be mutually exclusive and
exhaustive events with prior probabilities P(S 1),
P(S2),…,P(Sk). If an event A occurs, the posterior
probability of Si, given that A occurred is

PP((SSii))PP((AA|| SSii))
PP((SSii || AA)) 
 for ii 
for  11,, 22,...k
,...k

 PP((SSii))PP((AA|| SSii))
Note: Think about the definition of conditional probability and
the rule of total probability.
Ex 10: Suppose that 49% of the population are female.
Of the female patients, 8% are high risk for heart
attack, while 12% of the male patients are high risk. A
single person is selected at random and found to be
high risk. What is the probability that it is a male?

Define H: high risk F: female M: male

 Exercise 5
(Birthday Problem) Two people enters a room and their
birthdays (ignoring years) are recorded.
a. Identify the sample space .
b. What is the probability that both people
have the same birthday.
If 40 people enter a room, find the probabilities:
c. None of the people have the same birthday.
d. At least two of the people have the same birthday.
 Exercise 6
(Screening Tests) Suppose that a certain disease is present in
10% of the population, and that there is a screening test
designed to detect this disease if present. The test does not
always work perfectly.
Sometimes the test is negative when the disease is present, and
sometimes it is positive when the disease is absent.

Test is Positive(P) Test is Negative(N)

Disease Present(D) 0.88 0.22
Disease Absent(D^c) 0.05 0.85
 Exercise 6(continued)
a. P(D|N)
b. Find the probability of a false positive, that the test is
positive, given that the person is disease-free.
c. Find the probability of a false negative, that the test is
negative, given that the person has the disease.
d. Are either of the probabilities in parts b or c large enough
that you would be concerned about the reliability of this
screening method? Explain.
 Random Variables
• A function X, which assigns to each element c  one
and only one number X(c) = x is called a random
variable.

• If the range of X, {x : X(c)=x, c } is a countable set,

X is a discrete random variable, otherwise a continuous
random variable.
Ex11: X = length of a 2-year –old black bass
X = number of people in a room at a randomly
selected time of day
X = total number of points scored in a football
game
 Discrete Distributions
• X : discrete rv, pmf p(x)

X takes on only a finite or countably number of values and

p (x) : the probability of X  x

0 p(x) 1 and 
all possible x
p(x) 1
mean  E(X)

variance E(X -  ) E(X ) - {E(X)}

2 2 2 2

 x 2 p ( x) - { xp(x)}2
Ex12: Toss a fair coin three times and
define X = number of heads.
x
HHH
HHH x p(x)
1/8 3 P(x
P(x == 0)
0) == 1/8
1/8 0 1/8
HHT
HHT
1/8 2
P(x
P(x == 1)
1) == 3/8
3/8 1 3/8
HTH
HTH P(x
P(x == 2)
2) == 3/8
3/8 2 3/8
1/8 2 P(x
THH
THH
P(x == 3)
3) == 1/8
1/8 3 1/8
1/8 2
HTT
HTT 1/8 1 Probability
Probability
THT
THT Histogram
1/8 1 Histogramfor
forxx
TTH
TTH 1/8 1
TTT
TTT 1/8 0
 Continuous Random Variables
• probability density function(pdf) of X, f(x)


0  f ( x ) and f(x)dx 1
-

P ( a  X  b)  P ( a  X  b)  P ( a  X b)  P ( a  X b)
b
 f ( x ) dx
a

P ( X  x ) 0 for all x

f ( x ) x  P(x  X x  x) for small x

d d
f(x)  F ( x)  P( X  x)
dx dx
Expectation of a random variable
mean :   E ( X )  
all posible x
xp(x), for discrete rv


 xf(x)dx,
-
for continuous rv

var iance :  2

 E (X -  ) 2    (x -  ) 2 p ( x )
all possible x

2
 
  x p( x) -  
2
xp ( x ) 
all possible x  all possible x 

 E(X 2 ) - E(X)
2

s tan dard deviation : 

 Probability Distributions
• Probability distributions can be used to describe the
population.
– Shape: Symmetric, skewed, mound-shaped…
– Outliers: unusual or unlikely measurements
– Center and spread: mean and standard deviation.
A population mean is called  and a population
standard deviation is called 
Ex 13: Toss a fair coin 3 times and record x the
number of heads.

x p(x) xp(x) (x-2p(x)

12
12 1.5
0 1/8 0 (-1.5) (1/8)
2
  xp ( x ) 
 xp( x)  1.5
1 3/8 3/8 (-0.5)2(3/8)
88
2 3/8 6/8 (0.5)2(3/8)
3 1/8 3/8 (1.5)2(1/8)
 
22
((xx )) pp((xx))
22

 22  28125..09375
..28125 09375..09375
09375..28125
28125..75
75
  75 
 ..75 ..688
688
 Example
• The probability distribution for x the number of
heads in tossing 3 fair coins.

Symmetric;
• Shape? mound-shaped
• Outliers? None
• Center?  = 1.5
• Spread?  = .688


Ex14 : Toss two fair coins and let X equal the number
of heads observed.
a. Find the probability distribution of X
b. Find E(X)
c. Find the variance of X

Ex15: Two distinct integers are chosen at random and

without replacement from the first six positive integers.
Compute the expected value of the absolute value of
the difference of these two numbers.