1

1. Probability

2. Random variables

3. Some important theoretical probability distributions

• Normal distribution, Student’s t-distribution,

• Chi-square distribution, F distribution

4. Statistical Inference: Estimation

• Point Estimation
• Interval Estimation (Confidence Interval)

5. Statistical Inference: Hypothesis Testing

2
1.1 Probabilities and Events
1.2 Conditional Probability
1.3 Random Variables and Expected Values
1.4 Covariance and Correlation

3
Section 1.1
Probabilities and Events

4
Consider a random experiment
 The sample space, S, is the set of all possible
outcomes of the experiment.
 If there are m possible outcomes of the
experiment then we will generally number them
1 through m. Then S ={1, 2, …, m}
 When dealing with specific examples, we will
usually give more descriptive names to the
outcomes.

5
 Definitions
❖ Event any collection of results or outcomes of an
experiment.
❖ Simple Event an outcome or an event that cannot
be further broken down into simpler
components.
❖ Sample Space for an experiment consists of all
possible simple events; that is, the sample space
consists of all outcomes that can not be broken
down any further.

6
P denotes a probability.

A, B, C and E denote specific events.

P(A) denotes the probability

of event A occurring.

7
Assume that a given procedure has n different
simple events and that each of those simple events
has an equal chance of occurring. If event A can
occur in s of these n ways, then

s Number of Ways A can occur

P( A) = =
n Number of different simple events

8
❖ The probability of an impossible event is 0.
❖ The probability of an event that is certain to
occur is 1.
❖ For any event A, the probability of A is between
0 and 1 inclusive. That is:

0  P( A)  1

9
Let S = {1, 2, …, m}. Let pibe the probability that i is
the outcome of the experiment. Then:

0  pi  1, i = 1,2, ... m
m

 i
p
i =1
= 1

10
For any event A,
m
P ( A) =  pi
i A

m
P( S ) =  pi = 1
i =1

11
Possible Values
for Probabilities

12
Roll two dice. What
is the chance that
the sum is:
Equal to 7?
Equal to 2?
Even? Odd?

13
 Example: Sum of two dice

Roll two dice. What

is the chance that
the sum is:
Equal to 7?

14
 Example: Sum of two dice

Roll two dice. What

is the chance that
the sum is:
Equal to 7?
Equal to 2?

15
 Example: Sum of two dice

Roll two dice. What

is the chance that
the sum is:
Equal to 7?
Equal to 2?
Even?
18/36 = 1/2

16
 Example: Sum of two dice

Roll two dice. What

is the chance that
the sum is:
Equal to 7?
Equal to 2?
Even?
Odd?
18/36 = 1/2

17
Events A and B are disjoint (or mutually exclusive) if
they cannot occur at the same time. (That is,
disjoint events do not overlap.)

18
P(A∪B) = P(A) + P(B) – P(A∩B)

Where P(A∩B) denotes the

probability that A and B
both occur at the same
time as an outcome in a
trial or procedure.

19
For mutually exclusive events A, B
P(AUB) = P(A) + P(B)

20
 c
The complement of event A, denoted by A ,
consists of all outcomes in which the event A
does not occur.

21
P( A) + P( A ) = 1
c

P( A ) = 1 − P( A)
c

P( A) = 1 − P( A )
c

22
23
 𝑃(𝐵 ∩ 𝐴)
𝑃 𝐵|𝐴 =
𝑃(𝐴)

P(B|A) represents the probability of

event B occurring after it is assumed
that event A has already occurred (read
B|A as “B given A.”)

24
P( A  B) = P( A) • P( B | A)

25
 Two events A and B are independent of
P(B|A) = P(B)
 A and B are independent if the occurrence of one
does not affect the probability of the occurrence
of the other. In this case,
P(A∩B) = P(A)  P(B)
 If A and B are not independent, they are said to
be dependent.

26
To find the probability of at least one of something,
calculate the probability of none, then subtract that
result from 1. That is,

P(at least one) = 1 – P(none)

27
 If there are n students in the class, what is the
chance that at least two of them have the same
birthday?
Solution: P(at least 2 have same birthday)
= 1 – P(No coinciding birthdays)
Let Bi be the birthday of student number i.
The probability of no coinciding birthdays is:

P( B2 {B1}, B3 {B1 , B2 },...Bn {B1 ,...., Bn−1}

28
P(at least 2 have same birthday)
= 1 − P( No coinciding birthdays )
1 2 n −1
= 1 − 1 • (1 - )(1 − )...(1 − )
365 365 365
Q: How can we compute this for large n?
A: Approximate!

29
Log(P(No coinciding birthdays))
1 2 n −1
= log((1 − )(1 − )...(1 − ))
365 365 365
1 2 n −1
= log(1 − ) + log(1 − ) + ... + log(1 − )
365 365 365
1 2 n −1
− − − ... −
365 365 365
1 1
=− ( n(n − 1))
365 2

30
P(No coinciding birthdays)

n ( n - 1)
−
e 2365

P(At least 2 have same birthday)

n ( n - 1)
−
 1− e 2365

31
n
32
If B1 ,..., Bn is a disjoint partition of S , then

P ( A) = P( AB1 ) + P ( AB2 ) + ... + P( ABn )

= P( A | B1 ) P( B1 ) + P( A | B2 ) P( B2 ) + ... + P( A | Bn ) P( Bn )

33
Section 1.3
Random Variables and
Expected Values

34
❖ Random variable
Numerical quantities whose values are determined
by the outcome of the experiment are known as
random variables.
❖ Discrete random variable
RV takes either a finite number of values or a
countable number of values.

35
IfXis a random variable whose possible values are
x1 , x2 ,..., xn ,
then the set of probabilities
P{ X = xi }( j = 1,..., n)
is called the probability distribution of the random
variable. n

 P( X = x ) = 1
i =1
i

36
The expectation of a discrete random variable X is
denoted by E[X], and it represents the (weighted)
average value of the possible values of X.

n
[ X ] =  xi P( X = xi )
i =1

37
Suppose a, b are constants and X is a RV.
Then
E[aX + b] = aE[ X ] + b

38
Let X 1 , X 2 ,...., X n be RVs. Then

 n
 n
E  X i  =  E[ X i ]
 i =1  i =1

39
A random variable X, which is equal to 1 with
probability p and to 0 with probability 1-p, is said to be
a Bernoulli random variable with parameter p. its
expected value is

E[ X ] = 1 p + 0  (1 − p) = p

40
 The experiment has a fixed number of trials, n.
 The trials must be independent.
 Each trial must have all outcomes classified into two
categories (commonly referred to as success and
failure).
 The probability of a success remains the same, say p,
in all trials.

41
Let X be the total number of successes that occur in a
binominal experiment. X is called a binominal random
variable with parameters n and p. Let X j be the
number of successes in trial j. Then X j is a Bernoulli RV
and n
X =Xj
j =1
n n
E[ X ] =  E[ X j ] =  p = np
j =1 j =1

42
Rationale for the Binomial
Probability Formula
n! n− x
P( x) = • p • (1 − p)
x

(n − x)! x!
The number of outcomes The probability of x
with exactly x successes successes among n trials
among n trials for any one particular
order

43
E ( XY ) = {all ( x , y )} xyP( X = x, Y = y )
E ( XY ) =  x  y xyP( X = x, Y = y )
Is E ( XY ) = E ( X ) E (Y )?
Theorem: If X and Y are independent, then:

E ( XY ) = E ( X ) E (Y )

45
Proof: If X and Y are independent,
P( X = x, Y = y) = P( X = x) P(Y = y )
then
E ( XY ) = x  y xyP( X = x) P(Y = y )

= (x xP( X = x))( y yP(Y = y ))

= E ( X ) E (Y )

46
The variance of X, denoted by Var(X) is the mean
squared deviation of X from its expected value
 = E (X ) :
Var ( X ) = E[( X −  ) ] 2

The standard deviation of X, denoted by SD(X) is the

square root of the variance of X:

SD( X ) = Var ( X )

47
Claim: Var ( X ) = E ( X ) − E ( X )
2 2

Proof: E[( X −  ) = E[ X − 2 X +  ]
2 2 2

= E[ X ] − 2E[ X ] + 
2 2

= E[ X ] − 2 + 
2 2 2

= E[ X ] − E[ X ]
2 2

48
Claim: If X 1 , X 2 ,..., X n are independent

then

Var ( X 1 + X 2 + ... + X n )
= Var ( X 1 ) + Var ( X 2 ) + ... + Var ( X n )

49
 Proof: Suffices to prove for 2 random variables.

E[( X + Y − E ( X + Y )) ] = E[( X − E ( X ) + Y − E (Y )) ]
2 2

 
= E ( X − E ( X )) + 2 E[( X − E ( X ))(Y − E (Y ))] + E[(Y − E (Y )) ]
2 2

= Var ( X ) + Var (Y ) + 2 E[( X − E ( X ))]E[(Y − E (Y ))]

= Var ( X ) + Var (Y ) + 0

50
For Bernoulli random variable X,
Var [ X ] = E[( X − E[ X ]) ] 2

= E[( X − p) 2

= (1 − p)  p + (0 − p) (1 − p)
2 2

= (1 − p)  p  [(1 − p) + p]
= p(1 − p)
51
Let X be a binomial random variable with
parameters n and p. Let X j be Bernoulli RV
representing the number of successes in trial j.
Then X 1 , X 2,..., X n are independent RVs and
n
X =Xj
j =1
n n
Var ( X ) =  Var ( X j ) =  p(1 − p) = np (1 − p)
j =1 j =1

52
Section 1.4
Covariance and Correlation

53
Cov( X ,Y ) = E[( X − E[ X ])(Y − E[Y ])]

54
 Cov ( X , Y ) = E[ XY ] − E[ X ]E[Y ]
Proof:
Cov ( X , Y ) = E[( X − E[ X ])(Y − E[Y ])]
= E XY − E[ X ]Y − E[Y ] X + E[ X ]E[Y ]
= E[ XY ] − E[ X ]E[Y ] − E[ X ]E[Y ] + E[ X ]E[Y ]
= E[ XY ] − E[ X ]E[Y ]

55
Cov( X , Y ) = Cov(Y , X )
Var ( X ) = Cov( X , X )
Cov(cX , Y ) = c  Cov ( X , Y )
Cov(c, Y ) = 0

56
  X − E ( X ) Y − E (Y ) 
 ( X , Y ) = Corr ( X , Y ) = E   
 SD( X ) SD(Y ) 

57
Using the linearity of Expectation we get:

 (aX + b, cY + d ) =  ( X ,Y )
Correlation is invariant of scale change!

58
 (X − X ) (Y − Y )
Let X = *
and Y =
*
then
SD( X ) SD(Y )
E ( X ) = E (Y ) = 0 and SD (X ) = SD (Y ) = 1
* * * *

Corr ( X , Y ) = Cov ( X , Y ) = E ( X Y )
* * * *

59
 ( X , Y ) = 0  Cov( X , Y ) = 0
 E[ XY ] = E[ X ]E[Y ]

60
 −1   ( X ,Y )  1
Proof :
E ( X ) = E (Y ) = 1
*2 *2

0  E( X − Y ) = 1 +1+ 2 − E( X Y )
* * 2 * *

0  E ( X + Y ) = 1 + 1 + 2E ( X Y )
* * 2 * *

−1  E( X Y )  1
* *

61
62
Let X be the distance traveled by
a golf-ball hit by Tiger Woods.
Then X is a continuous random
variable.

63
  A continuous distribution is determined by a
probability density function f(x).
 Probabilities are defined by areas under the
graph of f ( x)  0
 The total area under the curve must equal 1.

b
P(a  X  b) =  f ( x)dx
a

64
Continuous Discrete

b
P(a  X  b) =  f ( x)dx P(a  X  b) = a X b P( X = x)
a

65
 Continuous  Discrete
• Expectation of a function g(X)

E[ g ( x)] =  g ( x ) f ( x ) dx E[ g ( X )] =  g ( x) P( X = x)
− all x

• Variance and SD

Var ( X ) = E[( X − E ( X )) ] 2

SD( X ) = Var ( X )

66
 Definition: For a random variable X, the function

F ( x) = P( X  x)
is called the cumulative distribution function (CDF)

 A distribution is called continuous when the CDF is

continuous.

67
 x
F ( x) = 
−
f (t )dt

b
P(a  X  b) =  f ( x)dx = F (b) − F (a)
a

68
69
Definition: A continuous random variable Z has a
standard normal distribution if Z has a probability
density of the following form:

( z )2
1 −
f ( z) = e 2
, ( −   z  )
2

70
The standard normal distribution is a
probability with mean equal to 0 and standard
deviation equal to 1, and the total area under
its density curve is equal to 1.

71
 2
( x)
 −
1 2


− 2
e dx = 1

 ( x) 2
1 −
E (Z ) =  x e 2
dx = 0
− 2
 ( x )2
1 −
Var ( Z ) =  dx = 1
2 2
x e
− 2
72
Z score will be positive
Finding the 95 th Percentile

73
 Finding z Scores
When Given Probabilities

Z score will be positive 1.645

Finding the 95 th Percentile

74
(One z score will be negative and the other positive)

Finding the Bottom 2.5% and Upper 2.5%

75
Finding z Scores When Given
Probabilities - cont

-1.96 1.96
(One z score will be negative and the other positive)

Finding the Bottom 2.5% and Upper 2.5%

76
77
Definition: If Z has a standard normal distribution
and  and   0 are constants then
X = Z + 
has a normal distribution with mean  and
standard deviation  .

78
E ( X ) = E (Z +  ) = E ( Z ) +  = 
Var ( X ) = Var (Z +  ) =  E ( Z ) = 
2 2

79
80
81
P ( c  X  d ) = P (c   Z +   d )
= P((c −  ) /   Z  (d −  ) /  )

82
 Standard Normal Distribution
0.4

f(z) 0.3

0.2

0.1
1.56

{
0.0
-5 -4 -3 -2 -1 0 1 2 3 4 5
Z

P(0  z  1.56) =
0.4406
P (1  Z  2) = 0.1359

Standard Normal Dis ribution

0.4

0.3
f(z)

0.2

0.1

0.0
-5 -4 -3 -2 -1 0 1 2 3 4 5
Z
 4-
87

Standard Normal Distribution

To find z such that
0.4

P(0  Z  z) = .40:
0.3

f(z)
0.2

P(0  Z  1.28)  .40 0.1

0.0
-5 -4 -3 -2 -1 0 1 2 3 4 5
Z
 4-
88

Example 4-5

X~N(160,302) P (100  X  180 )

 100 −  X −  180 −  
= P   ÷
    
 100 − 160 180 − 160 
= P  Z  ÷
 30 30 
(
= P − 2  Z  .6666 )
= 0 .4772 + 0 .2475 = 0 .7247
 4-
89

Example 4-6
P ( X  150 )
X~N(127,222)
X − 150 −  
= P  ÷
   
 150 − 127 
= P Z  ÷
 22 
(
= P Z  1.045 )
= 0.5 + 0.3520 = 0.8520
90
91