M131-Lecture Notes No. 4
M131-Lecture Notes No. 4
Smadi
Continuous Distributions
and
Normal Distribution
1
Continuous Probability Distributions
Since the continuous random variables are
uncountable, it is difficult to write down probabilities
of all possible events.
Instead, an equation or formula is used to describe a
continuous probability distribution.
The equation used to describe a continuous probability
distribution is called a probability density function
(p.d.f).
2
Properties of probability density function
3
Example: Given
2 x 0 x 1
f ( x)
0 otherwise
a. Verify that f(x) is a legitimate probability density
function
Answer:
1. f(x) ≥ 0?
for 0 x 1, f ( x) 0
2. The area under the p.d.f equals 1?
0 1
f ( x) dx
0 dx
2x dx
0
0 dx
1
0 x
2 1
0 0 1
4
b. Find
1 1
P X ?
2 2
1/ 2
1 1
P X
2 2
f ( x) dx
1 / 2
0 1/ 2
0 dx
1 / 2
2 x dx
0
1/ 2
x2 1
2
2 0 4
5
Example: Given the following p.d.f
cx 2 for 0 x 1
f ( x)
0 otherwise
a. Find the value of c?
Answer:
f ( x) dx 1
1 3
x
0 dx c 3
1
cx 2
0
1
c
1
3
then c = 3
1 2
x3 1
p (0 X ) 3 x dx 3
2
2 0
3 0
8
6
Expected Value (Mean):
7
Definition:
Let X be a random variable. Let g(X) be a function of X.
Then the expected value of g(X) is defined by
E ( g ( X )) g ( x) f ( x)dx
Provided that the integral exists.
Example:
E( X 2 ) f ( x)dx
2
x
Result: E(a+bX) = a + b E(X)
Result: If X and Y are random variables, then
E(X+Y) = E(X) + E(Y)
Variance:
2 E ( X ) 2 ( x ) 2 f ( x)dx
Result:
2 E ( X 2 ) ( )2
where σ: standard deviation of X (or σX)
8
Example: Given the following p.d.f
3x 2 for 0 x 1
f ( x)
0 otherwise
Find σ2?
2 E ( X 2 ) ( E ( X )) 2
1 1
E( X ) dx
2 3
x (3 x ) dx 3 x
0 0
x4 3
3
1
0
4 4
1 1
E ( X 2 ) x 2 (3x 2 )dx 3x 4 dx
0 0
x5 3
3
1
0
5 5
3 3 3
2 E ( X 2 ) ( ) 2 ( )2
5 4 80
Result:
Var(a+bX) = b2 Var(X)
Where a and b are real constants
9
Some Continuous Distributions
1- Uniform Distribution
2- Normal Distribution
1.Uniform Distribution:
The probability density function of the uniform random
variable X over the interval [a, b] is:
Denoted, X: U(a, b)
10
Result:
ab
1. E ( X )
2
(b a ) 2
2. Var ( X )
12
Proof:
E( X ) xf ( x)dx
b
1 x2
E( X ) x dx
b
a
ba 2(b a) a
b2 a 2 (b a )(b a )
E( X )
2(b a ) 2(b a)
(b a )
E( X )
2
Standard uniform distribution
Restricting a=0 and b=1, the resulting distribution U(0,1)
is called a standard uniform distribution, it is very
important in generating random numbers.
11
Example: Metro trains are scheduled every 5 minutes at
a certain station. A person comes to the station at a
random time. Let the random variable X count the
number of minutes he/she has to wait for the next train.
Assume X has a uniform distribution over the interval (0,
5). Find the probability that he/she has to wait at least
three minutes for the train.
Answer: As X follows uniform distribution over the
interval (0, 5)
then the p.d.f of X is
1 1 1
f ( x) , 0 x5
ba 50 5
Thus, the desired probability
5
P( X 3) f ( x)dx
3
5 5
1 1
dx (1)dx
3
5 53
1 5 1
x | (5 3)
5 3 5
= 0.4
12
2. Normal Distribution
Errors in measurements.
13
A continuous random variable X follows a normal
distribution if it has the following probability density
function (p.d.f.):
( x )2
1
f ( x) e 2 2
, x ,
2 2
, 2 0
f ( x ) dx 1
14
Result: It can be shown that, if X ~ N(µ ,σ2)
E(X) = µ
Var(X) = σ2
So, µ is the mean of the distribution and σ2 is the variance.
µ is called location parameter and σ2 is called shape
parameter.
15
Properties of Normal Distribution:
- Symmetric Bell-shaped
- Unimodal
- Extends to +/- infinity.
- Area under the curve = 1
- Symmetric about the center (mean)
- Mean = Median = Mode
- Extremely-large values and extremely-small values
are rare and occur near the tail ends
- Most-frequent values are clustered around the mean
and fall off smoothly in either side of it.
16
-
17
18
The Standard Normal Distribution
A standard normal distribution is a normal distribution
with mean 0 and standard deviation 1, Z: N(0,1). The
probability density function is
z2
1
f ( z) e 2
- z
2
19
Example-1: Find P(Z ≤ 1.18)?
From z-table
z .00 .01 ... .08 .09
0.0 .5000 .5040 ... .5319 .5359
0.1 .5398 .5438 ... .5714 .5753
... ... ... ... ... ...
1.0 .8413 .8438 ... .8599 .8621
1.1 .8643 .8665 ... .8810 8830
1.2 .8849 .8869 ... .8997 .9015
... ... ... ... ... ...
20
Z-table
21
22
Example-2: Find P(Z >-1.48)?
23
Example-4: Find the z value such that P(Z>z) = 0.20
24
Result:
If X ~ N(µ ,σ2), to find P(a<X<b)
Draw the normal curve in terms of X
X
Z : N (0,1)
Use the standardized normal table
25
c. Find the 90-th percentile?
i.e. find c such that P(X ≤ c) = 0.9
X 85 c 85
P( ) 0.9
5 5
c 85
P( Z a) 0.9, where a
5
From z-table a = 1.28
c 85
1.28
5
Then c = 91.4 is the 90th percentile
27
28
Example-1: The random variable X has a normal distribution with mean
1 and variance 20. Given that P(X<k) = 3 P(X>k), find k?
Answer:
29
Example: The random variable X has a normal distribution
N(μ, σ2). Given that P(X<10) = P(X>20) = 0.330, find μ and σ2?
Answer:
X⁓N(μ, σ2)
As normal distribution is symmetrical about the mean μ
P(X<10) = P(X>20) μ = (10+20)/2 =15
P(X>20) = 0.330
P(X<20) = 0.67
X 20 15
P 0.67
5
P Z 0.67
P Z c 0.67
where
5
c
P Z c 0.67
from the z-table c = 0.44
5
0.44
then σ = 11.36
30
Example: The random variable X has a normal distribution
N(μ, σ2). Given that P(X<40.0) = 0.05 and P(X<70.0) = 0.975,
find μ and σ2?
Answer:
P(X<40.0) = 0.05
X 40
P 0.05
40
P Z 0.05
from the z-table
40
1.645
1.645 40 .0 (1)
X 70
P(X<70.0) = 0.975 P 0.975
70
P Z 0.975
from the z-table
40
1.96
1.96 70 .0 (2)
Solving the following two equations:
1.645 40 .0 (1)
1.96 70 .0 (2)
3.605σ = 30
σ = 8.322
from (1)
1.96 70 .0
1.96 8.322 70 .0
16 .311 70 .0
μ = 53.689
31
Properties of Normal Distribution:
32
For independent X⁓N(μ1, σ12) and Y⁓N(μ2, σ22), a, b are
constants
E(aX±bY) = a E(X) ± b E(Y) = aμ1 ± bμ2
Var(aX±bY) = a2Var(X) +b2 Var(Y)
= a2 σ12+b2 σ22
aX + bY⁓N(aμ1 + bμ2, a2 σ12+b2 σ22)
Example:
The independent random variables X and Y have the
distributions
N(10, 9) and N(12, 16). Find P(2X>Y+Y)?
Answer:
P(2X>Y+Y) = P(2X-Y-Y>0)
E(2X-Y-Y) = 2(X) - 2E(Y) = 2(10) - 2E(12) = -4
Var(2X-Y-Y) = 22Var(X) + Var(Y) + Var(Y)
Var(2X-Y-Y) =22Var(X) + 2Var(Y)
= 4(9) + 2(16) = 68
2X-Y-Y⁓N(-4, 68)
P(2X-Y-Y>0) = 1- P(2X-Y-Y<0)
2 X Y Y (4) 0 (4)
1 P
68 68
4
1 P Z
68
33
Sampling Distribution
of the Sample Mean
34
Suppose that a sample of size n is drawn from a population
with mean μ and variance σ2. If the sample points are
denoted by X1,X2,…,Xn then the sample mean is
n
X X 2 ... X n X i
X 1 i 1
n n
Notice that the sample mean is a random variable, since it
depends of the sample drawn!
A common estimator for μ is the sample mean is X
X X 2 ... X n X i
X 1 i 1
n n
is an Estimator for the population mean μ.
35
How good the estimator X ?
Look to its probability distribution!
Important properties of the distribution of the sample
mean X
1. The mean or expected value of X ?
2. Variance or Standard Deviation of X ?
3. Shape of the distribution of X ?
18 18 19 20 20 21 58
19.33
6 3
N 6 2
58
( xi )
2
xi
3
2 i 1 i 1
N 6
58 2 58 58
(18 ) (18 ) 2 ... (21 ) 2
2 3 3 3
6
≈ 1.22
36
Select a samples of size 2 (n = 2). There are 15 possible
samples.
Find the mean of every possible sample where n = 2:
37
Sampling distribution for the sample mean X
38
The mean (expected value) of the sample X
i X
( X ) 2
Var( X ) X2 i 1
15
(18 19.33) 2 (18.5 19.33) 2 ... (20.5 19.33) 2
15
= 0.488
N n2
Var( X ) 2
X
N 1 n
39
If
n
0.05
N
then
2
Var( X ) X2
n
Example:
N n 2 6 2 1.22
Var( X ) 2
0.488
X
N 1 n 6 1 2
Theorem: If a random sample of size n is drawn from a
population with mean µ and variance σ2, then
(1) The mean or expected value of X is µ
2
(2) The variance of X , Var( X ) = n
(3) If the population distribution is normal then the
sample mean is normally distributed.
2
X : N ( , )
n
40
or
X
Z : N (0,1)
/ n
Very useful result in interval estimation!
41
Example: If a sample of size n = 25 is taken from a
normal distribution of µ=10 and σ=2, find P( X 9) ?
Answer:
P( X 9) 1 P( X 9)
Using
2
X : N ( , )
n
P( X 9) 1 P( X 9)
X 9 10
1 P
/ n 2 / 25
9 10
1 P Z
2/5
1 PZ 2.5
= 1 - 0.00621
= 0.99379
42
Result :
In statistics we call X1,…, Xn a random sample if
1. X1,…, Xn are independent.
2. X1,…, Xn all have the same distribution
If E(Xi)= µ and var( Xi) = σ2
Then
The Sample Sum ∑𝑛𝑖=1 𝑋𝑖 :
𝐸(∑𝑛𝑖=1 𝑋𝑖 ) = nµ and 𝑣𝑎𝑟(∑𝑛𝑖=1 𝑋𝑖 ) = nσ2 .
The Sample Mean : 𝑋̅ = 𝑛1 ∑𝑛𝑖=1 𝑋𝑖 :
E( X )
2
Var( X ) 2
x
n
S .D( X ) X
n
43
Result:
E( X )
X is an unbiased estimator for µ,
Proof:
1 n
E( X ) E X i
n i 1
1 n
E( X i )
n i 1
1 n
n i 1
1
n
n
44
Result:
2
Var ( X )
n
Proof.
1 n
Var( X ) Var X i
n i 1
1 n
2 Var ( X i )
n i 1
1 n
n 2
2
n
i 1
2
2
n
2
n
Random samples and the normal distribution
If X1,…, Xn are independent and Xi ~ N(µ, σ2) (i = 1,…,n)
then
The sum ∑𝑛𝑖=1 𝑋𝑖 ~ N(nµ, nσ2)
̅ 𝜎2
The sample mean 𝑋 ~ N(µ, 𝑛 )
45
Example: Suppose a random sample of size 25
observations is selected from a population that is normally
distributed with mean 106 and standard deviation 12 find
a. The mean and standard deviation of the sampling
distribution of the sample mean?
mean of X is 106
12
S.D of X 2.4
n 25
b. Find P(( X ) 4)
( X ) 4
P( )
/ n 12 / 25
P ( Z 1.67 ) 0.9525
46
The Central Limit Theorem
النظرية االساسية في تقارب التوزيعات االحتمالية
If X1, X2 ,… are independent and E(Xi )= µ and var(Xi )=
σ2, i = 1, 2,… then for large n
the sampling distribution of the sample mean X
approximately
2
𝑋̅ ~ N(µ, 𝜎 )
𝑛
The sampling distribution of the sample sum
approximately
S n N ( n , n 2 )
where Sn= X1+ X2+…+ Xn (Sample Sum)
47
48
Why CLT?
The normal distribution is important and curtail
distribution because of CLT, which states that the
distribution of the sum (or average) of a large number of
independent, identically distributed random variables
approaches a normal distribution, regardless of the original
distribution of the variables. This makes the normal
distribution a fundamental tool is statistical inference and
hypothesis testing
49
Normal Approximation to Binomial
Shape of the Binomial Distribution
The shape of the binomial distribution depends on the
values of n and p.
51
Example: Let X: B(100, 1/2)
Find P(20 X 80)?
80
100 i
0.5 (1 0.5)100i
i 20 i
Not easy!
Using CLT
To find P(20 X 80)?
Using the Normal approximation
a np X np b np
P( )
np(1 p ) np(1 p ) np(1 p )
a np b np
P( Z )
np(1 p ) np(1 p )
Using the z-table, we get the answer
52
Continuity correction factor by approximating
discrete distribution by continuous distribution!
Example: Suppose we toss a fair coin 20 times. Let X be
the random variable representing the number of heads
thrown. X ~ B(20, ½)
In this diagram, the rectangles represent the binomial
distribution and the curve is the normal distribution:
We want P(9 ≤ X ≤ 11), which is the red shaded area. Notice that
the first rectangle starts at 8.5 and the last rectangle ends at 11.5.
Using a continuity correction, therefore, our probability becomes
P(8.5 < X < 11.5) in the normal distribution.
53
Example: For example, Use normal approximation to a
binomial distribution with n 10 and p 0.5
Using normal approximation to find
P(3 ≤ X ≤ 7) ?
= P(2.5 ≤ X ≤ 7.5)
np = 5, np(1-p) = 2.5
2.5 np 7.5 np
P(2.5 X 7.5) P( Z ) P(1.58 Z 1.58)
np(1 p) np(1 p)
= 0.8858
Exact Answer
7
10
0.5i (1 0.5)10i
i 3 i
= 0.8996
54
Exercises
Q3: If X:N(260, 400), then for a random sample of size n = 10, the sampling distribution
of X is
20
(A) N(260, 40) (B) N(260, 20) (C) N (260, ) (D) None of the above
10
55
Q4: Which of the following statements describes the central limit theorem
A) For a random sample from any population with mean μ and variance σ2 and any
sample size the sampling distribution of the sample mean is exactly normal
distribution.
B) For a random sample from any population with mean μ and variance σ2 and large
sample size the sampling distribution of the sample mean is exactly normal
distribution.
C) For a random sample from any population with finite mean μ and variance σ2 and
large sample size the sampling distribution of the sample mean is approximately
normal distribution.
D) None of the above
56
Q10: In order to be accepted into a top university, applicants must score within the top 2.5%
on the SAT exam. Given that the test has a mean of 1000 and a standard deviation of
200, what is the lowest possible score a student needs to qualify for acceptance into the
university?
A) 1392
B) 1400
C) 1300
D) None of the above
Q12: Which of the following statements is not a property of the normal distribution?
(A) Bell shaped
(B) Symmetric about μ
(C) The mean and median may be different
(D) None of the above
Q14. Let Z be the standard normal random variable, then the value of P(-2.16 ≤ Z ≤ 0.55) =
(A) 0.2758 (B) 0.6934 (C) 0.4846 (D) None of the above
Q18: The time required to complete a final examination of MATH 235 is normally distributed,
with a mean of 80 minutes and a standard deviation of 10 minutes. Assume that the
examination period is 90 minutes in length, then the probability that a randomly selected
student will be unable to complete the exam in the allotted time is
(A) 0.3413 (B) 0.1587 (C) 0.8413 (D) one of the above
57
Question 2: Given the following probability density function
c 2x4
f ( x)
0 otherwise
a. Find the value of c?
b. Find P(X>1)?
c. Find E(X) and Var(x)?
Question 3: Assume that the test scores from a college admissions test are normally distributed,
with a mean of 450 and a standard deviation of 100.
a. What percentage of the people taking the test score between 400 and 500?
b. Suppose that someone receives a score of 630. What percentage of the people taking
the test score better? What percentage score worse?
58