Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous

STAT 151 Introduction to Statistical Theory

August 21, 2014
STAT 151 Class 1 Slide 1
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Instructor and TAs (weekly consultation location and hours, by
appointments only)
Denis HY LEUNG SOE 5047
email:[email protected]
phone: 68280396
Tues 4-6pm, Wed/Thurs 9-11am
TAs SOE Grp Study Rm 4-7 (Rm 4023)
Mon 12-3pm DANG VU Dieu Linh
[email protected]
Wed 12-3pm HO Yan Yan
[email protected]
Thu 4-7pm Muhammad Haz Bin KASMAN
[email protected]
Thu 12-3pm LAU Jin Wen
[email protected]
STAT 151 Class 1 Slide 2
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Essentials
Name card (rst few weeks)
Course webpage: http://economics.smu.edu.sg/faculty/
profile/9699/Denis%20LEUNG (NOT eLearn!)
Understanding of basic Calculus and Algebra Appendix in course
notes
Readings before each class
Projects vs Homework
Do not disturb others in class
If you missed a class, it is YOUR responsibility to nd out what you
have missed from your classmates or course webpage
Expected eort 5 hours per week
STAT 151 Class 1 Slide 3
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Assessments
Class Participation (10%)
Projects (40%)
1 project with presentations 16%
2 projects without presentations 24% (2 12%)
Each projects grade includes 6% individual assessment
(quizzes)
Exam (50%)
Closed book but one 2-sided A-4 cheat sheet is allowed
STAT 151 Class 1 Slide 4
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Structure of each class
New materials (< 2.5 hrs)
Break (15 mins)
Any other business (15 mins)
STAT 151 Class 1 Slide 5
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Data
Survival time (in years) in 70 cancer patients
0.67 0.01 0.48 1.06 0.85 0.45 0.19 0.78 1.23 0.18
0.37 2.17 0.15 1.19 0.58 1.22 0.72 0.67 1.42 0.02
0.12 0.50 0.15 0.81 0.64 0.22 0.05 0.55 0.46 0.83
0.46 0.56 0.82 0.07 2.28 0.34 0.64 0.09 0.77 0.26
0.45 0.41 0.23 0.16 0.39 0.29 0.62 1.09 0.14 0.49
0.66 0.89 0.99 0.98 0.95 0.14 0.03 0.01 2.62 0.99
0.08 1.39 1.31 0.50 0.74 1.19 0.15 0.14 1.18 1.53
Orange - Subtype I Black - Subtype II
Outcome (H vs. T) in tossing a coin 10 times
H H T H T T H T T T
Occurrence of nancial crises
Mexican
82
S&L
84
Black Mon.
87
Comm. RE
91
Asian
97
LTCM
98
Dotcom
00
Subprime
07
Euro
12
?
STAT 151 Class 1 Slide 6
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Sample vs. Population
(a) Data are a sample from a population that we want to study
e.g. Survival time of 70 patients (sample) out of all cancer
patients (population)
(b) We are interested in some characteristics of the population
e.g. average survival time of patients or percentage of patients
who live beyond 2 years
(c) Due to randomness, we cannot make denitive statements about
a particular unit in the population
e.g. We cannot tell how long the next patient would live
(d) We use the sample of data to help us answer the questions in (b)
STAT 151 Class 1 Slide 7
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Randomness
Observation from survival time data - Why do some patients live longer than
others?
Subtype I - 1.06 2.17 1.22 1.42 2.28 2.62 1.39 1.18 1.53
average 1.65
Subtype II - 0.67 0.01 0.48 0.85 0.45 0.19 0.78 1.23 0.18 0.37 0.15
1.19 0.58 0.72 0.67 0.02 0.12 0.50 0.15 0.81 0.64
0.22 0.05 0.55 0.46 0.83 0.46 0.56 0.82 0.07 0.34
0.64 0.09 0.77 0.26 0.45 0.41 0.23 0.16 0.39 0.29
0.62 1.09 0.14 0.49 0.66 0.89 0.99 0.98 0.95 0.14
0.03 0.01 0.99 0.08 1.31 0.50 0.74 1.19 0.15 0.14
average 0.50
I has a better chance of survival because it has a higher average than II
There are still variations within each subtype - inevitable in the real world
We attribute (accommodate) these (unexplained) variations as random
Chance and randomness can be described using probability theory
STAT 151 Class 1 Slide 8
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Denition of probability - Coin tossing data
Toss 1 2 3 4 5 6 7 8 9
Outcome H T T H H T H T H
P
r
o
p
o
r
t
i
o
n

o
f

h
e
a
d
s
0
.
0
0
.
2
0
.
4
0
.
6
0
.
8
1
.
0
1 2 3 4 9 50 100 500 1000
probability = 0.5
Number of tosses
The long run proportion (frequency) of heads is the probability of heads.
STAT 151 Class 1 Slide 9
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Insight from coin tossing experiment
Probability is the long run frequency of an outcome
Probability cannot predict individual outcomes
However, it can be used to predict long run trends
Probability always lies between 0 and 1, with a value closer to
1 meaning a higher frequency of occurrence
Probability is numeric in value so we can use it to:
- compare the relative chance between dierent outcomes
(events)
- carry out calculations
STAT 151 Class 1 Slide 10
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Probability Axioms - Urn model (1)- drawing marbles from an urn
(with replacement)
1
2
4
3
5
Probability
3
5
2
5
Draws
1 2 3 4 5 6 7 8 9
...
...
STAT 151 Class 1 Slide 11
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Urn model (2)
Five possible Outcomes: 1 2 4 3 5
Interested in Event A:
A={ 1 4 5 }; hence an event is a collection of
outcomes
P(A) =
3
5
= 0.6(60%) =
Number of marbles in A
Total number of marbles
STAT 151 Class 1 Slide 12
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Complementary events
Marbles in urn: 1 2 4 3 5
Interested in

A: (Not A)

A=

2 3

A, sometimes written as A
C
, is called the complementary
event of A
Chance of = 1 chance of
P(

A) = 1 P(A) = 1
3
5
=
2
5
STAT 151 Class 1 Slide 13
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Joint probability - independent events - Drawing a blue in draw 1 and
a green in draw 2 (with replacement)
A and B are independent means the occurrence of one event does
not change the chance of the other
1
2
4
3
5
Draws
1 2
A = { in 1st draw}
B = { in 2nd draw}
P(A) =
3
5
; P(B) =
2
5
P(A and B) =
3
5

2
5
=
6
25
= P(A)P(B)
STAT 151 Class 1 Slide 14
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Joint probability and disjoint events
Two events A and B are disjoint or sometimes called mutually
exclusive if they cannot occur simultaneously: P(A and B)=0
Example
P( and in a single draw) = 0
STAT 151 Class 1 Slide 15
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Conditional probability
Conditional probability is a useful quantication of how the assessment of
chance changed due to new information: If A happened, what is the chance
of B?
The conditional probability of B given A is written as P(B|A)
Example Drawing marbles WITHOUT replacement
?
A = { in 1st draw}
B = { in 2nd draw}
P(B) =
2
5
P(B|A) =
2
4
2
5

2
4
because has been drawn
P(A and B) =
2
4

3
5
= P(B|A)P(A) =
6
20
= P(B)P(A)
;
STAT 151 Class 1 Slide 16
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Conditional probability and independence
The following relationships hold if A and B are independent
P(A|B) = P(A)
P(B|A) = P(B)
P(A and B) = P(A)P(B)
Independence is NOT the same as mutually exclusive (disjoint),
which is P(A and B) = 0.
STAT 151 Class 1 Slide 17
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
The multiplication rule
P(AB) = P(A|B)P(B) = P(B|A)P(A) Multiplication Rule
Example
Marbles in urn: 1 2 4 3 5
P( and Odd) = P( |Odd)P(Odd) =

1
3

3
5

=
1
5
= P(Odd| )P( ) =

1
2

2
5

=
1
5
Rearranging the multiplication rule:
P(A|B) =
P(AB)
P(B)
STAT 151 Class 1 Slide 18
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Partition rule
P( ) = P( and odd) +P( and even)
= P({ 1 5 }) +P({ 4 })
=
2
5
+
1
5
=
3
5
STAT 151 Class 1 Slide 19
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Union of events (1)
Union of events can sometimes be best
visualized using a Venn diagram (John
Venn, 1834-1923)
Example What is the probability of drawing
a or an odd number ?
1
2
4
3
5
Urn
Green
Odd
1
2
4
3
5
;
STAT 151 Class 1 Slide 20
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Union of events (2)
P( or odd) = P( ) +P(Odd) P( and odd)
=
2
5
+
3
5

1
5
=
4
5
In general, if A and B are:
disjoint, then P(A or B) = P(A) +P(B)
not disjoint, then P(A or B) = P(A) +P(B) P(AB)
STAT 151 Class 1 Slide 21
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Probability tree
Probability tree is useful for studying combination of events. Branches of a tree are
conditional probabilities.
Example
Drawing two marbles from urn without replacement:
Draw 1
Draw 2
P( ) =
3
5

2
4
2
4
= P( | )
P( ) =
3
5

2
4
2
4
= P( | )
3
5
Draw 2
P( ) =
2
5

3
4
3
4
= P( | )
P( ) =
2
5

1
4
1
4
= P( | )
2
5
STAT 151 Class 1 Slide 22
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Bayes Theorem (Thomas Bayes, 1701-1761)
Trees are useful for visualizing P(B|A) when B follows from A in a natural (time)
order. Many problems require P(A|B), Bayes Theorem provides an answer.
Example
Testing for an infectious disease.
Disease
Test
P(

D

T) =
99
100

9
10

T
9
10
= P(

T|

D)
P(

D T) =
99
100

1
10 T
1
10
= P(T|

D)

D
99
100
Test
P(D

T) =
1
100

1
10

T
1
10
= P(

T|D)
P(D T) =
1
100

9
10 T
9
10
= P(T|D)
D
1
100
What is P(D|T) or P(

D|

T)?
STAT 151 Class 1 Slide 23
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Bayes Theorem (2)
P(D|T) =
P(D T)
P(T)
=
P(T|D)P(D)
P(T)
=
P(T|D)P(D)
P(T D) +P(T

D)

Partition rule
=
P(T|D)P(D)
P(T|D)P(D) +P(T|

D)P(

D)

Multiplication rule
=
9
10

1
100
9
10

1
100
+
1
10

99
100
=
1
12
In general,
P(A|B) =
P(B|A)P(A)
P(B)
STAT 151 Class 1 Slide 24
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Data revisited- Discrete vs. Continuous Data
Discrete - countable number of possible values
Examples
(1) Coin Tossing: H H T H T T H T T T
Two possible values: H or T
(2) Financial crisis: 1982, 1984, 1987,... (No. of crises in a
decade)
Many possible values: 0, 1, 2, 3, 4,...
Continuous - values fall in an interval (a, b), a could be and
b could be
Example
Survival time in cancer patients: 0.67 0.01 0.48 1.06 0.85 0.45 ...
0 < survival time < a positive number
STAT 151 Class 1 Slide 25
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
First look - summary statistics
Nave methods
Numerical summary: Min, max, mean, variance, etc.
Graphical summary: Histogram, pie chart, bar graph, etc.
Tabular summary: Tables of frequencies
Attributes
Quick and dirty
Identify potential problems, errors
Flaws
Dicult to generalize especially when the dataset is
complicated
Not easily portable cannot describe a chart or graph unless
you can see it!
STAT 151 Class 1 Slide 26
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
What do we do about data?
Financial crises data
Mexican
82
S&L
84
Black Mon.
87
Comm. RE
91
Asian
97
LTCM
98
Dotcom
00
Subprime
07
Euro
12
?
We may be interested in X, the number of crises in the next decade. Possible questions:
What is the probability of X=0, or 1, or 2, or 3 or...?
Probability distribution - tells us the long run frequencies (chances) of the possible
outcomes
On average, how many crises in a decade?
Expectation (Expected value) - tells us the weighted average of the possible
outcomes, where the weights are the probabilities
How likely will X dier from the expected?
Standard deviation (Variance) - tells us the weighted average of deviations of the
possible outcomes from the expected value
STAT 151 Class 1 Slide 27
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Discrete distributions - Tossing a coin
(H) or (T) ?
Long run frequencies
H T
1
2
1
2
The long run frequencies are 1/2 each for
H and T there is equal chance for H or T
;
STAT 151 Class 1 Slide 28
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Discrete distributions - Drawing a marble
1
2
4
3
5
1 2 3 4 5 ?
1 2 3 4 5
1
5
1
5
1
5
1
5
1
5
3
5
2
5
;
The long run frequencies tell us there is equal chance for 1, 2, 3, 4
and 5 but there is a higher chance for blue than green
STAT 151 Class 1 Slide 29
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Discrete probability distributions
X H T
P(X)
1
2
1
2
X 1 2 3 4 5
P(X)
1
5
1
5
1
5
1
5
1
5
3
5
2
5
X
a
1
a
2
a
3
...
P(X) P(a
1
) P(a
2
) P(a
3
)
...
Probability distribution function
;
A probability distribution summarizes the long run frequencies (chances) of the
possible outcomes of X
STAT 151 Class 1 Slide 30
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Probability distribution for a discrete random variable
X is the unknown outcome.
X is called a discrete random variable if its value can only come from a countable
number of possible values: a
1
, a
2
, ..., a
k
.
Examples
Coin toss: X = H or T (2 possible values)
Marbles: X = 1, 2, 3, 4, or 5 (5 possible values)
Financial crisis: X = 0, 1, 2, 3,... (Innite but countable number of possible
values)
P(X = a
i
) gives the probability X = a
i
and is called a probability distribution
function.
A valid probability distribution function must satisfy the following rules:
P(X = a
i
) must be between 0 and 1
We are certain that one of the values will appear, therefore:
P(X = a
1
or X = a
2
or ...X = a
k
) = P(X = a
1
) +P(X = a
2
) +... +P(X = a
k
)

X=a
1
,X=a
2
,...are disjoint events
= 1
STAT 151 Class 1 Slide 31
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Example: Probability distribution of the number from drawing a marble
X 1 2 3 4 5
P(X) 0.2 0.2 0.2 0.2 0.2
P(X = 1) = 0.2
P(X 3) = P(X = 3) +P(X = 4) +P(X = 5) = 0.6
P(X = 1.5) = 0
P(X > 5) = 0
1 = P(X = 1)+P(X = 2)+P(X = 3)+P(X = 4)+P(X = 5)
STAT 151 Class 1 Slide 32
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Continuous random variable
The survival time (X) of a cancer patient following diagnosis may be
any value in a range (e.g. 0 < X < some positive number).
X is called a continuous random variable if its value fall in a range
(a, b) (, ). For a continuous random variable X, P(X = x) for
any value x is 0, we can only talk about the probability of X falling in
a range
Examples
P(X > 2 years) = Probability of surviving beyond 2 years
P(0 < X 1 year) = Probability of dying within 1 year
P(X 2 years|X > 1 year) = Probability of dying within 2
years given surviving for 1 year?
STAT 151 Class 1 Slide 33
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Probability distribution for a continuous random variable
A continuous random variable, X, is a random variable with outcome that falls
within a range or interval, (a, b)
Examples
X = survival time of a patient: (a, b) = [0, 150] years
X = return from an investment of 10000; (a, b) = [10000, ) dollars
X = per capita income; (a, b) = [0, 1000000000000] dollars
The probability distribution of X is dened by a (probability) density
function (PDF), f (x). The interpretation of f (x) is similar to probability but
there are subtle dierences:
f (x) 0 for any value of x in (a, b)
f (x) = P(X = x) = 0 for any value of x
P(r X s) = P(r < X < s)

since P(X=r )=P(X=s)=0
probability of X between r and s
P(a X b) = 1 since X must fall within (a, b)
The cumulative distribution function (CDF for short), written as F(x), is
dened as P(X x)
STAT 151 Class 1 Slide 34
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Example - survival data
Density of survival time
X (Time in years)
D
e
n
s
i
t
y
0 1 2 3 4 5
f(x) = 1.5e
1.5x
f (x) = e
x
, > 0 is called an
exponential density
Dierent values of can be used
to model dierent populations
All exponential densities have the
same shape but dierent gradients
so it is easy to describe to others
Allows us to make probability
statements about survival times we
have and have not observed
The area under f (x) is 1 which
means the probability of dying
between time 0 and is 1
STAT 151 Class 1 Slide 35
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Some calculations
X (Time in years)
D
e
n
s
i
t
y
0 1 2 3 4 5
f(x) = 1.5e
1.5x
P(X > 2)
P(X > 2) = 0.049 is the red area under the curve and can be
obtained by analytical or numerical (computer) analysis
STAT 151 Class 1 Slide 36
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Some calculations
X (Time in years)
D
e
n
s
i
t
y
0 1 2 3 4 5
f(x) = 1.5e
1.5x
P(X < 1)
P(X 1) = P(0 < X 1) = 0.776 is the red area under the curve
and can be obtained by analytical or numerical analysis.
STAT 151 Class 1 Slide 37
Introduction Randomness Probability Probability rules Probability distributions Discrete Continuous
Some calculations
P(X 2|X > 1) =
P(1 < X and X 2)
P(X > 1)

conditional probability
=
P(1 < X 2)
P(X > 1)
=
1 P(X 1) P(X > 2)
1 P(X 1)
=
1 0.776 0.049
1 0.776
= 0.781
STAT 151 Class 1 Slide 38