EEF 271E

Probability and Statistics

Fall 2022

Lecture 02
Computing Probability

Lecture Notes 02 Computing Probability 1

 Lesson Overview

Computing Probability

– Relative frequency approach

– Classical (counting) approach
– Reasonable definition approach

Lecture Notes 02 Computing Probability 2

 Computing Probability

So far, we have seen only the

properties that a probability must
satisfy.
Thus, there still remains the
fundamental question:
‘’How do we compute probabilities
?’’

3
Lecture Notes 02
 Computing Probability
‘’How do we compute
probabilities ?’’
To start answering this question, we
the experimenters, must step in
with physical intuition, reasonable
assumptions etc.
Probability theory merely provides
a mathematical framework; we
have to connect it to ‘’real life’’.
4
Lecture Notes 02
 Computing Probability
‘’How do we compute
probabilities ?’’
Example:

The Probability of having a head (H)

for a fair coin toss etc.
(more on this soon…)

5
Lecture Notes 02
 Computing Probability

There are three ways we will

consider for ‘’assigning’’
probabilities.
1) Relative frequency approach.
2) Classical (counting) approach.
3) Reasonable definition approach.

6
Lecture Notes 02
 Relative frequency approach.
Given an event A, we define
as follows:
(i) Repeat experiment N times.
(ii) Count # of times A occurs;
let represent this with
Define
(where the symbol of # means number)
7
Lecture Notes 02
 Relative frequency approach.
The relative definition approach is
based on engineering intuition;
a ‘’more probable’’ event occurs

‘’more frequently’’.
There is and inherent, deep,
assumption that ‘’time averages’’
equal probability …

8
Lecture Notes 02
 Relative frequency approach.
Let us now check that the relative
frequency definition of Probability
satisfies the axioms…

(1)Is for any AєF ? YES.

(2) Is ? YES.

(3) If A and B satisfy

is ? YES.

9
Lecture Notes 02
 Calassical approach.

Classical approach is based on

‘’principle of sufficient reason’’.
It was proposed around 18th century in
connection with games of chance …

10
Lecture Notes 02
 Calassical approach.

Suppose

From physical reasoning, suppose the

elementary events , , …, are equally
likeley.
Then given any event
(note that it has m elements)

We define
11
Lecture Notes 02
 Calassical approach.

In other words, if # A denotes the

number of elements in the set A,

Why does definition make sense?

For any elementary event ,

12
Lecture Notes 02
 Calassical approach.

So that elementary events are all

equally likely (the assumption we
started with)… ,

*** WARNING: Remember that

holds if and only if all elementary

events are equally likely!
13
Lecture Notes 02
Calassical approach.

Example 1:
Resistance value 1Ω 10 Ω 100 Ω 1000 Ω
# of resistors 100 500 150 250

Resistors mixed together in a large bin,

Experiment: Draw one resistor at random
Find

14
Lecture Notes 02
Solution 1:

Here is the formal way to solve this

problem using the classical approach.
Label 1 Ω resistor as a1, a2, …, a100
10 Ω resistor as b1, b2, …, b500
100 Ω resistor as c1, c2, …, c150
1000 Ω resistor as d1, d2, …, d250
Ω ={ a1, a2, …, a100, b1, b2, …, b500, c1, c2, …,
c150,
d1, d2, …, d250 }
Lecture Notes 02
15
 Solution 1 (Cont.):

From physical reasoning all elementary

events are equally likely (since resistor
is drawn ‘’random’’ ).
Now, the event ‘’resistor has value of 1
Ω’’ same as {a1, a2, …, a100}
Thus

Lecture Notes 02 Computing Probability 16

Solution 1 (Cont.):

Similarly

Lecture Notes 02 Computing Probability 17

Does the definition of Probability from
Classical approach satisfy the axioms?

(1) P(A) ≥ 0 ? (YES)

(2) P(Ω) = 1 ? (YES, )
(3) If AB =  does
P(AB) = P(A) + P(B) ?
(YES, since if AB =  Y #(AB) = #A +
#B)
Lecture Notes 02 Computing Probability 18
 Though the Classical approach
appears limited in scope (applies
only when all elementary events are
equally likely), it is extremely
powerful since Probability can be
computed simply by ‘’counting’’.

More later…

Lecture Notes 02 Computing Probability 19

 Reasonable definition
approach.
- Very similar to Classical approach,
but more general.
- Applies to continuous sample space
also
- Idea: Define probabilities of some
basic events to whatever is
reasonable. Then use axioms and
properties of Probability of other
Lecture Notes 02 Computing Probability 20
events.
 Reasonable definition
approach.
What are the basic events mentioned
above?
For discrete sample space, one choice
of basic events is simply the set of
elementary events.

Lecture Notes 02 Computing Probability 21

 Examples:
Expt. Tossing a fair coin.
Outcomes: Head (H), Tail (T)
Sample space: Ω = {H, T}
DEFINE ,
Why ? Because it’s reasonable to
assume coin is fair (same
justification as with Classical
approach).
Lecture Notes 02 Computing Probability 22
 Expt: Tossing a biased coin.
Outcomes: H,T
Sample space: Ω = {H, T}
DEFINE ,
Why ? because have reason to
believe that the Probability
assignment is sensible …

Lecture Notes 02 Computing Probability 23

 Expt:
Peak a real number at random,
‘’uniformly’’ in [0,1].

Ω = [0, 1]
,
(You should check that this definition
satisfies all the axioms of
Probability).
Real number
0 a b 1

Lecture Notes 02 Computing Probability 24

 - It turns out here that the analog to
the ‘’elementary events’’ are simply
intervals.

The Probability of any event , can be

computed using this rule. (BTW «by
the way» to prove all this rigorously is
quite hard … ‘’measure theory’’ )
- Is this the only Probability rule
possible? (Answer NO!). Why this rule
then? (Answer: ‘’uniform’’)
Lecture Notes 02 Computing Probability 25
 Examples:
Expt: Pick a pair of real #s (x1, x2) in
[0,1]x[0,1]
Outcomes: real number pairs in [0,1]x[0,1]
x2
Sample space: Ω Event: A
x2

(1,1) (0,1) (1,1)

(0,1)
W
W
A

x1 x1
(0,0) (1,0) (0,0) (1,0)

Define
Lecture Notes 02 Computing Probability 26
 Sample space: Ω Event: A
x2 x2

(0,1) (1,1) (0,1) (1,1)

W
W
A
𝑃 ( 𝐴) = 𝐴𝑟𝑒𝑎 𝑜𝑓 𝐴
x1 x1
(0,0) (1,0) (0,0) (1,0)
x2 x2 x2

(1,1) (1,1) (1,1)

(0,1) (0,1) (0,1)
Ÿ Ÿ
Ÿ

A A A
(0,0) (1,0) x1 (0,0) (1,0) x1 (0,0) (1,0) x1

etc.

(A dart board provides a similar example…

Probability calculation are more complicated here,
as Notes
Lecture we 02
shall see…) Computing Probability 27
Example:
Two persons have a date at a given time, and each will
arrive at the meeting place with a delay between 0 and 1
hour. The first to arrive will wait for 15 minutes and will
leave if the other has not yet arrived.
What is the Probability that they will meet?
Solution:
Let the persons be Romeo & Juliet.
Assume that they will meet at 0 o’clock.
Romeo delays for x minutes
Juliet delays for y minutes
If |x-y|< 15 minutes they will meet.

Lecture Notes 03 Conditional Probability, Bayes' 28

Theorem
Solution:
Romeo delays for x minutes
Juliet delays for y minutes
If |x-y|< 15 minutes they will meet.
y
15 minutes 1/4 hours
1
Ÿ M event

M = { (x, y) | |x-y| ”1/4 , 0 ”x < 1, 0 ”y < 1 }

Ÿ = [(0,1)x(0,1)] : square area ( Sample Space)

1/4

x
1/4 1

Lecture Notes 03 Conditional Probability, Bayes' 29

Theorem
Solution:
Romeo delays for x minutes, Juliet delays for y minutes
If |x-y|< 15 minutes they will meet.
y
15 minutes 1/4 hours
1
Ÿ M event

M = { (x, y) | |x-y| ”1/4 , 0 ”x < 1, 0 ”y < 1 }

Ÿ = [(0,1)x(0,1)] : square area ( Sample Space)

1/4

x
1/4 1

Lecture Notes 03 Conditional Probability, Bayes' Theorem 30

 In summary:
 Probability is a function

 Satisfies some axioms, based on which other

properties can be derived.

 In order to compute probabilities need to know

probabilities of some basic events (elementary
events for discrete sample space, interval for
Ω=[0,1] etc. …)

 The probabilities of these basic events are either

estimated empirically (using relative frequency),
guessed or defined using reasoning.
Lecture Notes 02 Computing Probability 31