Scribd
Upload
4 views11 pages

ST3236 Note2

The lecture covers fundamental concepts in probability theory, including the probability space, conditional probability, and independence of events. Key topics include the definition of sample spaces, probability measures, and Bayes' theorem, illustrated with a practical example of doping tests in sports. The session concludes with a preview of upcoming topics such as random variables and expectations.

Uploaded by

Li Xuan Guang
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
4 views11 pages

ST3236 Note2

The lecture covers fundamental concepts in probability theory, including the probability space, conditional probability, and independence of events. Key topics include the definition of sample spaces, probability measures, and Bayes' theorem, illustrated with a practical example of doping tests in sports. The session concludes with a preview of upcoming topics such as random variables and expectations.

Uploaded by

Li Xuan Guang
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 11

LECTURE 2: Fundamental Concepts in Probability

Theory

Somabha Mukherjee

National University of Singapore

1 / 11
Outline

1 The Probability Space

2 Conditional Probability

3 Independence

2 / 11
Sample Space and Events

Suppose that a random experiment is performed, and its outcomes are


observed. The set of all possible outcomes is called the sample space,
denoted by Ω.

Roll a die once. The sample space

Ω = {1, 2, 3, 4, 5, 6}.

Toss a coin thrice. The sample space

Ω = {HHH, HHT , HTH, HTT , THH, THT , TTH, TTT }.

An event is a subset of the sample space Ω, i.e. a collection of some possible


outcomes.

For example, obtaining an odd number of heads in the first three tosses of a
coin is the event {HTT , THT , TTH, HHH}.

3 / 11
Probability Measure
A probability measure is a rule of assigning a valid chance (likelihood) to
each event.

Mathematically speaking, a probability measure is a function


P : Set of all events → [0, 1], satisfying the following two conditions:
1 P(Ω) = 1; T
2 If A1 , A2 , . . . is a countable collection of disjoint events (i.e. Ai Aj = ∅ for
i ̸= j), then  
[ X
P An  = P(An ).
n≥1 n≥1

If each outcome in a sample space is equally likely, one can imagine of


assigning the same probability to all the outcomes.

In this case, the probability of an event is given by:

|A| Total number of favorable outcomes


P(A) = = .
|Ω| Total number of possible outcomes

4 / 11
Some Properties of Probability:

P(Ac ) = 1 − P(A):
 [ 
1 = P(Ω) = P A Ac = P(A) + P(Ac ).

If A ⊆ B then P(A) ≤ P(B):


 [ 
P(B) = P A (B \ A) = P(A) + P(B \ A) ≥ P(A).

Boole’s Inequality: If A1 , A2 , . . . are events, then


 
[ X
P An  ≤ P(An ) EXERCISE!;
n≥1 n≥1

S T
P (A B) = P(A) + P(B) − P (A B) EXERCISE!

5 / 11
Outline

1 The Probability Space

2 Conditional Probability

3 Independence

6 / 11
What is Conditional Probability?
For two events A and B with P(B) > 0, the conditional probability of A given
B is defined as: T
P (A B)
P(A|B) = .
P(B)

It follows directly from the above definition, that:


 \ 
P A B = P(A|B)P(B) = P(B|A)P(A).

P(B|A)P(A)
Bayes Rule I: P(A|B) = P(B) EXERCISE !

Bayes Rule II: Let A1 , A2 , . .S. be a partition of the sample space Ω (i.e.
A1 , A2 , . . . are disjoint and j≥1 Aj = Ω.) Then for any event B with
P(B) > 0, we have:

P(B|Ai )P(Ai )
P(Ai |B) = P EXERCISE !
j≥1 P(B|Aj )P(Aj )

7 / 11
An Application of Bayes Formulae in Doping Tests
A certain doping test for football players is 99% accurate, meaning that it
shows a true positive result for players using drug with probability 0.99, and a
true negative result for players not using any drug with probability 0.99.

Let us assume that 0.1% of all football players use drug before a match.

Question: If a player selected at random tests positive, what is the probability


that he is actually a drug user?

Let A denote the event that the player is a drug user, and let B denote the
event that he tests positive. We are interested in P(A|B).

By Bayes formula, we have:


P(B|A)P(A)
P(A|B) =
P(B|A)P(A) + P(B|Ac )P(Ac )
0.99 × 0.001
= = 0.09.
(0.99 × 0.001) + (0.01 × 0.999)
Surprisingly, even if a player tests positive, there is only 9% chance that he is
actually drug user!
8 / 11
Outline

1 The Probability Space

2 Conditional Probability

3 Independence

9 / 11
What do we Mean by Independent Events?

Events A1 , A2 , . . . , An are said to be independent, if for every k ≤ n and


every 1 ≤ i1 < i2 < . . . < ik ≤ n,
 \ \ 
P Ai1 . . . Aik = P(Ai1 ) . . . P(Aik ).

Independence is different from pairwise independence, which means


 \ 
P Ai Aj = P(Ai )P(Aj ) for all i ̸= j .

Clearly, two events A and B are independent if and only if


 \ 
P A B = P(A)P(B).

Note that for two events A and B with P(B) > 0, A and B are independent
if and only if
P(A|B) = P(A).
if and only if “Conditional Probability of A given B does not depend on B.”

10 / 11
What will we do Next Day?

Introduce “random variables”, which are the central characters in probability


theory.

Introduce a notion of the average of a random variable, which is known as


“expectation”.

Talk about “conditional distributions” of random variables and introduce the


notion of “conditional expectations”.

All these concepts will be immensely useful in our future study of Markov
chains.

11 / 11

You might also like