Probability and Statistics ì

for Computer Science

“A major use of probability in

statistical inference is the
updating of probabilities
when certain events are
observed” – Prof. M.H.
DeGroot
Credit: wikipedia

Hongye Liu, Teaching Assistant Prof, CS361, UIUC, 9.7.2023

Counting: how many ways?
if we
put I hats /
indistinguishable)
on 7 people out of 10 people
randomly ?
St
-
Fl-
-

E
(i) =

1
Last time
✺More Probability Calculation
✺Conditional Probability
✺ Multiplication rule
✺ Bayes rule
Multiplication using conditional
probability
⇒ P (A ∩ B) = P (A|B)P (B) 0 6 ice cream
p(AlB)
.

/ 0.5
meat
8 4
cheesecake
Cheesecake
-

2
P(B
! 0.3
:

soup fish
0.8
0.2
vegetable P(C(B) = ?
(start) 8x0 5x 0
4
meat (MB)
-

0 . .

0.3 =8 0
*
0.2
.

0.4 P(B)
P(D) Juice fish
<B =

-
BrAnc+BUAnc
0.3 vegetable
A : meat B :

soup D :
juice
Objectives
✺Conditional Probability
✺ Review
✺ Bayes rule
✺ Total probability
✺ Independence
Conditional Probability
✺ The probability of A given B
Ω P (A ∩ B)
P (A|B) =
P (B)
B
A∩B
A
P (B) != 0
A

The area defined by the set B is the

new sample space for conditional P(A|
B)
Joint Probability Calculation
⇒ P (A ∩ B) = P (A|B)P (B)
PLANUCS
0.5
meat
=

p(An /C) pCC) ·

soup 0.3 fish

(start)
0.8
0.2
vegetable P(A)BIC P(B(c)
=
.

meat
· pic
0.3
0.2 0.4
Juice fish
P (soup ∩ meat) =
0.3 vegetable
P (meat|soup)P (soup)
= 0.5 × 0.8 = 0.4
Symmetry of joint event in terms of
conditional prob.
P (A ∩ B)
P (A|B) = P (B) != 0
P (B)

⇒ P (A ∩ B) = P (A|B)P (B)
⇒ P (B ∩ A) = P (B|A)P (A)
Symmetry of joint event in terms of
conditional prob.

∵ P (B ∩ A) = P (A ∩ B)
⇒
P (A|B)P (B) = P (B|A)P (A)
The famous Bayes rule
P (A|B)P (B) = P (B|A)P (A)

⇒ P (B|A)P (A)
P (A|B) =
P (B)
"

Thomas Bayes (1701-1761)

Bayes rule: lemon cars
There are two car factories, A and B, that
supply the same dealer. Factory A produced
1000 cars, of which 10 were lemons. Factory B
produced 2 cars and both were lemons. You
bought a car that turned out to be a lemon.
What is the probability that it came from
Er Bad Ez From B
factory B?
: 1 can =

2
14 2

P CEICEw PIEr
- -

, 1002

Is
-

psErIE)
=
1002

-E
-
-

--
12
-
(02
Bayes rule: lemon cars
Given the above information, what is the
probability that it came from factory A?
P (A|L) =?
Bayes rule: lemon cars
Given the above information, what is the
probability that it came from factory A?
P (A|L) =?
P (L|A)P (A)
P (A|L) =
P (L) Or in this case

P (A|L) = 1 − P (B|L) =
Total probability:
Sunny : 80 % to work out

Rainy :
309 to workout

Snowy :
409 to workout

# Sunny/#Rainy / Snowy
A
=

5 : 3 : 2

G
Sunnys P(W/Smn) Pun)
=

P (workout
-
(work out) ? (Rain)
P -
D(W/RarP
=

G Rains=
30 3
+pl . . .

(urlSnow P
8X15 +o
Swary1= P Isnow)
.

G
.

=0 .

-
· 4 X0 .
2 + ps ..
Total probability
P(B)
=
P(A , B) +
P(A2nB) +

P(AznB)
=
P(B(A ) P(A1)
.
+
P(B(A2) P(A2)
+
P(B(As) P(As)

Al Az , As
,

A1 are disjoint
B A3
,

AiUAzU As
-
B
A2
Total probability:
P(B)
=
P(ANB) + P(A nB)
= P>B(A) p(A) + P (B) AY PIAY
-

R
A
B P(B(A) =

C
A
P(B(AY =
Total probability general form
P(B) =
I P(BUAj)
j

I
=P(BIAj>P/Ai
Aj ,
Ar are

A1 disjoint
B A3
AjnAm =
j =
1
A2
Total probability:
8 46
P(meat) ?
.

ments
p(meat/soups P :
Samph
↓
0.5
0 meat -- PIjuice
ments

p(soup)
↓
soup 0.3 fish Assomps
·

Psmeatkamp
0.8
0.2
vegetable
(start)
0.3
meat P(soup/meat) =
?
0.2 meat
S 0.4 fish pssomp of

Pljnice) Juice
-

0.3 vegetable p > meat /

- .
4
--
0 .
46
Bayes rule using total prob.
P(AjB) =
PAS
-

-
Aj) P(Aj)

IP(B(Aj) P(Aj)
L
j

AnnAj= bedisjoint
if i =
j
Bayes rule: rare disease test
There is a blood test for a rare disease. The
frequency of the disease is 1/100,000. If one has it,
the test confirms it with probability 0.95. If one
doesn't have, the test gives false positive with
probability 0.001. What is P (D|T ), the probability
of having disease given a positive test result?
P (T |D)P (D) Using total prob.
P (D|T ) =
P (T ) 5
- 95.
-

P (T |D)P (D) I
> (0

= -

o
sol

P (T |D)P (D) + P (T |Dc )P (Dc ) (1-10-5)

,

-
- -
Bayes rule: rare disease test
There is a blood test for a rare disease. The
frequency of the disease is 1/100,000. If one has it,
the test confirms it with probability 0.95. If one
doesn't have, the test gives false positive with
probability 0.001. What is P (D|T ), the probability
of having disease given a positive test result?
P (T |D)P (D)
P (D|T ) =
P (T |D)P (D) + P (T |Dc )P (Dc )
-

-
Independence
✺One definition:
P (A|B) = P (A) or
P (B|A) = P (B)

Whether A happened doesn’t change

the probability of B and vice versa
Independence: example
✺ Suppose that we have a fair coin and it is
tossed twice. let A be the event “the first
toss is a head” and B the event “the two
outcomes are the same.” >
= M
A : :st H p(B(A) PcA)
and same
B -
: ist

TT
HH ,
8 .
5

-
I
&
✺ These two events are independent!
2

P(B) I =
Independence
✺Alternative definition
P (A|B) = P (A)
P (A ∩ B)
⇒ = P (A)
P (B)
⇒ P (A ∩ B) = P (A)P (B)
Testing Independence:
✺ Suppose you draw one card from a
standard deck of cards. E1 is the event
that the card is a King, Queen or Jack. E2
-

is the event the card is a Heart. Are E1 and

E2 independent?
p(Ei)
=

Y3 plE , nEn) =I
I p(EinEn)
=

P1E,pDE)
p(Ers=
Iindpt !
 Pairwise independence is not mutual
independence in larger context
! S
A1 A2 P(A1) = P(A2) = P(A3) = P(A4) = 1/4
⑧
A3 ⑲ P(A) ! =P(B) = P() x
=

A4
=
=

C
1 P(AnB) P, PASPIB) = I

I
-

A = A1 ∪ A2 ; P (A) =
2 P(Anc ? P(A>PIC) ? =
=

1 Px1) ·

B = A1 ∪ A3 ; P (B) = P(BUC) ? P(B)p() ?

=
=

2 B
1 P(A)PIB) PCC) 1
C = A1 ∪ A4 ; P (C) = P(AMBUC) > =
=

2 I= Pan
I
I
***I
*P (ABC) is the shorthand f or P (A ∩ B ∩ C)
Mutual independence
✺ Mutual independence of a collection
of events A1 , A2 , A3 ...An is :
P (Ai |Aj Ak ...Ap ) = P (Ai )
j, k, ...p != i
✺ It’s very strong independence!
Probability using the property of
Independence: Airline overbooking (1)
✺ An airline has a flight with 6 seats. They
always sell 7 tickets for this flight. If ticket
holders show up independently with
-

probability p, what is the probability that

the flight is overbooked ?
3
P(7 pl snow up
1st
Mandard
--n< th)
=

> --- paths

It)Plinds
. .

P 1 7
=

P
Probability using the property of
Independence: Airline overbooking (2)
✺ An airline has a flight with 6 seats. They
always sell 8 tickets for this flight. If ticket
holders show up independently with
probability p, what is the probability that
exactly 6 people showed up?

P(6 people showed up) =

18) Pep
Probability using the property of
Independence: Airline overbooking (3)
✺ An airline has a flight with 6 seats. They
always sell 8 tickets for this flight. If ticket
holders show up independently with
probability p, what is the probability that
the flight is overbooked ?

P(8pK
P( overbooked) = p(2ni
+

(2) P( > p8+

Assignments
✺ Module week3 on Canvas
✺ Next time: Random variable
Additional References
✺ Charles M. Grinstead and J. Laurie Snell
"Introduction to Probability”
✺ Morris H. Degroot and Mark J. Schervish
"Probability and Statistics”
See you next time

See
You!