Scribd
Upload
1 views17 pages

Joint Probability Distribution

Joint Probability Distribution

Uploaded by

nisharobinrohit
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
1 views17 pages

Joint Probability Distribution

Joint Probability Distribution

Uploaded by

nisharobinrohit
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

Joint Probability Distribution

Weather
Sunny Cloudy Rainy Snowy
Yes 0.25 0.15 0.05 0.13
Have
Fun? No 0.05 0.1 0.25 0.02
Distribution vs. Probability
discrete random variable

P(A) denotes a distribution


Probabilities for each value
P(A | B) denotes a different distribution across the
same values

P(A = a2) denotes the probability that A takes on the


value a2
P(A = a2 , B = b3) denotes the probability that A takes on
the value a2 and B takes on the value b3
Distribution
P(Weather)
Weather
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
Sunny Cloudy Rainy Snowy

P(Weather = Sunny) = 0.3


Distribution
P(Weather | Fun=Yes)
Weather given Fun=Yes
0.5
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
Sunny Cloudy Rainy Snowy

P(Weather = Sunny | Fun = Yes) = 0.431+


Distribution Statements

P(A) = P(A | C)
vs
P(A) = P(A | C=c5)
How many numbers (parameters) to
estimate from data?

• Number of diseases: n
• Number of symptoms: q
• Each symptom takes on s values

nsq - 1
Joint Probability Distribution
Weather
Sunny Cloudy Rainy Snowy
Yes 0.25 0.15 0.05 0.13
Have
Fun? No 0.05 0.1 0.25 0.02

Now add RV for Time Of Day in 6hr


blocks
Conditional Independence
• Model the disease influencing the symptoms
• But no symptom interactions given the
disease
• Conditional independence:

P(A|B,C) = P(A|C)
“A is conditionally independent of B given C”
More Conditional Independence

Suppose A is conditionally independent of B given C


Is B necessarily conditionally independent of A given C?

P(A|B,C)=P(A|C) ? P(A|B,C) = P(A|C)

P(B|A,C)=P(B|C) P(A,B,C)
=
P(A,C)
P(B,C) P(C)

P(A,B,C) P(B,C)
=
P(A,C) P(C)

P(B|A,C) = P(B|C)
YES, they are equivalent
More Conditional Independence
If A and B are conditionally independent given C
C can still depend on both A and B
P(C|A)  P(C) and P(C|B)  P(C)
A and B are not necessarily independent:
P(B|A)  P(B) and P(A|B)  P(A)
But, Given C,
A and B do not influence each other
Knowledge of C separates A and B
Suppose we do NOT know C,
then A and B MAY influence each other
(think about this one until it’s intuitive
– what does “influence” mean here?)
Suppose that Symptoms are Conditionally
Independent given the Disease

• We know John has a cold


• Congestion, a sore throat, headache, rash
are more likely but not necessary
• We discover he does in fact have a sore throat
• Is he now more or less likely to also have a
headache?
Bayesian Networks
(belief networks,…)

• Graphical model
– Probability + Graph theory
• Nodes – discrete random variables
• Directed Arcs – direct influence
– Usually “causality” (class, text,…)
– sometimes “dependency”
• Configuration of parents establishes contexts for a node
– Different distribution for each context
– Conditional Probability Tables (CPTs)
• No directed cycles (DAG)
• General graphical models
– Markov Networks
– Conditional random fields
– Dynamic Bayesian nets
– others
Dentist Example
3 Boolean Random Variables:
C
C – Patient has a cavity
A – Patient reports a
toothache
B – Dentist’s probe catches A B
on tooth
C directly influences A and B
A and B are conditionally independent given C
CPT at each node specifies a probability distribution for each context
(configuration of parents)
How many numbers do we need for the full joint?
How many for the Bayesian net?
How Many Numbers
C
Joint:
23 – 1 = 7 (why?)
Bayes Net:
A B
Distribution(s) at C
P(cavity), P(cavity)
Need one number
Distribution(s) at A
P(ache|cavity), P(ache|cavity)
P(ache| cavity), P(ache| cavity)
Distributions at B, like A
1 + 2 + 2 = 5 (a savings of 2!)
How Many Numbers?
• Add binary random variables for
– Weather W
– Weekday / Weekend D W
C D
• How many numbers for
the joint?
A B
• How many numbers
for the BN?
• What does the BN look like?
How Many Numbers?
• Suppose A,B,C,D,W are trinary
• How many numbers for
the joint? 35 – 1 = 242 C D
W
• How many numbers
for the BN? 2+2+2+6+6 = 18
A B
• Suppose A is binary,
B is trinary, C takes on 4 values
D takes on 5, W takes on 6?
W 5, D 4, C 3, A 4, B 8 = 24
Notation
• W, C, D, A, B are random variables
– C (binary presence of cavity) ranges over values
– C’s values might be Yes or No; 1 or 0; c1 or c2

• Probability distribution: P(C)


• Probability value: P(C=c1) or P(c1) or P(c)

• P(A,B,C,W,D) full joint probability distribution


• P(a,b,c,w,d) some particular entry in the joint

You might also like