An Introduction to Probabilistic Graphical Models 【微信公众号:一介狂书生】
An Introduction to Probabilistic Graphical Models 【微信公众号:一介狂书生】
MODELS
David Madigan
Rutgers University
[email protected]
Expert Systems
If A then C (p1)
If B then C (p2)
•Computationally intractable
•Inscrutable
•Requires vast amounts of data/elicitation
X Y|Z ! f X ,Y |Z ( x, y | z ) = f X |Z ( x | z ) fY |Z ( y | z )
Conditional Independence
A B C
C A|B
versus 250-1
Properties of Conditional Independence (Dawid, 1980)
CI 2: A B ∪ C [P] ⇒ A B [P]
CI 3: A B ∪ C [P] ⇒ A B | C [P]
(G) ⇔ (L)
X1
X1
X3
X3
X2
X2
A p(A,B,C,D,E,F,G,H,S) =
p(A)p(C|A)p(D|C)p(S|D,F)p(E|S)
C G B p(F|G)p(G|B)p(H|S,B)p(B)
D F ⇒
S
H
p(S|A,B,C,D,E,F,G,H) ∝
E p(S|D,F)p(E|S)p(H|S,B)
A B C
C A|B
A A D
A B C
B
C C B A C|B
A C A B | C,D A B C
C D | A,B
A C|B
A B C
A C|B
UG – ADG Intersection
UG ADG
Decomposable
•The intersection of each set with those earlier in the list is fully contained
in previous set
•Can compute cond. probabilities (e.g. Pr(X|V)) by message passing
(Lauritzen & Spiegelhalter, Dawid, Jensen)
Probabilistic Expert System
•Computationally intractable
•Inscrutable
•Requires vast amounts of data/elicitation
A
A
ψ (C,S,D) ← Pr(S|C, D)
E F ψ(A,E) ← Pr(E|A) Pr(A)
E F ψ (C,E) ← Pr(C|E)
ψ(F,D,B) ← Pr(D|F)Pr(B|F)Pr(F)
C D
B C D ψ (D,B,S) ← 1
B
S ψ (B,S,G) ← Pr(G|S,B)
S ψ (H,S) ← Pr(H|S)
H
G H
G
•Moralize
•Triangulate
Pr(C|B)=0.2 Pr(C|¬B)=0.6
A B C Pr(B|A)=0.5 Pr(B|¬A)=0.1
Pr(A)=0.7
ψ(A,B) ← Pr(B|A)Pr(A)
A B C
ψ (B,C) ← Pr(C|B)
B ¬B C ¬C
B ¬B
AB B BC A 0.35 0.35 B 0.2 0.8
1 1
¬A 0.03 0.27 ¬B 0.6 0.4
Survival Survival
No Yes No Yes
Less 3 176 1.7% Less 17 197 7.9%
Care Care
More 4 293 1.4% More 2 23 8.0%
Clinic A Clinic B
Survival
No Yes
Less 20 373 5.1%
Care
More 6 316 1.9%
Pooled
Collapsibility
• Five parameters:
Local and Global Independence
Bayesian learning
Consider a particular state pa(v)+ of pa(v)
Equivalence Classes and Chain Graphs
A C|B
b b b
a d a d a d
A D | B,C
c c c
D1 D2 D3
B C|A
b
a d A D | B,C
c B C
D4
Why is this a problem?
b b
b
c
a d a d
a d
c c
b b b
a d a d a d
c c c
D1 D2 D3
Essential Graphs
AMP (1995)
a b a b a b a b (c1!c2)
c c c c1
(a) (b) (c) (d)
“Equivalence”:
a ~ b iff a b
UG ADG
Decomposable CG
A D
C D | A,B or C D|A ?
C B
Cox & Wermuth (1996)