Causal and Probabilistic Reasoning

Slides Set 2:
Rina Dechter

Reading:
Darwiche chapter 4
Pearl (probabilistic): chapter 3

slides2 Winter 2024

Outline
◼ Basic of Probability Theory
◼ Bayesian Networks, DAGS, Markov(G)
◼ Graphoids axioms for Conditional
Independence
◼ d-separation: Inferring CIs in graphs

slides2 Winter 2024

Basics of Probabilistic Calculus
(Chapter 3)

Winter 2024 3
The Burglary Example

Earthquake Burglary

Alarm
Radio

Call

Winter 2024 4
Winter 2024 5
Winter 2024 6
Winter 2024 7
Winter 2024 8
Winter 2024 9
Winter 2024 10
Winter 2024 11
Alpha and beta are events

Winter 2024 12
Winter 2024 13
Burglary is independent of Earthquake

Winter 2024 14
Earthquake is independent of burglary

Winter 2024 15
Winter 2024 16
Winter 2024 17
Winter 2024 18
Winter 2024 19
Winter 2024 20
Winter 2024 21
Outline
◼ Basic of Probability Theory
◼ Bayesian Networks, DAGS, Markov(G)
◼ From a distribution to a BN
◼ From BN to distributions, DAGs, Markov(G)
◼ Parameterization
◼ Graphoids axioms for Conditional
Independence
◼ D-separation: Inferring CIs in graphs
slides2 Winter 2024
Outline
◼ Bayesian Networks, DAGS, Markov(G)
◼ From a distribution to a BN
◼ From BN to distributions, DAGs, Markov(G)
◼ Parameterization
◼ Graphoids axioms for Conditional
Independence
◼ D-separation: Inferring CIs in graphs

slides2 Winter 2024

 Bayesian Networks (BNs) in 2 ways:

From a distribution to a BN:

• A Bayesian network is factorize probability distribution along an ordering.
• The DAG emerging is a Bayesian network of the distribution
• The factorization is guided by a set of Markov assumption that transform
the chain product formula into a Bayesian network.

From a BN to a distribution:
• Generate a DAG with its Markov assumptions.
• Parameterize the DAG yielding a Bayesian network which corresponds to a
single probability distribution obtained by product.

• The BN distribution obeys additional independence assumption read from the DAG
and can be proved using the Graphoid axioms.

slides2 Winter 2024

 Earthquake Burglary

Alarm

John Marry

slides2 Winter 2024

slides2 Winter 2024
slides2 Winter 2024
 P(B,E,A,J,M)=?

P(B)P(E|B)P(A|B,E)P(J|B,E,A)P(M|B,E,A,J) =
P(B)P(E|B)P(A|B,E)P(J|A)P(M|A) =

pa(B) = {}, pa(E)={B},P(A)= {B,E}, pa(J) = {A}, pa(M) = {A}

slides2 Winter 2024

 P(B,E,A,J,M)=?

P(B)P(E|B)P(A|B,E)P(J|B,E,A)P(M|B,E,A,J) =
P(B)P(E|B)P(A|B,E)P(J|A)P(M|A) =

pa(B) = {}, pa(E)={B},P(A)= {B,E}, pa(J) = {A}, pa(M) = {A}

slides2 Winter 2024

slides2 Winter 2024
Outline
◼ Bayesian Networks, DAGS, Markov(G)
◼ From a distribution to a BN
◼ From BN to distributions, DAGs, Markov(G)
◼ Parameterization
◼ Graphoids axioms for Conditional
Independence
◼ D-separation: Inferring CIs in graphs

slides2 Winter 2024

 The causal interpretation

slides2 Winter 2024

But influences can be indirect as well.
For example…

slides2 Winter 2024

slides2 Winter 2024
slides2 Winter 2024
Graphs Convey Independence Statements

◼ Directed graphs by graph’s d-separation

◼ Undirected graphs by graph separation
◼ Goal: capture probabilistic conditional
independence by graphs.
◼ We focus on directed graphs.

slides2 Winter 2024

slides2 Winter 2024
slides2 Winter 2024
What are the Markov assumptions here?

slides2 Winter 2024

What are the Markov assumptions here?

slides2 Winter 2024

slides2 Winter 2024
slides2 Winter 2024
A Causal Bayesian Network A non-causal Bayesian Network

slides2 Winter 2024

Outline
◼ Bayesian Networks, DAGS, Markov(G)
◼ From a distribution to a BN
◼ From BN to distributions, DAGs, Markov(G)
◼ Parameterization
◼ Graphoids axioms for Conditional
Independence
◼ D-separation: Inferring CIs in graphs

slides2 Winter 2024

slides2 Winter 2024
slides2 Winter 2024
slides2 Winter 2024
slides2 Winter 2024
 Use GeNie/Smile
To create this network
slides2 Winter 2024
slides2 Winter 2024
slides2 Winter 2024
slides2 Winter 2024
slides2 Winter 2024
 Bayesian Networks: Representation
P(S)
Smoking BN = (G, Θ)
P(C|S) P(B|S)
lung Cancer Bronchitis
CPD:
C B D=0 D=1
0 0 0.1 0.9
0 1 0.7 0.3
P(X|C,S) P(D|C,B) 1 0 0.8 0.2
X-ray Dyspnoea 1 1 0.9 0.1

P(S, C, B, X, D) = P(S) P(C|S) P(B|S) P(X|C,S) P(D|C,B)

Conditional Independencies Efficient Representation

slides2 Winter 2024
Outline
◼ Basic of Probability Theory
◼ Bayesian Networks, DAGS, Markov(G)
◼ Graphoids axioms for Conditional
Independence
◼ d-separation: Inferring CIs in graphs

slides2 Winter 2024

 This independence follows from the Markov assumption

R and C are independent given A

slides2 Winter 2024
Properties of Probabilistic Independence
(Pearl ch 3)

◼ Symmetry:
◼ I(X,Z,Y) → I(Y,Z,X)

◼ Decomposition:
◼ I(X,Z,YW)→ I(X,Z,Y) and I(X,Z,W)

◼ Weak union:
◼ I(X,Z,YW)→I(X,ZW,Y)

◼ Contraction:
◼ I(X,Z,Y) and I(X,ZY,W)→I(X,Z,YW)

◼ Intersection:
◼ I(X,ZY,W) and I(X,ZW,Y) → I(X,Z,YW)
slides2 Winter 2024
slides2 Winter 2024
Pearl’s language:
If two pieces of information are irrelevant to X then each one is irrelevant to X

slides2 Winter 2024

Example: Two coins (C1,C2,) and a bell (B)

slides2 Winter 2024

slides2 Winter 2024
slides2 Winter 2024
slides2 Winter 2024
slides2 Winter 2024
When there are no constraints

slides2 Winter 2024

slides2 Winter 2024
slides2 Winter 2024
Properties of Probabilistic independence

◼ Symmetry:
◼ I(X,Z,Y) → I(Y,Z,X)
Graphoid axioms:
◼ Decomposition: Symmetry, decomposition
◼ I(X,Z,YW)→ I(X,Z,Y) and I(X,Z,W) Weak union and contraction

Positive graphoid:
◼ Weak union: +intersection
◼ I(X,Z,YW)→I(X,ZW,Y)

In Pearl: the 5 axioms

Contraction:
are called Graphids,
◼
◼ I(X,Z,Y) and I(X,ZY,W)→I(X,Z,YW)
the 4, semi-graphois
◼ Intersection:
◼ I(X,ZY,W) and I(X,ZW,Y) → I(X,Z,YW)

slides2 Winter 2024

Outline
◼ DAGS, Markov(G), Bayesian networks
◼ Graphoids: axioms of for inferring
conditional independence (CI)
◼ D-separation: Inferring CIs in graphs
◼ I-maps, D-maps, perfect maps
◼ Markov boundary and blanket
◼ Markov networks

slides3 COMPSCI 2021

Properties of Probabilistic independence

◼ Symmetry:
◼ I(X,Z,Y) → I(Y,Z,X)
Graphoid axioms:
◼ Decomposition: Symmetry, decomposition
◼ I(X,Z,YW)→ I(X,Z,Y) and I(X,Z,W) Weak union and contraction

Positive graphoid:
◼ Weak union: +intersection
◼ I(X,Z,YW)→I(X,ZW,Y)

In Pearl: the 5 axioms

Contraction:
are called Graphids,
◼
◼ I(X,Z,Y) and I(X,ZY,W)→I(X,Z,YW)
the 4, semi-graphois
◼ Intersection:
◼ I(X,ZY,W) and I(X,ZW,Y) → I(X,Z,YW)

slides2 COMPSCI 2021

slides2 COMPSCI 2021
slides2 COMPSCI 2021
Properties of Probabilistic independence

◼ Symmetry:
◼ I(X,Z,Y) → I(Y,Z,X)
Graphoid axioms:
◼ Decomposition: Symmetry, decomposition
◼ I(X,Z,YW)→ I(X,Z,Y) and I(X,Z,W) Weak union and contraction

Positive graphoid:
◼ Weak union: +intersection
◼ I(X,Z,YW)→I(X,ZW,Y)

In Pearl: the 5 axioms

Contraction:
are called Graphids,
◼
◼ I(X,Z,Y) and I(X,ZY,W)→I(X,Z,YW)
the 4, semi-graphois
◼ Intersection:
◼ I(X,ZY,W) and I(X,ZW,Y) → I(X,Z,YW)

slides2 COMPSCI 2021