On Continuous and Combinatorial Relaxations

of Graph Isomorphism

Albert Atserias
Universitat Politècnica de Catalunya
Barcelona, Catalonia, EU

Based on joint work with

Elitza Maneva (University of Barcelona)
Indistinguishability
Overview of the talk

Overview:

1. Iterated degree sequences and Weisfeiler-Lehman algorithm

2. Fractional isomorphisms and Sherali-Adams relaxations

3. Transfer Lemma

4. Indistinguishability in counting logics

5. Applications
 Part I

ITERATED DEGREE SEQUENCES

Iterated degree sequences

Let G = (V , E ) be a graph.
Use u to denote a vertex, and NG (u) for its neighborhood.

Start at the degree sequence:

d1 (u) := |NG (u)|,

d1 (G ) := {{ d1 (u) : u ∈ V }}.

Iterate:

di +1 (u) := {{ di (v ) : v ∈ NG (u) }},

di +1 (G ) := {{ di +1 (u) : u ∈ V }}.

Take the limit:

D(G ) := (d1 (G ), d2 (G ), d3 (G ), . . .).

Indistinguishability by iterated degree sequences

Definition:

G∼
=D H iff D(G ) = D(H).
Indistinguishability by iterated degree sequences

∼
=D is strong...

Theorem [Babai-Erdös-Selkow 80]:

Let G = G (n, 1/2) be drawn randomly. Then, a.s. as n → ∞,

for every H with n vertices we have G ∼
=D H iff G ∼
= H.
Indistinguishability by iterated degree sequences

∼
=D is strong...

Theorem [Babai-Erdös-Selkow 80]:

Let G = G (n, 1/2) be drawn randomly. Then, a.s. as n → ∞,

for every H with n vertices we have G ∼
=D H iff G ∼
= H.

But also weak...

Fact [Obvious]:

If G and H are both d-regular, then G ∼

=D H.
Types of k-tuples

For a k-tuple of vertices u = (u1 , . . . , uk ) ∈ V k ,

Define:

tpG (u) = “complete information about adjacencies,

non-adjacencies, equalities and non-equalities
between the components u1 , . . . , uk ”.

Example:

tpG (u1 , u2 , u3 ) = {E (1, 1), E (1, 2), E (1, 3),

E (2, 1), E (2, 2), E (3, 2),
E (3, 1), E (3, 2), E (3, 3),
1 6= 2, 1 6= 3, 2 = 3}
k-dimensional Weisfeiler-Lehman algorithm

Start at the type sequence:

ℓ0 (u) := tpG (u),

ℓ0 (G ) := {{ ℓ0 (u) : u ∈ V k }}.

Iterate:

ℓi +1 (u) := {{ (tpG (uv ), ℓi (u[1/v ]), . . . , ℓi (u[k/v ])) : v ∈ V }},

ℓi +1 (G ) := {{ ℓi +1 (u) : u ∈ V k }}.

Take the limit:

D k (G ) := (ℓ0 (G ), ℓ1 (G ), . . .).
Indistinguishability by k-dim WL

Definition:

G∼
=kWL H iff D k (G ) = D k (H).
Indistinguishability by k-dim WL

∼
=kWL is strong...

At least as strong as vertex-refinement:

G∼
6 D H =⇒ G ∼
= 6 1WL H
=

Theorem [Kucera 87]:

Let G = Greg (n, d) be drawn randomly. Then, a.s. as n → ∞,

for every H with n vertices we have G ∼
=2WL H iff G ∼
= H.
Indistinguishability by k-dim WL

∼
=kWL is strong...

At least as strong as vertex-refinement:

G∼
6 D H =⇒ G ∼
= 6 1WL H
=

Theorem [Kucera 87]:

Let G = Greg (n, d) be drawn randomly. Then, a.s. as n → ∞,

for every H with n vertices we have G ∼
=2WL H iff G ∼
= H.

Relevant note:
∼
=kWL is decidable in time nO(k) .
Is k-dim WL weak at all?

Truth is:

For years no two ∼

=37
WL -indistinguishable graphs were known...
It was even conjectured that no such graphs existed...
Is k-dim WL weak at all?

Truth is:

For years no two ∼

=37
WL -indistinguishable graphs were known...
It was even conjectured that no such graphs existed...

Theorem [Cai-Fürer-Immerman 92]:

There exists explicitely defined graphs Gn and Hn ,

with n vertices each and maximum degree 3, such that
Ω(n)
Gn ∼
=WL Hn yet Gn ∼
6 Hn .
=

Note:

Reasoning about = ∼k requires an excursion

WL
into finite model theory (more on this later).
CFI-construction
1. Start with a 3-regular graph G without Ω(n)-separators.
CFI-construction
1. Start with a 3-regular graph G without Ω(n)-separators.

2. Replace each vertex by gadget:

CFI-construction
1. Start with a 3-regular graph G without Ω(n)-separators.

2. Replace each vertex by gadget:

3. Let Gn be the result and let Hn = Gn + “one flip”.

 Part II

SHERALI-ADAMS RELAXATIONS
Adjacency matrices

Let G = (V G , E G ) and H = (V H , E H ) be graphs.

Say V G = V H = {1, . . . , n}.
Let A and B be their adjacency matrices.

   
0 1 1 1 1 1 0 1 1 1 0 0

 1 0 1 0 0 1 


 1 0 1 1 0 0 

 1 1 0 1 0 0   1 1 0 1 0 0 
A=  B = 

 1 0 1 0 1 0 


 1 1 1 0 1 1 

 1 0 0 1 0 1   0 0 0 1 0 1 
1 1 0 0 1 0 0 0 0 1 1 0
Permutation matrices and isomorphisms
A permutation matrix P is a real matrix such that
Pn
P =1 for every i ∈ {1, . . . , n},
Pnj=1 ij
i =1 ij = 1
P for every j ∈ {1, . . . , n},
Pij ∈ {0, 1} for every i , j ∈ {1, . . . , n}.

Properties:
• P TP = I ,
• A 7→ AP: permutes the columns of A,
• A 7→ P T A: permutes the rows of A,
• A 7→ P T AP: permutes the vertices.

Fact: The following are equivalent:

1. G ∼
= H,
2. there exists P ∈ Pn such that P T AP = B,
3. there exists P ∈ Pn such that AP = PB.
Doubly stochastic matrices and fractional isomorphisms

A doubly stochastic matrix S is a real matrix such that:

Pn
S = 1 for every i ∈ {1, . . . , n},
Pnj=1 ij
i =1 Sij = 1 for every j ∈ {1, . . . , n},
Sij ≥ 0 for every i , j ∈ {1, . . . , n}.

Relaxation of isomorphism:
• Replace “there exists P ∈ Pn such that AP = PB”
• by this “there exists S ∈ Sn such that AS = SB”.

In other words, let I (G , H) be the LP for Sn plus

Pn Pn
i =1 Aui Siv = j=1 Suj Bjv

for every u, v ∈ V G × V H .
Indistinguishability by fractional isomorphisms

Definition:

G∼
=F H iff I (G , H) 6= ∅.
Indistinguishability by fractional isomorphisms

Suppose G ∼
=F H. Then:
• |E G | = |E H |,
• actually d1 (G ) = d1 (H),
• and even D(G ) = D(H).
Indistinguishability by fractional isomorphisms

Suppose G ∼
=F H. Then:
• |E G | = |E H |,
• actually d1 (G ) = d1 (H),
• and even D(G ) = D(H).

Indeed:

Theorem [Ramana-Scheinerman-Ullman 94]

G∼
=F H iff G ∼
=D H.
Sherali-Adams relaxations

Let

P = {x ∈ Rn : Ax ≥ b},
P Z = convexhull{x ∈ {0, 1}n : Ax ≥ b}.

The Sherali-Adams levels are nested polytopes:

P = P0 ⊇ P1 ⊇ P2 ⊇ · · · ⊇ Pn = PZ

and the SA-rank of P is:

min{k : P k = P Z }.
Definition of P k in four steps

Let  
a1T x ≥ b1 


:  ...
 
P = x ∈ Rn  .
 
 T

a m x ≥ bm
 

be the LP.
Definition of P k in four steps

Step 1: Multiply each aiT x ≥ bi by all multipliers of the form

Y Y
xi (1 − xj )
i ∈I j∈J

for I , J ⊆ [n], |I ∪ J| ≤ k − 1, I ∩ J = ∅.
Definition of P k in four steps

Step 1: Multiply each aiT x ≥ bi by all multipliers of the form

Y Y
xi (1 − xj )
i ∈I j∈J

for I , J ⊆ [n], |I ∪ J| ≤ k − 1, I ∩ J = ∅.

Step 1 leaves an equivalent system of polynomials of degree k.

Definition of P k in four steps

Step 2: Expand the products and replace each square xi2 by xi .

Definition of P k in four steps

Step 2: Expand the products and replace each square xi2 by xi .

Step 2 leaves a system of multi-linear polynomials of degree k.

This is the integrality step: valid on {0, 1}n only.
Definition of P k in four steps

Q
Step 3: Linearize each monomial i ∈I xi by introducing a new
variable yI .
Definition of P k in four steps

Q
Step 3: Linearize each monomial i ∈I xi by introducing a new
variable yI .
k
Step 3 leaves a linear program Q k on the yI -variables in Rn .
This is the relaxation step.
Definition of P k in four steps

Step 4: Define

P k := {x ∈ Rn : ∃y ∈ Q k s.t. y{i } = xi for every i }.

Definition of P k in four steps

Step 4: Define

P k := {x ∈ Rn : ∃y ∈ Q k s.t. y{i } = xi for every i }.

Step 4 takes us back to Rn .

k
It’s the projection step: from Rn to Rn .
Solving P k

Note:

The polytope P k is definable by

an LP on nk variables and m · nk inequalities.

Therefore:

Feasibility and optimization of linear functions over P k

can be solved in time mO(1) nO(k) .
Indistinguishability by SA-levels of fractional isomorphisms

Definition:
G∼
=kSA H iff I (G , H)k 6= ∅.
 Part III

TRANSFER LEMMA
Statement of the transfer lemma

Transfer Lemma:

G ∼
=kWL H =⇒ G ∼k−1
=SA H =⇒ G ∼ k−1
=WL H.

Interpretation:

A geometric concept is captured by purely combinatorial means.

A combinatorial concept is captured by purely geometric means.
Proof of the transfer lemma

Intermediate notions of indistinguishability:

G ∼
=kWL H ⇒ G ∼
=kC H ⇒G ∼ k−1
=CS H ⇒G ∼ k−1
=EP H ⇒G ∼ k−1
=SA H

and
G ∼ k−1
=SA ∼k−1 H ⇒G =
H ⇒G = C
∼k−1 H.
WL
Proof of the transfer lemma

Intermediate notions of indistinguishability:

G ∼
=kWL H ⇒ G ∼
=kC H ⇒G ∼ k−1
=CS H ⇒G ∼ k−1
=EP H ⇒G ∼ k−1
=SA H

and
G ∼ k−1
=SA ∼k−1 H ⇒G =
H ⇒G = C
∼k−1 H.
WL

Here:
∼
=kC is indistinguishability by properties definable in
first-order logic with counting quantifiers and width k.
 Part IV

COUNTING LOGICS
Indistinguishability
Counting quantifiers

Counting witnesses:

∃≥i x(φ(x)) : there are at least i vertices x that satisfy φ(x).

Example:

ψd (x) := ∃≥d y (E (x, y )) ∧ ¬∃d+1 y (E (x, y )),

φ := ¬∃≥1 x(¬ψd (x)).

Note:

We used only two first-order variables (x and y )

where d + 1 are required in pure first-order logic.
Bounded width formulas
Example: First paths

P1 (x, y ) := E (x, y )
P2 (x, y ) := ∃z1 (E (x, z1 ) ∧ P1 (z1 , y ))
P3 (x, y ) := ∃z2 (E (x, z2 ) ∧ P2 (z2 , y ))
..
.
Pi +1 (x, y ) := ∃zi (E (x, zi ) ∧ Pi (zi , y )).

and then

∀x(¬P3 (x, x) ∧ ¬P5 (x, x) ∧ · · · ∧ ¬P2⌈n/2⌉−1 (x, x)).

Counting logic with k variables:

C k : collection of formulas for which
all subformulas have at most k free variables.
Indistinguishability by C k

Definition:

G∼
=kC H iff for every φ ∈ C k we have G |= φ ⇔ H |= φ.
Pebble game (without counting moves)

Forced win for Spoiler.

Pebble game (without counting moves)

Forced win for Spoiler.

Pebble game (without counting moves)

Forced win for Spoiler.

Pebble game (without counting moves)

Forced win for Spoiler.

Pebble game (without counting moves)

Forced win for Spoiler.

Pebble game (without counting moves)

Forced win for Spoiler.

Pebble game WITH counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Pebble game with counting moves

Forced win for Spoiler.

Systems with the back-and-forth properties

A winning strategy for the Duplicator in G ∼

=kC H is a non-empty
collection F of partial isomorphisms from G to H such that for
every f ∈ F we have:
Systems with the back-and-forth properties

A winning strategy for the Duplicator in G ∼

=kC H is a non-empty
collection F of partial isomorphisms from G to H such that for
every f ∈ F we have:
1. (bounded) |Dom(f )| ≤ k,
Systems with the back-and-forth properties

A winning strategy for the Duplicator in G ∼

=kC H is a non-empty
collection F of partial isomorphisms from G to H such that for
every f ∈ F we have:
1. (bounded) |Dom(f )| ≤ k,
2. (subfunction) For every g ⊆ f we have g ∈ F,
Systems with the back-and-forth properties

A winning strategy for the Duplicator in G ∼

=kC H is a non-empty
collection F of partial isomorphisms from G to H such that for
every f ∈ F we have:
1. (bounded) |Dom(f )| ≤ k,
2. (subfunction) For every g ⊆ f we have g ∈ F,
3. (back) If |Dom(f )| < k then:
for every X ⊆ VG there exists Y ⊆ VH with |Y | = |X | s.t.
for every v ∈ Y there exists u ∈ X with f ∪ {(u, v )} ∈ F,
Systems with the back-and-forth properties

A winning strategy for the Duplicator in G ∼

=kC H is a non-empty
collection F of partial isomorphisms from G to H such that for
every f ∈ F we have:
1. (bounded) |Dom(f )| ≤ k,
2. (subfunction) For every g ⊆ f we have g ∈ F,
3. (back) If |Dom(f )| < k then:
for every X ⊆ VG there exists Y ⊆ VH with |Y | = |X | s.t.
for every v ∈ Y there exists u ∈ X with f ∪ {(u, v )} ∈ F,
4. (forth) If |Dom(f )| < k then:
for every Y ⊆ VH there exists X ⊆ VG with |X | = |Y | s.t.
for every u ∈ X there exists v ∈ Y with f ∪ {(u, v )} ∈ F.
Counting pebble game vs. Weisfeiler-Lehman algorithm

Theorem [Immerman-Lander 90, Cai-Fürer-Immerman 92]

G∼
=kWL H ⇐⇒ G ∼
=k+1
C H.

Relevant note: From its definition, it is not even obvious that

G ∼
=kC H is decidable in time nO(k) .
From feasible solutions to systems with B&F

Wanted:
G∼
=kSA H =⇒ G ∼
=kC H
Ingredient 1:

Birkhoff decomposition theorem: every doubly stochastic matrix is

a convex combination of permutation matrices.

Ingredient 2:

Permutations preserve sizes of sets.

From systems with B&F to feasible solutions

Wanted:
G∼
=kC H =⇒ G ∼ k−1
=SA H
Ingredient 1:

A sliding game to account for AS = SB;

here is where the −1 is lost.

Ingredient 2:

Normalizing winning strategies into uniform ones.

 Part V

APPLICATIONS (or what to do of this?)

Isomorphism testing for special graphs

Theorem [Immerman-Lander 90, Grohe 98, ...]

1. If G is a tree, then G ∼
=2C H iff G ∼
= H, for every H.
∼ H iff G =
2. If G is planar, then G = 15 ∼ H, for every H.
C
3. · · ·

Corollary

For all such graph classes,

an explicit and poly-size LP
solves graph isomorphism.
SA-rank lower bounds
Consider the standard LP-relaxation of vertex cover:
P
minimize u∈V xu
subject to
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V .
SA-rank lower bounds
Consider the standard LP-relaxation of vertex cover:
P
minimize u∈V xu
subject to
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V .

We expect that the inequality

P
u∈V xu ≥ vc(G ) (1)

will not, in general, be valid over P k (G ) for any k = O(1).

SA-rank lower bounds
Consider the standard LP-relaxation of vertex cover:
P
minimize u∈V xu
subject to
xu + xv ≥ 1 for every (u, v ) ∈ E ,
xu ≥ 0 for every u ∈ V .

We expect that the inequality

P
u∈V xu ≥ vc(G ) (1)

will not, in general, be valid over P k (G ) for any k = O(1).

Indeed:

Theorem [exercise, also follows Schoenebeck 08]

There exist graphs G for which (2) is not valid over P Ω(n) (G ).
New proof

Sketch:
Ω(n)
1. Start with the n-vertex CFI graphs G ∼ =C H yet G ∼ 6 H.
=
Ω(n)
=C (G , H) yet G ∼
2. In particular (G , G ) ∼ = G and G ∼
6 H.
=
3. Apply the reduction from graph isomorphism to vertex cover.
Ω(n)
4. Get graphs A ∼ =
C B with vc(A) 6= vc(B).
Ω(n)
5. Apply transfer lemma and get A ∼
=SA B.

Final step:

A∼
=2k k k
SA B =⇒ opt(P (A)) = opt(P (B)).
SA-rank lower bounds
Consider the standard LP-relaxation of max-cut:
maximize 21 uv ∈E xuv
P

subject to
xuv = xvu
xuw ≤ xuv + xvw
xuv + xvw + xwu ≤ 2
0 ≤ xuv ≤ 1
SA-rank lower bounds
Consider the standard LP-relaxation of max-cut:
maximize 21 uv ∈E xuv
P

subject to
xuv = xvu
xuw ≤ xuv + xvw
xuv + xvw + xwu ≤ 2
0 ≤ xuv ≤ 1
We expect that the inequality
P
u∈V xu ≤ mc(G ) (2)
will not, in general, be valid over P k (G ) for any k = O(1).
SA-rank lower bounds
Consider the standard LP-relaxation of max-cut:
maximize 21 uv ∈E xuv
P

subject to
xuv = xvu
xuw ≤ xuv + xvw
xuv + xvw + xwu ≤ 2
0 ≤ xuv ≤ 1
We expect that the inequality
P
u∈V xu ≤ mc(G ) (2)
will not, in general, be valid over P k (G ) for any k = O(1).
Indeed:

Theorem [follows from Schoenebeck 08]

There exist graphs G for which (2) is not valid over P Ω(n) (G ).
New proof

Sketch:
Ω(n)
1. Start with the n-vertex CFI graphs G ∼ =C H yet G ∼ 6 H.
=
Ω(n)
=C (G , H) yet G ∼
2. In particular (G , G ) ∼ = G and G ∼
6 H.
=
3. Apply the reduction from graph isomorphism to max-cut.
Ω(n)
4. Get graphs A ∼ =
C B with mc(A) 6= mc(B).
Ω(n)
5. Apply transfer lemma and get A ∼
=SA B.

Final step:

A∼
=3k k k
SA B =⇒ opt(P (A)) = opt(P (B)).
Local LPs
Basic k-local LPs:
1. one variable xu for each k-tuple u ∈ V k ,
2. one inequality u∈V k au,v · xu ≥ bv for every k-tuple v ∈ V k ,
P

3. coefficients au,v depend only on the type tpG (u, v),

4. coefficients bv depend only on the type tpG (v).

Generic k-local LPs:

Unions of generic basic k-local LPs

(with coefficients given as a function of the types).

Instantiation of generic k-local LPs:

Let P is a generic k-local LP.

Then P(G ) is the LP associated to G .
Metric polytope

Recall the metric polytope:

1P
2 uv ∈E xuv ≥ W
xuv = xvu
xuw ≤ xuv + xvw
xuv + xvw + xuw ≤ 2
0 ≤ xuv ≤ 1

1. Objective function: basic 2-local LP

2. Symmetry constraint: two basic 2-local LPs
3. Triangle inequality: basic 3-local LP
4. Perimetric inequality: basic 3-local LP
5. Unit cube constraint: two basic 2-local LPs
Preservation of local LPs

Theorem: Let P be a generic k-local LP.

If G ∼
=kSA H, then P(G ) is feasible iff P(H) is feasible.

’Just do it’ proof:

1. Let {xu } be a feasible solution for P(G ).
2. Let {Xu,v } be a feasible solution for I (G , H)k .
3. Define: X
yv := Xu,v · xu .
u∈G k

4. Check that {yv } is a feasible solution for P(H).

More examples of local LPs

More examples:
1. maximum flows (2-local)
2. matchings on bipartite graphs (2-local)
3. relaxation of max-cut via the metric polytope (3-local)
4. relaxation of vertex cover (2-local)
5. r SA-levels of k-local LPs are O(kr )-local LPs.
Expressibility results

Consider the max-flow LP. It is 2-local. It is integral.

Corollary

G∼
=3C H ⇒ mf(G ) = mf(H).

Corollary

There exists a sentence in C 3 that,

over st-networks with n vertices, defines those
whose maximum flow is at least the out-degree of the source.
Expressibility results

Consider the metric polytope again.

Theorem [Barahona-Majoub 86]:

If G is a K5 minor-free graph, then mc(G ) = opt(P(G )).

Corollary

If G and H are K5 minor-free, then G ∼

=4C H ⇒ mc(G ) = mc(H).

Corollary

There exists a sentence in C 4 that,

over K5 minor-free n-vertex graphs, defines those
whose max-cut is at least n/4.
 Part VI

DISCUSSION AND OPEN PROBLEMS

Get new rank lower bounds from inexpressibility results?

Challenging problem:

Prove that an integrality gap of 2 − ǫ

resists Ω(n) SA-levels of vertex-cover.
Get new rank lower bounds from inexpressibility results?

Challenging problem:

Prove that an integrality gap of 2 − ǫ

resists Ω(n) SA-levels of vertex-cover.

What would be enough?:

Find G and H such that:
1. mc(G ) ≥ (2 − ǫ) · mc(H)
Ω(n)
2. G ∼
= C H.
New expressibility/inexpressibility results?

Challenging problem:

Is perfect matching definable in C O(1) ?

(answer is YES for bipartite graphs)
New expressibility/inexpressibility results?

Challenging problem:

Is perfect matching definable in C O(1) ?

(answer is YES for bipartite graphs)

SOLVED! [Anderson-Dawar-Holm 13]:

YES even for general graphs!

TODA!