Huynh Sang 20214930 - Nguyen The An 20210006 - Vu Tung Linh 20210523

Graph Partitioning Problem Fundamentals of Optimization

7 Algorithms
• Backtracking
• Integer/Constraint Programming

Overview
• Spectral Clustering
• Greedy Algorithm
• Kernighan-Lin Local Re nement
• Recursive Bisectioning
• Multilevel Partitioning
fi
 Problem Statement
Given: G = (V, E, w). Need to partition V into k subsets, such that:

❖ max Vi − min Vi ≤ α (alpha-constrained, original task)

i i

[ k ]
|V|
❖
Vi ≤ Vmax := (1 + ϵ) (epsilon-constrained, looks nicer since RHS is const)

And the total weight of cut-edges are minimized, i.e.

∑
❖
minimize w({u, v})
{u,v}∈C

❖ where C = {{u, v} ∈ E : u ∈ Vi, v ∈ Vj,1 ≤ i < j ≤ k}

 Backtracking Approach
For the backtracking problem, the hardest thing is to come up with an algorithm that can
generate all k-partitions ef ciently.
Note that, there exists a bijection between the set of all partitions and this set:
{(k1, . . . , kn) : ∀2 ≤ i ≤ n : k1 ≤ ki ≤ max kj + 1,ki ∈ ℤ}
1≤j<i

Which means, a number has to be in an existing partition or has to be in a new partition.

For instance, the 3-partition {{1,3},{2,4},{5}} correspond to the array [1,2,1,2,3].
fi
 Backtracking Approach
To further enhance the algorithm, we augment Branch and Bound:
❖ Generate an initial partition, call the cost lower_bound
❖ When building a solution, if our cost is already larger than lower_bound, then cut
that branch
And to even reduce the size of our search tree, we turn to our epsilon-constraint:
❖ When building a partition, if there is a subset whose size is already larger than
Vmax, then we can cut that branch.
 Integer/Constraint Programming
IP formulation:

∀{u, v} ∈ E, ∀p : euv ≥ + xu,p − xv,p ∀{u, v} ∈ E, ∀p : euv ≥ − xu,p + xv,p

∑ ∑
∀p : xv,p ≤ Vmax (*) ∀v ∈ V : xv,p = 1
v∈V p

❖ euv ∈ {0,1}: if {u, v} is a cut-edge or not,

❖ xv,p ∈ {0,1}: if v ∈ Vp or not.
❖ The rst two constraint allows for de nition of cut-edge
❖ The second two constraint allows for valid partition and valid subset size.
fi
fi
 Integer/Constraint Programming
If our problem is alpha-constrained, then we can replace constraint (*) with:

∑
❖
∀p : xv,p ≤ M
v∈V

∑
❖
∀p : xv,p ≥ m
v∈V

❖ M−m≤α
Ultimately, the objective function:

∑
minimize  euvw({u, v})
{u,v}∈E

We implement this model into both OR-Tools' CP solver and SCIP solver.
 Spectral Clustering
❖ Powerful algorithm for non-convex clusters
❖ Make use of the spectrum (eigenvector, eigenvalues) to reduce dimension
(similar concept as Principal Components Analysis)
❖ Input:
❖ W: our adjacency weight matrix (symmetric)
❖ D: degree matrix (di = deg vertex i) (diagonal)
❖ L = D − W: the Laplacian (symmetric)
 Spectral Clustering
Properties of the Laplacian:
❖ Positive-de nite
❖ Smallest eigenvalue is λ1 = 0, corresponding eigenvector is 1.
❖ Has n non-negative real eigenvalues λ1 ≤ λ2 ≤ . . . ≤ λn.
❖ Multiplicity of λ = 0 is the number of connected components in the graph.

Proof: refer to https://people.csail.mit.edu/dsontag/courses/ml14/notes/

Luxburg07_tutorial_spectral_clustering.pdf
fi
 Unnormalized Spectral Algorithm
Input: adjacency weight matrix W
❖ Compute the (unnormalized) Laplacian L
❖ Compute the rst k eigenvectors of L: v1, v2, . . . , vk
n×k
❖ Let U ∈ ℝ : matrix with columns v1, v2, . . . , vk
k
❖ For i = 1,...,n, let yi ∈ R be the vector corresponding to the i-th row of U.
k
❖ Cluster the points (yi), i = 1,...,n in R with the k-means algorithm into clusters
C1, . . . , Ck.
Output: Clusters A1, . . . , Ak with Ai = {j : yj ∈ Ci}.
fi
 Spectral Clustering
Implementation:

Note that, this is a very basic

spectral algorithm; there are
variations with normalized Laplacian,
etc.

Better to import from

sklearn.cluster.SpectralClustering
than to reimplement our own version
in practice.












Spectral Clustering

Clusterings from Ng, Jordan paper

 Greedy Graph Growing Partition

❖ Very simple greedy framework:

 Greedy Graph Growing Partition

❖ Our greedy function:

❖ Algorithm most likely is not going to perform well, but could run for many
iterations and get usable partition.
 Kernighan-Lin Local Refinement
❖ Historically important; one of the rst practical heuristics for graph
partitioning

❖ Invented by Kernighan and Lin (1970)

❖ Designing electronic layout of

Very Large-Scale Integration

A VLSI-Integrated Circuit die



fi
 Kernighan-Lin Local Refinement
❖ Perform re nement on bipartition P = (P1, P2)
❖ Choose X1 ∈ P1, X2 ∈ P2 to swap

❖ Algorithm choose iteratively:

❖ Choose 2 vertices from P1\X1, P2\X2
such that swapping them would
yield maximum decrease in cut-size
❖ Stop when no improvement can be made
fi




 Recursive Bisectioning
❖ Simply: Bisection using
Kernighan-Lin "recursively"
❖ Use random partition instead of greedy
to generate initial bipartition
❖ Only works for k being power of 2
❖ Implementation can get rid of
recursion using while loop
❖ A basic, naive multilevel partitioner






 Multilevel Algorithms
❖ A practical and useful paradigm in partitioning problems
❖ Phase 1: Graph contraction (coursening),
many approaches:

maximal matching,
maximal cliques, etc.
❖ Phase 2: Partition
When the contracted graph
is small enough, carry out
partition using IP/greedy/...














 Multilevel Algorithms
❖ Phase 3: Uncoursening:

Make use of local search heuristics

for re ning partitions.

Re ne partition = swap vertices from one

subset to another to decrease cost
or rebalance (such as Kernighan-Lin)
❖ Exists many other modules: METIS,
KaHyPar, etc. with good contraction and
re nement schemes




fi
fi
fi














Results: Datasets
 Results
Small sample: set2

Remarks in small test sets: avgtime optimal? avgcost

backtrack(alpha) 0.0839 yes 583
❖ Backtracking, IP, CP can deliver exact
ip(alpha) 0.0979 yes 583
solutions quickly
cp(alpha) 0.0321 yes 583
❖ Greedy algorithm may get optimal backtrack(eps) 0.0487 yes 523
results in smaller test sets; difference ip(eps) 0.0791 yes 523
from exact solution is acceptable cp(eps) 0.0258 yes 523
❖ With KL re nement, result becomes greedy(500) 0.1089 no 683
better greedy(1000) 0.2135 no 652
greedy+KL 0.0044 sometimes 597
❖ Spectral's imbalance is noticable
spectral 0.0047 not 477
balance
fi








 Results
Medium sample: set3 (n=50, sparse) , set4 (n=50, dense)

Remarks in medium sample set4 avgtime avgcost

❖ set3: IP/CP can terminate in <1s ip(alpha) limit to 10s 17549

❖ set4: cannot reach exact sol. cp(alpha) limit to 10s 15482

❖ epsilon formulations are slightly faster ip(eps) limit to 10s 17263

❖ IP performed worse than CP cp(eps) limit to 10s 15228

greedy(500) 1.4687 17114
❖ Greedy alone can yield usable results
❖ Greedy+KL: very good greedy(1000) 2.9194 16890

❖ Spectral: good in time & cost; greedy+KL 0.0062 15329

balancedness questionable spectral 0.0061 14431


 Results
set6.2 avgtime avgcost
ip(eps) limit to 3s 85042
cp(eps) limit to 3s 78492
Large sample: greedy(50) 1.1566 82878
set6.2 (n=100, m=2250, k=4) spectral 0.0075 73201
RB 0.0493 72660
❖ IP shows much poorer performance
comparing to CP set7.1 avgtime avgcost
❖ Recursive Bisection did very good (time and partition quality) greedy(1) 21.212 504179
set7.1 (n=1000, m=20000, k=2) greedy+KL 64.318 377295
❖ greedy+KL: impressive but slow
spectral 0.216 390367
kahip(strong) 9.243 379925
❖ greedy: not very competitive if
RB 58.2948 377948
run for few iterations
set7.2 avgtime avgcost
set7.2 (n=1000, m=20000, k=8)
greedy 23.534 878642
❖ kahip: excellent + can be fast if necessary
spectral 0.282 741639
❖ spectral: good and fast kahip(strong) 13.590 711914
❖ Recursive Bisection did good but slow (maybe as k grows big, RB takes more time)
kahip(fast) 0.677 732573
RB 77.8854 718323




 Future works
❖ Direct k-way partitioning (for arbitrary k's)
❖ Faster, more complex coursening and re nement techniques
❖ Explore different techniques, i.e. Genetic Algorithms, etc.
Final remarks:
❖ In practice, we should use available tools, e.g. KaHIP module, because it will
most likely perform better than our own implementation in both partition
quality and time taken.
❖ Very impressed about Recursive Bisectioning (implemented last night)
fi
THANKS FOR LISTENING