Combinatorial Optimization f (δ − (v)) = f (δ + (v)) for all v ∈ V \ {s, t} (flow

conservation: the total entering flow equals the

Jens Vygen total leaving flow at any vertex except s and t).
University of Bonn, Research Institute for The goal is to maximize f (δ − (t)) − f (δ + (t)), i.e.,
Discrete Mathematics, Lennéstr. 2, 53113 Bonn, the total amount of flow shipped from s to t. This
Germany is called the value of f .
The figure below shows an example. The left-
Combinatorial optimization problems arise in hand side displays an instance, the capacities are
numerous applications. In general, we look for shown next to the edges. The right-hand side
an optimal element of a finite set. However, this shows an s-t-flow of value 7. This is not optimal.
set is too large to be enumerated; it is implicitly
given by its combinatorial structure. The goal is 9
a b 4
a b
3 3
to develop efficient algorithms by understanding
and exploiting this structure. s 1 1 6 s 1 1 2
7 c t 3 c t
1 Some Important Problems 5 5

We first give some classical examples. We refer

to the chapter on −→ Graph Theory for basic no- 1.3 Matching
tation. In a digraph, we denote by δ + (X) and Given a finite undirected graph (V, E), find a
δ − (X) the set of edges leaving and entering X, matching M ⊆ E that is as large as possible.
respectively; here X can be a vertex or a set of (A matching is a set of edges whose endpoints are
vertices. In an undirected graph, δ(X) denotes all distinct.)
the set of edges with exactly one endpoint in X.

1.4 Knapsack
1.1 Spanning trees
Given n ∈ N, positive integers ai , bi (profit and
Here we are given a finite connected undirected weight of item i, for i = 1, . . . , n), and B (the
graph (V, E) (so V is the set of vertices and E
the set of edges) and weights on the edges, i.e.,
knapsack’s
P P I ⊆ {1, . . . , n}
capacity), find a subset
with i∈I bi ≤ B, such that i∈I ai is as large
c(e) ∈ R for all e ∈ E. The task is to find a set as possible.
P⊆ E such that (V, T ) is a (spanning) tree and
T
e∈T c(e) is minimum. (Recall that a tree is a
connected graph without cycles.) 1.5 Traveling salesman
The figure below shows on the left a set V of
Given a finite set X with metric d, find a bijection
eight points in the Euclidean plane. Assuming
π : {1, . . . , n} → X such that the length of the
that (V, E) is the complete graph on these points
corresponding tour,
and c is the Euclidean distances, the right-hand
side shows an optimal solution. n−1
X
d(π(i), π(i + 1)) + d(π(n), π(1)),
i=1

is as small as possible.

1.2 Maximum flows 1.6 Set covering

Given a finite directed graph (V, E), two vertices Given a finite set U and subsets S1 , . . . , Sn of U ,
s, t ∈ V (source and sink), and capacities u(e) ∈ find the smallest collection of these subsets whose
R≥0 for all e ∈ E, we look for an s-t-flow f :
S
union is U , i.e., I ⊆ {1, . . . , n} with i∈I Si = U
E → R≥0 with f (e) ≤ u(e) for all e ∈ E and and |I| minimum.

1
2

2 General Formulation and Goals However, for NP-hard problems (like the last
three examples in our list), this is impossible un-
2.1 Instances and solutions less P=NP, and consequently one is satisfied with
less (see Section 5).
These problems have many common features.
In each case, there are infinitely many in-
stances, each of which can be described (up to 2.3 Other Goals
renaming) by a finite set of bits, and in some cases Besides developing algorithms and proving their
a finite set of real numbers. correctness and efficiency, combinatorial opti-
For each instance, there is a set of feasible so- mization (and related areas) also comprises other
lutions. This set is finite in most cases. In the work:
maximum flow problem it is actually infinite, but
even here one can restrict w.l.o.g. to a finite set • analyze combinatorial structures, such as
of solutions; see below. graphs, matroids, polyhedra, hypergraphs;
Given an instance and a feasible solution, we • establish relations between different combi-
can easily compute its value. natorial optimization problems: reductions,
For example, in the matching problem, the in- equivalence, bounds, relaxations;
stances are the finite undirected graphs; for each
instance G the set of feasible solutions are the • prove properties of optimal (or near-optimal)
matchings in G; and for each matching, its value solutions;
is simply its cardinality. • study the complexity of problems and estab-
Even if the number of feasible solutions is fi- lish hardness results;
nite, it cannot be bounded by a polynomial in the
instance size (the number of bits that is needed • implement algorithms and analyze their
to describe the instance). For example, there practical performance;
are nn−2 trees (V, T ) with V = {1, . . . , n} (this
• apply combinatorial optimization problems
is Cayley’s formula). Similarly, the number of
to real-world problems.
matchings on n vertices, subsets of an n-element
set, and permutations on n elements, grow expo-
nentially in n. One cannot enumerate all of them 3 Greedy algorithm
in reasonable time except for very small n.
Whenever an instance contains real numbers, The spanning tree problem has a very simple so-
we assume that we can do elementary operations lution: the greedy algorithm does the job. We can
with them, or we actually assume them to be ra- start with the empty set and successively pick a
tionals with binary encoding. cheapest edge that does not create a cycle, until
our subgraph is connected. Formally:

2.2 Algorithms 1. Sort E = {e1 , . . . , em } so that c(e1 ) ≤ · · · ≤

c(em ).
The main goal in combinatorial optimization is to
devise efficient algorithms for solving such prob- 2. Let T be the empty set.
lems.
3. For i = 1, . . . , m do:
Efficient usually means polynomial-time (that if (V, T ∪ {ei }) contains no cycle,
is: the number of elementary steps can be then add ei to T .
bounded by a polynomial in the instance size).
Of course, the faster, the better. In our example, the first four steps would add
Solving a problem usually means always (for ev- the four shortest edges (shown in bold on the left-
ery given instance) computing a feasible solution hand side below). Then the dotted edge is exam-
with optimum value. ined, but it is not added as it would create a cycle.
We give an example of an efficient algorithm The right-hand side shows the final output of the
solving the spanning tree problem in Section 3. algorithm.
 3

Generalizations like optimization over the in-

tersection of two matroids or minimization of sub-
modular functions (given by an oracle) can also
be solved in polynomial time, with more compli-
This algorithm can be easily implemented so cated combinatorial algorithms.
that it runs in O(nm) time, where n = |V | and
m = |E|. With a little more care, a running
time of O(m log n) can be obtained. So this is 4 Duality and Min-Max Equations
a polynomial-time algorithm.
This algorithm computes a maximal set T such Relations between different problems can lead to
that (V, T ) contains no cycle. In other words, many important insights and algorithms. We give
(V, T ) is a tree. It is not completely obvious that some well-known examples.
the output (V, T ) is always an optimal solution,
i.e., a tree with minimum weight. Let us give the 4.1 Max-Flow Min-Cut Theorem
nice and instructive proof of this fact:
We begin with the maximum flow problem and its
relation to s-t-cuts. An s-t-cut is the set of edges
3.1 Proof of correctness leaving X (denoted by δ + (X)) for a set X ⊂ V
Let (V, T ∗ ) be an optimal tree, and choose T ∗ with s ∈ X and t ∈ / X.
so that |T ∗ ∩ T | is as large as possible. Suppose The total capacity of the edges in such an s-t-
T ∗ 6= T . cut, denoted by u(δ + (X)), is an upper bound on
All spanning trees have exactly |V | − 1 edges, the value of any s-t-flow f in (G, u). This is be-
implying that T ∗ \ T 6= ∅. Let j ∈ {1, . . . , m} be cause this value is precisely f (δ + (X))−f (δ − (X))
the smallest index with ej ∈ T ∗ \ T . for every set X containing s but not t, and
Since the greedy algorithm did not add ej to T , 0 ≤ f (e) ≤ u(e) for all e ∈ E.
there must be a cycle with edge set C ⊆ {ej } ∪ The famous max-flow min-cut theorem says
(T ∩ {e1 , . . . , ej−1 }) and ej ∈ C. that the upper bound is tight: the maximum
(V, T ∗ \ {ej }) is not connected, so there is a value of an s-t-flow equals the minimum capac-
set X ⊂ V with δ(X) ∩ T ∗ = {ej }. (Recall that ity of an s-t-cut.
δ(X) denotes the set of edges with exactly one In other words: if f is any s-t-flow with max-
endpoint in X.) imum value, then there is a set X ⊂ V with
Now |C ∩ δ(X)| is even, so at least two. Let s ∈ X, t ∈ / X, f (e) = u(e) for all e ∈ δ + (X),
ei ∈ (C ∩ δ(X)) \ {ej }. Note that i < j and thus and f (e) = 0 for all e ∈ δ − (X).
c(ei ) ≤ c(ej ). Indeed, if no such set exists, we can find a di-
Let T ∗∗ := (T ∗ \{ej })∪{ei }. Then (V, T ∗∗ ) is a rected path P from s to t in which each edge
tree with c(T ∗∗ ) = c(T ∗ ) − c(ej ) + c(ei ) ≤ c(T ∗ ). e = (v, w) is either an edge of G with f (e) < u(e),
So T ∗∗ is also optimal. But T ∗∗ has one edge or the reverse e0 := (w, v) is an edge of G with
more in common with T (the edge ei ) than T ∗ , f (e0 ) > 0. (This follows from letting X be the set
contradicting the choice of T ∗ . of vertices that are reachable from s along such
paths.)
Such paths are called augmenting paths because
3.2 Generalizations
along such a path we can augment the flow by
In general (and for any of the other problems increasing the flow on forward edges and decreas-
above), no simple “greedy” algorithm will always ing it on backward edges. Some flow algorithms
find an optimal solution. (but generally not the most efficient ones) start
The reason that it works for spanning trees is with the all-zero flow and successively find an
that here the feasible solutions form the bases of augmenting path.
a matroid. Matroids are a well-understood com- The figure below shows how to augment the
binatorial structure that can in fact be character- flow shown in Section 1.2 by one unit along the
ized by the optimality of the greedy algorithm. path a − c − b − t. The resulting flow with value 8,
4

shown on the right, is optimal, as is proved by the NP-complete to decide whether in a given di-
s-t-cut δ + ({s, a, c}) = {(a, b), (c, t)} of capacity 8. rected graph with vertices s and t there is a path
P from s to t and a path Q from t to s such that
a b a b P and Q are edge-disjoint.
4 3 4 3
s 1 1 2 s 1 0 3 4.3 LP duality
3 c t 4 c t The maximum flow problem (and also gener-
5 5
alizations like minimum-cost flows and multi-
The above relation also shows that for finding commodity flows) can be formulated as linear pro-
an s-t-cut with minimum capacity, it suffices to grams in a straightforward way.
solve the maximum flow problem. This can also Most other combinatorial optimization prob-
be used to compute a minimum cut in an undi- lems involve binary decisions and can be for-
rected graph or to compute the connectivity of a mulated naturally as (mixed-)integer linear pro-
given graph. grams. We give an example for the matching
Any s-t-flow can be decomposed into flows on s- problem.
t-paths, and possibly on cycles (but cyclic flow is The matching problem can be written as inte-
redundant as it does not contribute to the value). ger linear program
This decomposition can be done greedily, and
max 1>x : Ax ≤ 1, xe ∈ {0, 1} ∀e ∈ E

then the list of paths is sufficient to recover the
flow. This shows that one can restrict to a fi- where A is the vertex-edge-incidence matrix of
nite number of feasible solutions without loss of the given graph G = (V, E), 1 = (1, 1, . . . , 1)>
generality. denotes an appropriatePall-one vector (so 1>x is
just an abbreviation of e∈E xe ), and ≤ is meant
4.2 Disjoint paths component-wise. The feasible solutions to this
If all capacities are integral (i.e., are integers), one integer linear program are exactly the incidence
can find a maximum flow by always augmenting vectors of matchings in G.
by 1 along an augmenting path, until none ex- Solving integer linear programs is NP-hard in
ists anymore. This is not a polynomial-time algo- general (cf. Section 5.2), but linear programs
rithm (because the number of iterations can grow (without integrality constraints) can be solved in
exponentially in the instance size), but it shows polynomial time (−→ Continuous Optimization).
that in this case there is always an optimal flow This is one reason why it is often useful to con-
that is integral. An integral flow can be decom- sider the linear relaxation, which here is:
posed into integral flows on paths (and possibly
max 1>x : Ax ≤ 1, x ≥ 0 ,

cycles).
Hence, in the special case of unit capacities, an where 0 and 1 denote appropriate all-zero and
integral flow can be regarded as a set of pairwise all-one vectors, respectively. Now the entries of x
edge-disjoint s-t-paths. Therefore, the max-flow can be any real numbers between 0 and 1.
min-cut theorem implies the following theorem, The dual LP is:
due to Karl Menger:
Let (V, E) be a directed graph and s, t ∈ V . min y>1 : y>A ≥ 1, y ≥ 0 .

Then the maximum number of paths from s to
t that are pairwise edge-disjoint equals the mini- By weak duality, every dual feasible vector y
mum number of edges in an s-t-cut. yields an upper bound on the optimum. (Indeed,
Other versions of Menger’s theorem exist, for if x is the incidence vector of a matching M and
instance, for undirected graphs and for (inter- y ≥ 0 with y>A ≥ 1, then |M | = 1>x ≤ y>Ax ≤
nally) vertex-disjoint paths. y>1.)
In general, finding disjoint paths with pre- If G is bipartite, it turns out that these two
scribed endpoints is difficult: For example, it is LPs actually have integral optimal solutions. The
 5

minimal integral feasible solutions of the dual LP • run fast and produce good results in practice
are exactly the incidence vectors of vertex covers although there is no proof (heuristics).
(sets X ⊆ V such that every edge has at least one
endpoint in X). 5.1 Approximation algorithms
In other words, in any bipartite graph G, the
maximum size of a matching equals the minimum From a theoretical point of view, the notion of
size of a vertex cover. This is a theorem of Dénes approximation algorithms has proved to be most
Kőnig. It can also be deduced from the max-flow fruitful. For example, for the knapsack problem
min-cut theorem. (cf. Section 1.4) there is an algorithm that for any
For general graphs, this is not the case, as for given instance and any given number  > 0 com-
example the triangle (complete graph on three putes a solution at most 1 +  times worse than
vertices) shows. Nevertheless, the convex hull of the optimum, and whose running time is propor-
2
incidence vectors of matchings in general graphs tional to n . For the traveling salesman problem
can also be described well: it is (cf. Section 1.5 and −→ Traveling Salesman Prob-
n o lem), there is a 32 -approximation algorithm.
x : Ax ≤ 1, x ≥ 0, xe ≤ |A|
X  
2 ∀A ⊆ V , For set covering (cf, Section 1.6) there is no
e∈E[A] constant factor approximation algorithm unless
P=NP. But consider the special case where we
where E[A] denotes the set of edges whose end- ask for a minimum vertex cover in a given graph
points both belong to A. This was shown by Jack G: here U is the edge set of G and Si = δ(vi ) for
Edmonds (1965), who also found a polynomial- i = 1, . . . , n, where V = {v1 , . . . , vn } is the vertex
time algorithm. In contrast, the problem of find- set of G.
ing a minimum vertex cover in a given graph is Here we can use the above-mentioned fact that
NP-hard. the size of any matching in G is a lower bound.
Indeed, take any (inclusion-wise) maximal match-
5 Dealing with NP-Hard Problems ing M (e.g., found by the greedy algorithm), then
the 2|M | endpoints of the edges in M form a ver-
The other three problems mentioned above (see tex cover. As |M | is a lower bound on the opti-
Sections 1.4, 1.5, and 1.6) are NP-hard: they have mum, this is a simple 2-approximation algorithm.
a polynomial-time algorithm if and only if P=NP.
Since most researchers believe that P6=NP, they 5.2 Integer linear optimization
gave up looking for polynomial-time algorithms
for NP-hard problems. Weaker goals strive for Most classical combinatorial optimization prob-
algorithms that, for instance, lems can be formulated as integer linear programs
min c>x : Ax ≤ b, x ∈ Zn .

• solve interesting special cases in polynomial
time;
This includes all problems discussed in this chap-
• run in exponential time but faster than triv- ter, except the maximum flow problem, which is
ial enumeration; in fact a linear program (−→ Continuous Opti-
mization). Often the variables are restricted to 0
• always compute a feasible solution whose or 1. Sometimes, some variables are continuous,
value is at most k times worse than the opti- and others are discrete:
mum (so-called k-approximation algorithms;
min c>x : Ax + By ≤ d, x ∈ Rm , y ∈ Zn .

see Section 5.1);
• are efficient or compute good solutions for Such problems are called mixed-integer linear
most instances, in some probabilistic model; programs.
Discrete optimization comprises combinatorial
• are randomized (use random bits in their optimization but also general (mixed-)integer op-
computation) and have a good expected be- timization problems with no special combinato-
haviour; or rial structure.
6

For general (mixed-)integer linear optimiza- parts of a decision tree because lower and up-
tion, all known algorithms have exponential per bounds tell us that the unvisited parts can-
worst-case running time. The most successful al- not contain a better solution. How well this works
gorithms in practice use a combination of cutting mainly depends on how good the available bounds
planes and branch-and-bound (see Sections 6.2 are.
and 6.7). These are implemented in advanced
commercial software. Since many practical prob-
lems (including almost all classical combinato- 6.3 Reducing or decomposing the
rial optimization problems) can be described as instance
(mixed-)integer linear programs, such software
Often, an instance can be pre-processed by re-
is routinely used in practice to solve small and
moving irrelevant parts. In other cases, one can
medium-size instances of such problems. How-
compute a smaller instance or an instance with a
ever, combinatorial algorithms that exploit the
certain structure, whose solution implies a solu-
specific structure of the given problem are nor-
tion of the original instance.
mally superior, and often the only choice for very
large instances. Another well-known technique is divide-and-
conquer. In some problems, instances can be
decomposed/partitioned into smaller instances,
6 Techniques whose solutions can then be combined in some
way.
Since good algorithms have to exploit the struc-
ture of the problem, every problem requires differ-
ent techniques. Some techniques are quite general 6.4 Combinatorial or algebraic
and can be applied for a large variety of problems, structures
but in many cases they will not work well. Never-
theless we list the most important techniques that If the instances have a certain structure (like pla-
have been applied successfully to several combi- narity or certain connectivity or sparsity prop-
natorial optimization problems. erties of graphs, cross-free set families, matroid
structures, submodular functions, etc.), this must
usually be exploited.
6.1 Reductions Also, optimal solutions (of relaxations or the
Reducing an unknown problem to a known (and original problem) often have a useful structure.
solved) problem is the most important technique Sometimes (e.g., by sparsification or uncrossing
of course. To prove hardness, one proceeds the techniques) such a structure can be obtained even
other way round: we reduce a problem that we if it is not there originally.
know to be hard to a new problem (that then also Many algorithms compute and use a combi-
must be hard). If reductions work in both ways, natorial structure as a main tool. This is often
problems can actually regarded to be equivalent. a graph structure, but sometimes an algebraic
view can reveal certain properties. For instance,
the Laplacian matrix of a graph has many use-
6.2 Enumeration techniques ful properties. Sometimes simple properties, like
parity, can be extremely useful and elegant.
Some problems can be solved by skillful enumera-
tion. Dynamic programming is such a technique.
It works if optimal solutions arise from optimal 6.5 Primal-dual relations
solutions to “smaller” problems by simple opera-
tions. Dijkstra’s shortest path algorithm is a good We discussed LP duality, a key tool for many al-
example. Many algorithms on trees use dynamic gorithms, above. Lagrangian duality can also be
programming. useful for nonlinear problems. Sometimes other
Another well-known enumeration technique is kinds of duality, like planar duality or dual ma-
branch-and-bound. Here one enumerates only troids are very useful.
 7

6.6 Improvement techniques obtain more accurate, or even exact, solutions of

the original instance faster.
It is natural to start with some solution and it-
eratively improve it. The greedy algorithm and
finding augmenting paths can be considered as 6.9 Geometric techniques
special cases. In general, some way of measur- Geometric techniques are also playing an increas-
ing progress is needed so that the algorithm will ing role. Describing (the convex hull of) feasible
terminate. solutions by a polyhedron is a standard technique.
The general principle of starting with any feasi- Planar embeddings of graphs (if existent) can of-
ble solution and iteratively improving it by small ten be exploited in algorithms. Approximating a
local changes is called local search. Local search certain metric space by a simpler one is an im-
heuristics are often quite successful in practice, portant technique in the design of approximation
but in many cases no reasonable performance algorithms.
guarantees can be given.
6.10 Probabilistic techniques
6.7 Relaxation and rounding
Sometimes, a probabilistic view makes problems
Relaxations can arise combinatorially (by allow- much easier. For example, a fractional solution
ing solutions that do not have a certain prop- can be viewed as a convex combination of extreme
erty that was originally required for feasible solu- points, or as a probability distribution. Arguing
tions), or by omitting integrality constraints of a over the expectation of some random variables
description as an optimization problem over vari- can lead to simple algorithms and proofs. Many
ables in Rn . randomized algorithms can be derandomized, but
Linear programming formulations can imply this often complicates matters.
polynomial-time algorithms even if they have ex-
ponentially many variables or constraints (by the Further Reading
equivalence of optimization and separation). Lin-
1. Korte, B., Vygen, J. 2012. Combinatorial Op-
ear relaxations can be strengthened by adding timization: Theory and Algorithms. Berlin:
further linear constraints, called cutting planes. Springer; 5th edition
One can also consider non-linear relaxations. 2. Schrijver, A. 2003. Combinatorial Optimization:
In particular, semidefinite relaxations have been Polyhedra and Efficiency. Berlin: Springer
used for some approximation algorithms.
Of course, after solving a relaxation, the orig-
inally required property must be restored some-
how. If a fractional solution is made integral, this
is often called rounding. Sophisticated rounding
algorithms for various purposes have been devel-
oped.

6.8 Scaling and rounding

Often, a problem becomes easier if the numbers
in the instance are small integers. This can be
achieved by scaling and rounding, of course at
a loss of accuracy. The knapsack problem (cf.
Section 1.4) is a good example: the best algo-
rithms use scaling and rounding and then solve
the rounded instance by dynamic programming.
In some cases, a solution of the rounded in-
stance can be used in subsequent iterations to