CS675: Convex and Combinatorial Optimization

Spring 2018
Convex Sets

Instructor: Shaddin Dughmi

Outline

1 Convex sets, Affine sets, and Cones

2 Examples of Convex Sets

3 Convexity-Preserving Operations

4 Separation Theorems
 Convex Sets
A set S ⊆ Rn is convex if the line segment between any two points in S
lies in S. i.e. if x, y ∈ S and θ ∈ [0, 1], then θx + (1 − θ)y ∈ S.

Convex sets, Affine sets, and Cones 0/20

 Convex Sets
A set S ⊆ Rn is convex if the line segment between any two points in S
lies in S. i.e. if x, y ∈ S and θ ∈ [0, 1], then θx + (1 − θ)y ∈ S.

Equivalent Definition
S is convex if every convex combination of points in S lies in S.

Convex Combination
Finite: y is a convex combination of x1P , . . . , xk if
y = θ1 x1 + . . . θk xk , where θi ≥ 0 and i θi = 1.
General: expectation of probability measure on S.
Convex sets, Affine sets, and Cones 0/20
Convex Sets

Convex Hull
The convex hull of S ⊆ Rn is the smallest convex set containing S.
Intersection of all convex sets containing S
The set of all convex combinations of points in S

Convex sets, Affine sets, and Cones 1/20

Convex Sets

Convex Hull
The convex hull of S ⊆ Rn is the smallest convex set containing S.
Intersection of all convex sets containing S
The set of all convex combinations of points in S

A set S is convex if and only if convexhull(S) = S.

Convex sets, Affine sets, and Cones 1/20

 Affine Set
A set S ⊆ Rn is affine if the line passing through any two points in S
lies in S. i.e. if x, y ∈ S and θ ∈ R, then θx + (1 − θ)y ∈ S.

Obviously, affine sets are convex.

Convex sets, Affine sets, and Cones 2/20

 Affine Set
A set S ⊆ Rn is affine if the line passing through any two points in S
lies in S. i.e. if x, y ∈ S and θ ∈ R, then θx + (1 − θ)y ∈ S.

Obviously, affine sets are convex.

Equivalent Definition
S is affine if every affine combination of points in S lies in S.

Affine Combination
yPis an affine combination of x1 , . . . , xk if y = θ1 x1 + . . . θk xk , and
i θi = 1.

Generalizes convex combinations

Convex sets, Affine sets, and Cones 2/20
Affine Sets

Equivalent Definition II
S is affine if and only if it is a shifted subspace
i.e. S = x0 + V , where V is a linear subspace of Rn .

Any x0 ∈ S will do, and yields the same V .

The dimension of S is the dimension of subspace V .

Convex sets, Affine sets, and Cones 3/20

Affine Sets

Equivalent Definition II
S is affine if and only if it is a shifted subspace
i.e. S = x0 + V , where V is a linear subspace of Rn .

Any x0 ∈ S will do, and yields the same V .

The dimension of S is the dimension of subspace V .

Equivalent Definition III

S is affine if and only if it is the solution of a set of linear equations (i.e.
the intersection of hyperplanes).
i.e. S = {x : Ax = b} for some matrix A ∈ Rm×n and b ∈ Rm .

Convex sets, Affine sets, and Cones 3/20

Affine Sets

Affine Hull
The affine hull of S ⊆ Rn is the smallest affine set containing S.
Intersection of all affine sets containing S
The set of all affine combinations of points in S

Convex sets, Affine sets, and Cones 4/20

Affine Sets

Affine Hull
The affine hull of S ⊆ Rn is the smallest affine set containing S.
Intersection of all affine sets containing S
The set of all affine combinations of points in S

A set S is affine if and only if affinehull(S) = S.

Convex sets, Affine sets, and Cones 4/20

Affine Sets

Affine Hull
The affine hull of S ⊆ Rn is the smallest affine set containing S.
Intersection of all affine sets containing S
The set of all affine combinations of points in S

A set S is affine if and only if affinehull(S) = S.

Affine Dimension
The affine dimension of a set is the dimension of its affine hull

Convex sets, Affine sets, and Cones 4/20

 Cones
A set K ⊆ Rn is a cone if the ray from the origin through every point in
K is in K i.e. if x ∈ K and θ ≥ 0, then θx ∈ K.

Note: every cone contains 0.

Convex sets, Affine sets, and Cones 5/20

 Cones
A set K ⊆ Rn is a cone if the ray from the origin through every point in
K is in K i.e. if x ∈ K and θ ≥ 0, then θx ∈ K.

Note: every cone contains 0.

Special Cones
A convex cone is a cone that is convex
A cone is pointed if whenever x ∈ K and x 6= 0, then −x 6∈ K.
We will mostly mention proper cones: convex, pointed, closed,
and of full affine dimension.
Convex sets, Affine sets, and Cones 5/20
Cones

Equivalent Definition
K is a convex cone if every conic combination of points in K lies in K.

Conic Combination
y is a conic combination of x1 , . . . , xk if y = θ1 x1 + . . . θk xk , where
θi ≥ 0.

Convex sets, Affine sets, and Cones 6/20

Cones
Conic Hull
The conic hull of K ⊆ Rn is the smallest convex cone containing K
Intersection of all convex cones containing K
The set of all conic combinations of points in K

Convex sets, Affine sets, and Cones 7/20

Cones
Conic Hull
The conic hull of K ⊆ Rn is the smallest convex cone containing K
Intersection of all convex cones containing K
The set of all conic combinations of points in K

A set K is a convex cone if and only if conichull(K) = K.

Convex sets, Affine sets, and Cones 7/20
Cones

Polyhedral Cone
A cone is polyhedral if it is the set of solutions to a finite set of
homogeneous linear inequalities Ax ≤ 0.

Convex sets, Affine sets, and Cones 8/20

Outline

1 Convex sets, Affine sets, and Cones

2 Examples of Convex Sets

3 Convexity-Preserving Operations

4 Separation Theorems
 Linear Subspace: Affine, Cone
Hyperplane: Affine, cone if includes 0
Halfspace: Cone if origin on boundary
Line: Affine, cone if includes 0
Ray: Cone if endpoint at 0
Line segment

Examples of Convex Sets 9/20

 Polyhedron: finite intersection of halfspaces

Polytope: Bounded polyhedron

Examples of Convex Sets 10/20

 Nonnegative Orthant Rn+ : Polyhedral cone
Simplex: convex hull of affinely independent points
P
Unit simplex: x  0, i xi ≤P1
Probability simplex: x  0, i xi = 1.

Examples of Convex Sets 11/20

 Euclidean ball: {x : ||x − xc ||2 ≤ r} for center xc and radius r
Ellipsoid: x : (x − xc )T P −1 (x − xc ) ≤ 1 for symmetric P  0


Equivalently: {xc + Au : ||u||2 ≤ 1} for some linear map A

Examples of Convex Sets 12/20

 Norm ball: {x : ||x − c|| ≤ r} for any norm ||.||

Examples of Convex Sets 13/20

 Norm ball: {x : ||x − c|| ≤ r} for any norm ||.||

Norm cone: {(x, r) : ||x|| ≤ r}

Cone of symmetric positive semi-definite matrices M
Symmetric matrix A  0 iff xT Ax ≥ 0 for all x

Examples of Convex Sets 13/20

Outline

1 Convex sets, Affine sets, and Cones

2 Examples of Convex Sets

3 Convexity-Preserving Operations

4 Separation Theorems
 Intersection
The intersection of two convex sets is convex. This holds for the
intersection of an infinite number of sets.

Examples
Polyhedron: intersection of halfspaces
PSD cone: intersection of linear inequalities z T Az ≥ 0, for all
z ∈ Rn .

Convexity-Preserving Operations 14/20

 Intersection
The intersection of two convex sets is convex. This holds for the
intersection of an infinite number of sets.

Examples
Polyhedron: intersection of halfspaces
PSD cone: intersection of linear inequalities z T Az ≥ 0, for all
z ∈ Rn .

In fact, we will see that every closed convex set is the intersection of a
(possibly infinite) set of halfspaces.
Convexity-Preserving Operations 14/20
 Affine Maps
If f : Rn → Rm is an affine function (i.e. f (x) = Ax + b), then
f (S) is convex whenever S ⊆ Rn is convex
f −1 (T ) is convex whenever T ⊆ Rm is convex

f (θx + (1 − θ)y) = A(θx + (1 − θ)y) + b

= θ(Ax + b) + (1 − θ)(Ay + b))
= θf (x) + (1 − θ)f (y)

Convexity-Preserving Operations 15/20

 Examples
An ellipsoid is image of a unit ball after an affine map
A polyhedron Ax  b is inverse image of nonnegative orthant
under f (x) = b − Ax

Convexity-Preserving Operations 16/20

 Perspective Function
Let P : Rn+1 → Rn be P (x, t) = x/t.
P (S) is convex whenever S ⊆ Rn+1 is convex
P −1 (T ) is convex whenever T ⊆ Rn is convex

Convexity-Preserving Operations 17/20

 Perspective Function
Let P : Rn+1 → Rn be P (x, t) = x/t.
P (S) is convex whenever S ⊆ Rn+1 is convex
P −1 (T ) is convex whenever T ⊆ Rn is convex

Ax+b
Generalizes to linear fractional functions f (x) = cT x+d
Composition of perspective with affine.

Convexity-Preserving Operations 17/20

Outline

1 Convex sets, Affine sets, and Cones

2 Examples of Convex Sets

3 Convexity-Preserving Operations

4 Separation Theorems
 Separating Hyperplane Theorem
If A, B ⊆ Rn are disjoint convex sets, then there is a hyperplane
weakly separating them. That is, there is a ∈ Rn and b ∈ R such that
a| x ≤ b for every x ∈ A and a| y ≥ b for every y ∈ B.

Separation Theorems 18/20

 Separating Hyperplane Theorem (Strict Version)
If A, B ⊆ Rn are disjoint closed convex sets, and at least one of them
is compact, then there is a hyperplane strictly separating them. That is,
there is a ∈ Rn and b ∈ R such that a| x < b for every x ∈ A and
a| y > b for every y ∈ B.

Separation Theorems 18/20

 Farkas’ Lemma
Let K be a closed convex cone and let w 6∈ K. There is z ∈ Rn such
that z | x ≥ 0 for all x ∈ K, and z | w < 0.

Separation Theorems 19/20

Supporting Hyperplane

Supporting Hyperplane Theorem.

If S ⊆ Rn is a closed convex set and y is on the boundary of S, then
there is a hyperplane supporting S at y. That is, there is a ∈ Rn and
b ∈ R such that a| x ≤ b for every x ∈ S and a| y = b.

Separation Theorems 20/20