ISyE/Math/CS/Stat 525

Linear Optimization

2. The geometry of linear programming

Prof. Alberto Del Pia

University of Wisconsin-Madison

Based on the book Introduction to Linear Optimization by D. Bertsimas and J.N. Tsitsiklis
Outline

Sec. 2.1 We define a polyhedron as a set described by a finite number

of linear equality and inequality constraints.
Sec. 2.2 We study the basic geometric properties of polyhedra, with
emphasis on their “corner points”.
Sec. 2.3 We focus on the case where the feasible set is in the standard
form {x | Ax = b, x ≥ 0}.
Sec. 2.4 We study what happens when corner points arise in a
degenerate manner.
Sec. 2.5 We see under which conditions a polyhedron has corner points.
Sec. 2.6 We see that in this case, the search for optimal solutions to
LP problems can be restricted to corner points.
Sec. 2.7 We provide an alternative representation of polyhedra.
Sec. 2.8 We present the oldest method for solving LP problems.
General Optimization Problem
Consider the optimization problem
minimize f(x)
subject to x ∈ S,
where f : Rn → R and S ⊆ Rn .

Definition
The problem is infeasible if S = ∅.

Definition
The problem is unbounded if for each real number K there
exists x ∈ S such that f(x) < K.

Definition
The problem has an optimal solution if there exists x∗ ∈ S
such that f(x∗ ) ≤ f(x) for all x ∈ S.
General Optimization Problem
▶ In general, an optimization problem can be neither infeasible,
nor unbounded, nor have an optimal solution.
▶ Example:
minimize 1/x
subject to x ≥ 1.
LP Problem
Consider the LP problem
minimize c′ x
subject to Ax ≥ b,
where c ∈ Rn , b ∈ Rm and A ∈ Rm×n .

We will prove the following fundamental property:

Property 1:
Any LP problem is either infeasible, or it is unbounded, or it
has an optimal solution.
Local vs Global optimal solutions
Consider the optimization problem
minimize f(x)
subject to x ∈ S,
where f : Rn → R and S ⊆ Rn .

Definition
A vector x∗ ∈ S is called a (global) optimal solution if

f(x∗ ) ≤ f(x) for all x ∈ S.

Definition
A vector x∗ ∈ S is called a local optimal solution if ∃ϵ > 0
and a neighborhood N(x∗ , ϵ) = {x ∈ Rn : ∥x∗ − x∥ ≤ ϵ} of
x∗ such that f(x∗ ) ≤ f(x) for all x ∈ S ∩ N(x∗ , ϵ).
Local vs Global optimal solutions
Consider the optimization problem
minimize f(x)
subject to x ∈ S,
where f : Rn → R and S ⊆ Rn .

There can be local optimal solutions that are NOT global

optimal solutions.
Local vs Global optimal solutions
Consider the LP problem
minimize c′ x
subject to Ax ≥ b,

where c ∈ Rn , b ∈ Rm and A ∈ Rm×n .

We will prove the following fundamental property:

Property 2:
In a LP problem each local optimal solution is also a global
optimal solution.
2.1 Polyhedra and convex sets
Hyperplanes, halfspaces, and polyhedra
Hyperplanes and halfspaces

Definition 2.3
Let a be a nonzero vector in Rn and
let b be a scalar.
(a) The set {x ∈ Rn | a′ x = b} is
called a hyperplane.
(b) The set {x ∈ Rn | a′ x ≥ b} is
called a halfspace.

▶ A hyperplane is the boundary of a corresponding halfspace.

▶ A hyperplane is equal to the intersection of two halfspaces.
▶ The vector a in the definition of the hyperplane is
perpendicular to the hyperplane itself. Exercise: prove it!
Polyhedra

Definition 2.1
A polyhedron is a set that can be described in the form

{x ∈ Rn | Ax ≥ b},

where A is an m × n matrix and b is a vector in Rm .

▶ A polyhedron is the intersection of a finite number of

halfspaces.
Polyhedra

Definition 2.1
A polyhedron is a set that can be described in the form

{x ∈ Rn | Ax ≥ b},

where A is an m × n matrix and b is a vector in Rm .

▶ A polyhedron is the intersection of a finite number of

halfspaces.
Polyhedra

▶ As discussed in Section 1.1, the feasible set of any LP problem

can be described by inequality constraints of the form Ax ≥ b,
and is therefore a polyhedron.
▶ A set of the form

{x ∈ Rn | Ax = b, x ≥ 0}

is also a polyhedron, in a standard form representation.

Bounded Sets and Polytopes
A set can either “extend to infinity” or be confined in a finite
region.

Definition 2.2
A set S ⊂ Rn is bounded if there exists a constant K such
that the absolute value of every component of every element
of S is less than or equal to K.

x2

O K x1
Bounded Sets and Polytopes
A set can either “extend to infinity” or be confined in a finite
region.

Definition 2.2
A set S ⊂ Rn is bounded if there exists a constant K such
that the absolute value of every component of every element
of S is less than or equal to K.

Definition
A bounded polyhedron is called a polytope.
Why is LP special?
[spoilers ahead...]

Property 1:
Any LP problem is either infeasible, or it is unbounded, or it
has an optimal solution.

Property 2:
In a LP problem each local optimal solution is also a global
optimal solution.
Why is LP special?
[spoilers ahead...]

Property 1:
Any LP problem is either infeasible, or it is unbounded, or it
has an optimal solution.

Property 2:
In a LP problem each local optimal solution is also a global
optimal solution.

Property 3:
If the feasible region of a LP problem is a nonempty polytope,
then there exists an optimal solution that is a ‘corner point’.
2.2 Extreme points, vertices, and basic feasible
solutions
Extreme points, vertices, and basic feasible solutions

▶ We observed in Section 1.4 that an optimal solution to a LP

problem tends to occur at a “corner” of the polyhedron over
which we are optimizing.
▶ We now introduce three different ways of defining the concept
of a “corner”.
▶ We then show that all three definitions are equivalent.
Extreme points

Definition 2.6
Let P be a polyhedron. A vector x ∈ P is an extreme point of
P if we cannot find two vectors y, z ∈ P, both different from x,
and a scalar λ ∈ [0, 1], such that x = λy + (1 − λ)z.

▶ This definition is entirely

geometric: It depends on P,
but it does not depend on
the (algebraic)
representation of P.
Vertices

Definition 2.7
Let P be a polyhedron. A vector x ∈ P is a vertex of P if
there exists some c such that c′ x < c′ y for all y satisfying
y ∈ P and y ̸= x.

▶ A vertex of a polyhedron P
is the unique optimal
solution to some LP
problem with feasible set P.

▶ Also this definition is entirely geometric.

▶ We would like to have a definition which reduces to an
algebraic test.
Active constraints in x∗ ∈ Rn
Consider a polyhedron P ⊂ Rn defined in terms of the linear
equality and inequality constraints

a′i x ≥ bi , i ∈ M1 ,
a′i x ≤ bi , i ∈ M2 ,
a′i x = bi , i ∈ M3 ,

where M1 , M2 , and M3 are finite index sets, each ai is a vector in

Rn , and each bi is a scalar.

Definition 2.8
If a vector x∗ satisfies a′i x∗ = bi for i
in M1 , M2 , or M3 , we say that
constraint i is active at x∗ .

Example: P = {(x1 , x2 , x3 ) | x1 + x2 + x3 = 1, x1 , x2 , x3 ≥ 0}.

Basic (feasible) solutions

Definition 2.9
Consider a polyhedron P defined by linear equality and
inequality constraints, and let x∗ be an element of Rn .
(a) The vector x∗ is a basic solution if:
(i) All equality constraints are active;
(ii) Out of the constraints that are active at x∗ , there are n
of them that are linearly independent.
(b) If x∗ is a basic solution that satisfies all of the
constraints, we say that it is a basic feasible solution.

▶ We say that certain constraints a′i x ∼ bi are linearly

independent, meaning that the corresponding vectors ai are
linearly independent.
Basic (feasible) solutions

Definition 2.9
Consider a polyhedron P defined by linear equality and
inequality constraints, and let x∗ be an element of Rn .
(a) The vector x∗ is a basic solution if:
(i) All equality constraints are active;
(ii) Out of the constraints that are active at x∗ , there are n
of them that are linearly independent.
(b) If x∗ is a basic solution that satisfies all of the
constraints, we say that it is a basic feasible solution.

Example:
P = {(x1 , x2 , x3 ) : x1 + x2 + x3 = 1, x1 , x2 , x3 ≥ 0}.
Basic (feasible) solutions

Definition 2.9
Consider a polyhedron P defined by linear equality and
inequality constraints, and let x∗ be an element of Rn .
(a) The vector x∗ is a basic solution if:
(i) All equality constraints are active;
(ii) Out of the constraints that are active at x∗ , there are n
of them that are linearly independent.
(b) If x∗ is a basic solution that satisfies all of the
constraints, we say that it is a basic feasible solution.
Basic (feasible) solutions

▶ If the number m of constraints used to define a polyhedron

P ⊂ Rn is less than n, the number of active constraints at any
given point must also be less than n, and there are no basic or
basic feasible solutions.
Equivalence of definitions
▶ We have given three different definitions that are meant to
capture the same concept.
▶ Two of them are geometric (extreme point, vertex) and the
third is algebraic (basic feasible solution).
▶ All three definitions are equivalent and, for this reason, the
three terms can be used interchangeably.

Theorem 2.3
Let P be a nonempty polyhedron and let x∗ ∈ P. Then, the
following are equivalent:
(a) x∗ is a vertex;
(b) x∗ is an extreme point;
(c) x∗ is a basic feasible solution.
Equivalence of definitions
▶ To prove Theorem 2.3 we need the following result.

Theorem 2.2 (edited)

Let a′i x = bi , i ∈ I, be constraints in Rn . The following are
equivalent:
(a) There exist n constraints among a′i x = bi , i ∈ I, which are
linearly independent.
(b) The span of the vectors ai , i ∈ I, is all of Rn , that is, every
element of Rn can be expressed as a linear combination of
the vectors ai , i ∈ I.
(c) The system of equations a′i x = bi , i ∈ I, has a unique
solution.

Proof: Exercise. You can prove Theorem 2.2 using basic facts from
linear algebra (Theorem 1.3(a) in Section 1.5).
Now let’s prove Theorem 2.3!
Equivalence of definitions

▶ Theorem 2.3 states that a vector is a basic feasible solution if

and only if it is an extreme point.
▶ We have seen that the definition of an extreme point does not
depend on the representation of a polyhedron.
▶ We conclude that the property of being a basic feasible
solution is also independent of the representation of the
polyhedron.
Equivalence of definitions

▶ Theorem 2.3 states that a vector is a basic feasible solution if

and only if it is an extreme point.
▶ We have seen that the definition of an extreme point does not
depend on the representation of a polyhedron.
▶ We conclude that the property of being a basic feasible
solution is also independent of the representation of the
polyhedron.
▶ Question: Does the property of being a basic solution depend
on the representayion of the polyhedron?
Equivalence of definitions

Corollary 2.1
Given a finite number of linear inequality constraints, there
can only be a finite number of basic or basic feasible
solutions.

Proof.
▶ Consider a system of m linear inequality constraints in Rn .
▶ At any basic solution, there are n linearly independent
active constraints that define it.
▶ Therefore, the number of basic solutions is bounded above
by the number of ways that we can choose n constraints
out of m.
▶ This number is finite.
Equivalence of definitions

Corollary 2.1
Given a finite number of linear inequality constraints, there
can only be a finite number of basic or basic feasible
solutions.

▶ Although the number of basic feasible solutions is

guaranteed to be finite, it can be very large.
▶ For example, the unit cube

{x ∈ Rn | 0 ≤ xi ≤ 1, i = 1, . . . , n}

is defined by 2n constraints, but has 2n basic feasible

solutions.
Adjacent basic solutions
Adjacent basic solutions

▶ Two distinct basic solutions

to a set of linear constraints
in Rn are said to be adjacent
if we can find n − 1 linearly
independent constraints that
are active at both of them.
▶ If two adjacent basic
solutions are also feasible,
then the line segment that
joins them is called an edge
of the feasible set.
Adjacent basic solutions

▶ Two distinct basic solutions

to a set of linear constraints
in Rn are said to be adjacent
if we can find n − 1 linearly
independent constraints that
are active at both of them.
▶ If two adjacent basic
solutions are also feasible,
then the line segment that
joins them is called an edge
of the feasible set.
2.3 Polyhedra in standard form
Polyhedra in standard form

▶ The definition of a basic solution refers to general polyhedra.

▶ We will now specialize to polyhedra in standard form.

▶ Let P be a polyhedron in standard form

P = {x ∈ Rn | Ax = b, x ≥ 0}.

▶ Let the dimensions of A be m × n.

The full row rank assumption on A

▶ In most of our discussion of standard form problems, we will

make the assumption that the m rows of the matrix A are
linearly independent.
▶ We now see that when P is nonempty, linearly dependent
rows of A correspond to redundant constraints that can be
discarded.
▶ Therefore, our linear independence assumption can be made
without loss of generality.
The full row rank assumption on A

Theorem 2.5
Let P = {x | Ax = b, x ≥ 0} be a nonempty polyhedron,
where A is a matrix of dimensions m × n, with rows a′1 , . . . , a′m .
Suppose that rank(A) = k < m and that the rows a′i1 , . . . , a′ik
are linearly independent. Consider the polyhedron

Q = {x | a′i1 x = bi1 , . . . , a′ik x = bik , x ≥ 0}.

Then Q = P.

Proof: Easy exercise.

Example 2.3

▶ Consider the nonempty polyhedron defined by

2x1 + x2 + x3 = 2
x1 + x2 = 1
x1 + x3 = 1
x1 , x2 , x3 ≥ 0.
Example 2.3

▶ Consider the nonempty polyhedron defined by

2x1 + x2 + x3 = 2
x1 + x2 = 1
x1 + x3 = 1
x1 , x2 , x3 ≥ 0.

▶ The corresponding matrix A has rank two.

▶ This is because the last two rows (1, 1, 0) and (1, 0, 1) are
linearly independent, but the first row (2, 1, 1) is equal to the
sum of the other two.
Example 2.3

▶ Consider the nonempty polyhedron defined by

2x1 + x2 + x3 = 2
x1 + x2 = 1
x1 + x3 = 1
x1 , x2 , x3 ≥ 0.

▶ The corresponding matrix A has rank two.

▶ This is because the last two rows (1, 1, 0) and (1, 0, 1) are
linearly independent, but the first row (2, 1, 1) is equal to the
sum of the other two.
▶ Thus, the first constraint is redundant and after it is
eliminated, we still have the same polyhedron.
The full row rank assumption on A

▶ Notice that the polyhedron Q in Theorem 2.5 is in standard

form like P:
Q = {x | Dx = f, x ≥ 0}.
▶ Moreover, D is a k × n submatrix of A, with rank equal to k.
▶ We conclude that as long as the feasible set is nonempty, a
LP problem in standard form can be reduced to an equivalent
standard form LP problem (with the same feasible set) in
which the equality constraints are linearly independent.
Polyhedra in standard form: basic solutions

▶ Let’s get back to our task of specializing the definition of a

basic solution to polyhedra in standard form.
▶ Let P be a polyhedron in standard form

P = {x ∈ Rn | Ax = b, x ≥ 0}.

▶ Let the dimensions of A be m × n.

▶ We can now assume, without loss of generality, that the m
rows of the matrix A are linearly independent.
▶ Since the rows are n-dimensional, this requires that m ≤ n.
Polyhedra in standard form: basic solutions

▶ Recall that at any basic solution, there must be n linearly

independent constraints that are active.
▶ Furthermore, every basic solution must satisfy the equality
constraints Ax = b, which provides us with m active
constraints; these are linearly independent because of our
assumption on the rows of A.
▶ In order to obtain a total of n active constraints, we need to
choose n − m of the variables xi and set them to zero, which
makes the corresponding nonnegativity constraints xi ≥ 0
active.
▶ However, for the resulting set of n active constraints to be
linearly independent, the choice of these n − m variables is not
entirely arbitrary.
Polyhedra in standard form: basic solutions

Theorem 2.4
Consider the constraints Ax = b and x ≥ 0 and assume that
the m × n matrix A has linearly independent rows. A vector
x ∈ Rn is a basic solution if and only if we have Ax = b, and
there exist distinct indices B(1), . . . , B(m) ∈ {1, . . . , n} such
that:
(a) The columns AB(1) , . . . , AB(m) of A are linearly
independent;
(b) If i ̸= B(1), . . . , B(m), then xi = 0.

Proof idea:
n m n−m
( ) ( )
m A m B C
Active constraints: =
n−m 0 I n−m 0 I
Polyhedra in standard form: basic solutions

In view of Theorem 2.4, all basic solutions to a standard form

polyhedron can be constructed according to the following
procedure.

Procedure for constructing basic solutions

1. Choose m linearly independent columns AB(1) , . . . , AB(m) .
2. Let xi = 0 for all i ̸= B(1), . . . , B(m).
3. Solve the system of m equations Ax = b for the unknowns
xB(1) , . . . , xB(m) .
Polyhedra in standard form: basic solutions

We can use a similar procedure to construct all basic feasible

solutions to a standard form polyhedron.
▶ If a basic solution constructed according to this procedure is
nonnegative, then it is feasible, and it is a basic feasible
solution.
▶ Conversely, since every basic feasible solution is a basic
solution, it can be obtained from this procedure.
Example 2.1

Let the constraint Ax = b be of the form

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0 12
 x =  .
1 0 0 0 0 1 0 4
0 1 0 0 0 0 1 6
Example 2.1

Let the constraint Ax = b be of the form

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0 12
 x =  .
1 0 0 0 0 1 0 4
0 1 0 0 0 0 1 6

▶ B(1) = 4, B(2) = 5, B(3) = 6, B(4) = 7.

▶ The columns A4 , A5 , A6 , A7 are linearly independent.
▶ The corresponding basic solution is

x = (0, 0, 0, 8, 12, 4, 6).

▶ x≥0 ⇒ x is a basic feasible solution.

Example 2.1

Let the constraint Ax = b be of the form

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0 12
 x =  .
1 0 0 0 0 1 0 4
0 1 0 0 0 0 1 6

▶ B(1) = 3, B(2) = 5, B(3) = 6, B(4) = 7.

▶ The columns A3 , A5 , A6 , A7 are linearly independent.
▶ The corresponding basic solution is

x = (0, 0, 4, 0, −12, 4, 6).

▶ x5 = −12 < 0 ⇒ x is not feasible.

Polyhedra in standard form: basic solutions

▶ If x is a basic solution, the variables

xB(1) , . . . , xB(m)

are called basic variables; the remaining variables are called

nonbasic.
▶ Similarly, B(1), . . . , B(m) are called basic indices, and
{1, . . . , n} \ {B(1), . . . , B(m)} are called nonbasic indices.
▶ The columns
AB(1) , . . . , AB(m)
are called the basic columns and, since they are linearly
independent, they form a basis of Rm .
Polyhedra in standard form: basic solutions
▶ By arranging the m basic columns next to each other, we
obtain an m × m matrix B, called a basis matrix:
[ ]
B = AB(1) AB(2) · · · AB(m) .

▶ Note that a basis matrix B is invertible because the basic

columns are linearly independent.
▶ We can similarly define a vector xB with the values of the
basic variables:  
xB(1)
 
xB =  ...  .
xB(m)
▶ The basic variables are determined by solving the equation
BxB = b whose unique solution is given by

xB = B−1 b.
Example 2.1
   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 
0 12
 x= .
1 0 0 0 0 1 0  4
0 1 0 0 0 0 1 6

▶ We chose columns A4 , A5 , A6 , A7 and obtained the basic feasible

solution x = (0, 0, 0, 8, 12, 4, 6).
▶ Basic indices: 4, 5, 6, 7.
▶ Basic variables: x4 , x5 , x6 , x7 .
▶ Basic columns: A4 , A5 , A6 , A7 .
Example 2.1
   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 
0 12
 x= .
1 0 0 0 0 1 0  4
0 1 0 0 0 0 1 6

▶ Basis matrix B and vector xB :

   
1 0 0 0 x4
0 1 0 0 x5 
B= 0 0 1 0 ,
 xB =  
x6  .
0 0 0 1 x7

▶ Note that Bi = AB(i) for every i = 1, . . . , m.

Different bases

▶ We say that two bases are distinct or different if they involve

different sets {B(1), . . . , B(m)} of basic indices.
▶ If two bases involve the same set of indices in a different
order, they will be viewed as one and the same basis.
Example 2.1

Suppose there is an eighth column A8 = A7 .

   
1 1 2 1 0 0 0 0 8
0 1 6 0 1 0 0 0   
 x = 12 .
1 0 0 0 0 1 0 0 4
0 1 0 0 0 0 1 1 6

▶ The two sets of columns {A3 , A5 , A6 , A7 } and

{A3 , A5 , A6 , A8 } coincide.
▶ The corresponding sets of basic indices {3, 5, 6, 7} and
{3, 5, 6, 8} are different.
▶ We have two different bases.
Example 2.1

Suppose there is an eighth column A8 = A7 .

   
1 1 2 1 0 0 0 0 8
0 1 6 0 1 0 0 0   
 x = 12 .
1 0 0 0 0 1 0 0 4
0 1 0 0 0 0 1 1 6

▶ The two sets of columns {A3 , A5 , A6 , A7 } and

{A3 , A5 , A6 , A8 } coincide.
▶ The corresponding sets of basic indices {3, 5, 6, 7} and
{3, 5, 6, 8} are different.
▶ We have two different bases.
Polyhedra in standard form: basic solutions
Intuitive view of basic solutions.
▶ Recall our interpretation of the
constraint
∑
n
Ax = b ⇔ Ai xi = b
i=1

as a requirement to synthesize the

vector b ∈ Rn using the resource
vectors Ai (Section 1.1).
▶ In a basic solution, we use only m Question: In this example
of the resource vectors, those m = 2. Which are the
associated with the basic variables. bases that yield basic
▶ In a basic feasible solution, this is feasible solutions?
accomplished using a nonnegative
amount of each basic vector.
Correspondence of bases and basic solutions
Correspondence of bases and basic solutions

▶ A basis uniquely determines a basic solution, thus different

basic solutions must correspond to different bases.
▶ However, two different bases may lead to the same basic
solution.
▶ Example: If we have b = 0, then every basis matrix leads to
the same basic solution, namely, the zero vector.
▶ This phenomenon will have some important algorithmic
implications.
Adjacent basic solutions and adjacent bases
Adjacent basic solutions and adjacent bases

▶ Recall that two distinct basic solutions are said to be adjacent

if there are n − 1 linearly independent constraints that are
active at both of them.
▶ For standard form problems, we define two bases to be
adjacent if they share all but one basic columns.

Exercise: Show that:

1. Adjacent basic solutions can always be obtained from two
adjacent bases.
2. Conversely, if two adjacent bases lead to distinct basic
solutions, then the latter are adjacent.
(Hint: Use proof of Theorem 2.4.)
Example 2.2

In Example 2.1 we considered constraint Ax = b of the form

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0   
 x = 12 .
1 0 0 0 0 1 0 4
0 1 0 0 0 0 1 6
Example 2.2

In Example 2.1 we considered constraint Ax = b of the form

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0   
 x = 12 .
1 0 0 0 0 1 0 4
0 1 0 0 0 0 1 6

▶ The bases {A4 , A5 , A6 , A7 } and {A3 , A5 , A6 , A7 } are

adjacent because all but one columns are the same.
▶ The corresponding basic solutions x = (0, 0, 0, 8, 12, 4, 6)
and x = (0, 0, 4, 0, −12, 4, 6) are adjacent: we have n = 7
and a total of six common linearly independent active
constraints: the four equality constraints, x1 ≥ 0, and
x2 ≥ 0.
Example 2.2

In Example 2.1 we considered constraint Ax = b of the form

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0   
 x = 12 .
1 0 0 0 0 1 0 4
0 1 0 0 0 0 1 6

▶ The bases {A4 , A5 , A6 , A7 } and {A3 , A5 , A6 , A7 } are

adjacent because all but one columns are the same.
▶ The corresponding basic solutions x = (0, 0, 0, 8, 12, 4, 6)
and x = (0, 0, 4, 0, −12, 4, 6) are adjacent: we have n = 7
and a total of six common linearly independent active
constraints: the four equality constraints, x1 ≥ 0, and
x2 ≥ 0.
2.4 Degeneracy
Degeneracy

▶ We now consider again general polyhedra.

▶ According to our definition, at a basic solution, we must have
n linearly independent active constraints.
▶ This allows for the possibility that the number of active
constraints is greater than n.
▶ In this case, we say that we have a degenerate basic solution.

Definition 2.10
A basic solution x ∈ Rn is said to be degenerate if more than n
of the constraints are active at x.
Degeneracy

▶ If n = 2, a degenerate basic
solution is at the intersection of
three or more lines.
Degeneracy

▶ If n = 3, a degenerate basic
solution is at the intersection of
four or more planes.

▶ The presence of degeneracy can strongly affect the behavior

of LP algorithms.
Example 2.4
Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤ 8 (1)
x2 + 6x3 ≤ 12 (2)
x1 ≤ 4 (3)
x2 ≤ 6 (4)
x1 , x2 , x3 ≥ 0.
Example 2.4
Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤ 8 (1)
x2 + 6x3 ≤ 12 (2)
x1 ≤ 4 (3)
x2 ≤ 6 (4)
x1 , x2 , x3 ≥ 0.

▶ Consider x = (2, 6, 0).

Example 2.4
Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤ 8 (1)
x2 + 6x3 ≤ 12 (2)
x1 ≤ 4 (3)
x2 ≤ 6 (4)
x1 , x2 , x3 ≥ 0.

▶ Consider x = (2, 6, 0).

There are exactly three active and linearly independent
constraints: (1), (4), and x3 ≥ 0.
Example 2.4
Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤ 8 (1)
x2 + 6x3 ≤ 12 (2)
x1 ≤ 4 (3)
x2 ≤ 6 (4)
x1 , x2 , x3 ≥ 0.

▶ Consider x = (2, 6, 0).

There are exactly three active and linearly independent
constraints: (1), (4), and x3 ≥ 0.
Thus it is a nondegenerate basic feasible solution.
Example 2.4
Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤ 8 (1)
x2 + 6x3 ≤ 12 (2)
x1 ≤ 4 (3)
x2 ≤ 6 (4)
x1 , x2 , x3 ≥ 0.

▶ Consider x = (2, 6, 0).

There are exactly three active and linearly independent
constraints: (1), (4), and x3 ≥ 0.
Thus it is a nondegenerate basic feasible solution.
▶ Consider x = (4, 0, 2).
Example 2.4
Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤ 8 (1)
x2 + 6x3 ≤ 12 (2)
x1 ≤ 4 (3)
x2 ≤ 6 (4)
x1 , x2 , x3 ≥ 0.

▶ Consider x = (2, 6, 0).

There are exactly three active and linearly independent
constraints: (1), (4), and x3 ≥ 0.
Thus it is a nondegenerate basic feasible solution.
▶ Consider x = (4, 0, 2).
There are four active constraints: (1), (2), (3), and x2 ≥ 0,
and three of them are linearly independent.
Example 2.4
Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤ 8 (1)
x2 + 6x3 ≤ 12 (2)
x1 ≤ 4 (3)
x2 ≤ 6 (4)
x1 , x2 , x3 ≥ 0.

▶ Consider x = (2, 6, 0).

There are exactly three active and linearly independent
constraints: (1), (4), and x3 ≥ 0.
Thus it is a nondegenerate basic feasible solution.
▶ Consider x = (4, 0, 2).
There are four active constraints: (1), (2), (3), and x2 ≥ 0,
and three of them are linearly independent.
Thus it is a degenerate basic feasible solution.
Degeneracy in standard form polyhedra
Degeneracy in standard form polyhedra
▶ At a basic solution of a polyhedron in standard form, the m
equality constraints are always active.
▶ Therefore, having more than n active constraints is the same
as having more than n − m variables at zero level.
▶ This leads us to the next definition which is a special case of
Definition 2.10.

Definition 2.11
Consider the standard form polyhedron

P = {x ∈ Rn | Ax = b, x ≥ 0}

and let x be a basic solution. Let m be the number of rows of

A. The vector x is a degenerate basic solution if more than
n − m of the components of x are zero.
Example 2.5

Consider the polyhedron P = {x | Ax = b, x ≥ 0}, where

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0 12
A= 
1 0 0 0 0 1 0 , b =  4  .
 

0 1 0 0 0 0 1 6
Example 2.5

Consider the polyhedron P = {x | Ax = b, x ≥ 0}, where

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0 12
A= 
1 0 0 0 0 1 0 , b =  4  .
 

0 1 0 0 0 0 1 6

▶ Consider the basis consisting of the linearly independent

columns A1 , A2 , A3 , A7 .
▶ The corresponding basic (feasible) solution is
x = (4, 0, 2, 0, 0, 0, 6).
▶ It is degenerate, because we have four variables that are
zero, whereas n − m = 7 − 4 = 3.
Example 2.5

Consider the polyhedron P = {x | Ax = b, x ≥ 0}, where

   
1 1 2 1 0 0 0 8
0 1 6 0 1 0 0 12
A= 
1 0 0 0 0 1 0 , b =  4  .
 

0 1 0 0 0 0 1 6

▶ Consider now the adjacent basis consisting of the linearly

independent columns A1 , A3 , A4 , A7 .
▶ The corresponding basic (feasible) solution is again
x = (4, 0, 2, 0, 0, 0, 6).
Degeneracy in standard form polyhedra

▶ This example suggests that we can think of degeneracy in the

following terms.
▶ We pick a basic solution by picking n linearly independent
constraints to be satisfied with equality, and we realize that
certain other constraints are also satisfied with equality.
Degeneracy in standard form polyhedra

▶ If the entries of A or b were chosen at random, this would

almost never happen.
▶ Also, if the coefficients of the active constraints are slightly
perturbed, degeneracy can disappear.

▶ In practical problems, however, the entries of A and b often

have a special (nonrandom) structure, and degeneracy is
common.
Degeneracy in standard form polyhedra

To visualize degeneracy in standard form polyhedra, we assume

that n − m = 2.

Example:

n=3
m=1

▶ At a nondegenerate basic solution, exactly n − m of the

constraints xi ≥ 0 are active and the corresponding
variables are nonbasic.
Degeneracy in standard form polyhedra

To visualize degeneracy in standard form polyhedra, we assume

that n − m = 2.

Example:

n=6
m=4

▶ At a nondegenerate basic solution, exactly n − m of the

constraints xi ≥ 0 are active and the corresponding
variables are nonbasic.
Degeneracy in standard form polyhedra

To visualize degeneracy in standard form polyhedra, we assume

that n − m = 2.

Example:

n=6
m=4

▶ At a degenerate basic solution, more than n − m of the

constraints xi ≥ 0 are active, and there are usually several
ways of choosing which n − m variables to call nonbasic.
▶ In that case, there are several bases corresponding to that
same basic solution.
Degeneracy is not a purely geometric property
Degeneracy is not a purely geometric property

▶ Degeneracy of basic feasible solutions may depend on the

particular representation of a polyhedron.
▶ Let’s see an example.
Degeneracy is not a purely geometric property

Example: Consider the standard form polyhedron

P = {(x1 , x2 , x3 ) | x1 − x2 = 0, x1 + x2 + 2x3 = 2, x1 , x2 , x3 ≥ 0}.

▶ n = 3, m = 2, n − m = 1.
▶ (1, 1, 0) is a nondegenerate basic feasible solution because
only one variable is zero.
▶ (0, 0, 1) is a degenerate basic feasible solution because two
variables are zero.
Degeneracy is not a purely geometric property

The same polyhedron can also be described in the form

P = {(x1 , x2 , x3 ) | x1 − x2 = 0, x1 + x2 + 2x3 = 2, x1 , x3 ≥ 0}.

▶ Note that this polyhedron is not in standard form.

▶ The vector (0, 0, 1) is now a nondegenerate basic feasible
solution, because there are only three active constraints.
Degeneracy is not a purely geometric property

The same polyhedron can also be described in the form

P = {(x1 , x2 , x3 ) | x1 − x2 = 0, x1 + x2 + 2x3 = 2, x1 , x3 ≥ 0}.

▶ Note that this polyhedron is not in standard form.

▶ The vector (0, 0, 1) is now a nondegenerate basic feasible
solution, because there are only three active constraints.
Question: Can you construct a polytope with a basic feasible
solution that is degenerate in every possible representation?
Degeneracy is not a purely geometric property
▶ Example: consider a standard form polyhedron

P = {x | Ax = b, x ≥ 0}.
▶ Let x∗ be a nondegenerate basic feasible solution.
▶ Exactly n − m of the variables x∗i are equal to zero.
Degeneracy is not a purely geometric property
▶ Example: consider a standard form polyhedron

P = {x | Ax = b, x ≥ 0}.
▶ Let x∗ be a nondegenerate basic feasible solution.
▶ Exactly n − m of the variables x∗i are equal to zero.

▶ We now represent P in the form

P = {x | Ax ≥ b, −Ax ≥ −b, x ≥ 0}.

▶ Active constraints at x∗ :
▶ n − m variables set to zero,
▶ all m inequality constraints from Ax ≥ b,
▶ all m inequality constraints from −Ax ≥ −b.
▶ Total active constraints at x∗ : n − m + 2m = n + m.
⇒ x∗ is degenerate.
▶ Under the second representation every basic feasible solution
is degenerate.
2.5 Existence of extreme points
Existence of extreme points

▶ We will see necessary and sufficient conditions for a

polyhedron to have at least one extreme point.
▶ Observe that not every polyhedron has this property.
▶ Example: If n ≥ 2, a halfspace in Rn is a polyhedron without
extreme points.

▶ More generally, as argued in Section 2.2, if the matrix A has

fewer than n rows, then the polyhedron {x ∈ Rn | Ax ≥ b}
does not have a basic feasible solution.
Existence of extreme points
▶ The existence of an extreme point depends on whether a
polyhedron contains an infinite line or not.

Definition 2.12
A polyhedron P ⊂ Rn contains a line if there exists a vector
x ∈ P and a nonzero vector d ∈ Rn such that x + λd ∈ P for
all λ ∈ R.

Exercise: Show that if P contains a line {x̄ + λd | λ ∈ R}, then P

contains all lines {x + λd | λ ∈ R} for every x ∈ P.
Existence of extreme points
Theorem 2.6
Suppose that the polyhedron

P = {x ∈ Rn | a′i x ≥ bi , i = 1, . . . , m}

is nonempty. Then, the following are equivalent:

(a) The polyhedron P has at least one extreme point.
(b) The polyhedron P does not contain a line.
(c) There exist n vectors out of the family a1 , . . . , am ,
which are linearly independent.

Proof idea:
▶ (a) ⇒ (c): Immediate, from definition of basic solution.
▶ (c) ⇒ (b): By contradiction. Assume P contains a line
{x + λd | λ ∈ R}. Then d must be orthogonal to each vector ai .
Existence of extreme points
Theorem 2.6
Suppose that the polyhedron

P = {x ∈ Rn | a′i x ≥ bi , i = 1, . . . , m}

is nonempty. Then, the following are equivalent:

(a) The polyhedron P has at least one extreme point.
(b) The polyhedron P does not contain a line.
(c) There exist n vectors out of the family a1 , . . . , am ,
which are linearly independent.

Proof idea:
▶ (b) ⇒ (a): Constructive:
1. Start from x ∈ P and move along a direction d
where all active constraints remain active.
2. Stop when a new constraint is about to be violated.
3. Now there is one more linearly independent
active constraint. Repeat.
Exercise: Write a complete proof.
Existence of extreme points

Examples of polyhedra that do not contain a line:

▶ Polytopes. Why?
▶ The positive orthant {x | x ≥ 0}. Why?
▶ Polyhedra in standard form. Why?

Theorem 2.6 then implies:

Corollary 2.2
Every nonempty bounded polyhedron and every nonempty
polyhedron in standard form has at least one basic feasible
solution.
2.6 Optimality of extreme points
Optimality of extreme points
▶ We have established the conditions for the existence of
extreme points.
▶ We now confirm the intuition developed in Chapter 1:

Theorem 2.7
Consider the LP problem of minimizing c′ x over a
polyhedron P. Suppose that P has at least one extreme
point and that there exists an optimal solution. Then, there
exists an optimal solution which is an extreme point of P.

▶ We have seen that polyhedra in standard form, as well as

bounded polyhedra, have extreme points.
▶ In these cases, Theorem 2.7 implies that if there exists an
optimal solution, then there exists an extreme point that is
optimal.
Optimality of extreme points
▶ We have established the conditions for the existence of
extreme points.
▶ We now confirm the intuition developed in Chapter 1:

Theorem 2.7
Consider the LP problem of minimizing c′ x over a
polyhedron P. Suppose that P has at least one extreme
point and that there exists an optimal solution. Then, there
exists an optimal solution which is an extreme point of P.

Let’s prove Theorem 2.7!

Optimality of extreme points
▶ The next result is stronger than Theorem 2.7.
▶ It shows that the existence of an optimal solution can be
taken for granted, as long as the optimal cost is not −∞.

Theorem 2.8
Consider the LP problem of minimizing c′ x over a
polyhedron P. Suppose that P has at least one extreme
point. Then, either the optimal cost is equal to −∞, or
there exists an extreme point which is optimal.

Proof idea:
▶ Similar to the proof of Theorem 2.6 (b) ⇒ (a).
The difference is that we also make sure that
the cost does not increase.
▶ This shows that ∀x ∈ P, either the optimal cost
is −∞, or ∃ extreme point with cost ≤.
▶ Exercise: Write a complete proof.
Optimality of extreme points
▶ The next result is stronger than Theorem 2.7.
▶ It shows that the existence of an optimal solution can be
taken for granted, as long as the optimal cost is not −∞.

Theorem 2.8
Consider the LP problem of minimizing c′ x over a
polyhedron P. Suppose that P has at least one extreme
point. Then, either the optimal cost is equal to −∞, or
there exists an extreme point which is optimal.

▶ For a general LP problem, if the feasible set has no extreme

points, then Theorem 2.8 does not apply.
▶ Example?
Optimality of extreme points

▶ On the other hand, any LP problem can be transformed into

an equivalent problem in standard form to which Theorem 2.8
does apply.
▶ This establishes the following corollary.

Corollary 2.3
Consider the LP problem of minimizing c′ x over a nonempty
polyhedron. Then, either the optimal cost is equal to −∞ or
there exists an optimal solution.
Optimality of extreme points
▶ The result in Corollary 2.3 should be contrasted with what
may happen with a nonlinear cost function.
▶ For example, in the problem
minimize 1/x
subject to x ≥ 1,
the optimal cost is not −∞, but an optimal solution does not
exist.
Why is LP special?
[2/3 spoilers understood...]

Property 1 [Corollary 2.3]:

Any LP problem is either infeasible, or it is unbounded, or it
has an optimal solution.

Property 2:
In a LP problem each local optimal solution is also a global
optimal solution.

Property 3 [Theorem 2.7]:

If the feasible region of a LP problem is a nonempty polytope,
then there exists an optimal solution that is an extreme point.
2.7 Representation of polytopes
Representation of polytopes

Definition 2.5
Let x1 , . . . , xk ∈ R n
∑kand λ1 , . . . , λk ∈ R be such that λi ≥ 0 for
i = 1, . . . , k and i=1 λi = 1.
∑
(a) The vector ki=1 λi xi is said to be a convex combination
of x1 , . . . , xk .
(b) The convex hull of x1 , . . . , xk is the set of all possible
convex combinations of x1 , . . . , xk .

▶ A vector x ∈ P is an extreme point of P if x is not a convex

combination of two vectors in P.
Representation of polytopes

▶ So far, we have been representing polyhedra in terms of their

defining inequalities.
▶ The next theorem provides an alternative:

Theorem 2.9
A nonempty polytope is the convex hull of its extreme points.

We will not prove this.

Representation of polytopes

▶ So far, we have been representing polyhedra in terms of their

defining inequalities.
▶ The next theorem provides an alternative:

Theorem 2.9
A nonempty polytope is the convex hull of its extreme points.

We will not prove this.

▶ There is a converse to Theorem 2.9 asserting that the convex

hull of a finite number of points is a polyhedron.
▶ This result is proved in the next section.
Example 2.6

▶ Consider the polytope

P = {(x1 , x2 , x3 ) | x1 + x2 + x3 ≤ 1, x1 , x2 , x3 ≥ 0}.
▶ It has four extreme points:
xA = (0, 0, 1), xB = (1, 0, 0), xC = (0, 1, 0), xD = (0, 0, 0).
Example 2.6

▶ Consider the polytope

P = {(x1 , x2 , x3 ) | x1 + x2 + x3 ≤ 1, x1 , x2 , x3 ≥ 0}.
▶ It has four extreme points:
xA = (0, 0, 1), xB = (1, 0, 0), xC = (0, 1, 0), xD = (0, 0, 0).

▶ The vector x = (1/3, 1/3, 1/4) belongs to P.

▶ It can be represented as

1 1 1 1
x = xA + xB + xC + xD .
4 3 3 12
Example 2.6

▶ Consider the polytope

P = {(x1 , x2 , x3 ) | x1 + x2 + x3 ≤ 1, x1 , x2 , x3 ≥ 0}.
▶ It has four extreme points:
xA = (0, 0, 1), xB = (1, 0, 0), xC = (0, 1, 0), xD = (0, 0, 0).

▶ Viceversa, consider the vector

1 A 1 B 1 C 1 D (1 1 1)
x + x + x + x = , , .
3 6 6 3 6 6 3
▶ It can be checked that it belongs to P.
2.8 Projections of polyhedra: Fourier-Motzkin
elimination
Projections of polyhedra: Fourier-Motzkin elimination

▶ We present the oldest method for solving LP problems.

▶ This method is not practical because it requires a very large
number of steps.
▶ However, it has some interesting theoretical consequences.
Projections of polyhedra: Fourier-Motzkin elimination

▶ The key to this method is the

concept of a projection:
▶ Let k, n be nonnegative integers
with k ≤ n.
▶ The projection πk (x) of a vector
x ∈ Rn is

πk (x) = πk (x1 , . . . , xn ) = (x1 , . . . , xk ).

Projections of polyhedra: Fourier-Motzkin elimination

▶ The projection Πk (S) of a set

S ⊆ Rn is

Πk (S) = {πk (x) | x ∈ S}.

▶ An equivalent definition is

Πk (S) = {(x1 , . . . , xk ) | ∃xk+1 , . . . , xn s.t. (x1 , . . . , xn ) ∈ S}.

▶ Note that S is nonempty if and only if Πk (S) is nonempty.

Projections of polyhedra: Fourier-Motzkin elimination

▶ Consider the problem of deciding whether a given polyhedron

P ⊆ Rn is nonempty.
▶ Suppose we can construct the set

Πn−1 (P) ⊆ Rn−1 .

▶ We can then consider the equivalent problem of deciding

whether Πn−1 (P) is nonempty.
▶ If we keep eliminating variables one by one, we eventually
arrive at the set
Π1 (P) ⊆ R.
▶ Its emptiness is easy to check. How?
Projections of polyhedra: Fourier-Motzkin elimination

▶ The main disadvantage of this method is that while each step

reduces the dimension by one, a large number of constraints is
usually added.
▶ The number of constraints can increase exponentially with the
problem dimension (see Exercise 2.20).
Projections of polyhedra: Fourier-Motzkin elimination

We now describe the Fourier-Motzkin elimination method.

▶ We are given a polyhedron P:
{ }
∑ n
P = x ∈ Rn | aij xj ≥ bi , i = 1, . . . , m .
j=1

▶ We wish to eliminate xn and construct the projection Πn−1 (P).

Projections of polyhedra: Fourier-Motzkin elimination

Elimination algorithm
∑n
1. Rewrite each constraint j=1 aij xj ≥ bi in the form
∑
n−1
ain xn ≥ − aij xj + bi , i = 1, . . . , m.
j=1

▶ If ain ̸= 0, divide both sides by ain .

▶ By letting x̄ = (x1 , . . . , xn−1 ), we obtain an equivalent
representation of P:
xn ≥ di + f′i x̄ if ain > 0,
dj + f′j x̄ ≥ xn if ajn < 0,
0 ≥ dk + f′k x̄ if akn = 0.

Here, each di , dj , dk ∈ R, and each fi , fj , fk ∈ Rn−1 .

Projections of polyhedra: Fourier-Motzkin elimination

Elimination algorithm
▶ By letting x̄ = (x1 , . . . , xn−1 ), we obtain an equivalent
representation of P:
xn ≥ di + f′i x̄ if ain > 0,
dj + f′j x̄ ≥ xn if ajn < 0, (*)
0 ≥ dk + f′k x̄ if akn = 0.
Here, each di , dj , dk ∈ R, and each fi , fj , fk ∈ Rn−1 .
2. Let Q be the polyhedron in Rn−1 defined by the
constraints
dj + f′j x̄ ≥ di + f′i x̄ if ain > 0 and ajn < 0,
0 ≥ dk + f′k x̄ if akn = 0. (**)
Example 2.7
Consider the polyhedron defined by the constraints

x1 + x2 ≥ 1
x1 + x2 + 2x3 ≥ 2
2x1 + 3x3 ≥ 3
x1 − 4x3 ≥ 4
−2x1 + x2 − x3 ≥ 5.
Example 2.7
Consider the polyhedron defined by the constraints

x1 + x2 ≥ 1
x1 + x2 + 2x3 ≥ 2
2x1 + 3x3 ≥ 3
x1 − 4x3 ≥ 4
−2x1 + x2 − x3 ≥ 5.

1. We rewrite these constraints in the form

0 ≥ 1 − x1 − x2
x3 ≥ 1 − x1 /2 − x2 /2
x3 ≥ 1 − 2x1 /3
−1 + x1 /4 ≥ x3
−5 − 2x1 + x2 ≥ x3 .
Example 2.7

1. We rewrite these constraints in the form

0 ≥ 1 − x1 − x2
x3 ≥ 1 − x1 /2 − x2 /2
x3 ≥ 1 − 2x1 /3
−1 + x1 /4 ≥ x3
−5 − 2x1 + x2 ≥ x3 .
Example 2.7
2. Then, the set Q is defined by the constraints

0 ≥ 1 − x1 − x2
−1 + x1 /4 ≥ 1 − x1 /2 − x2 /2
−1 + x1 /4 ≥ 1 − 2x1 /3
−5 − 2x1 + x2 ≥ 1 − x1 /2 − x2 /2
−5 − 2x1 + x2 ≥ 1 − 2x1 /3.

1. We rewrite these constraints in the form

0 ≥ 1 − x1 − x2
x3 ≥ 1 − x1 /2 − x2 /2
x3 ≥ 1 − 2x1 /3
−1 + x1 /4 ≥ x3
−5 − 2x1 + x2 ≥ x3 .
Projections of polyhedra: Fourier-Motzkin elimination

Theorem 2.10
The polyhedron Q constructed by the elimination algorithm is
equal to the projection Πn−1 (P) of P.

Let’s prove it!

Projections of polyhedra: Fourier-Motzkin elimination

▶ Notice that for any vector x = (x1 , . . . , xn ), we have

πn−2 (x) = (x1 , . . . , xn−2 ) = πn−2 (πn−1 (x)).

▶ Accordingly, for any polyhedron P, we also have

Πn−2 (P) = Πn−2 (Πn−1 (P)).

▶ By generalizing this observation, we see that the set Π1 (P)

can be obtained by applying the elimination algorithm n − 1
times.
Projections of polyhedra: Fourier-Motzkin elimination

▶ Each application of the elimination algorithm can increase the

number of constraints substantially.
▶ The polyhedron Π1 (P) obtained may be described by a very
large number of constraints.
▶ Of course, since Π1 (P) is one-dimensional, almost all of these
constraints will be redundant.
▶ This is of no help: in order to decide which ones are
redundant, we must, in general, enumerate them.
Projections of polyhedra: Fourier-Motzkin elimination

The elimination algorithm has an important theoretical

consequence:
▶ The projection Πk (P) can be generated by repeated
application of the elimination algorithm.
▶ The elimination algorithm always produces a polyhedron.
⇒ A projection Πk (P) of a polyhedron is also a polyhedron.

Corollary 2.4
Let P ⊂ Rn+k be a polyhedron. Then, the set

{x ∈ Rn | there exists y ∈ Rk such that (x, y) ∈ P}

is also a polyhedron.
Projections of polyhedra: Fourier-Motzkin elimination

Corollary 2.6
The convex hull of a finite number of vectors is a polyhedron.

Proof:
▶ The convex hull of x1 , . . . , xk is the set

{x ∈ Rn | there exists λ ∈ Rk such that (x, λ) ∈ P},

{ ∑ k ∑
k }
where P = (x, λ) | x = λi xi , λi = 1, λ ≥ 0 .
i=1 i=1

▶ Corollary 2.4 implies that it is a polyhedron.

Projections of polyhedra: Fourier-Motzkin elimination
The elimination algorithm can be used to solve LP problems.
▶ Consider the LP problem

min{c′ x | Ax ≥ b}.

▶ Define a new variable x0 and consider the set

{(x0 , x) | Ax ≥ b, x0 = c′ x}.

▶ If we use the elimination algorithm n times to eliminate the

variables x1 , . . . , xn , we are left with the set

Q = {x0 | there exists x such that Ax ≥ b, x0 = c′ x},

and the optimal cost is equal to the smallest element of Q.

▶ An optimal solution x can be recovered by backtracking
(Exercise 2.21).