Optimization

Chapter 2: The Geometry of Linear

Programming

Dr. Nguyễn Ngọc Phan

Department of Mathematics - Mechanics - Informatics

VNU Hanoi University of Science

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
1 A polyhedron is a set that can be described in the form
{x ∈ Rn : Ax ≥ b}, where A is an m × n matrix and b is
a vector in Rn .

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
1 A polyhedron is a set that can be described in the form
{x ∈ Rn : Ax ≥ b}, where A is an m × n matrix and b is
a vector in Rn .
2 The set S ⊂ Rn is bounded if there exists a constant K
such that the absolute value of every component of every
element of S is less than or equal to K .

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
1 A polyhedron is a set that can be described in the form
{x ∈ Rn : Ax ≥ b}, where A is an m × n matrix and b is
a vector in Rn .
2 The set S ⊂ Rn is bounded if there exists a constant K
such that the absolute value of every component of every
element of S is less than or equal to K .

Definition
Let a be a nonzero vector in Rn and let b be a scalar.
1 The set {x ∈ Rn : aT x = b} is called a hyperplane.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
1 A polyhedron is a set that can be described in the form
{x ∈ Rn : Ax ≥ b}, where A is an m × n matrix and b is
a vector in Rn .
2 The set S ⊂ Rn is bounded if there exists a constant K
such that the absolute value of every component of every
element of S is less than or equal to K .

Definition
Let a be a nonzero vector in Rn and let b be a scalar.
1 The set {x ∈ Rn : aT x = b} is called a hyperplane.
2 The set {x ∈ Rn : aT x ≥ b} is called a half plane.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A set S ∈ Rn is convex if for any x, y ∈ S, and any λ ∈ [0, 1],
we have λx + (1 − λ)y ∈ S.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A set S ∈ Rn is convex if for any x, y ∈ S, and any λ ∈ [0, 1],
we have λx + (1 − λ)y ∈ S.

Therefore, a set is convex if the segment joining any two of its

elements is contained in the set.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Let x 1 , . . . , x k be vectors in Rn and let λ1 , . . . , λk ≥ 0 such
that
λ1 + · · · + λk = 1.

The vector ki=1 λi x i is said to be a convex combination

P
1

of the vectors x 1 , . . . , x k .

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Let x 1 , . . . , x k be vectors in Rn and let λ1 , . . . , λk ≥ 0 such
that
λ1 + · · · + λk = 1.

The vector ki=1 λi x i is said to be a convex combination

P
1

of the vectors x 1 , . . . , x k .
2 The convex hull of the vectors x 1 , . . . , x k is the set of all
convex combinations of these vectors.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
1 The intersection of convex sets is convex.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
1 The intersection of convex sets is convex.
2 Every polyhedron is a convex set.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
1 The intersection of convex sets is convex.
2 Every polyhedron is a convex set.
3 A convex combination of a finite number of elements of a
convex set also belongs to that set.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
1 The intersection of convex sets is convex.
2 Every polyhedron is a convex set.
3 A convex combination of a finite number of elements of a
convex set also belongs to that set.
4 The convex hull of a finite number of vectors is a convex
set.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Let P be a polyhedron. A vector x ∈ P is an extreme
point of P if we cannot find two vectors y , z ∈ P, both
different from x, and a scalar λ ∈ [0, 1], such that
x = λy + (1 − λ)z.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Let P be a polyhedron. A vector x ∈ P is an extreme
point of P if we cannot find two vectors y , z ∈ P, both
different from x, and a scalar λ ∈ [0, 1], such that
x = λy + (1 − λ)z.
Let P be a polyhedron. A vector x ∈ P is a vertex of P if
there exists some c such that c T x < c T y for all y
satisfying y ∈ P and y ̸= x.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Let P be a polyhedron. A vector x ∈ P is an extreme
point of P if we cannot find two vectors y , z ∈ P, both
different from x, and a scalar λ ∈ [0, 1], such that
x = λy + (1 − λ)z.
Let P be a polyhedron. A vector x ∈ P is a vertex of P if
there exists some c such that c T x < c T y for all y
satisfying y ∈ P and y ̸= x.

In other words, x is a vertex of P if and only if P is on one

side of a hyperplane which meets P only at the point x.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Let x ∗ be an element of Rn and let I be the set of indices of
constraints that are active at x ∗ . The following are equivalent:
1 There exist n vectors in the set {ai : i ∈ I } which are
linear independent.
2 The span of vectors ai , i ∈ I , is Rn . That is, every
element of Rn is a linear combination of ai ’s.
3 The system of equations aiT x = bi , i ∈ I has a unique
solution.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Consider a polyhedron P defined by linear constraints, and let
x ∗ be an element of Rn .
1 The vector x ∗ is a basic solution if
1 All equality constraints are active

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Consider a polyhedron P defined by linear constraints, and let
x ∗ be an element of Rn .
1 The vector x ∗ is a basic solution if
1 All equality constraints are active
2 Out of the constraints that are active at x ∗ , n of them
are linear independent.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Consider a polyhedron P defined by linear constraints, and let
x ∗ be an element of Rn .
1 The vector x ∗ is a basic solution if
1 All equality constraints are active
2 Out of the constraints that are active at x ∗ , n of them
are linear independent.
2 If x ∗ is a basic solution that satisfies all constraints, we
say that it is a basic feasible solution.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Let P be a nonempty polyhedron and let x ∗ ∈ P. The
following are equivalent:
1 x ∗ is a vertex,
2 x ∗ is an extreme point,
3 x ∗ is a basic feasible solution.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Let P be a nonempty polyhedron and let x ∗ ∈ P. The
following are equivalent:
1 x ∗ is a vertex,
2 x ∗ is an extreme point,
3 x ∗ is a basic feasible solution.

Corollary
Given a finite number of linear inequality constraints, there
can only be a finite number of basic or basic feasible solutions.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Consider the constraints Ax = b and x ≥ 0 and assume that
the m × n matrix A has m linearly independent rows. A vector
x ∈ Rn is a basic solution if and only if we have Ax = b, and
there exist indices B(1), . . . , B(m) such that:
1 The columns AB(1) , . . . , AB(m) are linear independent.
2 If i ̸= B(1), . . . , B(m), then xi = 0.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example 1. Let the constraint Ax = b be of the form

1 1 2 1 0 0 0 8
   

 0 1 6 0 1 0 0   12 
   
x = 
 4 .
 
 1 0 0 0 0 1 0 
   
0 1 0 0 0 0 1 6

Choose A4 , A5 , A6 , A7 as our basic columns. They are linearly

independent and the corresponding basis matrix is the identity.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example 1. Let the constraint Ax = b be of the form

1 1 2 1 0 0 0 8
   

 0 1 6 0 1 0 0   12 
   
x = 
 4 .
 
 1 0 0 0 0 1 0 
   
0 1 0 0 0 0 1 6

Choose A4 , A5 , A6 , A7 as our basic columns. They are linearly

independent and the corresponding basis matrix is the identity.
We then obtain the basic solution x = (0, 0, 0, 8, 12, 4, 6)
which is nonnegative and, therefore, is a basic feasible solution.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example 1. Let the constraint Ax = b be of the form

1 1 2 1 0 0 0 8
   

 0 1 6 0 1 0 0   12 
   
x = 
 4 .
 
 1 0 0 0 0 1 0 
   
0 1 0 0 0 0 1 6

Choose A4 , A5 , A6 , A7 as our basic columns. They are linearly

independent and the corresponding basis matrix is the identity.
We then obtain the basic solution x = (0, 0, 0, 8, 12, 4, 6)
which is nonnegative and, therefore, is a basic feasible solution.
Another basis is obtained by choosing the columns A3 , A5 , A6 ,
A7 . The corresponding basic solution is (0, 0, 4, 0, −12, 4, 6),
not feasible because x5 = −12 < 0.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Two distinct basic solutions are said to be adjacent if there are
n − 1 linearly independent constraints that are active at both
of them.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Two distinct basic solutions are said to be adjacent if there are
n − 1 linearly independent constraints that are active at both
of them.
For standard form problems, we also say that two bases are
adjacent if they share all but one basic column.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Two distinct basic solutions are said to be adjacent if there are
n − 1 linearly independent constraints that are active at both
of them.
For standard form problems, we also say that two bases are
adjacent if they share all but one basic column.
Adjacent basic solutions can always be obtained from two
adjacent bases.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Two distinct basic solutions are said to be adjacent if there are
n − 1 linearly independent constraints that are active at both
of them.
For standard form problems, we also say that two bases are
adjacent if they share all but one basic column.
Adjacent basic solutions can always be obtained from two
adjacent bases.
Example 2. In Example 1, the bases {A4 , A5 , A6 , A7 } and
{A3 , A5 , A6 , A7 } are adjacent because all but one columns are
the same.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Two distinct basic solutions are said to be adjacent if there are
n − 1 linearly independent constraints that are active at both
of them.
For standard form problems, we also say that two bases are
adjacent if they share all but one basic column.
Adjacent basic solutions can always be obtained from two
adjacent bases.
Example 2. In Example 1, the bases {A4 , A5 , A6 , A7 } and
{A3 , A5 , A6 , A7 } are adjacent because all but one columns are
the same.
The corresponding basic solutions x = (0, 0, 0, 8, 12, 4, 6) and
x = (0, 0, 4, 0, −12, 4, 6) are adjacent.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Two distinct basic solutions are said to be adjacent if there are
n − 1 linearly independent constraints that are active at both
of them.
For standard form problems, we also say that two bases are
adjacent if they share all but one basic column.
Adjacent basic solutions can always be obtained from two
adjacent bases.
Example 2. In Example 1, the bases {A4 , A5 , A6 , A7 } and
{A3 , A5 , A6 , A7 } are adjacent because all but one columns are
the same.
The corresponding basic solutions x = (0, 0, 0, 8, 12, 4, 6) and
x = (0, 0, 4, 0, −12, 4, 6) are adjacent.
We have n = 7 and a total of six common linearly independent
active constraints: x1 ≥ 0, x2 ≥ 0 and four equality
constraints.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A basic solution x ∈ Rn is said to be degenerate if more than
n of the constraints are active at x.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A basic solution x ∈ Rn is said to be degenerate if more than
n of the constraints are active at x.

Example. Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤8
x2 + 6x3 ≤ 12
x1 ≤4
x2 ≤6
x1 , x2 , x3 ≥ 0.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A basic solution x ∈ Rn is said to be degenerate if more than
n of the constraints are active at x.

Example. Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤8
x2 + 6x3 ≤ 12
x1 ≤4
x2 ≤6
x1 , x2 , x3 ≥ 0.

We have
x 1 = (2, 6, 0) is a nondegenerate basic feasible solution,

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A basic solution x ∈ Rn is said to be degenerate if more than
n of the constraints are active at x.

Example. Consider the polyhedron P defined by the constraints

x1 + x2 + 2x3 ≤8
x2 + 6x3 ≤ 12
x1 ≤4
x2 ≤6
x1 , x2 , x3 ≥ 0.

We have
x 1 = (2, 6, 0) is a nondegenerate basic feasible solution,
x 2 = (4, 0, 2) is a degenerate basic feasible solution.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
Consider the standard form polyhedron

P = {x ∈ Rn : Ax = b, x ≥ 0}

and let x be a basic solution. Let m be the number of rows of

A. The vector x is degenerate if more than n − m of the
components of x are zero.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example. Consider again the constraints Ax = b, x ≥ 0

1 1 2 1 0 0 0 8
   

 0 1 6 0 1 0 0   12 
   
x = 
 4 .
 
 1 0 0 0 0 1 0 
   
0 1 0 0 0 0 1 6

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example. Consider again the constraints Ax = b, x ≥ 0

1 1 2 1 0 0 0 8
   

 0 1 6 0 1 0 0   12 
   
x = 
 4 .
 
 1 0 0 0 0 1 0 
   
0 1 0 0 0 0 1 6

Consider the basis consisting of the linearly independent

columns A1 , A2 , A3 , A7 . To calculate the corresponding basic
solution, we first set the nonbasic variables x4 , x5 , and x6 to
zero, and then solve the system Ax = b for the remaining
variables, to obtain x = (4, 0, 2, 0, 0, 0, 6).

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example. Consider again the constraints Ax = b, x ≥ 0

1 1 2 1 0 0 0 8
   

 0 1 6 0 1 0 0   12 
   
x = 
 4 .
 
 1 0 0 0 0 1 0 
   
0 1 0 0 0 0 1 6

Consider the basis consisting of the linearly independent

columns A1 , A2 , A3 , A7 . To calculate the corresponding basic
solution, we first set the nonbasic variables x4 , x5 , and x6 to
zero, and then solve the system Ax = b for the remaining
variables, to obtain x = (4, 0, 2, 0, 0, 0, 6).
This is a degenerate basic feasible solution, because we have a
total of four variables that are zero, whereas n − m = 3.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A polyhedron P ∈ Rn contains a line if there exists a vector
x ∈ P and a nonzero vector d ∈ Rn such that x + λd ∈ P for
all scalars λ.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Definition
A polyhedron P ∈ Rn contains a line if there exists a vector
x ∈ P and a nonzero vector d ∈ Rn such that x + λd ∈ P for
all scalars λ.

Theorem
Suppose that the polyhedron

P = {x ∈ Rn : Ax ≥ b}

is nonempty. Then the following are equivalent:

1 The polyhedron P has at least one extreme point.
2 The polyhedron P does not contain a line.
3 There exist m vectors out of the family A1 , . . . , An , which
are linearly independent.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Every nonempty polyhedron in standard form has at least one
basic feasible solution.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Every nonempty polyhedron in standard form has at least one
basic feasible solution.

Theorem
Consider the linear programming problem of minimizing c T x
over a polyhedron P. Suppose that P has at least one extreme
point and that there exists an optimal solution. Then there
exists an optimal solution which is an extreme point of P.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Consider the linear programming problem of minimizing c T x
over a polyhedron P. Suppose that P has at least one extreme
point. Then, either the objective function is equal to −∞, or
there exists an extreme point which is optimal.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
Consider the linear programming problem of minimizing c T x
over a polyhedron P. Suppose that P has at least one extreme
point. Then, either the objective function is equal to −∞, or
there exists an extreme point which is optimal.

Corollary
Consider the linear programming problem of minimizing c T x
over a nonempty polyhedron. Then, either the optimal value is
equal to −∞, or there is an optimal solution.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example. Consider the constraints Ax = b, x ≥ 0. Find all
bases of matrix A and basic feasible solutions corresponding to
each of these bases.

 x1 − x2 + x3 = 1

3x1 + x2 − x3 = 7

xj ≥ 0, j = 1, 2, 3.


Describe geometrically the feasible set.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Theorem
A linear programming problem has an optimal solution if and
only if the set of feasible solutions is nonempty and the
objective function is bounded.

Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example. Solve the linear programming problem

f (x) = 2x1 + 3x2 → max

subject to 
4x1 + 2x2 + x3 = 5

x1 + 3x2 =1

xj ≥ 0, j = 1, 2, 3.


Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming

Example. Solve the linear programming problem

f (x) = −2x1 + x2 → min

subject to 

 x1 + 2x2 ≥ 2

2x − 3x ≤ 6
1 2


 4x1 + 5x2 ≤ 20
x1 , x2 ≥ 0.


Dr. Nguyễn Ngọc Phan The Geometry of Linear Programming