MTL103 Spring 2023 Lecture 5

Optimization Methods and Applications

Lecturer: Prof. Minati De

Vaibhav Mishra Riya Sawhney Somaditya Singh Kushal Kumar Gupta

2020CS10448 2020CS10374 2020CS10389 2020CS10355

Dhruv Tyagi Akarsh Jain

2020CS10341 2020CS10318

11 January 2023

1 Overview
In the previous chapter, we started modeling the linear programming problem geometrically and
introduced the notions of Polyhedra, convex sets, Hyperplane, Halfspaces and Extreme point.
We shall build upon the terminologies in this chapter to further expand the model and build towards
solving the problems belonging to the LP class geometrically.

2 Main Section
Definition: Vertex Of A Polyhedron
Let P be a polyhedron. A vector x ∈ P is a vertex of P if there exists some c such that cT x < cT y
for all y satisfying y ∈ P and y ̸= x.

Intuitively, a vertex is a unique solution to a linear programming problem for the set of feasible
solutions P , with respect to cost vector c. In other words, x is a vertex of P iff P is on one side of a
hyperplane (the hyperplane {y|c′ y = c′ x}) which meets P only at the point x.

Figure 1: The line at the bottom touches P at a single point and x is a vertex. On the other hand,
w is not a vertex because there is no hyperplane that meets P only at w.

1
 Consider a polynomial P ⊂ Rn defined in terms of the linear equality and inequality constraints

a′i x ≥ bi , i ∈ M1 ,

a′i x ≤ bi , i ∈ M2 ,

a′i x = bi , i ∈ M3 ,

where M1 , M2 , and M3 are finite index sets, each ai is a vector in Rn , and each bi is a scalar.

Definition: Active or Binding Constraint

If a vector x* satisfies a′i x∗ = bi , for some i in M1 , M2 , or M3 , we say that the corresponding
constraint is active or binding at x*.

Definition: Basic Solution

Consider a polyhedron P defined by linear equality and inequality constraints, and let x∗ be
an element of Rn The vector x∗ is a basic solution if:

• All equality constraints are active.

• Out of the constraints that are active at x∗ , there are n of them that are linearly indepen-
dent.

Definition: Basic feasible solution

The vector x∗ is a Basic Feasible Solution if
x∗ is a basic solution and
all constraints are satisfied.

Examples of Basic Solution and Basic Feasible Solution:

In reference to Figure 2, we note that points A, B, and C are basic feasible solutions. Point D is not
a basic solution because it fails to satisfy the equality constraint. Point E is feasible, but not basic.
If the equality constraint x1 + x2 + x3 = 1 were to be replaced by the constraints x1 + x2 + x3 ≥ 1 and
x1 + x2 + x3 ≤ 1, then D would be a basic solution, according to our definition. This shows that
whether a point is a basic solution or not may depend on the way that a polyhedron is
represented.

2
Figure 2: Let P = {(x1 , x2 , x3 )|x1 + x2 + x3 = 1, x1 , x2 , x3 > 0}. There are three constraints that are
active at each one of the points A, B, C and D. There are only two constraints that are active at
point E, namely x1 + x2 + x3 = 1 and x2 = 0

Figure 3: The points A, B, C, D, E, F are all basic solutions because at each one of them, there are
two linearly independent constraints that are active. Points C, D, E, F are basic feasible solutions.

3 Basics of Linear Algebra

Theorem: Let x∗ ∈ Rn and I = {i|aTi x∗ = bi } be the set of indices of active constraints at x∗ .
Then the following are equivalent:

1. ∃n vectors in the set {ai |i ∈ I}, which are linearly independent.

2. The span of the vectors in {ai |i ∈ I} is all of Rn

3. The system of equations {aTi x = bi |i ∈ I} has a unique solution.

Proof
2 =⇒ 1

Suppose that the vectors ai , i ∈ I span all of Rn . Then the span of these vectors has dimension n.
Hence, n of these form a basis of Rn and therefore are linearly independent.

3
1 =⇒ 2

Suppose that the vectors ai , i ∈ I are linearly independent. Then the subspace spanned by these
vectors has dimension n and must be equal to Rn . Hence, every element in Rn is a linear combination
of ai , i ∈ I.
This establishes 1 is equivalent to 2.

¬3 =⇒ ¬2

If the system of equation {aTi x = bi |i ∈ I} has a multiple solutions (say x1 and x2 ), then the non-zero
vector d = x1 − x2 satisfies a′i d = 0 for all i ∈ I. Since d is orthogonal to each vector ai hence, it is
not a linear combination of them and therefore ai , i ∈ I does not span Rn .

¬2 =⇒ ¬3

If ai , i ∈ I does not span Rn , then choose a vector d which orthogonal to the subspace spanned by
these vectors. If x satisfies {aTi x = bi |i ∈ I}, then x + d also satisfies the system of equations. Hence,
there are multiple solutions.
This establishes 3 is equivalent to 2 (which is equivalent to 1).

Theorem:
Suppose that span S of the vectors x1 , . . . xk has dimension m. Then,

1. ∃ a basis of S consisting of m of the vectors x1 , . . . xk

2. if l ≤ m and x1 , . . . xl are linearly independent, we can form a basis of S by starting with
x1 , . . . xl and choosing m − l of the vectors from xl+1 , . . . xk

Proof
Part 2

If every vector xl+1 , . . . , xk can be expressed as a linear combination of x1 , . . . xl , then every vector
in the span of x1 , . . . , xk is also a linear combination of x1 , . . . , xl , and the latter form a basis(in
particular, m = k). Otherwise, atleast one of the vectors xl+1 , . . . xk is linearly independent from
x1 , . . . xk . By picking one such vector, we now have l + 1 of the vectors x1 , . . . , xk that are linearly
independent. By repeating this process m − l times, we end up with the desired basis of S.

Part 1

Part 1 is a special case of part2 with l = 0.

4
Equivalence of Extreme Point, Vertex and Basic Feasible So-
lution
Let us recapitulate the definitions of extreme point, vertex and basic feasible solution of a
polyhedron-

• A vector x ∈ P is a an extreme point of P if we cannot find two vectors y, z ∈ P , both

different from x, and a scalar λ ∈ [0, 1] such that x = λy + (1 − λ)z.

• A vector x ∈ P is a vertex of P if there exists some c such that cT x < cT y for all satisfying
y ∈ P and y ̸= x.

• The vector x∗ is a basic feasible solution if-

– All equality constraints are active;

– Out of all the constraints that are active at x∗ , there are n of them that are linearly
independent;
– All constraints are satisfied.

We have given so far three different definitions that are meant to capture the same concept; two
of them are geometric (extreme point, vertex) and the third is algebraic (basic feasible solution).
Fortunately, all three definitions are equivalent as we prove next and, for this reason, the three terms
can be used interchangeably.

Theorem:
Let P be the non-empty polyhedron in Rn defined by the constraint equations, and x∗ ∈ P .
Then the following are equivalent:

1. x∗ is a Vertex
2. x∗ is an Extreme point
3. x∗ is a Basic Feasible Solution w.r.t. the constraints

Proof
1 =⇒ 2

Given that x∗ is a vertex, we know that for some c ∈ Rn , cT x∗ < cT y, ∀y ∈ P, y =

̸ x∗ . Now assume
that x∗ is not an extreme point. Then x∗ = λy + (1 − λ)z for some y, z ∈ P , 0 < λ < 1. Then we
have
cT x∗ = λcT y + (1 − λ)cT z < cT x∗

which is a contradiction. Hence x∗ is an extreme point.

2 =⇒ 3

The proof is by contradiction. Let S = {i | aTi x∗ = bi }, that is, the indices of constraints that are
binding at x∗ . We assume that S includes all the equality constraints, and later we will justify this

5
assumption. If x∗ is not a Basic Feasible Solution, there are < n linearly independent constraints
in S. Thus ∃d ∈ Rn which is orthogonal to the subspace spanned by the vectors ai . Thus x∗ + ϵd
satisfies the constraints ∈ S, ∀ϵ ∈ R. Let J be the set of indices of the inequality constraints that
are not binding at x∗ . If we choose ϵ such that | ϵaTj d |<| bj − aTj x∗ |, ∀j ∈ J, then y = x∗ + ϵd and
z = x∗ − ϵd satisfy the constraints in J as well, and so y, z ∈ P . But x = y+z 2
which is a contradiction
since x was assumed to be an extreme point.
Note that if S did not include all the linearly independent equality constraints, then the same proof
above could be repeated as the equality constraints that were present could not have spanned Rn .

3 =⇒ 1

Let {(ai , bi )}ni=1 be the set of linearly independent binding constraints at x∗ . Here we have assumed
that the constraints are of the form aTi x = bi or aTi x ≤ bi . Then x∗ is the unique point in Rn where
these constraints are binding. Consider the cost vector c = ni=1 ai . Then for any point y ∈ P ,
P

y ̸= x∗ , ∃i such that aTi y < bi . Thus cT x∗ > cT y for all such y. Thus x∗ is a vertex.