Linear Programming

LUISS

1/18
(LUISS) Linear Programming 1 / 18
Contents of the week

Some prototypical examples of linear programs

LPs in standard and canonical forms
Polyhedra and polytopes
Extreme points and basic feasible solutions
Optimal solutions
Informal description of the Simplex method

2/18
(LUISS) Linear Programming 2 / 18
 Linear Programming

Linear programming problems

Definition
A linear program consists in maximizing or minimizing a linear function under a
set of linear equality and inequality constraints

3/18
(LUISS) Linear Programming 3 / 18
 Linear Programming

Example: Production planning

A small manufacturer produces tools of types A and B, which require different

amounts of labor, wood and metal, and yield different profits as shown below
Tool Labor Wood Metal Profit
type [hr/unit] [kg/unit] [kg/unit] [e/unit]
A 1.0 1.0 2.0 50
B 2.0 1.0 1.0 40
Available are 120 hours of labor, 70 units of wood, and 100 units of metal.
What is the production mix that maximizes profit ?

4/18
(LUISS) Linear Programming 4 / 18
 Linear Programming

Example: Production planning

A small manufacturer produces tools of types A and B, which require different

amounts of labor, wood and metal, and yield different profits as shown below
Tool Labor Wood Metal Profit
type [hr/unit] [kg/unit] [kg/unit] [e/unit]
A 1.0 1.0 2.0 50
B 2.0 1.0 1.0 40
Available are 120 hours of labor, 70 units of wood, and 100 units of metal.
What is the production mix that maximizes profit ?


 max 50 xA + 40 xB
 s.t. xA + 2 xB ≤ 120


xA + xB ≤ 70
2 xA + xB ≤ 100




xA ≥ 0, xB ≥ 0


4/18
(LUISS) Linear Programming 4 / 18
 Linear Programming

Example: Production planning



 max 50 xA + 40 xB
s.t. xA + 2 xB ≤ 120




xA + xB ≤ 70


 2 xA + xB ≤ 100
xA ≥ 0




xB ≥ 0


5/18
(LUISS) Linear Programming 5 / 18
 Linear Programming

Example: Production planning



 max 50 xA + 40 xB
s.t. xA + 2 xB ≤ 120




xA + xB ≤ 70


 2 xA + xB ≤ 100
xA ≥ 0




xB ≥ 0


Optimal solution: xA∗ = 30 / xB∗ = 40

Maximum profit is e 3100
Total labor used is 110 [hr]
Total wood used is 70 [kg]
Total metal used is 100 [kg]

5/18
(LUISS) Linear Programming 5 / 18
 Linear Programming

Example: Diet problem

Given a set of foods with their nutrient information and cost per serving,
determine the number of meal servings that satisfy a set of nutritional
requirements at minimum cost.
Food Calories [cal] Vitamin A [mcg] Cost [e]
Carrots 40 450 0.12
Milk 63 150 0.15
Chicken 220 5 2.00

6/18
(LUISS) Linear Programming 6 / 18
 Linear Programming

Example: Diet problem

Given a set of foods with their nutrient information and cost per serving,
determine the number of meal servings that satisfy a set of nutritional
requirements at minimum cost.
Food Calories [cal] Vitamin A [mcg] Cost [e]
Carrots 40 450 0.12
Milk 63 150 0.15
Chicken 220 5 2.00

Suppose that the maximum number of servings is 10 for each food, and there are
restrictions on the daily calories (between 1600 and 2500 [cal]) and the amount of
Vitamin A (between 800 and 1100 [mcg]).

6/18
(LUISS) Linear Programming 6 / 18
 Linear Programming

Example: Diet problem



 min 0.12 x1 + 0.15 x2 + 2.0 x3
s.t. 1600 ≤ 40 x1 + 63 x2 + 220 x3 ≤ 2500


 800 ≤ 450 x1 + 150 x2 + 5 x3 ≤ 1100
0 ≤ x1 , x2 , x3 ≤ 10


7/18
(LUISS) Linear Programming 7 / 18
 Linear Programming

Example: Diet problem



 min 0.12 x1 + 0.15 x2 + 2.0 x3
s.t. 1600 ≤ 40 x1 + 63 x2 + 220 x3 ≤ 2500


 800 ≤ 450 x1 + 150 x2 + 5 x3 ≤ 1100
0 ≤ x1 , x2 , x3 ≤ 10


Optimal solution: x1∗ = 0.0 (carrot) / x2∗ = 7.16 (milk) / x3∗ = 5.22 (chicken)
Minimum cost is e 11.519
Total number of Calories is 1600
Total amount of Vitamin A is 1100

7/18
(LUISS) Linear Programming 7 / 18
 Linear Programming

Example: Optimal transport

A gadget manufacturer has two factories F1 and F2 with production capacities 6
and 9, from which it serves three retail centers C1 , C2 , C3 with demands 8, 5, 2.
The transportation costs are shown below
C1 C2 C3
F1 5 5 3
F2 6 4 1
Which demand should be served from each factory?

8/18
(LUISS) Linear Programming 8 / 18
 Linear Programming

Example: Optimal transport

A gadget manufacturer has two factories F1 and F2 with production capacities 6
and 9, from which it serves three retail centers C1 , C2 , C3 with demands 8, 5, 2.
The transportation costs are shown below
C1 C2 C3
F1 5 5 3
F2 6 4 1
Which demand should be served from each factory?
Amount shipped from factory Fi to retail center Cj = xi,j


 min 5 x1,1 + 5 x1,2 + 3 x1,3 + 6 x2,1 + 4 x2,2 + x2,3
s.t. x1,1 + x1,2 + x1,3 = 6




x2,1 + x2,2 + x2,3 = 9



x1,1 + x2,1 = 8
x1,2 + x2,2 = 5




x 1,3 + x2,3 = 2




xi,j ≥ 0 (∀i = 1, 2)(∀j = 1, 2, 3)


8/18
(LUISS) Linear Programming 8 / 18
 Linear Programming

Matrix notation for linear programs

A linear program with only inequality constraints

Pn
max i=1 ci xi




 a x + a x + ··· + a x
11 1 12 2 1n n ≤ b1
..



 .
am1 x1 + am2 x2 + · · · + amn xn ≤ bm

can be written in compact form using matrix notation

max c t x

Ax ≤ b

where A is an m × n matrix, b ∈ Rm , and c ∈ Rn .

9/18
(LUISS) Linear Programming 9 / 18
 Linear Programming

Matrix notation for linear programs

Similarly, a linear program with only equality plus sign constraints

 Pn
 min i=1 ci xi


 a11 x1 + a12 x2 + · · · + a1n xn = b1



..
 .
 am1 x1 + am2 x2 + · · · + amn xn = bm




x1 ≥ 0, . . . , xn ≥ 0


can be expressed in compact form as


 min c t x
Ax = b
x ≥0


where A is an m × n matrix, b ∈ Rm , and c ∈ Rn .

10/18
(LUISS) Linear Programming 10 / 18
 Linear Programming

Transformations of linear programs

There are several forms of linear programs, each of which can be transformed into
another equivalent form by using the following simple tricks:

A minimization problem is converted to maximization (and vice-versa) by

changing the sign of objective function.
An inequality ≥ is converted to ≤ by multiplying by -1.
A sign constraint xi ≥ 0 can be included as an additional row in Ax ≤ b.
An unconstrained variable xi ∈ R can be replaced by xi = xi+ − xi− with
xi+ ≥ 0 and xi− ≥ 0.
An inequality at x ≤ b can be converted to an equality at x + s = b plus an
additional sign constraint s ≥ 0. Here s is called a slack variable.
An equality at x = b is equivalent to two inequalities at x ≤ b and at x ≥ b.
Alternatively, an equality can be used to eliminate one variable from the
problem.

11/18
(LUISS) Linear Programming 11 / 18
 Linear Programming

Transformations of linear programs

There are several forms of linear programs, each of which can be transformed into
another equivalent form by using the following simple tricks:

A minimization problem is converted to maximization (and vice-versa) by

changing the sign of objective function.
An inequality ≥ is converted to ≤ by multiplying by -1.
A sign constraint xi ≥ 0 can be included as an additional row in Ax ≤ b.
An unconstrained variable xi ∈ R can be replaced by xi = xi+ − xi− with
xi+ ≥ 0 and xi− ≥ 0.
An inequality at x ≤ b can be converted to an equality at x + s = b plus an
additional sign constraint s ≥ 0. Here s is called a slack variable.
An equality at x = b is equivalent to two inequalities at x ≤ b and at x ≥ b.
Alternatively, an equality can be used to eliminate one variable from the
problem.

11/18
(LUISS) Linear Programming 11 / 18
 Linear Programming

Transformations of linear programs

There are several forms of linear programs, each of which can be transformed into
another equivalent form by using the following simple tricks:

A minimization problem is converted to maximization (and vice-versa) by

changing the sign of objective function.
An inequality ≥ is converted to ≤ by multiplying by -1.
A sign constraint xi ≥ 0 can be included as an additional row in Ax ≤ b.
An unconstrained variable xi ∈ R can be replaced by xi = xi+ − xi− with
xi+ ≥ 0 and xi− ≥ 0.
An inequality at x ≤ b can be converted to an equality at x + s = b plus an
additional sign constraint s ≥ 0. Here s is called a slack variable.
An equality at x = b is equivalent to two inequalities at x ≤ b and at x ≥ b.
Alternatively, an equality can be used to eliminate one variable from the
problem.

11/18
(LUISS) Linear Programming 11 / 18
 Linear Programming

Transformations of linear programs

There are several forms of linear programs, each of which can be transformed into
another equivalent form by using the following simple tricks:

A minimization problem is converted to maximization (and vice-versa) by

changing the sign of objective function.
An inequality ≥ is converted to ≤ by multiplying by -1.
A sign constraint xi ≥ 0 can be included as an additional row in Ax ≤ b.
An unconstrained variable xi ∈ R can be replaced by xi = xi+ − xi− with
xi+ ≥ 0 and xi− ≥ 0.
An inequality at x ≤ b can be converted to an equality at x + s = b plus an
additional sign constraint s ≥ 0. Here s is called a slack variable.
An equality at x = b is equivalent to two inequalities at x ≤ b and at x ≥ b.
Alternatively, an equality can be used to eliminate one variable from the
problem.

11/18
(LUISS) Linear Programming 11 / 18
 Linear Programming

Transformations of linear programs

There are several forms of linear programs, each of which can be transformed into
another equivalent form by using the following simple tricks:

A minimization problem is converted to maximization (and vice-versa) by

changing the sign of objective function.
An inequality ≥ is converted to ≤ by multiplying by -1.
A sign constraint xi ≥ 0 can be included as an additional row in Ax ≤ b.
An unconstrained variable xi ∈ R can be replaced by xi = xi+ − xi− with
xi+ ≥ 0 and xi− ≥ 0.
An inequality at x ≤ b can be converted to an equality at x + s = b plus an
additional sign constraint s ≥ 0. Here s is called a slack variable.
An equality at x = b is equivalent to two inequalities at x ≤ b and at x ≥ b.
Alternatively, an equality can be used to eliminate one variable from the
problem.

11/18
(LUISS) Linear Programming 11 / 18
 Linear Programming

Transformations of linear programs

There are several forms of linear programs, each of which can be transformed into
another equivalent form by using the following simple tricks:

A minimization problem is converted to maximization (and vice-versa) by

changing the sign of objective function.
An inequality ≥ is converted to ≤ by multiplying by -1.
A sign constraint xi ≥ 0 can be included as an additional row in Ax ≤ b.
An unconstrained variable xi ∈ R can be replaced by xi = xi+ − xi− with
xi+ ≥ 0 and xi− ≥ 0.
An inequality at x ≤ b can be converted to an equality at x + s = b plus an
additional sign constraint s ≥ 0. Here s is called a slack variable.
An equality at x = b is equivalent to two inequalities at x ≤ b and at x ≥ b.
Alternatively, an equality can be used to eliminate one variable from the
problem.

11/18
(LUISS) Linear Programming 11 / 18
 Linear Programming

Transformations of linear programs

With these simple transformations every linear program can be stated in any of
the following forms, for appropriately defined A, b, and c.

max c t x

Compact Form
Ax ≤ b

 max c t x
Canonical Form Ax ≤ b
x ≥0


 max c t x
Standard Form Ax = b
x ≥0


12/18
(LUISS) Linear Programming 12 / 18
 Linear Programming

Feasibility, polyhedra and polytopes

The set of vectors satisfying the equalities and inequalities of a linear program is
called its feasible set.
These are a special class of convex sets called polyhedra. When they are bounded
they are called polytopes.

Example:

2x1 + x2 ≥ 1

13/18
(LUISS) Linear Programming 13 / 18
 Linear Programming

Feasibility, polyhedra and polytopes

The set of vectors satisfying the equalities and inequalities of a linear program is
called its feasible set.
These are a special class of convex sets called polyhedra. When they are bounded
they are called polytopes.

Example: x2

2x1 + x2 ≥ 1

x1

13/18
(LUISS) Linear Programming 13 / 18
 Linear Programming

Feasibility, polyhedra and polytopes

The set of vectors satisfying the equalities and inequalities of a linear program is
called its feasible set.
These are a special class of convex sets called polyhedra. When they are bounded
they are called polytopes.

Example: x2

2x1 + x2 ≥ 1
x1 + 3x2 ≥ 2

x1

13/18
(LUISS) Linear Programming 13 / 18
 Linear Programming

Feasibility, polyhedra and polytopes

The set of vectors satisfying the equalities and inequalities of a linear program is
called its feasible set.
These are a special class of convex sets called polyhedra. When they are bounded
they are called polytopes.

Example: x2

2x1 + x2 ≥ 1
x1 + 3x2 ≥ 2
− 3x1 + x2 ≥ −3

x1

13/18
(LUISS) Linear Programming 13 / 18
 Linear Programming

Feasibility, polyhedra and polytopes

The set of vectors satisfying the equalities and inequalities of a linear program is
called its feasible set.
These are a special class of convex sets called polyhedra. When they are bounded
they are called polytopes.

Example: x2

2x1 + x2 ≥ 1
x1 + 3x2 ≥ 2
− 3x1 + x2 ≥ −3
3x1 − 2x2 ≥ −3

x1

13/18
(LUISS) Linear Programming 13 / 18
 Linear Programming

Feasibility, polyhedra and polytopes

The set of vectors satisfying the equalities and inequalities of a linear program is
called its feasible set.
These are a special class of convex sets called polyhedra. When they are bounded
they are called polytopes.

Example: x2

2x1 + x2 ≥ 1
x1 + 3x2 ≥ 2
− 3x1 + x2 ≥ −3
3x1 − 2x2 ≥ −3
− 2x1 − 3x2 ≥ −6
x1
Observe the corners !

13/18
(LUISS) Linear Programming 13 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1 + 2x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1 + 2x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1 + 2x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1 + 2x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1 + 2x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1 + 2x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Solving a linear program

x2

Example:

min x1 + 2x2
x ∈P P

x1

14/18
(LUISS) Linear Programming 14 / 18
 Linear Programming

Basic Feasible Solutions (BFS)

Let P = {x ∈ Rn : Ax ≤ b} be a nonempty and bounded polyhedron (this

requires more inequalitites than variables) and consider the linear program

min c t x


Ax ≤ b

A basic solution (BS) of the system Ax ≤ b is any point obtained by forcing n of

these inequalities to be satisfied as equalities, and such that the resulting system
of linear equations has a unique solution (i.e. n linearly independent equations).
If the resulting point lies in P (i.e. if it satisfies the remaining inequalities) it is
called a basic feasible solution (BFS).

Theorem
A polytope P = {x ∈ Rn : Ax ≤ b} has finitely many BS.
min{c t x : x ∈ P} is attained at least at one BFS.

15/18
(LUISS) Linear Programming 15 / 18
 Linear Programming

Basic Feasible Solutions (BFS)

Let P = {x ∈ Rn : Ax ≤ b} be a nonempty and bounded polyhedron (this

requires more inequalitites than variables) and consider the linear program

min c t x


Ax ≤ b

A basic solution (BS) of the system Ax ≤ b is any point obtained by forcing n of

these inequalities to be satisfied as equalities, and such that the resulting system
of linear equations has a unique solution (i.e. n linearly independent equations).
If the resulting point lies in P (i.e. if it satisfies the remaining inequalities) it is
called a basic feasible solution (BFS).

Theorem
A polytope P = {x ∈ Rn : Ax ≤ b} has finitely many BS.
min{c t x : x ∈ P} is attained at least at one BFS.

15/18
(LUISS) Linear Programming 15 / 18
 Linear Programming

The Simplex method – idea

A brute force algorithm for solving a linear program is to try out all the possible
BS until we find the best BFS (if any). This is prohibitively slow.
The Simplex method explores the BFS in a smarter way, moving from one BFS to
a “neighboring” one which has strictly smaller objective value. Two BFS are
neighbors if the equalities that define each one differ only by a single equation.

16/18
(LUISS) Linear Programming 16 / 18
 Linear Programming

The Simplex method – idea

A brute force algorithm for solving a linear program is to try out all the possible
BS until we find the best BFS (if any). This is prohibitively slow.
The Simplex method explores the BFS in a smarter way, moving from one BFS to
a “neighboring” one which has strictly smaller objective value. Two BFS are
neighbors if the equalities that define each one differ only by a single equation.

16/18
(LUISS) Linear Programming 16 / 18
 Linear Programming

Simplex method – properties

While the Simplex can be very slow in the worst case, it is extremely efficient in
most practical cases.

The Simplex method allows for unbounded polyhedra and is able to detect when
the problem is infeasible, when it has no optimal solutions, and when it does. In
the latter case it finds one optimal BFS.

In principle the method could cycle in degenerate cases (different basis defining
the same BFS) but there exist anti-cycling strategies.

17/18
(LUISS) Linear Programming 17 / 18
 Linear Programming

Simplex method – properties

While the Simplex can be very slow in the worst case, it is extremely efficient in
most practical cases.

The Simplex method allows for unbounded polyhedra and is able to detect when
the problem is infeasible, when it has no optimal solutions, and when it does. In
the latter case it finds one optimal BFS.

In principle the method could cycle in degenerate cases (different basis defining
the same BFS) but there exist anti-cycling strategies.

17/18
(LUISS) Linear Programming 17 / 18
 Linear Programming

Simplex method – properties

While the Simplex can be very slow in the worst case, it is extremely efficient in
most practical cases.

The Simplex method allows for unbounded polyhedra and is able to detect when
the problem is infeasible, when it has no optimal solutions, and when it does. In
the latter case it finds one optimal BFS.

In principle the method could cycle in degenerate cases (different basis defining
the same BFS) but there exist anti-cycling strategies.

17/18
(LUISS) Linear Programming 17 / 18
 Linear Programming

Simplex method – Degeneracy & Cycling

18/18
(LUISS) Linear Programming 18 / 18