15.

053
z

February 13, 2007

The Geometry of Linear Programs

the geometry of LPs illustrated

Quotes of the day

You don't understand anything until you learn it more than one way. Marvin Minsky

One finds limits by pushing

them.
Herbert Simon

Goal of this Lecture

Present the Geometry of Linear Programs

A key way of looking at LPs Others are algebraic and economic Some basic concepts 2-dimensional (2 variable) linear programs) 3-dimensional (3 variable) linear programs Properties of the set of feasible solutions and of optimal solutions generalizable to all linear programs

A Two Variable Linear Program

(a variant of the DTC example)

z = 3x + 5y 2x + x + x + x 3y 2y y

objective

10
y 6 5 4 3

(1) (2) (3) (4) (5) (6)

4

x, y 0

We could have used the original variable names of K and S, but it is simpler to use x and y since we usually think of the two axes as the x and y axis.

Finding an optimal solution

z

Introduce yourself to your partner Try to find an optimal solution to the linear program, without looking ahead.

Finding an optimal solution to a 2-variable LP can be challenging until you have seen the theory. Finding an optimal solution to an LP with more than a few variables and constraints is very hard to do by hand (or at least prone to errors) and we typically use a computer.

Some Basic Concepts y

5 4 3 2 1

A point is represented as a pair (x, y). For example, (2, 3). Sometimes, we will call (x, y) a vector. In that case, it is often represented with a line segment directed from the origin.

We go through this review pretty quickly

Lines
Every pair of (distinct) points determines a unique line. y p1 = (1, 5) p1 5 p2 = (4, 2)
4 3 2 1

L: x + y = 6. Alternative representation of the line: (1-)p1 + p2 for - . L = (1, 5) + (3, -3) for - .

p2

The alternative representation is really important, as we shall see on the next few slides.

Rays
y
5 4 3 2 1

Every pair of (distinct) points determines a unique ray beginning at the first point. p1 p1 = (1, 5) p2 = (4, 2) Ray: (1-)p1 + p2 for 0 . p2 = (1, 5) + (3, -3) for 0 .

Line segments
y
5 4 3 2 1

Every pair of points determines a unique line segment. p1 p1 = (1, 5) p2 = (4, 2) Segment: (1-)p1 + p2 for 0 1. p2 = (1, 5) + (3, -3) for 0 1.

We keep seeing (1-)p1 + lambda p2 as the formula. But the representation of the line segment is the most useful for our purposes.

Inequalities y
5 4 3 2 1

An inequality with two variables determines a unique half-plane

x+ 2y 6

10

A half plane contains the line as well as all points on one side of the line.

Graphing the Feasible Region

y
5 4 3 2 1

We will construct and shade the feasible region one or two constraints at a time.

11

y
5 4 3 2 1

Graph the Constraints: (1) 2x+ 3y 10 x 0 , y 0. (6)

2x + 3y = 10

12

OK. This is really three constraints despite what was said on the last slide.

Add the Constraint: (2) x + 2y 6 y

5 4 3 2 1

x + 2y = 6

13

Add the Constraint: x + y 5

y
5
4
3
2
1

x + y = 5

A constraint is called redundant if deleting the constraint does not increase the size of the feasible region. x + y = 5 is redundant

14

We dont concern ourselves much with redundant constraints 15.053. In principle, we could delete a redundant constraint because it might make the problem easier to solve. But in reality, it doesnt help much. But it is a widely used concept.

Add the Constraints: x 4; y 3 y

5 4 3 2 1

We have now graphed the feasible region.

15

How do we maximize z = 3x + 5y ? x
5 4 3 2 1

Lets avoid adding a 3rd dimension. Find a feasible solution such that 3x + 5y = p for different values of p. Choose p as large as possible.

16

If you can do so, try to maximize the objective function before looking ahead.

y
3

How do we maximize 3x + 5y ?
Is there a feasible solution such that 3x + 5y = 8? Is there a feasible solution such that 3x + 5y = 11?

1 3x + 5y = 8

3x + 5y = 11

x
17

The lines such as 3x + 5y = 8 are often called isoquant lines or isoprofit lines or isocost lines. Note that if the objective function is linear, then all isoprofit lines are parallel to each other.

y
3

Find the maximum value p such that there is a feasible solution with 3x + 5y = p. Move the line with profit p parallel as much as possible. This is called the geometric method for optimizing in 2D

3x + 5y = 16 The optimal solution occurs at a corner point.

3x + 5y = 11 3x + 5y = 8 1 2 3 4

x
18

The geometric method for solving a 2-variable LP is to move the isoprofit line so that (1) there is still at least one feasible point on the line (2) moving it any further would make all points on the line infeasible. If there is exactly one feasible point on the line it will be a corner point, which is defined on the next slide.

Corner Points
z

A corner point of the feasible region is a point that is not the midpoint of two other points of the feasible region.

Where are the corner points of this feasible region?

19

Note that this is a very simple definition of a corner point. It also leads to some unintuitive aspects. First of all, a corner point only makes sense if the feasible region is convex, that is, if two points p1 and p2 are feasible, then every point on the line segment [p1, p2] is also feasible. We will discuss convexity later in this lecture. Second, if the feasible region is a disk, then every point on the outside of the disk is a corner point even though there are no corners to a disk.

Solving for the Corner Point

In two dimensions, a corner point lies at the intersection of two lines. x + 2y = 6 2x + 3y = 10 y 3 2 1 x +2y = 6 2x + 4y = 12 2x + 3y = 10 2x +3y = 10 y=2

20

Its very useful that the corner point lies at the intersection of two lines. Then solving a system of equations with two variables and two equations will give the value of the corner point.

Solving for Corner Points

In three dimensions, a corner point is the intersection of three constraints. (3 planes) 0 x 2
0 y 2
z
0 z 2
x -y +z3
x y What is the red corner point?

21

The red corner point is the intersection of three planes

x = 2
z = 2
x - y + z = 3
The unique solution is x = 2, y = 1, z = 2.

An important difference between the geometry and the algebra

2x + y = 9
5 4 3 2 1

x + 2y = 9

Usually, corner points can be described in a unique way as an intersection of two lines (constraints). But not always. 2x + y 9 x + 2y 9 0x3 0y3 The point (3, 3) can be written as the intersection of two 5 lines in 6 ways. 6

22

This turns out to be very important in later lectures when we consider the simplex algorithm, which is the algebraic technique for solving a linear program. Within the simplex algorithm, there may be many structural descriptions (bases) that correspond to the same solution. This leads to a number of technical issues that need to be resolved. Well return to this slide later in the subject when we discuss degeneracy.

Remainder of the lecture: Corner Points.

Does every linear program with a feasible solution have a corner point? No. Consider the LP: Maximize y no constraints on x subject to 0 y 1

23

Even a person who has studied linear programming for a long time can forget that not all linear programs have corner points. Fortunately, all linear programs with non-negativity constraints do have corner points. We will generally deal with linear programs that have non-negativity constraints.

Theorem. If there is a feasible solution, and if there is no feasible line, then there is a corner point. Corollary. Any LP in which each variable is non-negative has a corner point.

y
5 4 3 2 1 1 2 3 4 5 6

24

If there is an optimal solution, and if there is a corner point, is there always an optimal solution that is a corner point? y
3 YES! It is true even if there is an entire line segment that is optimal.

25

It is also possible that the entire feasible region is optimal, but this can only happen when the objective is to maximize (or minimize) 0x + 0y. In this case, all corner points are optimal.

If there is an optimal solution, then there is an optimal solution that is a corner point.

se s. t o t. f p pr o of in it ts = a

26

In three dimensions, the isoprofit points form a plane. For example, we may have an objective of 2x + 3y + w =. Well try to avoid having z as one of the variables since z usually denotes the value of the objective function.

What types of Linear Programs are there?

There is no feasible solution. There is a feasible solution and an optimal solution. There is a feasible solution and the objective value is unbounded from below max x s.t. x + 2y -1 x 0, y 0 max x s.t. x + 2y 1 x 0, y 0 max s.t. x - 2y 1 x 0, y 0
27

Case 1 unfortunately happens too frequently, often because of an error. Sometimes it comes because the decision maker is trying to accomplish too much with too little. For example, in the application to radiation therapy to destroying brain tumors, a doctor may require a very large dose of radiation to the tumor while requiring a very low dose to non-tumor cells. But this may lead to a linear program with no feasible solution. Case 2 will always occur when the feasible region is non-empty and bounded. However, it can also occur with the feasible region is unbounded. For example, min {x : x >= 1}. Case 3 can only happen when the feasible region is unbounded.

Any other types

z

Is it possible to have an LP such that the feasible region is bounded, and such that there is no optimal solution? No. But it could happen if we
permitted strict inequality
constraints.
Maximize x

subject to 0 < x < 1

28

It might seem obvious that if an LP feasible region is bounded, then it must have an optimal solution. But this relies on the fact that the inequalities are not strict, as the example above shows. From a mathematical perspective, an LP feasible region is closed; that is, if a sequence of feasible points converges then it converges to a point that is feasible. But this is a digression, and is not used elsewhere in 15.053.

Convex Sets
A set S is convex if for every two points in the set, the line segment joining the points is also in the set; that is, if p1, p2 S, then so is(1-)p1 + p2 for [0,1] Theorem. The feasible region of a linear program is convex.

y
3

p1

p2

29

Convexity is of some use for linear programming. It is critically important in nonlinear programming. Non-linear programs are extremely hard to solve in general (impossible may be a better word); however, when the objective function is convex, they are often tractible. We will use properties of convexity a number of times in the remainder of this lecture and in the next lecture. Notice that we are using the same description of a line segment as earlier.

More on Convexity
Which of the following are convex ? or not ?

5 4 3 2 1

30

The doughnut (object 5) is not convex because it contains a hole in it.

The 8 points (object 8) is not convex because it only contains those 8 points and not
points in between.

Which shapes are convex Which regions are LP feasible regions? Which are the corner points of each shape?

31

Not convex: heart and moon

Not LP: heart and moon and circle. (The circle cannot be expressed as a finite
number of linear inequalities.)
Corner points: those that are not the midpoints of two other points of the object.
The top of the heart is not a corner point since it is the midpoint of two other points
of the heart. The points on the outside curve are corner points except for the points
below the top two curves.
All outside points on the circle are corner points, despite the fact that in common
usage we would not think of a circle as having any corners.
The cube has 8 corner points.
The outside points on the left of the moon are corner points. The outside on the
right side are not.
The infinite region has 4 corner points.

On corner points
z

Corner points make more sense if the region is convex. We are only concerned about corner points of linear programs.

32

When we use corner points in linear programming, it will make intuitive sense. It is not very useful to talk of corner points of non-convex regions.

A Theorem on Corner Points

z

Theorem. Every corner point of an LP is an optimal solution for some linear objective.

www.mathematica.com

33

All one has to two in two dimensions is to find a line that goes through the corner
point but doesnt touch any other feasible point.
In three dimensions, one needs to find a plane that goes through a corner point and
doesnt touch any other feasible point.
Intuitively, this is quite easy to do. However, it is not so easy to prove that it is
always possible, even though it is.
In fact, in an n-variable LP, every corner point is an optimal solution for some linear
objective.

Theorem: The feasible region of an LP is convex.

Proof illustrated. Let p1 = (1, 2). Let p2 = (3, 1). Suppose that both points satisfy one of the constraints: say ax + by c.

y
3 2 1

Then 1a + 2b c and 3a + 1b c. Suppose that p3 = (1-)p1 + p2 = (1 - )(1, 2) + (3, 1) = (1 + 2, 2 - ) p1 p3 Claim: p3 satisfies the inequality. 1a + 2b c p2 3a + 1b c (1-)

(1 + 2)a + (2 - )b c 1 2 3 4

34

To prove that a set is convex, one goes back to the definition. One shows that is p1 and p2 are both in the set and if p3 is on the line segment joining p1 and p2, then p3 must also be in the set. In the case of a linear programming feasible region, one needs to show that p3 satisfies each of the linear equalities and inequalities. The above is an outline of the proof why this is true with two variables. Indeed, the proof idea can be generalized to show that it is true regardless of the number of variables.

The proof continued

z

Every inequality is satisfied by p3. So, p3 is feasible. Equalities are also satisfied.

35

And now, its time for ..

36

Who wants a piece of candy is not stored on the web.

Summary: 2D Geometry helps guide the intuition

z z z z

2D visualization infeasibility and unboundedness Corner Points and their significance Convexity of the feasible region

37