Linear Programming

Linear Programming
• In a linear programming problem, there is a set of
variables, and we want to assign real values to
CISC5835, Algorithms for Big Data
them so as to
• satisfy a set of linear equations and/or linear
CIS, Fordham Univ. inequalities involving these variables, and
• maximize or minimize a given linear objective function.

Instructor: X. Zhang

Example: profit maximization LP formulation

• A boutique chocolatier has two products:
• its flagship assortment of triangular chocolates, called Pyramide,
• and the more decadent and deluxe Pyramide Nuit.

• How much of each should it produce to maximize profits?

• Every box of Pyramide has a a profit of $1.
• Every box of Nuit has a profit of $6. A linear equation of x1 and x2 defines a line in
the two-dimensional (2D) plane
• The daily demand is limited to at most 200 boxes of Pyramide and 300
A linear inequality designates a half-space (the
boxes of Nuit.
region on one side of the line)
• The current workforce can produce a total of at most 400 boxes of The set of all feasible solutions of this linear
chocolate per day. program, that is, the points (x1,x2) which satisfy
all constraints, is the intersection of five half-
• Let x1 be # of boxes of Pyramide, x2 be # of boxes of Nuit spaces.

It is a convex polygon.
3 4
 Maximize Profit Maximize Profit (cont’d)
• Find point(s) in feasible region • All points that lie on line x1 + 6x2
(shaded part) at which objective = c (for some constant c) achieve
same profit c
function (x1+6x2) is maximized.
feasible region: a • As c increases, “profit line” moves
• feasible regions decided by polygon parallel to itself, up and to the right.
linear constraints
• To maximize c: move line as far
• Note: All points on line x1 + 6x2 up as possible, while still touching
= c (for some constant c) feasible region.
achieve same profit c • Optimum solution: very last
feasible point that profit lines sees
• e.g., points (0, 200), (200, 1000/6)
and must therefore be a vertex of
lie on x1 + 6x2 = 1200, both yield
polygon.
profit $1200 (0,200)
(200,1000/6)
• so are all points in the line
segment
5 6

Maximize Profit (cont’d) Simplex Method

• All points that lie on line x1 + Simplex method: devised by George Dantzig in 1947.
6x2 = c (for some constant c) • Starts at a vertex, and repeatedly looks for an adjacent vertex
achieve same profit c (connected by an edge of the feasible region) of better objective value.

• As c increases, “profit line” • In this way it does hill-climbing on vertices of the polygon, walking
from neighbor to neighbor so as to steadily increase profit along the
moves parallel to itself, up and way.
to the right.
• Upon reaching a vertex that has no better neighbor, simplex declares
• To maximize c: move line as it to be optimal and halts.
far up as possible, while still
touching feasible region. Why does this local test imply global optimality?
considering think of profit line passing through this vertex. Since all the
• Optimum solution: very last
vertex’s neighbors lie below the line, the rest of the feasible polygon must
feasible point that profit lines also lie below this line.
sees and must therefore be a
vertex of polygon.
7 8
 A few comments A few comments
Simplex Method is a kind of hill climbing technique: • Linear programming: a special case of convex
• a mathematical optimization technique which belongs to the family of optimization.
local search. • Convex optimization: minimizing convex functions over
• It is an iterative algorithm that starts with an arbitrary solution to a convex sets.
problem, then attempts to find a better solution by incrementally • Simple ex: What if objective function is: maximize x12+x22 ?
changing a single element of the solution.
• What does the “profit” lines look like?
• If the change produces a better solution, an incremental change is
made to the new solution, repeating until no further improvements can
be found.

9 10

Simplex Algorithm: details Simplex Algorithm: detail

• Convert the problem into standard form • Convert standard form into slack form

• slack form: (N, B, A, b, c, v) N: set of non-basic variables (those on the right hand sides, and in
object functions)
B: the set of basic variables
A: matrix (ai,j)
(bi): the vector
(ci): the coefficients in object function

Basic solution: set all non-basic variables to 0, and

calculate basic variables accordingly.
11 12
 13 14

What if basic solution not feasible?

• or the problem is not feasible, or is unbounded?

15 16
 Practice Higher Dimension
• Consider the following linear program: What if there is a third and even more exclusive line of chocolates,
called Pyramide Luxe. One box of these will bring in a profit of $13.
• plot the feasible region and find optimal • Nuit and Luxe require same packaging machinery, except that Luxe
solution uses it three times as much, which imposes another constraint x2 +
• What if objective is to minimize 5x+3y? 3x3 ≤ 600

17 18

Another Problem Transport Networks

Duckwheat is produced in Kansas and Mexico and consumed in New • Given a directed graph G=(V,E), two nodes s, t in
York and California.
V (source and sink), and capacities ce on edges
• Kansas produces 15 shnupells of buckwheat and Mexico 8.
• Model some transport system (a network of oil
• New York consumes 10 shnupells and California 13.
pipelines, computer networks, …)
• Transportation costs per shnupell are $4 from Mexico to New York, $1
from Mexico to California, $2 from Kansas to New York, and $3 and • Question: How to transport as much as goods from
from Kansas to California. s to t using the network using?
Write a linear program that decides the amounts of duckwheat (in shnupells
and fractions of a shnupell) to be transported from each producer to
each consumer, so as to minimize the overall transportation cost.
A Network

19 20
 Flow in Networks Max. Flow in Networks
• A shipping scheme/plan assign fe to each edge, • Input: G=(V,E), edge capacity ce
and has following properties • Output: fe of each edge (# of var = |E|)
• 0<=fe<=ce (capacity) • Linear Programming problem
• for all nodes u except s and t, amount of flow • constraints are all linear!
entering u equals amount leave u (conserved)
• maximize: f(d,t)+f(e,t)

A Network A flow in the network: value is 7 21 A Network A flow in the network: value is 7 22

Ford-Fulkerson Alg. Summary

• Input: G=(V,E), edge capacity ce • Linear Programming: assign values to variables
• Output: fe of each edge (# of var = |E|) subject to linear constraints, with goal of
minimizing (or maximizing) a linear function
• Many problems can be formulated as LP
• if values can only be integer, then it’s a harder
problem
• e.g., Knaksack problems
• Ideas of Simplex alg.

A Network A flow in the network: value is 7 23 24