Linear Programming

Fundamentals
Mohammed Brahimi
ENSIA/Intelligent Systems Enginnering

October 5, 2024
Outline
Linear Programming: A Historic and Vital Tool in OR

What is Linear Programming (LP) ?

Linear Programming applications

LP assumptions

Graphical solution of LP

Summary

2/46
Linear Programming (LP): A Historic and
Vital Tool in OR
• 1950: LP Discovery - A Landmark Moment in OR for Linear Programming.
• Linear Programming’s Impact: Saving Money in Diverse Industries, such as
education, forestry, petroleum, ...

• In a survey of Fortune 500 firms, 85% of the respondents said they had used LP

3/46
Prototype example
You run a company that produces two products: chairs and tables.

• You have 500 hours of labor, 400 m2 of wood, and 300 m2 of fabric available.

• Each chair requires 3 hours of labor, 4 m2 of wood, and 2 sm2 of fabric.

• Each table requires 4 hours of labor, 6 m2 of wood, and 3 m2 of fabric.

• You sell each chair for DZD 100 and each table for DZD 150.

4/46
Prototype example
You run a company that produces two products: chairs and tables.

• You have 500 hours of labor, 400 m2 of wood, and 300 m2 of fabric available.

• Each chair requires 3 hours of labor, 4 m2 of wood, and 2 m2 of fabric.

• Each table requires 4 hours of labor, 6 m2 of wood, and 3 m2 of fabric.

• You sell each chair for DZD 100 and each table for DZD 150.

How many chairs and tables should you produce in order to maximize
your profit?

5/46
Prototype example
• Decision variables ?

• Objective function ?

• Constraints ?

6/46
Prototype example
• Decision variables:
– Let x1 be the number of chairs to produce.
– Let x2 be the number of tables to produce.
• Objective function:
– Maximize 100x1 + 150x2
• Constraints:
– 3x1 + 4x2 ≤ 500 (labor constraint)
– 4x1 + 6x2 ≤ 400 (wood constraint)
– 2x1 + 3x2 ≤ 300 (fabric constraint)
– x1, x2 ≥ 0 (non-negativity constraint)

7/46
Prototype example
Maximize 100x1 + 150x2
Subject to 3x1 + 4x2 ≤ 500
4x1 + 6x2 ≤ 400
2x1 + 3x2 ≤ 300
x1 , x2 ≥0

What is the solution approach for this problem on a two-dimensional

plane?

8/46
What is Linear Programming (LP) ?

An optimization model is a Linear Program (or LP) if it has continuous variables, a single
linear objective function, and all constraints are linear equalities or inequalities.

9/46
What is Linear Programming (LP) ?
• Decisions variables : that we seek to determine
– x1 , x2 , ..., xn
• Linear objetive function: (goal) that we need to optimize (maximize or minimize)
– z = c∑
1 x1 + c2 x2 + ... + cn xn
– z = ni=1 ci xi
• Linear constraints: that the solution must satisfy
– a1j x1 + a2j x2 + ... + anj xn ≤ bj for j = 1...m

10/46
What is Linear Programming (LP)?
Maximize z = c1 x1 + c2 x2 + · · · + cn xn
Subject to a11 x1 + a21 x2 + · · · + an1 xn ≤ b1 ,
a12 x1 + a22 x2 + · · · + an2 xn ≤ b2 ,
..
.
a1m x1 + a2m x2 + · · · + anm xn ≤ bm ,
xi ≥ 0 (1 ≤ i ≤ n)
• Here, xi represents the decision variables, ci , aij and and bj are fixed constants
extracted from the studied problem. xi ≥ 0 represents non-negativity constraint.

11/46
What is Linear Programming (LP)?
• Any LP problem can be converted to the previous form 11.

• An equality (=) can be represented by two inequalities (≤ and ≥)

• An inequality ≥ can be converted to ≤ by multiplying by −1.

′ ′
• A negative variable xi can be replaced with xi = −xi and xi ≥ 0.
• An unrestricted variables (i.e., it can take both positive and negative values) can be
− + −
replaced by two variables xi = xi − xi and xi , xi ≥ 0.
+

12/46
LP Applications: Resource Allocation
• LP commonly used for allocating resources to activities with limited resources

• The objective is to choose levels of activities that achieve the best overall
measure of performance.

• E.g. Optimizing the scheduling of personnel and equipment in a hospital or

allocating budgets across different departments in a company

• Organizations can make the most efficient use of their resources and achieve
better performance outcomes using LP.

13/46
14/46
Example: Resource Allocation
• Context: manufacturing company has two factories and three product lines.

• The company’s objective: maximize the profit.

• Constraint 1: each factory has a limited capacity for producing each product line.

• Constraint 2: limited demand for each product line.

15/46
Example: Resource Allocation
• Context: manufacturing
Maximize 15x1 + 20x2 + 10x3
company has two factories and
Subject to:
three product lines.
2x1 + 3x2 + x3 ≤ 30 (Factory 1 capacity constraint)
• The company’s objective:
maximize the profit.
x1 + 2x2 + 4x3 ≤ 40 (Factory 2 capacity constraint)
x1 + x2 + x3 ≤ 20 (Product 1 demand constraint)
• Constraint 1: each factory has
a limited capacity for producing 2x2 + 3x3 ≤ 25 (Product 2 demand constraint)
each product line. x1 + 2x3 ≤ 15 (Product 3 demand constraint)
• Constraint 2: limited demand x1 , x2 , x3 ≥ 0 (Non-negativity constraint)
for each product line.

16/46
LP Applications
1. Production planning:
– Determine optimal mix of products to produce
– Determine quantity of each product to manufacture
– Utilize resources such as labor, materials, and machines efficiently
2. Transportation and distribution:
– Minimize transportation costs
– Satisfy demand and supply constraints
3. Portfolio optimization:
– Select best mix of assets
– Maximize returns while minimizing risks

17/46
LP Applications
4. Marketing optimization:
– Optimize advertising campaigns and promotions
– Determine best media channels, advertising frequency, and promotion offers
– Maximize sales and profits
5. Energy and environmental management:
– Optimize use of energy resources
– Reduce environmental impacts
– Determine optimal mix of energy sources and production technologies

6. Other applications ...

18/46
LP assumptions
• LP assumptions are implicit in the mathematical formulation.

• In order to use LP from a modeling perspective, it is necessary for the assumptions

about the data to be fulfilled..

Understanding LP assumptions can help evaluate how well linear

programming applies to a problem.

19/46
LP assumptions: Proportionality
• The contribution of each variable to the objective function z and functional
constraints is proportional to xj .

• This is represented by the terms cj xj in the objective function and ai jxj in the
functional constraint.

In a LP model, only variables with an exponent of 1 are allowed in the

terms of any function, including the objective function and constraints.

20/46
LP assumptions: Proportionality
Proportionality Violated

x1 Proportionality Satisfied Case 1 Case 2 Case 3

0 0 0 0 0
1 3 2 3 3
2 6 5 7 5
3 9 8 12 6
4 12 11 18 6

21/46
LP assumptions: Proportionality

22/46
LP assumptions: Additivity
• Proportionality assumption in LP model restricts exponents to 1.

• It does not forbid cross-product terms like xi xj .

• Additivity assumption in LP model rules out cross-product terms.

Every function in the model can be represented as the sum of the

separate contributions of the individual activities.

23/46
LP assumptions: Additivity
Value of the objective function z

Additivity Violated

(x1 , x2 ) Additivity Satisfied Case 1 Case 2

(1,0) 3 3 3
(0,1) 5 5 5

(1,1) 8 9 7

24/46
LP assumptions: Divisibility
• The divisibility assumption allows decision variables to have values that are not
limited to integers.

• Integer Programming can be used if the propblem requires integer decision

variables.

LP allows non-integer decision variable values, implying the possibility

to run at fractional levels, which can result in more efficient solutions
in some scenarios.

25/46
LP assumptions: Certainty
• The certainty assumption in LP models assumes that all the LP coefficients are
known fixed constants.

• In practical applications, satisfying the certainty assumption in can be challenging

as parameter values are often estimated or predicted values.

• Therefore, sensitivity analysis is crucial to identify sensitive parameters that may

affect the optimal solution.

26/46
LP assumptions: Conclusion
• Mathematical models are just approximations and simplifying assumptions.

• Reasonably high correlation between the model’s prediction and the actual
problem is sufficient for useful analysis.

• If any of the assumptions are violated significantly, alternative models are available.

• LP outperforms other complex methods in terms of its effectiveness and efficiency.

27/46
Graphical solution of LP
The graphical solution includes two steps:
1. Determination of the feasible solution space (feasible region).
2. Determination of the optimum solution among all points of the feasible region.

28/46
Determination of the feasible region

1. Feasible region: set of possible solutions that meet the problem constraints.

2. Graph inequalities: represent constraints on a coordinate plane.

3. Shade region: satisfy all constraints to determine feasible solution space.

29/46
30/46
Determination of the optimum solution

1. Optimum solution: determine best solution from feasible solution space.

2. Graph the objective function: Fix z and draw the line called isoprofit/ iso-cost

3. Shift the line: shift the iso-profit line in a parallel manner from its original position
in a direction that increases of z.

31/46
32/46
Why study the graphical solution of LP ?
• In practice, LPs can have hundreds or even thousands of variables and constraints.
• Two-variable LP may seem limited, but the graphical solution offers a key insight.
• The optimal solution of an LP is always at a corner point of the feasible region.

Key result
The graphical solution of a two-variable LP problem provides a
fundamental insight into LP optimization, which is that the optimal
solution is always associated with a corner point of the feasible
region.

33/46
Why Corner Points help to solve LP ?
• This result limits the search for the optimal solution from an infinite number of
feasible points to a finite number of corner points.
• Simplex method is based on this powerful result to solve complex LP problems.
• In two dimension case, the upper bound of the number of corner points is
(n) n(n−1)
Number of corner points ≤ 2 = 2 : where n is the number of constraints.

Key result
Intelligently navigating the corner points of a LP problem can simplify
solving very large LP problems using optimization algorithms.

34/46
Convex Sets, Extreme Points, and LP
Convex set
A set of points S is a convex set if the line segment joining any pair of
points in S is wholly contained in S.
S is convex ⇒ ∀A, B ∈ S, β ∈ [0, 1], β A + (1 − β)B ∈ S

Extreme Point (Corner point)

For any convex set S, a point P in S is an extreme point if each line
segment that lies completely in S and contains the point P has P as an
endpoint of the line segment.

35/46
Non-convex Sets Convex Sets

36/46
Convex Sets, Extreme Points, and LP
• Feasible region cannot comprise several disconnected feasible regions.

• Convexity guarantees the absence of local optima.

• No local optimum that could trap the solver.

Key result
Feasible region convexity is a critical property and it assures that if the
LP problem has an optimal solution, then at least one corner point of
the feasible region is a global optimum.

37/46
LP Special Cases
1. Alternative or multiple optimal solutions: LPs have an infinite number of optimal
solutions.

2. Infeasible LPs: LPs have no feasible solutions.

3. Unbounded LPs: There are points in the feasible region with arbitrarily large (in a
max problem) z-values.

38/46
Alternative or multiple optimal solutions
• The isoprofit intersects an entire line
segment corresponding to the
constraint.
• The decision maker can use a
secondary criterion to choose
between optimal solutions.

39/46
Unbounded LPs
• Objective function can be arbitrarily
large.
• Unbounded optimal solution should
not occur in a correctly formulated
LP.
• Modify the problem formulation to
remove the unboundedness to make
the problem solvable.

40/46
Infeasible LPs
• Feasible region to be empty (contain no points), resulting in an infeasible LP.
• Because the optimal solution to an LP is the best point in the feasible region, an
infeasible LP has no optimal solution.
• Over-constraining the problem, which makes the constraints are too tight to allow
any feasible solution.
• Introduction of incompatible constraints, such as when one constraint requires a
variable to be positive while another requires it to be negative.
• Modify the problem formulation to remove the contradictory constraint or
constraints and make the problem feasible.

41/46
Summary
• LP problem has three parts:
1. Decision variables: variable that we can control and aim to optimize by finding the best
values.
2. Objective function: a linear function of decision variables that we want to maximize or
minimize.
3. Constraints: linear inequalities or equalities that restrict the values of decision variables.

• Optimal solution to the LP is a point in the feasible region that maximizes /

minimizes the objective function.

• Verifying LP assumptions with data assesses the feasibility of using LP.

42/46
Summary
• The feasible region for any LP is a convex set, and an optimal solution to an LP is an
extreme point of the feasible region.

• To solve a max LP with two decision variables graphically, follow these steps:
1. Step 1: Graph the feasible region.
2. Step 2: Draw an isoprofit line.
3. Step 3: Move parallel to the isoprofit line in the direction of increasing z. The last point in
the feasible region that contacts an isoprofit line is an optimal solution to the LP.

• The optimal solution to a max LP with two decision variables is the extreme point
of the feasible region that lies on the highest isoprofit line.

43/46
Summary
Four cases that can occur when solving an LP:
1. Case 1: The LP has a unique solution.
– Only one point in the feasible region that maximizes or minimizes the objective function.
2. Case 2: The LP has more than one optimal solution.
– Only one point in the feasible region that maximizes or minimizes the objective function.
– The feasible region has multiple points that optimize the objective function.
– Graphically, the isoprofit line last hits an entire line segment before leaving the feasible
region.

44/46
Summary
5. Case 3: The LP is infeasible.
– No points in the feasible region.
– The LP has no feasible solution.
6. Case 4: The LP is unbounded.
– There are points in the feasible region with arbitrarily large objective function values.
– Graphically, when we move parallel to an isoprofit line in the direction of increasing z, we
never lose contact with the LP’s feasible region.

45/46
References
1. Winston, Wayne & Goldberg, Jeffrey. Operations research: applications and
algorithms.

2. Frederick S. Hillier and Gerald J. Lieberman. Introduction to Operations Research.

3. Hamdy, A. Taha. Operations Research: An Introduction

4. Robin J. Wilson. Introduction to graph theory.

46/46