LINEAR OPTIMIZAITION(Math3122)

Exit exam short note

Walle Tilahun1
1 MIZAN TEPI UNIVERSITY

COLLEGE OF NATURAL AND COMPUTATIONAL SCIENCES

December 21, 2023

√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Introduction Linear Programming
Key points under this course

Linear Programming Problem (LPP): The process of

optimizing a linear objective function subject to linear equality
and inequality constraints.‘.
convex: A set is convex if the line segment between any two
points in the set lies entirely within the set
Concave: A function is concave if its epigraph is a convex set.‘
Hyperplane: A flat affine subspace of one dimension less than
its ambient space.‘
Polyhedron: A bounded or unbounded subset of n-dimensional
space that is defined as the intersection of finitely many
half-spaces.‘
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

· · · Cont’d

‘
Feasible Region: The set of all feasible solutions to a linear
programming problem, defined by the intersection of the
constraints.
Decision Variable: Variables in a linear programming problem
that represent decisions to be made.‘
Basic Variable: Variables that correspond to the basic feasible
solution in a linear programming problem.‘
Slack Variable: Variables added to convert inequalities into
equations in a linear programming problem.(i.e By adding
so-called slack variables. For each of the less-than
inequalities)‘
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

· · · Cont’d

Surplus Variable: A Posetive Variables subtracted to convert

inequalities into equations in a linear programming
problem.(i.e By subtracting so-called Surplus variables. For
each of the greater-than inequalities)“
Constraint Function: The function representing the
constraints in a linear programming problem.‘
Objective Function: The function to be maximized or
minimized in a linear programming problem.‘
Non-negativity: The requirement that decision variables in a
linear programming problem must be non-negative.‘
Optimal Solution: The best feasible solution that optimizes
the objective function in a linear programming problem.‘ √
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Solution: A proposal of specific values for the decision
variables is called a solution. A solution (x1 , x2 , ..., xn ) is called
feasible if it satisfies all of the constraints. It is called optimal
if in addition it attains the desired maximum/minimum. ‘
Infeasible Solution: A solution that violates one or more
constraints in a linear programming problem.‘
Bounded Solution: A solution with finite values for decision
variables in a linear programming problem.‘
Unbounded Solution: A solution with unbounded values for
decision variables in a linear programming problem.‘
Feasible Region: The set of all feasible solutions to a linear
programming problem, defined by the intersection of the
constraints.‘
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Geometry of LP
Convex Set:Z ∈ Rn is a convex combination of two points x
and y if ∃λ ∈ [0, 1] :z = λx + (1 − λ)y.

6y    
5 7
= x2 x3 =
5 5
4
Z = 14 x1 + 13 x2 + 0x3 + 5
12 x4

2 
1
 
= x1 7
2 x4 =
1
x
2 4 6 8 10 √
walleTilahun

Figure: Representation of polyhedral set

Walle Tilahun LINEAR OPTIMIZATION
· · · Cont’d

from figure1 Z ∈ Rn is convex combination of x1 ,x2 ,x3 ,. . . ,xn

if ∃λ1 , λ2 , λ3 , . . . , λn ≥ 0 suchthat
n
X n
X
λi = 1 and Z= λi xi (1)
i=1 i=1

√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Highlighting text

In this slide, some important text will be highlighted because it’s

important. Please, don’t abuse it.
Remark
An objective function is a linear function in two or more variables
that is to be optimized (maximized or minimized).

Important theorem
A linear programming problem consists of an objective function to
be optimized subject to a system of constraints.

Examples
The constraints are a system of linear inequalities that represent
certain restrictions in the problem. √
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Highlighting text

Examples
The solution set to a system of inequalities is the region that
satisfies all inequalities in the system is called the feasible set

Examples
The optimal solution is the point that maximizes or minimizes the
objective function, and the optimal value is the maximum or
minimum value of the function.

√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Solving Linear Programming Problems – The Graphical
Method
Examples
1. Graph the system of constraints. This will give the feasible set.
2. Find each vertex (corner point) of the feasible set.
3. Substitute each vertex into the objective function to determine
which vertex optimizes the objective function.
4. State the solution to the problem.

Examples
1.An unbounded set is a set that has no bound and continues
indefinitely. A Lp problem with an unbounded set may or may not
have an optimal solution, but if there is an optimal solution, it
occurs at a corner point.
2.A bounded set is a set that has a boundary around the feasible √
walleTilahun
set. A Lp problem with a bounded set always has an optimal
solution.
Walle Tilahun LINEAR OPTIMIZATION
Fundamental Theorem of Linear Programming

maximize c1 x1 + · · · + cn xn objective function

a11 x1 + · · · + a1n xn ≤ b1 constraint function
.. (2)
subject to .
am1 x1 + · · · + amn xn ≤ bm constraint function
x1 , . . . , xn ≥ 0 non -negetivity

maximize CTX
AX ≤ B (3)
subject to
x1 , . . . , xn ≥ 0
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Highlighting text

To solve a linear programming problem, we first need to know the

Fundamental Theorem of Linear Programming:
Remark
Given that an optimal solution to a linear programming problem
exists, it must occur at a vertex of the feasible set.

Remark
If the optimal solution occurs at two adjacent vertices of the
feasible set, then the linear programming problem has infinitely
many solutions. Any point on the line segment joining the two
vertices is also a solution.
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

 x2 3x1 + 2x2 ≤ 18
6 G(0,6)
Zmax = 13x1 + 10x2
E(0,5)

D(2,4.5)
4
x1 + 4x2 ≤ 20
C(4,3)

3x1 + 4x2 ≤ 24
2 Feasible region

x1 ≥ 0
x2 ≥ 0 B(6,0) x1
A(0,0) F(8,0)
2 4 6 8

Figure: 1

from Figure:2, given above. A. Find the maximum value and the
point where the maximum occurs.
B. Find the minimum value and the point where the minimum √
walleTilahun

occurs.
maximize 13x1 + 10x2
Walle Tilahun LINEAR OPTIMIZATION
Linear Programming General Form

Linear programming is a subset of optimization problems that have

the following form:
Pn
minf (x) = i=1 ci xi s.t Ax ≥ b where A is a m x m matrix and
b ∈ Rn .

This effectively is a summarization of a series of linear inequalities

of the form: a1,1 x1 + a1,2 x2 , , a1,n xn ≥ b1 .

Remarks:
1 maxf (x) = −min[−f (x)]
2 A1 x ≥ b ⇔ (−A1 )x ≤ −b1
3 A1 x = b1 ⇔ A1 x ≤ b1 & A1 x ≥ b1 √
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

 Z = 4x1 + 11x2
3

1 Feasible region

x1 ≥ 0 x2 ≥ 0

−0.5 0.5 1 1.5 2 2.5

Figure: walle
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Example of F in two dimensions
Definition
F is bounded if contained in a finite polytope and unbounded
otherwise.
x2
4

2 x1 ≤ 2

1 x1 +x2 ≥ 1
x1 −x2 ≥ −1
0 x1
-1 0 1 2 3 4 5
x1 ≥ 0 √
walleTilahun

-1 x2 ≥ 0
Walle Tilahun LINEAR OPTIMIZATION
unbounded LPP

To solve a linear programming problem, we first need to Sketch

grahical method.
EXAMPLE :2
A problem is unbounded if it has feasible solutions with arbitrarily
large objective values. For example, consider

maximize x1 − 4x2
−2x1 + x2 ≤ −1 (5)
subject to −x1 − 2x2 ≤ −2
x1 , x2 ≥ 0.

solution
joining the two vertices is also a solution. √
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Standard form for LP problems

If a problem has no feasible solution, then the problem itself is

called infeasible. For example, consider

maximize 5x1 + 4x2

x1 + x2 ≤ 2 (6)
subject to −2x1 − 2x2 ≤ −9
x1 , x2 ≥ 0.

√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Standard form for LP problems

maximize c1 x1 + · · · + cn xn objective function

a11 x1 + · · · + a1n xn = b1 constraint function
.. (7)
subject to .
am1 x1 + · · · + amn xn = bm constraint function
x1 , . . . , xn ≥ 0 non -negetivity

minimizing objective function (if not, multiply by -1);

variables ≥ 0; (if not, substitution)
all constraints are equalities; (+/- slack/surplus variables)
bi ≥ 0. (if not, multiply by -1) √
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Linear algebra: definitions
 
v1
 v2 
column vector V ∈ Rnx1 : V =  . ;
 
 .. 
vn
VT R1xn VT
 
row vector ∈ : = v1 , v2 , . . . , vn
 
a11 a12 . . . a1n
 a21 a22 . . . a2n 
matrix A ∈ Rmxn =  .
 
.. .. .. 
 .. . . . 
am1 am2 . . . amn
n
matrix V , W ∈ R , scalar
productV .W = Σni=1 Vi .Wi = V T W = VW T
Rank of A ∈ Rmxn , ρ(A), max linearly independent
rows/columns.
√
Rank of B ∈ Rmxm invertible walleTilahun

⇐⇒ ρ(B) = m ⇐⇒ det(B) ̸= 0.,

Walle Tilahun LINEAR OPTIMIZATION
Systems of linear equations

A Systems of equations in matrix form: a system of m

equations in n variables cab be written as:Ax = b, constriant
fun A ∈ Rmxm , b ∈ Rm , x ∈ Rn ≥ 0
Theorem of Rouch´e-Capelli:
Ax = b has a unique solutions ⇐⇒ ρ(A) = ρ(A|b) = r (No of
variable)

Examples
Elementary row operations:
1. swap row i and row j;
2. multiply row i by a non-zero scalar;
3.substitute row i by row i plus times row j(α ∈ R).
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

Basic solutions

Assumptions: system AX = b, A ∈ Rmxn

⇐⇒ ρ(A) = m, m < n.
Basis of A: square submatrix with maximum rank, B ∈ Rmxm
mxm , det(B) ̸= 0
 B∈R
A = [B|N],
X
X = B , XB ∈ Rm , XN ∈ Rn−m
XN
 
X
AX = b =⇒ [B|N] B = B ∗ XB + N ∗ XN = b
XN
=⇒ B ∗ XB = b − N ∗ XN =⇒ XB = B −1 ∗ b − B −1 ∗ N ∗ XN
imposing
 XN = 0,−1weobtain a so called basic solution:
XB B b
X = =
XN O
√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

 many different basic solutions by choosing a different basis of
A
variables equal to 0 are n − m (or more: degenerate basic
solutions)
basis B gives a feasible basic solution if XB = B −1 b ≥ 0

Examples
Write the following a LPP in standard form and find Basic
solutions(if any?)

A.max 13x1 + 10x2 B.min − 13x1 − 10x2

s.t 3x1 + 4x2 ≤ 24 s.t 3x1 + 4x2 ≤ 24
x1 + 4x2 ≤ 20 x1 + 4x2 ≤ 20 (8)
3x1 + 2x2 ≤ 18 3x1 + 2x2 ≤ 18
x1 , x2 ≥ 0 x1 , x2 ≥ 0 √
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION

3x1 +4x2 +s1 = 24
x1 +4x2 + s2 = 20    
3 4 1 0 0 24
3x1 +2x2 +s3 = 18 A=1 4 0 1 0 ,b=20
3 2 0 0 1 18

√
walleTilahun

Walle Tilahun LINEAR OPTIMIZATION