Optimizationan dcontrol

i
it

n minimum

what characteristics

are
of optimum

M tion

Cn o

CN 20 2nd derivative positive

f

is

dealing with constraint optimizationonly

we are

Leastsquareexampleaf
An icecream seller to how

find many
icecreamta
sell
ni temperature i

day
Ice cream sold i

ye day

ya aus

I to find a b minimizedistance

bluepointfline

Find a b
mign.SE
in yi
a
ni BI


General formulation

T.co

fa
aibfferentoaseem

7edmed

or.mn

min e

F
II

g omophtimFation
o i c 9mi

ein constraint

to

subject

cow IO ie inequality

constraint

1stderivative jacobean gradient

2nd derivative matrin

Feasible

I NE Rn Cical o IEE Cicala iEI

nor a setof real na that

fulfil constraints

General optimizationproblem

min N 2 2tCnz 1 sit Mknzgo

Mtnz IZ

G NEM Ea Cf G no quality constraint

2 M na

Is 42

Move add equality constraint m o

gene Mz M2 E 3

2 M

Geng 7

I 1,23

Gr m

special cases

LMI Ctu s t Ain bi o EEE

minnern
finear

Ain bi Zo Eez

programming

al'tfoam

Ag n beg

Ain n bon

Feasobleyet in this case is a

polygon

The minimum Is always althe corner

19Plauadraticpregrammin

ut G din

min
ne Rn t n t

s t Ait n bi e o EEE

q.tn Zo i


minimum optimum is not at the corner

I.ngp.alltheconstraintsarelinear.hu

eeobjetieel

canbeaguadraticeatpressoon

LPenampter

LP example

n e tonnes A

RE tonnes B

t 7000 M t 6000Nz

constts 4000Mt 300 Nz 100 000

60 m t 80Nz E 2000

Optimizationproblem

In 7000 M 6000 Rz

s t 400024 300oz I 100000

60M 8092 2000

An Eb

O s O o

L by

II

MATLAB optimization toolbox

tinprey function

Convexity

Conuexeet The sets Connex

es ant acte I

my yes

non Connex
Connex set

set

Conuexfun ation

S R is conuex

n Es

y f Cantccaly Extent C a fly

A E

O D

tog

Conue set as well

iqeme.B.dk

as objectivefunction are convex set convexfunction

fog

geobalm.m.mg

r
set
feasible

conuex optimization problems are easier to

solve
are
Integerproblems

never conuen

foes can uen

f is concave

Lecture 2 10 Jan 2029

KKT condition complete desc in booby

Definitions

subject tee EEE

Minger flat Coconino

CP

Cian Io ice

Feasible set I NE
Ral Geng 0 EEE Coca 20 EEI

Global solution

local solution

strict local solution

fooga

fan

TIdoealminimumntyI.IR
seal

global

Gfeasibeeset


EAT

whenfeasible set come en

Coca E E E are linear

CiGy i E I are concave

Pr s
tsroblems

Proof by contradiction

Affume

Gender optimizationproblem n local solution

There exists at s t Cn
f Cray e f

n n

Defining 2 2 nd t l a n a E Lo I

FEI f Can team

t

SafCn tf a
fend

faux Cc a
floral tf asfcm

f tE f feel final

add subtractfEn

LO

in ne
fo l
fenix

jangle nixisglobal
meaning
solution
2 in the

that there is apoint neighbourhood

which the value ofobjective

of n
forlesser see it isn't
is even a local solution
function

Of book

K KT conditions 12.1 to 12.3

fans city

Thtifeier

KKTconditions FqLCn H O

g n o ti EE

Ci Ent Hi EI
20

dot 20 C Hi I

Ai qCn o to E EUI

Usefulfact

Gradient is orthogonal to lines Cleuellines

contour

MgkD.d fem nine

descent

direction
d flat 22,1

Will I

Given a direction d Then

CO ed
if
a descentdirection

Dfid is

acid O E d

Is feasible
a direction for day
o

Mccoy

cent 0


acid 20 e
de a
feasible detection for clonko

conto

g
55,420

Enample 12.1 12.3

Equality constraint

Casey

min Netz s t M't ne 2 0

n
clearly emiston feasible

n a descent direction

arnica
of

µ d'T Ct
Fomedonsuatinthint

O
Ff

as i m
E.Iqconse
qq.ee

d annt.a
if.eocmt.aco

oycqqqy

qq.gg ig aoeeatoee.eonf

Peony n there are no feasible

descent direction

FfCn X acCn

Caseii Inequality constraint

min oytz Sit 2 92 not 20

no clearly exists

feasiseeoggceeef.eu

there exists feasible

n

descent direction

d cons O

dat Afca O

n nofeasible descent direction

final D actual 4 20

complementary condition A cCn O

Jttn iEIIG.cn

Active set EU o

o.EE

Set offeasible directions deipnfdTGCH

fcng dTFGC.nl oiEIBAl

CaseI

min netz set 9 Coy 2 M2 NE 70

Ca NEO


n U

n fill

m s Aln Of nil ft Cnn 2

F buy R2 F Cn did'T If

not e
AGE I 22g

F Cn d DT and DT
f 20

Zo

f 9

madeseenffedngiiffYIL.gtxixec.cn

ai.ia zo

II Lectures optimalitycondaforconstrainedoptimization

Gradient

man o

Eating

itssymmetric foot
Qf_

dream

uessoanofanlayangian
g
ftp.P

mm

uedzess

F
Ballon string with constraints

of

minimize M2tnEtnz

Mutefliers can bet on

quality

inequality canneetbe ne

KKT conditions Theoremal First order necessarycondition

KKI min 2mHz sit Mtn 1 0

fix

CnH fans AC by

A Gretna 1


stationarity

0.8 1 49 t m tic

0
22 7 0 2
42

21

Hfimatsoeity

netz I

1 4 4,3

94 42

2T

µ kgn

KK 2

min min Set 9 Ca 2 m NE IO

Cn 2420

Cn 220

L mm A G m ni Tim Az Nz

KKI IL n t d I

dm

IL K t 2nd Az

Dnc

2 ni NE I 0

zo
primary feasibility

Na 20

1 20

20 dual feasibility

Az

20

Az

ti Cz m nil o

complementarity

Az ni o cand n

Iz Cud O

Case I

Assume 9 O

we are not on 2 ne NE butisomewhere

in the feasible


Nz 7 0 Nz EO 12 0

my 93 0 n
0,13 0

other coneen as well

checking similarly

X D Kkt

Oo
ooo
fulfills

Many but thats not a minimum ie a

8 is

neeta mon even when It KKT as KKT

fulfils
candor are

only necessary

Assuming'd 0 doesn't help

Casey Assume nitni 2

M so is no solution 42 0

Nz o ll 43 0

Nz I

2222 0

3k 2

Mtn z

Nz 3

N 2

43 43

ne


Solvability

Multiplier are shadow prices

densitiuityandhagrayemulk.pk Baak

Given n H Acne

Relan constraint 0 i Etten n I

Ci Cn 20 film Z E E 70

Assume that Acnex unchanged

Cnt Ces f Cn

f FfCn nice n Gaffney

from KKT conditions 17L

Tf E diThi 0

so

Tf Shiki

substituting

E djeg.cn nice n

jEAn


cjcnxcej cjcnxyiagtln.ee n

Aj Cj nice Cj cn
irontaomi

j µn

o 0

exceptfor jai

when E

XiE Cj n CED

for o

D
df

Tangential

Lecce Linear independent constraint qualification

doesn'thold
If 4cg

we not

might get Tf
value lagrange n
unique

of
daesmtaed
qzy

De

qs criticalcane Refervideo lecture

17th Jan 19

17thJan 19

Lecture 4: Linear programming

Brief recap: Linear algebra, Real analysis

Linear programming, formulation, standard form

KKT conditions for linear programming

Dual problem, weak&strong duality

Reference: N&W Ch.13.1 (also: 12.9)

(Linear algebra: App A.1, Real analysis: App A.2)

Man singular matchin At exists eigenvalues A

of 0

www.ntnu.no TTK4135 Optimization and control


F na
e4

LinearProgramming

linear

Recap Algebra

Norms

comma
row

Fundamental theorem of linear algebra

Condiber

Well conditioned

Ill conditioned if the matronis almostsingular

can bedetermined

use condition
2

Types of constrained optimization problems

Linear programming

Convex problem

Feasible set polyhedron

Quadratic programming

Convex problem if

Feasible set polyhedron

Nonlinear programming

In general non-convex!

www.ntnu.no TTK4135 Optimization and control

3

www.ntnu.no TTK4135 Optimization and control

4
>> help norm

norm Matrix or vector norm.

Norms norm(X,2) returns the 2-norm of X.
norm(X) is the same as norm(X,2).

norm(X,1) returns the 1-norm of X.
norm(X,Inf) returns the infinity norm of X.

Vector norms: Hassomethingtadawithsizeaf norm(X,'fro') returns the Frobenius norm of X.

nectar

Tatar
www.l

fee'Emitti

Matrix norms (same definition as vector norm):

largestrow
sum

Eminem

af

r
largesteigenvalue
J

largestcolumn
sum

www.ntnu.no TTK4135 Optimization and control

5

Fundamental theorem of linear algebra

pythinkasanoperator

projectionof

e ninrange

space

D mapsies

mapsto

prajectionofn

innullspace

Domain Range

www.ntnu.no TTK4135 Optimization and control

6

Condition number in eigenvalues eigenvalues aremore
spread if

spread longerna

lessspreadeigenvalues small

Well-conditioned: A small perturbation gives small changes

Ill-conditioned: A small perturbation gives large changes

use
we scaling

suchcases
for

Condition number:
>> help cond
cond Condition number with respect to inversion.

cond(X) returns the 2-norm condition number (the

e
ratio of the largest singular value of X to the smallest).
Large condition numbers indicate a nearly singular

Kappa
A small condition number (say, 1-100) implies the matrix.

matrix is well-conditioned, a large condition number cond(X,P) returns the condition number of X in P-norm:

(say, >10 000) implies the matrix is ill-conditioned.
The condition number (2-norm) of the above matrices NORM(X,P) * NORM(INV(X),P).

are 6.9 and 400 000, respectively. where P = 1, 2, inf, or 'fro'.

www.ntnu.no TTK4135 Optimization and control



Matrinfactor ization

solve linear egn system

A n b

newer use in verse in this course

1 Lo decomposition Gaussianelimination

For generalmatrinA

Symmetric pd matronA Chalesky decomposition

indefinite matchin LDL factorization

Symon

other important factorizations

ORdecomposition

EigenvalueEspectraydecomposition

singular valued decomposition

Someanal Gradient hessian

ysis

Directional derivative

Lipschitz continuity

Mean value theorem

x x

Matrix factorizations

Solve linear equation system:

In ac ice, ne e e he in e e. I i inefficien and n able .

Instead, use matrix factorizations:

General matrix A: Use LU-decomposition (Gaussian elimination)

Due to triangular structure of L and U, we easily solve the two linear systems by substitution

Symmetric pd matrix A: Use Cholesky decomposition

Half the cost of LU. Solve system as for LU.

For symmetric, indefinite matrices: Use LDL-factorization instead (book: )

Generally, algorithms use permutations:

Other important factorizations value decourfon

A = QR: Finds orthogonal basis for nullspace of A
fornon
peigen square
matrices

Eigenvalue (spectral) decomposition, singular value decomposition

www.ntnu.no TTK4135 Optimization and control



ed CsuD

This is entension
of eigenvalue decomposition for non

square matrices

AE RmXa

m n

Uµµ

orthogonal matrices

diagonal G

S on

Tite Z re

Some analysis (A.2)

Gradient and Hessian: For a function , , the

gradient and Hessian are

Directional derivative: The directional derivative of is

Also valid when is not continuously differentiable. When ,

Lipschitz continuity: A function is Lipschitz continuous in a

neighborhood , if

www.ntnu.no TTK4135 Optimization and control

9

Mean value theorem

For , , we have

wikipedia.org

www.ntnu.no TTK4135 Optimization and control



CHI Linearfroghlmmitf

thisdoesn't
adding make

differenceas itonly
any
changethevalue offunckat

Minnan fCn CTN d notthepoint

ftp.izgdeyba.intbut

Bothconstraints

est Ci Catan be O EE E as wellas

objfune
are linear

Gr
ain b IO ie I

Standard foam LP

An b

minne In it primal

NIO

endaeedform

Transformationtaste

we have min Ctn est An Eb

Say

Trick1 slackvariables

min En s t Antz b 220

Tricks splitting or in positive andnegativepart

N ont n ktzo n zo
n ont w

It 3 min CT at n s t A fat a tz b
at

µ at 20 a 20 2.20

min et E o

Int

s t

A A
b
Tat

Yutzy 20

10

LP example: Farming

A farmer wants to grow apples (A)

and bananas (B)

He has a field of size 100 000 m2

Growing 1 tonne of A requires an

area of 4 000 m2, growing 1 tonne

of B requires an area of 3 000 m2

A requires 60 kg fertilizer per tonne grown, B requires 80 kg

fertilizer per tonne grown

The profit for A is 7000 per tonne (including fertilizer cost), the

profit for B is 6000 per tonne (including fertilizer cost)

The farmer can legally use up to 2000 kg of fertilizer

He wants to maximize his profits

www.ntnu.no TTK4135 Optimization and control

11

Farming example: Geometric

interpretation and solution

www.ntnu.no TTK4135 Optimization and control



Introducing slack variables

mini 7000 M 6000Nz OZ t Oe Zz

S't 100 000

4000M t 30 Nz t 2

Gong t 80 Nz t Zz 2 000

M 20 Nz O Z Z O Zz IO

LP sceln it has two cases feasibleset

special

infeasible An _b has no solution Gre 0

unbounded enist sequence nter sit Ink so

Nsisump tion A C 112mm man An b

I
if not then
may endup

with infeasible or

unbounded Solh

which means we have less rice

of linear gas than thenee

of variables seethat we have a region to teach forthe

optimalsolution

Rankof matron A m E n

constraints

of
max no no
of
thatueecanhaueuaewabl.ee
independent

12

KKT conditions (Theorem 12.1)

Lagrangian:

KKT-conditions (First-order necessary conditions): If is a

local solution and LICQ holds, then there exist such that

(stationarity)

(primal feasibility)

(dual feasibility)

(complementarity condition/

complementary slackness)

Either or

www.ntnu.no TTK4135 Optimization and control

13

Linear programming, standard form and KKT

LP, standard form:

Lagrangian:

I
can betueor ue d

KKT-conditions (necessary and sufficient for LP): Hasteebetue

Lagrangemultiplier

TnLCn X 0 9 forinequality

Xs Lagrangemultiplier of equalities

s Lagrangemultiplier forinequalities
www.ntnu.no TTK4135 Optimization and control


KKTforLP

Lcn A En XT An b Stn

KII 17mL C ATA six

An b

n 20

8 20

Sini O i4 r n

KKT
for LPs are
necessary
and sufficient

Preff Assume E tulfilsKkT

Tisa

feasibleft
cTn ATf ts n f c Atx ts
IEEE
login
feandtfmbi.es

HEE n

ts

b 20

I bTy Ink

Explanations

i
T T
T b
A TA Ifor Enke ATx
IA

CATAK n
toptimal

Atb bTX i be X TAn tskTn

e
I I btl
cImmunity
candor
ofka
scalar 1

Ad 1

changing

rom column That is Ctn ZcTn See n is aglobal

doesn't makes

solution

difference

Dual problem

Max BTX ft ATTIC dual sedLdefined
Gotra

on Rm

XERM

mEc

Rewrite for KKT go.fmaiFREE
zoaerm.in Rs bIIaec

BTX sit c ATA 20

mini

I X X bit nT c Atl

Is

modified

optimization
lagrangian

variable lagrangian

multifier

interchanged than

regular

17 L x btAn

KKifal n O

six c AT 20

2 20

nikcc ATMi O.it in

sik

SK GATAK

Primal and dual has same KKT conditions

I feasible

Weakduality for primal

5 feasible for dual

c.tn z cTn btl I STI

nobjectivefunction
values

in

omfg dualitygap

BTI

St Theorem 13 I
y

or dual hasaffinite solution so has the other

1
If primal
cTn bTx

and the objedi heegu_al_rsi.e

If primal or dual unbounded the other is infeasible

3 dual of a dual is primal

Ytitygopiusedtfno.ba

aae.yueaeeaeaupsaue.on

is convoyed or not it should be 0

Praef bycontradiction
suppose primal is unbounded Enist feasible s t

CT NH N


suppose dual is feasible enist I

s E ATI Ec

ATI E C

XT A new E calKI

b TI E CT Nlt a

contradiction

sensitivity

make small perturbation in bi bi bite

them

Ennen CTnold I Eti

14

Linear programming solutions

The feasible set is a polytope (a convex

set with flat faces)

The objective function contours are planar

obj.ms as itis
alinearfunC

Three possible cases for solutions:

No solutions: Feasible set is empty

or problem is unbounded 52

0

One solution: A vertex

Infinite number of solutions:

An edge i a l i n

a LPlies onthe uerten

Solution of

Two ofsolvingDPproblems course

method

1 Simplexmethod Partofthis

Interiorpointmethod chic notpart

www.ntnu.no
ofthiscourse TTK4135 Optimization and control
1

Lecture 5: Solving LPs the simplex method

Brief recap previous lecture

The geometry of the feasible set

Basic feasible points, The fundamental theorem of

linear programming

The simplex method

Example 13.1

Some implementation issues

Reference: N&W Ch.13.2-13.3, also 13.4-13.5

www.ntnu.no TTK4135 Optimization and control

2

Linear programming, standard form and KKT: recap

LP:

LP, standard form:

Lagrangian:

KKT-conditions (LPs: necessary and sufficient for optimality):

www.ntnu.no TTK4135 Optimization and control

3

Duality

Primal problem Dual problem

Identical KKT conditions!

Equal optimal value:

Weak duality:

Duality gap:

Strong duality (Thm 13.1):

i) If primal or dual has finite solution, both are equal

ii) If primal or dual is unbounded, the other is infeasible

www.ntnu.no TTK4135 Optimization and control

4

LP: Geometry of the feasible set

Basic optimal point (BOP)

Basic feasible point (BFP)

(if they exist)

In general, the BFP has

at most m non-zero

components

Inspired by drawings by Miguel A. Carreira-Perpinan

www.ntnu.no TTK4135 Optimization and control



22 Jan 19 Lecture 5

easibleset

LPiGeometryoff

Basicfeasiblepain tBEP Axsume A

fullhour rank
BEP

n ix
n is
if

feasible

There is indenset Bcn

an C I in sit
LP min Ctn

mean
a
B En contains m indices

at a
AEI TETI

Rankofit m if Ben ni O

B Ai Egging is non

singular

column iof An

Mxn

basis matrix

B is called basis the LP

n
Ahasfull rank
as
for
rank B am

If

n 5 me 5

Ef
i 3
It

A n b

B Cn 2,3 NCH 1 4,5

Simpler methodgenerates iterates that are BEP and

converges to a solution if

i there are BEP

ie one of them is a solution BOP

Fundamental theorem for LP

13.2

For standard

If Pig feasible point

there is a
LPhas solutions
there is a BEP
solution is a Bop

ii If one
Iii P as feasible and bounded there is a solution
If

Theoremisis

All vertices
of the feasible polytope
meRn Ane b n Zo
I

are BEPs and all Bets are vertices

theorem If LP is bounded and non degenerate then the

simple BOP

A LP is degenerate there is a BEP

Degeneracy if

with me o
for EC Cng

LP KKT conditions (necessary&sufficient)

Simplex method iterates BFPs until one that fulfills KKT is found.

Each step is a move from a vertex to a neighboring vertex (one

change in the basis), that decreases the objective

www.ntnu.no TTK4135 Optimization and control



Csetofenduc

simpler definitions B Basey

B Basis matrix

N E in 1B

NB Basicfeasiblepoint

BmIm AiTic N LAIT ie r

B ni Tiegs nm no nm O from

icy

definitionof Bep

B ISM splits C matrices also in

points liken
feasibleand nonfeasible

CB Gee

Procedure 13.1

0n the simpler algorithm

Gillen N A BEP citingAmaterinascombination

afmaff.gg

KKT 2 An _Boyz Mane BRB b

KKT 3 1 20 my O

KKT 5 NTS NBTspot

NmTSm o by choosing o
in
weassumehereAhasfuel

KIT ATy ts C Of nwo

An b t.TK s7wmkohfIem

ears o
20 sofromKent2 Bnpgb incaseof

n.fiEoi a.z Eb
nrs
risende

enEoiKkT
ns

l StATd
CSBtBTd

Cg c

Fm Bmt x Gen smtmtxe.cn e

Incl fromusingKKT5,550 g Using d in

TIG 20

stimeconsuming
she_Gu out

A LBTTCB KkT4


Gm 20 then KKT Ok is BOP

If n

If not

g one inden t Sq co

Choose g s

increase
ng along An b until another

component Np becomes Zera

JV B Repeat with new BEP

M 9 The most timeconsumingpart seducingmaterinegns Bobet

and Btg CB

Alsa as thesematrices are

veryhugemaybeof order cooee becauseof

the size

of optimizationproblems therefore never use inverseoffroad

use to decompositoonfesplitting matronintolowerRapper

always matow

Boyz b triangle

B LU NB b

Unpeg

Ly b
µ t

which are much

easier to solve

and bysubstitution

section

Readbgooke simplex

on

www.ntnu.no TTK4135 Optimization and control

7

Check KKT-conditions for BFP

Given BFP , and corresponding basis . Define

Partition , and :

KKT conditions

KKT-2: (since x is BFP)

KKT-3: (since x is BFP)

KKT-5: if we choose

KKT-1:

KKT-4: Is ?

If , then the BFP fulfills KKT and is a solution

If not, change basis, and try again

E.g. pick smallest element of sN (index q), increase xq along Ax=b until xp becomes zero.

Move q from to , and p from to . This guarantees decrease of objective, and no

c cling (if non-degenerate).

www.ntnu.no TTK4135 Optimization and control

8

Example 13.1 figure

www.ntnu.no TTK4135 Optimization and control



Enample B min 4M Zz

NEIR2

St Mtn E5

jIf

conuoa.gtaseo.name

Mtnztnz 5

204 4221 24 8

M 20 220 20,2420

f I og't

First iteration Eter2 iter3

13 3,1313

Be 3,4 V 1,2 N 4,23 4,2

N l343

am my 8

B B'b
f if

no

Csis's
I

g i

sn.cm mix
f fi I

I E'd

m
t

sm

EIisniE.gg
I

is

c CS 4 is optimal
pick q

Let I enter B

Increase m An b

along Ng becomes zero

first

Let p 4 leave and enter IV

Simplex in 3D

wikipedia.org

www.ntnu.no TTK4135 Optimization and control

10

Linear algebra LU factorization

Two linear systems must be solved in each iteration:

(to find the direction to check when inreasing )

We also had . Since is not needed in the iterations, e don t

need to solve this (apart from in the final iteration)

This is the major work per iteration of simplex, efficiency is important!

B is a general, non-singular matrix

Guaranteed a solution to the linear systems

LU factorization is the appropriate method to use (same for both systems)

Don t use matri in ersion!

In each step of Simplex method, one column of B is replaced:

Can update ( maintain ) the LU factori ation of B in a smart and efficient fashion

No need to do a new LU factorization in each step, save time!

www.ntnu.no TTK4135 Optimization and control

11

Other practical implementation issues (Ch. 13.5)

Selection of entering inde q

Dant ig s rule: Select the index of the most negative element in sN

Other rules have proved to be more efficient in practice

Handling of degenerate bases/degenerate steps (when a positive xq is

not possible)

If no degeneracy, each step leads to decrease in objective and convergence in finite number of

iterations is guaranteed (Theorem 13.4)

Degenerate steps lead to no decrease in objective. Not necessarily a problem, but can lead to cycling

(we end up in the same basis as before)

Practical algorithms uses perturbation strategies to avoid this

Starting the simplex method

We assumed an initial BFP available but finding this is as difficult as solving the LP

Normally, simplex algorithms have two phases:

Phase I: Find BFP

Phase II: Solve LP

Phase I: Design other LP with trivial initial BFP, and whose solution is BFP for original problem

Presolving (Ch. 13.7)
Reducing the size of the problem before solving, by various tricks to eliminate variables and

constraints. Size reduction can be huge. Can also detect infeasibility.

www.ntnu.no TTK4135 Optimization and control



8 Beak

PhaseGm

Haneda I find a startingpoint

forsimpler

min 1 i 2 e 1,1 1T

2E RM NER

Ant EZ b E

min n
FET diag Ejj

220 Ejj I if bj70

if byLO

N O

f lbjI j I m

Mom BE 1,2 n ntm

mm

N has ntm n n indices

corresponding to
basicfeasiblepoint

Run simpler andfind solution which minimumvalue

of objective fund etz

min eTz o Ee L l l 1 2 0

2 0 minhappenswhen

2 0

An tEz And b

thisoptimalpaintforthisproblemcan now

beused as basic feasiblepointfor

the

originalproblem

12

Simplex an active set method

Complexity:

Typically, at most 2m to 3m iterations

Worst case: All vertices must be visited (exponential complexity in n)

Compare interior point method: Guaranteed polynomial complexity, but in

practice hard to beat simplex on many problems

Active set methods (such as simplex method):

Maintains explicitly an estimate of the set of inequality constraints that are

active at the solution (the set for the simplex method)

Makes small changes to the set in each iteration (a single index in simplex)

Next week: Active set method for QP

www.ntnu.no TTK4135 Optimization and control

1

24th jah.IO

Lecture 6: Quadratic programming

Quadratic programming; convex and non-convex QPs

Equality constrained QPs

Building block of general QP solvers (next time)

Reference: N&W Ch.15.3-15.5, 16.1-2,4-5

www.ntnu.no TTK4135 Optimization and control

2

Types of constrained optimization problems

Linear programming

Convex problem

Feasible set polyhedron

Quadratic programming

Convex problem if

Feasible set polyhedron

Nonlinear programming

In general non-convex!

www.ntnu.no TTK4135 Optimization and control

3

Last time: The simplex method for LP

The Simplex algorithm

The feasible set of LPs are (convex) polytopes

LP solution is a vertex ( co ne ) of the feasible set

Simplex works by going from vertex to neighbouring

e e ( hich a e all ba ic fea ible poin , BFP) in ch

a manner that the objective decreases in each iteration.

In each iteration, we solve a linear system to find which

component in the ba i ( e of no ac i e con ain ) e ho ld change

Almost guaranteed convergence (if LP not unbounded or infeasible)

Complexity:

Typically, at most 2m to 3m iterations

Worst case: All vertices must be visited (exponential complexity in n)

Active set methods (such as simplex method):

Maintains explicitly an estimate of the set of inequality constraints that are

active at the solution (the set for the simplex method)

Makes small changes to the set in each iteration (a single index in simplex)

Today, and next lecture: Active set method for QP

www.ntnu.no TTK4135 Optimization and control



because
adf.ffaegmentgtfo.ws

quadrateprogramming
NTGu In quadfun came

Minh fan Is Hessian

deenitheed
that

ofobjf.me

s t CiEng a
ain bi o EE E caremtraint
linear

Ci Cn Ait n bz Zo EI

Feasible set is convex all constraints are linear

Objective function G o positivedefinite strictlycome

Connex
of G20 G20 positivesemidefinite 9Pconnex
ifGao wegetup

non convex if Giesendefinite

uadhaticipregramimiy_

Linear quadratic control Mpc

sap sequentialquadraticprogramming usedto transform non

linear toquadraticform

Why are we interested in QPs?

Three (main) reasons:

I i he mo ea nonlinea p og amming p oblem ( o pecial

that it is given a separate name; quadratic programming)

ea : efficien algo i hm e i , e peciall fo con e QP

The QP i he ba ic b ilding block of SQP ( e en ial ad a ic

p og amming ), a common method for solving general nonlinear

programs

Topic in end of course (N&W Ch. 18)

QPs are very much used in control, especially as solvers in what

i called MPC ( Model P edic i e Con ol )

Topic in a few weeks

Al o ed in finance ( Portifolio op imi a ion ), ome pe of Machine

Lea ning/ eg e ion p oblem , con ol alloca ion, economic ,

www.ntnu.no TTK4135 Optimization and control

5

Convex QP

Feasible set is (convex) polytope

Objective is quadratic function, which can be non-convex

(concave or indefinite), convex or strictly convex

www.ntnu.no TTK4135 Optimization and control

6

QP example: Farming

example with changing prices

A farmer wants to grow apples (A)

and bananas (B)

He has a field of size 100 000 m2

Growing 1 tonne of A requires an

area of 4 000 m2, growing 1 tonne

of B requires an area of 3 000 m2

A requires 60 kg fertilizer per tonne grown, B requires 80 kg

fertilizer per tonne grown

The profit for A is (7000 200 ) per tonne (including fertilizer

cost), the profit for B is (6000 140 ) per tonne (including

fertilizer cost)

The farmer can legally use up to 2000 kg of fertilizer

He wants to maximize his profits

www.ntnu.no TTK4135 Optimization and control



Farming objectivefunction
6000 1402 Nz

maxmn 7000 200M Mt

minzynz 200242 140222 7000M 60002

minna kfmn.IT 4oooz0goJfnIftGooo 6oodI

LP farming example: Geometric

interpretation and solution

www.ntnu.no TTK4135 Optimization and control

8

QP farming example: Geometric

interpretation and solution

SolutionofQPcan beinside

feasibleregion

Solutionmightas well lie on

a corner

www.ntnu.no TTK4135 Optimization and control



Equality-Constrained QP

• Problem:

1 "

minJ $ 5$ + 7 " $

-∈ℝ 2

8. :. : *$ = +

• KKT conditions are:∇- ℒ $ ∗ , '∗ = 0, *$ = +

• Matrix form:

5 −*" $ ∗ = −7

* 0 '∗ +

– Combining KKT-1 (stationarity) and KKT-2 (Primal feasibility)

12

Equality constraint Cfpo

FQP
minnern nTGntcTn Sit An b

E of

AE 112mm man hank CA m

KK T

Lcn t nTG n Ctn AT An b

In Lcn d ATX

Izcan Gn Gta c
to be

we assume G symmetricalways

i n a I HEH

An b

mat

q oµ

Karmann

fg

KKT conditions (Theorem 12.1)

Lagrangian:

KKT-conditions (First-order necessary conditions): If is a

local solution and LICQ holds, then there exist such that

(stationarity)

(primal feasibility)

(dual feasibility)

(complementarity condition/

complementary slackness)

Either or

(strict complimentarity: Only one of them is zero)

www.ntnu.no TTK4135 Optimization and control

10

Example 16.2

Note symmetry of G.

Always possible!

Flings
makeGSymon
>> G = [6 2 1; 2 5 2; 1 2 4]; c = [-8; -3; -3]; A = [1 0 1;0 1 1]; b = [3;0];

>> K = [G, -A'; A, zeros(2,2)];

>> K\[-c;b] % X = A\B is the solution to the equation A*X = B

ans = usesLUdecomposition in this case

2.0000

-1.0000

1.0000
3.0000

-2.0000

www.ntnu.no TTK4135 Optimization and control



1 "
min $ 5$ + 7 " $

Theorem 16.2 -∈ℝ 2

J

8. :. : *$ = +

"

Let * have full row-rank, and the reduced Hessian = 5= > 0

(positive-definite). Then the vector $ ∗ satisfying:

5 −*" $ ∗ = −7

' ∗ +

* 0

Is the unique global solution of the QP.

Also known as Lemma 16.3 allows us to go from current guess to best guess.

If reduced Hessian positive semi-definite, then $ ∗ is a local minimizer.

13

Direct Solution of KKT

5 *" −? = 7 + 5$

* 0 ' *$ − +

• Factoring the full KKT system

– Good ol’ Gaussian Elimination

• Schur-Complement method

– Requires 5 nonsingular

• Null-Space Method

– What we’ve prepared for

14

cheuyewmina.tt

ntP

n ntp

l I s.is

am t notes

Il I t y

e.mn

E I III

Eatrin

11

Fundamental theorem of linear algebra

www.ntnu.no TTK4135 Optimization and control

12

Example 16.2

Note symmetry of G.

Always possible!

>> G = [6 2 1; 2 5 2; 1 2 4]; c = [-8; -3; -3]; A = [1 0 1;0 1 1]; b = [3;0];

>> K = [G, -A'; A, zeros(2,2)];

>> K\[-c;b] % X = A\B is the solution to the equation A*X = B

ans = >> [Q,R,P] = qr(A')

2.0000 Q =

-1.0000

1.0000 -0.7071 0.4082 -0.5774
0 -0.8165 -0.5774

3.0000
-2.0000 -0.7071 -0.4082 0.5774

R =

-1.4142 -0.7071
0 -1.2247

0 0

P =

1 0

0 1

www.ntnu.no TTK4135 Optimization and control



min CntpTGCmp A Ekman

s t A Cmp b AP 0

Mall A WER l Auto

tt has

kemma l6 3
full rank IG so

then non
singular

Ga Aoi

Ian

A we 0

we 52W

QuetATu O

GZU AT neo

uteru Prooflemma16.3

directions in the millspace of

quality constraints reduced

hessian

13

Direct solutions of KKT system (16.2)

Full space:

Use LU

Or better: Since KKT-matrix is symmetric, use LDL-method

Cholesky cannot be used, since KKT-matrix is indefinite for m 1

Reduced space, efficient if n-m ≪ n:

Solve two much smaller systems using LU and Cholesky

both with complexity that scales with n3

Main complexity is calculating basis for nullspace. Usual method is using QR.

Alternative to direct methods: Iterative methods (16.3)

For very large systems, can be parallelized

www.ntnu.no TTK4135 Optimization and control



Null-Space method minJ

`∈ℝ 2
$ + ? " 5($ + ?) + 7 " ($ + ?)

8. :. : *($ + ?) = +

• * ∈ ℝP×R , m ≤ D, rank A = m, have candidate $

• = is basis for Null(*), = ∈ ℝR×(RZP)

• O is basis for Range(*" ), Y ∈ ℝR×P

• Trying to solve:

5 *" −? = 7 + 5$
∗

* 0 ' *$ − +

Ap A n b

AYpg AzPz An b

Ay Py e Amb ta

15

Gp t Atl

CTG n

G YPy G 2kt ATX Ct Gn

IG Ypg zTGzpz zt et Gm

ET Gz Pz 2T GyPy 2T CtGm Cq

moue solve

for Pg
solve

for Pz y
2 come
for ardecompa
Cray

aka ntp

Concluding remarks

• Why solve equality QP? Active set strategy!

• For EQP, if KKT matrix nonsingular, invert and done!

• More advanced scenario: Use direct solution methods!

• Way too many variables? Use iterative methods to solve!

• Why care about QPs? Control!

16

1
29 1 19

Lecture 7: Quadratic programming

Recap last time EQPs

Active set method for solving QPs

For medium-sized problems for large problems, interior point

methods may be faster

Example 16.4

Reference: N&W Ch.15.3-15.5, 16.1-2,4-5

www.ntnu.no TTK4135 Optimization and control

2

Quadratic programming

(solving quadratic programs, QPs)

I

wesemidefiniteforMPC

Feasible set convex (as for LPs)

We say that the QP is (strictly) convex if G ≥ (>) 0

www.ntnu.no TTK4135 Optimization and control

3

Important special case:

Equality-constrained QP (EQP)

Basic assumption:

A full row rank

KKT-conditions (KKT system, KKT matrix):

Solvable when (columns of Z basis for nullspace of A):

tree

ifobjfund

How to solve KKT system (KKT matrix indefinite, but symmetric):

Full-space: Symmetric indefinite (LDL) factorization:

Reduced space: Use Ax=b to eliminate m variables. Requires computation

of Z, basis for nullspace of A, which can be costly. Reduced space method

can be faster than full-space if n-m ≪ n.

www.ntnu.no TTK4135 Optimization and control

 R dikedspac.ee

KKT conditions (Theorem 12.1)

Lagrangian:

KKT-conditions (First-order necessary conditions): If is a

local solution and LICQ holds, then there exist such that

(stationarity)

(primal feasibility)

(dual feasibility)

(complementarity condition/

complementary slackness)

Either or

(strict complimentarity: Only one of them is zero)

www.ntnu.no TTK4135 Optimization and control



IP En

minner IgnTGn

s t Ain bi O EEE

ain bi ZO EE I

LCH lqnTGntctn feuetica.tn bi

Kat Gn t C t E Xiao O CD

E EUI

aint bj O EEE G

aint bi 20 LEI

Xi 20 i C I 4

aiTn bi O IGI E

1 aint

Acey EE EUI bi O

lp Reformulation of KKT

Gn c E
iexacnxytiai o

foriEVtcn1 Xi o complimentary

asaiTn boforale
inactivesets

Ait bi O I Eftcay

aint bi 20 iEIlACn

Ai IO IE IN Acn

the a satisfies Kat n is a global solution

and G20

ie KK T cand n are well

sufficient as

Preeaf Assume n feasible at n

First rate n n Gn 4 thenYE tiGtiai

EE

XiaoCn n t E ti ait n n
Ee

iEAcn

ing thatarepartof
aerueset

Xi Tn aetna

Ee t.ca nedicaIingiInIo

O 7O

qty Cn n JTG n t n ng t
n t nextCn nd

Iq

nextGmt t CTW t Cn n JTGCn n t na Gnkt

o
g Gra G20

20 20

I qcn

Ado IG 2 SO 2 basis
for nullspace of active constraints

ne strict unique local solution

indefinite maybe several local solutions hard tofind

global

Degeneracy a strict complementarity does not hold

b at theoptimum

linearly dependent constraints

Mustbesafeguarded
5

Nonconvex QP

www.ntnu.no TTK4135 Optimization and control

6

Degeneracy

www.ntnu.no TTK4135 Optimization and control

7

General QP problem

Lagrangian

KKT conditions
General:

Defined via active set:

www.ntnu.no TTK4135 Optimization and control



Observatoire If ACat known then 9PMeduces to an E P

Min G n f CT

Ent n

At ain bi i C A Cna

summa

Onestep
of the active set algorithm
for QP
lala current estimate
of Atul

given
of
current feasibleestimate n

na e

Consider thelE9PI

KII.MIL

gtE.tctcmtsRgAk

fgIJiewkbk

ien

fminplzptcptffnktcfps.LA o

gut

solution pie

Pk

O
If He20 then all KKT
fulfilled n area

Eti ai tk

i c talk ga

tk 0 Kkt notfulfilled thenpickindex

If

i
of a
negative Xi and removethis inden

from talk

startouer

Pk40 If a
key natpk feasible ue.at all constraints
set ink start

over

my

If not find the blocking constraint

aitchKtxa pay 7 bi fol Aitp ko

ME min bi I a a step
length

afip.tn

Mate n k't Mapk Hkey KlKl Ej

start over

Active set method for convex QP

www.ntnu.no TTK4135 Optimization and control



Minner G D cnzz.SI

Egf t

M 22 220

M 22 6 ZO

M 22 2 20

2420

Nz 20

a
I
e
L 3

no Lalo 3,5

Zo

canoe
to KITE

go

crib Eftp.ficzsIL

II

www im n

po

i iii
Min f

f If 51oz

it o
H L

ftp.o

I EH

s 191

o startouer i
Lp by from

y
n Int 3

Zo

g
I

minp pit pity 5Pa

K f

set peo

Xa n't

p lo feasible

blzelot

Example 16.4

www.ntnu.no TTK4135 Optimization and control

10

Another example (N. Gould)

www.ntnu.no TTK4135 Optimization and control

11

How to find feasible initial point?

Same way as for LP:

Phase I: Define another optimization problem with known feasible

initial point, where solution is feasible for original problem.

Phase II: Solve original problem.

Al erna i e me hod: Big M

Relax all constraints; penalize constraint violations in objective

www.ntnu.no TTK4135 Optimization and control

1

3M

Lecture 8: Open loop dynamic optimization

Static vs dynamic optimization (and quasi-dynamic )

Dynamic optimization = optimization of dynamic systems

How to construct objective function for dynamic optimization

Batch approach vs recursive approach for solving dynamic

optimization problems

Video lecture 12 onwards

F
Reference: B&H Ch. 3,4

IM

www.ntnu.no TTK4135 Optimization and control

2

Static vs dynamic optimization

When using optimization for solving practical problems (that is, we

optimize some process) we have two cases:

The model of the process is time independent, resulting in static

optimization

Common in finance, economic optimization,

Recall farming example

The model of the process is time dependent, resulting in

dynamic optimization

The typical case in control

The process is a mechanical system (boat, drone, ), chemical process

(e.g. chemical reactor),

F&H argues for a third option called quazi-dynamic optimization

The process is slowly time-varying, and can be assumed to be static for the

purposes of optimization

We take care of the time-varying effects by resolving regularly (or when the

model has changed sufficiently)

www.ntnu.no TTK4135 Optimization and control

3

Oil production

(example of quasi-dynamic

I
optimization, ex. 2 in B&H)

www.corec.no

www.ntnu.no TTK4135 Optimization and control



e
T

water

9i g 9a 93 97 9is fog ago9i i f is

i4 is

g gas que xwi.fi

F TE u age and go.it is

Eggman 20 12

Fyi if

E c que
fue man

EI 9

sit Ck

mine pro o i

f 7g i 9we 9on Fu
4variable
all sweets for

similarythese

Parameters Quee i Ii
Ffgman 9agmax

age

At

Iago am changesslowly

resolve optimizationproblem with new

parameter values
4

Possible objectives in dynamic optimization

Penalize deviations from a constant reference/setpoint

(regulation) or deviations from a reference trajectory (tracking).

Very often used in optimization for control.

Economic objectives. Optimize economic profit: maximize

production (e.g. oil), and/or minimize costs (e.g energy or raw

material)

Limit tear and wear of equipment (e.g. valves)

Reach a specific endpoint, possibly avoiding obstacles

Reach a specific endpoint as fast as possible

www.ntnu.no TTK4135 Optimization and control



Dynamic models on this course

n state
Dixme qedels.C nonlinear

state

out ya pinput

htt Cht Ut nonlinear

e
time
Jt

Nta Atm Beat Clineartimeuahuantg

NEH Ant But linear time invariants

Linearization for transforming nonlinear fee linear

stationary ft NI go.cn t.it

define'perbubations ng Ey tort
et tout

Uf

Nute gcnt.at g Ettore uItdu continuous

gig
assuming

Tetouan
often tggntzl.int

went uI

At

II true

MIMI wnearizationishelpful

getnonlinear

Bf

duty At Inft Bydat deviations aresmall



X.c.gl problem

Mt

minz ft L ft Contai ut

to

E 0
Nti ft Gk cf

S't MT

ht GE ut 0 too M1

No

given objfuncodependson

a futurestate current

input

Z given

Uo M M Nz g Une MM pm

1k

M predoctoon horizon

ft cnet.ae
stagecast

h Ent ut Eco state and input constraints

NEER
n E Rra
u

ententpredictionhorizontheneeof

veryoften decisionvariables resionearlywith

predictionhorizon

states
Ndola nhigh
decisionzariones

n 4 input eotgxioe.co
nosomesteps
if timesteps
aan
decision 2 20 480
Ulam Sugg a high
 Ut
NE

I 2 3 i Mt M

Standardstagecost
ft latte Ut Ignite 9th atte t dntNtti
I
9 20 positivesemidefinite

UttRt at du t Ut
d
Ry o positivedefinite

specialcases I oscillatory variable

t Hint PenalizeYt instead ofnee

YiuQI Yu Ate Him It Httentte

 referencetracking or followingsome important variable
tracking
qc ninety q Kainuiet

nite AtHete CataldoT9E

nth t 4247 944
T Its aconstant
day t value now doesnt
depend on state
i removedfrom
objectivefune
PIECES

ft Entre UE t DuttSB ut

n
a But Ut Ut
o

o l alone but gu high

b n wzation

minz FEI LAI 9 at that Rat

s t nu Ant C Bat t O N 1

alone ne s ahigh

eueeah Inna

n.ge
 CE 1

min

tf
o

off
f 0 O O A BI O
l i

quality constraint in matrin form

l t

a.m
Batchapproac hun2

m A not Belo
Nz Am Boy AZ not ABuot Boy

in I

L
4Mt

X SX s ne U

X s no smu E
if if
a

E
min NTg ut pi u fifer
U
Ig n
tf
onion 1g shots UJTg fsnnots.ci tlgUTRU

alone high

es
1 Ii
L
5

Example: path planning

Ademoye et al., Path planning via CPLEX optimization, 40th SE Symp. on System Theory, 2008.
www.ntnu.no TTK4135 Optimization and control
6

Time-optimal pendulum trajectories

Swing-up

Obstacle avoidance

Developed in MSc-thesis by
P. Giselsson, LTH, Sweden

www.ntnu.no TTK4135 Optimization and control

7 1

Why quadratic objective?

0.9

0.8

0.7

0.6

x2
0.5

0.4

0.3

0.2

0.1

0
Two reasons: -1 -0.8 -0.6 -0.4 -0.2 0
x
0.2 0.4 0.6 0.8 1

Because it is convenient, mathematically

for analysis and numerical optimization, and smoothness
Lead to linear gradients

Because it is natural; the effect is often desirable

Tends to ignore small deviations
Tends to punish large deviations

Other objectives are possible

www.ntnu.no TTK4135 Optimization and control

8
Linear quadratic control:
Dynamic optimization without constraints

Three approaches for solution

Batch approach v1, full space solve as QP discussed
Batch approach v2, reduced space solve as QP earlier
Recursive approach solve as linear state feedback

www.ntnu.no TTK4135 Optimization and control

9
Linear Quadratic Control
Batch approach v1, Full space QP

Formulate with model as equality constraints, all inputs and states as

optimization variables: EQP!

www.ntnu.no TTK4135 Optimization and control

10
Linear Quadratic Control
Batch approach v2, Reduced space QP

Use model to eliminate states as variables

Future states as function of inputs and initial state

Insert into objective (no constraints!)

Solution found by setting gradient equal to zero:

www.ntnu.no TTK4135 Optimization and control

11
Linear Quadratic Control
Recursive approach

By writing up the KKT-conditions, we can show (we will do this later)

that the solution can be formulated as:

www.ntnu.no TTK4135 Optimization and control

12

Comments to the three solution approaches

All give same numerical solution
If problem is strictly convex (Q psd, R pd), solution is unique
The batch approaches give an open-loop solution, the recursive
approach give a closed-loop solution
Implies the recursive solution is more robust in implementation

vs

Constraints:
Straightforward to add constraints to batch approaches (both becomes
convex QPs)
Much more difficult to add constraints to the recursive approach

Can we
How toAto add feedback (and thereby robustness) to batch
approaches?
Model predictive control!

www.ntnu.no TTK4135 Optimization and control

13
determines howmuchemphasis is
The significance of weigths given on whetherminimizing oneof
theslates or minimizing the Input
variable
lect.cz art2
o

www.ntnu.no TTK4135 Optimization and control

14

Open loop vs closed loop

Next time: How to use open-loop optimization for closed-loop
(feedback!)
This is called Model Predictive Control

www.ntnu.no TTK4135 Optimization and control

1
Lecture 9: Closed loop dynamic optimization
Model Predictive Control (MPC)
Model Predictive Control (MPC)
Open loop vs closed loop dynamic optimization
Feasibility
Stability

Reference: B&H Ch. 3.3-4.2.2

www.ntnu.no TTK4135 Optimization and control

2

Zeilinger, Jones, Borrelli, Morari

www.ntnu.no TTK4135 Optimization and control

3

From ETH

www.ntnu.no TTK4135 Optimization and control

4
Last time: Dynamic open-loop optimization
(with linear state-space model)

b Future performanceof
costfunC
QRS treesemidefinite

www.ntnu.no TTK4135 Optimization and control

5

The significance of weigths

M5

takes more time to reach

input
future

www.ntnu.no TTK4135 Optimization and control

QPformulationo
mzin nite 9 Mu th UETRat
sit Nee Ane But to n Nt
alone my E nhith t l n
M
alone sues ahigh t 0 Mt

zt not net unit nae'T

min 2T Gz G RG r niAnotBn
2

sit
Ag bet A
Ag Io ko Ia.rs bet
Yo

1
ni
bing.IEaa
6
Linear quadratic control:
Dynamic optimization without constraints

Three approaches for solution

Ba ch a oach 1, f ll ace solve as QP optimalopen
Batch approach 2, ed ced ace solve as QP loopoptimized
MIMO
Recursive approach solve as linear state feedback forSaso
section 3 5 F H

www.ntnu.no TTK4135 Optimization and control

7
Linear Quadratic Control Doneinpractice in
Tz
Ba ch a oach 1, F ll ace QP Assignments

Fullspace
I
also called
multifdeshooting

Formulate with model as equality constraints, all inputs and states as

optimization variables: EQP!

www.ntnu.no TTK4135 Optimization and control

8
Linear Quadratic Control
are
Ba ch a oach 2, Red ced ace QP
most
commonMPC
for

Use model to eliminate states as variables

Future states as function of inputs and initial state

Insert into objective (no constraints!)

an
is
Solution found by setting gradient equal to zero:
su asuu.ru
asnnotsm
sansatan u
F e.scui asu.ee sn

www.ntnu.no TTK4135 Optimization and control

9
Linear Quadratic
mm Control
Recursive approach

By writing up the KKT-conditions, we can show (we will do this later)

that the solution can be formulated as:

fegdbf.cm

www.ntnu.no TTK4135 Optimization and control

10

Comments to the three solution approaches

All give same numerical solution
If problem is strictly convex (Q psd, R pd), solution is unique
The batch approaches give an open-loop solution, while the
recursive approach give a closed-loop solution
Implies the recursive solution is more robust in implementation

vs

What about constraints:

Straightforward to add constraints to batch approaches (both becomes
convex QPs)
Much more difficult to add constraints to the recursive approach

Can we handle constraints (use batch approaches) and have

feedback (and thereby robustness)?
Model predictive control!
g
www.ntnu.no TTK4135 Optimization and control
perpC
algorithm
t O do
for l 2
determine ht
soiree open loaf oft problem with noent
Applyfirst input of solution u E no
end

whyrepeated optimization Feedback

11

Model predictive control principle model

plant
Zuo
na
ni une
ch uz um
a

www.ntnu.no TTK4135 Optimization and control

12

MPC illustration
Illustration

www.ntnu.no TTK4135 Optimization and control

13

Open-loop vs closed-loop trajectories

Closed-loop trajectories different from open-loop (optimized) trajectories!

It is the closed-loop trajectories that must analyzed for feasibility and stability.

www.ntnu.no TTK4135 Optimization and control

 does the MPC AP have a solution
Feasibility
In general feasibility is not guaranteed
Must be handled

Commonsolytion black variables intheoryonlylinear

inpracticalpossibletohavei
asquadratic
q
thigh Intent 9 Hey tf attRaz r'tITE
it se LET
l i
s t nu Ant But 0 I htt

Et alone E of E alright E t l he

alone y E uhigh E 0 N
E ZO
Ena M IN 15 4
is large enough
g E o is solution
if possible
Exactpenalty functionimplies that if thereexistsLafeasible solution
to the E E should always bezogggo
originalproblem Caiithout
Stability If always stable
MPC

E Design MPCforsystem ntte i 2 net Ut no

constraints

Lo
Z
min E
Uo q k atte th ut

MI k
s t M l 2 NotUo
Nz 1.2M t Cy l 2 l 2Notyo tu
l 44 Not l 2 tee

substitute States

mina CLE
Ny
twenty
Nz
2 2
no 42 u

Find solution by setting Offa o

Offa
E lo2Notuo t le 2 l 44 Not l 2 14 tr UE
O

gdfy C 44 not E Zao toy they O

ae
aCiaa ut
EE't
uo e
14

MPC optimality implies stability?

MPC solution

MPC closed loop

not
is stable
unstable stable
Mpc
necessarily only if
always

Name latte1 2

unstable

www.ntnu.no TTK4135 Optimization and control

15

MPC and stability

Nominal vs robust stability
nominal abili : The model ed in o imi a ion i co ec (no
model- lan mi ma ch , no di bance )
ob abili i abili nde model- lan mi ma ch and/o
disturbances (more difficult to analyze)

Requirements for stability:

Stabilizability ( (A,B) stabilizable )
Detectability ( (A,D) detectable )
D is a matrix such that Q = DTD ( ha i , D is matrix square root of Q )
De ec abili : No mode can g o o infini i ho being i ible h o gh
Q

www.ntnu.no TTK4135 Optimization and control

System
Mete Ane t Bae
controllable rank LB AB An B
if na
stablizable
if rank XI A BJ Nn T I E eight

desired response

k ut at
R
n.FI o
Inter att t

a o D DDT R 0

Nominal CA B stablizable
stability if
LAD detectable

MPC
stability approach
MI
Min E
to tant d req tight Ry t ignorePam
16

How to achieve
nominal stability?
Choose prediction horizon equal to infinity (N = ∞)
Usually not possible
For given N, design Q and R such that MPC is stable (cf. example)
Difficult, and not always possible!
Change the optimization problem such that
The new problem gives a finite upper bound of infinite horizon problem cost
The constraints is guaranteed to hold after the prediction horizon

Terminal cost

Terminal constraint
Fai l aigh fo a d o do in heo , b can be cl m in ac ice
Typically, in practice: Choose N la ge
Stability guaranteed for N large enough, but difficult/conservative to compute this limit
Shorter N often OK
So what i la ge eno gh in ac ice? R le of h mb: longe than dominating dynamics
www.ntnu.no TTK4135 Optimization and control
17

Why MPC over PI?

Advantages of MPC
MPC handles constraints in a transparent way
Physical constraints (actuator limits), performance constraints,
safety limi ,
MPC is by design multivariable (MIMO)
MPC gives optimal e fo mance

Disadvantage with MPC

Online complexity
Requires models! Increased commisioning cost?
Difficult to maintain?

www.ntnu.no TTK4135 Optimization and control

18

S ee e and hif
How MPC (or better control in general) improves profitability

Ac i e con ain

Backoff Backoff = loss (lower throughput,

off-spec product, ...)

Time

Squeeze
(e.g. MPC)

Shift

www.ntnu.no TTK4135 Optimization and control

1
Lecture 10: More Model Predictive Control 71
(MPC)
Recap: Model Predictive Control (MPC)
Recap: Feasibility&stability
Output feedback
Target calculation
Offset-free MPC (integral action in MPC)

Reference: B&H Ch. 4.2.3-4.2.4

www.ntnu.no TTK4135 Optimization and control

2

Linear MPC; open loop dynamic optimization

called linear because we have a linear model and all
constraints are linear

www.ntnu.no TTK4135 Optimization and control

3

Model predictive control principle

www.ntnu.no TTK4135 Optimization and control

4

Open-loop vs closed-loop

www.ntnu.no TTK4135 Optimization and control

5
MPC and feasibility
Is there always a solution to the MPC open-loop optimization
problem?
Not necessarily state (or output) constraints may become
infeasible, for example after a disturbance
P ac ical ol ion: Sof con ain (o e ac enal fo m la ion )
Sof en a e con ain b adding lack a iable

PenalizeC so that it
is as small aspossible

Soft constraints complicates stability theory

Side- ema k: In MPC abili heo , ec i e fea ibili i an
im o an conce ( hen o don ha e of con ain )
Recursive feasibility: feasibility now implies feasibility in the future
In general, proving recursive feasibility is similar to proving stability

www.ntnu.no TTK4135 Optimization and control

6

MPC optimality implies stability?

MPC solution

MPC closed loop

w
r amace tooaggretine
andunstable
www.ntnu.no TTK4135 Optimization and control
7

MPC and stability

Requirements for stability:
tocheckwe dohaveafun in
I did
Stabilizability ( (A,B) stabilizable ) check matrass
reg
Detectability ( (A,D) detectable )
D is a matrix such that Q = DTD ( ha i , D is matrix square root of Q )
De ec abili : No mode can g o o infini i ho being i ible h o gh Q

How to design MPC schemes with guaranteed nominal stability:

Choose prediction horizon equal to infinity (usually not possible)
Change the optimization problem (add terminal cost/terminal
constraints) such that
The ne oblem i an e a o ima ion of infini e ho i on oblem
The constraints holds after the prediction horizon notmuch
usefulinpractice
For given N, choose Q and R such that MPC is stable (cf. example)
Difficult, and not always ueeogf
mateu.ca
possible!
Typically, in practice: Choose N la ge eno gh
Usually works; since for MPC, plant is often open loop stable
Wha i la ge eno gh ? Longe han domina ing d namic , b ho e can be OK
www.ntnu.no TTK4135 Optimization and control
Linear MPC controller o
Algorithm 445
for te 0 I 2
Determine at
Solve a p
gNI
min E
E O
Ent 94 tightRut
St n Ant But E OI N I

i
name ensign
alone E ut E Uhigh E I M

No Nt
control mane Ue
Ue Uo Applyfirst from
the solution above
end

Obseruation E MPC control law is nonlinear Ut EfEnt

Rm

T
piecewise
stateful affine
Ut
H.am s ICABTIM

e9a
 kh Iem
Outputfeedbacko
f
Cue make an observer
Real system
Pb assume
he y E end
EAg
qp T
7
gain
II
I i

1 iesiinator
Eiffel
I measurement
comedian

here are
corrupted
ye noise
here by
obseries
Linearstateestimator

Atte Anne t Butt L Lye Cane

Requires CA C observable
are now having
Reference tracking LT
nastpeaa.dtofyaeepqathn.me
in constrasttee
Controlled variables Yt Hmt regularcontrol
theorywhere
we control
Control objective we might
y
Ye Yref controlsome
othervariable
Assume is constant Rt
yay onpeicewiseconstant A.us
paxsumiyLI
inuertibl
T
steady state ng Ans t Bus Ns CATBUS
i
Ys Has years
 fi If
A B H a D
EEE
I
2 inputs
I control war

Ys 3.33 8 33 Us

Obserue
a
Input constraint limit possibleYs
G Oo E ne
Ef 0
Eye Ell 66

several us same
Ys
give

us E so
g
so
E
g

g Cruise control
7µg
t
Y Eosin im
from caratfront
is maintained
We are reformulating optimization problem to include

Tref

Targetcalculation
UstRs Us
Tsing Hq 2TG z
s t ng Ans t Bug
t
ef n S Hns Yay E 2
alone E ng s nhigh
whom
us uhigh

ya f9a F RE
Solve
min
IokaI9nt aIR In initial formulation heflaee
t Nta Ant But t oi my objective function with

mi
HomE NtEnhign t b in HaiHnsT9 YetiHnstight Rft
aloneE high
utE U E4 M
done so that we have
no at
bettercontrolauthority
and can acheiveoptimumvalue
of us
8

Offset-free MPC (or MPC with integral action)

From description:
MPC with nonlinear model and a linear (input) disturbance model with one disturbance
state: xt = f(xt,ut)+Bd dt . All states are measured (yt = xt).
A linear observer is designed as a steady-state Kalman filter for the linearized augmented
model at the final equilibrium.
The forward-looking nature of the MPC controller allows to react to disturbances by
considering obstacles in the environment and drastic replanning when necessary.

From Off e -free MPC explained: novelties, subtleties, and applications - G. Pannocchia,
M. Gabiccini, A. Artoni, NMPC 2015 - Seville, Spain September 17 - 20, 2015.

www.ntnu.no TTK4135 Optimization and control

 Its

the
 IT Biased estimate

Now we have included model noise as well in

estimator 2
for
Ik

I Te unbaised estimate
 freempt
Offset f integral action

t ffset
ef.ee

unmadelled disturbance
offset in controlled variables

Model with disturbance

Nta Ant Butt Addt
t

Cut tcdd dtu dt

yet d
as we don't have

anydynamicmodel
fordisturbane
Idea Augmentrapcmadel

H
iii It faith 3
ye Ee coiffed
 Offsetfreecanti
1 Use state estimator to estimate ndt

2 Use ex in the MPC

M
7 requires c ca to be observable
Ao Ed
Thisrequires dim Cdt 2
dimly
calculation must be modified to take
Target
disturbances into account

Industrial phactice Ad 0 CD I
data If t
left Jt
1

444
Lecture 11: Linear Quadratic (LQ) Control

Recap: Model Predictive Control (MPC)

LQ; a special case of MPC with a very attractive solution
(MPC sometimes called constrained LQ )

Finite horizon LQ control

Infinite horizon LQ control

Reference: B&H Ch. 4.3-4.4

www.ntnu.no TTK4135 Optimization and control

2

Linear MPC; open loop dynamic optimization

www.ntnu.no TTK4135 Optimization and control

3

Model predictive control principle

www.ntnu.no TTK4135 Optimization and control

 Finitehoring
min
2 EIRN
ft Ito MIN Antti t at Rat
St Ney Ant But t 0 M1

no
given T
t.FI TABfIsZ UE mi Unt naut

Ut Kent 920
R 0

Tin This is an Egp

The solution of29problemis an explicitcontroller instead ofsolving
a complexQPproblem at each time step
KKTcandi tions.ee
L Z A Az i Am IntoIgntitffntentkattrut
I
t for t o
to make problemsimmer 11 Atty Nta Ane Bat
dim sameas n

172 L Z X Am O
and Nti Ane But O Ee Cicely O IE E

R UE t BTAtty O t O I my
 Of Ont Att Atlee O t I my
dat t
notincludinglast
t

Of 9am Ine O ue.r.it she i e lastelement

Omni
Ncte Ant But
g Ue Rt BTX

Xm 9mm

6
Guess ht _Pent Lassiquziyotherttobelinear fun

Pee Ptt 20
Nem AntetB RtBtttte

Ncte Anet B Rt BTPennetta

solve for Ncte

It BR BT Pty Nth ANE

t
Ntfe fItBRtBTPty Any
 A ne te t ATAem O

Ant Pentt AT Pete Ntt O

One Pint AT
Pu CI t Bre BTPEMAne O

fff Pz AT Ptfe It BRt BTPm A Ut O

0
onlypossible ifthisis o

Ut Rt BTPete Ntt

u RtBTPzy It BRt BTPm A Kt

Kt
Kent fulfilled if
Eff
Ut Kt Nt

where
Kt Rt BTPete It B Rt B Pay A It
Pt CftATP It BRt BP YA E O All

Pm g
o
the 9am E fat
at
4

Necessary conditions (Ch. 12, N&W)

Lagrangian:

www.ntnu.no TTK4135 Optimization and control

5

Second-order conditions

Critical directions:

The critical directions are the allowed directions where it is not clear
from KKT-conditions whether the objective will decrease or increase

www.ntnu.no TTK4135 Optimization and control

6

Thm 16.4: For convex QP, KKT is sufficient

From N&W, p. 464:
KKT conditions

That is, since the solution of the Riccati equation implies the
KKT conditions are fulfilled, Thm 16.4 means that Riccati
equation gives the global solution

Side-remark: It is, in fact, the unique global solution. If G is positive definite (implied by
Q positive definite), this follows from the proof of Thm 16.4. If Q positive semidefinite,
further arguments are necessary (for instance using Thm 12.6 as in the note).

www.ntnu.no TTK4135 Optimization and control

 7
Finite horizon LQ controller

State feedback solution

(discrete) Riccati equation

www.ntnu.no TTK4135 Optimization and control

 m PgGt
Example Eg12 B
10
min E
o kata t LMUE
b t Nu 12M t Ut t O NH

Pu Qi
t
Pee 9 t t l 2pm Pee 1.2 It 1.44k E0 M
ltd htPtah
t
KE dy Pty 1.2 1.2 Pty
Itt
Pepe
le t Ptty
8

Linear quadratic control

The optimal solution to LQ control is a linear, time-varying state feedback:

Note that the gain matrix Kt is independent of the states. It can therefore
be computed in advance (knowing At, Bt, Qt, Rt)

The matrix (difference) equation

is called the (discrete) Riccati equation

Note that the boundar condition is given at the end of the hori on, and
the Pt -matrices must be found iterating backwards in time

www.ntnu.no TTK4135 Optimization and control

9

Example

LQ solution

15 1
r = 0.1 r = 0.1
r=1 r=1
10
r = 20 r = 20
t

0.5

t
P

x
5

0 0
1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11
t t

1.5 0

1 -0.5
t

t
K

0.5 -1

0 -1.5
0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11
t t

www.ntnu.no TTK4135 Optimization and control

10

MPC vs LQ
The difference between finite horizon LQ and MPC open loop problem is
constraints

Mo constraints

LQ open loop solution:

That is: The MPC control law when there are no constraints, is

www.ntnu.no TTK4135 Optimization and control

11

Previous example (MPC optimality implies stability?)

MPC solution

MPC closed loop

www.ntnu.no TTK4135 Optimization and control

 I eML9 f LOR linear quadratic regulator

N
min E
E O
Ig htt 9 ate that Rut
S t Nta Ant But E 0 a

Fact steady state backwards in time solution ofRiceati

equation
infinite horizon solution

PE P
steady state Pete

theorem The solution to when it exists is

givenby

Ut K Ne
t
K Rt Btp It EeBTp
t
P At ATP It BR BTp A

When does it enists

CAB is stablizable

Is infinite horizon 29 stable

A D is detectable
Yes if
12

Increasing LQ horizon

Horizon 20
HorizonNN==320
40
80
160
640
5

3
t
P

0
0 100 200 300 400 500 600

till 0 then can

If we
go
be assumed thatPt a
constant
www.ntnu.no TTK4135 Optimization and control
Pg66 B M
car is stable

I
13
Controllability vs stabilizability
Observability vs detectability
Stabilizable: All unstable modes are controllable
(that is: all uncontrollable modes are stable)
TgoesLee0onitself
Detectability: All unstable modes are observable
(that is: all unobservable modes are stable)

Controllability implies stabilizability

Observability implies detectability

www.ntnu.no TTK4135 Optimization and control

 I 8dat't L
mm ut

b O b O stablizable controllable
b 40 bz o E stablizable
b 0 boffo Not stablizable

a Lao

9,20 geo Detectable observable

9,20 92 0 Detectable

DTD
9
Y
o

9 o 9220 Not detectable

A
Ogg off DTD
2
14

Riccati equations
Discrete-time Riccati equation in the note (and lecture)

However, another, equivalent, form is found in other sources:

The latter is more numericall stable due to enforced s mmetr

The trick used to get the different formulas is the Matri Inversion
Lemma (a ver useful Lemma in control theor , optimi ation, )

ARE ContinuousAlgebraicRiccate
Discrete-time Algebraic Riccati equation (DARE) in the note (and lecture) qn
Other form (e.g. Matlab)

Note: This is a quadratic equation with two

>> help dare
solutions. The one we want is the positive dare Solve discrete-time algebraic Riccati equations.
definite solution (the stabili ing solution).
[X,L,G] = dare(A,B,Q,R,S,E) computes the unique
stabilizing solution X of the discrete-time
www.ntnu.no TTK4135 Optimization
algebraic Riccati equation and control
EX DARE P cigar I
http

Pcrtp htt 1.44 RP

p Cry c 44k P 4 0
I
DARE is aquadraticegn
1

Lecture 12: Summing up MPC and LQ(G)

Recap (mostly): Model Predictive Control (MPC), finite and

infinite horizon LQ
Output feedback LQ: LQG
Aspects related to implementation of MPC in industry

Reference: B&H Ch. 4.5-4.6

www.ntnu.no TTK4135 Optimization and control

2

Linear MPC: repeated open loop dynamic optimization

www.ntnu.no TTK4135 Optimization and control

3

Model predictive control principle

www.ntnu.no TTK4135 Optimization and control

4

LQ: MPC open loop problem without constraints

Solution: LTV state feedback

www.ntnu.no TTK4135 Optimization and control

5

Linear quadratic control; some observations

The optimal solution to LQ control is a linear, time-varying state feedback:

The matrix (difference) equation

is called the (discrete-time) Riccati equation.

Note that the gain matrix Kt and the Riccati equation is independent of the
states. It can therefore be computed in advance (knowing At, Bt, Qt, Rt).

N e ha he b da c di i i gi e a he e d f he h i ,a d
the Pt -matrices must be found iterating backwards in time.

www.ntnu.no TTK4135 Optimization and control

6

Example

LQ solution

15 1
r = 0.1 r = 0.1
r=1 r=1
10
r = 20 r = 20
t

0.5

t
P

x
5

0 0
1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11
t t

1.5 0

1 -0.5
t

t
K

0.5 -1

0 -1.5
0 1 2 3 4 5 6 7 8 9 10 11 0 1 2 3 4 5 6 7 8 9 10 11
t t

www.ntnu.no TTK4135 Optimization and control

7

Increasing LQ horizon

Horizon 20
HorizonNN==320
40
80
160
640
5

3
t
P

0
0 100 200 300 400 500 600

Infinite horizon LQ solution is steady-state finite horizon LQ solution!

www.ntnu.no TTK4135 Optimization and control
8

Infinite horizon LQ (LQR)

This has a solution provided (A,B) is stabilizable

Then the optimal solution is the LTI state feedback

This solution is guaranteed to be closed-loop stable (eigenvalues of A-BK

stable) if (A,D) is detectable, where Q = DTD

Being a state feedback solution, it implies some robustness (more on this later)
www.ntnu.no TTK4135 Optimization and control
9
Controllability vs stabilizability
Observability vs detectability
Stabilizable: All unstable modes are controllable
(that is: all uncontrollable modes are stable)

Detectability: All unstable modes are observable

(that is: all unobservable modes are stable)

Controllability implies stabilizability

Observability implies detectability

www.ntnu.no TTK4135 Optimization and control

10

Riccati equations
Discrete-time Riccati equation in the note (and lecture)

Typically, other forms are used, e.g. (from Wikipedia):

The a e i e e ica ab e d e e f ced e

The ic ed ge he diffe e f a i he Ma i I e i
Le a (a e ef Le ai c he )

Discrete-time Algebraic Riccati equation (DARE) in the note (and lecture)

Other form (e.g. Matlab)

Note: This is a quadratic equation with two

>> help dare
solutions. The one we want is the positive dare Solve discrete-time algebraic Riccati equations.
defi i e i ( he abi i i g i ).
[X,L,G] = dare(A,B,Q,R,S,E) computes the unique
stabilizing solution X of the discrete-time
www.ntnu.no TTK4135 Optimization
algebraic Riccati equation and control
11

LQR vs MPC
LQR i MPC i h c ai , i i i ea a e feedbac
MPC i i i e i i ai (QP)

Sometimes constraints can be active when far from setpoint, but

become irrelevant close to setpoint.
MPC ed ce LQR c e setpoint (under some simple modifications of MPC
problem)

Consider double integrator example:

www.ntnu.no TTK4135 Optimization and control

12

LQR vs MPC, II
Region (polytopic set) where LQR solution is optimal
(where we can assume problem unconstrained)

MPC solution has larger feasible region than LQR solution!

www.ntnu.no TTK4135 Optimization and control

13

LQR vs MPC, III

In fact, MPC solution is piecewise linear, defined on polytopic regions

Proof/computation of this is an exercise in studying KKT conditions

(Not very difficult, but was not realized before ca. 2000)
But: quickly becomes very complex, except for very small systems.
Bemporad, A., Morari, M., Dua, V., Pistikopoulos, E.N.: The explicit linear quadratic regulator for constrained systems. Automatica 38(1), 3 20 (2002)
Tøndel, P., Johansen, T.A., Bemporad, A.: An algorithm for multi-parametric quadratic programming and explicit MPC solutions. Automatica 39(3), 489 497 (2003)

www.ntnu.no TTK4135 Optimization and control

14

LQ regulator (LQR)

www.ntnu.no TTK4135 Optimization and control

15

LQ and robustness
SISO LQ regulators have 60 degrees phase margin and 6dB
gain margin
Can be extended to MIMO systems

LQ

However, usually one does not measure all the states

www.ntnu.no TTK4135 Optimization and control

16

Output feedback MPC

www.ntnu.no TTK4135 Optimization and control

17
Lecture parts
LQG: Linear Quadratic Gaussian 0 00 to 191.25

www.ntnu.no TTK4135 Optimization and control

 So we are able to separate
thedynamics
of the
completesystem into
state feedback
control dynamics
estimator dynamics

estimator system estimatorhastobeatleast 5times

CAKFC is
faster than fasterthanthe controlsystem
controlsystemCA BK

4 4.3 linear MPC with infinite horizon Lcecontrol

Stability of
discussed in video 19 40 till end lecturer 6
partI
but not covered on lecture slides
Axsignment 7 problem 4 is based on this concept
18

LQG and robustness

Doyle, 1978:

Lead a f e ea ch i b c i he 80 (a d a e ),
not topic of this course

www.ntnu.no TTK4135 Optimization and control

19

Complexity reduction strategies in MPC

Input blocking (or move blocking) reduce number of QP variables

I cide i reduce number of QP constraints

Only check constraints at certain time instants, rather than at all times on horizon

www.ntnu.no TTK4135 Optimization and control

20
Production optimization in process industry
Rea - i e i i ai , S ead - a e i i ai

In a plant we (usually) have

Degrees of freedom (u a e e i g )
States (x e ea e a d e e )
Disturbances (d)

How to use the DOFs? What should we optimize?

Often translates to throughput maximization active constraints!

Mathematical formulation

www.ntnu.no TTK4135 Optimization and control

21

Control and optimization hierarchy

stae
Iusually
models
Production optimization (RTO)
Minimize E to obtain setpoints (reference
trajectories) for control layer

a
iEmgi 1 5155am
Model Predictive Control (MPC)
use stepresponsemodels
Follow trajectories,
not statefeedback
keep process within constraints

Base control (PID)

Stabilize integrating and unstable process
parts

© Skogestad
www.ntnu.no TTK4135 Optimization and control
22
NMPC Discrete-time Open Loop
Optimal Control Problem
Solve

subject to t t+1 t+N t+P

with

www.ntnu.no TTK4135 Optimization and control

23

Incentives for MPC

Regularity, handling constraints transparently
( e aci g gic), be e c ( i i i g,
MIMO), ed ced ai e a ce,
Also directly economy?

But MPC follows setpoints given by RTO

how does MPC improve economy?
RTO:

MPC: setpaint following

www.ntnu.no TTK4135 Optimization and control

24
S ee e a d hif This is how MPC improveseconomy
Traditional view of how MPC (or improved control) improves economy

Ac i e c ai

Backoff Backoff = loss (lower throughput,

off-spec product, ...)

Time

Squeeze
(e.g. MPC)

Shift

www.ntnu.no TTK4135 Optimization and control

25

Squeeze & shift

COST FUNCTION

N Histogram
LEVEL 0 / LEVEL 1
2
Sigma 1 -- Sigma 2

Sigma 2
LEVEL 2
OFF
Q1 -- Q2
SPEC
W2
1.5

DELTA COST (W2-W1)

W1

Sigma 1
0.5

SQUEEZE
Q2 Q1 QUALITY
0
0 50 100 150 200 250 300 350 400 450
SHIFT
© Richalet
www.ntnu.no TTK4135 Optimization and control
26
Economic optimization and MPC
(current research)

S ee e a d hif Indirect
economic optimization by (N)MPC

Can we use a more direct

approach?
D a ic RTO (= Economic MPC )
D h squeeze shift
inone
together
RTO: Minimize E subject
step
to steady state model
D-RTO: Minimize E subject
to dynamic model
MPC: Minimize J subject
to dynamic model

www.ntnu.no TTK4135 Optimization and control

27

RTO + NMPC = Economic MPC (D-RTO)

Opportunities with D-RTO Challenges with D-RTO
Optimize performance also Managing complexity
under transients/non-steady
state (transients can be long!) Large, dynamic, non-linear
Disturbances
optimization problems have to
Grade changes
be solved at high sample rates

Consistent models for

performance optimization Developing theory&algorithms
for tuning (control performance
Consistent specification of
vs economy trade-off) and
process control objectives
stability
Optimization software
becoming mature
Leverage from industrial
(N)MPC implementations
www.ntnu.no TTK4135 Optimization and control
28

Simulation study: Offshore oil and gas production

Together with Anders Willersrud & Cybernetica AS

Short-term production
optimization: Maximize (e.g.)
oil production, given a long-
term recovery strategy

Ad ch e f d ci g
wells to obtain largest
ib e i d ci

Wells have different gas-oil

ratios (GOR)

www.ntnu.no TTK4135 Optimization and control

29

Simulation study

www.ntnu.no TTK4135 Optimization and control

30
Model developed in
Modelling&Control Modelica/Dymola,
249 dynamic states
NMPC tool:
Cybernetica CENIT
with Dymola model
Using unreachable
setpoint for zoil

www.ntnu.no TTK4135 Optimization and control

31

Production optimization
We want to maximize oil production
Assumption: Bottleneck is gas processing capacity
Implication: Upper limit on inlet separator pressure is active
constraint
to oil ratio
Due to different GOR in wells, it can be many combinations of
choke openings that meet active constraint
C e i a i :U ea i g d ce a a ac i e
constraint

We want to optimize online!

www.ntnu.no TTK4135 Optimization and control

32

Results unreachable setpoints

Initial state: Inlet

separator constraint
active

NMPC start: 10 min.

www.ntnu.no TTK4135 Optimization and control

 ables
Controlinputblocking
wingman M i controlinputdecision
variables considerably
reduces nu
thus reduces
e na
of
runtime

N nos na ofcontrolinputs

to

slackuaeeiabl.es To increase the

feasible region and then we
put penalty on E so that the value of extension E is not
high
1
Example
Adaptive cruise control
• Cruise control: Control system that adjust throttle to keep speed constant
• Adaptice cruise control (ACC): Adapt speed when vehicle in front of you
Two modes of control:
– Speed control when no vehicle in front
– Distance control when vehicle in front – a radar sensor measures distance

https://www.continentalmitsubishi.com/blog/mitsubishi-adaptive-cruise-control-explained/

• Simplified case study: MPC for distance control

Inspired by
• Takahama, Akasaka: “Model Predictive Control Approach to Design Practical Adaptive Cruise Control for Traffic Jam”, 2017
• ACC example in Matlab MPC toolbox

www.ntnu.no TTK4135 Optimization and control

2

ACC Modeling
Host car Lead car

Desired distance, hr
Relative distance, h

Host car velocity, vh Lead car velocity, vl

Note that “throttle dynamics” is most inaccurate part

of this model. In reality acceleration and braking
has quite different dynamics.

www.ntnu.no TTK4135 Optimization and control

3

ACC state-space model

www.ntnu.no TTK4135 Optimization and control

4

ACC control performance specifications

• Control distance to vehicle in front

• While not using too much input (not accelerating excessively)

• Limit maximum accelerations

• Limit speed

www.ntnu.no TTK4135 Optimization and control

5

MPC formulation
• Assume d = 0

• Weights:

• Constraints:

www.ntnu.no TTK4135 Optimization and control

6

Scenario 1: No disturbances Onlypossible

with
MPC not with
• No disturbances 29R as LQRcovethail
– lead car velocity constant, no resistant force constraints
• Start 10m behind desired headway,
same velocity as leader car

limit on
undershoot
useslackvariable
relax to uaitate
constraint but
put aheavylime
so that it
bouncesback
asquicklyas
possible

www.ntnu.no TTK4135 Optimization and control

7

Scenario 2: Disturbance, no compensation

www.ntnu.no TTK4135 Optimization and control

8

Offset free control I to

compensate for disturbances
• Make disturbance observer:

Why we needtargetcalculation
Nt
Zt
• Do target calculation min UE
– Not implemented yet t 0 tf I
2
6 Us
inputneededto counteract
resistiveforces
www.ntnu.no TTK4135 Optimization and control
9

Scenario 3: Disturbance, Offset-free MPC

Still not completely offset-free, since

I haven’t added target calculation

www.ntnu.no TTK4135 Optimization and control

1
Lecture 13: Unconstrained optimization
26 19
Optimality conditions for unconstrained optimization
Ingredients in a general algorithm for unconstrained optimization
Descent directions (steepest descent, Newton, Quasi-Newton)
How far to walk in descent direction (line search, trust region)
Termination criteria
Scaling

Reference: N&W Ch.2.1-2.2

www.ntnu.no TTK4135 Optimization and control

 Uncured optimization
min it city _o i EE
NER
flog
Geng 70 EEI

today E 0 I unconstrained optimization

n e

Inlhat is solution
Global minimizer Cn all
tri fCn Ef for n

Local minimizer
f Cn flag for all ENEMY
e n

Strict local Gy all NENCHY

minimizer
flag f for
2
Learning goal Ch. 2, 3 and 6: Understand this slide.
Line-search unconstrained optimization
1. Initial guess x0
2. While termination criteria not fulfilled
a) Find descent direction pk from xk
b) Find appropriate step length αk ; set xk+1 = xk + αk pk
c) k = k+1 A comparison of steepest
descent and Newton's method.
3. xM = x*? (possibly check sufficient conditions for optimality) Newton's method uses
curvature information to take a
more direct route. (wikipedia.org)

Termination criteria: Descent directions: Step length (Wolfe):

Stop when first of these become true: Steepest descent
(necessary condition)
(no progress) Newton
(no progress)
(kept on too long) Quasi-Newton

How to calculate derivatives (Ch. 8)?

How many iterations? (Convergence rates)
www.ntnu.no TTK4135 Optimization and control
3

Local and global minimizers

www.ntnu.no TTK4135 Optimization and control

4

(Strict and non-strict optimizers)

strict minimizer Valleyofminimizer

www.ntnu.no TTK4135 Optimization and control

5

Taylor expansions
From Calculus?

In this course:

www.ntnu.no TTK4135 Optimization and control

6

Taylor s theorem

to ntp
a p
First order: If f is continuously differentiable,

quality
g
f Cutts
approx
forest a
fCnTp
Second order: If f is twice continuously differentiable

k PTIZCf uttppoet
If coup Ffca f
t

www.ntnu.no TTK4135 Optimization and control

 theorem Necessary condition

n local solution n E C
f d FfEnix O

and one time continuously

differentiable functions

Proof By contradiction
Axxume a local solution and FfCn TO
teepeedescent FfCH pTl7fCn Hafen 11 so
direction Since Cn is continuous there exists T 70 At
f
pHfCn ttp so for all te Lo t
from taylor forany E E o t
Cn t Ep fCnk t Ept FfGetty t E Lo EI
p f

n tT
Http f cut
n is not local min contradiction
Paced theorem

g
nk
e
m

ga
theorem sufficientconditions
E C N T Cnk 0 172 0
f f fluty
Hunt
is strict local solution
Proof IfCng O
If continuous Exists970 At ThfCa 70
forall n E n IHn n Her
openbowl aroundnk
Taylor for any b Hpu ca

t ecoD
f Great
p f Cn t fcnkltptlg.ptofCnattp p
O 7O
fenix

look that
If we
for only points satisfy sufficient
condition then we miss some minimum
might
na
g
n

M
local
f convex n min

n min
global
 General line search algorithm
for solving minnfees

1 Initial bro
guess no
27 While termitmia notfulfilled
2cg find a des n
pk Cfrannee
2b walk along pie to Mate Mate Nkt Mapk
24 ke Rtl
end
3 n
Nm

Tench criteria Given small E 0

But thenhere
If
11nee n 11 E E I Cnd LE
flak we need to
know nk which
we dam it

Most important criteria

I Ilflow 11 E E
2 11 nee nee Il E E Oe11me Hill E Ell aka Il
3 I flail flake se artfluid fluke fselfemail
i e
stop makingprogress in obj function
value

4 K2 Konan Gone too far no stamina to do iterations

anyfurther
check all terminate when holds
Inpractice first
 Descentdine ction

im

at
bae TfCnl

Maindescentdirections
e teddies pk TfEnea
Used in most
of machinelearningalgorithms
r
2 Mutton pea Ff laid aflaw
I ma
more
computationally expensive

a.am
e
c
I dgcnII.t

t 172
meet'sr an

fGH flak p o

3
Q on
Pk BI Ff Gk
t.BE
fnTfs approximatinghessian
 How should walk along f k
far we

e Line search Find that approximately solues

mind f Cnetxpk Ak

set nku Nk t
Nkpk

2 Trust Construct mkCp

legion a
ofCnetp
b s t MkCH S Mk lo
find p
and p En trust region

Mk Cf fCnut fLuft
p Ig Tfendto
t PT

approx
flak t TfCnutp pT p
min mkLpl
PERn Ilp11 a
ie
plieswithintrust region
7

Quadratic approximation to objective function

Minimize approximation:

Newton step :

initial
minimizing quadrate
approximation

1stiteration

Chong & Zak, An introduction to optimization

www.ntnu.no TTK4135 Optimization and control

8

Steepest descent directions vs Newton

directions from objective function
approximations
Newton uses curvature
information
I

From Edgar, Himmelblau, Lasdon: Optimization of Chemical Processes

www.ntnu.no TTK4135 Optimization and control

9

Steepest descent vs Newton

zigzagsteps more
aligned

Boyd & Vanderberghe, P. Abbeel

www.ntnu.no TTK4135 Optimization and control

10

Line search and trust region steps

late concentratetee
line search here

www.ntnu.no TTK4135 Optimization and control

11

Scaling, scale invariance

Tv
variables approx of Poorlyscale easier
thesame size to in wrong
go
dirn awayfrom mi

Better scaled

www.ntnu.no TTK4135 Optimization and control

 In chemical processes usebaninsteadof
scaling units of variables for betterscaled variables
Poorly sealed some directionsgive muchlarger
change in objective function than others

Steepest descent sensitive to scaling

Menton scale invariant

ef fCn 1,24099422 poorly scaled

steepest descent

f Emine

Newton

t in
p ma subagent

scale invariance more robust algorithms

gives
BUT I Aled STYX
1

8144
Lecture 14: Globalization strategies
Two basic globalization strategies: line search (Ch. 3) and trust-
region (Ch. 4, not syllabus)
N e: gl bali a i d e i l ha e ea ch for global optimum, but we make
the algorithm work far from a (local or global) optimum!
Step-length, Wolfe conditions
Step-length computation
Hessian modifications

Reference: N&W Ch.3-3.1, 3.4, 3.5

www.ntnu.no TTK4135 Optimization and control

2

A general algorithm for unconstrained optimization

1. Initial guess x0
2. While termination criteria not fulfilled
a) Find descent direction pk from xk
b) Walk along pk to xk+1 (how long? line search!)
c) k = k+1
3. xM = x*? (possibly check sufficient conditions for optimality)

Termination criteria: Descent directions:

Stop when first of these become true: Steepest descent
(necessary condition)
(no progress) Newton
(no progress)
(kept on too long) Quasi-Newton

A comparison of steepest
descent (green) and
Newton's method (red) for
minimizing a function
(with small step sizes).
Newton's method uses
curvature information to take
a more direct route.
(wikipedia.org)
www.ntnu.no TTK4135 Optimization and control
3

Steepest descent direction vs Newton direction

from objective function approximation

From Edgar, Himmelblau, Lasdon: O i i a i f Che ical P ce e

www.ntnu.no TTK4135 Optimization and control

min
Zern flat
line search Nka Nkt pk

pie Brit
Ifk
Ba I steepest descent

Bk Thf o
Newton

Bar 172ft quasi newton

Bk o pie is descent direction

Newton dir is not necessarily a descent direction

Proof pet fee KTBTetf.ee

Ff 0

How to choose me
I
ii 9 10 4
 Enact line search 19K
a
floretxpie
a
ay mina

Inpractice in exact line search

Look for a that
fulfills
1
Sufficient decrease text
Ca E lGtgx

value g lo 4
typical
2 Curvature condition
G I Go 0 40

9 10 4

7 It I
These are the keolfe conditions
1
f CnutAapa E fCmd t G Ma 7ftpie
2
RfCnet Kpk Effiepk
OL G C G C I
4

Why sufficient decrease?

Decrease not enough, need sufficient decrease (1st Wolfe condition)

www.ntnu.no TTK4135 Optimization and control

5

Sufficient decrease

www.ntnu.no TTK4135 Optimization and control

6

Curvature condition

www.ntnu.no TTK4135 Optimization and control

7

Backtracking Line Search

www.ntnu.no TTK4135 Optimization and control

Line search
by interpolation
Initial
guess Ao
If Wolfe 0K exit
Do quadratic interpolation

q A X't o x
k

I
9 04012
to
210Ko Geo
01 o Ro

If wolfe 0k four exit

da cubic interpolation
ax't bae t lo a
Ofc t g lo
a
fee book
b

q see book

If ueolfe OK for a exit

Reduce do start cheek
Hessianmultiplication fermented
Newton Aku Nk t Nk pk
t
pk Effie Pfk
in solve
practice Tfa Pk Rfk

Eme far from an optimum

Tff typically net p d

possibly no solution to

possibly Pk is not descent direction

Solution Replace 172ft with 17Hok t Ek 70

Several Ek Ch3 4
ways of constructing
Faure
Ek Tye I

0
Zoe if 172ft O
Sy
Amin 712 t d otherwise
fa
172ft Mtn
rn
8

Line search Newton

www.ntnu.no TTK4135 Optimization and control

9 are
convergence rates more
Local convergence rates relevant near optimum

Steepest descent:
Linear convergence

Newton:
Quadratic convergence

Quasi-Newton:
Superlinear convergence

www.ntnu.no TTK4135 Optimization and control

1

513 Lecture 15: Quasi-Newton

Quasi-Newton
Q-N efficiently produce good search directions
Steepest descent: Many iterations, but each iteration cheap (need only gradient)
Newton: Few iterations, but each iteration expensive (need also Hessian)
Quasi-Newton: Achieve few iterations by approximating the Hessian using only the
gradient
Secant condition
BFGS (and DFP) Hessian approximation update formulas

What is this used for? A lot!

Machi e lea i g, Image ce i g,

Reference: N&W Ch.6-6.1 (Superficially 7.1)

www.ntnu.no TTK4135 Optimization and control

2

Line-search unconstrained optimization

1. Initial guess x0
2. While termination criteria not fulfilled
a) Find descent direction pk from xk
b) Find appropriate step length αk ; set xk+1 = xk + αk pk
c) k = k+1
3. xM = x*? (possibly check sufficient conditions for optimality)

Termination criteria: Descent directions:

Stop when first of these become true: Steepest descent
(necessary condition)
(no progress) Newton
(no progress)
(kept on too long) Quasi-Newton

A comparison of steepest
descent (green) and
Newton's method (red) for
minimizing a function
(with small step sizes).
Newton's method uses
curvature information to take
a more direct route.
(wikipedia.org)
www.ntnu.no TTK4135 Optimization and control
3
Line search

Conditions for a good step length: Wolfe conditions

www.ntnu.no TTK4135 Optimization and control

4

Hessian modification
For to be a descent direction, we need

In general, this does not hold true for Newton, . We

therefore modify the Hessian when it is not positive definite:

A good, but inefficient method:

M e efficie d imila cha ge d i g fac i ai ce
E.g. m dified Cholesky me h d

www.ntnu.no TTK4135 Optimization and control

Mention
model at me
fan
f Cneatp mk p fat
7fktptlgptitfkp.fm me

I
Ma

bk ang min mealp Tifa Rfk

p
me 17feet71 p o

veryfast convergence
Expensive to calculate Ffa
quasi
Meuet Cfm
model
fan at my

t
ftnktp.IE MkCp featTifap k p Bap t
 Have to choose Bk
1 Bk o so that we can produce descent direction
2
BE 728k forfast convergence
3
Only dependence on Ffk

Consider none

Mkte b fkn t Tlfkutp t L BTBkap

I
I
a'Kkk X
mean lo fkn
want
mantapa Tfkµ Bky9kpk Rfk
Bay Kpk Hokey Tok
L
SENka Nk Lk
Secant equation BaeSk yea

THI Pfk featMfa Mea Ma ifusingTaylor

Bien 0 SkiBaySk Slaty k 0 Sktfee o Bean 0

shouldbe
 Requires Sktya 70 Holds automatically if
1
f is a convex function or
2 hefulfills the Wolfe conditions

Infinitely many choicesfulfill Bkask ya

Choose one closest to Bk

Bka Aly miz11B BKH B BT 0 BSk_yk

Solution with matrin norm weighted Frobenius norm

DEP Bkey I fee
yeskt BK I fkskyE theykyat
E if
siege
since what we need pk FETFfk can we update

MKE fee instead

Matrix inversion lemma
t
At UCV At At U Ct VA UfVA
usefulincentraltheory
mannfitteretc

IIg.iq
Bien Bist Bse Ya fog BgYI

et
Alternatively Choose
Hku min HH Hull st H HT O
Of
My E Sk

1Hkti LI
fkskyIJHKII
fkyksktltfkskskhg.gs
BEGS is considered the most effective 9M
formula

Hk positive definite if yetsk o

Sk 0

y
T 2
NTH Csisa
yeaskin
Kun n Hk n
flag
a
tf
unless
ZO Sian _0
only 0 when n o

0
except when n o

Have to choose Ho
Ho YI
5

BFGS

www.ntnu.no TTK4135 Optimization and control

6

BFGS method

www.ntnu.no TTK4135 Optimization and control

7

Local convergence rates

Steepest descent:
Linear convergence
blow
Newton:
Quadratic convergence
extremely
Quasi-Newton:
fasts
Superlinear convergence

www.ntnu.no TTK4135 Optimization and control

8

Example (from book)

Using steepest descent, BFGS and inexact Newton on
Rosenbrock function

Iterations from starting point (-1.2,1):

Steepest descent: 5264
BFGS: 34
Newton: 21

wikipedia.org

Last iterations; value of

www.ntnu.no TTK4135 Optimization and control

9

http://www.cs.cmu.edu/afs/cs/academic/class/15780-s16/www/slides/optimization.pdf

www.ntnu.no TTK4135 Optimization and control

10

http://www.cs.cmu.edu/afs/cs/academic/class/15780-s16/www/slides/optimization.pdf

www.ntnu.no TTK4135 Optimization and control

 CHI LBFGS_
n 1000000 elements in te
Ef 5 10
na of independent nCnz_H
elements
4000GB
Ace
using BEGS becomes infeasible
BEGS
use
store
limey
lost in 5 20
Sa m
ya
Use
only chose to calculate Hkufrom L recursively
BFGSqn
11

Quasi-Newton
in machine learning

Proceedings of the 28 th International Conference on Machine Learning, Bellevue, WA, USA, 2011

www.ntnu.no TTK4135 Optimization and control

1
Lecture 16: Calculating derivatives (Ch. 8), and
13
Derivative-free optimization (Ch. 9)
Brief recap linesearch unconstrained optimization

Calculating derivatives (gradient/Jacobian and Hessian)

What can you do when obtaining derivatives is impractical?

Derivative-free optimization, Nelder-Mead

Reference: N&W Ch. 8.1, (8.2), Ch. 9.1, 9.5

www.ntnu.no TTK4135 Optimization and control

2

Line-search unconstrained optimization

1. Initial guess x0
2. While termination criteria not fulfilled
a) Find descent direction pk from xk
b) Find appropriate step length αk ; set xk+1 = xk + αk pk
c) k = k+1 A comparison of steepest
descent and Newton's method.
3. xM = x*? (possibly check sufficient conditions for optimality) Newton's method uses
curvature information to take a
more direct route.
(wikipedia.org)

Termination criteria: Descent directions: Step length (Wolfe):

Stop when first of these become true: Steepest descent
(necessary condition)
(no progress) Newton
(no progress)
(kept on too long) Quasi-Newton

www.ntnu.no TTK4135 Optimization and control

3

Quasi-Newton: BFGS method

BEGS

www.ntnu.no TTK4135 Optimization and control

4

Example (from book)

Using steepest descent, BFGS (Quasi-Newton) and inexact
Newton on Rosenbrock function

Iterations from starting point (-1.2,1):

Steepest descent: 5264
BFGS: 34
Newton: 21

wikipedia.org

Last iterations; value of

www.ntnu.no TTK4135 Optimization and control

5

Line-search unconstrained optimization

1. Initial guess x0
2. While termination criteria not fulfilled
a) Find descent direction pk from xk
b) Find appropriate step length αk ; set xk+1 = xk + αk pk
c) k = k+1 A comparison of steepest
descent and Newton's method.
3. xM = x*? (possibly check sufficient conditions for optimality) Newton's method uses
curvature information to take a
more direct route.
(wikipedia.org)

Termination criteria: Descent directions: Step length (Wolfe):

Stop when first of these become true: Steepest descent
(necessary condition)
(no progress) Newton
(no progress)
(kept on too long) Quasi-Newton

Need derivatives! How to compute them?

And what if derivatives are not available, or too expensive to compute?
www.ntnu.no TTK4135 Optimization and control
6

Finding derivatives of vector functions

By hand
Time consuming and (very!) error prone for large problems
But can give fast and efficient code
Symbolic differentiation
Computer algebra systems (CAS)
Maple, Mathematica, Matlab symbolic toolbox, ...
May result in very long code that is expensive to evaluate (and compile!)

Numerical differentiation (finite differences)

Easy to implement (do it yourself), but may have low accuracy

Automatic (Algorithmic) Differentiation (AD)

Best option!
Relatively easy to implement using the right software
Exact up to machine precision

www.ntnu.no TTK4135 Optimization and control

7

Numerical differentiation (finite differences)

Scalar : For some small ,

Directional derivative of

Full gradient : Directional derivatives along all axes

( )
Note: Not necessary to calculate full gradient if you only need directional
derivative! (Also valid for AD!)

How to choose epsilon?

Theoretical error proportional to , but too small gives numerical noise
Rule of thumb: , where eps is machine precision (or precision
of computing f) (IEEE double precision: ) double64 2016
for
www.ntnu.no TTK4135 Optimization and control
Theoretical accuracy offinitedifferences
fansHE L
Assume11172
Taylor
If Cntp tocastFfcnfp Ig p Tff http pl tEto
E 1gHell 11172 tht tf p11
f
inner product less than
norm individual element
of
ballpH Ha fCmpIII 11pm

Lz Ilp112
T
Let OO 0
p Eep ei 0,1 0 E 70

I f Ent Ee 3 fans E
Ef ei I E LezE
L
I fcnteeiz
fcnl ffm.IE LEE

ldfoni
toCEX
fcnteeil.to
htt
function evaluations
Tueasideddifferences
dotn fC eei to ET
2E
2nsfunction evaluations

we here that tuea sided differences

can
clearly see
are much more accurate than one sided differences
as have rather than E as we see in one sided
they
difference However sided differences
two
require more
computations
8

Ta he e

First order: If f is continuously differentiable,

Second order: If f is twice continuously differentiable

www.ntnu.no TTK4135 Optimization and control

9

Approximating the Hessian

In many cases, the gradient is available, but not the Hessian.
Then use finite differences on the gradient:

If he g adie i a ai ab e, e fi i e diffe e ce ice :

www.ntnu.no TTK4135 Optimization and control

10

AD Automatic (algorithmic) differentiation

Software tools that automatically computes derivatives of your code
The principle is simple: Extensive/automated use of chain rule

Two modes
Forward
Reverse (adjoint)

Two (main) implementation variants

Source code transformation
Operator overloading

Requires (more or less) that your implementation is differentiable

Example software: ADOL-C, CppAD, CasADi,

www.ntnu.no TTK4135 Optimization and control

11

AD forward and reverse

Forward mode
Both and are calculated by forward traversing
computation graph

Reverse mode
First, calculate by traversing graph forward
Then, calculate derivatives by traversing graph backward

www.ntnu.no TTK4135 Optimization and control

12

AD forward vs. reverse

Given a function
Costs of calculating derivatives with AD:
F ad de ( e c ):
Forward mode (entire Jacobian):
Re e e de ( e ):
Reverse mode (entire Jacobian):

(Forward mode: Similar cost as numerical differentiation, but

more accurate)

If m >> n, forward mode is fastest

If n >> m, reverse mode is fastest

www.ntnu.no TTK4135 Optimization and control

13

Implementing AD
Prototype procedure:
1. Dec e igi a c de i i i ic f c i (e.g. 1x2, sin(x),
ln(x), etc.)
2. Diffe e ia e i i ic f c i ( b ica , a ea i )
( i ( ) = c ( ), e c.)
3. Put everything together according to the chain rule
(either forward or reverse)

How to automatically transform your program into a program with

derivatives? Two approaches:
Source code transformation (Typical: C, Fortran)
Operator overloading (C++, Fortran 90, Java, Matlab, P h , )

www.ntnu.no TTK4135 Optimization and control

14

Example (C/C++)

function.c

double f(double x1, double x2, double x3) {

double x4, x5, x6, x7;

x4 = x1*x2;
x5 = cos(x1);
x6 = x3*x5;
x7 = x4 + x6 + x3;

return x7;
}

function.c

www.ntnu.no TTK4135 Optimization and control

15

Source code transformation (forward mode)

diff_function.c
double* f(double x1, double x2, double x3, double dx1, double dx2, double dx3) {
double x4, x5, x6, x7, dx4, dx5, dx7, df[2];

x4 = x1*x2;
dx4 = dx1*x2 + x1*dx2;
x5 = cos(x1);
dx5 = -sin(x1)*dx1;
x6 = x3*x5;
dx6 = dx3*x5 + x3*dx5;
x7 = x4 + x6 + x3;
dx7 = dx4 + dx6 + dx3;

df[0] = x7;
df[1] = dx7;

return df;
}

function.c AD tool diff_function.c compiler diff_function.o

www.ntnu.no TTK4135 Optimization and control

16
Operator overloading example (using CppAD)
Implement function as you do normally, but with other types (here: using
C++ templates):
function.cpp
template <class vector>
vector f(vector x) {
vector ... // possible temporary variables

return x[1]*x[2] + x[3]*cos(x[1]) + x[3];

}

Record operation sequence when you use the function:

...
CppAD::vector<ADdouble> x(3), f_res;
x[1] = pi; x[2] = 4; x[3] = 3;

// declare that x contains the independent variables (and start recording)

CppAD::Independent(x);

f_res = f(x);

// create the AD function object F : x -> f_res (and stop recording)

CppAD::ADFun<double> F(x, f_res);

std::vector<double> jac( NS*NS ) = F.Jacobian;

...

function.cpp and user program object file with

compiler derivatives
AD library
www.ntnu.no TTK4135 Optimization and control
17

Software etc.
General information
http://www.autodiff.org/
http://en.wikipedia.org/wiki/Automatic_differentiation

Many libraries of different maturity/robustness/performance, for

different languages and for different applications

Some mature libraries

C++: ADOL-C, CppAD
Developed for control&optimization: CasADi (Matlab/Octave, Python, C++)

Book:
A. Griewank, A. Wa he , E a a i g De i a i e : P i ci e a d Tech i e f
A g i h ic Diffe e ia i , 2 d edi i . SIAM, 2008.

www.ntnu.no TTK4135 Optimization and control

18

AD example (from R. Ringset)

Calculate gradient of

at

www.ntnu.no TTK4135 Optimization and control

19

AD forward mode

www.ntnu.no TTK4135 Optimization and control

20

AD forward mode

www.ntnu.no TTK4135 Optimization and control

21

AD forward mode

www.ntnu.no TTK4135 Optimization and control

22

AD forward mode

www.ntnu.no TTK4135 Optimization and control

23

AD forward mode

www.ntnu.no TTK4135 Optimization and control

24

AD forward and reverse

Forward mode
Both and are calculated by forward traversing the graph

Reverse mode
First, calculate by traversing graph forward
Then, calculate derivatives by traversing graph backward

www.ntnu.no TTK4135 Optimization and control

25

AD reverse mode

www.ntnu.no TTK4135 Optimization and control

26

AD reverse mode

www.ntnu.no TTK4135 Optimization and control

27

AD reverse mode

www.ntnu.no TTK4135 Optimization and control

28

AD reverse mode

www.ntnu.no TTK4135 Optimization and control

29

AD reverse mode

www.ntnu.no TTK4135 Optimization and control

30

AD reverse mode
mode node Of e I Cabuays
Right dni

www.ntnu.no TTK4135 Optimization and control

31

Example: optimization using CasADi

CasADi (https://casadi.org/)
CasADi is a symbolic framework for numeric optimization implementing automatic
differentiation in forward and reverse modes on sparse matrix-valued computational
graphs.

rosenbrock.m
import casadi.*

% Create NLP: Solve the Rosenbrock problem:

% minimize x^2 + 100*z^2
% subject to z + (1-x)^2 - y == 0
x = SX.sym('x');
y = SX.sym('y');
Define variables z = SX.sym('z');
v = [x;y;z];
f = x^2 + 100*z^2;
Define objective and constraints g = z + (1-x)^2 - y;
nlp = struct('x', v, 'f', f, 'g', g);

% Create IPOPT solver object

Create solver object solver = nlpsol('solver', 'ipopt', nlp);

% Solve the NLP

Solve the opt problem res = solver('x0' , [2.5 3.0 0.75],... % solution guess
'lbx', -inf,... % lower bound on x
'ubx', inf,... % upper bound on x
'lbg', 0,... % lower bound on g
'ubg', 0); % upper bound on g

% Print the solution

f_opt = full(res.f) % >> 0
x_opt = full(res.x) % >> [0; 1; 0]
lam_x_opt = full(res.lam_x) % >> [0; 0; 0]
lam_g_opt = full(res.lam_g) % >> 0

www.ntnu.no TTK4135 Optimization and control

32

Derivative-free optimization
If you with reasonable effort can implement derivatives
(gradients, possibly Hessian), use them!
A a e efficie ha i g he !
However, sometimes, obtaining derivatives is prohibitive
Examples:
Your objective function (and possibly constraints) are calculated using a (large)
b ac -b i a ha i e e i e
Your objective function (and possibly constraints) contain code that is not (easily)
diffe e iab e ( e ica i e a e e f fi i e diffe e ce diffic )
Often models from computational fluid dynamics (CFD)
(but recently, some CFD software exports some sort of derivative information)

Thi i ae de i a i e-f ee i i ai eh d
www.ntnu.no TTK4135 Optimization and control
33

Derivative-free optimization (DFO)

DFO use function values at a set of sample points to determine
new iterates. Coarsely, two different classes of methods:
1. M de -ba ed : U e a e i b i d a approximate model of the
objective function, then find search directions from approximate model
2. Me ahe i ic : Of e inspired b ce e i a e, ch a ge e ic
ag i h , i a ed a ea i g , a ic e a i i ai , f ac
i i ai ,

Many of the metaheuristic e h d c ai d g ba

i i ai a d ac e -diffe e iab e be , b hi
must be interpreted with care. Guarantees are seldom given.
Use if it is more important to find a good/improved objective function value
rather than the precise (local) optimum.

DFO generally works best if the number of optimization variables

is relatively small

Here: Nelder-Mead (old&simple, but fairly good) Ch9

www.ntnu.no TTK4135 Optimization and control
1 119

can evaluate food but not

minnfen afCng
simpler generalized triangle
a
N3 NG
En

I
1
Given initial simplex S M ok anti
singular materinVC
S is non nom nom nm n
is non
singular
it means no 3 points are on the same line

Let vertices are sorted

fam tf Cz E E f Cann

Define centroid I
in Gini

E.EE
i

nl 4 It44 n't n.n

worst ig
Msm worst
 N best
define line
silt I t t
Nnn E

OneiterationofMMCreplacinfant
G and th
Compute a
Freflection point
f f n

G GOTO
If fam s
f E
f Cnn Replace ann with a
andthen sort

else
if f c
fore
imm
Ieee Ii Em iE si
else replace ante with 2 te GO TO
else I
if f fCnn contraction

if f Cnn E
fi Ef Emm outfidontract
compute a Cts fig f EE t's point

replace ante with GOTO

o
If fqf f a
is
else
Cf f Cann
insidecontract
and
compute a
k f E f Elk point

replace ante with GOTO

If figff ante a
k
else
replace E 2,3
nil yzk.tn n n H shrink
complex
34

One iteration of Nelder-Mead

www.ntnu.no TTK4135 Optimization and control

35

One iteration of Nelder-Mead

Reflection

www.ntnu.no TTK4135 Optimization and control

36

One iteration of Nelder-Mead

Expansion

www.ntnu.no TTK4135 Optimization and control

37

One iteration of Nelder-Mead

Contraction (outside)

www.ntnu.no TTK4135 Optimization and control

38

One iteration of Nelder-Mead

Contraction (inside)

www.ntnu.no TTK4135 Optimization and control

39

One iteration of Nelder-Mead

Shrinkage

www.ntnu.no TTK4135 Optimization and control

Concierge
The
average
value
It f Cai decrease in

each iteration
possibly exceptforshrinkage

Termination
I fend f Cann LE

Not a
global optimalalgorithm depends where we
start with
40
Suited for problemswith limited decision
Examples variables complex function evaluations

wikipedia.org

www.ntnu.no TTK4135 Optimization and control

1
Lecture 17: Ne me h d f l i g
43
nonlinear equations (Ch. 11)
A brief summary of Ch. 10
Nonlinear equations
Ne me h d f l i g li ea e ai (Ch. 11)
Convergence
Merit functions

Reference: N&W Ch. 11-11.1

www.ntnu.no TTK4135 Optimization and control

2

Gradient and Jacobian

The gradient of a scalar function of several variables is

Say . We define the Jacobian as

the m by n matrix

www.ntnu.no TTK4135 Optimization and control

3

A brief aside: Nonlinear least squares (Ch. 10)

Consider the following problem: We have a number of (noisy) data

and want to fit the function

to the data

(Nonlinear) least squares formulation:

Outlierspresent not a
good idea to use
squareforerr
residual

Generalizations:
(Statistical) Machine Learning: Regression, or parametric learning
Control theory: System identification (fitting dynamic models to data)

www.ntnu.no TTK4135 Optimization and control

4

How to solve nonlinear least squares

residuals optimization
Formulation as unconstraind optimization problem (m>>n): variables

Say we want to use Ne method. We need gradient and

Hessian of objective function:
First find gradient of residuals :

Gradient of objective :

www.ntnu.no TTK4135 Optimization and control

5

Gauss-Newton method
For these problems, a good approximation of the Hessian is

The Gauss-Newton method for nonlinear least squares

problems: Use Ne method with this Hessian-
approximation
Note: Only first-order derivatives are needed!
Make it work far from solution: Use linesearch, Wolfe-conditions,
etc. (same as before)

(Using the same approximation with trust-region instead of linesearch is the

Levenberg-Marquardt algorithm implemented in Matlab-function lsqnonlin)

www.ntnu.no TTK4135 Optimization and control

6

Linear least squares

Say you want to fit a polynomial

to data
Define and least squares optimization problem

where the regressor matrix A is

Easy to show that solution is given from

Solve by Cholesky or (better) QR (see book 10.2)

Observe: The Gauss-Newton
approximation
www.ntnu.no is exact forOptimization
TTK4135 linear problems!
and control
Chile Manlineakeguations
N ne uz m
qCns O
Gla o M
kLn

i where am

knbr o Ig
Problemtosolue find ns.t.eecn
o
O

unit
r Cnt 0 can have no solution one
Tingle unique sol
or solutions
many
7

Why study nonlinear equations

Given nonlinear system , the steady state is found by
solving

Flowsheet analysis in
chemical engineering
(steady state simulators)

ModSim: Using implicit Runge-Kutta, we need to solve nonlinear

equations
Ne method for nonlinear equations is important for
deriving/analysing SQP methods (next time)
www.ntnu.no TTK4135 Optimization and control
Multidimensionality
rcmp roast F Cnttf pdt

model
of rcmatp
htnktpk t.skCp eeGua t Fenip

Milk o
pie FenderCme
In

I
em

1
2km Nk
Pk
Newton's algorithm nonlinear equations
for
choose no k 0
while Heilman E
Solue I lawfk rank
Nkt Nut pig
k KH
end
lathy called newton
minn fca fromligneous
To solve search
for ns.t
If Cn O
Jacobian Ca
of Ff
Cal
If
t
Pk IfCny IfCa Newton direction

the Convergence of Newton's method is quadratic

for real smooth
Proof scalar case
Newton's method May Kk teck
ri Ck

Taylor rcmp real th Ensp th that t

tpp

remain
rfk raffa

renal tithed f ef.cz

tee matte tn
2Mlink

sina.is
Tf
EM in a region around n

rCnka
T
EMkCn
 Potentialproblemaeithalevetonymeth
1
Computing Fema may be expensive andfor 7cal may
become not invertible
Solution Do as in quasi Newton ApproximateFlock using
acne and modify to avoid singularity

2 Ensure convergence whenstarting

farfrom solution by
using line search

No
objective functionfor line search
Solution8 define merit function

When Fein is non

singular
Pk Flink eeGue is a descent
Cn
direction
for f
PEFfCme RE FCnnTreme McnuttFlnKIF Tri
HUNT
Newton's algorithm nonlinear equations
for
choose no k O
while Hulman E
Solue I laidpie rank
Nkt Nkt pig
k late
end Mate Nkt xKpk
Taksatisfies
wolfecondnforfcn
8

Ne me h d

By Ralf Pfeifer - de:Image:NewtonIteration Ani.gif, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=2268473

www.ntnu.no TTK4135 Optimization and control

14 3 k 180
12 SU Peegramminf
today Local SAPfor equality constained NLP
En Minner M Nz St Metz 1 0

Lagrangian Lcn D m na Unita 1

KIT o
e l 2Mt

0 l 227 0

242 22 1 0

F na

a f to
LQcalsqpforegualitycanstainedNLP.fi
minne fan at cent 44
1,72

L CnD Cn XTCcn
f P Jacobian
Acnst 79Cn Fsln Cn
17cm

KKI FNL na Tf cm AGT1 0

Ccn 0

writing as a system of nonlinear gas

Final

Nonlinear
µ Yin
with
0

ntm equations and

yn system
ntm variables
Solve using Newton's method slide na 6
Newtonis method for solving FGsd O
Gillen No Xo
while Il FCnH Il 7 E
some
a o

Ii II Yy
qq.gqq.ua

K1 H
Iend

Theoreml6 rs Ax umption
The K KT system has a solution KKT matrin non singular
a ACn is a rank matrin LIC 9
full
b dt n for all d fo s t ACnk d 0
treedefinite on the nullspaceofAcn
This Newton as an excellent alg forsolving1 1 when
alfto
starting close six
AlternatiuederiuationofkktsystemT

e t Clark 0
min few
Approximate at Caa Aia

mpinep.fcnxltafcnkltpttzpttnntcnk.tk p
pY
st cand t ACK p O

Ifp f fcnedtaflnkjtptlg.pt172mLCneadp
lagrange
ITfunk tACnHp
multiplier

KKTconditions

Tp Ilp e FfGk Than Hnk p ACork Tl O

Clink tACnk p O
viii
i.a
cnn.gg
q tt d
Subtract Alak Tdk from both sides first line Kix
KKTsystem

1
ll 7 tl l
µ Xk

That is solving AP Given above gives pie and Xian

Tueainterpretationsofalgaritt y
analysisquadrate
conuoyenc
1 Newton's method for solving KKTsystem
2 sequential solution to Cfpapproximations ofCry
extensionto
inequalities

Cn At 914 0 i EE
Minne f
City O E C I

Given no Xo
Aff
while not convoyed
Salue Cfp
min fcnwt7fcnftptkptthnttnk.tk
s t City aCiEngTp O EEE
KE k 4 Cn t TheCnTp 70 i E I

µ my
 q
me

Reason
fGua t T
f WT p tIgpt172mLthis Xk p t th Cena t Akap

f Gud t XT e Cnk t 17
fcnkltptXIACnkptbzptnxnnslnk.tk

L CK da t Tha LthaXK t PTthanLENKXK p

quadraticapproximation oflagrangian
1

Lecture 18: Sequential Quadratic Programming

• Recap: Newton s method for solving nonlinear equations

• Recap: Equality-constrained QPs

• SQP for equality-constrained nonlinear programming problems

Reference: N&W Ch.18-18.1

www.ntnu.no TTK4135 Optimization and control

2

Types of constrained optimization problems

• Linear programming
Convex problem
Feasible set polyhedron

• Quadratic programming
Convex problem if
Feasible set polyhedron

• Nonlinear programming
In general non-convex!

www.ntnu.no TTK4135 Optimization and control

3

Nonlinear programming problem

• Example:

www.ntnu.no TTK4135 Optimization and control

4

The Lagrangian
For constrained functions, introduce modification of objective
function (the Lagrangian):

• Multipliers for equality constrains may have both signs in a

solution
• Multipliers for inequality constraints cannot be negative (cf.
shadow prices)
• For (inequality) constraints that are inactive, multipliers are zero

www.ntnu.no TTK4135 Optimization and control

5

KKT conditions (Theorem 12.1)

KKT-conditions (First-order necessary conditions): If is a

local solution and LICQ holds, then there exist such that

(stationarity)

(primal feasibility)

(dual feasibility)
(complementarity condition/
complementary slackness)

Either or

(strict complimentarity: Only one of them is zero)

www.ntnu.no TTK4135 Optimization and control
6

Newton s method for solving nonlinear equations (Ch. 11)

• Solve equation system
• Assume Jacobian exists and is continuous
• Taylor:

• (Local) convergence rate (Thm 11.2): Quadratic convergence if is

Lipschitz continuous (that is, very good convergence rate)

• If we set , then this method corresponds to Newton s

method for minimizing

www.ntnu.no TTK4135 Optimization and control

7

Equality-constrained QP (EQP)
Basic assumption:
A full row rank

• KKT-conditions (KKT system, KKT matrix):

• Solvable when (columns of Z basis for nullspace of A):

• How to solve KKT system (KKT matrix indefinite, but symmetric):

Full-space: Symmetric indefinite (LDL) factorization:
Reduced space: Use Ax=b to eliminate m variables. Requires computation
of Z, basis for nullspace of A, which can be costly. Reduced space method
can be faster than full-space if n-m ≪ n.
www.ntnu.no TTK4135 Optimization and control
8
Local SQP-algorithm for solving equality-
constrained NLPs

EQP:

www.ntnu.no TTK4135 Optimization and control

1
Lecture 19: Practical SQP algorithms for
13 nonlinear programming

Recap: Local SQP algorithms for equality-constrained NLPs

Extension to inequalities

Globalization of SQP-algorithms
makingthealgorithmwork away
far
Computation/approximation of the Hessian
fromoptimum
Linesearch
Other issues
Infeasible linearized constraints
The Maratos effect

Reference: N&W Ch. 18.2, 18.3, 15.4

www.ntnu.no TTK4135 Optimization and control

2

Ne eh df i g i ea e ai (Ch. 11)
Solve equation system
Assume Jacobian exists and is continuous
Taylor:

(Local) convergence rate (Thm 11.2): Quadratic convergence if is

Lipschitz continuous (that is, very good convergence rate)

www.ntnu.no TTK4135 Optimization and control

3

Equality-constrained NLP
Lagrangian:
KKT-system:

T e: U e Ne eh df i ea e ai KKT-system:

where

Consider this quadratic approximation to the objective (or Lagrangian):

KKT:

If we let , it is clear that the two KKT systems give

equivalent solutions
Newton-viewpoint: quadratic convergence locally
QP-viewpoint: provides a means for practical implementation and extension to inequality constraints
Assumptions for the above: 1) full row rank (LICQ),
2) on tangent space of constraints

www.ntnu.no TTK4135 Optimization and control

4

Local SQP-algorithm for solving NLPs

Only equality constraints:

With inequality constraints (IQP method):

Thm 18.1: Alg. 18.1 identifies (eventually) the optimal active set of constraints (under
a i ). Af e , i beha e ike Ne eh df e ai c ai ed be .

A e a i e (EQP e h d): Mai ai a ki g e (a i ai f he ac i e e )

in Alg. 18.1, solve equality-constrained QP in each iteration. May be more efficient for
large-scale problems.
www.ntnu.no TTK4135 Optimization and control
 Globalization inake it work far from optimum
Probed unconstrained
minnfans

quadratic minpfcneelttfcnedtptlg.ptEfcmalp
approximation
pre HyenasHfW

Validdir Descentdetection Thfence 70

Have 1
Modify the hessian
cue ensure
reawadirn 4 Approximate thehessianusing BEGS

line search Measure progress in fens akee nut

gaffe
I mask
howfar
shouldwego
 Constrained case
s t City 0 ice
min fans
City 20 i EI
quadratic mipnfcnajtafcndtptfzpTILCY.tk
nu p
problem s t
city t FCilnsTp O ie E

canst 71CityTp 20 iEI

Valid dina 7 Cfp well defined BEN L node 0 on tangent

spaceofconstraintsfaced

How 1 17miLEK Xia or

Modify
we ensure
valid dir
4 Approximate twitches k using BEGS

linesearch Measure progress inobjective andconstraint

Mate nictakpk T
uselmeleitfunction

dimenitfunction
O Cn m f Cnt tu feel city ME ions

µ penalty parameter
EET max 0 2

ME is not differentiable
Defis A function 0Cn µ is exact
0 the local
if for any µ I let solution
of theMLP
is a minimizer of 0

Third is exact with µ mail I I i E E UI

In implementation start with µ small and increase

gradually
Assume I 0 then 0 Cn µ flattell cCulll
We use merit funetoon as an alternate tee objectivefun
when we use line search be 02 algorithm allows iterates to
violate the constraints
we need directional derivative 70 Cai Miftp.k
But is not defined everywhere
Instead use D l 0 Cnn Mk Aa
Itf
iIT41

Theoremis Pk and Aq from 9 P

then
B 0 LakeMk bis E pie thatcnn.tkPk CathHalla 112k
That 9
fee is descent direction if
1 than f Cma Xk 70 and 2 µ Il XianIl a
 111111
largestabsolute value
I 11411 is sum
ofabsolutevalues
Linesearch o
Nkt Matak pk
Xk should
fulfill
0 lnktaapk.ua E0ilnk Mk ty Dl0iGkMk pk

compare Istuealfe condition flnatakpk EfCnk t4M7fWTpk

5

Quasi-Newton for unconstrained problems

termination criteria

www.ntnu.no TTK4135 Optimization and control

6

Quasi-Newton for SQP

SQP needs Hessian of Lagrangian, but this require second derivatives
of objective and constraints, which may be expensive
Quasi-Newton (BFGS) very successful for unconstrained optimization
can we do the same in the constrained case?
Unconstrained case: Constrained case:

Problem: BFGS gives positive definite Hessian approximation, while

Hessian of Lagrangian is not necessarily positive definite (not even
close to a solution). That is, the approximation may be bad.

P ib e i :A i a e ed ced He ia (He ia
nullspace of constraints) instead. This reduced Hessian is much more
likely to be positive definite (recall sufficient conditions).

www.ntnu.no TTK4135 Optimization and control

7
Line search Merit function
G ba i a i
How far to walk along p? Linesearch (or trust region)!
Unconstrained optimization: The Armijo (Wolfe) condition ensure
sufficient decrease of objective function:

Constrained optimization: Must check both objective and constraint!

Merit function (for line search): Function that measure progress in both:
l1 merit function:
Fifune
howmuch
nailated
oast

Thm 18.2:
That is: pk is a descent direction for merit function if Hessian of Lagrangian is positive
definite and µ is large enough

www.ntnu.no TTK4135 Optimization and control

8

A practical line search SQP method

Backtrackinglinesearch

www.ntnu.no TTK4135 Optimization and control

9

Maratos effect
Maratos effect: A merit function may reject good steps!
Ex. 15.4:

pk good step even if

both objective and
constraint violation
increase

Remedy:
Use a merit function that does not suffer from the Maratos effect
U e - e a eg ( e ai a i c ea e i ei f ci )
U e ec d- de c ec i ( he Maratos effect occurs)
www.ntnu.no TTK4135 Optimization and control
10

NLP software
SNOPT
e large-scale linear and nonlinear problems; especially recommended if some of the
constraints are highly nonlinear, or constraints respectively their gradients are costly to evaluate
and second derivative information is unavailable or hard to obtain; assumes that the number of
f ee variables is modest.
Licence: Commercial Stanford
IPOPT
i ei i e h d f a ge- ca e NLP
License: Open source (but good linear solvers might be commercial) corn
WORHP
interior ft
SQP solver for very large problems, IP at QP level, exact or approximate second derivatives,
various linear algebra options, varius interfaces
Licence: Commercial, but free for academia
KNITRO
trust region interior point method, efficient for NLPs of all sizes, various interfaces
License: Commercial
( a d several others, including fmincon in Matlab Optimization Toolbox)

«Decision tree for optimization software»: http://plato.asu.edu/sub/nlores.html

www.ntnu.no TTK4135 Optimization and control

11

Example: optimization using CasADi

CasADi (https://casadi.org/)
CasADi is a symbolic framework for numeric optimization implementing automatic
differentiation in forward and reverse modes on sparse matrix-valued computational graphs.
i e face to IPOPT/BONMIN, BlockSQP, WORHP, KNITRO and SNOPT

rosenbrock.m
import casadi.*

% Create NLP: Solve the Rosenbrock problem:

% minimize x^2 + 100*z^2
% subject to z + (1-x)^2 - y == 0
x = SX.sym('x');
y = SX.sym('y');
Define variables z = SX.sym('z');
v = [x;y;z];
f = x^2 + 100*z^2;
Define objective and constraints g = z + (1-x)^2 - y;
nlp = struct('x', v, 'f', f, 'g', g);

% Create IPOPT solver object

Create solver object solver = nlpsol('solver', 'ipopt', nlp);

% Solve the NLP

Solve the opt problem res = solver('x0' , [2.5 3.0 0.75],... % solution guess
'lbx', -inf,... % lower bound on x
'ubx', inf,... % upper bound on x
'lbg', 0,... % lower bound on g
'ubg', 0); % upper bound on g

% Print the solution

f_opt = full(res.f) % >> 0
x_opt = full(res.x) % >> [0; 1; 0]
lam_x_opt = full(res.lam_x) % >> [0; 0; 0]
lam_g_opt = full(res.lam_g) % >> 0

www.ntnu.no TTK4135 Optimization and control

1
3119
Lecture 20: NMPC and summing up

Course in a nutshell:
Unconstrained optimization
Steepest descent, Newton, Quasi-Newton
Globalization (line-search and Hessian modification), derivatives
Constrained optimization
Optimality conditions, KKT
Linear programming: SIMPLEX
Quadratic programming: Active set method
Nonlinear programming: SQP
Control and optimization
LQ control
MPC
Today: Nonlinear MPC, e ac ica /i d ia i e MPC

Reference: F&H 4.5, 4.6

Q&A session: Thursday 30 May? (Exam: Monday 03 June)
www.ntnu.no TTK4135 Optimization and control
2

Line-search unconstrained optimization

1. Initial guess x0
2. While termination criteria not fulfilled
a) Find descent direction pk from xk
b) Find appropriate step length αk ; set xk+1 = xk + αk pk
c) k = k+1 A comparison of steepest
descent and Newton's method.
3. xM = x*? (possibly check sufficient conditions for optimality) Newton's method uses
curvature information to take a
more direct route.
(wikipedia.org)

Termination criteria: Descent directions: Step length (Wolfe):

Stop when first of these become true: Steepest descent
(necessary condition)
(no progress) Newton
(no progress)
(kept on too long) Quasi-Newton

www.ntnu.no TTK4135 Optimization and control

3

Quasi-Newton: BFGS method

www.ntnu.no TTK4135 Optimization and control

4

Example (from book)

Using steepest descent, BFGS (Quasi-Newton) and inexact
Newton on Rosenbrock function

Iterations from starting point (-1.2,1):

Steepest descent: 5264
BFGS: 34
Newton: 21

wikipedia.org

Last iterations; value of

www.ntnu.no TTK4135 Optimization and control

5

Types of constrained optimization problems

Linear programming
Convex problem
Feasible set polyhedron

Quadratic programming
Convex problem if
Feasible set polyhedron

Nonlinear programming
In general non-convex!

www.ntnu.no TTK4135 Optimization and control

6

KKT conditions (Theorem 12.1)

KKT-conditions (First-order necessary conditions): If is a

local solution and LICQ holds, then there exist such that

(stationarity)

(primal feasibility)

(dual feasibility)
(complementarity condition/
complementary slackness)

Starting point for all algorithms for constrained optimization in this course!

www.ntnu.no TTK4135 Optimization and control

7

Linear programming, standard form and KKT: recap

LP:

LP, standard form:

Lagrangian:

KKT-conditions (LPs: necessary and sufficient for optimality):

www.ntnu.no TTK4135 Optimization and control

8

Check KKT-conditions for BFP

Given BFP , and corresponding basis . Define

Partition , and :

KKT conditions
KKT-2: (since x is BFP)
KKT-3: (since x is BFP)
KKT-5: if we choose
KKT-1:

KKT-4: Is ?

If , then the BFP fulfills KKT and is a solution

If not, change basis, and try again
E.g. pick smallest element of sN (index q), increase xq along Ax=b until xp becomes zero.
Move q from to , and p from to . This guarantees decrease of objective, and no
c c i g (if -degenerate).
www.ntnu.no TTK4135 Optimization and control
9

www.ntnu.no TTK4135 Optimization and control

10

General QP problem

Lagrangian

KKT conditions
General: Defined via active set:

www.ntnu.no TTK4135 Optimization and control

11

Active set method for convex QP

www.ntnu.no TTK4135 Optimization and control

12

Example 16.4

www.ntnu.no TTK4135 Optimization and control

13

Equality-constrained NLP
Lagrangian:
KKT-system:

T e: U e Ne eh df i ea e ai KKT-system:

where

Consider this quadratic approximation to the objective (or Lagrangian):

KKT:

If we let , it is clear that the two KKT systems give

equivalent solutions
Newton-viewpoint: quadratic convergence locally
QP-viewpoint: provides a means for practical implementation and extension to inequality constraints
Assumptions for the above: 1) full row rank (LICQ),
2) on tangent space of constraints

www.ntnu.no TTK4135 Optimization and control

14

Local SQP-algorithm for solving NLPs

Only equality constraints:

With inequality constraints (IQP method):

Thm 18.1: Alg. 18.1 identifies (eventually) the optimal active set of constraints (under
a i ). Af e , i beha e i e Ne eh df e ai c ai ed be .

www.ntnu.no TTK4135 Optimization and control

15

A practical line search SQP method

www.ntnu.no TTK4135 Optimization and control

16

NLP software
SNOPT
e large-scale linear and nonlinear problems; especially recommended if some of the
constraints are highly nonlinear, or constraints respectively their gradients are costly to evaluate
and second derivative information is unavailable or hard to obtain; assumes that the number of
f ee variables is modest.
Licence: Commercial
IPOPT
i ei i e h d f a ge- ca e NLP
License: Open source (but good linear solvers might be commercial)
WORHP
SQP solver for very large problems, IP at QP level, exact or approximate second derivatives,
various linear algebra options, varius interfaces
Licence: Commercial, but free for academia
KNITRO
trust region interior point method, efficient for NLPs of all sizes, various interfaces
License: Commercial
( a d several others, including fmincon in Matlab Optimization Toolbox)

«Decision tree for optimization software»: http://plato.asu.edu/sub/nlores.html

www.ntnu.no TTK4135 Optimization and control

17

Example: optimization using CasADi

CasADi (https://casadi.org/)
CasADi is a symbolic framework for numeric optimization implementing automatic
differentiation in forward and reverse modes on sparse matrix-valued computational graphs.
i e face to IPOPT/BONMIN, BlockSQP, WORHP, KNITRO and SNOPT

rosenbrock.m
import casadi.*

% Create NLP: Solve the Rosenbrock problem:

% minimize x^2 + 100*z^2
% subject to z + (1-x)^2 - y == 0
x = SX.sym('x');
y = SX.sym('y');
Define variables z = SX.sym('z');
v = [x;y;z];
f = x^2 + 100*z^2;
Define objective and constraints g = z + (1-x)^2 - y;
nlp = struct('x', v, 'f', f, 'g', g);

% Create IPOPT solver object

Create solver object solver = nlpsol('solver', 'ipopt', nlp);

% Solve the NLP

Solve the opt problem res = solver('x0' , [2.5 3.0 0.75],... % solution guess
'lbx', -inf,... % lower bound on x
'ubx', inf,... % upper bound on x
'lbg', 0,... % lower bound on g
'ubg', 0); % upper bound on g

% Print the solution

f_opt = full(res.f) % >> 0
x_opt = full(res.x) % >> [0; 1; 0]
lam_x_opt = full(res.lam_x) % >> [0; 0; 0]
lam_g_opt = full(res.lam_g) % >> 0

www.ntnu.no TTK4135 Optimization and control

18

Model predictive control

model-based
optimizer Process

desired
outputs inputs outputs

measurements

A model of the process is used to compute the control signals

(inputs) that optimize predicted future process evolution

www.ntnu.no TTK4135 Optimization and control

19

Operational hierarchy before and after MPC

www.ntnu.no TTK4135 Optimization and control

20
S ee e a d hif
How advanced control (MPC) improves economy
Ac i e c ai

Backoff Backoff = loss (lower throughput,

off-spec product, ...)

Time

Squeeze
(e.g. MPC)

Shift

www.ntnu.no TTK4135 Optimization and control

21
Embedded Model Predictive Control
PhD project Giorgio Kufoalor

www.ntnu.no TTK4135 Optimization and control

22

Main contributions to fill the gap

PhD project Giorgio Kufoalor

Step-response MPC on ultra-reliable, resource

constrained, industrial hardware
Detailed study on MPC formulations and solver methods to achieve fast and
reliable solutions
Achieve significant savings, both in
computations and memory usage
Exploiting problem structure and
specifics of computing platform
A a ic c de ge e a i (a )

Development of new multistage QP framework, tailored to step-

response MPC
All extensively tested on a realistic subsea compact separation
simulator using hardware-in-the-loop testing

D. K. Kufoalor, G. Frison, L. Imsland, T. A. Johansen, J. B. Jørgensen, Block Factorization of Step Response Model Predictive Control Problems, J. Process Control, Vol. 53, May, pp. 1 14, 2017;
D. K. M. Kufoalor, S. Richter, L. Imsland, T. A. Johansen, Enabling Full Structure Exploitation of Practical MPC Formulations for Speeding up First-Order Methods, 56th IEEE Conference on Decision and Control, 2017
D. K. M. Kufoalor, T. A. Johansen, L. S. Imsland, Efficient Implementation of Step Response Models for Embedded Model Predictive Control, Computers & Chemical Engineering, Volume 90, July, Pages 121 135, 2016
D. K. M. Kufoalor, V. Aaker, L. S. Imsland, T. A. Johansen, G. O. Eikrem, Automatically Generated Embedded Model Predictive Control: Moving an Industrial PC-based MPC to an Embedded Platform,
J. Optimal Control - Applications and Methods, vol. 36, pp. 705 727, 2015

www.ntnu.no TTK4135 Optimization and control

23

Linear MPC; open loop dynamic optimization

www.ntnu.no TTK4135 Optimization and control

24

Model predictive control principle

www.ntnu.no TTK4135 Optimization and control

 Nanak rape
minz 7 St Catti Ut

gCnt Ut
S't atte E 0 My
h Che Ue
Yt
E e high
yean ye g
alone e ut uhigh

Stage cost St
Sz Catti Ut Antti t UIRut
L
YI
Regulation

Chen
T
neg Utaref

ft
Tracking Ss Cnt Ut Ig hktm.ee yt78T9lhhaiHyII
Euereto TRu ueref
k

Economic St Cat Ut That

ye
25

audio The three ways of implementing NMPC

g45
F Sequential (single shooting) methods
Have only inputs (control moves) as optimization variables, simulate to

I
calculate objective and state constraints (and gradients)
Re i a i i ai be ih c e
Standard SQP methods are suitable
Simultaneous methods
Have both inputs and states as optimization variables, include model as
equality constraints
Results in huge optimization problem, but equality constraints are very
structured ( a e , a f e )
Must use solvers that exploit structure (e.g. IPOPT)
Inbetween: Multiple shooting
Divide horizon into b-h i , e i g e-shooting on each sub-horizon
a d add e a i c ai g e each b-horizon together
Re i b c - c ed i i ai be
Ideally use solvers that exploit this structure (but not many exists)

Wha i be ? De e d
www.ntnu.no TTK4135 Optimization and control
26

NMPC example: van der Pol

Controlled van der Pol oscillator

Discretization (here: Euler)

In general, discretization scheme important: more accurate discretization
may make it possible to use longer sample intervals (however, several
aspects influences this decision)

Stability dependent on horizon length

I a ce f a - a

Implementation: Tor Aksel Heirung

www.ntnu.no TTK4135 Optimization and control

27

Output feedback MPC

www.ntnu.no TTK4135 Optimization and control

 outlputfeedbadampcc.cn onlineaeeMPc

t.to
I

te gCnI.ue tkCyt ifz

if h Cine uit
E KT
entendedkalmanfilter
28
MPC and feasibility
Is there always a solution to the MPC open-loop optimization
problem?
Not necessarily state (or output) constraints may become
infeasible, for example after a disturbance
P ac ica i :S f c ai ( e ac e a f ai )
S fe ( ea ) aec ai b addi g ac a iab e

Audio
01
1 32

www.ntnu.no TTK4135 Optimization and control

29

Complexity reduction strategies in MPC

Input blocking (or move blocking) reduce number of QP variables

I cide i reduce number of QP constraints

Only check constraints at certain time instants, rather than at all times on horizon

www.ntnu.no TTK4135 Optimization and control

 Cybernetica
• Cybernetica provides advanced model-based control systems for the process
industry
– Based on non-linear first principles (mechanistic) models
– Nonlinear state- and parameter estimation (EKF, MHE)
– Online dynamic optimization (nonlinear model predictive control, NMPC)
Process physics

Model fit for purpose

CENIT
x : Model states Kernel
Model Plant
Plant personnel
dx y : Measurements Predictive
f x, u, , v Controller
dt z : Output variables
Model OPC
y g x, , w u : Input variables
Component
z h x, v : Process disturbances
w : Measurement noise
: Model parameters Estimator
Process data

30
 #4
Åsgard
Norne #7
Heidrun
Snøhvit

Total in Statoil:
92 MPC Applications

#28
Mongstad

#5
Kollsnes
Gullfaks/Tordis #2

#25
Kårstø
#21
Kalundborg
Stig Strand, Statoil