Chalmers University of Technology/Gothenburg University Optimisation

Mathematical Sciences Course information 2019

Emil Gustavsson August 28, 2019

TMA947 Nonlinear optimisation, 7.5 credits

MMG621 Nonlinear optimisation, 7.5 credits
The purpose of this basic course in optimization is to provide
(I) knowledge of some important classes of optimization problems and of application
areas of optimization modelling and methods;
(II) practice in describing relevant parts of a real-world problem in a mathematical opti-
mization model;
(III) an understanding of necessary and sufficient optimality criteria, of their consequences,
and of the basic mathematical theory upon which they are built;
(IV) examples of optimization algorithms that are naturally developed from this theory,
their convergence analysis, and their application to practical optimization problems.
EXAMINER: Michael Patriksson, professor of applied mathematics, Matematiska Veten-
skaper, room L2084; tel: 772 3529; e-mail: [email protected]

LECTURER/COURSE RESPONSIBLE: Emil Gustavsson, PhD, Business area leader

within Machine Learning, Data Science, and Optimization, Fraunhofer-Chalmers Centre,
http://www.fcc.chalmers.se, e-mail: [email protected]

EXERCISE ASSISTANT: Edvin Åblad, PhD student, Matematiska Vetenskaper and

Fraunhofer-Chalmers Centre; e-mail: [email protected]

EXERCISE ASSISTANT: Quanjiang Yu, PhD student, Matematiska Vetenskaper,

room L2033; tel: 772 1094; e-mail: [email protected]

Course presentation

CONTENTS: The main focus of the course is on optimization problems in continuous

variables; it builds a foundation for the analysis of an optimization problem. We can
roughly separate the material into the following areas:
Convex analysis: convex set, polytope, polyhedron, cone, representation theorem, ex-
treme point, Farkas Lemma, convex function

Optimality conditions and duality: global/local optimum, existence and uniqueness

of optimal solutions, variational inequality, Karush–Kuhn–Tucker (KKT) conditions,
complementarity conditions, Lagrange multiplier, Lagrangian dual problem, global
optimality conditions, weak/strong duality

1
Linear programming (LP): LP models, LP algebra and geometry, basic feasible so-
lution (BFS), the Simplex method, termination, LP duality, optimality conditions,
strong duality, complementarity, interior point methods, sensitivity analysis

Nonlinear optimization methods: direction of descent, line search, (quasi-)Newton

method, Frank–Wolfe method, gradient projection, exterior and interior penalty,
sequential quadratic programming

We also touch upon other important problem areas within optimization, such as integer
programming and network optimization.

PREREQUISITES: Passed courses on analysis (in one and several variables) and lin-
ear algebra; familiarity with matrix/vector notation and calculus, differential calculus.
Reading Chapter 2 in the book (i) below provides a partial background, especially to the
mathematical notation used and most of the important basic mathematical terminology.

ORGANIZATION: Lectures, exercises, a project assignment, and computer exercises.

COURSE LITERATURE:

(i) An Introduction to Continuous Optimization, 3rd edition by N. Andréasson, A. Ev-

grafov, E. Gustavsson, Z. Nedělková, M. Patriksson, K. C. Sou, and M. Önnheim,
published by Studentlitteratur in 2016 and found in the Cremona book store
(ii) Hand-outs from books and articles

COURSE REQUIREMENTS: The course content is defined by the literature references

in the plan below.

EXAMINATION:

• Written exam (first opportunity 31/10, 8.30–13.30)—gives 6 credits

• Project assignment—gives 1.5 credits
• Two correctly solved computer exercises

BONUS SYSTEM:

• Active participation during exercises gives at most 2 bonus points

• The bonus points are valid one year

COURSE EVALUATION: Three meetings between the Examiner and randomly se-
lected course representatives will be organized. All students will be asked to fill a ques-
tionnaire.

2
SCHEDULE:

Lectures: on Mondays 08.00–09.45 and Tuesdays 15.15–17.00. Lectures are given in En-
glish. For locations, see the schedule below.
Exercises: on Mondays 10.00–11.45 and Fridays 08.00–09.45 in two parallel groups. For
locations, see the schedule below.
Project: Teachers are available for questions in the computer rooms, which are also
booked for work on the project, on 3/10 (room: MVF25) at 15.15–19.00. (Pres-
ence is not obligatory.) At other times, work is done individually. Deadline for
handing in the project model (part 1): 27/9. Deadline for handing in the project
report (part 2): 11/10.
Computer exercises: The computer exercises are scheduled to take place when also
teachers are available, Computer exercise 1 on 19/9, 26/9 and Computer exercise
2 on 10/10 and 17/10 (room booked: MVF25), and on all occasions at 15.15–19.00.
(Presence is not obligatory.) The computer exercises can be performed individually,
but preferably in groups of two (and strictly not more than two). Deadline for
handing in the report, unless passed through oral examination on site during the
scheduled sessions: 3/10 (Computer exercise 1), 20/10 (Computer exercise 2).
Important note: The computer exercises require at least one hour of preparation
each; having done that preparation, two–three hours should be enough to complete
an exercise by the computer.

Information about the project and computer exercises are found on the web page
http://www.math.chalmers.se/Math/Grundutb/CTH/tma947/1920/

This course information, the course literature, project and computer exercise materials,
most hand-outs and previous exams will also be found on this page.

COURSE PLAN, LECTURES:

Le 1 (3/9) Course presentation. Subject description. Week 1

Course map. Applications. Notations.
Optimization modelling. Modelling. Problem analysis. Classification.
(i): Chapter 1, 2

Le 2 (9/9) Convexity. Convex sets and functions. Polyhedra. The Representation Week 2
Theorem. Fourier elimination. Farkas’ Lemma.
(i): Chapter 3

Le 3 (10/9) Optimality conditions, introduction. Local and global optimality. Existence

of optimal solutions. Feasible directions. Necessary and sufficient conditions for local or
global optimality when the feasible set is convex. The Separation Theorem.
(i): Chapter 4

Le 4 (16/9) Unconstrained optimization methods. Search directions. Line searches. Week 3

3
Termination criteria. Steepest descent. Derivative-free methods.
(i): Chapter 11

Le 5 (17/9) Optimality conditions. Introduction to the primal–dual optimality condi-

tions. Geometric optimality conditions. The Fritz John optimality conditions.
(i): Chapter 5.1–5.4

Le 6 (23/9) The Karush–Kuhn–Tucker conditions. Constraint qualifications. The Week 4

Karush–Kuhn–Tucker conditions: necessary and sufficient conditions for local or global
optimality.
(i): Chapter 5.5–5.9

Le 7 (24/9) Convex duality. The Lagrangian dual problem. Weak and strong duality.
Obtaining the primal solution.
(i): Chapter 6

Le 8 (30/9) Linear programming. Introduction to linear programming. Modelling. Ba- Week 5

sic feasible solutions and extreme points (algebra versus geometry in linear programming).
The simplex method, introduction.
(i): Chapter 7, 8

Le 9 (1/10) Linear programming, continued. The Simplex method. The revised Simplex
method. Phase I and II. Degeneracy. Termination. Complexity.
(i): Chapter 9

Le 10 (7/10) Linear programming duality. Sensitivity analysis. Week 6

(i): Chapter 10

Le 11 (8/10) Convex optimization. Optimality conditions over convex sets. Subgradient

methods.
(i): Chapter 3, 4.4, 6.4

Le 12 (14/10) Integer programming. Applications. Modelling. Week 7

(ii): On integer programming

Le 13 (15/10) Nonlinear optimization methods: convex feasible sets. The gradient pro-
jection method. The Frank–Wolfe method. Simplicial decomposition. Applications.
(i): Chapter 12

4
Le 14 (21/12) Nonlinear optimization methods: general sets. Penalty and barrier meth- Week 8
ods. Interior point methods for linear programming, orientation.
(i): Chapter 13

Le 15 (22/12) An overview of the course.

5
COURSE PLAN, EXERCISES:

Ex 1 (6/9) Modelling. Week 1

(i): Chapter 1

Ex 2 (9/9) Convexity. Polyhedra. Representation. Farkas’ Lemma. Week 2

(i): Chapter 3

Ex 3 (13/9) Local and global minimum. Feasible sets. Optimality conditions. Weier-
strass’ Theorem. Separation.
(i): Chapter 4

Ex 4 (16/9) Unconstrained optimization. Week 3

(i): Chapter 11

Ex 5 (20/9) The KKT conditions.

(i): Chapter 5

Ex 6 (23/9) Lagrangian duality. Week 4

(i): Chapter 6

Ex 7 (27/9) Geometric solution of LP problems. Standard form. The geometry of the

Simplex method. Basic feasible solution.
(i): Chapters 7, 8

Ex 8 (30/9) The Revised Simplex method. Phase I & II.

(i): Chapter 9

Ex 9 (4/10) Duality in linear programming. Week 5

(i): Chapter 10.1–10.4

Ex 10 (7/10) Sensitivity analysis in linear programming.

(i): Chapter 10.5

Ex 11 (11/10) Subgradient optimization methods. Week 6

(i): Chapter 6.4

6
Ex 12 (14/10) Algorithms for convexly constrained optimization. Week 7
The Frank–Wolfe and simplicial decomposition algorithms.
(i): Chapter 12

Ex 13 (18/10) Constrained optimization methods. Penalty methods. Repetition.

(i): Chapter 13

Ex 14 (21/10) Old exam. Week 8