Scribd
Upload
5 views

Lecture 4 Convex Optimization

lecture notes

Uploaded by

funtwang
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
5 views

Lecture 4 Convex Optimization

lecture notes

Uploaded by

funtwang
Copyright
© © All Rights Reserved
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 24

Convex Optimization

Nicholas Ruozzi
University of Texas at Dallas
General Optimization

subject to:

2
General Optimization

is not necessarily convex


subject to:

3
General Optimization

subject to: Constraints do not need to


be linear

4
Example

subject to:

5
Example

subject to:

6
Convex Optimization

subject to:

In a convex optimization problem must be convex and must be


affine, i.e.,

7
Convex Optimization

subject to:

The constraints form a convex set: if both satisfy all of the


constraints, then so does any for

8
Local and Global Optima
• Every local optimum of a convex optimization problem is a
global optimum: let be a local minimum, a global minimum,
and

𝑦
𝑅
𝑥

9
Local and Global Optima
• Every local optimum of a convex optimization problem is a
global optimum: let be a local minimum, a global minimum,
and

is a local minimum
implies that there
exists a radius such
𝑦 that for all points
within the ball of
𝑅 radius of ,
𝑥

10
Local and Global Optima
• Every local optimum of a convex optimization problem is a
global optimum: let be a local minimum, a global minimum,
and

Let

𝑦
then
𝑅
𝑧
𝑥

11
Local and Global Optima
• Every local optimum of a convex optimization problem is a
global optimum: let be a local minimum, a global minimum,
and

By convexity,

𝑦
And
𝑅
𝑧
𝑥 So, , a contradiction

12
Local Optima

• For convex differentiable functions, is a local minimum of over


a convex set if the directional is nonnegative along any direction
inside the constraint set

• For all ,

• Note that if for some , then it must correspond to a descent


direction, i.e., if we take a small enough step in this direction,
the function will decrease in value

13
Linear Programming Problems
For

subject to:

14
Quadratic Programming Problems
For

subject to:

Recall must be positive


semidefinite for this to
be a convex optimization
problem
15
Least Squares Regression
• Given data points and , find the best fit line

16
Least Squares Regression
• Given data points and , find the best fit line

This is a convex optimization problem, why?

17
Least Squares Regression
• Given data points and , find the best fit polynomial of degree

This is a convex optimization problem, why?

18
Projections

• Given a set , the projection of a point onto is the point such


that the distance between and is less than or equal to the
distance between and any other

𝑧 𝑥

19
Projections

• Given a set , the projection of a point onto is the point such


that the distance between and is less than or equal to the
distance between and any other

20
Projections

• Given a set , the projection of a point onto is the point such


that the distance between and is less than or equal to the
distance between and any other

Convex if is a convex set, e.g., is a line or a plane

21
Projections

• Given a set , the projection of a point onto is the point such


that the distance between and is less than or equal to the
distance between and any other

Can project under different notions of distance as well

22
Smallest Enclosing Ball

• Find the ball of smallest radius that encloses a collection of


points

23
Smallest Enclosing Ball

• Find the ball of smallest radius that encloses a collection of


points

such that for


and

24

You might also like