Scribd
Upload
9 views

Optimization

Uploaded by

Thanh Trúc Lê Nguyễn
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
9 views

Optimization

Uploaded by

Thanh Trúc Lê Nguyễn
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 8

Optimization

Wednesday, October 15, 2014 2:25 PM

===============================================
KEY CONCEPTS OF OPTIMIZATION
============================
MATHEMATICAL BACKGROUND

---------------------------------------------
Optimization and Root Finding

Root Finding = Optimization = GUESSING and SEARCHING for a POINT on a Function


• Root Finding = Searching for ZEROS
• Optimization = Searching for MINIMUM and MAXIMUM

---------------------------------------------
Mathematical Definition of Optimization
Given
might be defined by (optional):
•
•
Optimization = FINDING such that is Maximum or Minimum
• is n-dimensional DESIGN VECTOR
• is OBJECTIVE FUNCTION
• are INEQUALITY CONSTRAINTS ( are constants)
• are EQUALITY CONSTRAINTS ( are constants)

---------------------------------------------
Classification of Optimization
*Linear vs Quadratic vs Nonlinear*
Constraints Optimization Form
Linear Linear LINEAR PROGRAMMING
Quadratic Linear QUADRATIC PROGRAMMING
Nonlinear NONLINEAR PROGRAMMING

*Constrained vs Unconstrained*
Constrained Optimization
Unconstrained Optimization

===============================================
Theoretical Model in Computing Page 1
===============================================
ONE-DIMENSIONAL UNCONSTRAINED OPTIMIZATION
============================
KEY CONCEPTS

---------------------------------------------
Multimodal Function

• Optima can occur many times throughout the interval

*3 ways to distinguish global from local optimum*


1. Graph the function
2. Use randomly generated starting guess pick the largest optima
3. Perturbing starting point see if routine results better optimum

---------------------------------------------
Unimodal Function

============================
OPTIMIZATION METHODS

---------------------------------------------
Overall Strategy for Unimodal function
1. Pick initial two points to BRACKET extremum
2. Pick third point WITHIN interval (to determine whether maximum occurred)
3. Pick fourth point to determine whether maximum occurs in first three or last three points.

*Should choose intermediate points to MINIMIZE FUNCTION EVALUATION by REPLACING OLD


values with NEW values*

---------------------------------------------
Golden-Section Search

Theoretical Model in Computing Page 2


•
•

*Algorithm for Golden-Section Search*

1. Pick two initial guesses


2. Select two interior points

3. Evaluate function values at


4. Find new values for
a.
b.
5. Calculate new
6. Start new iteration
a.

*Example: find maximum of *

Theoretical Model in Computing Page 3


---------------------------------------------
Parabolic Interpolation
Use the parabolic interpolation between three initial points to guess the maximum

*Algorithm is similar to Golden-Section Search, but , so they must


be compared carefully*

*Example: find maximum of *

---------------------------------------------
Newton's Method

Newton's Method is used to solve

Theoretical Model in Computing Page 4


*Example: find maximum of *

===============================================
MULTI-DIMENSIONAL UNCONSTRAINED OPTIMIZATION
============================
KEY CONCEPTS

---------------------------------------------
Classification of Multi-dimensional optimization techniques
Two classes of Multi-dimensional techniques:
1. Gradient / Descent / Ascent Method = require DERIVATIVE EVALUATION
a. Direct Calculation from Derivative
b. Steepest Ascent Method
2. Non-gradient / Direct Method = NOT require derivative evaluation
a. Random Search
b. Univariate
c. Pattern

============================
DIRECT METHODS

---------------------------------------------
Random Search
*Mechanism = based on EVALUATION FUNCTION at RANDOMLY selected values of INDEPENDENT
VARIABLES*

Advantages • Work with DISCONTINUOUS, NONDIFFERENTIABLE functions


• Always find GLOBAL OPTIMUM
Disadvantages • Onerous with large number of independent variables
• NOT EFFICIENT

---------------------------------------------
Univariate Search
*Mechanism = change ONE variable at a time sequence of one-dimentional searches*

Theoretical Model in Computing Page 5


---------------------------------------------
Pattern Direction Search
*Mechanism = shoot directly along ridge towards maximum*

============================
GRADIENT METHODS

---------------------------------------------
Tool: Gradients
*Gradient is similar to First-Derivative in one-dimensional functions*

*Meaning of Gradients*
• "What DIRECTION is steepest ascend?"

• "How much we will GAIN by taking that step?"

---------------------------------------------

Theoretical Model in Computing Page 6


---------------------------------------------
Tool: Hessians
*Hessian is similar to Second-Derivative in one-dimensional functions*

*Meaning of Hessians*
• Evaluate whether current point is Maximum or Minimum
○

○
○

---------------------------------------------
Direct Calculation Method
*Process*
1. Find partial derivatives
2. Solve
3. Calculate
4. Use

*Sample*

---------------------------------------------
Steepest Ascent Method
*Process*
1. Calculate at given point
2. Write as function of
○

Theoretical Model in Computing Page 7


○
3. Based on , turn
4. Locate such that is maximum / minimum
5. Find new based on
6. Repeat from step 1

*Sample*

Theoretical Model in Computing Page 8

You might also like