Part 4

Optimization

Prepared by: Engr. Lucia V. Ortega 1/10/2022 1

 Optimization – involves searching for either the minimum or maximum.

Prepared by: Engr. Lucia V. Ortega 1/10/2022 2

 One-Dimensional Unconstrained Optimization

A. Bracketing Method
1. Golden-Section Search
2. Quadratic Interpolation
B. Open Method
Newton’s Method

Prepared by: Engr. Lucia V. Ortega 1/10/2022 3

Golden-Section Search
Steps:
1. Guess an initial value of the root,
𝑥𝑙 and 𝑥𝑢 . 𝑓 𝑥𝑙 𝑓 𝑥𝑢 < 0.
2. Determine two interior points
using the golden ratio
5−1
𝑑= 𝑥𝑢 − 𝑥𝑙
2
𝑥1 = 𝑥𝑙 + 𝑑
𝑥2 = 𝑥𝑢 − 𝑑

Prepared by: Engr. Lucia V. Ortega 1/10/2022 4

 Golden-Section Search

3. The function is evaluated at theses two interior points. Two results can occur.
a. If 𝑓 𝑥1 > 𝑓 𝑥2 , then the domain of x to the left of 𝑥2 , from 𝑥𝑙 to 𝑥2 , can be eliminated
because it does not contain the maximum. For this case, 𝑥2 becomes the new 𝑥𝑙 for the
next round.
b. If 𝑓 𝑥2 > 𝑓 𝑥1 , then the domain of x to the right of 𝑥1 , from 𝑥1 to 𝑥𝑢 , can be
eliminated because it does not contain the maximum. For this case, 𝑥1 becomes the new 𝑥𝑢
for the next round.
4. Termination criteria
𝑥𝑢 −𝑥𝑙 5−1
𝜀𝑎 = 1 − 𝑅 100% R=
𝑥𝑜𝑝𝑡 2

Prepared by: Engr. Lucia V. Ortega 1/10/2022 5

Problem 1:
Solve for the value of x that maximizes 𝑓 𝑥 = −1.5𝑥 6 − 2𝑥 4 + 12𝑥 using the golden-section search.
Employ initial guesses of 𝑥𝑙 = 0 and 𝑥𝑢 = 2 and perform three iterations.

𝑰 𝒙𝒍 𝒇 𝒙𝒍 𝒙𝟐 𝒇 𝒙𝟐 𝒙𝟏 𝒇 𝒙𝟏 𝒙𝒖 𝒇 𝒙𝒖 𝒅
1 0.00000000 0.00000000 0.76393202 8.18788519 1.23606798 4.81418201 2.00000000 -104.00000000 1.23606798

2 0.00000000 0.00000000 0.47213595 5.54963687 0.76393202 8.18788519 1.23606798 4.81418201 0.76393202

3 0.47213595 5.54963687 0.76393202 8.18788519 0.94427191 8.67784172 1.23606798 4.81418201 0.47213595

… … … … … … … … … …
26 0.91691033 8.69792982 0.91691489 8.6979298252 0.91691770 8.6979298250 0.91692225 8.69792982 0.00000737

27 0.91691033 8.69792982 0.91691315 8.6979298251 0.91691489 8.6979298252 0.91691770 8.69792983 0.00000455

28 0.91691315 8.69792983 0.91691489 8.6979298252 0.91691596 8.6979298252 0.91691770 8.69792983 0.00000281

Prepared by: Engr. Lucia V. Ortega 1/10/2022 6

 Quadratic Interpolation

Steps:
1. Guess an initial value of the root, 𝑥0 , 𝑥𝑙 , and 𝑥𝑢 . 𝑓 𝑥𝑙 𝑓 𝑥𝑢 < 0.
2. Solve for 𝑥3
𝑓 𝑥0 𝑥12 −𝑥22 +𝑓 𝑥1 𝑥22 −𝑥02 +𝑓 𝑥2 𝑥02 −𝑥12
𝑥3 =
2𝑓 𝑥0 𝑥1 −𝑥2 +2𝑓 𝑥1 𝑥2 −𝑥0 +2𝑓 𝑥2 𝑥0 −𝑥1
3. A strategy similar to golden-section search can be employed to determine
which point should be discarded.

Prepared by: Engr. Lucia V. Ortega 1/10/2022 7

Problem 2:
Solve for the value of x that maximizes 𝑓 𝑥 = −1.5𝑥 6 − 2𝑥 4 + 12𝑥 using the golden-section search.
Employ initial guesses of 𝑥0 = 0, 𝑥1 = 1, and 𝑥2 = 2 and perform three iterations.

I 𝐱𝟎 𝐟 𝐱𝟎 𝐱𝟏 𝐟 𝐱𝟏 𝐱𝟐 𝐟 𝐱𝟐 𝐱𝟑 𝐟 𝐱𝟑
1 0.00000000 0.00000000 1.00000000 8.50000000 2.00000000 -104.00000000 0.57024793 6.57990854

2 0.57024793 6.57990854 1.00000000 8.50000000 2.00000000 -104.00000000 0.81243059 8.44652312

3 0.81243059 8.44652312 1.00000000 8.50000000 2.00000000 -104.00000000 0.90771630 8.69575392

… … … … … … … … …
9 0.91687266 8.69792978 0.91690077 8.69792982 1.00000000 8.50000000 0.91691250 8.69792983

10 0.91687266 8.69792978 0.91691250 8.69792983 1.00000000 8.50000000 0.91691305 8.69792983

11 0.91691250 8.69792983 0.91691305 8.69792983 1.00000000 8.50000000 0.91691494 8.69792983

Prepared by: Engr. Lucia V. Ortega 1/10/2022 8

 Newton’s Method

𝑓′ 𝑥𝑖
Formula: 𝑥𝑖+1 = 𝑥𝑖 −
𝑓" 𝑥𝑖

Problem:
Solve for the value of x that maximizes 𝑓 𝑥 = −1.5𝑥 6 − 2𝑥 4 + 12𝑥 using the golden-
section search. Employ initial guesses of 𝑥0 = 2 and perform three iterations.
Solution:
𝑓 𝑥 = −1.5𝑥 6 − 2𝑥 4 + 12𝑥
𝑓′ 𝑥 = −9𝑥 5 − 8𝑥 3 + 12 𝑓" 𝑥 = −45𝑥 4 − 24𝑥 2

Prepared by: Engr. Lucia V. Ortega 1/10/2022 9

 Newton’s Method
𝑰 𝒙𝒊 𝒇 𝒙𝒊 𝒇′𝒙𝒊 𝒇" 𝒙𝒊
1 2.00000000 -104.00000000 -340.00000000 -816.00000000
2 1.58333333 -17.20289643 -109.31260851 -342.98133681
3 1.26462034 3.92461739 -33.28977528 -153.47647787

… … … … …
4 1.04771560 8.17861564 -8.56281128 -80.56831818
5 0.94143547 8.68184520 -1.33088363 -56.62002371
6 0.91792994 8.69790303 -0.05284773 -52.17080295
7 0.91691697 8.69792983 -0.00009395 -51.98540229
8 0.91691516 8.69792983 0.00000000 -51.98507199

Prepared by: Engr. Lucia V. Ortega 1/10/2022 10

 Multidimensional Unconstrained Optimization

• Direct Method
Random Search
• Gradient Methods
1. Gradients and Hessians
2. Steepest Ascent Method

Prepared by: Engr. Lucia V. Ortega 1/10/2022 11

 Direct Methods

Direct methods vary from simple brute force approaches to a more elegant techniques that
attempt to exploit the nature of the function.
Random Search – a simple example of brute force approach. This method repeatedly
evaluates the function at randomly selected values of the independent variables. If sufficient
number of samples are conducted, the optimum will eventually be located.
For a given 𝑓(𝑥, 𝑦), let r be the random number generator
𝑥 = 𝑥𝑙 + 𝑥𝑢 − 𝑥𝑙 𝑟 𝑦 = 𝑦𝑙 + 𝑦𝑢 − 𝑦𝑙 𝑟

Prepared by: Engr. Lucia V. Ortega 1/10/2022 12

Problem 3: 𝑰 𝒙 𝒚 𝒇(𝒙, 𝒚)
1 -2 1 -2
Use random number generator to locate the
2 -1.96 1.02 -1.7452
maximum of 𝑓 𝑥, 𝑦 = 𝑦 − 𝑥 − 2𝑥 2 − 3 -1.92 1.04 -1.5008
2𝑥𝑦 − 𝑦 2 in the domain bounded by 𝑥 = … … … …
− 2 to 2 and 𝑦 = 1 to 3. 23 -1.12 1.44 1.2032
24 -1.08 1.46 1.2292
Solution:
25 -1.04 1.48 1.2448
𝑥 = 𝑥𝑙 + 𝑥𝑢 − 𝑥𝑙 𝑟 26 -1 1.5 1.25
𝑥 = −2 + 2 − −2 𝑟 27 -0.96 1.52 1.2448
𝑥 = −2 + 4𝑟 28 -0.92 1.54 1.2292
29 -0.88 1.56 1.2032
𝑦 = 𝑦𝑙 + 𝑦𝑢 − 𝑦𝑙 𝑟
… … … …
𝑦 =1+ 3−1 𝑟 97 1.84 2.92 -24.9632
𝑦 = 1 + 2𝑟 98 1.88 2.94 -25.7068
Assume 𝒓 = 𝟎. 𝟎𝟏 99 1.92 2.96 -26.4608
100 1.96 2.98 -27.2252
101 2 3 -28

Prepared by: Engr. Lucia V. Ortega 1/10/2022 13