Scribd
Upload
49 views17 pages

#9 Steepest Descent

Uploaded by

Robert Dalton
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
49 views17 pages

#9 Steepest Descent

Uploaded by

Robert Dalton
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

ME 782 Design Optimization

Steepest Descent
Steepest Descent

• Negative gradient moves


towards minimum
• Norm of the gradient reduces
f(x) for a multivariable function

f ( x4 )  0
' f ' ( x6 )  0

s
s
f ' ( x5 ) = 0
x4 x5 x6 x

8/16/2024 D.N. Manik 2


Steepest Descent

• Start with an initial point xi

• Determine gradient f (x ) and i

set the direction of search s = −f (xi )


as negative of the gradient
xi +1 = xi +  s
• Determine the optimum value of 𝛾
8/16/2024 D.N. Manik 3
Consider the given equation.

f ( x1 , x2 ) = 20 + 3x1 + 7 x2 + 10 x12 + 5x22 + 3x1 x2

Beginning from (1,1) determine the next best point in the


steepest descent direction in search of minimum value.
The gradient at (1,1) is given by
 3  20 3  1  26
  f (x0 ) = d + Hx =   +     = 
7   3 10 1  20

The search direction based on steepest descent is given by


 −26 
s1 = −  f (x0 ) =  
 −20 
The normalized search direction is given by

1  −26  −0.7926 
s1 = =
2  −20  −0.6097 
26 + 20 
2
  

The next value is given by


1  −0.7926 
x1 = x 0 +  s1 =   +   −0.6097 
1  
1 − 0.7926 
= 
1 − 0.6097 
𝑓 𝑥1 𝑥2 = 20 + 3𝑥1 + 7𝑥2 + 10𝑥12 + 5𝑥22 + 3𝑥1 𝑥2

The given objective function can be written as follows

f ( x1 +  s1 , x2 +  s2 ) = 20 + 3 ( x1 +  s1 ) + 7 ( x2 +  s2 ) + 10 ( x1 +  s1 )
2

+ 5 ( x2 +  s2 ) + 3 ( x1 +  s1 )( x2 +  s2 )
2

Where  is the extent to which the current search can be pursued

from points x1 , x2 in the direction indicated in the coordinate directions s1 , s2

Derivative of the above equation is given by

df ( x1 +  s1 , x2 +  s2 )
= 3s1 + 7 s2 + 20 s1 ( x1 +  s1 ) + 10s2 ( x2 +  s2 )
d
+ 3s2 ( x1 +  s1 ) + 3s1 ( x2 +  s2 ) = 0
−3s1 − 7 s2 − 20 x1s1 − 10 x2 s2 − 3 x1s2 − 3x2 s1
=
20 s12 + 10 s22 + 6 s1s2

x1 = 1
x2 = 1
s1 = −0.7926
s2 = −0.6097

From equations (3.20) and (3.21)


−3  −0.7926 − 7  −0.6097 − 20  −0.7926 − 10  −0.6097
−3  −0.6097 − 3  −0.7926
= = 1.71
2 ( −0.7926 ) + 10 ( − 0.6097 ) + 6  −0.7926  −0.6097
2 2
Hence, the next iterated value is given by
1  −0.7926 
x1 = x 0 +  s1 =   + 1.71  
1
  −0.6097 
 −0.3554 
= 
 −0.0426 

 can also be computed from the following equation

 −0.7926 
 26 20  
f (x0 ) s1
T
 −0.6097 
 =− T =− = 1.71
s1 H s1  20 3   −0.7926 
 −0.7926 −0.6097   
 3 10   −0.6097 
Steepest Descent
Minimize
f ( x1 , x2 ) = 5.5 x1 − 2 x2 + 6 x + 3x + 4 x1 x2
2
1
2
2

Using the method of steepest descent

12 x1 + 4 x2 + 5.5
f =  
 6 x 2 + 4 x1 − 2 
Let the initial guess be given by

x T
0 =  0 0
 −5.5
s = −f ( x 0 ) =  
 2 

8/16/2024 D.N. Manik 9


Contour plot
f ( x1 , x2 ) = 5.5 x1 − 2 x2 + 6 x + 3x + 4 x1 x2
2
1
2
2
Steepest Descent

0  −5.5  −5.5 


x1 = x 0 +  s =   +   2  =  2 
0    
f  x1 (  )  = 0.5 − 0.1145 + C
2

df
=  − 0.1145 = 0
d
 = 0.1145

0  −5.5 -0.6300 


x1 = x 0 +  s =   + 0.1145   = 
0
   2   0.2291 
8/16/2024 D.N. Manik 11
1.1438 
s = −f ( x1 ) =  
 3.1455 
-0.6300  1.1438 
x 2 = x1 +  s =   + 3.1455
 0.2291   
-0.6300 + 1.1438 
= 
 0.2291 + 3.1455 
f  x 2 (  )  = 51.5 2 − 11.2024 + C
df
= 103 − 11.2024 = 0  = 0.1079
d
 −0.63  1.1438  -0.5066 
x 2 = x1 +  s =   + 0.1079   = 
 0.2291  3.1455   0.5684 
8/16/2024 D.N. Manik 12
 −1.6941
s = −f ( x 2 ) =  
 0.6160 
-0.5066   −1.6941
x3 = x 2 +  s =   + 0.6160 
 0.5684   
 -0.5066-1.6941 
= 
0.5684 + 0.6160 
f  x3 (  )  = 14.1839 2 − 3.2495 + C
df
= 28.3678 − 3.2495 = 0  = 0.1145
d
 −0.5066   −1.6941 -0.7007 
x3 = x 2 +  s =   + 0.1145   = 
 0.5684   0.6160   0.6390 
8/16/2024 D.N. Manik 13
0.3523 
s = −f ( x3 ) =  
 0.9689 
-0.7007  0.3523 
x 4 = x3 +  s =   + 0.9689 
 0.6390   
 -7007+0.3523 
= 
0.6390 + 0.9689 
f  x 4 (  )  = 4.9262 2 − 1.0628 + C
df
= 9.8525 − 1.0628 = 0  = 0.1079
d
 −0.7007  0.3523   -0.6627 
x 4 = x3 +  s =   + 0.1079   = 
 0.6390   0.9689   0.7435 
8/16/2024 D.N. Manik 14
 −0.5218
s = −f ( x 4 ) =  
 0.1898 
-0.6627   −0.5218
x5 = x 4 +  s =   + 0.1898 
 0.7435   
 -0.6627-0.5218 
= 
0.7435 + 0.1898 
f  x5 (  )  = 1.3457 2 − 0.3083 + C
df
= 2.6914 − 0.3083 = 0  = 0.1145
d
 −0.6627   −0.5218 -0.7225 
x5 = x 4 +  s =   + 0.1145   = 
 0.7435   0.1898   0.7652 
8/16/2024 D.N. Manik 15
0.1085 
s = −f ( x5 ) =  
 0.2984 
-0.7225 0.1085 
x 6 = x5 +  s =   + 0.2984 
 0.7652   
 −0.7225 + 0.1085 
= 
 0.7652 + 0.2984 
f  x 6 (  )  = 0.4674 2 − 0.1008 + C
df
= 0.9348 − 0.1008 = 0  = 0.1079
d
 −0.7225 0.1085   -0.7107 
x 6 = x5 +  s =   + 0.1079   = 
 0.7652   0.2984   0.7974 
8/16/2024 D.N. Manik 16
Conclusion

8/16/2024 D.N. Manik 17

You might also like