Optimization Presentation 2015
Optimization Presentation 2015
A Hands-On Workshop
Farrukh Nagi
Universiti Tenaga Nasional PutraJaya Campus
12th Nov, 2015
http://metalab.uniten.edu.my/~farrukh/OPTIM15.zip
1
Introduction To Optimization
PART I
Contents
PART 1 INTRODUCTION TO OPTIMIZATION
1. Introduction to Optimization
What is optimization
Optimization Methods
Linear Programming
Fundamentals of non-linear optimization
Optimality Conditions
Steepest Descent Methods
Optimization Exercise & Solutions
What is Optimization?
Optimization is an iterative process
by which a desired solution(max/min)
of the problem can be found while
satisfying all its constraint or bounded
conditions.
Optimization problem could be linear
or non-linear.
2. OPTIMIZER GUI
@Objective function .m
Mathematical model .m
Simulink Block model . mdl
3. SIMULINK OPTIMIZER
Objective function.mdl
See Slide 22
contd
CONSULTANCY WORK
i) Plant Noise control Optimization
OPTIM15\Noise\Conn_octave3_j_10.m
RESEARCH WORK
Noise Control
Component
Location
illumination
modeling
10
Optimization Methods
One-Dimensional Unconstrained Optimization
Golden-Section Search
GA
Quadratic Interpolation
Presentation
Newton's Method
Multi-Dimensional Unconstrained Optimization
PSO
Presentation
Non-gradient or direct methods
Gradient methods
Linear Programming (Constrained)
SA presentation
Graphical Solution
Simplex Method
Genetic Algorithm (GA) Survival of the fittest principle based upon
evolutionary theory \OPTIM15\GA\GA Presentation.ppt
Particle Swarm Optimization (PSO) Concept of best solution in
the neighborhood : \OPTIM15\PSO\ PSO Presentation.ppt
Others .
11
Fundamentals of Optimization
12
13
13
A set S is convex if the line segment joining any two points in the
set is also in the set.
convex
not convex
convex
not convex
not convex
14
Contd
Optimality Conditions
f ( X *) 0
x
H 2
f
xy
2 f
yx
2 f
y 2
16
-1
-2
-2
-1
17
Contd
Maximize X1 + 1.5 X2
Subject to:
X1 + X2 150
0.25 X1 + 0.5 X2 50
X1 50
X2 25
X1 0, X2 0
Optimal points
Local minima/maxima points: A point or Solution x* is at local point
if there is no other x in its Neighborhood less than x*
Global minima/maxima points: A point or Solution x** is at global
point if there is no other x in entire search space less than x**
18
19
Example
20
Example (cont.)
21
C:\OPTIM15/Presentatio
n/Problems and
Excercises.doc
Microsoft Word
Document
22
22
MATLAB Optimization
PART II
23
@functions
Scripts
MATLAB
MATLAB
>>Command
Window
>>Optimtool
Simulink
Simulink
Model.mdl
Ports/
Block
update
-GUI-
Design
Optimization
Model
Block
24
Function Optimization
min f x
x
g x 0
hi x 0 Equality Constraints
~
Subject to:
Inequality Constraints
x L k xk xU k Side Constraints
Where:
f x
~
x
~
g x 0
hi x 0
~
Equality Constraints
Inequality Constraints
x L k xk xU k
Side Constraints
26
Optimization Toolbox
Is a collection of functions that extend the capability of MATLAB.
The toolbox includes routines for:
Unconstrained optimization
Constrained nonlinear optimization, including goal attainment
problems, minimax problems, and semi-infinite minimization
problems
Quadratic and linear programming
Nonlinear least squares and curve fitting
27
Unconstrained Minimization
x x1
~
x2
Steps:
Create an M-file that returns the function value (Objective
Function). Call it objfun.m
fminunc
28
x2
f=exp(x(1))*(4*x(1)^2+2*x(2)^2+4*x(1)*x(2)+2*x(2)+1);
Objective function
29
options = optimset(LargeScale,off);
[xmin,feval,exitflag,output]=
fminunc(objfun,x0,options);
Output arguments
Input arguments
30
Results
xmin =
0.5000
-1.0000
feval =
1.3028e-010
exitflag =
1
output =
iterations: 7
funcCount: 40
stepsize: 1
firstorderopt: 8.1998e-004
algorithm: 'medium-scale: Quasi-Newton line search'
31
32
34
Non-Linear least
squares
35
Non-Linear least
squares
36
Simplex Methods
Minimize
37
Derivative-free optimization
Downhill
simplex
method
38
Derivative-free optimization
Downhill
simplex
method
39
39
40
Example (cont.)
41
Example (cont.)
42
43
43
44
45
Fish Harvesting
WWTP_RivPol
Noise Control
Turbine controller
Data Fitting
1. Hydrology .../hydrology/
2. Anthropometric.../anthropometry/
hydrology
Anthropometry
46
Two modes:
Large-Scale: interior-reflective Newton
Medium-Scale: quasi-Newton (BFGS)
Not preferred in solving highly discontinuous functions.
This
Constrained Minimization
Vector of
Lagrange
Multiplier at
optimal point
[xmin,feval,exitflag,output,lambda,grad,hessian]
=
fmincon(fun,x0,A,B,Aeq,Beq,LB,UB,NONLCON,options,
P1,P2,)
48
Example
function f = myfun(x)
min f x x1 x2 x3
x
~
Subject to:
f=-x(1)*x(2)*x(3);
2 x12 x2 0
x1 2 x2 2 x3 0
x1 2 x2 2 x3 72
0 x1 , x2 , x3 30
1 2 2
A
, B
1
2
2
0
72
LB 0 , UB
30
30
30
49
Example (Cont.)
For
2 x12 x2 0
50
Example (Cont.)
Initial guess (3 design variables)
x0=[10;10;10];
A=[-1 -2 -2;1 2 2];
B=[0 72]';
LB = [0 0 0]';
UB = [30 30 30]';
1 2 2
0
A
, B
72
1
2
2
0
30
30
LB 0 , UB
0
30
[x,feval]=fmincon(@myfun,x0,A,B,[],
[],LB,UB,@nonlcon)
CAREFUL!!!
fmincon(fun,x0,A,B,Aeq,Beq,LB,UB,NONLCON,options,P1,P2
,)
51
Example (Cont.)
Warning: Large-scale (trust region) method does not currently solve this type of problem, switching to medium-scale (line
search).
> In D:\Programs\MATLAB6p1\toolbox\optim\fmincon.m at line 213
In D:\usr\CHINTANG\OptToolbox\min_con.m at line 6
Optimization terminated successfully:
Magnitude of directional derivative in search direction less than 2*options.TolFun and maximum constraint violation is less
than options.TolCon
Active Constraints:
2
9
x=
0.00
0.00
16.231
feval =
-4.657237250542452e-025
x1 2 x2 2 x3 0 Const. 1
x1 2 x2 2 x3 72 Const. 2
0 x1 30
Const. 3
Const. 5
Const. 4
Const. 7
0 x2 30
Const. 6
0 x3 30
2 x12 x2 0
Const. 8
Const. 9
Sequence: A,B,Aeq,Beq,LB,UB,C,Ceq
52
53
Constrained Optimization
An optimization algorithm is large scale when it uses linear algebra that does not need
to store, nor operate on, full matrices. In contrast, medium-scale methods internally
create full matrices and use dense linear algebra
The definition is based on the Karush-Kuhn-Tucker (KKT) conditions. The KKT
conditions are analogous to the condition that the gradient must be zero at a minimum,
modified to take constraints into account. The difference is that the KKT conditions hold
for constrained problems. The KKT conditions use the auxiliary Lagrangian function:
54
fmincon Algorithms
fmincon has four algorithm options: a) interior-point,
b) active-set, c) SQP, d) trust-region-reflective
a) An Interior point method is a linear or
nonlinear programming method (Forsgren et al.
2002) that achieves optimization by going
through the middle of the solid defined by the
problem rather than around its surface
b) Active Set Approach
Equality constraints always
remain in the active set Sk. The
search
direction
dk
is
calculated and minimizes the
objective
function
while
remaining on active constraint
boundaries.
55
1
1 e
1.0986
( x b2 )
b1
Find b1 and
b2
>>recfit
>>b =
%recfun.m
function y=recfun(b)
0.0603
global
data; 1.0513
x=data(:,1);
Rexp=data(:,2);
Rcal=1./(1+exp(1.0986/b(1)*(x-b(2))));
% the calculated value from the model
%y=sum((Rcal-Rexp).^2);
y=Rcal-Rexp;
% the sum of the square of the
difference
%between calculated value and 57
experimental value
SIMULINK OPTIMIZATION
RESPONSE
PART III
58
2)
Objective Function
3)
4)
5)
6)
59
Out-Port
Parameter
Optimized
60
/PSO/PSO.m
61
62
functionfunct=motor_select_ga_fun(x)
Ki=x(1);
Ka=x(2);
assignin('base','Ki',Ki);
assignin('base','Ka',Ka);
[t,xout,err]=sim('lead_screw_model_opt_Motor_ga',[010]);
K / (L s + R )
i
K /( J s + B)
a
funct=sqrt(sum(err).^2);
%motor_select_opt_ga.m
%lead_screw_model_opt_Motor_ga.mdl
%motor_select_ga_fun.m
%loadgaoptions.mat
clc
Ki=0.1;
Ka=1;
u0=[Ki,Ka];
options=gaoptimset(options,'PopulationSize',10,'TimeLimit',100,'Generations',200,'StallTimeLimit',20,'TolCon',1e6,'TolFun',1e6)
[x,funct]=ga(@motor_select_ga_fun,2,[],[],[],[],[0.0010.001],[],[],options)
63
Block
Parameter
Optimized
Gain
num(s)
10
num(s)
-K-
0.5s+1
Constant1
den(s)
K /(L s + R )
i
K /(J s + B)
a
64
OPTIM15\leadscrew\lead_screw_model_optimization.mdl
65
Block
Parameter
Optimized
Simulink
Optimization
Response GUI
Block
Parameter
Optimized
66
OPTIM15\Buck_Boost\Buck_PWM_OPT12_Fs_Optim
67
Block
Parameter
Optimized
f(x)=0
Block Solver
Configuration2
Parameter
Optimized
-KGain
A
+
Current
Sensor
Friction
Torque
DC Motor
Tm - Time constant
N
3mm Lead
Worm Gear
Screw
PS S
Zero N t<2s,
1000N 4s<t<6s,
4000N t>6s
Load
Force
DC Voltage Source
R rpm
Speed
Sensor
Pmm/s
Speed
(rpm)
Linear
Speed
Speed
linear
Torqueconstant=0.0637Nm/A
68
%OPTIM15\lead screw\lead_screw_model_optimization_GA.m
%OPTIM15\lead screw\lead_screw_model_optimization_GA.mdl
%OPTIM15\lead screw\lead_screw_model_optimization_ga_fun.m
%lead_screw_model_optimization_GA.m
clc
Tm=0.1;
N=0.6
x0=[Tm N];
lb=[0.01 0.01];
ub=[1 1];
options=optimset('LargeScale','on','Display','iter','Maxiter',100,'MaxFunEvals',100,'TolX',1e6,'TolFun',1e-6);
[x,funct]=fmincon(@lead_screw_ga_fun,x0,[],[],[],[],lb,ub,[],options)
function fspeed=lead_screw_ga_fun(x)
t=[0 20];
assignin('base','Tm', x(1));
assignin('base','N',x(2));
[t, xout, err]=sim('lead_screw_model_optimization_GA',t);
fspeed=sum(sqrt((2000-err).^2))
69
OPTIM15\leadscrew\lead_screw_model_optimization_bnd.mdl
70
OPTIM15\leadscrew\lead_screw_model_optimization_ref.mdl
71
White Box Fit the data to model derived from scientific knowledge,
physical Laws such as Ohms law, Newton Laws, Thermodynamics,
Gravity.
ii)
Grey Box Fit the data to any other model such as Autoregressive ,
polynomial or Laplace Transfer . The model is chosen by the user.
iii)
Black Box Fit the data to the model chosen by the System Identification
software
Objective
Function
72
__1____
(L s+R_a)
(Js+B)
To rque
Current
= 2rad/sec
[0.51]
[0 .010.00003]
%IEEE_OPTIM_2014\lead screw\vcb_mot_opt.m
%VCB_motor_opt_port.mdl
%vcb_mot_fun.m
clc
a=1;
b=1;
c=0.1;
d=0.1;
x0=[a b c d];
lb=[0.01 0.01 0.01 0.01];
ub=[10 10 10 10];
options=optimset('LargeScale','on','Display','iter','Maxiter',200,'MaxFunEvals',200,'TolX',1e-6,'TolFun',1e-6);
[x,funct]=fmincon(@vcb_mot_fun,x0,[],[],[],[],lb,ub,[],options)
t=[0 199.99];
[t, xout, spout]=sim('vcb_motor_opt_port',t);
plot(mtrtmed,spout)
hold
grid
plot(mtrtmed,mtrsigd)
hold off
Speed
function fspeed=vcb_mot_fun(x)
load mtrsig mtrsig
load mtrtme mtrtme
mtrsigd=downsample(mtrsig,10);
mtrtmed=downsample(mtrtme,10);
t=[0 199.99];
assignin('base','a', x(1));
assignin('base','b',x(2));
assignin('base','c', x(3));
assignin('base','d', x(4));
assignin('base','mtrsigd', mtrsigd);
assignin('base','mtrtmed',mtrtmed);
[t, xout, spout]=sim('vcb_motor_opt_port',t);
fspeed=sum(sqrt((mtrsigd-spout).^2));
73
74
OPTIM15\lead screw\vcb_motor_opt_Motor_trk.mdl
75
END
QUESTIONS ?
76
B. GENETIC ALGORITHM
3. GA function optimization
(OPTIM15\GA\Live_fn_simGA.mdl)
4. DC Motor Parameter Optimization with GA
(OPTIM15\Diodes_amps\lead_screw_model_opt_Motor_ga.mdl)\
________________________________________________________________
77
_
________________________________________________________
________________________________________________________
79
REFERENCES
1. Optimization toolbox for use with MATLAB, User Guide, The MathWorks
Inc. 2006
2. Applied Optimization with MATLAB Programming, P. Venkataraman, Wiley
Inter Science, 2002
3. Optimization for Engineering Design, Kalyanmoy Deb, Prentice Hall, 1996.
4. Convex Optimization, Stephen Boyd and Lieven Vandenberghe, CUP 2004.
5. Numerical Recipes in C (or C++): The Art of Scientific Computing, W. H.
Press, Brain P. F. Saul A. T. W. T. Vetterling CUP ,1992/2002
6. http://users.powernet.co.uk/kienzle/octave/optim.html
7. http://www.cse.uiuc.edu/eot/modules/optimization/SteepestDescent/
80