Proceedings of DETC’02

ASME 2002 Design Engineering Technical Conferences

and Computers and Information in Engineering Conference
Montreal, Canada, September 29-October 2, 2002
Proceedings of DETC/CIE:
DETC2002/CIE-34403
Computers and Information in Engineering Conference
Sept. 29- Oct. 2, 2002 Montreal
CIE-34403
The Synaps Pointer Optimization Engine
Dr. Alex Van der Velden, Synaps, Inc. Dr. David Kokan, Synaps, Inc.
[email protected] [email protected]

Abstract combination of optimization methods that would

work better than random guessing.
This paper describes a innovative optimization
algorithm called Pointer. The Pointer software Knowing the type of system you have is critical
consists of a complementary set of optimization to being able to solve the system efficiently.
algorithms: linear simplex, sequential quadratic Optimization theory has always been developed
programming (SQP), downhill simplex, and the other way around. Assuming that the system
genetic algorithms. Pointer can efficiently solve has a certain mathematical form, what is the most
a wide range of problems in a fully automatic cost efficient way of optimizing this system?
manner due to a special automatic control Over time, a large collection of optimization
algorithm. The goal of the Pointer software is to methods based on such assumptions were
make optimization more accessible to non-expert developed. Each of these methods had many
users without sacrificing performance. degrees of freedom to adapt the method to the
problem at hand. In essence, the specific
Introduction problem that people wanted to solve (e.g. design
a lighter structure) was now transformed into
finding the right algorithm for designing the
Optimization has always been around. Our
structure. This second task was in many ways a
species has and continues to evolve through an
harder task than the original. For many engineers
optimization scheme described by Darwin’s
and scientists, the problem was transformed from
survival of the fittest theory. Such a trial-and-
a known to an unknown domain. As a
error strategy can be attributed to past progress in
consequence, optimization technology did not
human endaveours. Even though this approach
become the big commercial success many had
might be best for a completely unknown
hoped for.
environment, it is obviously not ideal when more
is known about the enviroment.
Thus the issue of control became a driving factor
in the development of the Pointer optimization
A mathematican friend of mine once said that he
engine. Pointer uses a proprietary algorithm that
worked on optimization for about a year before
automatically controls a set of optimization
he gave up because “there is nothing you can say
resources. Similarly, Pointer efficiently solves a
about an arbitrary system.” He was right to
wide range of problems in a fully automatic
some extent. However, based on many years of
manner by harnessing and leveraging the power
experience in applying optimization to various
of a group of distinct complementary
engineering problems we can make the following
optimization algorithms.
observations. First of all, a system can be
classified once more is known about it.
Secondly, once classified, there is some

1 Copyright © 2002 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/conferences/idetc/cie2002/69723/ on 06/05/2017 Terms of Use: http://www.asme.o

 Figure 1 - Pointer Control Algorithm

The Pointer control algorithm varies the

optimizer settings in such a way that either:
Pointer Control Algorithm
a) the best answer is found in the shortest
Pointer derives its name from the Pointer dog. time. In this case the optimizers are
Even though Pointer dogs are all genetically configured such that the maximum rate
similar, they can have very specialized skills. of improvement of the objective
Some can ‘point’ out drugs while others can function is achieved from a single
‘point’ out where a victim is buried under an starting position.
avalanche. These special skills are passed down
from generation to generation. b) the most experience is gained. The
optimizer settings are selected in such a
The Pointer software consists of a way that the highest optimizer
complementary set of optimization algorithms. robustness is achieved for a given
Currently Pointer uses linear simplex, sequential optimization time. The robustness is
quadratic programming (SQP), downhill simplex, defined as the mean harmonic error of
and genetic algorithms. All have control the best objective function (ever found)
parameters (such as initial step sizes) that need to normalized by the local optimum for a
be set properly in order to operate efficiently. set of random starting vectors. This
This process illustrated in Figure 1 is called approach is an order of magnitude
optimizer training. The inputs to the process are slower than merely finding the best
the design variables, constraints and the search answer, but the experience gained will
objective, with the total time available for the allow you to solve similar classes of
search and any previous experience in terms of problem in the shortest possible time.
the selection and right settings of the optimizers.

2 Copyright © 2002 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/conferences/idetc/cie2002/69723/ on 06/05/2017 Terms of Use: http://www.asme.o

 Usually it is not possible for the user to identify Pointer Core Algorithms
the problem topology in such a way that he/she
can select the proper algorithms and settings. A The core Pointer algorithms were selected based
real-life example is shown in Figure 2. The on their individual performance and the added
topology is that of the drag of an airfoil performance in the set of algorithms used.
computed by CFD as a function of its shape.
Previously, the user (from experience) assumed For smooth problems, the best optimizer is the
that the drag was a smooth function of the sequential quadratic programming (SQP)
geometry. From the starting point, a small step algorithm. In our experience have never found
was selected to calculate the gradients accurately. an instance where the (older) gradient optimizers
Because the topology of the drag had surface outperformed SQP.. The SQP algorithm uses
waves, the steepest downhill slope was function calls close to the starting point to
considered to be along this wave. If the user had determine the topology of the problem. SQP is
picked a larger step size he would have widely used in trajectory optimization and
concluded that the direction normal to the waves structural optimization.
has the ‘relevant’ steepest downhill slope.
The SQP algorithm used in Pointer is NLPQL,
developed by Dr. Klaus Schittkowski [1], which
solves general nonlinear mathematical
programming problems with equality and
inequality constraints. It is assumed that all
problem functions are continuously
differentiable. Proceeding from a quadratic
approximation of the Lagrangian function and a
linearization of the constraints, a quadratic
subproblem is formulated and solved by the dual
code QL. Subsequently a line search is
performed with respect to two alternative merit
Figure 2 - CFD Problem Topology functions and the Hessian approximation is
updated by a modified BFGS-formula.
These waves are not caused by the physics of the
problem (drag), but by numerical interference of
For non-smooth continuous problems, the
converging loops inside the numerical code that
downhill-simplex method is unmatched.
calculates the drag. Almost all simulation codes
Downhill-simplex is especially popular in the
that use partial differential equations exhibit such
chemical engineering field. Unlike SQP that
behavior. Note that the spikes in this plot
starts the computation near the starting point,
represent code-crashes, another difficult
downhill simplex starts the computations from
topological feature not inherent in the real
the edges of a computational domain. Pointer
physics.
utilizes a modified version of Nelder and Mead’s
[2] implementation. The simplex is a
A dozen approaches were identified to correct geometrical body with n+1 vertices represented
pathological topologies and find the correct by a triangle in two dimensions and a tetrahedron
relevant search direction. These pathological in three. The method calculates and compares
topologies are created by the type of numerical the objective function at the vertices of a simplex
algorithm used in the simulation code and not by in the variable space, selects the worst one, and
the specific design problem the simulation code moves this point through the opposite face of the
addresses. Therefore, the solution that Pointer simplex to a lower point. If this new vertex is
presents is valid over a wide range of topologies. better, the old one is deleted. If there is no
improvement after a number of steps, the method
“shrinks” the simplex by reducing the length of
each side, thus trapping the optimum solution.
Instead of moving the best point directly in the
direction of the steepest decent (the approach of
SQP) it moves the worst points of the initial set

3 Copyright © 2002 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/conferences/idetc/cie2002/69723/ on 06/05/2017 Terms of Use: http://www.asme.o

 in the direction of the best. Thus SQP and author. The evolution strategy starts out with a
downhill simplex have little overlap in terms of large number of points inside the computational
assumptions and computational use of objective domain. In the case of mutation, each of the
function data. points produces new points that are normally
distributed around the original point. The best
A third algorithm in Pointer is the so-called point out of this set is selected. In the case of
evolution or genetic algorithm. SQP and recombination, a random number of points
downhill simplex determine analytically where exchange parameter values. Instead of the step
the best answer is based on previous objective size (for SQP) and the size of the simplex
function calls. Genetic algorithms work well (downhill-simplex) the volatility of the search is
because they incorporate randomness in their determined by the standard deviation of the
search. It gives the algorithm the ability to average mutation.
correct deterministic search bottlenecks. No
objective function continuity is required for this In our experience, genetic algorithms almost
method. always work. However, they are not often the
best algorithms to use because of their very high
The algorithm in Pointer is based on Professor computational expense.
Schwefel’s evolution strategy [3], later modified
by Dr. Mathias Hadenfeld [4] and the first

Pointer’s Performance on Bench- bar under tension (wire). The weight of the truss
mark Problems structure is therefore a discontinuous function of
the locations of the joints.
Pointer has been tested on a variety of problems,
from classic ‘hard’ problems such as traveling The optimization task was to minimize the
salesman, to the design of the hull shape for a weight of the truss in 30 minutes. In 1993
hydrofoil. Mathias Hadenfeld completed the task of
optimizing the truss from a random starting point
[5]. Hadenfeld was an expert user of
However, any optimization problem can usually
optimization. He was free to tweak and restart
be solved when the right answer is already
the solution until he felt that no more progress
known and the starting point ‘tweaked’. This
could be made, but he could not change the
comparison will therefore focus on the added
starting point. His results are shown in Figure 5
value of Pointer versus the already good set of
(blue symbols). On average he had more success
core algorithms.
with the genetic algorithm than the downhill
simplex method, and the gradient method failed
Although many problems have been solved with
completely. On average his answers were 15%
Pointer, we pick a truss optimization problem
away from the known optimum due to the
because it represents a typical design problem in
difficult topology of the problem.
mechanical or aerospace engineering [5]. Figure
3 shows a truss as it supports a series of point
In 1999 the same problem was given to 13
loads. The bridge is supported at the right edge
groups of up to 7-15 professional engineers.
and free at the left edge. It has six joints whose
They were given Pointer and its graphical user
positions are chosen by the optimizer so as to
interface displaying the truss. They were asked
minimize the weight or cost subject to a given set
to minimize the weight without using the
of loads. The weight is computed by determining
optimization. They had to manually input the
the minimum bar dimensions that can support the
joint coordinates and run the analysis to see
applied load
whether an improvement was made. Since an
analysis could be done in less than a second, 30
The problem is hard because, as the joints
minutes was ample time to complete the task.
locations move with respect to each other, the
The number of iterations was between 10 and 30.
loads on the bars change from compression to
In our experience this also represents the average
tension and back. A bar under compression has
number of industrial product design iterations.
to be much thicker (thin-walled cylinder) than a

4 Copyright © 2002 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/conferences/idetc/cie2002/69723/ on 06/05/2017 Terms of Use: http://www.asme.o

 Figure 3 – A 27 bar truss supporting a load on a bridge Fig 4: Weight of bar as function of x,y node

The groups of engineers came up with vastly We also compared Pointer versus a dozen high
different answers. Some were more than twice as quality optimizers on standard benchmark tests. The
heavy as the best solution. However, some came up test we used was devised by Dr. Sandgren [6] to
with solutions that were within 5% of the right represent a wide variety of hard optimization
answer. It is interesting to note that they only problems in all fields of mathematics and
achieved these results through the use of the engineering. Though the problems are hard, they
graphical representation of the truss. In a were typically not as hard as the once we experience
subsequent test without the graphics, no engineer working with clients.
came within 20% of the optimizer’s best answer.
Figure 6 shows the result of the test. Nevertheless,
Next, Pointer was given the control over the core maybe not surprisingly, Pointer performed
optimizers (red symbol). Just like the expert, exceedingly well on the test. It was the only code
Pointer required less function calls to find the capable of solving all the problems. This is all the
answer with genetic algorithm than with the more impressive considering that the same standard
downhill simplex algorithm. But with its standard settings were used for all the test cases and that there
settings, which allow the optimal use of all was no expert intervention in the process. In terms
algorithms, Pointer was able to consistently get the of speed it is usually comparable to the best codes
right answer from any starting point with just 1000 for each of the specific problems. The ‘best’
function calls (<1 minute). benchmark results were provided by Dr. Egorov [7]
the author of IOSO. Other codes in the comparison
Even though the group of experts got very different were IOSO [7], NLPQL [1], DONLP [8] and
answers when solving the problem manually, all MOST [9].
achieved the correct answer when using Pointer
under its standard settings.

5 Copyright © 2002 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/conferences/idetc/cie2002/69723/ on 06/05/2017 Terms of Use: http://www.asme.o

 Pointer

Expert Expert
manual W/optimizer

Figure 5 - Design of a truss for minimum weight using various tools

Figure 6 - Performance of Pointer as compared to the fastest algorithm (used by an expert) known
for each test problems.

6 Copyright © 2002 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/conferences/idetc/cie2002/69723/ on 06/05/2017 Terms of Use: http://www.asme.o

 Conclusions Ph.D. Thesis, Purdue University,
December 1977
This paper describes a innovative
optimization algorithm called Pointer. 7. Egorov I.N. 1998 “Indirect
The Pointer software consists of a Optimization Methods on the basis of
complementary set of optimization self-organization”. Optimization
algorithms: linear simplex, sequential Techniques and Applications (ICOTA
quadratic programming, downhill simplex, ’98), vol 2. pp. 683-691
and genetic algorithms. Since all the
optimizer control parameters are 8. Dr. Spellucci:
automatically set with a special control (http://www.mathematik.th-
algorithm, Pointer can efficiently solve a darmstadt.de/ags/ag8/spellucci/spellucci.
wide range of problems in a fully automatic html)
manner. This was demonstrated by
benchmarking the code with Eric 9. Pincus, Jonathan. ``Transistor Sizing.''
Sandgren’s optimization benchmark test. UCB//CSD-86-285, February 1986. 102
Pointer was the only algorithm that could pages.
solve all problems and was usually as fast
as the best algorithm known for each of the
individual test problems.

References
1. Schittkowski, K., “NLPQL: A Fortran
subroutine for solving constrained
nonlinear programming problems”,
Annals of Operations Research, Vol.5,
485-500, 1985.

2. 2. Nelder J.A. & Mead R. “Downhill

simplex method in multidimensions”.
Computer Journal, 7, pp. 308-313,
1965.

3. Schwefel H.P. “Evolutionsstrategie

und numerische optimierung”, PhD
thesis, Verfahrenstechnik TUBerlin,
1975.

4. Hadenfeld, M., “Anwendung von

Evolutionsstrategien fuer die
Optimierungsaufgaben beim Vorentwurf
von Flugzeugen”, Braunschweig
/Bremen 1993.

5. Ashley, H., “On Making Things the

Best – Aeronautical uses of
Optimization”, Wright brothers
lectureship in aeronautics, Journal of
Aircraft. vol. 19 No. 1, 1982.

6. Eric Sandgren, “The Utility of

Nonlinear Programming Algorithms"

7 Copyright © 2002 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/pdfaccess.ashx?url=/data/conferences/idetc/cie2002/69723/ on 06/05/2017 Terms of Use: http://www.asme.o