Foulds HeuristicProblemSolvingApproach 1983
Foulds HeuristicProblemSolvingApproach 1983
Author(s): L. R. Foulds
Source: The Journal of the Operational Research Society , Oct., 1983, Vol. 34, No. 10
(Oct., 1983), pp. 927-934
Published by: Palgrave Macmillan Journals on behalf of the Operational Research
Society
REFERENCES
Linked references are available on JSTOR for this article:
http://www.jstor.com/stable/2580891?seq=1&cid=pdf-
reference#references_tab_contents
You may need to log in to JSTOR to access the linked references.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms
Operational Research Society and Palgrave Macmillan Journals are collaborating with JSTOR to
digitize, preserve and extend access to The Journal of the Operational Research Society
INTRODUCTION
some kind of score, and an optimization process is carried out. This relatively new area
of mathematics is termed combinatorial optimization. It is evident that many problems
faced by O.R. practitioners fail within this compass, including: sequencing, scheduling,
routing, layout and design problems. There is usually a finite number of feasible solutions
to each instance of these problems. Therefore it seems intuitive that the fundamental
theorem of combinatorial optimization could be employed: examine every feasible solution
and choose the best. Unfortunately, there are usually far too many solutions for this
approach to be practical. As an example, consider the travelling salesman problem. Here
a salesman must find a closed tour, visiting each of a given number n, of cities exactly
once, travelling minimum distance. If the distance function is asymmetric, there are n\
possible tours. If each tour could be evaluated in a billionth ofa second, it would still take
over 1600 years to examine them for /7 = 21. (Other solution procedures for this problem
will be discussed later.)
We have to be able to do better than complete enumeration. In order to be able to
compare alogrithms, we need some way of judging the effectiveness of an algorithm.
Edmonds2 provided the following convention, which has been generally accepted.
Computer scientists often use the term 'polynomially-bound time', as it is assumed that
computational time is linearly proportional to the number of elementary computational
steps. The size of a specific instance of a problem is defined to be the number of symbols
required to describe it.
Given a combinatorial optimization problem, it is natural to ask whether a
polynomially-bounded time algorithm has been found for it. The set of problems for which
the answer is "yes" is denoted by P (for polynomial). There are a few problems for which
it has been shown that membership in P is impossible. There exists a third set of problems
whose status is unknown?it is not known whether they are in P or not. There is at least
one algorithm known for each of these problems, but no known effective algorithms have
as yet been reported. This set is denoted by NP (for non-deterministic polynomial).
The following caveat should be made here. There are problems in P which, for practical
size (say n), are intractable and problems not in P which are tractable. For instance, a
problem in P of size n = 1000 is intractable if its solution time is ofthe order of n9 (= IO27).
But a problem not in P of the same size with solution time of the order e?001n is tractable.
Some tensor inversion problems have the former character, and some mathematical
programming problems the latter. The relative efficiency of the ellipsoidal and simplex
algorithms for linear programming further brings out this point.
NP contains a subset of problems termed NP-complete. Each problem in the set has the
following properties: (1) it belongs to NP; (2) if an effective algorithm exists for it, then
an effective algorithm exists for every problem in NP, i.e. P = NP. This means that
NP-complete problems are the hardest in NP. Many of the problems in NP have defied
the attempts to find effective algorithms of some of the best mathematicians over the past
30 years. There is also more objective circumstantial evidence that P ^ NP. Thus it seems
unlikely that an effective algorithm exists for any of the NP -complete problems. Hence
the fact that a problem is NP -complete is considered by some to be justification for
approximate procedures to be applied to it. Many commonly encountered O.R. problems
are TVP-complete. So we now turn our attention to the aim of developing approximate
procedures with good performance guarantees which are also effective in the sense defined
earlier.
928
of Descartes and Leibniz, the subject became known as heuristics. Allied with logic,
philosophy and psychology, it has the aim of investigating the methods of invention and
discovery. The name itself was derived from the Greek heuriskein?to discover. Today the
term heuristic means a method which, on the basis of experience or judgement, seems likely
to yield a good solution to a problem but which cannot be guaranteed to produce an optimum.
Most people use heuristics every day:
*In order to guard against a chill, take a warm coat to work when the weather forecast
is bad.
*In order to ensure a supply of small change on hand, go to the bank and change money
regularly.
*In order to be certain of being able to buy theatre tickets, book early.
Heuristics are common in business too:
There has been much work published on heuristic problem solving, and a few of
more important contributions will now be briefly discussed. Further references to o
more specialized work appear later. Pioneering work was done by Newell and Simon
the computer simulation of human thinking. They devised a computer program called Th
General Problem Solver (GPS) to stimulate human problem-solving behaviour. It app
operators to objects to attain goals. Its processes are geared towards three types of g
(transformation, operator application and difference reduction) and methods for attainin
them. The authors furnish an example in which the goal is to transform one symbolic lo
expression into another. The operations are certain allowed manipulations, and the ob
are the expressions themselves. Tests for differences that must be recognized between pa
of expressions and these entities are fed into the computer. The steps used by the compu
and by a human subject in solving the problem are summarized in a table. It show
number of similarities and differences. They also presented this work at a conferenc
Germany.4 Newell and Simon5 went on to develop the idea of problem-solving mach
They discuss computer programs that not only play games but which also loc
warehouses and prove theorems in geometry. Newell and Simon6 also proposed
detailed and powerful variable iteration technique for solving test problems by computer
A CLASSIFICATION OF HEURISTICS
929
EXAMPLES OF HEURISTICS
To make the ideas of the last section more lucid we present a number o
illustrating the different strategies. To make this meaningful we must
problem for which a heuristic discussed is designed. In order to avoid lengthy
of a variety of problems, we introduce but one problem. We use as our v
section a problem referred to in the second section; the travelling sales
(T.S.P.). This problem can be formulated as;
Minimize
? ?d^ (i)
subject to
n
i *h (4)
930
The problem (1), (2),. .., (5) is TVP-complete, and thus heuristic solution procedures are
in order. There is no suggestion that the heuristics about to be presented are superior t
all others for the T.S.P. They are introduced, not as the most efficient T.S.P. procedures
but in order to illustrate the ideas of the last section.
available. It does this iteratively using the r-optimal approach, but r is not fixed. The key
to the success of the procedure is the following theorem.
This allows the procedure to consider only sequences of interchanges whose partial sum
of gains is always positive (improving). The authors present a detailed statement of their
heuristic and a numerical example. The computational experience they report is very
impressive for small problems (with no more than 42 cities). Here the optimal solution can
be obtained by branch and bound enumeration in reasonable computational time as a
check on the quality of heuristic solutions. It appears that the probability of obtaining an
optimal solution by their heuristic is close to one. Computational time is 0(n2).
932
Returning to the point of guaranteeing good performance on realistic and not just small
test problems, we turn to the question of validation. Sometimes this can be achieved by
constructing large-scale problems with known optimal solutions. That is, this solution is
defined and then the rest of the data for the problem is "built around it" ensuring
optimality. Although such problems are inherently artificial, careful construction can
provide useful test problems. Despite the previous remark, it is often still quite useful to
compare heuristic performance with that of an algorithm on a wide variety of small and
medium-sized test problems if possible.
When validation is difficult, but it is relatively easy to generate feasible solutions, the
following design approach of Baum and Carlson14 may be valuable. Called the Better Than
Most approach, B.T.M., it randomly generates solutions, selecting those that are feasible.
The best solutions among these are selected according to the optimality criteria of the
analyst, where some of these criteria may be difficult to quantify. Baum and Carlson
discuss the obvious question of how large a sample must be taken in order to guarantee
that the best solution identified will be, with a specified probability, within a given
percentage of optimality.
This approach leads to the question of when to terminate a heuristic that generates one
or more feasible solutions early in its execution and then spends most of its time searching
for improvements. McRoberts15 outlines and provides examples of the application of
extreme value distributions as mechanisms for estimating optimum limits. Given such
estimates, he further illustrates the development of rules applicable to the decision of
whether to continue or terminate.
When optimality criteria are difficult to quantify, it may be reasonable to contemplate
the design of a heuristic which involves interaction with an analyst during its executio
Using his intuition and experience, he may be able to guide the heuristic's search proce
far more efficiently than a predetermined set of rules. There is the further strategy o
relaxing certain difficult constraints and having the analyst choose from a variety
solutions proposed by the interactive problem. The analyst then, on the basis of his choice,
may be able to set slightly more constrained problems for the interactive heuristic. These
approaches have been successfully used to solve large-scale vehicle scheduling problems
heuristically.
1. They can often be implemented by a practitioner who lacks advanced training, by virtue
of their simplicity.
2. They require relatively modest computational time.
3. They are flexible, often allowing use as planning tools where many scenarios and set
of non-quantifiable constraints must be considered.
One disadvantage is the fear that solutions far below optimality will be produced. It
often not clear how significant the deficiency is. There can be further problems with
implementation. Most ofthe literature on the topic deals with successfully conceptualizing
heuristic solutions for idealized models. (It is a shame that more negative results are no
reported as these often contribute to the learning process of others as well.) Many of t
heuristics posed do not pass the test of being subjected to real-world problems becaus
of model deficiency. There are often complications not present in the model for which the
heuristic is designed. Thus successful heuristic implementation often lags behind heuristic
development by a considerable degree.
In order to safeguard against the error of introducing a heuristic at the expense o
existing or new standard techniques, the management should ask the following question
1. Will we spend more than we save by developing and implementing a new heuristic?
2. Will the decision process be unnecessarily slowed down by its introduction?
933
3. Is there a chance that work quality will be reduced to a level that negates the savings
brought about by introduction?
4. Are the results, when actually implemented, not really significantly better than those
produced before introduction?
If the answer to any of these questions is in the affirmative, the manager should seriously
question the value of the introduction of the heuristic proposed.
CONCLUSIONS
REFERENCES
1C. W. Churchman (1970) Operations research as a profession. Mgmt Sci. 17, B37-B
2J. Edmonds (1965) Paths, trees and flowers. Can. J. Math. 17, 449-467.
3A. Newell and H. A. Simon (1961) Computer simulation of human thinking. Scienc
4 A. Newell and H. A. Simon (1961) A program that simulates human problem solving. Pro
Automata. Oldenburg, Munich.
5A. Newell and H. A. Simon (1964) Problem-solving machines. Int. Sci. Technol. 36,
6A. Newell and H. A. Simon (1965) Simulation of human processing of informatio
Computing (J. Heller, P. D. Lax, D. H. Lehmer and R. D. Richtmyer, Eds). Slaught Mem
10, Am. math. Mon. 72, No. 2, Part 11, 111-118.
7H. Muller-Merbach (1976) A morphological classification of heuristic techniques. Z. O
8H. Muller-Merbach (1969) Operations Research. Methods and Models for Optimal Plan
Vahlen, Berlin.
9N. Christofides (1976) Worst case analysis of a new heuristic for the travelling salesman
Carnegie-Mellon University, Pittsburgh, Pennsylvania.
10 S. Lin and B. Kerningham (1973) An effective heuristic algorithm for the travelling sa
Res. 21, 498.
11 R. M. Karp (1976) The probabilistic analysis of some combinatorial search algorithms. In
Directions and Recent Results in Algorithms and Complexity (J. B. Traub, Ed.). Acade
12 P. Krolak, W. Felts and G. Marble (1971) A man-machine approach toward solving th
problem. CACM 14, 327-334.
13 M. R. Garey and D. S. Johnson (1976) Approximation algorithms for combinatorial p
bibliography. In Proc. Symp. on New Directions and Recent Results in Algorithms and Com
Ed), pp. 41-52. Academic Press, New York.
14 S. Baum and R. Carlson (1979) On solutions that are better than most. Omega 7, 24
15 K. L. McRoberts (1971) A search model for evaluation of combinatorially explosive p
1331-1349.
16 J. Beardwood, J. H. Halton and J. M. Hammersley (1959) The shortest path through many points. In Pro
Camb. Phil. Soc. 55, 299-327.
934