Let's recap your heads.
In the last lesson, we saw that the travelling salesman
problem, or, the TSP, is complex to solve. However, it is possible to drastically
reduce the complexity of the resolution algorithm, provided that the produced
solution is not necessarily the optimal one, but a good enough one.
In this lesson, I'm going to introduce the notion of heuristic algorithm, which is a
technique used to more quickly solve problems when the classical methods are
too slow to find an exact solution. In other words, the objective of a heuristic is
to produce a solution in a reasonable time frame that's good enough for solving
the problem at hand.
Usually, heuristics trade optimality, completeness, accuracy, or precision for
speed. In a way, a heuristic can be considered a shortcut.
A heuristic most often translates an intuition that is not necessarily supported by
formal arguments.
Example with graph traversal algorithms
By now, you should be familiar with several graph traversal algorithms, such as
the BFS or DFS, the result of which is to obtain a tree covering a graph from a
given vertex.
If you think about it, using one route rather than another can be in itself a
heuristic.
For example, consider the following problem: from a wikipedia page, for
example the wikipedia page on the traveling salesman problem, we want to
arrive at another wikipedia page, for example the wikipedia page on the number
42, by following hypertext links.
Question: should a DFS or BFS be used?
There's no doubt that both methods will get the expected result, given the very
high connectivity of the wikipedia hyperlink graph, but they will certainly not get
there in the same way.
What is a good heuristic ?
So, what's a 'good' heuristic?
A heuristic is most often inspired by an intuition, and so it's not easy to evaluate
its quality beforehand.
A heuristic should have two important qualities : to provide a gain, and be
limited in terms of complexity. Let's examine these two features in some more
detail.
Provide a gain
To provide a gain, a heuristic should improve the performance of an algorithm
according to a given criterion and compared to other heuristics.
For example, imagine that I want to get from this classroom, the yellow X on the
map, to the canteen, the red X on the map, in a minimum number of steps.
One possible heuristic to get there is to move in what I know to be the general
direction of the canteen, ignoring the presence of walls or obstacles. This
heuristic is probably going to have a generally better outcome than if I use a
random orientation. But in some cases, I might find myself at a dead end, in
which case the heuristic does not provide a gain, and even worse, does not
provide a solution.
Limited complexity
With regards to the limited complexity characteristic, if using heuristics is as
costly as exploring all the possibilities, then there's no point - you might as well
not use it!
Using the same example of me finding my way from this classroom to the
canteen, let's assume that to find my way I have to explore all possible paths
and count the number of steps for each of them. I can then of course choose
the shortest path, but as I've had to explore every possibility before doing so, I
will have had to walk much farther than if I'd chosen a simpler heuristic.
Assessing the quality
To assess the quality of a heuristic, it's generally easier to perform this
evaluation ex post. This means that a choice on the heuristic is made,
implemented, and then tested on a large number of problem instances, and,
finally, the results are compared to the results obtained with other heuristics.
This ends today's subject on heuristics. In the next video, you'll learn how
heuristics can help to provide good enough solutions for the TSP.
