PROBLEM SOLVING

TECHNIQUES:
Heuristic Search
 Heuristic Search Techniques
• Direct techniques (blind search) are not
always possible (they require too much time
or memory).

• Weak techniques can be effective if applied

correctly on the right kinds of tasks.
(Heuristic Search Techniques)

– Typically require domain specific information.

 Example: 8 Puzzle

1 2 3 1 2 3
7 8 4 8 4
6 5 7 6 5
1 2 3 GOAL 1 23
8 4 7 84
7 6 5 6 5

right

1 23 1 23 123
7 84 7 84 7 4
6 5 6 5 6 85

Which move is best?

 8 Puzzle Heuristics
• Blind search techniques used an arbitrary
ordering (priority) of operations.
• Heuristic search techniques make use of
domain specific information - a heuristic.
• What heurisitic(s) can we use to decide
which 8-puzzle move is “best” (worth
considering first).
 8 Puzzle Heuristics
• For now - we just want to establish some
ordering to the possible moves (the values
of our heuristic does not matter as long as it
ranks the moves).
• Later - we will worry about the actual
values returned by the heuristic function.
 A Simple 8-puzzle heuristic
• Number of tiles in the correct position.
– The higher the number the better.
– Easy to compute (fast and takes little memory).
– Probably the simplest possible heuristic.
 Another approach
• Number of tiles in the incorrect position.
– This can also be considered a lower bound on
the number of moves from a solution!
– The “best” move is the one with the lowest
number returned by the heuristic.
– Is this heuristic more than a heuristic (is it
always correct?).
• Given any 2 states, does it always order them
properly with respect to the minimum number of
moves away from a solution?
1 2 3 GOAL 1 23
8 4 7 84
7 6 5 6 5

right

1 23 1 23 123
7 84 7 84 7 4
6 5 6 5 6 85
h=2 h=4 h=3
 Another 8-puzzle heuristic
• Count how far away (how many tile
movements) each tile is from it’s correct
position.
• Sum up this count over all the tiles.
• This is another estimate on the number of
moves away from a solution.
1 2 3 GOAL 1 23
8 4 7 84
7 6 5 6 5

right

1 23 1 23 123
7 84 7 84 7 4
6 5 6 5 6 85
h=2 h=4 h=4
 Techniques
• There are a variety of search techniques that
rely on the estimate provided by a heuristic
function.
• In all cases - the quality (accuracy) of the
heuristic is important in real-life application
of the technique!
 Hill Climbing
• Variation of generate-and-test approach:
– generation of next state depends on feedback from the
test procedure.
– Test now includes a heuristic function that provides a
guess as to how good each possible state is.
• There are a number of ways to use the information
returned by the test procedure.
• Searching for a goal state = Climbing to the top of
a hill
 Simple Hill Climbing
• Use heuristic to move only to states that are
better than the current state.

• Always move to better state when possible.

• The process ends when all operators have

been applied and none of the resulting states
are better than the current state.
 Simple Hill Climbing
Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or there are no
new operators left to be applied:
- Select and apply a new operator
- Evaluate the new state:
goal  quit
better than current state  new current state

15
 Simple Hill Climbing
Function Optimization

y = f(x)
y

x
 Potential Problems with
Simple Hill Climbing
• Will terminate when at local optimum.

• The order of application of operators can

make a big difference.

• Can’t see past a single move in the state

space.
 Steepest-Ascent Hill Climbing
• A variation on simple hill climbing.
• Instead of moving to the first state that is better,
move to the best possible state that is one move
away.
• The order of operators does not matter.
• Not just climbing to a better state, climbing up the
steepest slope.
• Considers all the moves from the current state.
• Selects the best one as the next state.
 Steepest-Ascent Hill Climbing
(Gradient Search)
Algorithm
1. Evaluate the initial state.
2. Loop until a solution is found or a complete
iteration produces no change to current state:
- SUCC = a state such that any possible successor of the
current state will be better than SUCC (the worst state).
- For each operator that applies to the current state, evaluate
the new state:
goal  quit
better than SUCC  set SUCC to this state
- SUCC is better than the current state  set the current
state to SUCC. 19
 Hill Climbing: Disadvantages
Local maximum
A state that is better than all of its neighbours,
but not better than some other states far
away.

20
 Hill Climbing: Disadvantages
Plateau
A flat area of the search space in which all
neighbouring states have the same value.

21
 Hill Climbing: Disadvantages
Ridge
The orientation of the high region, compared
to the set of available moves, makes it
impossible to climb up. However, two
moves executed serially may increase the
height.

22
 Hill Climbing: Disadvantages

Ways Out

• Backtrack to some earlier node and try

going in a different direction.
• Make a big jump to try to get in a new
section.
• Moving in several directions at once.
23
 Hill Climbing: Disadvantages
• Hill climbing is a local method:
Decides what to do next by looking only at
the “immediate” consequences of its
choices.
• Global information might be encoded in
heuristic functions.

24
Hill Climbing: Disadvantages
(Example)
Start Goal
A D
D C
C B
B A
Blocks World

25
 Hill Climbing: Disadvantages
Start Goal
A D
0 4
D C
C B
B A
Blocks World
Local heuristic:
+1 for each block that is resting on the thing it is supposed to
be resting on.
-1 for each block that is resting on a wrong thing.
26
Hill Climbing: Disadvantages
0 A 2

D D
C C
B B A

27
Hill Climbing: Disadvantages
D 2
C
B A

A 0
0
D
C C D C 0

B B A B A D

28
Hill Climbing: Disadvantages
Start Goal
A D
-6 6
D C
C B
B A
Blocks World
Global heuristic:
For each block that has the correct support structure: +1 to
every block in the support structure.
For each block that has a wrong support structure: -1 to
every block in the support
29 structure.
Hill Climbing: Disadvantages
D -3
C
B A

A -6
-2
D
C C D C -1

B B A B A D

30
 Hill Climbing: Conclusion
• Can be very inefficient in a large, rough
problem space.

• Global heuristic may have to pay for

computational complexity.

• Often useful when combined with other

methods, getting it started right in the right
general neighbourhood. 31
 Heuristic Dependence
• Hill climbing is based on the value assigned
to states by the heuristic function.
• The heuristic used by a hill climbing
algorithm does not need to be a static
function of a single state.
• The heuristic can look ahead many states, or
can use other means to arrive at a value for
a state.
 Best-First Search
• Combines the advantages of Breadth-First and
Depth-First searchs.
– DFS: follows a single path, no need to generate all
competing paths.
– BFS: doesn’t get caught in loops or dead-end-paths.
– Combining the two is to follow a single path at a time,
but switch paths whenever some competing path look
more promising than the current one.
• Best First Search: explore the most promising path
seen so far.
 Best-First Search (cont.)
While goal not reached:
1. Generate all potential successor states and
add to a list of states.
2. Pick the best state in the list and go to it.

• Similar to steepest-ascent, but don’t throw

away states that are not chosen.
 Best-First Search
A A A

B C D B C D
3 5 1 3 5
E F
4 6
A A

B C D B C D
5 5
G H E F G H E F
6 5 4 6 6 5 6
I J
2 1
35
 Best-First Search
• OPEN: nodes that have been generated, but
have not examined.
This is organized as a priority queue.

• CLOSED: nodes that have already been

examined.
Whenever a new node is generated, check
whether it has been generated before.
36
 Best-First Search
Algorithm
1. OPEN = {initial state}.
2. Loop until a goal is found or there are no
nodes left in OPEN:
- Pick the best node in OPEN
- Generate its successors
- For each successor:
new  evaluate it, add it to OPEN, record its parent
generated before  change parent, update successors
37
 Best-First Search
• Greedy search:
h(n) = estimated cost of the cheapest path
from node n to a goal state.
• Uniform-cost search:
g(n) = cost of the cheapest path from the
initial state to node n.

38
 Best-First Search
• Greedy search:
h(n) = estimated cost of the cheapest path
from node n to a goal state.
Neither optimal nor complete

• Uniform-cost search:
g(n) = cost of the cheapest path from the
initial state to node n.
Optimal and complete, but very inefficient
Example: City with straight-line dist.
Greedy best-first search example
Greedy best-first search example
Greedy best-first search example
Greedy best-first search example
 GBFS is not complete

c
b g
a goal state
start state

f(n) = straightline distance

 Properties of greedy best-first
search
• Complete? No – can get stuck in loops.
• Time? O(bm), but a good heuristic can give
dramatic improvement
• Space? O(bm) - keeps all nodes in memory
• Optimal? No
e.g. AradSibiuRimnicu
VireaPitestiBucharest is shorter!
 Best-First Search
• Algorithm A* (Hart et al., 1968):

f(n) = g(n) + h(n)

h(n) = cost of the cheapest path from node n to a

goal state.
g(n) = cost of the cheapest path from the initial
state to node n.

Under the right conditions A* provides the cheapest

cost solution in the optimal time! 47
 A * search
• Idea: avoid expanding paths that are already
expensive
• Evaluation function f(n) = g(n) + h(n)
• g(n) = cost so far to reach n
• h(n) = estimated cost from n to goal
• f(n) = estimated total cost of path through n to goal
• Greedy Best First search has f(n)=h(n)
• Uniform Cost search has f(n)=g(n)
 A* Algorithm
1. Create a priority queue of search nodes (initially the
start state). Priority is determined by the function f(n)
2. While queue not empty and goal not found:
get best state x from the queue.
If x is not goal state:
generate all possible children of x (and save path
information with each node).
Apply f to each new node and add to queue.
Remove duplicates from queue
x Use f(n) to pick the best
 Admissible heuristics
• A heuristic h(n) is admissible if for every node n,
h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal
state from n.
• An admissible heuristic never overestimates the cost to
reach the goal, i.e., it is optimistic
• Example: hSLD(n) (never overestimates the actual road
distance)
 Admissible heuristics
E.g., for the 8-puzzle:
• h1(n) = number of misplaced tiles
• h2(n) = the sum of the distances of the tiles from their goal positions
(Manhattan distance/city block distance)
(i.e., no. of squares from desired location of each tile)

• h1(S) = ?
• h2(S) = ?
 Admissible heuristics
E.g., for the 8-puzzle:
• h1(n) = number of misplaced tiles
• h2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

• h1(S) = ? 8
• h2(S) = ? 3+1+2+2+2+3+3+2 = 18
The average solution cost for a randomly generated 8-puzzle instance is about
22 steps. The branching factor is about 3.
 Dominance
• If h2(n) ≥ h1(n) for all n (both admissible)
• then h2 dominates h1
• h2 is better for search: it is guaranteed to expand
less or equal no. of nodes.

• Typical search costs (average number of nodes expanded-

d is the solution depth):

• d=12 IDS = 3,644,035 nodes

A*(h1) = 227 nodes
A*(h2) = 73 nodes
• d=24 IDS = too many nodes
A*(h1) = 39,135 nodes
A*(h2) = 1,641 nodes
A* Optimality and Completeness
• If the heuristic function h is admissible the
algorithm will find the optimal (shortest
path) to the solution in the minimum
number of steps possible (no optimal
algorithm can do better given the same
heuristic).
Exercise
 Constraint Satisfaction
• Many AI problems can be viewed as problems of
constraint satisfaction in which the goal is to discover
some problem state that satisfies a given set of constraints.

Cryptarithmetic puzzle:
SEND

MORE
MONEY

57
 Constraint Satisfaction
• As compared with a straightforard search procedure,
viewing a problem as one of constraint satisfaction
can reduce substantially the amount of search.

• Operates in a space of constraint sets.

• Initial state contains the original constraints given in

the problem.

• A goal state is any state that has been constrained

“enough”.
58
 Example: Crypt-arithmetic Puzzle

SEND
+ MOR E
----------
MONEY

Variables: S, E, N, D, M, O, R, Y Soln.:95
67
Domains: 1085
[0..9] for S, M, E, N, D, O, R, Y
====
Search space: 1,814,400 10652
Aside: could have [1..9] for S and M
 Constraints
Option 1:
C1a) 1000 S + 100 E + 10 N + D +
1000 M + 100 O + 10 R + E
= 10000 M + 1000 O + 100 N + 10 E + Y SEND
Or use 5 equality constraints, using auxiliary +MORE
“carry” variables C1, …, C4 Є [0…9] ----------
Option 2: C1b) MONEY
D+E = 10 C1 + Y Which constraint set better
C1 + N + R = 10 C2 + E for solving? C1a or C1b? Why?
C2 + E + O = 10 C3 + N
C3 + S + M = 10 C4 + O C1b, more “factored”. Smaller
C4 =M pieces. Gives more propagation!

Need two more sets of constraints:

C2) S =/= 0, M =/= 0
C3) S =/= M, S =/= O, … E =/= Y (28 not equal constraints)
Note: need to assert everything!
Alt. “All_diff(S,M,O,...Y)” for C3.
 Some Reflection:
Reasoning/Inference vs. Search

How do human solve this?

What is the first step?

1) M = 1, because M =/= 0 and …

the carry over of the addition of two
digits (plus previous carry) is at most 1.

Actually, a somewhat subtle

piece of mathematical background
Also, what made us focus knowledge.
on M?
Experience / intuition …
1) M = 1, because M =/= 0 and …
the carry over of the addition of two
digits (plus previous carry) is at most 1.

2) O = 0. Because M=1 and we have to have a carry to the next

column. S + 1 + C3 is either 10 or 11. So, O equals 0 or 1.
1 is taken. So, O = 0.

3) S = 9. There cannot be a carry to the 4th column (if

there were, N would also have to be 0 or 1. Already
taken.). So, S = 9.

A collection of “small pieces” of local reasoning,

using basic constraints from the rules of arithmetic.
A logic (SAT) based encoding will likely “get these steps.”
 And further it goes…

Largely, a clever chain of reasoning / inference /

propagation steps (no search) except for…
exploring 2 remaining options (i.e., search) to
find complete solution.
Human problem solving nicely captures key idea behind how to solve
CSPs: replace as much of search by propagation (inference/reasoning).

One difficulty: Humans often use subtle background knowledge.

Can search be completely avoided?

A.: Most likely NO. General cryptarithmetic is NP-complete

(Eppstein 1987)