Informed search algorithms

Outline
• Greedy best-first search
• A* search
• Memory-bound search
• Bidirectional heuristic search
• Heuristics
 Review: Tree search
• A search strategy is defined by picking the
order of node expansion
•
 Best-first search
• Idea: use an evaluation function f(n) for each node
– estimate of "desirability“
– Expand most desirable unexpanded node

• Implementation:
Order the nodes in fringe in decreasing order of
desirability
• Special cases:
– greedy best-first search
– A* search

•
Romania with step costs in km
 Greedy best-first search
• Evaluation function f(n) = h(n) (heuristic) =
estimate of cost from n to goal

• e.g., hSLD(n) = straight-line distance from n

to Bucharest

• Greedy best-first search expands the node

that appears to be closest to goal
Greedy best-first search
example
Greedy best-first search
example
Greedy best-first search
example
Greedy best-first search
example
 Properties of greedy best-first
search
• Complete? No – can get stuck in loops,
e.g., Iasi  Neamt  Iasi  Neamt 
• Time? O(bm), but a good heuristic can give
dramatic improvement
• Space? O(bm) -- keeps all nodes in
memory
• Optimal? No
 A* search
• Idea: avoid expanding paths that are
already expensive
• Evaluation function f(n) = g(n) + h(n)
• g(n) = cost so far to reach n
• h(n) = estimated cost from n to goal
• f(n) = estimated cost of the best path that
continues from n to a goal
•
•
Romania with step costs in km
A search example
*
A search example
*
A search example
*
A search example
*
A search example
*
A search example
*
• A* is efficient because it prunes away search tree nodes that
are not necessary for finding an optimal solution.
• Timisoara has f = 447 and Zerind has f = 449 Even though
they are children of the root and would be among the first
nodes expanded by uniform-cost or breadth-first search, they
are never expanded by A* search because the solution with f =
417 is found first.
 Algorithm (A*)
• Initialization OPEN list with initial node; CLOSED= ; g =
0, f = h, Found = false;
• While (OPEN ≠  and Found = false )
{1
– Remove the node with the lowest value of f from OPEN to
CLOSED and call it as a Best_Node.
– If Best_Node = Goal state then Found = true else
{2
• Generate the Succ of Best_Node
• For each Succ do
{3
– Compute g(Succ) = g(Best_Node) + cost of getting from
Best_Node to Succ.
 A* - Contd..
– If Succ  OPEN then /* already being generated
but not processed */
{4
» Call the matched node as OLD and add it in the list
of Best_Node successors.
» Ignore the Succ node and change the parent of
OLD, if required.
- If g(Succ) < g(OLD) then make
parent of OLD to be Best_Node and change
the values of g and f for OLD
- If g(Succ) >= g(OLD) then
ignore
}4
 A* - Contd..
– If Succ  CLOSED then /* already processed */
{5
» Call the matched node as OLD and add it in the list
of Best_Node successors.
» Ignore the Succ node and change the parent of
OLD, if required
- If g(Succ) < g(OLD) then make parent of OLD to be
Best_Node and change the values of g and f for OLD.
- Propogate the change to OLD’s children using depth
first search
- If g(Succ) >= g(OLD) then do nothing
}5
 A* - Contd..
– If Succ  OPEN or CLOSED
{6
» Add it to the list of Best_Node’s successors
» Compute f(Succ) = g(Succ) + h(Succ)
» Put Succ on OPEN list with its f value
}6
}3 /* for loop*/
}2 /* else if */
}1 /* End while */
• If Found = true then report the best path else report
failure
• Stop
 Admissible heuristics
• A heuristic h(n) is admissible if for every node n,
h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal
state from n.
• key property is admissibility: An admissible heuristic never
overestimates the cost to reach the goal, i.e., it is optimistic

• Theorem: If h(n) is admissible, A* using TREE-

SEARCH is optimal.
•
Example: hSLD(n) (never overestimates the
actual road distance)
 Consistency
• A heuristic is consistent if, for every node
and every successor of generated by an
action we have:

• Theorem: If h(n) is consistent, A* using

GRAPH-SEARCH is optimal
Triangle inequality
 Admissibility of A*
• A search algorithm is admissible, if
– for any graph, it always terminates in an optimal path
from initial state to goal state, if path exists.
• If heuristic function h is underestimate of actual value
from current state to goal state, then the it is called
admissible function.
• Alternatively we can say that A* always terminates with
the optimal path in case
– h(x) is an admissible heuristic function.
 Example – Underestimation – f=g+h
Here h is underestimated
A

Underestimated
(1+3)B (1+4)C (1+5)D

3 moves away from goal

(2+3) E

3 moves away from goal

(3+3) F
 Explanation -Example of
Underestimation
• Assume the cost of all arcs to be 1. A is expanded to B, C and
D.
• ‘f’ values for each node is computed.
• B is chosen to be expanded to E.
• We notice that f(E) = f(C) = 5
• Suppose we resolve in favor of E, the path currently we are
expanding. E is expanded to F.
• Clearly expansion of a node F is stopped as f(F)=6 and so we
will now expand C.
• Thus we see that by underestimating h(B), we have wasted
some effort but eventually discovered that B was farther away
than we thought.
• Then we go back and try another path, and will find optimal
path.
 Example – Overestimation
Here h is overestimated
A
Overestimated

(1+3) B (1+4) C (1+5) D

(2+2) E

(3+1)F

(4+0) G
 Explanation –Example of
Overestimation
• A is expanded to B, C and D.
• Now B is expanded to E, E to F and F to G for a solution
path of length 4.
• Consider a scenario when there a direct path from D to
G with a solution giving a path of length 2.
• We will never find it because of overestimating h(D).
• Thus, we may find some other worse solution without
ever expanding D.
• So by overestimating h, we can not be guaranteed to
find the cheaper path solution.
 Search contours
• A useful way to visualize a search is to draw contours in the state space, just like the
contours in a topographic map. Inside the contour labeled 400, all nodes have
 Monotonicity
• A heuristic function h is monotone if
  states Xi and Xj such that Xj is successor of
Xi
h(Xi)– h(Xj) ≤ cost (Xi, Xj)
where, cost (Xi, Xj) actual cost of going from Xi to Xj
– h (goal) = 0

• In this case, heuristic is locally admissible i.e.,

consistently finds the minimal path to each state
they encounter in the search.
• The monotone property in other words is that
search space which is every where locally consistent
with heuristic function employed i.e., reaching each
state along the shortest path from its ancestors.
• With monotonic heuristic, if a state is rediscovered,
it is not necessary to check whether the new path is
shorter.
• Each monotonic heuristic is admissible
• A cost function f(n) is monotone if
f(n) ≤f(succ(n)), n.
• For any admissible cost function f, we can
construct a monotone admissible function.
 Satisficing search: Inadmissible heuristics and
weighted A* Inadmissible heuristic

Inadmissible heuristics

•A* search has many good qualities, but it expands a lot

of nodes.
•allow A* search to use an inadmissible heuristic—one
that may overestimate with the heuristic can potentially
be more accurate, thereby reducing the number of nodes
expanded.
Weighted A* search
• Weighted A* search: weight the heuristic value more heavily,
the evaluation function f(n) = g(n) + W × h(n), W > 1.

• Example: road engineers know the concept of a detour index,

which is a multiplier applied to the straight-line distance to
account for the typical curvature of roads.
• A detour index of 1.3: if two cities are 10 miles apart in
straight-line distance, a good estimate of the best path between
them is 13 miles.
• For most localities, the detour index ranges between 1.2 and
1.6.
• Searches that evaluate states by combining and in various
ways; weighted A* can be seen as a generalization of the
others:
• A* search: g(n) + h(n) (W = 1)
• Uniform-cost search: g(n) (W = 0)
• Greedy best-first search: h(n) (W = ∞)
• Weighted A* search: g(n) + W × h(n) (1 < W < ∞)
 Properties of A*
• Complete? Yes
(unless there are infinitely many nodes with f ≤ f(G);
can’t happen if step-cost  ε > 0)
• Time/Space? Exponential O(bd)
except if:
• Space? Keeps all nodes in memory
• Optimal? Yes
* *
| h (n )  h (n ) |O
(with: Tree-Search, admissible heuristic;(logh (n))
Graph-Search, consistent heuristic)
• Optimally Efficient? Yes
(no optimal algorithm with same heuristic is guaranteed to expand fewer
nodes)
 Memory-bounded search
• The main issue with A* is its use of memory.
• Solution: Memory is split between the frontier and the reached
states
– some implementations keep a state in only one of the two places, saving a
bit of space at the cost of complicating (slowing down) the algorithm.
– only check the frontier for redundant paths, and to eliminate the reached
table.
– keep reference counts of the number of times a state has been reached,
and remove it from the reached table when there are no more ways to
reach the state.
– For example, on a grid world where each state can be reached only from
its four neighbours, once we have reached a state four times, we can
remove it from the table.
 Memory-bounded search
• Beam search
• Iterative-deepening A* search (IDA*)
• Recursive best-first search (RBFS)
 Beam search
• Beam search limits the size of the frontier
• keep only the nodes with the best f-scores,
discarding any other expanded nodes
Iterative-deepening A* search (IDA*)
• At each iteration, perform a DFS cutting off a branch when
its total cost (g+h) exceeds a given threshold.

• Threshold(contour): finds a node just beyond that

contour, and uses that node’s -cost as the next
contour.
• This threshold starts at the estimate of the cost of the initial
state, and increases for each iteration of the algorithm.

• At each iteration, the threshold used for the next iteration

is the minimum cost of all values exceeded the current
threshold.

• In other words, each iteration exhaustively searches an f-

contour, finds a node just beyond that contour, and uses
that node’s f-cost as the next contour.
Ist iteration ( Threshhold = 5) O 5

O O O O O
6 8 4 8 9
   

IInd iteration ( Threshhold = 6) O 5

O O

O O O O O O
7 5 9 8 4 7
   

IIIrd iteration ( Threshhold = 7) O 5

O O

O O O

O O O O O O Goal
8 9 4 4 8
  
 Contd..
• Given an admissible monotone cost function, IDA* will find a
solution of least cost or optimal solution if one exists.

• IDA* not only finds cheapest path to a solution but uses far
less space than A* and it expands approximately the same
number of nodes as A* in a tree search.

• An additional benefit of IDA* over A* is that it is simpler to

implement, as there are no open and closed lists to be
maintained.

• A simple recursion performs DFS inside an outer loop to

handle iterations.
• Each iteration exhaustively searches an f-contour,
• For problems like the 8-puzzle where each path’s f-cost is an
integer, this works very well, resulting in steady progress
towards the goal each iteration.
• If the optimal solution has cost then there can be no more than
iterations (for example, no more than 31 iterations on the
hardest 8-puzzle problems).
• But for a problem where every node has a different f-cost,
each new contour might contain only one new node, and the
number of iterations could be equal to the number of state.
 Recursive best-first search (RBFS)
• RBFS resembles a recursive depth first search, but rather than
continuing indefinitely down the current path, it uses the
f_limit variable to keep track of the f-value of the best
alternative path available from any ancestor of the current
node.

• If the current node exceeds this limit, the recursion unwinds

back to the alternative path.

• RBFS replaces the f-value of each node along the path with a
backed-up value—the best f-value of its children.
Stages in an RBFS search for the shortest route to Bucharest. The f-limit value for each
recursive call is shown on top of each current node, and every node is labeled with its -
cost. (a) The path via Rimnicu Vilcea is followed until the current best leaf (Pitesti) has
a value that is worse than the best alternative path (Fagaras). (b) The recursion unwinds
and the best leaf value of the forgotten subtree (417) is backed up to Rimnicu Vilcea;
then Fagaras is expanded, revealing a best leaf value of 450.
(c) The recursion unwinds and the best leaf value of the forgotten subtree
(450) is backed up to Fagaras; then Rimnicu Vilcea is expanded. This time,
because the best alternative path (through Timisoara) costs at least 447,
the expansion continues to Bucharest.
• RBFS is somewhat more efficient than IDA*, but still suffers from
excessive node regeneration.

• IDA* and RBFS suffer from using too little memory.

• Between iterations, IDA* retains only a single number: the current f-cost
limit.

• RBFS retains more information in memory, but it uses only linear space:
even if more memory were available, RBFS has no way to make use of it.
Because they forget most of what they have done.

• both algorithms may end up reexploring the same states many times over.
 MA* (memory-bounded A*) and
SMA* (simplified MA*)
• SMA* proceeds just like A*, expanding the best leaf
until memory is full.
• At this point, it cannot add a new node to the search
tree without dropping an old one.
• SMA* always drops the worst leaf node—the one
with the highest f-value.
• Like RBFS, SMA* then backs up the value of the
forgotten node to its parent.
• SMA* is a fairly robust choice for finding optimal solutions, particularly
when the state space is a graph, action costs are not uniform, and node
generation is expensive compared to the overhead of maintaining the
frontier and the reached set.
 Bidirectional heuristic search
• Nodes going in the forward direction (with the initial state
as root)

• Nodes in the backward direction (with a goal state as

root)

• Forward and backward searches are solving the same

problem, they have different evaluation functions.
• Lower bound on the cost of a solution that follows the
path from the initial state to then somehow gets to then
follows the path to the goal as

• Evaluation function that mimics the criteria lb

f2(n) = max(2g(n), g(n) + h(n))

• f2 function guarantees that we will never expand a node

(from either frontier) with
• Bidirectional search is sometimes more efficient
than unidirectional search, sometimes not.

• bidirectional search has the same asymptotic

complexity as A*.

• Bidirectional search with the f2 evaluation

function and an admissible heuristic h is
complete and optimal.
 Heuristic Functions
• How the accuracy of a heuristic affects
search performance, and also consider
how heuristics can be invented?
Example: Eight puzzle problem (EPP)
 Evaluation function - f
Example: Solve Eight puzzle problem using A* algorithm

 Evaluation function f (X) = g (X) + h(X)

h1 (X) = the number of tiles not in their goal
position in a given state X
g(X) = depth of node X in the search tree
 Initial node has f(initial_node) = 8
 Apply A* algorithm to solve it.
 The choice of evaluation function critically determines search
results.
 Start State
Search Tree f = 0+4
3 7 6
5 1 2
4 8

up left right
(1+3) (1+5) (1+5)
3 7 6 3 7 6 3 7 6
5 2 5 1 2 5 1 2
4 1 8 4 8 4 8

up left right
(2+3) (2+3) (2+4)
3 6 3 7 6 3 7 6
5 7 2 5 2 5 2
4 1 8 4 1 8 4 1 8

left right
(3+2) (3+4)
3 6 3 6
5 7 2 5 7 2
4 1 8 4 1 8
down
(4+1)
5 3 6 right 5 3 6 Goal
7 2 7 2 State
4 1 8 4 1 8
 Harder Problem
• Harder problems (8 puzzle) can’t be solved by
heuristic function defined earlier.

•
•A better estimate function is to be thought.
h2 (X) = the sum of the distances of the tiles
from their goal position in a given state X
• Initial node has h2(initial_node) = 3+1+2+2+2+3+3+2 = 18
 Admissible heuristics
E.g., for the 8-puzzle:
• h1(n) = number of misplaced tiles
• h2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

• h1(S) = ? 8
• h2(S) = ? 3+1+2+2+2+3+3+2 = 18
•
Effect of heuristic accuracy on performance

• Effective branching factor and Effective depth.

• Effective branching factor b*: quality of a heuristic.

• For example, if A* finds a solution at depth 5
using 52 nodes, then the effective branching
factor is 1.92.

• A well-designed heuristic would have a value of

close to 1, allowing fairly large problems to be
solved at reasonable computational cost.
 Effective depth

• Effect of A* pruning with a given heuristic is that it reduces

the effective depth by a constant compared to the true
depth.

• This means that

for an uninformed search.
 Dominance
• If h2(n) ≥ h1(n) for all n (both admissible)
• then h2 dominates h1
• h2 is better for search
• Every node that is surely expanded by A* search with h2 is also
surely expanded with h1 and might cause other nodes to be expanded
as well.
• Typical search costs (average number of nodes
expanded):

• d=12 IDS = 3,644,035 nodes

A*(h1) = 227 nodes
A*(h2) = 73 nodes

• d=24 IDS = too many nodes

A*(h1) = 39,135 nodes
A*(h2) = 1,641 nodes
Comparison of the search costs and effective branching
factors for 8-puzzle problems using breadth-first search, A*
h1 and h2. Data are averaged over 100 puzzles for each
solution length d from 6 to 28.
 Relaxed problems
• A problem with fewer restrictions on the actions
is called a relaxed problem

• If the rules of the 8-puzzle are relaxed so that a

tile can move anywhere, then h1(n) gives the
shortest solution

• If the rules are relaxed so that a tile can move to

any adjacent square, then h2(n) gives the
shortest solution
Generating heuristics from subproblems:
Pattern databases

• A subproblem of the 8-puzzle instance.

• The task is to get tiles 1, 2, 3, 4, and the blank into their correct
positions, without worrying about what happens to the other tiles.
• The cost of the optimal solution of this subproblem is a lower bound
on the cost of the complete problem.
• It turns out to be more accurate than Manhattan distance in some
cases
 Generating heuristics with landmarks

• Example: online services that host maps with tens of

millions of vertices and find cost optimal driving
directions in milliseconds.

• Best search algorithms we have considered so far are

about a million times slower?

• precomputation of some optimal path costs.

• precomputation can be time-consuming, it need only be
done once, and then can be amortized over billions of
user search requests.
• A better approach is to choose a few (10 or 20) landmark
points from the vertices.
 Learning to search better
• Metalevel state space: Each state in a metalevel state
space captures the internal (computational) state of a
program that is searching in an ordinary state space
such as the map of Romania.

• Learn from the experiences to avoid exploring

unpromising subtrees.
 Learning heuristics from
experience
• Machine learning
• Reinforcement learning