Moving On…

 Two-player adversary search is nice,

10. Single-agent Search but not all interesting problems can be
mapped to games
 Large class of optimization problems
Jonathan Schaeffer that all have the same search properties
[email protected]  Find the best search value from the
perspective of a single player
www.cs.ualberta.ca/~jonathan
 Single-agent search

1 9/9/02 2

Applications Why Alpha-Beta First?

 Pathfinding  Many of the performance
 Dynamic programming enhancements we saw in alpha-beta
 Job shop scheduling translate to single-agent search
 DNA sequence alignment  Most originated with alpha-beta, and
 Scheduling were adopted by other classes of
 Planning search algorithms
 Constraint satisfaction
 …
9/9/02 3 9/9/02 4

1
 Application: Pathfinding Application
 Consider a sample application
 Find a minimal cost path from a start
node to a goal node GOAL
 Can move one square horizontally or
vertically, each with a cost of one
 Can be generalized to include diagonals
 Can be generalized to include variable
costs
START
9/9/02 5 9/9/02 6

Solution 1 Solution 1
 Trivial solution
7 6 7 7
 Explore outward from the start node
GOAL
until reaching the goal node 6 5 7 6 7

 Can use iterative deepening to 5 4 5 6

guarantee minimal cost path
4 3 2 3 4 5
 Try paths of length 1, then 2, etc.
3 1 2 4
START
2 1 0 1 2 3
9/9/02 7 9/9/02 8

2
 Solution 1 Solution 2
 Note that more than one path can lead  Trivial observation that searching to
to a node depth 1 is a waste of time since we are
 Some of these paths are non-optimal obviously more than 1 away from the
 Note that cycles are possible goal
 Observation: we need to eliminate  Add to the search an evaluation
duplicate states! function that estimates the distance to
the goal
 What is a simple estimator of distance?
9/9/02 9 9/9/02 10

Solution 2 Manhattan Distance

 For pathfinding, a good estimate of distance
to go is the Manhattan distance
 Number of horizontal and vertical moves to the GOAL
goal node
 Cost of reaching a node is now two parts:
 Distance already traveled
 Estimate of distance to go
 If the cost of a node exceeds the iterative
deepening threshold, then stop searching that
path START 1 + 4 =5

9/9/02 11 9/9/02 12

3
 Pathfinding IDA*
 Iterative deepening A*
 The cost of a node is (using A* terms)
GOAL  f=g+h
5+2=7 7+0=7 6+1=7
 g = cost incurred to get to this node
4+3=7 5+2=7  h = heuristic estimate of getting to goal
 Iterative deepening iterates on a threshold
3+4=7 2+3=5 3+2=5 4+3=7  Search a node as long as f <= threshold
 Either find a solution (done), or fail, in which case
1+4=5 2+3=5
the threshold is increased and a new search started
START
1+6=7 0+5=5 1+4=5 2+5=7
9/9/02 13 9/9/02 14

IDA* (1) IDA* (2)

IDA*( state s, int g, threshold t ) {
threshold = Eval( s );
h = Eval( s );
done = false; if( h == 0 ) return( true );
f = g + h;
while( not done ) { if( f > threshold ) return( false );
done = IDA*( s, 0, threshold ); for( i = 1; i <= numchildren; i++ ) {
done = IDA*( s.child[ i ], g + cost( child[ i ] ), t );
if( done == false ) threshold++; if( done == true ) return( true );
}
}
return( false );
}

9/9/02 15 9/9/02 16

4
 IDA* Comments IDA* Tree
 Automatically builds a variable-depth search
 Provably bad lines are cutoff as soon as possible  Depth-first search
 When the cutoff occurs depends on the quality of
the evaluation function  Root’s value = T
v <= T
 Storage requirements are trivial; just the  Search nodes <= T
recursion stack  Search nodes <= T+1
 Iteration i+1 repeats all the work of iteration i! v <= T+1
 Repeat until solution
 For some domains you can do better than
iterate by 1 v <= T+2
 Use the mimimum f-value seen at a leaf node
during an iteration as the next threshold
9/9/02 17 9/9/02 18

IDA* Comments Manhattan Distance

 Is IDA* guaranteed to produce an  Computes a direct path from a node to
optimal answer? the goal
 Yes!  Ignores all obstacles, which can only
 But only if… lengthen the path
 The evaluation function has to be  Therefore it is an admissible heuristic
admissible:
 It must always be a lower bound on the
true solution length

9/9/02 19 9/9/02 20

5
 Monotonicity Examining h
 Most admissible heuristics also have  Simplified cost of a search
the monotonicity property  Uniform branching factor b
 The f values never decrease along a  Search depth d
path if monotonicity holds  Ignore all other enhancements
 If you have a non-monotonic heuristic,  No heuristic: bd
one can always modify the search to  Average heuristic value is h: bd-h
make the heuristic monotonic…
 The quality of the heuristic has an enormous
 How?
impact on the search efficiency

9/9/02 21 9/9/02 22

Examining h Eliminating Redundant Nodes

 What does it mean to iterate?  Need to eliminate duplicate nodes
 If the first iteration finds an answer, then  Trivial optimization for many domains is
h had no error to disallow move reversals
 If a second iteration is required, then  For more sophisticated detection of
there is an error of 1 in h redundant nodes, we can use a
 The number of iterations indicates the transposition table
degree of error in h

9/9/02 23 9/9/02 24

6
 Transposition Table Sliding Tile Puzzle
 Store the t and g values in the table, and only
search a transpositon node with the smallest
g, and only once for the current t
 Use table only to indicate which nodes not to Sam Lloyd’s
search creation was the
 No need to store values, since the search Rubik’s Cube of
stops when a solution is found
the 1800s.
 All other TT issues (table size, hashing, table
entry replacement) remain the same as for
two-player games
9/9/02 25 9/9/02 26

Experiments A*
 Korf problem set of 100 positions  Single-agent search began in the 1960s
with the A* algorithm [2]
 Search 36700  This algorithm dominated AI search for
 Search - move reversals 100 two decades, but has competition now
from IDA*
 Search + TT (256K) 37
 Why teach IDA* first? Easy to explain
once you’ve seen Alpha-Beta

9/9/02 27 9/9/02 28

7
 A* Search Frontier
 Each iteration of IDA* re-searches the
7 6 7 7
tree over again beginning at the root
GOAL
 All that overhead can be eliminated… 6 5 7 6 7

 … by keeping track of the search 5 4 5 6

frontier, and only expanding nodes on 4 3 2 3 4 5
the frontier
3 1 2 4
 A* is a best-first search algorithm
START
2 1 0 1 2 3
9/9/02 29 9/9/02 30

A* Data Structure A* Algorithm (1)

 OpenList  Take best (first) node from OpenList
 List of nodes in the tree that are not yet  Check for solution
fully considered  Expand all the children
 Ordered from best to worst f value  Move node to the ClosedList
 ClosedList  As far as we know, done with this node
 Nodes that have been fully expanded
 No longer on any optimal path

9/9/02 31 9/9/02 32

8
 A* Algorithm (2) A* (1)
 Expanding a child A*( state s ) {
s.g = 0; s.h = Eval( s ); s.f = s.g + s.h; s.parent = null;
 Check if seen before Open/ClosedList done = false;
 If the node has been seen before with the same or better push s on OpenList
g value, then reject while( OpenList != empty && done == false ) {
 Add to OpenList for consideration pop s from head of OpenList
if( s is a goal node ) { done = true; break; }
 In effect the lists act as a cache of previously foreach( i = i; i <= Children( s ); i++ ) {
seen results Consider( s, s.child[i ] );
 NOTE: the algorithm requires all nodes to be }
add s to ClosedList
in these lists, unlike a TT }
return( done );
9/9/02 33 9/9/02 } 34

A* (2) Example
Consider( state from, state to ) {
newg = from.g + Cost( from, to ); 6
if( ( to is in OpenList or ClosedList ) and
5 GOAL
( to.g <= newg ) ) return;
to.g = newg; to.h = Eval( to ); 4
to.f = to.g + to.h; to.parent = from;
if( to is in ClosedList ) remove to from ClosedList 3
if( to is not in OpenList ) insert to in OpenList sorted 2
by f-value
} 1 START

9/9/02 35 9/9/02 A B C D E F 36

9
 Example Example
 Step 1: Initialize  Step 3: Expand C2
 ( C1, 0 + 5 = 5, null )  ( C3, 2 + 3 = 5, C2 ) ( D1, 1 + 4 = 5, C1 )
 () ( D2, 2 + 3 = 5, C2 ) ( B1, 1 + 6 = 7, C1 )
 Step 2: Expand C1  ( C1, 0 + 5 = 5, null ) (C2, 1 + 4 = 5, C1 )
 ( C2, 1 + 4 = 5, C1 ) (D1, 1 + 4 = 5, C1 )  Why isn’t C1 added to the OpenList?
( B1, 1 + 6 = 7, C1 )
 C1 is found in the ClosedList with a lower
 ( C1, 0 + 5 = 5, null ) g value

9/9/02 37 9/9/02 38

Example Sorting Open List

 Step 4: Expand C3  Sort by increasing f value, but what
 ( D3, 3 + 2 = 5, C3 ) ( D1, 1 + 4 = 5, C1 ) about ties?
( D2, 2 + 3 = 5, C2 ) ( B1, 1 + 6 = 7, C1 )  Break ties based on g value
( B3, 3 + 4 = 7, C3 )
 Larger g values mean more accurate
 ( C1, 0 + 5 = 5, null ) (C2, 1 + 4 = 5, C1 ) information and less heuristic
( C3, 2 + 3 = 5, C2 ) approximation

9/9/02 39 9/9/02 40

10
 A* IDA* versus A*
 Does not have the iterative overhead of  For many types of problems, IDA* flounders
IDA* in the cost of the re-searches, causing many
to prefer A* over IDA*
 Only expands nodes that are shown to  Why?
be relevant  But… IDA* is handicapped with no storage!
 Needs to maintain a history of all nodes  A* uses a closed list -- in effect a perfect cache of
previously searched previously seen states
IDA* uses almost no storage
 In practice, faster than IDA*, but A* runs 

 IDA* with a transposition table can be competitive

out of memory very quickly! with A*

9/9/02 41 9/9/02 42

Which to Choose? References

 IDA* is guaranteed to work, albeit [1] R. Korf. “Best-first Iterative-Deepening: An
possibly more slowly Optimal Admissible Tree Search”, Artificial
 A* is more efficient, but can run out of Intelligence, vol. 27, no.1, pp. 97-109, 1985.
memory [2] P. Hart, N. Nilsson and B. Raphael. “A
 Can also run slower because of cache Formal Basis for the Heuristic Determination
effects of Minimum Cost Paths”, IEEE Trans. Syst.
 The right choice depends on properties Sci. Cyber., vol. 4, no. 2, pp. 100-107, 1968.
of your application

9/9/02 43 9/9/02 44

11