Solving problems by searching

UNIT-II

1
 Problem-solving agents
 Decide what to do by finding sequences of actions
that lead to desirable states.
 Goal formulation: first step of problem solving.
Depends on current situation and agent’s
performance measure.
 Actions: cause transitions between world states.
 Problem formulation: deciding which actions and
states to consider, given a goal.
 Contd.
 Search
 considering sequences of actions that lead to states of

the world of known value.

 choosing the best one.

 Search algorithm
 Takes a problem as input.

 Returns a solution in the form of an action sequence.

 Execution
 Carrying out the actions sequence.

 Agent design
 formulate, search, execute
Problem-solving agents

4
 Example: Romania
 On holiday in Romania; currently in Arad.
 Flight leaves tomorrow from Bucharest.
 Formulate goal:
 be in Bucharest.
 Formulate problem:
 states: various cities.
 actions: drive between cities.
 Find solution:
 sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharest

5
Example: Romania

6
 Single-state problem formulation
A problem is defined by four items:

 initial state e.g., "at Arad“

 actions or successor function S(x) = set of (action,state) pairs
e.g., S(Arad) = {<Arad  Zerind, Zerind>, … }
 goal test, can be
explicit, e.g., x = "at Bucharest"
implicit, e.g., Checkmate(x)
 path cost (additive)
e.g., sum of distances, number of actions executed, etc.
c(x,a,y) is the step cost, assumed to be ≥ 0

A solution is a sequence of actions leading from the initial state to a goal

state. Optimal path is the one with lowest path cost.

7
 Selecting a state space
 Real world is absurdly complex
 state space must be abstracted for problem solving
 (Abstract) state = set of real states
 (Abstract) action = complex combination of real actions
 e.g., "Arad  Zerind" represents a complex set of possible routes,
detours, rest stops, etc.
 (Abstract) solution =
 set of real paths that are solutions in the real worl.
 Each abstract action should be "easier" than the original
problem.

8
 Vacuum world state space graph

 states?
 actions?
 goal test?
 path cost? 9
 Vacuum world state space graph

 states? Integer,dirt and robot location

 actions? Left, Right, Suck
 goal test? no dirt at all locations
 path cost? 1 per action
10
 Example: The 8-puzzle

 states?
 actions?
 goal test?
 path cost?

11
 Example: The 8-puzzle

 states? locations of tiles

 actions? move blank left, right, up, down
 goal test? = goal state (given)
 path cost? 1 per move
12
 Example: robotic assembly

 states?: real-valued coordinates of robot joint

angles parts of the object to be assembled
 actions?: continuous motions of robot joints
 goal test?: complete assembly
 path cost?: time to execute
13
 Tree search algorithms
 Search Process
 Building a search tree.
 Search Tree
 Generated by the initial state and the successor function.
 Expanding
 Apply successor function to the current state.
 Generating
 Result of expansion
 Search strategy
 The choice of which state to expand.
14
 Tree search algorithms
 Node data structure
 State: state in the state space.
 Parent-node: reference to parent.
 Action: operator applied to generate node.
 Depth: depth of node.
 Path-cost: path cost to node.
 Fringe
 The collection of nodes that have been
generated but not yet expanded.
15
 Tree search algorithms
 Basic idea:
 Simulated exploration of state space by generating
successors of already-explored states.

16
Tree search example

17
Tree search example

18
Tree search example

19
 Search strategies
 A search strategy is defined by picking the order of node
expansion.
 Strategies are evaluated along the following dimensions:
 completeness: Is the algorithm guaranteed to find a solution when
there is one?
 time complexity: How long does it take to find a solution?
 space complexity: How much memory is needed to perform the
search?
 optimality: Does it find an optimal solution?
 Let
 b is max no of successors of any node.
 d is depth of the shallowest goal node.
 m is max length of any path.

20
 Uninformed search strategies
 Uninformed or blind search strategies use only the
information available in the problem definition.
 Strategy is informed or heuristic if it knows that one non-
goal state is more promising than other.
 Blind search
 Breadth-first search
 Uniform-cost search
 Depth-first search
 Depth-limited search
 Iterative deepening search
 Bidirectional search

21
 Breadth-first search
 Expand root node first, then its successors and then their
successors and so on.
 Implementation:
 fringe is a FIFO queue, i.e., new successors go at end.

 Shallow nodes are expanded before deeper nodes.

22
 Breadth-first search
 Expand root node first, then its successors and then their
successors and so on.
 Implementation:
 fringe is a FIFO queue, i.e., new successors go at end.

 Shallow nodes are expanded before deeper nodes

23
 Breadth-first search
 Expand root node first, then its successors and then their
successors and so on.
 Implementation:
 fringe is a FIFO queue, i.e., new successors go at end.

 Shallow nodes are expanded before deeper nodes

24
 Breadth-first search
 Expand root node first, then its successors and then their
successors and so on.
 Implementation:
 fringe is a FIFO queue, i.e., new successors go at end.

 Shallow nodes are expanded before deeper nodes

25
 Properties of breadth-first search

 Complete? Yes (if b is finite)

 Time? 1+b+b2+b3+… +bd + (bd+1-b) = O(bd+1)
 Space? O(bd+1) (keeps every node in memory)
 Optimal? Yes (if cost = 1 per step)
 Space is the bigger problem (more than time).

26
 Uniform-cost search
 Optimal with any step cost.
 Expand least-cost unexpanded n nodes.
 Implementation:
 fringe = queue ordered by path cost
 Equivalent to breadth-first if step costs all equal

 Complete? Yes, if step cost ≥ ε

 Time? # of nodes with path cost ≤ cost of optimal solution,
O(b(C*/ ε)) where C* is the cost of the optimal solution
 Space? same
 Optimal? Yes – nodes expanded in increasing path cost

27
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

28
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

29
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

30
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

31
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

32
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

33
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

34
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

35
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

36
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

37
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

38
 Depth-first search
 Expand deepest unexpanded node.
 Implementation:
 fringe = LIFO queue, i.e., put successors at front.

39
 Properties of depth-first search
 Complete? No: fails in infinite-depth spaces.
 complete in finite spaces
 Time? O(bm): terrible if m is much larger than d
 but if solutions are dense, may be much faster than
breadth-first.
 Space? O(bm), i.e., linear space.
 Optimal? No

40
 Depth-limited search
 For unbounded trees
 depth-first search with depth limit l i.e. nodes at depth l
have no successors.
 Incomplete if l<d.
 Non optimal if l>d.

41
 Iterative deepening search
 Finds the best depth limit.
 Gradually increases the limit until a goal is found.
 Combines benefits of BFS and DFS.
 States are generated multiple times.

42
Iterative deepening search l =0

43
Iterative deepening search l =1

44
Iterative deepening search l =2

45
Iterative deepening search l =3

46
 Iterative deepening search
 Number of nodes generated in an iterative
deepening search to depth d with branching factor
b:
NIDS = d b1 + (d-1)b2 + … + 3bd-2 +2bd-1 + 1bd

47
 Properties of IDS
 Complete? Yes
 Time? O(bd)
 Space? O(bd)
 Optimal? Yes.

48
 Bidirectional search
 Idea is to run two simultaneous searches.
 One forward from the initial state
 One backward from the goal.
 Stops when the two searches meet in the middle.
 Motivation is (bd/2)+(bd/2) is much less than (bd).
 Searching backward is not an easy job.

49
Bidirectional search

50
Summary of algorithms

51
Informed Search Algorithms
 Informed Search Strategies
• These use problem-specific knowledge.
• Finds solutions more efficiently.
• General approach is best-first search.
 Best-first search
• Idea: use an evaluation function f(n) for selecting a node
for expansion.
– estimate of "desirability“
– Expand most desirable unexpanded node

• Implementation:
priority queue, a data structure that will maintain the
fringe in ascending order of f-values.
 Best-first search
• Family of best first search algorithms exists with different
evaluation functions.
• A key component is a heuristic function h(n):
h(n) = estimated cost of the cheapest path from node n to
a goal node.
• If n is goal node, h(n)=0.
• Special cases:
– greedy best-first search
– A* search
 Greedy best-first search
• Evaluation function f(n) = h(n) (heuristic)
= estimate of cost from n to goal.
• e.g., hSLD(n) = straight-line distance from n
to Bucharest.
• Greedy best-first search expands the node
that appears to be closest to goal.
Romania with step costs in km
Greedy best-first search
example
Greedy best-first search
example
Greedy best-first search
example
Greedy best-first search
example
 Properties of greedy best-first
search
• Complete? No – can get stuck in loops,
e.g., Iasi  Neamt  Iasi  Neamt 
• Time? O(bm), but a good heuristic can give
dramatic improvement
• Space? O(bm) -- keeps all nodes in
memory
• Optimal? No
 A* search
• Idea: avoid expanding paths that are
already expensive.
• Evaluation function f(n) = g(n) + h(n).
• g(n) = cost so far to reach n.
• h(n) = estimated cost from n to goal.
• f(n) = estimated total cost of path through
n to goal.
A* search example
A* search example
A* search example
A* search example
A* search example
A* search example
 Admissible heuristics
• A heuristic h(n) is admissible if for every node n,
h(n) ≤ h*(n), where h*(n) is the true cost to reach
the goal state from n.
• An admissible heuristic never overestimates the
cost to reach the goal, i.e., it is optimistic.
• Example: hSLD(n) (never overestimates the
actual road distance).
• Theorem: If h(n) is admissible, A* using TREE-
SEARCH is optimal.
 Optimality of A* (proof)
• Suppose some suboptimal goal G2 has been generated and is in the
fringe. Let n be an unexpanded node in the fringe such that n is on a
shortest path to an optimal goal G.
•

• f(G2) = g(G2) since h(G2) = 0

• g(G2) > g(G) since G2 is suboptimal
• f(G) = g(G) since h(G) = 0
• f(G2) > f(G) from above.
 Optimality of A* (proof)
• Suppose some suboptimal goal G2 has been generated and is in the
fringe. Let n be an unexpanded node in the fringe such that n is on a
shortest path to an optimal goal G.
•

• f(G2) > f(G) from above

• h(n) ≤ h*(n) since h is admissible
• g(n) + h(n) ≤ g(n) + h*(n)
• f(n) ≤ f(G)
Hence f(G2) > f(n), and A* will never select G2 for expansion.
 Properties of A*
• Complete? Yes.
• Time? Exponential.
• Space? Keeps all nodes in memory.
• Optimal? Yes.
 Admissible heuristics
• E.g., for the 8-puzzle:
• h1(n) = number of misplaced tiles
• h2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

• h1(S) = ?
• h2(S) = ?
 Admissible heuristics
• E.g., for the 8-puzzle:
• h1(n) = number of misplaced tiles
• h2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

• h1(S) = ? 8
• h2(S) = ? 3+1+2+2+2+3+3+2 = 18
 Dominance
• If h2(n) ≥ h1(n) for all n (both admissible)
• then h2 dominates h1
• h2 is better for search.

Learning heuristics from experience.

 Properties of Heuristic
Algorithms
• Admissibility: Algorithm A is admissible if it is guaranteed to
return an optimal solution when one exists.

• Completeness: Algorithm A is complete if it is guaranteed

to return a solution when one exists.

• Dominance: A1 dominates A2 if the heuristic function of A1

that of A2.

• Optimality: Algorithm A is optimal over a class of algorithms if

A dominates all members of the class.
 Local search algorithms
• In many optimization problems, the path to the
goal is irrelevant; the goal state itself is the
solution.
• State space = set of "complete" configurations.
• Find configuration satisfying constraints, e.g., n-
queens.
• In such cases, we can use local search
algorithms.
• keep a single "current" state, try to improve it.
 Example: n-queens
• Put n queens on an n × n board with no
two queens on the same row, column, or
diagonal.
 Hill-climbing search
• "Like climbing Everest in thick fog with
amnesia”.
Hill-climbing search
 Problems
• Local maxima: a local maximum is a peak that is higher
than each of its neighboring states but lower than the
global maximum. Hill-climbing algorithms that reach the
vicinity of a local maximum will be drawn upward toward
the peak but will then be stuck with nowhere else to go.
• Ridges: Ridges result in a sequence of local maxima
that is very difficult for greedy algorithms to navigate.
• Plateaux: a plateau is a flat area of the state-space
landscape. It can be a flat local maximum, from which no
uphill exit exists, or a shoulder, from which progress is
possible. A hill-climbing search might get lost on the
plateau.
Exploring the Landscape
local maximum

plateau

ridge
 Hill Climbing: Disadvantages
Local maximum
A state that is better than all of its
neighbours, but not better than some other
states far away.

33
 Hill Climbing: Disadvantages
Plateau
A flat area of the search space in which all
neighbouring states have the same value.

34
 Hill Climbing: Disadvantages
Ridge: The orientation of the high region,
compared to the set of available moves,
makes it impossible to climb up. However,
two moves executed serially may increase
the height.

35
 Simulated Annealing search
• Idea: escape local maxima by allowing some
"bad" moves but gradually decrease their
frequency.
• Annealing is the process used to temper or
harden metals and glass by heating them to a
high temperature and then gradually cooling
them, thus allowing the material to coalesce into
a low energy crystalline state.
 Simulated Annealing search
• Not best but picks a random move.
• If it improves situation, then accepted.
• Otherwise with some probability less than 1.
• The probability decreases exponentially with the
badness of the move
– The amount E by which the evaluation is worsened.
– As T goes down.
– Bad moves are allowed at the start then
decreases.
Simulated annealing search
 Simulated annealing search
• One can prove: If T decreases slowly enough,
then simulated annealing search will find a
global optimum with probability approaching 1

• Widely used in VLSI layout, airline scheduling,

etc.
 Local beam search
• Keep track of k states rather than just one.
• Start with k randomly generated states
• At each iteration, all the successors of all k
states are generated
• If any one is a goal state, stop; else select the k
best successors from the complete list and
repeat.
 Genetic algorithms
• A successor state is generated by combining two parent
states.
• Start with k randomly generated states (population).
• A state is represented as a string over a finite alphabet
(often a string of 0s and 1s).
• Evaluation function (fitness function). Higher values for
better states.
• Produce the next generation of states by selection,
crossover, and mutation
 Genetic algorithms

• Fitness function: number of non-attacking pairs of

queens (min = 0, max = 8 × 7/2 = 28)
• 24/(24+23+20+11) = 31%
• 23/(24+23+20+11) = 29% etc
Genetic algorithms
Algorithm
Example
Example
Contd.
Adversarial Search
 Overview of Games
 Games are a form of multi-agent environment
• What do other agents do and how do they affect our success?
• Cooperative vs. competitive multi-agent environments.
• Competitive multi-agent environments give rise to adversarial
search (games)
 Why study games?
• Fun; historically entertaining
• Interesting subject of study because they are hard
• Easy to represent and agents restricted to small number of actions
 Relation of Games to
Search
 Search – no adversary
• Solution is (heuristic) method for finding goal
• Heuristics techniques can find optimal solution
• Evaluation function: estimate of cost from start to goal through
given node
• Examples: path planning, scheduling activities
 Games – adversary
• Solution is strategy (strategy specifies move for every possible
opponent reply).
• Time limits force an approximate solution
• Evaluation function: evaluate “goodness” of
game position
• Example: chess,
 Game setup
 Two players: MAX and MIN
 MAX moves first and they take turns until the game is over.
Winner gets award, looser gets penalty.
 Games as search:
• Initial state: e.g. board configuration of chess
• Successor function: list of (move,state) pairs specifying
legal moves.
• Terminal test: Is the game finished?
• Utility function: Gives numerical value of terminal states.
E.g. win (+1), loose (-1) and draw (0) in tic-tac-toe (next)
 MAX uses search tree to determine next move.
Partial Game Tree: Tic-Tac-Toe
 Optimal strategies
 Find the contingent strategy for MAX assuming an
infallible MIN opponent.
 Assumption: Both players play optimally !!
 Given a game tree, the optimal strategy can be
determined by using the minimax value of each
node:

MINIMAX-VALUE(n)=
UTILITY(n) If n is a terminal
maxs  successors(n) MINIMAX-VALUE(s) If n is a max node
mins  successors(n) MINIMAX-VALUE(s) If n is a min node
 Minimax

The minimax decision

 What if MIN does not play
optimally?

 Definition of optimal play for MAX assumes MIN

plays optimally: maximizes worst-case outcome
for MAX.
 But if MIN does not play optimally, MAX will do
even better.
 Minimax Algorithm
function MINIMAX-DECISION(state) returns an action
inputs: state, current state in game
vMAX-VALUE(state)
return the action in SUCCESSORS(state) with value v
function MAX-VALUE(state) returns a utility value
if TERMINAL-TEST(state) then return UTILITY(state)
v∞
for a,s in SUCCESSORS(state) do
v  MAX(v,MIN-VALUE(s))
return v
function MIN-VALUE(state) returns a utility value
if TERMINAL-TEST(state) then return UTILITY(state)
v∞
for a,s in SUCCESSORS(state) do
v  MIN(v,MAX-VALUE(s))
return v
 Minimax Properties
Complete
• Yes, if tree is finite
Optimal
• Yes, against an optimal opponent
Time complexity
• O(bm)
Space complexity
• O(bm)
 Multiplayer games
 Games allow more than two players
 Single minimax values become vectors
- pruning
 Alpha-Beta Example
Do DF-search until first leaf

Range of possible values

[-∞,+∞]

[-∞, +∞]
Alpha-Beta Example
(continued)

[-∞,+∞]

[-∞,3]
Alpha-Beta Example
(continued)

[-∞,+∞]

[-∞,3]
Alpha-Beta Example
(continued)

[3,+∞]

[3,3]
Alpha-Beta Example
(continued)
[3,+∞]
This node is worse
for MAX

[3,3] [-∞,2]
Alpha-Beta Example
(continued)

[3,14] ,

[3,3] [-∞,2] [-∞,14]

 Alpha-Beta Example
(continued)

[3,5] ,

[3,3] [−∞,2] [-∞,5]

 Alpha-Beta Example
(continued)

[3,3]

[3,3] [−∞,2] [2,2]

 Alpha-Beta Example
(continued)

[3,3]

[3,3] [-∞,2] [2,2]

Alpha-Beta Algorithm
function ALPHA-BETA-SEARCH(state) returns an action
inputs: state, current state in game
vMAX-VALUE(state, - ∞ , +∞)
return the action in SUCCESSORS(state) with value v

function MAX-VALUE(state, , ) returns a utility value

if TERMINAL-TEST(state) then return UTILITY(state)
v-∞
for a,s in SUCCESSORS(state) do
v  MAX(v,MIN-VALUE(s,  , ))
if v ≥  then return v
  MAX( ,v)
return v
Alpha-Beta Algorithm

function MIN-VALUE(state,  , ) returns a utility value

if TERMINAL-TEST(state) then return UTILITY(state)
v+∞
for a,s in SUCCESSORS(state) do
v  MIN(v,MAX-VALUE(s,  , ))
if v ≤  then return v
  MIN( ,v)
return v
 General alpha-beta Pruning
 Consider a node n
somewhere in the tree
 If player has a better
choice at
• Parent node of n
• Or any choice point
further up
 n will never be reached in
actual play.
 Hence when enough is
known about n, it can be
pruned.