CSE 359 (CEN 307) &

CSE 360 (CEN 308)

Artificial Intelligence
By
Md. Palash Uddin
Assistant Professor
Dept. of CSE
Hajee Mohammad Danesh Science and Technology
University, Dinajpur. 1
 Search Strategies
• A search strategy is defined by picking the order of node expansion

• Strategies are evaluated along the following dimensions:

– completeness: does it always find a solution if one exists?
– time complexity: number of nodes generated
– space complexity: maximum number of nodes in memory
– optimality: does it always find a least-cost solution?

• Time and space complexity are measured in terms of

– b: maximum branching factor of the search tree
– d: depth of the least-cost solution
– m: maximum depth of the state space (may be ∞)
 Evaluating Search Strategies
• Completeness
– Guarantees finding a solution whenever one exists
• Time Complexity
– How long (worst or average case) does it take to find a solution?
Usually measured in terms of the number of nodes expanded
• Space Complexity
– How much space is used by the algorithm? Usually measured in
terms of the maximum size that the “OPEN" list becomes
during the search
• Optimality/Admissibility
– If a solution is found, is it guaranteed to be an optimal one? For
example, is it the one with minimum cost?
 Search Algorithms for Goal-based Agent
• Uninformed Blind search
– Breadth-first
– uniform first
– depth-first
– Iterative deepening depth-first
– Bidirectional
– Branch and Bound
• Informed (heuristic) search
– Greedy best-first
– A*
– Memory-bounded heuristic search
– And more….
• Local search and optimization
– Hill-climbing
– Simulated annealing
– Genetic algorithms
 Uninformed search
• Search does not depend on the nature of solution

• Given a state, we only know whether it is a goal state or not

• Cannot say one nongoal state looks better than another nongoal state

• Can only traverse state space blindly in hope of somehow hitting a

goal state at some point
– Also called blind search

– Blind does not imply unsystematic!

 Breadth-First Search (BFS)
• Expand shallowest unexpanded node

• Fringe: nodes waiting in a queue to be explored, also called OPEN

• Implementation:
– For BFS, fringe is a first-in-first-out (FIFO) queue
– new successors go at end of the queue

• Repeated states?
– Simple strategy: do not add parent of a node as a leaf
 Breadth-First Search (BFS)
• Algorithm outline:
– Always select from the OPEN the node with the smallest depth for
expansion, and put all newly generated nodes into OPEN
– OPEN is organized as FIFO (first-in, first-out) list, I.e., a queue.
– Terminate if a node selected for expansion is a goal
• Properties
– Complete
– Optimal (i.e., admissible) if all operators have the same cost. Otherwise,
not optimal but finds solution with shortest path length (shallowest
solution).
– Exponential time and space complexity,
O(b^d) nodes will be generated, where
d is the depth of the solution and
b is the branching factor (i.e., number of children) at each node
Breadth-First Search (BFS)
 Example: Map Navigation
State Space:
S = start, G = goal, other nodes = intermediate states, links = legal transitions

A B C

S G

D E F
BFS Search TreeA B C

S G
S
D E F

Queue = {S}

Select S

Goal(S) = true?

If not, Expand(S)
 BFS Search TreeA B C

S G
S
D E F
A D

Queue = {A, D}

Select A

Goal(A) = true?

If not, Expand(A)
 BFS Search TreeA B C

S G
S
D E F
A D

B D Queue = {D, B, D}

Select D

Goal(D) = true?

If not, expand(D)
 BFS Search TreeA B C

S G
S
D E F
A D

B D A E Queue = {B, D, A, E}

Select B
etc.
 BFS Search TreeA B C

S G
S
D E F
A D

B D A E

C E S E S B B F

Level 3
Queue = {C, E, S, E, S, B, B, F}
 BFS Search TreeA B C

S G
S
D E F
A D

B D A E

C E S E S B B F

D F A B F D C E A C G

Level 4
Expand queue until G is at front
Select G
Goal(G) = true
Breadth-First Search
 Breadth-First Search (BFS) Properties
• Complete? Yes

• Optimal? Only if path-cost = non-decreasing function of

depth

• Time complexity O(bd)

• Space complexity O(bd)

• Main practical drawback? exponential space complexity

 Complexity of Breadth-First Search
• Time Complexity d=0
– assume (worst case) that there is 1 goal leaf
at the RHS at depth d d=1
– so BFS will generate
d=2
= b + b2+ ..... + bd + bd+1 - b G
= O (bd+1)

d=0
• Space Complexity
– how many nodes can be in the queue (worst-
d=1
case)?
– at depth d there are bd+1 unexpanded nodes in d=2
the Q = O (bd+1) G
Examples of Time and Memory Requirements for Breadth-First Search

Assuming b=10, 10000 nodes/sec, 1kbyte/node

Depth of Nodes
Solution Generated Time Memory

2 1100 0.11 seconds 1 MB

4 111,100 11 seconds 106 MB

8 109 31 hours 1 TB

12 1013 35 years 10 PB
 Depth-First (DFS)
• Algorithm outline:
– Always select from the OPEN the node with the greatest depth for
expansion, and put all newly generated nodes into OPEN
– OPEN is organized as LIFO (last-in, first-out) list
– Terminate if a node selected for expansion is a goal

• Properties:
May not terminate without a "depth bound," i.e., cutting off search
below a fixed depth D (How to determine the depth bound?)
Not complete (with or without cycle detection, and with or without a
cutoff depth)
Exponential time, O(b^d), but only linear space, O(bd), required
Can find deep solutions quickly if lucky
When search hits a deadend, can only back up one level at a time even if
the "problem" occurs because of a bad operator choice near the top of the
tree. Hence, only does "chronological backtracking"
 DFS Search TreeA B C

S G
S
D E F
A D

Queue = {A,D}
 DFS Search TreeA B C

S G
S
D E F
A D

B D
Queue = {B,D,D}
 DFS Search TreeA B C

S G
S
D E F
A D

B D
Queue = {C,E,D,D}

C E
 DFS Search TreeA B C

S G
S
D E F
A D

B D
Queue = {D,F,D,D}

C E

D F
 DFS Search TreeA B C

S G
S
D E F
A D

B D
Queue = {G,D,D}

C E

D F

G
 Depth-First Search (DFS) Properties
• Complete?
– Not complete if tree has unbounded depth
• Optimal?
– No
• Time complexity?
– Exponential
• Space complexity?
– Linear
 What is the Complexity of Depth-First Search?
• Time Complexity
– maximum tree depth = m d=0
– assume (worst case) that there is
1 goal leaf at the RHS at depth d d=1
– so DFS will generate O (bm) d=2
G

d=0

• Space Complexity d=1

– how many nodes can be in the
queue (worst-case)? d=2
– at depth m we have b nodes d=3
– and b-1 nodes at earlier depths
– total = b + (m-1)*(b-1) = O(bm) d=4
Examples of Time and Memory Requirements for Depth-First Search

Assuming b=10, m = 12, 10000 nodes/sec, 1kbyte/node

Depth of Nodes
Solution Generated Time Memory

2 1012 3 years 120kb

4 1012 3 years 120kb

8 1012 3 years 120kb

12 1012 3 years 120kb

 Comparing DFS and BFS
• Time complexity: same, but
– In the worst-case BFS is always better than DFS
– Sometime, on the average DFS is better if:
• many goals, no loops and no infinite paths
• BFS is much worse memory-wise
• DFS is linear space
• BFS may store the whole search space (Exponential)
• In general
• BFS is better if goal is not deep, if infinite paths, if many loops, if small search space
• DFS is better if many goals, not many loops,
• DFS is much better in terms of memory
• Selection strategy from OPEN
– BFS by FIFO – Queue
– DFS by LIFO - stack
• BFS always terminate if goal exist but DFS on locally finite infinite tree
• Guarantee shortest path to goal - BFS
 DFS with a depth-limit L
• Standard DFS, but tree is not explored below some depth-
limit L

• Solves problem of infinitely deep paths with no solutions

– But will be incomplete if solution is below depth-limit

• Depth-limit L can be selected based on problem knowledge

– E.g., diameter of state-space:
• E.g., max number of steps between 2 cities
– But typically not known ahead of time in practice
Depth-First Search with a depth-limit, L = 5
Depth-First Search with a depth-limit=5
Depth-First Search with a depth-limit=5
 Iterative Deepening Search (IDS)
• Run multiple DFS searches with increasing depth-limits
• BFS and DFS both have exponential time complexity O(b^d)

 BFS is complete but has exponential space complexity

 DFS has linear space complexity but is incomplete

• Iterative deepening search

L=1

 While no solution, do
 DFS from initial state S0 with cutoff L

 If found goal, stop and return solution,

 else, increment depth limit L

Iterative deepening search L=0
Iterative deepening search L=1
Iterative deepening search L=2
Iterative Deepening Search L=3
Iterative deepening search
 Properties of Iterative Deepening Search
• Space complexity = O(bd)
• (since its like depth first search run different times, with maximum depth
limit d)
• Time Complexity
• b + (b+b2) + .......(b+....bd) = O(bd)
(i.e., asymptotically the same as BFS or DFS to limited depth d in the
worst case)
• Complete?
• Yes

• Optimal
• Only if path cost is a non-decreasing function of depth
• IDS combines the small memory footprint of DFS, and has the
completeness guarantee of BFS
• In practice, IDS is the preferred uniform search method with a large
search space and unknown solution depth
 Bidirectional Search
• Idea
– simultaneously search forward from S and backwards from G
– stop when both “meet in the middle”
– need to keep track of the intersection of 2 open sets of nodes

• What does searching backwards from G mean

– need a way to specify the predecessors of G
• this can be difficult,
• e.g., predecessors of checkmate in chess?
– what if there are multiple goal states?
– what if there is only a goal test, no explicit list?

• Complexity
– time complexity at best is: O(2 b(d/2)) = O(b (d/2))
– memory complexity is the same
Bi-Directional Search
 Uniform Cost Search
• Optimality: path found = lowest cost
– Algorithms so far are only optimal under restricted
circumstances
• Let g(n) = cost from start state S to node n
• Uniform Cost Search:
– Always expand the node on the fringe with minimum cost
g(n)
– Note that if costs are equal (or almost equal) will behave
similarly to BFS
• A Generalized version of Breadth-First Search
– C(ni, nj) = cost of going from ni to nj
– Guarantee to find the minimum cost path
– Dijkstra Algorithm
 Uniform-Cost (UCS)
• Algorithm outline:
– Always select from the OPEN the node with the least g(.) value for expansion,
and put all newly generated nodes into OPEN
– Nodes in OPEN are sorted by their g(.) values (in ascending order)
– Terminate if a node selected for expansion is a goal
Uniform Cost Search
 Uniform-Cost (UCS)
• Complete (if cost of each action is not infinitesimal)
– The total # of nodes n with g(n) <= g(goal) in the state space is finite
– If n’ is a child of n, then g(n’) = g(n) + c(n, n’) > g(n)
– Goal node will eventually be generated (put in OPEN) and selected for
expansion (and passes the goal test)
• Optimal/Admissible
– Admissibility depends on the goal test being applied when a node is removed
from the OPEN list, not when it's parent node is expanded and the node is first
generated (delayed goal testing)
– Multiple solution paths (following different backpointers)
– Each solution path that can be generated from an open node n will have its path
cost >= g(n)
– When the first goal node is selected for expansion (and passes the goal test), its
path cost is less than or equal to g(n) of every OPEN node n (and solutions
entailed by n)
• Exponential time and space complexity, O(b^d) where d is the
depth of the solution path of the least cost solution
Comparison of Uninformed Search Algorithms
 When to use what
• Depth-First Search:
– Many solutions exist
– Know (or have a good estimate of) the depth of solution
• Breadth-First Search:
– Some solutions are known to be shallow
• Uniform-Cost Search:
– Actions have varying costs
– Least cost solution is the required
This is the only uninformed search that worries about costs.
• Iterative-Deepening Search:
– Space is limited and the shortest solution path is required
 Summary
• A review of search
– a search space consists of states and operators: it is a graph
– a search tree represents a particular exploration of search space

• There are various strategies for “uninformed search”

– breadth-first
– depth-first
– iterative deepening
– bidirectional search
– Uniform cost search

• Various trade-offs among these algorithms

– “best” algorithm will depend on the nature of the search problem