CS414-Articial Intelligence

Lecture 5: Basic Search Algorithms

Waheed Noor
Computer Science and Information Technology,
University of Balochistan,
Quetta, Pakistan
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 1 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 2 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 3 / 48
Search Algorithms
Search plays important role in solving AI problems.
Normally, the result of a search algorithm is a solution in the form
of a set of actions that lead to the goal. (But not always)
For any problem, once the problem is properly formulated and
initial state is identied, search algorithm is applied to identify best
possible sequences of state-action to achieve the goal.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 4 / 48
Searching & Optimization
In almost every problem:
Initial situation and goal is given.
At any state in the state space, we are faced with the set of actions
Executing a particular sequence of such actions may or may not
achieve the goal.
Search is the process of identifying several such sequences and
choosing one that achieves the goal.
For some applications, each sequence of actions may be
associated with a certain cost.
A search problem that is not only to reach to a goal but also
achieving the minimal cost is an optimization problem.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 5 / 48
An Example
(a) (b)
Figure : Source:http://www.ics.forth.gr/cvrl/
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 6 / 48
Searching the State Space
The set of all possible sequences of legal moves form a tree:
The nodes of the tree are labeled with states (the same state
could label many different nodes).
The initial state is the root of the tree.
For each of the legal follow-up moves of a given state, any node
labeled with that state will have a child labeled with the follow-up
state.
Each branch corresponds to a sequence of states (and thereby
also a sequence of moves).
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 7 / 48
State Vs. Node
A state is a (representation of) a physical conguration
A node is a data structure constituting part of a search tree
includes parent, children, depth, path cost g(x)
States do not have parents, children, depth, or path cost!
The EXPAND function creates new nodes, lling in the various elds
and using the SUCCESSORFN of the problem to create the
corresponding states.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 8 / 48
Basic Search Algorithm: In General
Basic idea:
ofine, simulated exploration of state space by generating successors
of already-explored states (i.e.,expanding states)
Algorithm 1: Basic Tree Search Algorithm
Input: problem, strategy
Output: solution or failure
initialize the search tree using the initial state of problem
while Expansion is possible do
choose a leaf node for expansion according to strategy
if the node contains a goal state then
Return: the corresponding solution
else
expand the node and add the resulting nodes to the search tree
end
end
Return failure
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 9 / 48
Search Tree Example
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 10 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 11 / 48
Infrastructure for Search Algorithm
For each node n of the tree, we have a structure that contains ve
components:
n.STATE: the state in the state space to which the node
corresponds.
n.PARENT: the node in the search tree that generated this node.
n.ACTION: the action that was applied to the parent to generate the
node.
n.DEPTH: the depth of tho node n, i.e., number of nodes in the path
from the rood to this node n.
n.PATH-COST: the cost, traditionally denoted by g(n), of the path from
the initial state to the node, as indicated by the parent
pointers.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 12 / 48
Infrastructure for Search Algorithm
General search tree algorithm, shown above, also needs to know the
list of unexpanded nodes and among them which node to choose for
expansion.
Node selection for expansion depends on the search strategy that
we will see shortly.
The list of available nodes for expansion (unexpanded yet) is
maintained in the form a collection called fringe or frontier.
Generally, this collection of nodes is implemented as a queue.
Since, sometime it may be necessary for search strategy to check
each node in the list for possible expansion.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 13 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 14 / 48
Search Strategy
Denition
A strategy is dened by picking the order of node expansion
Strategies are evaluated based on:
completenessdoes it always nd a solution if one exists?
time complexitynumber of nodes generated/expanded
space complexitymaximum number of nodes in memory
optimalitydoes it always nd a least-cost solution?
Time and space complexity are measured in terms of
bmaximum branching factor of the search tree
ddepth of the least-cost solution
mmaximum depth of the state space (may be )
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 15 / 48
Types of Search Strategy
Denition (Uninformed Search Algorithms)
Algorithms that are given no information about the problem other than
its denition. Although some of these algorithms can solve any
solvable problem, none of them can do so efciently.
Denition (Informed Search Algorithms)
Informed search algorithms, on the other hand, can do quite well given
some guidance on where to look for solutions. That is, it may have
access to some function that estimate the cost of the solution.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 16 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 17 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 18 / 48
Breadth-First Search
Denition
The shallowest unexpanded node is chosen for expansion, i.e., root
node is expanded rst, then all the successors of the root node are
expanded next, then their successors, and so on.
They are implemented using FIFO data structures, the algorithm will
maintain a list of the currently active paths and in each round of the
algorithm repeat following three steps until solution is not found or no
further expansion is possible.
1
Remove the rst path from the list of paths.
2
Generate a new path for every possible follow-up nodes.
3
Append the list of newly generated paths to the end of the list of
paths (to ensure paths are really being visited in breadth-rst
order).
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 19 / 48
Breadth-First Search: Example
(a) (b)
(c) (d)
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 20 / 48
Breadth-First Search: Rating
Complete Yes (if b is nite)
Time 1 +b +b
2
+b
3
+ . . . +b
d
+b(b
d
1) = O(b
d+1
), i.e.,
exponential in d
Space O(b
d+1
) (keeps every node in memory)
Optimal Yes (if cost = 1 per step); not optimal in general
Space is the big problem; can easily generate nodes at 100MB/sec
so 24hrs = 8640GB.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 21 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 22 / 48
Uniform-cost search
Denition
Unlike BFS, instead of expanding shallowest node, uniform-cost
Search (UCS) expands the node with lowest path cost.
Explanation
UCS nds the least-cost path through a graph by maintaining an
ordered list of nodes in order of least-greatest cost(Priority Queue).
This allows us to evaluate the least cost path rst.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 23 / 48
How UCS Works
The algorithm uses the accumulated path cost and a priority
queue to determine the path to evaluate.
The priority queue (least cost) contains the nodes to be evaluated.
As node children are evaluated, we add their cost to the node with
the aggregate sum of the current path.
This node is then added to the queue, and when all children have
been evaluated, the queue is sorted in order of ascending cost.
When the rst element in the priority queue is the goal node, then
the best solution has been found.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 24 / 48
Example
Figure : Graph of an example problem with respective tree.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 25 / 48
Solution
Step-by-Step Execution of UCS
Step0: { [ S , 0 ] }
Step1: { [ S->A , 1 ] , [ S->G , 12 ] }
Step2: { [ S->A->C , 2 ] , [ S->A->B , 4 ] , [ S->G , 12] }
Step3: { [ S->A->C->D , 3 ] , [ S->A->B , 4 ] ,
[ S->A->C->G , 4 ] , [ S->G , 12 ] }
Step4: { [ S->A->B , 4 ] , [ S->A->C->G , 4 ] ,
[ S->A->C->D->G , 6 ] , [ S->G , 12 ] }
Step5: { [ S->A->C->G , 4 ] , [ S->A->C->D->G , 6 ] ,
[ S->A->B->D , 7 ] , [ S->G , 12 ] }
Step6 Output as S->A->C->G.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 26 / 48
UCS: Rating
Complete: Yes (if b is nite)
Time: O(b
d
)
Space: O(b
d
)
Optimal: Yes, if cost on the edges are non-negative.
Good things about UCS
Can set to prune (remove) duplicate entries in the queue.
Can set to search exhaustively for optimal solution, if possible,
among different lest-cost candidate paths to the goal.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 27 / 48
Class Activity
Class Activity
Draw expected graph based on BFS and UCS algorithms and output
possible resulting optimal path.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 28 / 48
Quiz
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 29 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 30 / 48
Depth-First Search
Denition (DFS)
A graph searching algorithm that begins at the root node, and
exhaustively searches each branch to its greatest depth before
backtracking to previously unexplored branches.
Explanation
Nodes found but yet to be reviewed are stored in a LIFO queue
(stack).
Exhaustive means DFS can search every node in the graph to nd
the goal. If the goal is not present in the graph, the algorithm will
terminate, but will search each and every node in a systematic way.
Therefore, the DFS need to store only single path from node to leaf
along with unexpanded sibling at each level of depth. Each node
explored is put on the front of frontier.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 31 / 48
Progress of DFS
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
A
B C
D E F G
H I J K L M N O
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 32 / 48
Variants of DFS
Graph Version Vs. Tree Version
1 Graph version avoids redun-
dant paths and it is complete
in nite state spaces, since it
will eventually expand every
node.
Can not avoid loops, however
modication can be made to
avoid loops. General, both
are incomplete when depth
spaces are innite.
2 Non-Optimal Non-Optimal
3 Time complexity is bounded
by the size of the state space:
Not good if state space is in-
nite
O(b
m
), where m is the maxi-
mum depth of any node: Ter-
rible.
4 Space complexity is not better
than BFS
Linear O(bm), since nodes
from root to leaf along the,
and unexplored siblings are
stored at a time
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 33 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 34 / 48
Depth-limited Search
Denition
Depth-limited search (DLS) is the variant of DFS with pre-specied
depth limit. That is if depth limit is denoted by then no successor are
added further to the node at the depth . The DLS overcomes the
failure of DFS in innite space and solve the innite path problem.
If is not properly chosen in the case when d is unknown, < d,
then DLS is incomplete that is if the goal is beyond the depth limit.
The depth limit also reduces the scope of the search.
It is also suboptimal, since the algorithm may nd rst path to the
goal instead of shortest path.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 35 / 48
DLS Continue
The depth limit is often based on the domain knowledge of the
problem.
Example
In the map of Romania, we can see that there are 20 cities, therefore
possibly = 19 since if solution exists that would reach at the depth of
19. Or if we look at the map carefully every city from any city can be
reach in 9 steps that would be more precise depth limit. In such case it
is called diameter of the state space.
DLS Rating
Complete: No (Generally, if < d)
Time Complexity: O(b

)
Space Complexity: O(b)
Optimal: No.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 36 / 48
An Example
Figure : DLS example with different depth limit
Home Work
Given the map of Romania, what will be the minimum depth limit
and generate graphs using DLS based on your reported for:
From Arad to Busharest.
From Iasi to Zerind.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 37 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 38 / 48
Iterative Deepening Search
Reading
Read it by your self and solve the above problem for IDS as well.
You are also supposed to read Bidirectional Search.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 39 / 48
Outline
1
Basic search algorithms
2
Infrastructure for Search Algorithms
3
Search Strategy
4
Uninformed Search Algorithms
Breadth-First Search
Uniform-cost search
Depth-rst search
Depth-limited Search
Iterative Deepening Search
5
Informed Search
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 40 / 48
Informed Search
What we have seen so far are all those search algorithms that
operate with out any domain knowledge, and try to solve the
problem in brute-force fashion.
Such methods are insufcient for most problems specially when
the problem is large and complex.
Now we will look into another type of search techniques called
informed search.
Denition
Informed Search Incorporates a heuristic in the search that determines
the quality of any state in the search space. In a graph search, this
results in a strategy for node expansion. The heuristic may consider
the problem knowledge to guide the search strategy.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 41 / 48
Best First Search I
In Best-FS state space is evaluated on the basis of an evaluation
function f (n) that incorporates a heuristic function h(n) and
estimated cost g(n) from start node to n, i.e., f (n) = h(n).
This search strategy is also called A* Search.
The h(n) can be an educated guess such that the cheapest path
from the state at node n to the goal.
In Romania example, this may be a straight line between state at
node n to goal.
The g(n) is the cost from the start node to node n.
The algorithm is often implemented by maintaining two list, an
open one and a closed one.
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 42 / 48
Best First Search II
Open list is a priority queue consist nodes yet to be visited sorted
by their evaluation function.
Closed list contains nodes that have already been evaluated.
Best-FS specializes to BFS when f (n) = h(n) and to UFS when
f (n) = g(n).
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 43 / 48
An Example
Consider again the Romania example
Bucharest
Giurgiu
Urziceni
Hirsova
Eforie
Neamt
Oradea
Zerind
Arad
Timisoara
Lugoj
Mehadia
Dobreta
Craiova
Sibiu
Fagaras
Pitesti
Rimnicu Vilcea
Vaslui
Iasi
Straightline distance
to Bucharest
0
160
242
161
77
151
241
366
193
178
253
329
80
199
244
380
226
234
374
98
Giurgiu
Urziceni
Hirsova
Eforie
Neamt
Oradea
Zerind
Arad
Timisoara
Lugoj
Mehadia
Dobreta
Craiova
Sibiu
Fagaras
Pitesti
Vaslui
Iasi
Rimnicu Vilcea
Bucharest
71
75
118
111
70
75
120
151
140
99
80
97
101
211
138
146 85
90
98
142
92
87
86
Figure : Romania map with SLD
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 44 / 48
Greedy Fashion
Rimnicu Vilcea
Zerind
Arad
Sibiu
Arad Fagaras Oradea
Timisoara
Sibiu Bucharest
329 374
366 380 193
253 0
Rimnicu Vilcea
Zerind
Arad
Sibiu
Arad Fagaras Oradea
Timisoara
329 374
366 176 380 193
Zerind
Arad
Sibiu Timisoara
253 329 374
Arad
366
(a) The initial state
(b) After expanding Arad
(c) After expanding Sibiu
(d) After expanding Fagaras
Figure : In the greedy fashion
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 45 / 48
A* Search Example
(a) The initial state
(b) After expanding Arad
(c) After expanding Sibiu
Zerind
Arad
Sibiu Timisoara
447=118+329 449=75+374 393=140+253
Arad
366=0+366
(d) After expanding Rimnicu Vilcea
(e) After expanding Fagaras
(f) After expanding Pitesti
Zerind
Arad
Sibiu
Arad
Timisoara
Rimnicu Vilcea Fagaras Oradea
447=118+329 449=75+374
646=280+366 413=220+193 415=239+176 671=291+380
Zerind
Arad
Sibiu
Arad
Timisoara
Fagaras Oradea
447=118+329 449=75+374
646=280+366 415=239+176
Rimnicu Vilcea
Craiova Pitesti Sibiu
526=366+160 553=300+253 417=317+100
671=291+380
Zerind
Arad
Sibiu
Arad
Timisoara
Sibiu Bucharest
Rimnicu Vilcea Fagaras Oradea
Craiova Pitesti Sibiu
447=118+329 449=75+374
646=280+366
591=338+253 450=450+0 526=366+160 553=300+253 417=317+100
671=291+380
Zerind
Arad
Sibiu
Arad
Timisoara
Sibiu Bucharest
Rimnicu Vilcea Fagaras Oradea
Craiova Pitesti Sibiu
Bucharest Craiova Rimnicu Vilcea
418=418+0
447=118+329 449=75+374
646=280+366
591=338+253 450=450+0 526=366+160 553=300+253
615=455+160 607=414+193
671=291+380
Figure : The A* implementation
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 46 / 48
A* Search
A* Search Rating
Complete: Yes
Time Complexity: O(b
d
)
Space Complexity: O(b
d
)
Optimal: Yes.
A* search algorithm have two avors, the graph based and tree based.
Conditions for Optimality: Admissibility and Consistency
Admissibility: That the heuristic is admissible i.e., that h(n) will never
overestimate the cost to reach the goal. For A*, h(n) is admissible,
since straight line is the shortest path between any two points.
Consistent: Consistency (also called monotonicity) describes that the
cost along the path is increasing or decreasing. For A* graph version,
the g(n) is the increasing function of n along the path..
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 47 / 48
References I
Waheed Noor (CS&IT, UoB, Quetta) CS414-Articial Intelligence September-October 2014 48 / 48