WIN SEM (2022-23) CSE3026 ETH AP2022236000316 Reference Material I 08-Feb-2023 4.informed (Heuristic) Search

Problem Solving by Searching
Search Methods :
informed (Heuristic) search
Using problem specific knowledge
to aid searching
 Without incorporating
Search
knowledge into searching, one
can have no bias (i.e. a everywhere!!
preference) on the search
space.
 Without a bias, one is forced

to look everywhere to find the
answer. Hence, the
complexity of uninformed
search is intractable.
2
Using problem specific knowledge
to aid searching
 With knowledge, one can search the state space as if he was
given “hints” when exploring a maze.
 Heuristic information in search = Hints
 Leads to dramatic speed up in efficiency.
B C D E
Search only
F G H I J
in this
subtree!! K L M N
O 3
More formally, why heuristic
functions work?
 In any search problem where there are at most b choices
at each node and a depth of d at the goal node, a naive
search algorithm would have to, in the worst case, search
around O(bd) nodes before finding a solution
(Exponential Time Complexity).
 Heuristics improve the efficiency of search algorithms

by reducing the effective branching factor from b to
(ideally) a low constant b* such that
 1 =< b* << b
4
Heuristic Functions
 A heuristic function is a function f(n) that gives an estimation on the “cost”
of getting from node n to the goal state – so that the node with the least
cost among all possible choices can be selected for expansion first.
 Three approaches to defining f:
 f measures the value of the current state (its “goodness”)
 f measures the estimated cost of getting to the goal from the current state:
 f(n) = h(n) where h(n) = an estimate of the cost to get from n
to a goal
 f measures the estimated cost of getting to the goal state from the current
state and the cost of the existing path to it. Often, in this case, we decompose
f:
 f(n) = g(n) + h(n) where g(n) = the cost to get to n (from initial
state)
5
Approach 1: f Measures the Value of
the Current State
 Usually the case when solving optimization problems

 Finding a state such that the value of the metric f is optimized
 Often, in these cases, f could be a weighted sum of a set of

component values:
 N-Queens
 Example: the number of queens under attack …
 Data mining
 Example: the “predictive-ness” (a.k.a. accuracy) of a rule discovered
6
Approach 2: f Measures the Cost to
the Goal
A state X would be better than a state Y if the

estimated cost of getting from X to the goal is
lower than that of Y – because X would be closer
to the goal than Y
7
Approach 3: f measures the total cost of the
solution path (Admissible Heuristic
Functions)
 A heuristic function f(n) = g(n) + h(n) is admissible if h(n) never
overestimates the cost to reach the goal.
 Admissible heuristics are “optimistic”: “the cost is not that much …”
 However, g(n) is the exact cost to reach node n from the initial state.
 Therefore, f(n) never over-estimate the true cost to reach the goal
state through node n.
 Theorem: A search is optimal if h(n) is admissible.
 I.e. The search using h(n) returns an optimal solution.
 Given h2(n) > h1(n) for all n, it’s always more efficient to use h2(n).
 h2 is more realistic than h1 (more informed), though both are optimistic.
8
Traditional informed search
strategies
 Greedy Best first search
 “Always chooses the successor node with the best f value”
where f(n) = h(n)
 We choose the one that is nearest to the final state among
all possible choices
 A* search
 Best first search using an “admissible” heuristic function f
that takes into account the current cost g
 Always returns the optimal solution path
9
Informed Search Strategies
Best First Search

An implementation of Best
First Search
function BEST-FIRST-SEARCH (problem, eval-fn)
returns a solution sequence, or failure
queuing-fn = a function that sorts nodes by eval-

fn
return GENERIC-SEARCH (problem,queuing-fn)
11
Greedy Search
eval-fn: f(n) = h(n)
Greedy Search
Start State Heuristic: h(n)
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
Goal f(n) = h (n) = straight-line distance heuristic
13
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
14
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
15
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
16
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
17
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
18
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
19
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
20
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
21
Greedy Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
22
Greedy Search: Tree Search
Start
A
23
Start
A 75
118
[329] 140 [374] B

C
[253] E
24
Start
A 75
118
[329] 140 [374] B

C
[253] E
80 99
[193] [178]
G F
[366] A
25
Start
A 75
118
[329] 140 [374] B

C
[253] E
80 99
[193] [178]
G F
[366] A
211
[253] E I [0]
Goal
26
Start
A 75
118
[329] 140 [374] B

C
[253] E
80 99
[193] [178]
G F
[366] A
211
[253] E I [0]
Goal
Path cost(A-E-F-I) = 253 + 178 + 0 = 431
27
dist(A-E-F-I) = 140 + 99 + 211 = 450
Greedy Search: Optimal ?
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
dist(A-E-G-H-I) =140+80+97+101=418 28
Greedy Search: Complete ?
A 75
118 A 366
140 B B 374
C
111 ** C 250
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
29
Start
A
30
Start
A 75
118
[250] 140 [374] B

C
[253] E
31
Start
A 75
118
[250] 140 [374] B

C
[253] E
111
[244] D
32
Start
A 75
118
[250] 140 [374] B

C
[253] E
111
[244] D
Infinite Branch !
[250] C
33
Start
A 75
118
[250] 140 [374] B

C
[253] E
111
[244] D
Infinite Branch !
[250] C
[244] D
34
Start
A 75
118
[250] 140 [374] B

C
[253] E
111
[244] D
Infinite Branch !
[250] C
[244] D
35
Greedy Search: Time and
Space Complexity ?
Start
A 75
118 • Greedy search is not optimal.
140 B
C
111
•Greedy search is incomplete
E
D 80 99
without systematic checking of
repeated states.
G F
97
•In the worst case, the Time
and Space Complexity of
H 211
Greedy Search are both O(bm)
101
I Where b is the branching factor and
Goal
m the maximum path length 36
A* Search
eval-fn: f(n)=g(n)+h(n)
A* (A Star)
 Greedy Search minimizes a heuristic h(n) which is an
estimated cost from a node n to the goal state. Greedy
Search is efficient but it is not optimal nor complete.
 Uniform Cost Search minimizes the cost g(n) from the

initial state to n. UCS is optimal and complete but not
efficient.
 New Strategy: Combine Greedy Search and UCS to get an

efficient algorithm which is complete and optimal.
38
A* (A Star)
 A* uses a heuristic function which
combines g(n) and h(n): f(n) = g(n) + h(n)
 g(n) is the exact cost to reach node n from

the initial state.
 h(n) is an estimation of the remaining cost

to reach the goal.
39
A* (A Star)
g(n)
f(n) = g(n)+h(n) n
h(n)
40
A* Search
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 98
101 I 0
I
Goal f(n) = g(n) + h (n)
g(n): is the exact cost to reach node n from the initial state. 41
A* Search: Tree Search
A Start
42
A Start
118 75
140
E [393] B [449]
[447] C
43
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
44
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
97
[415] H
45
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
97
[415] H
101
Goal I [418]
46
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
97
[415] H I [450]
101
Goal I [418]
47
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
97
[415] H I [450]
101
Goal I [418]
48
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
97
[415] H I [450]
101
Goal I [418]
49
A* with f() not Admissible
h() overestimates the cost to

reach the goal state
A* Search: h not admissible !
A 75
118 A 366
140 B B 374
C
111 C 329
E
D 80 99 D 244
E 253
G F
F 178
97 G 193
H 211 H 138
101 I 0
I
Goal f(n) = g(n) + h (n) – (H-I) Overestimated
g(n): is the exact cost to reach node n from the initial state. 51
A Start
52
A Start
118 75
140
E [393] B [449]
[447] C
53
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
54
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
97
[455] H
55
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[413] G F [417]
97
[455] H Goal I [450]
56
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[473] D [413] G F [417]
97
[455] H Goal I [450]
57
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[473] D [413] G F [417]
97
[455] H Goal I [450]
58
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[473] D [413] G F [417]
97
[455] H Goal I [450]
59
A Start
118 75
140
E [393] B [449]
[447] C
80 99
[473] D [413] G F [417]
97
[455] H Goal I [450]
A* not optimal !!!

60
A* Algorithm
A* with systematic checking for

repeated states …
A* Algorithm
1. Search queue Q is empty.
2. Place the start state s in Q with f value h(s).
3. If Q is empty, return failure.
4. Take node n from Q with lowest f value.
(Keep Q sorted by f values and pick the first element).
5. If n is a goal node, stop and return solution.
6. Generate successors of node n.
7. For each successor n’ of n do:
a) Compute f(n’) = g(n) + cost(n,n’) + h(n’).
b) If n’ is new (never generated before), add n’ to Q.
c) If node n’ is already in Q with a higher f value, replace it
with current f(n’) and place it in sorted order in Q.
End for
8. Go back to step 3.
62
A* Search: Analysis
Start
•A* is complete except if there is an
118
A 75 infinity of nodes with f < f(G).
140 B •A* is optimal if heuristic h is
C
111 admissible.
E
D 80 99 •Time complexity depends on the
G F
quality of heuristic but is still
exponential.
97
H
•For space complexity, A* keeps all
211
nodes in memory. A* has worst
101
case O(bd) space complexity, but an
I
Goal iterative deepening version is
possible (IDA*). 63
A* Search
 Advantages:
 A* search algorithm is the best algorithm than other search
algorithms.
 A* search algorithm is optimal and complete.
 This algorithm can solve very complex problems.
 Disadvantages:
 It does not always produce the shortest path as it mostly
based on heuristics and approximation.
 A* search algorithm has some complexity issues.
 The main drawback of A* is memory requirement as it
keeps all generated nodes in the memory, so it is not
practical for various large-scale problems. 64
Iterative Deepening A*
Iterative Deepening A*:IDA*
 Use f(N) = g(N) + h(N) with admissible

and consistent h
 Each iteration is depth-first with cutoff

on the value of f of expanded nodes
66
IDA* Algorithm
 In the first iteration, we determine a “f-cost limit” – cut-off value
f(n0) = g(n0) + h(n0) = h(n0), where n0 is the start node.
 We expand nodes using the depth-first algorithm and backtrack

whenever f(n) for an expanded node n exceeds the cut-off value.
 If this search does not succeed, determine the lowest f-value

among the nodes that were visited but not expanded.
 Use this f-value as the new limit value – cut-off value and do
another depth-first search.
 Repeat this procedure until a goal node is found.

67
f(N) = g(N) + h(N)
8-Puzzle with h(N) = number of misplaced tiles
4
Cutoff=4
68
f(N) = g(N) + h(N)
4
Cutoff=4 4
6 6
69
f(N) = g(N) + h(N)
4
Cutoff=4 4 5
6 6
70
f(N) = g(N) + h(N)
4
Cutoff=4 4 5
6 6
71
f(N) = g(N) + h(N)
6 5
4
Cutoff=4 4 5
6 6
72
f(N) = g(N) + h(N)
4
Cutoff=5
73
f(N) = g(N) + h(N)
4
Cutoff=5 4
6 6
74
f(N) = g(N) + h(N)
4
Cutoff=5 4 5
6 6
75
f(N) = g(N) + h(N)
4
Cutoff=5 4 5
6 6
76
f(N) = g(N) + h(N)
4 5
Cutoff=5 4 5
6 6
77
f(N) = g(N) + h(N)
4 5 5
Cutoff=5 4 5
6 6
78
f(N) = g(N) + h(N)
4 5 5
Cutoff=5 4 5
6 6
79
Conclusions
 Frustration with uninformed search led to the idea
of using domain specific knowledge in a search so
that one can intelligently explore only the relevant
part of the search space that has a good chance of
containing the goal state. These new techniques
are called informed (heuristic) search strategies.
 Even though heuristics improve the performance

of informed search algorithms, they are still time
consuming especially for large size instances.
80

WIN SEM (2022-23) CSE3026 ETH AP2022236000316 Reference Material I 08-Feb-2023 4.informed (Heuristic) Search

Uploaded by

Copyright:

Available Formats

WIN SEM (2022-23) CSE3026 ETH AP2022236000316 Reference Material I 08-Feb-2023 4.informed (Heuristic) Search

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

WIN SEM (2022-23) CSE3026 ETH AP2022236000316 Reference Material I 08-Feb-2023 4.informed (Heuristic) Search

Uploaded by

Copyright:

Available Formats

Problem Solving by Searching

 Without a bias, one is forced

 Heuristics improve the efficiency of search algorithms

 Three approaches to defining f:

 f measures the value of the current state (its “goodness”)

 Usually the case when solving optimization problems

 Often, in these cases, f could be a weighted sum of a set of

A state X would be better than a state Y if the

Best First Search

queuing-fn = a function that sorts nodes by eval-

return GENERIC-SEARCH (problem,queuing-fn)

[329] 140 [374] B

[329] 140 [374] B

[329] 140 [374] B

[329] 140 [374] B

[250] 140 [374] B

[250] 140 [374] B

[250] 140 [374] B

[250] 140 [374] B

[250] 140 [374] B

 Uniform Cost Search minimizes the cost g(n) from the

 New Strategy: Combine Greedy Search and UCS to get an

 g(n) is the exact cost to reach node n from

 h(n) is an estimation of the remaining cost

h() overestimates the cost to

[473] D [413] G F [417]

[473] D [413] G F [417]

[473] D [413] G F [417]

[473] D [413] G F [417]

A* not optimal !!!

A* with systematic checking for

 Use f(N) = g(N) + h(N) with admissible

 Each iteration is depth-first with cutoff

f(n0) = g(n0) + h(n0) = h(n0), where n0 is the start node.

 We expand nodes using the depth-first algorithm and backtrack

 If this search does not succeed, determine the lowest f-value

 Repeat this procedure until a goal node is found.

 Even though heuristics improve the performance

You might also like