Chapter 4
Chapter 4
Chapter 4
(Adapted from Stuart Russel, Dan Klein, and others. Thanks guys!)
1
Outline
• Hill-climbing
• Simulated annealing
• Genetic algorithms (briefly)
• Local search in continuous spaces (very briefly)
2
Motivation: Types of problems
Local Search Algorithms
shoulder
local maximum
"flat" local maximum
state space
current
state
Pure hill climbing search on 8-queens: gets stuck 86% of time! 14%
success
Overall Plan for all variants: Build in ways to allow *some* non-optimal moves
à get out of local maximum and onward to global maximum
Idea:
ü escape local maxima by allowing some random “bad” moves
ü but gradually decrease the degree and frequency
ü à jiggle hard at beginning, then less and less to find global maxima
11
Local beam search
12
Genetic algorithms
Steps:
1. Rank current population (of states) by fitness function
2. Select states to cross. Random plus weighted by fitness (more fit=more likely)
3. Randomly select “crossover point”
4. Swap out whole parts of states to generate “offspring” 13
5. Throw in mutation step (randomness!)
Genetic Algorithm: N-Queens example
Genetic algorithms: analysis
+ = !!!
From researchGate.net
Katieluethgeospatial.blogspot.com
Searching Continuous spaces
Approaches:
Discretization methods turn continuous space into discrete space
• e.g., empirical gradient search considers ±δ change in each coordinate
• If you make δ small enough, you get needed accuracy
∂f ∂f ∂f ∂f ∂f ∂f
⎛ ⎞
∇f =
⎜ ⎟
⎜
, , , , , ⎟
∂x 1
⎝
∂y 1 ∂x 2 ∂y 2 ∂x 3 ∂y 3 ⎠
• Start state = 1
• One solution:
1. Suck,
2. if(state=5) then
[right,suck] ]
• Plan:
– Build a belief state space based on the real state space
– Search that state space using the usual search techniques!
Start!
• Online search
– Idea: Search as you go. Interleave search + action
– Pro: actual percepts prune huge subtrees of search space @ each move
– Con: plan ahead less à don’t foresee problems
• Best case = wasted effort. Reverse actions and re-plan
• Worst case: not reversible actions. Stuck!
• LRTA* à many variants; vary in selecting next action and updating rules
Chapter 4: Summary
• Classic search must be adapted and modified for the real world
– Hill-climbing: can be seen as DFS + h(n) ... with depth limit of one.
– Beam search: can be seen as Best First...with Frontier queue limit = k.
– Stochastic techniques (incl. simulated annealing) = seen as Best-first with
weighted randomized Q selection.
– Belief State Search = identical to normal search...only searching belief space
– Online Search: Applied DFS or local searching
• With high cost of backtracking and becoming stuck
• Pruning by moving before complete plans made.