Beyond Classical Search

Syed Rizwan
 Agenda
o Local Search
o Hill Climbing
o Simulated Annealing
o Genetic Algorithms
o Beam Search
o Searching With Partial Information
 What we have
seen so far?
o Our search agent was search systematically:
o It keeps more than one paths (if available) in memory
o It keeps of all the nodes that have been discovered, ex-
plored or which haven’t been discovered
o Solve problems where the solution is the path to the goal
 Local Search
Algorithms
o There are optimization problems where the path is
irrelevant, what we are after is the goal itself, for
example:
o Sudoku
o Crosswords
o N-Queen Problem
o Travelling salesman problem
o Timetable scheduling
o etc
 Local Search
Algorithms
o While the traditional search agents use incremental
formulations, local searches use set of “complete”
configurations
o We have to find configurations that satisfy some
constraints, e.g., Sudoku
o In such cases, we can make use of local search al-
gorithms
o We start by having a random solution to the prob-
lem, “current state” it may not be correct, and
then improve it incrementally.
 Local Search
Algorithms
o Think of a local search agent as an agent climbing a
hill where there is lot of fog and the agent has am-
nesia
o It uses very little memory, because they only keep
into memory the current state
o They find reasonable solutions (not always optimal)
in potentially exponentially large spaces, where tra-
ditional approaches are not suitable
o They are often used to solve optimization problems,
e.g. Travelling Salesman Problem
 Example: n-Queen Problem
o Put n queens on an n × n board with no two queens
on the same row, column, or diagonal
 State Space Landscape
Objective Function

Current State
 Hill-Climbing Search
"Like climbing Everest in thick fog with amnesia"
 Hill-Climbing Search
o Objective function?
o Number of attacking queens
 Hill Climbing
Properties
o It is also called greedy local search
o Complete?
o Optimal?
o It can get stuck when:
o At local maxima/minima
o Ridges
o Plateaus
 Hill Climbing
Properties
o The steepest-ascent hill climbing solves only 14% of
the randomly generated 8-queen problems with an
avg. of 4 steps
o Allowing sideways move raises the success rate to
94% with an avg. of 21 steps, and 64 steps for each
failure
 Variants of Hill Climbing
o Stochastic hill climbing:
o chooses at random from among uphill moves
o converges more slowly, but finds better solutions in some land-
scapes
o First-choice hill climbing:
o generate successors randomly until one is better than the current
o good when a state has many successors
o Random-restart hill climbing:
o conducts a series of hill climbing searches from randomly gener-
ates initial states, stops when a goal is found
o It’s complete with probability approaching 1
 More on Random-Restart
Hill Climbing
o Assume each hill climbing search has a probability p of suc-
cess, then the expected number of restarts required is 1/p
o For 8-queen problem, p = 14%, so we need roughly 7 itera-
tions to find a goal
o Expected # of steps = cost_to_success + (1-p)/p *
cost_to_failure
o Random-restart hill climbing is very effective for n-queen
problem
o 3 million queens can be solved < 1 min
 Some Thoughts
o NP-hard problems typically have an exponential
number of local maxima/minima to get stuck on
o A hill climbing algorithm that never makes “down-
hill” (or “uphill”) moves is guaranteed to be incom-
plete
o A purely random walk moving to a successor chosen
uniformly at random is complete, but extremely in-
efficient
o What should we do?
o Simulated annealing (P115)
 Intuition
o Suppose we want to go to a Goal, while using Eu-
clidean distance between the current position and
the goal position as the heuristic
Euclidean Distance

Agent
Goal
 Intuition
o Suppose we want to go to a Goal, while using Eu-
clidean distance between the current position and
the goal position as the heuristic

Euclidean Distance

Agent
Goal
 Intuition
o Suppose we want to go to a Goal, while using Eu-
clidean distance between the current position and
the goal position as the heuristic
W
a
l
l

Agent
Goal
 Intuition
o Suppose we want to go to a Goal, while using Eu-
clidean distance between the current position and
the goal position as the heuristic
W
a
l
l

Goal
Agent
 What is Simulated
Annealing?
o The process used to harden metals and glass by
heating them to a high temperature and then grad-
ually cooling them, thus allowing the material to
reach a low-energy crystalline state
Ping-Pong Ball Example
 Simulated Annealing
o Main idea: escape the local minima/maxima by selecting a bad move
with some “probability”, this probability will decrease as we move
towards a more stable state
 Analysis of
Simulated Annealing
o One can prove: If T decreases slowly enough, then
simulated annealing search will find a global opti-
mum with probability approaching 1
o Widely used in VLSI layout, airline scheduling, etc.
 Local Beam Search
o Idea:
o Keep track of k states rather than just one
o Start with k randomly generated states
o At each iteration, all the successors of all k states are generated
o If any one is a goal state, stop; else select the k best successors from the com-
plete list and repeat
o Is it the same as running k random-restart searches?
o Useful information is passed among the k parallel search threads
o Stochastic beam search: similar to natural selection, offspring of a
organism populate the next generation according to its fitness
 Genetic Algorithms
o A successor state is generated by combining two parent
states
o Start with k randomly generated states (population)
o A state is represented as a string over a finite alphabet
(often a string of 0s and 1s or digits)
o Evaluation function (fitness function). Higher values for
better states
o Produce the next generation of states by selection,
crossover, and mutation
Genetic Algorithms
Example: 8-Queen
 More on Genetic Algorithms
o Genetic algorithms combine an uphill tendency with
random exploration and exchange of information
among parallel search threads
o Advantages come from “crossover”, which raise the
level of granularity
A Genetic Algorithm
 Searching With
Partial Information
o We have covered: Deterministic, fully observable  single-state
problem
o agent knows exactly which state it will be in
o solution is a sequence
o • Deterministic, non-observable  multi-state problem
o Also called sensorless problems (conformant problems)
o agent may have no idea where it is
o solution is a sequence
o • Nondeterministic and/or partially observable  contingency
problem
o percepts provide new information about current state
o often interleave search, execution
o solution is a tree or policy
o • Unknown state space  exploration problem (“online”)
o states and actions of the environment are unknown
 Example: Vacuum World
o • Single-state, start in #5.
Solution?
– [Right, Suck]

o • Multi-state, start in #[1, 2, …, 8].

Solution?
– [Right, Suck, Left, Suck]
 Contingency Problem
o • Contingency, start in #5 &
7.
o Nondeterministic: suck may
dirty a clean carpet
o local sensing: dirt, location only
at current location
o Solution?
o Percept:
[Left, Clean]  [Right, if dirty
then Suck]
 Online search problems
o Agent knowledge:
o ACTION(s): list of allowed actions in state s
o C(s,a,s’): step-cost function (! After s’ is determined)
o GOAL-TEST(s)
o An agent can recognize previous states.
o Actions are deterministic.
o Access to admissible heuristic h(s)
e.g. manhattan distance
 Online search problems
o Objective: reach goal with minimal cost
o Cost = total cost of travelled path
o Competitive ratio=comparison of cost with cost of the so-
lution path if search space is known.
o Can be infinite in case of the agent
accidentally reaches dead ends
 The adversary argument

o Assume an adversary who can construct the

state space while the agent explores it
o Visited states S and A. What next?
o Fails in one of the state spaces

o No algorithm can avoid dead ends in all state

spaces.
 Online search agents
o The agent maintains a map of the environment.
o Updated based on percept input.
o This map is used to decide next action.

Note difference with e.g. A*

An online version can only expand the node it is physically in (local
order)
 Online DF-search
function ONLINE_DFS-AGENT(s’) return an action
input: s’, a percept identifying current state
static: result, a table indexed by action and state, initially empty
unexplored, a table that lists for each visited state, the action not yet tried
unbacktracked, a table that lists for each visited state, the backtrack not yet
tried
s,a, the previous state and action, initially null

if GOAL-TEST(s’) then return stop

if s’ is a new state then unexplored[s’]  ACTIONS(s’)
if s is not null then do
result[a,s]  s’
add s to the front of unbackedtracked[s’]
if unexplored[s’] is empty then
if unbacktracked[s’] is empty then return stop
else a  an action b such that result[b, s’]=POP(unbacktracked[s’])
else a  POP(unexplored[s’])
s  s’
return a
 Online DF-search, example
o Assume maze problem on 3x3
grid.
o s’ = (1,1) is initial state
o Result, unexplored (UX),
unbacktracked (UB), …
are empty
o S,a are also empty
 Online DF-search, example
o GOAL-TEST((,1,1))?
o S not = G thus false
o (1,1) a new state?
o True
o ACTION((1,1)) -> UX[(1,1)]
o {RIGHT,UP}

o s is null?
o True (initially)
o UX[(1,1)] empty?
o False
o POP(UX[(1,1)])->a
o A=UP
o s = (1,1)
o Return a
 Online DF-search, example
o GOAL-TEST((2,1))?
o S not = G thus false
o (2,1) a new state?
o True
o ACTION((2,1)) -> UX[(2,1)]
o {DOWN}

o s is null?
o false (s=(1,1)) S
o result[UP,(1,1)] <- (2,1)
o UB[(2,1)]={(1,1)}
o UX[(2,1)] empty?
o False
o A=DOWN, s=(2,1) return A
 Online DF-search, example
o GOAL-TEST((1,1))?
o S not = G thus false
o (1,1) a new state?
o false
o s is null?
o false (s=(2,1))
o result[DOWN,(2,1)] <- (1,1)
o UB[(1,1)]={(2,1)}
o UX[(1,1)] empty?
o False
o A=RIGHT, s=(1,1) return
A
 Online DF-search,
example
o GOAL-TEST((1,2))?
o S not = G thus false
o (1,2) a new state?
o True,
UX[(1,2)]={RIGHT,UP,LEFT}
o s is null?
o false (s=(1,1))
o result[RIGHT,(1,1)] <- (1,2)
o UB[(1,2)]={(1,1)}
o UX[(1,2)] empty?
o False S
o A=LEFT, s=(1,2) return
A
 Online DF-search,
example
o GOAL-TEST((1,1))?
o S not = G thus false
o (1,1) a new state?
o false
o s is null?
o false (s=(1,2))
o result[LEFT,(1,2)] <- (1,1)
o UB[(1,1)]={(1,2),(2,1)}
o UX[(1,1)] empty?
o True
o UB[(1,1)] empty? False
o A= b for b in result[b,
(1,1)]=(1,2)
o B=RIGHT
o A=RIGHT, s=(1,1) …
 Online DF-search
o Worst case each node is
visited twice.
o An agent can go on a long
walk even when it is close
to the solution.
o An online iterative
deepening approach solves
this problem.
o Online DF-search works
only when actions are
reversible.
 Online local search
o Hill-climbing is already online
o One state is stored.
o Bad performance due to local maxima
o Random restarts impossible.
o Solution: Random walk introduces exploration (can produce exponentially many
steps)
 Online local search
o Solution 2: Add memory to hill climber
o Store current best estimate H(s) of cost to reach goal
o H(s) is initially the heuristic estimate h(s)
o Afterward updated with experience (see below)
o Learning real-time A* (LRTA*)
 Credits
o Artificial Intelligence: a modern approach, 3rd edi-
tion \
By Stuart Russell. And Peter Norvig

http://aima.cs.berkeley.edu/