Solving Problems by

Searching

1
 Outline
• Problem-solving agents
• Problem formulation
• Basic search algorithms
• Heuristic search
• Beyond classical search

2
 Problem-solving agents
• Problem-solving agents use atomic
representation
– States of the world are considered as wholes, with
no internal structure visible to the problem-solving
algorithms

B C

B C
(a) Atomic (b) Factored (c) Structured

3
 Search problems in real world

Route Planning Robot Navigation

4
 Toy problems

2 8 3 1 2 3
1 6 4 8 4
7 5 7 6 5

8-puzzle 8-queens

5
 Problem formulation
• All these problems have a common structure:
– An initial situation and a certain goal.
– Different simple actions available to us (e.g. “turn left” vs.
“turn right”). Executing a particular sequence of such actions
may or may not achieve the goal.
– Search is the process of inspecting several such sequences and
choosing one that achieves the goal.
– For some applications, each sequence of actions may be
associated with a certain cost.

6
 Problem formulation
• States • Transition model
• Initial state • Goal test
• Actions • Path cost

2 8 3 1 2 3
1 6 4 8 4
7 5 7 6 5

7
 Searching the state space
• All possible sequences of legal moves form a tree

– The nodes of the tree are labelled with states

– The initial state is the root of the tree.
– For each of the legal follow-up moves of a
given state, any node labelled with that state
will have a child labelled with the follow-up
state.
– Each branch corresponds to a sequence of
states (and thereby also a sequence of moves)

8
 Single-state problem formulation
A problem is defined by four items:
1. initial state e.g., "at Arad“
2. actions or successor function S(x) = set of action–state pairs
– e.g., S(Arad) = {<Arad → Zerind, Zerind>, … }
–

3. goal test, can be

– explicit, e.g., x = "at Bucharest"
– implicit, e.g., Checkmate(x)
–
4. path cost (additive)
– e.g., sum of distances, number of actions executed, etc.
– c(x,a,y) is the step cost, assumed to be ≥ 0
• A solution is a sequence of actions leading from the initial state
to a goal state

9
 Search strategies
• A search strategy is defined by picking the order of
node expansion
• Types of search strategies
1. Uninformed search: no problem specific information
2. Heuristic search: existence of problem-specific information
• Strategies are evaluated along the following dimensions:
– completeness: does it always find a solution if one exists?
– time complexity: number of nodes generated
– space complexity: maximum number of nodes in memory
– optimality: does it always find a least-cost solution?

10
 Uninformed search strategies
• Uninformed search strategies use only the information
available in the problem definition
– Breadth-first search
– Depth-first search
– Depth-limited search
– Iterative deepening search
– Uniform-cost search

11
 Uninformed search strategies

12
 Depth-limited search

• Set the depth-limit to 1. Only level 0 and 1 get expanded in

DFS sequence (A->B->C)
• No satisfactory result, because we could not reach the goal
node I

13
 Iterative deepening
– The goal node is H and initial
depth-limit is 1. It will
expand level 0 and 1 and will
terminate with A->B->C.
– Change the depth-limit to 3.
It will again expand the nodes
from level 0 till level 3 and
the search terminate with
A->B->D->F->E->H where
H is the desired goal node.

14
 Uniform-cost search
• Expand least-cost unexpanded node, a.k.a. Dijkstra’s Algorithm
[Dijkstra 1959]
• Implementation:
– fringe = queue ordered by path cost
• Equivalent to breadth-first if step costs all equal

Difference from BFS:

The goal test is applied to a node only
when it is selected for expansion not
when it is first generated because the first
goal node which is generated may be on
a suboptimal path.

15
 Outline
• Problem-solving agents
• Problem formulation
• Basic search algorithms
• Heuristic search
• Beyond classical search

16
 Heuristic-guided search

Optimization
Search problems
problems

• Every move is associated with a cost

• Find a solution path that minimizes the overall cost

• Greedy best-first search

• A* search

17
 Greedy best-first search
• Idea: use an evaluation function f(n) for each node
→Expand most promising paths
• Evaluation function
f(n) = h(n) (heuristic)
= estimate of cost from n to goal
e.g., hSLD(n) = straight-line distance from n to Bucharest

• Greedy best-first search expands the node that appears to be

closest to goal

18
 A* search
• Idea: avoid expanding paths that are expensive

• Evaluation function f(n) = g(n) + h(n)

• g(n) = cost so far to reach n

• h(n) = estimated cost from n to goal
• f(n) = estimated total cost of path through n to goal

19
 A* search
S is the root node, and G is the goal node.
• f(n) = g(n) + h(n)
• g(n) = cost so far to reach n
• h(n) = estimated cost from n to goal
• f(n) = estimated total cost of path
through n to goal

What is the path to reach G?

20
 Admissible heuristics
• A heuristic h(n) is admissible if for every node n,
h(n) ≤ h*(n), where h*(n) is the true cost to reach the goal state
from n.
h(n) ≤ h*(n) is underestimation of the cost value
h(n) ≥ h*(n) is overestimation of the cost value

• An admissible heuristic never overestimates the cost to reach the

goal

21
 Consistent heuristics
• A heuristic is consistent if for every node n, every successor n'
of n generated by any action a, then

h(n) ≤ c(n,a,n') + h(n')

• If h is consistent, we have

f(n') = g(n') + h(n')

= g(n) + c(n,a,n') + h(n')
≥ g(n) + h(n)
= f(n)
i.e., f(n) is non-decreasing along any path.

22
 Optimality of A*
• Theorem: If h(n) is admissible, A* using TREE-SEARCH is
optimal; If h(n) is consistent, A* using GRAPH-SEARCH is
optimal
• Proof:
– Suppose some suboptimal goal G2 has been generated and is in the fringe.
Let n be an unexpanded node in the fringe such that n is on a shortest
path to an optimal goal G.
– Can proof that A* will never select G2 for expansion

23
 Properties of A*
• Complete? Yes (unless there are infinitely many
nodes with f ≤ f(G) )

• Time? Exponential

• Space? Keeps all nodes in memory

• Optimal? Yes

24
 Heuristic function
Admissible heuristics
E.g., for the 8-puzzle:

• h1(n) = number of misplaced tiles

• h2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

• h1(S) = ?
• h2(S) = ?

25
 Admissible heuristics
E.g., for the 8-puzzle:

• h1(n) = number of misplaced tiles

• h2(n) = total Manhattan distance
(i.e., no. of squares from desired location of each tile)

• h1(S) = ? 8
• h2(S) = ? 3+1+2+2+2+3+3+2 = 18

26
 Dominance
• If h2(n) ≥ h1(n) for all n (both admissible)
• then h2 dominates h1

===》 h2 is better for search

• Typical search costs (average number of nodes expanded):

• d=12 IDS = 3,644,035 nodes (ITERATIVE-DEEPENING-

SEARCH)
A*(h1) = 227 nodes
A*(h2) = 73 nodes

• d=24 IDS = too many nodes

A*(h1) = 39,135 nodes
A*(h2) = 1,641 nodes
27
 Summary
• Breadth First Search
• Uniform Cost Search
• Greedy Best-first Search
• A*

28
 Summary
• Breadth First Search
– explore equally in all directions
• Uniform Cost Search
– prioritize which paths to explore and favors lower cost paths
• Greedy Best-first Search
– expand the node that appears to be closest to goal
• A* : consider both the cost from start to current node n and estimated cost
from n to goal
BFS cost UCS
g(n)
heuristic

h(n)

Greedy A*

f(n) = g(n) + h(n)

29
 Beyond Classical Search

30
 Outline
• Local search algorithms
• Hill-climbing search
• Simulated annealing search
• Local beam search
• Genetic algorithms

31
 Local search algorithms
• Search algorithms like breadth-first, depth-first or A*
explore all the search space systematically by keeping
one or more paths in memory and by recording which
alternatives have been explored.
• When a goal is found, the path to that goal constitutes a
solution.
• Local search algorithms can be very helpful if we are
interested in the solution state but not in the path to
that goal. They operate only on the current state and
move into neighboring states.

32
 Example: n-queens
• Put n queens on an n×n board with no two queens on
the same row, column, or diagonal

33
 Local search algorithms
• Local search algorithms have 2 key advantages:
– They use very little memory
– They can find reasonable solutions in large or infinite
(continuous) state spaces.
• Some examples of local search algorithms are:
– Hill-climbing
– Random walk
– Simulated annealing

34
 Hill-climbing search
• “Like climbing Everest in thick fog with amnesia”
• Problem: depending on initial state, can get stuck
in local maxima

35
 Hill-climbing search: 8-queens problem

• h = number of pairs of queens that are attacking each other,

either directly or indirectly
• h = ？ for the above state
36
 Hill-climbing search: 8-queens problem

• A local minimum with h = 1

37
 Local search: hill climbing
• Start with a complete state
• Move to successor with best (or at least better) objective value
– Successor: move one queen within its column
Q Q Q
Q Q Q
Q Q Q Q
Q Q Q no more
Q Q Q improvements
Q Q Q
Q Q
Q Q Q

4 attacking pairs 3 attacking pairs 2 attacking pairs

• Local search can get stuck in a local optimum

local optimum global optimum

38
 Avoiding getting stuck with local search
• Random restarts: restart at a random point in the search
space
• Simulated annealing:
– Generate a random successor (possibly worse than current state)
– Move to that successor with some probability that is sharply
decreasing in the badness of the state
– Also, over time, as the “temperature decreases,” probability of
bad moves goes down

39
 Annealing
• The process of annealing contains two steps:
– Increase the temperature to a maximum value at which the
solid melts.
– Decrease carefully the temperature until the particles arrange
themselves in the ground state of the solid. Ground state is a
minimum energy state of the solid.
• The ground state of the solid is obtained only if the
maximum temperature is high enough and the cooling
is done slowly.

Source https://baijiahao.baidu.com/s?id=1662474966979928280

40
 Simulated annealing
• To apply simulated annealing with optimization purposes we
require the following:
– A successor function that returns a “close” neighboring solution given the
actual one. This will work as the “disturbance” for the particles of the
system.
– A target function to optimize that depends on the current state of the
system. This function will work as the energy of the system.
• Process
– Start with a randomized state
– Move to neighboring states …
successor returns a state
ΔE<0, ΔE>0,
good move compare with the bad move
current state
accept the state as
accept the state current state by
as current state probability p
(determined by T)

41
 Simulated annealing
Decrease the temperature slowly, accepting less bad moves at
each temperature level until at very low temperatures the
algorithm becomes a greedy hill-climbing algorithm.

The distribution used to

decide if we accept a bad
movement is know as
Boltzman distribution.
where γ is the current configuration of the system, E γ is the
energy related with it, and Z is a normalization constant.

42
 Simulated annealing search
• Idea: escape local maxima by allowing some "bad"
moves but gradually decrease their frequency
start

generate a state
current

yes output
T=0?
current
end
no
next ← a randomly selected successor of current

change T
by schedule
ΔE ← VALUE[next] - VALUE[current]

ΔE < 0 ?

current ← next current ← next only with

probability exp(-ΔE/T)

43
 Practical issues with simulated
annealing
• Cost function must be carefully developed and
should be fast
• The energy function of the left would work with
Simulated Annealing while the one of the right
would fail.

44
 Local beam search
• Keep track of k states rather than just one

• Start with k randomly generated states

• At each iteration, all the successors of all k states are

generated

• If any one is a goal state, stop; else select the k best

successors from the complete list and repeat.

45
 Outline
• Local search algorithms
• Hill-climbing search
• Simulated annealing search
• Local beam search
• Genetic algorithms

46
 Genetic algorithms

Algorithms in Nature

Credits:
Ziv Bar-Joseph, CMU https://www.cs.cmu.edu/~02317/

47
 What is GA
• A search technique used in computing to find true
or approximate solutions to optimization and
search problems.
• Global search heuristics.
• A particular class of evolutionary algorithms that
use techniques inspired by evolutionary biology
such as inheritance, mutation, selection, and
crossover (also called recombination).

48
 What is GA
• The evolution usually starts from a population
of randomly generated individuals and
happens in generations.

• In each generation, the fitness of every

individual in the population is evaluated,
multiple individuals are selected from the
current population (based on their fitness), and
modified to form a new population.

49
 What is GA
• The new population is used in the next iteration
of the algorithm.

• The algorithm terminates when either a

maximum number of generations has been
produced, or a satisfactory fitness level has
been reached for the population.

No convergence rule
or guarantee!
50
 Vocabulary
• Individual - Any possible solution
• Population - Group of all individuals
• Fitness – Target function that we are optimizing (each
individual has a fitness)
• Trait - Possible aspect (features) of an individual
• Genome - Collection of all chromosomes (traits) for an
individual.

51
 Basic genetic algorithm
• Start with a large “population” of randomly
generated “attempted solutions” to a problem
• Repeatedly do the following:
– Evaluate each of the attempted solutions
– (probabilistically) keep a subset of the best
solutions
– Use these solutions to generate a new population
• Quit when you have a satisfactory solution (or you
run out of time)

52
 General framework of GC
Generate Initial Population

Fitness Function
Evaluate Fitness

Yes
Termination Condition? Best Individual

No

Select Parents
Crossover, Mutation

Generate New Offspring

53
 Example: the MAXONE problem
Suppose we want to maximize the number of ones in
a string of l binary digits
• An individual is encoded (naturally) as a string of l
binary digits
• The fitness f of a candidate solution to the
MAXONE problem is the number of ones in its
genetic code
• We start with a population of n random strings.
Suppose that l = 10 and n = 6

10
54
 Example (initialization)
We toss a fair coin 60 times and get the
following initial population:
s1 = 1111010101 f (s1) = 7
s2 = 0111000101 f (s2) = 5
s3 = 1110110101 f (s3) = 7
s4 = 0100010011 f (s4) = 4
s5 = 1110111101 f (s5) = 8
s6 = 0100110000 f (s6) = 3

12
55
 Step 1: Selection
We randomly (using a biased coin) select a subset of
the individuals based on their fitness:
Individual i will have a f (i)

probability to be chosen  f (i)

i

1 Area is
2
n Proportional
to fitness
value
3
…
4

13
56
 Selected set
Suppose that, after performing selection, we get
the following population:

s1` = 1111010101 (s1)

s2` = 1110110101 (s3)
s3` = 1110111101 (s5)
s4` = 0111000101 (s2)
s5` = 0100010011 (s4)
s6` = 1110111101 (s5)

14
57
 Step 2: crossover
• Next we mate strings for crossover. For each
couple we first decide (using some pre-
defined probability, for instance 0.6) whether
to actually perform the crossover or not
• If we decide to actually perform crossover, we
randomly extract the crossover points

15
58
 Crossover result
Before crossover:
s1` = 1111010101 s2` = 1110110101

After crossover:
s1`` = 1110110101 s2`` = 1111010101

16
59
 Step 3: mutations
The final step is to apply random mutations: for each bit that we are to copy to
the new population we allow a small probability of error (for instance 0.1)

Initial strings After mutating

s1`` = 1110110101 s1``` = 1110100101
s2`` = 1111010101 s2``` = 1111110100
s3`` = 1110111101 s3``` = 1110101111
s4`` = 0111000101 s4``` = 0111000101
s5`` = 0100011101 s5``` = 0100011101
s6`` = 1110110011 s6``` = 1110110001

17
60
 And now, iterate …

In one generation, the total population fitness

changed from 34 to 37, thus improved by ~9%

At this point, we go through the same process

all over again, until a stopping criterion is met

61
 Biological motivation

62
 GA operators
• Methods of Representation
• Methods of Selection
• Methods of Reproduction

63
 Common representation methods
• Binary strings.
• Arrays of integers (usually bound)
• Arrays of letters
• ….

64
 Selection

• Selecting individuals to be parents

• Chromosomes with a higher fitness value will have a
higher probability of contributing one or more
offspring in the next generation

65
 Methods of selection
• Roulette-wheel selection.
• Elitist selection.
• Fitness-proportionate selection.
• Scaling selection.
• Rank selection.
• …

66
 Roulette wheel selection
• Conceptually, this can be represented as a
game of roulette - each individual gets a slice
of the wheel, but more fit ones get larger slices
than less fit ones.

67
 Roulette wheel selection

No. String Fitness % Of Total

1 01101 169 14.4

2 11000 576 49.2

3 01000 64 5.5

4 10011 361 30.9

Total 1170 100.0

68
 Other selection methods

• Elitist selection:
Chose only the most fit members of each
generation.
• Cutoff selection:
Select only those that are above a certain
cutoff for the target function.

69
 Methods of reproduction
• There are primary methods:
–Crossover
–Mutation

70
 Methods of reproduction:
Crossover
– Two parents produce two offspring
– Two options:
1.The chromosomes of the two parents are copied
to the next generation
2.The two parents are randomly recombined
(crossed-over) to form new offsprings

71
 Several possible crossover strategies

• Randomly select a single point for a

crossover
• Multi-point crossover
• Uniform crossover

72
 Crossover
• Single point crossover

Cross point
• Two point crossover (Multi point crossover)

73
 Uniform crossover
• A random subset is chosen
• The subset is taken from parent 1 and the other bits from parent 2.

Subset: BAABBAABBB (Randomly generated)

Parents: 1010001110 0011010010

Offspring: 0011001010 1010010110

74
 Methods of reproduction: Mutations

– Generating new offspring from single parent

75
 Benefits of genetic algorithms
• Concept is easy to understand
• Modular, separate from application
• Supports multi-objective optimization
• Always an answer; answer gets better with time.
• Easy to exploit previous or alternate solutions
• Flexible building blocks for hybrid applications

76
 Local Search in
Continuous Spaces

77
 Discrete and continuous
environments
• Most real-world environments are continuous
• Most of the algorithms in previous lectures
cannot handle continuous state and action space,
as they have infinite branching factors.

78
 Local search in continuous spaces
• Suppose we want to place 3 new airports in
Romania, such that the sum of squared distances
from each city to its nearest airport is minimized
• The state space is then defined by the
coordinates of the airports:
– 6-D state space
– Object function

Let Ci be the set of cities whose closest airport (in the current state) is airport i

79
 Local search in continuous spaces

80
 Particle Swarm Optimization

81
 Characteristics of swarms

• Composed of many individuals

• Individuals are homogeneous
• Local interaction based on simple rules
• Self-organization ( No centralized Control)

82
 Particle Swarm Optimization

Source https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSKhLOQonxCPbIKl6GE0UWhaJJByuKpYFtWDUovH1Ss0HUaaWcq

83
 Particle Swarm Optimization (PSO)

• PSO is stochastic optimization technique proposed

by Kennedy and Eberhart (1995)
• A population based search method with position of
particle is representing solution and Swarm of
particles as searching agent.
• PSO is a robust evolutionary optimization
technique based on the movement and intelligence
of swarms.
• PSO find the minimum value for the function.

84
 PSO search scheme
• The idea is similar to bird flocks searching for
food.
– Bird = a particle, Food = a solution
– pbest : the best solution achieved so far by that
particle.
– gbest : the best value obtained so far by any particle
in the neighborhood of that particle.
- The basic concept of PSO lies in accelerating each
particle toward its pbest and the gbest locations, with a
random weighted acceleration at each time.

85
 PSO search scheme
- PSO uses a number of agents, i.e., particles, that
constitute a swarm flying in the search space looking for
the best solution.

- Each particle is treated as a point (candidate solution)

in a N-dimensional space which adjusts its “flying”
according to its own flying experience as well as the
flying experience of other particles.

86
 Global best

New Velocity

Position X Personal best

87
 Particle Swarm Optimization (PSO)
Each particle tries to modify its position X using the following
formula:
X (t+1) = X(t) + V(t+1) (1)
V(t+1) = wV(t) + c1 ×rand ( ) × ( Xpbest - X(t)) + c2 ×rand ( ) × ( Xgbest - X(t)) (2)

V(t) velocity of the particle at time t

X(t) Particle position at time t
w Inertia weight
c1 , c2 learning factor or accelerating factor
rand uniformly distributed random number
between 0 and 1
Xpbest particle’s best position
Xgbest global best position

Slides from Nasser M.

88
 PSO algorithm
Input: Randomly initialized position and velocity of Particles:
Xi (0) andVi (0)
Output: Position of the approximate global minimum X*

1: while terminating condition is not reached do

2: for i = 1 to number of particles do
3: Calculate the fitness function f
4: Update personal best and global best of each particle
5: Update velocity of the particle using Equation 2
6: Update the position of the particle using equation 1
7: end for
8: end while

89