Méthodes de Résolution des Problèmes

 Prof. Nadjet KAMEL

 Introduction
 Examples of Complex Problems
 Algorithms for Search problems
◦ Uninformed Search : BFS, DFS
◦ Informed Search: A*
 Algorithms for Optimization problems
◦ Hill Climbing
◦ Tabu Search
◦ Simulated Annealing
◦ Genetic Algorithms
◦ Particle Swarm Optimization
 Some typical Artificial Intelligence Problems
◦ Playing games : chess game. The way the Computer choose which move to make is govered by
AI
◦ Navigation : find the shortest road to go from A to B, the road that has the less trafic
 Path finding where we have different possibilities; find the most optimal path
◦ Planning and scheduling
 Find an optimal sequence of actions to solve a given task
 Formalize the problem
◦ Goal : objective task
◦ Actions : possible moves
◦ States : what is changing if we are performing one move
 Some problems

 Traveling sales person problem(TSP):

"Given a list of cities and the distances between each pair of
cities, what is the shortest possible route that visits each city exactly
once and returns to the origin city?".

◦ If n cities then nbre. of different paths among them are (n-1) !

◦ Time to examine single path is proportional to n .
◦ Total search time required T (total) = n ( n-1) ! = n !,

◦ If n = 10 then 10 ! = 3, 628 , 800 paths are possible. This is very large nb.
◦ This phenomenon of growing nb. of possible paths as n increases is called “
Combinatorial Explosion”
Some problems
 The knapsack problem
Given a set of items, each with a weight and a value, determine the number of each item
to include in a collection so that the total weight is less than or equal to a given limit and
the total value is as large as possible.

◦ Maximize Subject to
n n
 W and xi  0,1
v x
i =1
i i w x
i =1
i i

 0-1 knapsack problem : restricts the number of each kind of item to zero or one.
Example :
Values = [20, 5, 10, 40, 15, 25] Sol = items 1 and 4.
Weight = [1, 2, 3, 8, 7, 4] Weight = 1+8=9
Limit = 10 Value = 20+40= 60
 Example of Complex Problems

 The N Queen problem

Place N chess queens on an N×N
chessboard so that no two queens
attack each other.
 Example :
◦ For N=8 we have 64·63 ···57 ≈1.8 ×1014 possible
configurations to investigate
 Example of Complex Problems

 The Puzzel Problem:

◦ Given a 3×3 board with 8 tiles (every tile has one number from 1
6 4 7
to 8) and one empty space.
◦ The objective is to place the numbers on tiles to match the final 5 2
configuration using the empty space. 3 8 1
◦ We can slide four adjacent (left, right, above, and below) tiles into
the empty space.
Example
 State = permutation of {, 1,2,3,4,5,6,7,8} 1 2
 9 = 362880 possible states 3 4 5
 Solution = {, 1,2,3,4,5,6,7,8}
6 7 8
 Actions = {up, down, left, right}
Search Problems
 The state space of a problem includes:
 An initial state
 One or more goal states.
 Sequence of intermediate states through which the system makes transition
while applying various rules.

 State space may be a tree or a graph.

 A solution is a sequence of actions which transforms the

start state to a goal state
• Idea: Problem Solving as Search
• Basic formalism as State-Space Graph
• Graph explored by Tree Search
• Different algorithms to explore the graph
2 Types of search
 Uninformed or blind search
◦ – no clue whether one node is better than any other
◦ – search blindly and exhaustively
◦ – taking into account only the path already explored
◦ – depth-first search, breadth-first search, uniform-cost search

 Informed search
◦ – Try to take the goal into account
◦ – Need heuristic function to approximate distance to the goal
◦ – guided by heuristic, but heuristic might be wrong …
◦ – (greedy) best-first search, A* algorithm
Example : Romania
 On holiday in Romania; currently in Arad.
 Flight leaves tomorrow from Bucharest

 Formulate goal:
 – be in Bucharest

 Formulate problem:
 – states: various cities
 – actions: drive between cities

 Find solution:
 – sequence of cities, e.g., Arad, Sibiu, Fagaras, Bucharest
State Space Problem Formulation
 A problem is defined by four items:
 1. set of states, e.g., "at some city“

 2. initial state, e.g., "at Arad“

 3. actions or successor function

◦ – S(x) = set of action–state pairs, e.g., S(Arad) = {<Arad → Zerind, Zerind>, … }
◦ – or simply S(x) = successor states, e.g., S(Arad) = {Zerind, Sibiu, Timisoara }

 4. goal test, can be

◦ – explicit, by giving a list of nodes: e.g., x = "at Bucharest"
◦ – implicit, by Boolean function: e.g., Checkmate(x)

 option: path cost (additive) e.g., sum of distances, number of actions executed, etc

 A solution is a sequence of actions leading from the initial state to a goal

 Another Example: The 8-puzzle

 Start state Goal state

• states? locations of tiles, e.g. (7,2,4,5,O,6,8,3,1)

• actions? move blank left, right, up, down
• goal test? goal state is given: (O,1,2,3,4,5,6,7,8)
• path cost? 1 per move
From Problems to Graphs
 a (directed) graph is a couple (N,A) where
 – N is a set of nodes
 – A of ordered pairs (couples) of nodes from A called arcs.

 a path from node s to node g is a sequence of nodes

 ⟨n0 , n1 ,..., nk ⟩ s.t. s=n0 , g=nk , and ⟨ni-1 ,ni⟩ ∈ A

 A cycle is a nonempty path such that the end node is the same as the start node

 arcs can be labeled by a cost function : A → ℝ+

 For a path p = ⟨n0, n1,..., nk⟩, the cost of path p is

cost(p) = cost(⟨n0,n1⟩) + ...+ cost(⟨nk-1,nk⟩)

 Unlabeled arcs ⟨n,n’⟩ ∈ A , cost (⟨n,n’⟩ ) = 1

Graph defined by a problem
 Nodes: set of nodes generated by iteratively applying actions
from starting node
◦ N = { n /  kℕ , n= SK(n₀)}, for initial state n₀
 Arcs: defined by successor function (i.e. actions)
◦ A = { (n,S(n)) / n  N}

 Graph is big ! e.g., for 8-puzzle problem: 9! = 362,880 states

 Graph is implicit
 Graph can be infinite …
State Space Graph
 State space graph: A mathematical representation of a search problem
◦ Nodes are problem configurations
◦ Arcs represent transitions. Transitions in the graph correspond to actions
◦ The goal test is a set of goal nodes
◦ The goal is to reach those nodes starting from an initial node.

a G
 In a search graph, each state occurs only once
b c

e
d f
S h
p r
q
 Search Tree

State Space Graph Search Tree

a G Each NODE d e p
b c
in the search tree b c e h r q
d
e is an entire PATH
f a a h r p q f
S in the state space graph.
h
p q f q c G
p q r
q c G a

a
 Search Tree

•A search tree: 6 4 7

•The start state is the root node 5 2

3 8 1
•Nodes show states U
D
•Children correspond to successors
R L
6 4 7 6 4 7 6 4 7
6 7
5 2 5 2 5 8 2
5 4 2
3 8 1 3 8 1 3 1
3 8 1
Example: Romania

 Partial search trees for finding a route from Arad to Bucharest.

◦ Nodes that have been expanded are shaded;
◦ nodes that have been generated but not yet expanded are outlined in bold;
◦ nodes that have not yet been generated are shown in faint dashed lines
 Example: 8 Puzzel Problem
S G 1 2 3
1 2 3 1 2 3 4 6
4 6 4 5 6 7 5 8
R U
7 5 8 7 8 D
1 2 3 2 3 1 2 3
4 moves = {Up, Down, Right, Left}
4 6 1 4 6 7 4 6

U 7 5 8 7 5 8 5 8
D R D R
U
L R
1 2 3 1 2 3 1 2 3 1 3 2 3 1 2 3 1 2 3 1 2 3
4 5 6 4 6 4 6 4 2 6 1 4 6 4 6 4 6 7 4 6
7 8 7 5 8 7 5 8 7 5 8 7 5 8 7 5 8 7 5 8 5 8
R
L
1 2 3 1 2 3
4 5 6 4 5 6
7 8 7 8
Tree search algorithms
 Basic idea:
◦ simulated exploration of state space by generating successors
of already-explored states (a.k.a. expanding states)
Search strategies
 A search strategy is defined by the order of node expansion

 Strategies are evaluated along the following dimensions:

◦ – completeness: does it always find a solution if one exists?
◦ – time complexity: number of nodes generated
◦ – space complexity: maximum number of nodes in memory
◦ – optimality: does it always find a least-cost solution?

 Time and space complexity are measured in terms of

 – b: maximum branching factor of the search tree
 – d: depth of the least-cost solution
 – m: maximum depth of the state space (may be ∞)

 – l: the depth limit (for Depth-limited complexity)

 – C*: the cost of the optimal solution (for Uniform-cost complexity)
 – ε: minimum step cost, a positive constant (for Uniform-cost complexity)
Classification fo search stategies
1- Uninformed Search stategies
◦ Search without information
◦ More complexity
 Depth First Search (DFS), Breadth First Search (BFS), …
2- Informed Search
◦ Search with information
◦ We can estimate how far it is from the goal
◦ Less complexity
 A*, Heuritic DFS, BFS, …
Uninformed Search stategies
Depth-First Search
 Strategy: expand deepest unexpanded node

 Implementation:
 – put new successor nodes in front of frontier
 – i.e. select last element added to the frontier
 – i.e. treat frontier as a stack, i.e. LIFO
 Uniformed search stratigies:
DFS : Depth First Search
A
 Strategy: Expand deepest unexpanded node
 Uses Stack (LiFo)
 Time complexity: O(bm); terrible if m is much larger than d
 Space: O(bm) B C
◦ Not complete:
 d could be infinite, so only if we prevent cycles
D E F G
 Not optimal:
◦ it finds the “leftmost” solution, regardless of depth or cost
D
A B D E C F G B E E F
C C C C G G
Properties of depth-first search
 Complete? No: fails in infinite-depth state-spaces, and state-
spaces with loops
◦ But can be modified to avoid repeated states along path
 complete in finite spaces

 Time? O(bm): terrible if m is much larger than d

◦ but if solutions are dense, may be much faster than breadth-first

 Space? O(bm), i.e., linear space!

 Optimal? No
Uninformed Search stategies
Breadth-First Search
 Strategy: expand shallowest unexpanded node

 Implementation:
◦ – put new successor nodes at end of frontier
◦ – i.e. select first element added to the frontier
◦ – i.e. treat frontier as a queue, i.e. FIFO
 Uniformed search stratigies:
BFS : Breadth First Search
 Strategy: Expand shallowest A KLM
unexpanded node BCD LMN
 Uses FiFo Queue 0 A
CDEF MN
 Complete if b is finite
 Time complexity = O(bd)
DEFGH N 1 B C D

 Space complexity = O(bd) EFGHI

2 E F G H I
(keeps every node in FGHIJK
memory)
GHIJK J K L M
 Optimal if cost = 1 per step 3
HIJKL
 Space is the bigger problem IJKL 4 N
(more than time) JKLM
Properties of breadth-first search
 Complete? Yes (if b is finite)

 Time? 1+b+b2+b3+… +bd + b(bd-1) = O(bd+1)

 Space? O(bd+1) (keeps every node in memory)

 Optimal? Yes (if cost = 1 per step)

 Space is the bigger problem (more than time)

 Uninformed Search stategies
Depth limited search

 Depth First Search with

depth limit l to avoid infinite
depth, i.e., nodes at depth l have
no successors
 Uniformed search stratigies:
Iterative Deepening search

 It gradually increases the limit—first 0, then 1, then 2,

and so on—until a goal is found.

 In general, iterative deepening is the preferred

uninformed search method when there is a large search
space and the depth of the solution is not known
 Complete
 Time complexity = O(bd)
 Space complexity= O (bd)
 Optimal if step cost = 1
 Uniformed search stratigies:
Iterative Deepening search. Example

 Iterative deepening search l =0

 Uniformed search stratigies:
Iterative Deepening search. Example

 Iterative deepening search l =1

 Uniformed search stratigies:
Iterative Deepening search. Example

 Iterative deepening search l =2

 Uniformed search stratigies:
Iterative Deepening search. Example

 Iterative deepening search l =3

Uninformed Search stategies
Bidirectional Search
 Bidirectional search
◦ run two simultaneous searches—one forward from the initial state and the other
backward from the goal—stopping when the two searches meet in the middle
 Time Complexity : O(bd/2)
 Complete if b is finite and if both directions use breadth-first search
 Optimal if both directions use breadth-first search and step costs are all identical
Backtracking
 L’approche de « backtracking » explore systématiquement l’espace exhaustif
des solutions
 Le « backtracking » est une technique algorithmique méthodique qui essaie
plusieurs configurations jusqu’à en trouver une qui soit une solution.
 On dispose d’un ensemble de configurations « solutions potentielles » dont
on ne possède préalablement aucune information
 On recule devant la sélection d’une mauvaise configuration
 On favorise les configurations les plus prometteuses si plusieurs se
présentent. C1

C2
P
Ci

Cn
Boolean Explore (noeud N)
{
Si N est une solution
return True
Sinon Si N est un échec
return False
Sinon Pour tout fils F étendant N
return Explore(F)
return False /* tous les autres cas */
}
Exemple: Recherche de circuit hamiltonien
On dispose d’un graphe G(X,E)

Problème : On veut rechercher un circuit éventuel qui passe par tous les nœuds de G
Un backtracking serait de
1. Commencer à un noeud n0 1
2. Explorer systématiquement les nœuds adjacents et marquer les nœuds visités
3. Progresser récursivement vers le nœud n0
2 3 6
Exemple : le graphe suivant contient un circuit hamiltonien
1 3
1 2 4
1 2 7 5
5 3 7

6 3 4 6 4 7 4 5
4 5 4 6 7
5 1

1 5