www.rejinpaul.

com
CS8691 – Artificial Intelligence

UNIT II PROBLEM SOLVING METHODS

Problem solving Methods - Search Strategies- Uninformed - Informed - Heuristics
- Local Search Algorithms and Optimization Problems - Searching with Partial
Observations - Constraint Satisfaction Problems – Constraint Propagation -
Backtracking Search - Game Playing - Optimal Decisions in Games – Alpha - Beta
Pruning - Stochastic Games.

Searching for Solutions

Search techniques use an explicit search tree that is generated by the initial
state and the successor function that together define the state space. In general,
we may have a search graph rather than a search tree, when the same state can
be reached from multiple paths

Example Route finding problem

The root of the search tree is a search node corresponding to the initial state,
In(Arad).

The first step is to test whether this is a goal state.

Apply the successor function to the current state, and generate a new set of
states

In this case, we get three new states: In(Sibiu),In(Timisoara), and In(Zerind).

Now we must choose which of these three possibilities to consider further.

CSE – Panimalar Engineering College 1

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Continue choosing, testing, and expanding until either a solution is found or

there are no more states to be expanded.

The choice of which state to expand is determined by the search strategy

Tree Search algorithm

Task : Find a path to reach F from A

1. Start the sequence with the initial state and check whether it is a goal state or
not.

a, If it is a goal state return success.

b. Otherwise perform the following sequence of steps

From the initial state (current state) generate and expand the new set of states.
The collection of nodes that have been generated but not expanded is called as
fringe. Each element of the fringe is a leaf node, a node with no successors in
the tree.

Expanding A

Expanding B

Expanding C

CSE – Panimalar Engineering College 2

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Sequence of steps to reach the goal state F from (A = A - C - F)

2. Search strategy: In the above example we did the sequence of choosing,
testing and expanding until a solution is found or until there are no more states
to be expanded. The choice of which state to expand first is determined by
search strategy.
3. Search tree: The tree which is constructed for the search process over the
state space.
4. Search node: The root of the search tree that is the initial state of the
problem.

The general tree search algorithm

function TREE-SEARCH(problem. strategy) returns a

solution or failure
initialize the search tree using the initial state of
problem
loop do
if there are no candidates for expansion then return
failure
choose a leaf node for expansion according to strategy
if the node contains a goal state then return the
corresponding solution
else expand the node and add the resulting nodes to the
search tree

There are many ways to represent nodes, but we will assume that a node is a
data structure with five components:

STATE: the state in the state space to which the node corresponds
PARENT-NODE: the node in the search tree that generated this node;
ACTION (RULE): the action that was applied to the parent to generate the
node;
PATH-COST: the cost, traditionally denoted by g(n) , of the path from the initial
state to the node
DEPTH: the number of steps along the path from the initial state.

The collection of nodes represented in the search tree is defined using set or
queue representation.

CSE – Panimalar Engineering College 3

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Set : The search strategy would be a function that selects the next node to be
expanded from the set

Queue: Collection of nodes are represented, using queue. The queue operations
are defined as:

MAKE-QUEUE(elements) - creates a queue with the given elements

EMPTY(queue)-returns true only if there are no more elements in the queue.
REM0VE-FIRST(queue) - removes the element at the front of the queue and
returns it
INSERT ALL (elements, queue) - inserts set of elements into the queue and
returns the resulting queue.
FIRST (queue) - returns the first element of the queue.
INSERT (element, queue) - inserts an element into the queue and returns the
resulting queue

The general tree search algorithm with queue representation

function TREE-SEARCH(problem,fringe) returns a

solution, or failure
fringe <- INSERT(MAKE-NODE(INITIAL-STATE[problem]),
fringe)
loop do
if EMPTY?(fringe) then return failure
node <- REMOVE-FIRST(fringe)
ifGOAL-TEST[problenl]applied to STATE[node] succeeds
then return SOLUTION(node)
fringe <- INSERT-ALL(EXPAND(node, problem),fringe)

CSE – Panimalar Engineering College 4

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

function EXPAND(node, problem) returns a set of nodes

successors <- the empty set
for each <action, result> in SUCCESSOR-FN
[problem](STATE[node])do
S <- a new NODE
STATE[s] <- result
PARENT-NODE[s] <- node
ACTION[s] <- action
PATH-COST[s] <- PATH-COST[node]+STEP-COST(node,action,s)
DEPTH[s] <- DEPTH[node] + 1
add s to successors
return successors

Example: Route finding problem

Task. : Find a path to reach E using Queuing function in general tree search
algorithm

Measuring problem solving performance

The search strategy algorithms are evaluated depends on four important

criteria’s. They are:

CSE – Panimalar Engineering College 5

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

(i) Completeness : The strategy guaranteed to find a solution when there is

one.
(ii) Time complexity : Time taken to run a solution
(iii) Space complexity : Memory needed to perform the search.
(iv) Optimality : If more than one way exists to derive the solution then the
best one is Selected

Definition of branching factor (b): The number of nodes which is connected

to each of the node in the search tree. Branching factor is used to find space and
time complexity of the search strategy

Solving Problems by Searching

The searching algorithms are divided into two categories

1. Uninformed Search Algorithms (Blind Search)

2. Informed Search Algorithms (Heuristic Search)

There are six Uninformed Search Algorithms

1. Breadth First Search

2. Uniform-cost search
3. Depth-first search
4. Depth-limited search
5. Iterative deepening depth-first search
6. Bidirectional Search

There are three Informed Search Algorithms

1. Best First Search

2. Greedy Search
3. A* Search
Blind search Vs Heuristic search

Blind search Heuristic search

CSE – Panimalar Engineering College 6

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

No information about the number of The path cost from the current state to
steps (or) path cost from current state the goal state is calculated, to select
to goal state the minimum path cost as the next
state.
Less effective in search method More effective in search method
Problem to be solved with the given Additional information can be added as
information assumption to solve the problem

Breadth-first search

Breadth-first search is a simple strategy in which the root node is expanded

first, then all the successors of the root node are expanded next, then their
successors, and so on. In general, all the nodes are expanded at a given depth
in the search tree before any nodes at the next level are expanded.

Breadth-first search can be implemented by calling TREE-SEARCH with an empty

fringe that is a first-in-first-out (FIFO) queue, assuring that the nodes that are
visited first will be expanded first.

In other words, calling TREE-SEARCH(Problem, FIFO-QUEUE())results in a

breadth-first search. The FIFO queue puts all newly generated successors at the
end of the queue, which means that shallow nodes are expanded before deeper
nodes

Breadth first search trees after node expansions

Example: Route finding problem

CSE – Panimalar Engineering College 7

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Task: Find a ,path from. S to G using BFS

The path in the 2nd depth level is selected, (i.e) SBG{or) SCG.

CSE – Panimalar Engineering College 8

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Algorithm :

function BREADTH-FIRST-SEARCH(problem) returns a solution, or failure

node ←a node with STATE = problem.INITIAL-STATE, PATH-COST = 0
if problem.GOAL-TEST(node.STATE) then return SOLUTION(node)
frontier ←a FIFO queue with node as the only element
explored ←an empty set
loop do
if EMPTY?( frontier ) then return failure
node←POP( frontier ) /* chooses the shallowest node in frontier */
add node.STATE to explored
for each action in problem.ACTIONS(node.STATE) do
child ←CHILD-NODE(problem, node, action)
if child.STATE is not in explored or frontier then
if problem.GOAL-TEST(child.STATE) then return SOLUTION(child)
frontier ←INSERT(child, frontier )

Time and space complexity:

Example:

Time complexity

= 1 +b + b 2 + . . . . . . . + b d

= O(b d)

The space complexity is same as time complexity because all the leaf nodes of
the tree must be maintained in memory at the same time = O(b d)

Completeness: Yes

Optimality: Yes, provided the path cost is a non decreasing function of the
depth of the node

CSE – Panimalar Engineering College 9

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Advantage: Guaranteed to find the single solution at the shallowest depth level

Disadvantage: Suitable for only smallest instances problem (i.e.) (number of

levels to be minimum (or) branching factor to be minimum)
')

Uniform-cost search

function UNIFORM-COST-SEARCH(problem) returns a solution, or failure

node ←a node with STATE = problem.INITIAL-STATE, PATH-COST = 0
frontier ←a priority queue ordered by PATH-COST, with node as the only
element
explored ←an empty set
loop do
if EMPTY?( frontier ) then return failure
node←POP( frontier ) /* chooses the lowest-cost node in frontier */
if problem.GOAL-TEST(node.STATE) then return SOLUTION(node)
add node.STATE to explored
for each action in problem.ACTIONS(node.STATE) do
child ←CHILD-NODE(problem, node, action)
if child.STATE is not in explored or frontier then
frontier ←INSERT(child, frontier )
else if child.STATE is in frontier with higher PATH-COST then
replace that frontier node with child

Breadth-first search is optimal when all step costs are equal, because it always
expands the shallowest unexpanded node. By a simple extension, we can find an
algorithm that is optimal with any step cost function. Instead of expanding the
shallowest node, uniform-cost search expands the node n with the lowest
path cost. Note that if all step costs are equal, this is identical to breadth-first
search.

Uniform-cost search does not care about the number of steps a path has, but
only about their total cost.

Example: Route finding problem

CSE – Panimalar Engineering College 10

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Task : Find a minimum path cost from S to G

Since the value of A is less it is expanded first, but it is not optimal.

B to be expanded next

SBG is the path with minimum path cost.

No need to expand the next path SC, because its path cost is high to reach C
from S, as well as goal state is reached in the previous path with minimum cost.

Time and space complexity:

Time complexity is same as breadth first search because instead of depth level
the minimum path cost is considered.

CSE – Panimalar Engineering College 11

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Time complexity: O(b d) Space complexity: O(b d)

Completeness: Yes Optimality: Yes

Advantage: Guaranteed to find the single solution at minimum path cost.

Disadvantage: Suitable for only smallest instances problem.

Depth-first search

Depth-first search always expands the deepest node in the current fringe of
the search tree

The search proceeds immediately to the deepest level of the search tree, where
the nodes have no successors. As those nodes are expanded, they are dropped
from the fringe, so then the search "backs up" to the next shallowest node that
still has unexplored successors. This strategy can be implemented by TREE-
SEARCH with a last-in-first-out (LIFO) queue, also known as a stack.

Depth first search tree with 3 level expansion

Example: Route finding problem

CSE – Panimalar Engineering College 12

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Task: Find a path from S to G using DFS

The path in the 3rd depth level is selected. (i.e. S-A-D-G

Algorithm:

function DFS(problem) return a solution or failure

TREE-SEARCH(problem, LIFO-QUEUE())

Time and space complexity:

In the worst case depth first search has to expand all the nodes

Time complexity : O(bm).

The nodes are expanded towards one particular direction requires memory for
only that nodes.

Space complexity : O(bm)

CSE – Panimalar Engineering College 13

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

b=2
m = 2 :. bm=4

Completeness: No

Optimality: No

Advantage: If more than one solution exists (or) number of levels is high then
DFS is best because exploration is done only in a small portion of the whole
space.

Disadvantage: Not guaranteed to find a solution

Depth - limited search

1. Definition: A cut off (maximum level of the depth) is introduced in this search
technique to overcome the disadvantage of depth first search. The cutoff value
depends on the number of states.

Example: Route finding problem

The number of states in the given map is 5. So, it is possible to get the goal
state at a maximum depth of 4. Therefore the cutoff value is 4

CSE – Panimalar Engineering College 14

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Task : Find a path from A to E.

A recursive implementation of depth-limited search

function DEPTH-LIMITED-SEARCH(problem, limit) returns a

solution, or failure/cutoff
return RECURSIVE-DLS(MAKE-NODE(INITIAL-STATE [problem]),
problem, limit)

function RECURSIVE-DLS(node, problem, limit) returns a

solution, or failure/cutoff
cutoff-occurred? <- false
if GOAL-TEST[problem](STATE[node]) then return
SOLUTION(node)
else if DEPTH[node] =limit then return cutoff
else for each successor in EXPAND(node, problem) do
result <- RECURSIVE-DLS(successor, problem, limit)
if result = cutoff then cutoff-occurred?<- true
else if result  failure then return result
if cutoff-occurred? then return cutoff else return
failure

Time and space complexity:

CSE – Panimalar Engineering College 15

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

The worst case time complexity is equivalent to BFS and worst case DFS.
Time complexity : O(bl)

The nodes which is expanded in one particular direction above to be stored.

Space complexity : O(bl)

Optimality: No, because not guaranteed to find the shortest solution first in the
search technique.

Completeness : Yes, guaranteed to find the solution if it exists.

Advantage: Cut off level is introduced in the DFS technique

Disadvantage : Not guaranteed to find the optimal solution.

Iterative deepening search

Iterative deepening search

Definition: Iterative deepening search is a strategy that sidesteps the issue of

choosing the best depth limit by trying all possible depth limits.

Example: Route finding problem

Task: Find a path from A to G

Limit = 0

Limit = 1

CSE – Panimalar Engineering College 16

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Limit = 2

Solution: The goal state G can be reached from A in four ways. They are:

1. A – B – D - E – G ------- Limit 4
2. A - B - D - E - G ------- Limit 4
3. A - C - E - G ------- Limit 3
4. A - F - G ------ Limit2

Since it is a iterative deepening search it selects lowest depth limit (i.e.) A-F-G is
selected as the solution path.

The iterative deepening search algorithm :

function ITERATIVE-DEEPENING-SEARCH (problem) returns a

solution, or failure
inputs : problem
for depth <- 0 to  do
result <-DEPTH-LIMITED-SEARCH(problem, depth)
if result  cutoff then return result

Time and space complexity :

Iterative deepening combines the advantage of breadth first search and depth
first search (i.e) expansion of states is done as BFS and memory requirement is
equivalent to DFS.

Time complexity : O(bd)

Space Complexity : O(bd)

Optimality: Yes, because the order of expansion of states is similar to breadth

first search.

CSE – Panimalar Engineering College 17

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Completeness: yes, guaranteed to find the solution if it exists.

Advantage: This method is preferred for large state space and the depth of the
search is not known.

Disadvantage : Many states are expanded multiple times

Example : The state D is expanded twice in limit 2

Bidirectional search

Definition : Bidirectional search is a strategy that simultaneously search both

the directions (i.e.) forward from the initial state and backward from the goal,
and stops when the two searches meet in the middle.

Example: Route finding problem

Task : Find a path from A to E.

Search from forward (A) :

Search from backward (E) :

Time and space complexity:

The forward and backward searches done at the same time will lead to the
solution in O(2bd/2) = O(bd/2)step, because search is done to go only halfway

CSE – Panimalar Engineering College 18

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

If the two searches meet at all, the nodes of at least one of them must all be
retained in memory requires O(bd/2) space.
Optimality: Yes, because the order of expansion of states is done in both the
directions.

Completeness: Yes, guaranteed to find the solution if it exists.

Advantage : Time and space complexity is reduced.

Disadvantage: If two searches (forward, backward) does not meet at all,

complexity arises in the search technique. In backward search calculating
predecessor is difficult task. If more than one goal state 'exists then explicit,
multiple state search is required

Comparing uninformed search strategies

Criterion Breadth Uniform Depth Depth Iterative Bi

First Cost First Limited Deepening direction
Complete Yes Yes No No Yes Yes
Time O(bd) O(bd) O(bm) O(bl) O(bd) O(bd/2)
Space O(bd) O(bd) O(bm) O(bl) O(bd) O(bd/2)
Optimal Yes Yes No No Yes Yes

Avoiding Repeated States

The most important complication of search strategy is expanding states that

have already been encountered and expanded before on some other path

A state space and its exponentially larger search tree

CSE – Panimalar Engineering College 19

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

The repeated states can be avoided using three different ways. They are:

1. Do not return to the state you just came from (i.e) avoid any successor that is
the same state as the node's parent.
2. Do not create path with cycles (i.e) avoid any successor of a node that is the
same as any of the node's ancestors.
3. Do not generate any state that was ever generated before.

The general TREE-SEARCH algorithm is modified with additional data structure,

such as :

Closed list - which stores every expanded node.

Open list - fringe of unexpanded nodes.

If the current node matches a node on the closed list, then it is discarded and it
is not considered for expansion. This is done with GRAPH-SEARCH algorithm.
This algorithm is efficient for problems with many repeated states

function GRAPH-SEARCH (problem, fringe) returns a

solution, or failure
closed <- an empty set
fringe <- INSERT (MAKE-NODE(INITIAL-STATE[problem]),
fringe)
loop do
if EMPTv?(fringe) then return failure
node <- REMOVE-FIKST (fringe)
if GOAL-TEST [problem ](STATE[node]) then return SOLUTION
(node)
if STATE [node] is not in closed then
add STATE [node] to closed
fringe <- INSERT-ALL(EXPAND(node, problem), fringe)

The worst-case time and space requirements are proportional to the size of the
state space, this may be much smaller than O(bd)

Informed search and exploration

Uninformed search strategies can find solutions to problems by systematically

generating new states and testing them against the goal. These strategies are
inefficient in most cases.

CSE – Panimalar Engineering College 20

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

An informed search Strategy uses problem-specific knowledge and it can find

solutions more efficiently.

Informed Heuristic Search Strategies

An informed search strategy uses problem-specific knowledge beyond the

definition of the problem itself and it can find solutions more efficiently than an
uninformed strategy.

The general approach is best first search that uses an evaluation function in
TREE-SEARCH or GRAPH-SEARCH.

Best-first search is an instance of the general TREE-SEARCH or GRAPH-SEARCH

algorithm in which a node is selected for expansion based on an evaluation
function, f(n)

The node with the lowest evaluation is selected for expansion, because the
evaluation measures distance to the goal.

Best-first search can be implemented within our general search framework via a
priority queue, a data structure that will maintain the fringe in ascending order of
f –values

Implementation of Best-first search using general search algorithm

function BEST-FIRST-SEARCH(problem, EVAL-FN) returns a

solution sequence
inputs: problem, a problem
EVAL-FN, an evaluation function
QUEUEING -FN<- a function that orders nodes by EVAL-FN
return TREE-SEARCH(problem, QUEUEING-FN)

The key component of these algorithms is a heuristic functions denoted h(n)

h(n) = estimated cost of the cheapest path from node n to a goal node.

One constraint: if n is a goal node, then h(n) = 0

The two types of evaluation functions are:

(i) Expand the node closest to the goal state using estimated cost as the
evaluation is called greedy best first search.

CSE – Panimalar Engineering College 21

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

(ii) Expand the node on the least cost solution path using estimated cost and
actual cost as the evaluation function is called A*search

Greedy best first search (Minimize estimated cost to reach a goal)

Definition : A best first search that uses h(n) to select next node to expand is
called greedy search
Evaluation function : The estimated cost to reach the goal state, denoted by
the letter h(n)

h(n)= estimated cost of the cheapest path from the state

at node n to a goal state

Algorithm :

Function GREEDY-BEST-FIRST SEARCH (problem) returns a

solution or failure
return BEST-FIRST-SEARCH (problem, h)

Example 1 : Route Finding Problem

Problem : Route finding Problem from Arad to Burcharest

Heuristic function : A good heuristic function for route-finding problems is

Straight-Line Distance to the goal and it is denoted as h SLD(n).

CSE – Panimalar Engineering College 22

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

hSLD(n) = Straight-Line distance between n and the goal

locatation

Note : The values of hSLD(n) cannot be computed from the problem description
itself. Moreover, it takes a certain amount of experience

Values of hSLD-straight-line distances to Bucharest

Solution :

From the given graph and estimated cost, the goal state is identified as B u c h
a r e s t from Arad. Apply the evaluation function h (n) to find a path from Arad
to Burcharest from A to B

CSE – Panimalar Engineering College 23

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

The first node to be expanded from Arad will be Sibiu, because it is closer to
Bucharest than either Zerind or Timisoara.

The next node to be expanded will be Fagaras, because it is closest.

Fagaras in turn generates Bucharest, which is the goal.

For this particular problem, greedy best-first search using h SLD finds a solution
without ever expanding a node that is not on the solution path; hence, its search
cost is minimal. It is not optimal, however: the path via Sibiu and Fagaras to
Bucharest is 32 kilometers longer than the path through Rimnicu Vilcea and
Pitesti. This shows why the algorithm is called "greedy'-at each step it tries to get
as close to the goal as it can.

Minimizing h(n) is susceptible to false starts. Consider the problem of getting

from Iasi to Fagaras. The heuristic suggests that Neamt be expanded first,
because it is closest to Fagaras, but it is a dead end. The solution is to go first to
Vaslui-a step that is actually farther from the goal according to the heuristic-and
then to continue to Urziceni, Bucharest, and Fagaras.

Time and space complexity : Greedy search resembles depth first search,
since it follows one path to the goal state, backtracking occurs when it finds a
dead end. The worst case time complexity is equivalent to depth first search,
that is O(b m), where m is the maximum depth of the search space. The greedy

CSE – Panimalar Engineering College 24

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

search retains all nodes in memory, therefore the space complexity is also O(bm)
The time and space complexity can be reduced with good heuristic function.

Optimality : It is not optimal, because the next level node for expansion is
selected only depends on the estimated cost and not the actual cost.

Completeness : No, because it can start down with an infinite path and never
return to try other possibilities.

Example 2 : Finding the path from one node to another node

Solution :

From the given graph and estimated cost, the goal state IS identified as B from
A.

Apply the evaluation function h(n) to find a path from A to B

CSE – Panimalar Engineering College 25

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

From F, goal state B is reached. Therefore the path from A to Busing greedy
search is A - S - F - B = 450 (i.e) (140 + 99 + 211)

A* search ( Minimizing the total estimated solution cost)

The most widely-known form of best-first search is called A* search (pronounced

"A-star search"). A* search is both complete and optimal.
It evaluates nodes by combining g(n), the cost to reach the node, and h(n.),the
cost to get from the node to the goal

f(n) =g(n) + h(n)

g(n) - path cost from the start node to node n

h(n) - estimated cost of the cheapest path from n to the goal
f (n) - estimated cost of the cheapest solution through n

A* Algorithm

function A* SEARCH(problem) returns a solution or failure

return BEST-FIRST-SEARCH (problem, g+h)

CSE – Panimalar Engineering College 26

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Example 1 : Route Finding Problem

Problem : Route finding Problem from Arad to Burcharest

Heuristic function : A good heuristic function for route-finding problems is

Straight-Line Distance to the goal and it is denoted as h SLD(n).

hSLD(n) = Straight-Line distance between n and the goal locatation

Values of hSLD-straight-line distances to Bucharest

Stages in an A* search for Bucharest. Nodes are labeled with f (n) = g

(n) + h(n)

CSE – Panimalar Engineering College 27

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Example 2 : Finding the path from one node to another node

CSE – Panimalar Engineering College 28

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Solution:

From the given graph and estimated cost, the goal state is identified as B from A
Apply the evaluation function f(n) = g(n) +h(n) to find a path from A to B

CSE – Panimalar Engineering College 29

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

From P, goal state B is reached. Therefore the path from A to B using A* search
is A – S - R - P -B : 418 (ie) {140 + 80 + 97 + 101), that the path cost is less
than Greedy search path cost.

Time and space complexity: Time complexity depends on the heuristic

function and the admissible heuristic value. Space complexity remains in the
exponential order.

The behavior of A* search

Monotonicity (Consistency)

In search tree any path from the root, the f- cost never decreases. This condition
is true for almost all admissible heuristics. A heuristic which satisfies this property
is called monotonicity(consistency).

A heuristic h(n) is consistent if, for every node n and every successor n' of n
generated by any action a, the estimated cost of reaching the goal from n is no

CSE – Panimalar Engineering College 30

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

greater than the step cost of getting to n' plus the estimated cost of reaching the
goal from n':

If the heuristic is non-monotonic, then we have to make a minor correction that

restores monotonicity.

Example for monotonic

Let us consider two nodes n and n’, where n is the parent of n’

For example

g(n) = 3 and h(n) = 4. then f(n) = g(n) + h(n) = 7.

g(n’) = 54 and h(n’) = 3. then f(n’) = g(n’) + h(n’) = 8

Example for Non-monotonic

Let us consider two nodes n and n’, where n is the parent of n’. For example

g(n) = 3 and h(n) = 4. then f(n) = g(n) + h(n) = 7.

g(n’) = 4 and h(n’) = 2. then f(n’) = g(n’) + h(n’) = 6.

To reach the node n the cost value is 7, from there to reach the node n' the
value of cost has to increase as per monotonic property. But the above example
does not satisfy this property. So, it is called as non-monotonic heuristic.

How to avoid non-monotonic heuristic?

We have to check each time when we generate anew node, to see if its f-cost is
less that its parent’s f-cost; if it is we have to use the parent’s f- cost instead.

Non-monotonic heuristic can be avoided using path-max equation.

CSE – Panimalar Engineering College 31

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

f(n') = max (f{n), g(n') + h(n'))

Optimality

A* search is complete, optimal, and optimally efficient among all algorithms

A* using GRAPH-SEARCH is optimal if h(n) is consistent.

Completeness

A* is complete on locally finite graphs (graphs with a finite branching factor)

provided there is some constant d such that every operator costs at least d.

Drawback

A* usually runs out of space because it keeps all generated nodes in memory

Memory bounded heuristic search

The simplest way to reduce memory requirements for A* is to adapt the idea of
iterative deepening to the heuristic search context, resulting in the iterative-
deepening A" (IDA*) algorithm.

The memory requirements of A* is reduced by combining the heuristic function

with iterative deepening resulting an IDA* algorithm.

Iterative Deepening A* search (IDA*)

Depth first search is modified to use an f-cost limit rather than a depth limit for
IDA* algorithm.

Each iteration in the search expands all the nodes inside the contour for the
current f-cost and moves to the next contour with new f - cost.

Space complexity is proportional to the longest path of exploration that is bd is

a good estimate of storage requirements

Time complexity depends on the number of different values that the heuristic
function can take on

Optimality: yes, because it implies A* search.

Completeness: yes, because it implies A* search.

CSE – Panimalar Engineering College 32

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Disadvantage: It will require more storage space in complex domains (i.e) Each
contour will include only one state with the previous contour. To avoid this, we
increase the f-cost
limit by a fixed amount  on each iteration, so that the total number of iteration
is proportional to 1/  . Such an algorithm is called  admissible.

The two recent memory bounded algorithms are:

 Recursive Best First Search (RBfS)

 Memory bounded A* search (MA*)

Recursive Best First Search (RBFS)

A recursive algorithm with best first search technique uses only linear space.
It is similar to recursive depth first search with an inclusion (i.e.) keeps track of
the f-value of the best alternative path available from any ancestor of the current
node.

If the current node exceeds this limit, the recursion unwinds back to the
alternative path and replaces the f-value of each node along the path with the
best f-value of its children.

The main idea lies in to keep track of the second best alternate node (forgotten
node) and decides whether it's worth to reexpand the subtree at some later
time.

CSE – Panimalar Engineering College 33

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Algortihm For Recursive Best-First Search

function RECURSIVE-BEST-FIRST-SEARCH(problem) returns a

solution, or failure
RBFS(problem,MAKE-NODE(INITIAL-STATE[problem]),  )
function, RBFS(problem, node, f_limit) returns a
solution, or failure and a new f-cost limit
if GOAL-TEST[problem](state) then return node
successors <- EXPAND(node, problem)
if successors is empty then return failure, 
for each s in successors do
f[s]<-max(g(s) + h(s),f[node])
repeat
best <- the lowest f-value node in successors
if f[best] > f_limit then return failure,f[best]
alternative <- the second-lowest f-value among successors
result,f[best]<-
RBFS(problem,best,min(f_limit,alternative))
if result  failure then return result

Stages in an RBFS search for the shortest route to Bucharest .

CSE – Panimalar Engineering College 34

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Example:

a) After expanding A, S, and R, the current best leaf(P) has a value that is worse
than the best alternative path (F)

f-limit value of each recursive call is shown on top of each current node. After
expanding R, the condition f[best] >f-limit (417 > 415) is true and returns
f[best] to that node.

b) After unwinding back to and expanding F

Here the f[best] is 450 and which is greater than the f-limit of 417. Therefore if
returns and unwinds with f[best] value to that node.

CSE – Panimalar Engineering College 35

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

c) After switching back to Rand expanding P.

The best alternative path through T costs at least 447, therefore the path
through R and P is considered as the best one.

Time and space complexity : RBFS is an optimal algorithm if the heuristic

function h(n) is admissible. Its time complexity depends both on the accuracy of
the heuristic function and on how often the best path changes as nodes are
expanded. Its space complexity is O(bd), even though more is available.

A search techniques (algorithms) which uses all available memory are:

a. MA* (Memory - bounded A*)

b. SMA* (Simplified MA*)

Simplified Memory - bounded A* search (SMA*)

SMA* algorithm can make use of all available memory to carry out the search.

Properties of SMA* algorithm:

(a) It will utilize whatever memory is made available to it.

(b) It avoids repeated states as far as its memory allows.

It is complete if the available memory is sufficient to store the deepest solution

path.
It is optimal if enough memory is available to store the deepest solution path.
Otherwise, it returns the best solution that can be reached with the available
memory.

Advantage: SMA* uses only the available memory.

CSE – Panimalar Engineering College 36

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Disadvantage: If enough memory is not available it leads to unoptimal

solution.

Space and Time complexity: depends on the available number of node.

The SMA* Algorithm

function SMA*(problem) returns a solution sequence

inputs: problem, a problem
local variables: Queue, a queue of nodes ordered by
f-cost

Queue<-MAKE-QUEUE({MAKE-NODE(INITIAL-STATE[problem])} )
loop do
if Queue is empty then return failure
n<-deepest least-f-cost node in Queue
if GOAL-TEST(n) then return success
s<-NEXT-SUCCESSOR(n)
if s is not a goal and is at maximum depth then
f{s)<- 
else
f{s)<- MAX(f(n), g(s)+h(s))
if all of n’s successors have been generated then
update n’s f-cost and those of its ancestors if necessary
if SUCCESSORS(n) all in the memory then remove n from
Queue
if memory is full then
delete shallowest, highest-f-cost node in Queue
remove it from its parent's successor list
insert its parent on Queue if necessary
insert s on Queue
end

CSE – Panimalar Engineering College 37

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Example:

The values at the nodes are given as per the A* function i.e. g+h=f

From the Figure we identified that the goal states are D,F,J,I because the h value
of these nodes are zero (marked as a square)

Available memory - 3 nodes of storage space.

Task: Find a optimal path from A to anyone of the goal state.

Solution:

CSE – Panimalar Engineering College 38

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

HEURISTIC FUNCTIONS

The 8-puzzle was one of the earliest heuristic search problems.

Given :

CSE – Panimalar Engineering College 39

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Task : Find the shortest solution using heuristic function that never over
estimates the number of steps to the goal.

Solution : To perform the given task two candidates are required, which are
named as h1 and h2

h1 = the number of misplaced tiles.

All of the eight tiles are out of position in the above figure, so the start state
would have hl = 8. hl is an admissible heuristic, because it is clear that any tile
that is out of place must be moved at least once.

h2 = the sum of the distances of the tiles from their goal positions. Because tiles
cannot move along diagonals, the distance we will count is the sum of the
horizontal and vertical distances. This is called as the city block distance or
Manhattan distance. h2 is also admissible, because any move can do is move
one tile one step closer to the goal. Tiles 1 to 8 in the start state give a
Manhattan distance of

h2=3+1+2+2+2+3+3+2=18.

True solution cost is h1 + h2 = 26

Example :

h1=7

h2 = 2 + 3 + 3 + 2 + 4 + 2 + 0 + 2 = 18

True Solution Cost is h1 + h2 = 25

Effective branching factor(b*)

CSE – Panimalar Engineering College 40

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

In the search tree, if the total number of nodes expanded by A* for a particular
problem is N, and the solution depth is d, then b* is the branching factor that a
uniform tree of depth d, would have N nodes. Thus:

N= l+ b* + (b*)2 + (b*)3 + +(b*)d

Example:

For example, if A* finds a solution at depth 5 using 52 nodes, then the effective
branching factor is 1.92.

Depth = 5
N = 52

Effective branching factor is 1.92.

Relaxed problem

A problem with less restriction on the operators is called a relaxed problem. If

the given problem is a relaxed problem then it is possible to produce good
heuristic function. Example: 8 puzzle problem, with minimum number of
operators.

Local Search Algorithms And Optimization Problems

In many optimization problems, the path to the goal is irrelevant; the goal state
itself is the solution.

The best state is identified from the objective function or heuristic cost function.
In such cases, we can use local search algorithms (ie) keep only a single current
state, try to improve it instead of the whole search space explored so far

For example, in the 8-queens problem, what matters is the final configuration of
queens, not the order in which they are added.

Local search algorithms operate a single current state (rather than multiple
paths) and generally move only to neighbors of that state. Typically, the paths
followed by the search are not retained.

They have two key advantages:

CSE – Panimalar Engineering College 41

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

(1) They use very little memory-usually a constant amount; (2) They can often
find reasonable solutions in large or infinite (continuous) state spaces for which
systematic algorithms are unsuitable.

The local search problem is explained with the state space land scape. A
landscape has:

Location - defined by the state

Elevation - defined by the value of the heuristic cost function or objective

function, if elevation corresponds to cost then the lowest valley (global
minimum) is achieved. If elevation corresponds to an objective function, then the
highest peak (global maximum) is achieved.

A complete local search algorithm always finds a goal if one exists, an optimal
algorithm always finds a global minimum/maximum.

A one-dimensional state space landscape in which elevation

corresponds to the objective function.

Applications

Integrated - circuit design

Factory - floor layout
Job-shop scheduling
Automatic programming
Vehicle routing
Telecommunications network Optimization
Advantages

 Constant search space. It is suitable for online and offline search

CSE – Panimalar Engineering College 42

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

 The search cost is less when compare to informed search

 Reasonable solutions are derived in large or continuous state space for
which systematic algorithms are unsuitable.

Some of the local search algorithms are:

1. Hill climbing search (Greedy Local Search)

2. Simulated annealing
3. Local beam search
4. Genetic Algorithm (GA)

Hill Climbing Search (Greedy Local Search)

The hill-climbing search algorithm is simply a loop that continually moves in

the direction of increasing value. It terminates when it reaches a "peak" where
no neighbor has a higher value. The algorithm does not maintain a search tree,
so the current node data structure need only record the state and its objective
function value. At each step the current node is replaced by the best neighbor;

Hill-climbing search algorithm

function HILL-CLIMBING(problem) returns a state that is a

local maximum
inputs: problem, a problem
local variables:current, a node and neighbor, a node
current <- MAKE-NODE(INITIAL-STATE[problem])
loop do
neighbor <- a highest-valued successor of current
if VALUE[neighbor] <= VALUE[current] then
return STATE[current]
current <- neighbor

To illustrate hill-climbing, we will use the 8-queens, where each state has 8
queens on the board, one per column. The successor function returns all possible
states generated by moving a single queen to another square in the same
column (so each state has 8 x 7 = 56 successors).

Hill-climbing algorithms typically choose randomly among the set of best

successors, if there is more than one.

The heuristic cost function h is the number of pairs of queens that are attacking
each other, either directly or indirectly.

CSE – Panimalar Engineering College 43

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

The global minimum of this function is zero, which occurs only at perfect
solutions.

An 8-queens state with heuristic cost estimate h = 17, showing the value of h for
each possible successor obtained by moving a queen within its column. The best
moves are marked.

A local minimum in the 8-queens state space; the state has h = 1 but every
successor has a higher cost.

CSE – Panimalar Engineering College 44

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Hill climbing often gets stuck for the following reasons:

Local maxima or foot hills : a local maximum is a peak that is higher than
each of its neighboring states, but lower than the global maximum

Example :

The evaluation function value is maximum at C and from their there is no path
exist for expansion. Therefore C is called as local maxima. To avoid this state,
random node is selected using back tracking to the previous node.

Plateau or shoulder: a plateau is an area of the state space landscape where

the evaluation function is flat. It can be a flat local maximum.

Example :

The evaluation function value of B C D are same, this is a state space of plateau.
To avoid this state, random node is selected or skip the level (i.e) select the
node in the next level

CSE – Panimalar Engineering College 45

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Ridges: Ridges result in a sequence of local maxima that is very difficult for
greedy algorithms to navigate. But the disadvantage is more calculations to be
done function

Structure of hill climbing drawbacks

Variants of hill-climbing

Stochastic hill climbing - Stochastic hill climbing chooses at random from

among the uphill moves; the probability of selection can vary with the steepness
of the uphill move.

First-choice hill climbing - First-choice hill climbing implements stochastic hill

climbing by generating successors randomly until one is generated that is better
than the current state

Random-restart hill climbing - Random-restart hill climbing adopts the well

known adage, "If at first you don't succeed, try, try again." It conducts a series
of hill-climbing searches from randomly generated initial state, stopping when a
goal is found.

Simulated annealing search

An algorithm which combines hill climbing with random walk to yield both
efficiency and completeness

In metallurgy, annealing is the process used to temper or harden metals and

glass by heating them to a high temperature and then gradually cooling them

When the search stops at the local maxima, we will allow the search to take
some down Hill steps to escape the local maxima by allowing some "bad" moves
but gradually decrease their size and frequency. The node is selected randomly
and it checks whether it is a best move or not. If the move improves the
situation, it is executed.  E variable is introduced to calculate the probability of
worsened. A Second parameter T is introduced to determine the probability.

CSE – Panimalar Engineering College 46

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

The simulated annealing search algorithm

function SIMULATED-ANNEALING(problem, schedule) returns a

solution state
inputs: problem, a problem
schedule, a mapping from time to "temperature"
local variables: current, a node
next, a node
T, a "variable" controlling the probability of downward
steps
current <- MAKE-NODE(INITIAL-STATE[problem])
for t<- l to  do
T <- schedule[t]
if T = 0 then return current
next <- a randomly selected successor of current
 E <- VALUE[next] - VALUE[current]
if  E > 0 then current <- next
else current <- next only with probability e  E/T

Property of simulated annealing search

T decreases slowly enough then simulated annealing search will find a global
optimum with probability approaching one

Applications

VLSI layout
Airline scheduling

Local beam search

Local beam search is a variation of beam search which is a path based algorithm.
It uses K states and generates successors for K states in parallel instead of one
state and its successors in sequence. The useful information is passed among the
K parallel threads.

The sequence of steps to perform local beam search is given below:

CSE – Panimalar Engineering College 47

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

 Keep track of K states rather than just one.

 Start with K randomly generated states.
 At each iteration, all the successors of all K states are generated.
 If anyone is a goal state stop; else select the K best successors from the
complete list and repeat.

This search will suffer from lack of diversity among K states.

Therefore a variant named as stochastic beam search selects K successors at

random, with the probability of choosing a given successor being an increasing
function of its value.

Genetic Algorithms (GA)

A genetic algorithm (or GA) is a variant of stochastic beam search in which

successor states are generated by combining two parent states, rather than by
modifying a single state

GA begins with a set of k randomly generated states, called the population.

Each state, or individual, is represented as a string over a finite alphabet.

For Example an 8 queen’s state could be represented as 8 digits, each in the

range from 1 to 8.

Initial population: K randomly generated states of 8 queen problem

Individual (or) state: Each string in the initial population is individual (or)
state. In one state, the position of the queen of each column is represented.

Example: The state with the value 24748552 is derived as follows:

The Initial Population (Four randomly selected States) are :

CSE – Panimalar Engineering College 48

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Evaluation function (or) Fitness function: A function that returns higher

values for better State. For 8 queens problem the number of non attacking pairs
of queens is defined as fitness function

Minimum fitness value is 0

Maximum fitness value is : 8*7/2 = 28 for a solution

The values of the four states are 24, 23, 20, and 11.

The probability of being chosen for reproduction is directly proportional to the

fitness score, which is denoted as percentage.

24 / (24+23+20+11) = 31%
23 / (24+23+20+11) = 29%
20 / (24+23+20+11) = 26%
11 / (24+23+20+11) = 14%

Selection : A random choice of two pairs is selected for reproduction, by

considering the probability of fitness function of each state. In the example one
state is chosen twice probability of 29%) and the another one state is not
chosen (Probability of 14%)

Cross over: Each pair to be mated, a crossover point is randomly chosen. For
the first pair the crossover point is chosen after 3 digits and after 5 digits for the
second pair.

CSE – Panimalar Engineering College 49

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

offspring : Offspring is created by crossing over the parent strings in the

crossover point. That is, the first child of the first pair gets the first 3 digits from
the first parent and the remaining digits from the second parent. Similarly the
second child of the first pair gets the first 3 digits from the second parent and
the remaining digits from the first parent.

The 8-queens states corresponding to the first two parents

Mutation : Each location is subject to random mutation with a small

independent probability. One digit was mutated in the first, third, and fourth
offspring

CSE – Panimalar Engineering College 50

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Production of Next Generation of States

The initial population in (a) is ranked by the fitness function in (b), resulting in
pairs for mating in (c). They produce offspring in (d), which are subject to
mutation in(e).

function GENETIC-ALGORITHM(population, FITNESS-FN)

returns an individual
inputs: population, a set of individuals
FITNESS-FN, a function that measures the fitness of an
individual
repeat
new-population <- empty set
loop for i from 1 to SIZE(population) do
x <- RANDOM-SELECTION(Population,FITNESS-FN)
y <- RANDOM-SELECTION(Population,FITNESS-FN)
child <- REPRODUCE(yX),
if (small random probability) then child MUTATE(chi1d)
add child to new-population
population <- new-population
until some individual is fit enough, or enough time has
elapsed
return the best individual in population, according to
FITNESS-FN
function REPRODUCE(x,y), returns an individual
inputs: x, y, parent individuals
n <- LENGTH(x)
c <- random number from 1 to n
return APPEND(SUBSTRING(x,1,c),SUBSTRING(y, c + 1, n ))

The sequence of steps to perform GA is summarized below:

 A successor state is generated by combining two parent states.

 Start with K randomly generated states population
 Each state or individual is represented as a string over a finite alphabet
(often a string of O's and 1's)

CSE – Panimalar Engineering College 51

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

 Evaluation function (fitness function) is applied to find better states

with higher values.
 Produce the next generation of states by selection, crossover and
mutation

Local Search In Continuous Spaces

Local Search in continuous space is the one that deals with the real world
problems.

 One way to solve continuous problems is to discretize the

neighborhood of each state.
 Stochastic hill climbing and simulated annealing are applied directly in
the continuous space
 Steepest - ascent hill climbing is applied by updating the formula of
current state

x <- x +   f(x)

 - small constant
 f(x) - magnitude & direction of the steepest slope.
 Empirical gradient, line search, Newton-Raphson method can be
applied in this domain to find the successor state.
 Local search methods in continuous space may also lead to local
maxima, ridges and plateau. This situation is avoided by using random
restart method.

Online Search Agents and Unknown Environments

Online search agent operates by interleaving computation and action, that is first
it takes an action, then it observes the environment and computes the next
action, whereas ,the offline search computes complete solution (problem solving
agent) before executing the problem solution.

online search agents suits well for the following domains.

 Dynamic or Semi dynamic domain

 Stochastic domain

Online search is a necessary idea for an exploration problem, where the states
and actions are unknown to the agent. For example, consider a newborn baby
for exploration problem and the baby's gradual discovery of how the world
works is an online search process

CSE – Panimalar Engineering College 52

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Online search problems

An online search problem can be solved by an agent executing actions rather

than by a computational process. The agent knows the following terms to do the
search in the given environment

 ACTIONS(S) - which returns a list of actions allowed in state s ;

 c(s, a, s’) - The step-cost function known to the agent when it reaches
s’
 GOAL-TEST(S).

Searching With Partial Information

When the knowledge of the states or actions is incomplete about the

environment, then only partial information is known to the agent. This
incompleteness lead to three distinct problem types. They are:
(i) Sensorless problems (conformant problems) : If the agent has no
sensors at all, then it could be in one of several possible initial states, and each
action might
therefore lead to one of possible successor states.

(ii) Contigency problems: If the environment is partially observable or if

actions are uncertain, then the agent's percepts provide new information after
each action. A problem is called adversarial if the uncertainty is caused by the
actions of another agent. To handle the situation of unknown circumstances the
agent needs a contigency plan.

(iii) Exploration problem: It is an extreme case of contigency problems, where

the states and actions of the environment are unknown and the agent must act
to discover
them.

CONSTRAINT SATISFACTION PROBLEMS(CSP)

Constraint satisfaction problems (CSP) are mathematical problems where one

must find states or objects that satisfy a number of constraints or criteria. A
constraint is a restriction of the feasible solutions in an optimization problem.

Some examples for CSP's are:

The n-queens problem

A crossword puzzle
A map coloring problem
The Boolean satisfiability problem

CSE – Panimalar Engineering College 53

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

A cryptarithmetic problem

All these examples and other real life problems like time table scheduling,
transport scheduling, floor planning etc. are instances of the same pattern,

A Constraint Satisfaction Problem(or CSP) is defined by a set of variables

{X1,X2,….Xn,} and a set of constraints {C1,C2,…,Cm}. Each variable Xi has a
nonempty domain D, of possible values. Each constraint Ci involves some
subset of variables and specifies the allowable combinations of values for that
subset.

A State of the problem is defined by an assignment of values to some or all of

the variables,{Xi = vi, Xj = vj,…}. An assignment that does not violate any
constraints is called a consistent or legal assignment.

A complete assignment is one in which every variable is mentioned, and a

solution to a CSP is a complete assignment that satisfies all the constraints.
Some CSPs also require a solution that maximizes an objective function.

Example for Constraint Satisfaction Problem :

The map coloring problem. The task of coloring each region red, green or blue in
such a way that no neighboring regions have the same color.

Map of Australia showing each of its states and territories

We are given the task of coloring each region either red, green, or blue in such a
way that the neighboring regions must not have the same color.

CSE – Panimalar Engineering College 54

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

To formulate this as CSP, we define the variable to be the regions: WA, NT, Q,
NSW, V, SA, and T.

The domain of each variable is the set {red, green, blue}.

The constraints require neighboring regions to have distinct colors: for example,
the allowable combinations for WA and NT are the pairs

{(red,green),(red,blue),(green,red),(green,blue),(blue,red),(blue,green)}.

(The constraint can also be represented as the inequality WA  NT)

There are many possible solutions, such as

{ WA = red, NT = green, Q = red, NSW = green, V = red ,SA = blue,T = red}.

Constraint Graph : A CSP is usually represented as an undirected graph, called

constraint graph where the nodes are the variables and the edges are the binary
constraints.

The map-coloring problem represented as a constraint graph.

CSP can be viewed as a standard search problem as follows :

 Initial state : the empty assignment {},in which all variables are
unassigned.
 Successor function : a value can be assigned to any unassigned
variable, provided that it does not conflict with previously assigned
variables.
 Goal test : the current assignment is complete.
 Path cost : a constant cost(E.g.,1) for every step.

CSE – Panimalar Engineering College 55

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Every solution must be a complete assignment and therefore appears at depth n

if there are n variables. So Depth first search algorithms are popular for CSPs.

Varieties of CSPs

Discrete variables

Discrete variables can have

 Finite Domains
 Infinite domains

Finite domains

The simplest kind of CSP involves variables that are discrete and have finite
domains.

Map coloring problems are of this kind. The 8-queens problem can also be
viewed as finite-domain CSP, where the variables Q 1,Q2,…..Q8 are the positions
each queen in columns 1,….8 and each variable has the domain
{1,2,3,4,5,6,7,8}.

If the maximum domain size of any variable in a CSP is d, then the number of
possible complete assignments is O(d n) – that is, exponential in the number of
variables.

Finite domain CSPs include Boolean CSPs, whose variables can be either true
or false.

Infinite domains

Discrete variables can also have infinite domains – for example, the set of
integers or the set of strings. With infinite domains, it is no longer possible to
describe constraints by enumerating all allowed combination of values. For
example, if Jobl, which takes five days, must precede Jobs, then we would need
a constraint language of algebraic inequalities such as

Startjob1 + 5 <= Startjob 3.

Continuous domains

CSPs with continuous domains are very common in real world. For example, in
operation research field, the scheduling of experiments on the Hubble Telescope
requires very precise timing of observations; the start and finish of each

CSE – Panimalar Engineering College 56

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

observation and maneuver are continuous-valued variables that must obey a

variety of astronomical, precedence and power constraints.

The best known category of continuous-domain CSPs is that of linear

programming problems, where the constraints must be linear inequalities
forming a convex region. Linear programming problems can be solved in time
polynomial in the number of variables.

Varieties of constraints :

Unary constraints – Which restricts a single variable.

Example : SA  green

Binary constraints - relates pairs of variables.

Example : SA  WA

Higher order constraints involve 3 or more variables.

Example : cryptarithmetic puzzles. Each letter stands for a distinct digit

The aim is to find a substitution of digits for letters such that the resulting sum is
arithmetically correct, with the added restriction that no leading zeros are
allowed.

Constraint graph for the cryptarithmetic Problem

Alldiff constraint can be broken down into binary constraints - F  T, F  U, and

so on.
The addition constraints on the four columns of the puzzle also involve several
variables and can be written as

CSE – Panimalar Engineering College 57

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

O + O = R + 10 . X1
X1 + W + W = U + 10 . X2
X1 + T + T = O + 10 . X3
X3 = F

Where X1, X2, and X3 are auxiliary variables representing the digit (0 or 1)
carried over into the next column.

Real World CSP's : Real world problems involve read-valued variables,

 Assignment problems Example : who teaches what class.

 Timetabling Problems Example : Which class is offered when & where?
 Transportation Scheduling
 Factory Scheduling

Backtracking Search for CSPs

The term backtracking search is used for depth-first search that chooses
values for one variable at a time and backtracks when a variable has no legal
values left to assign.

Part of search tree generated by simple backtracking for the map

coloring problem

CSE – Panimalar Engineering College 58

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Improving backtracking efficiency is done with general purpose methods, which

can give huge gains in speed.

 Which variable should be assigned next and what order should be tried?
 What are the implications of the current variable assignments for the other
unassigned variables?
 Can we detect inevitable failure early?

Variable & value ordering: In the backtracking algorithm each unassigned

variable is chosen from minimum Remaining Values (MRV) heuristic, that is
choosing the variable with the fewest legal values. It also has been called the
"most constrained variable" or "fail-first" heuristic.

If the tie occurs among most constrained variables then most constraining
variable is chosen (i.e.) choose the variable with the most constraints on
remaining variable. Once a variable has been selected, choose the least
constraining value that is the one that rules out the fewest values in the
remaining variables.

Propagating information through constraints

So far our search algorithm considers the constraints on a variable only at the
time that the variable is chosen by SELECT-UNASSIGNED-VARIABLE. But by
looking at some of the constraints earlier in the search, or even before the
search has started, we can drastically reduce the search space.

Forward checking

One way to make better use of constraints during search is called forward
checking. Whenever a variable X is assigned, the forward checking process

CSE – Panimalar Engineering College 59

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

looks at each unassigned variable Y that is connected to X by a constraint and

deletes from Y ’s domain any value that is inconsistent with the value chosen for
X.

The progress of a map-coloring search with forward checking.

In forward checking WA = red is assigned first; then forward checking deletes

red from the domains of the neighboring variables NT and SA. After Q = green,
green is deleted from the domains of NT, SA, and NSW. After V = blue, blue is
deleted from the domains of NSW and SA, leaving SA with no legal values. NT
and SA cannot be blue

Constraint propagation

Although forward checking detects many inconsistencies, it does not detect all of
them.

Constraint propagation is the general term for propagating the implications of

a constraint on one variable onto other variables.

Constraint propagation repeatedly enforces constraints locally to detect

inconsistencies. This propagation can be done with different types of consistency
techniques. They are:

Node consistency (one consistency)

Arc consistency (two consistency)
Path consistency (K-consistency)

Node consistency

 Simplest consistency technique

 The node representing a variable V in constraint graph is node consistent if
for every value X in the current domain of V, each unary constraint on V is
satisfied.
 The node inconsistency can be eliminated by simply removing those values
from the domain D of each variable V that do not satisfy unary constraint on
V.

CSE – Panimalar Engineering College 60

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Arc Consistency

The idea of arc consistency provides a fast method of constraint propagation

that is substantially stronger than forward checking. Here, 'arc’ refers to a
directed arc in the constraint graph, such as the arc from SA to NSW. Given the
current domains of SA and NSW, the arc is consistent if, for every value x of SA,
there is some value y of NSW that is consistent with x.

In the constraint graph, binary constraint corresponds to arc. Therefore this type
of consistency is called arc consistency.

Arc (vi, vj) is arc consistent if for every value X the current domain of vi there is
some value Y in the domain of vj such vi =X and vj=Y is permitted by the binary
constraint between vi and vj

Arc-consistency is directional ie if an arc (vi, vj) is consistent than it does not

automatically mean that (vj, vi) is also consistent.

An arc (vi, vj) can be made consistent by simply deleting those values from the
domain of Di for which there is no corresponding value in the domain of D j such
that the binary constraint between Vi and vj is satisfied - It is an earlier detection
of inconstency that is not detected by forward checking method.

The different versions of Arc consistency algorithms are exist such as AC-I,
AC2,AC-3, AC-4, AC-S; AC-6 & AC-7, but frequently used are AC-3 or AC-4.

AC - 3 Algorithm

In this algorithm, queue is used to cheek the inconsistent arcs.

When the queue is not empty do the following steps:

Remove the first arc from the queue and check for consistency.

CSE – Panimalar Engineering College 61

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

If it is inconsistent remove the variable from the domain and add a new
arc to the queue
Repeat the same process until queue is empty

function AC-3( csp) returns the CSP, possibly with

reduced domains
inputs: csp, a binary CSP with variables {X1, X2, . . . ,
Xn}
local variables: queue, a queue of arcs, initially all
the arcs in csp
while queue is not empty do
(Xi, Xj) <- REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUEXS(xi,xj) then
for each Xk in NEIGHBORS[Xj) do
add (Xk , Xi)to queue

function REMOVE-INCONSISTENT-VALUEXS(xi,xj) returns true

iff we remove a value
removed <-false
for each x in DOMAIN[xi] do
if no value y in DOMAIN[xj] allows (x, y) to satisfy the
constraint between Xi and Xj
then delete x from DOMAIN[Xi]removed <- true
return removed

k-Consistency (path Consistency)

A CSP is k-consistent if, for any set of k - 1 variables and for any consistent
assignment to those variables, a consistent value can always be assigned to any
kth variable

1-consistency means that each individual variable by itself is consistent;

this is also called node consistency.
2-consistency is the same as arc consistency.
3-consistency means that any pair of adjacent variables can always be
extended to a third neighboring variable; this is also called path
consistency.

CSE – Panimalar Engineering College 62

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Handling special constraints

Alldiff constraint - All the variables involved must have distinct values.

Example: Crypt arithmetic problem

The inconsistency arises in Alldiff constraint when m>n (i.e.) m variables are
involved in the constraint and n possible distinct values are there. It can be
avoided by selecting the variable in the constraint that has a singleton. domain
and delete the variable's value from the domain of remaining variables, until the
singleton variables are-exist. This simple algorithm will resolve the inconsistency
of the problem.

Resource constraint (Atmost Constraint) - Higher order constraint or

atmost constraint, in which consistency is achieved by deleting the maximum
value of any domain if it is not consistent with minimum values; of the other
domains.

Intelligent backtracking

Chronological backtracking : When a branch of the search fails, back up to

the preceding variable and try a different value for it (i.e.) the most recent
decision point is revisited. This will lead to inconsistency in real world problems
(map coloring problem) that can't be resolved. To overcome the disadvantage of
chronological backtracking, an intelligence backtracking method is proposed.

Conflict directed backtracking: When a branch of the search fails, backtrack

to one of the set of variables that caused the failure-conflict set. The conflict set
for variable X is the set of previously assigned variables that are connected to X
by constraints. A backtracking algorithm that was conflict sets defined in this way
is called conflict directed backtracking

Local Search for CSPs

Local search method is effective in solving CSP's, because complete state

formulation is defined.
Initial state - assigns a value to every variable.
Successor function - works by changing the value of each variable

Advantage : useful for online searching when the problem changes.

Ex : Scheduling problems

The MIN-CONFLICTS algorithm for solving CSPs by local search.

CSE – Panimalar Engineering College 63

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

function MIN-CONFLICTS (CSP, max-steps) returns a

solution or failure
inputs: csp, a constraint satisfaction problem
max-steps, the number of steps allowed before giving up
current <- an initial complete assignment for csp
for i = 1 to max-steps do
if current is a solution for csp then return current
var <- a randomly chosen, conflicted variable from
VARIABLES[CSP]
value <- the value v for var that minimizes
CONFLICTS(var, v, current, csp)
set var = value in current
return failure

A two-step solution for an &-queens problem using min-conflicts. At each stage,

a queen is chosen for reassignment in its column. The number of conflicts (in
this case, the number of attacking queens) is shown in each square.

The structure of problems

The complexity of solving a CSP-is strongly related to the structure of its

constraint graph. If CSP can be divided into independent sub problems, then
each sub problem is solved independently then the solutions are combined.
When the n variables are divided as n/c subproblems, each will take dc work to
solve. Hence the total work is O(dc n/c). If n=lO, c=2 then 5 problems are
reduced and solved in less time.

Completely independent sub problems are rare, in most cases sub problems of a
CSP are connected

The way how to convert the constraint graph of a tree structure CSP into linear
ordering of the variables consistent with the tree is shown in Figure. Any two
variables are connected by atmost one path in the tree structure

CSE – Panimalar Engineering College 64

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

If the constraint graph of a CSP forms a tree structure then it can be solved in
linear time number of variables). The algorithm has the following steps.
1. Choose any variable as the root of the tree and order the variables from the
root to the leaves in such a way that every node's parent in the tree preceeds it
in the ordering label the variables X l .... Xn in order, every variable except the
root has exactly one parent variable.

2. For j from n down 2, apply arc consistency to the arc (Xi , Xj), where Xi is the
parent of Xj removing values from Domain [Xi] as necessary.

3. For j from 1 to n, assign any value for Xj consistent with the value assigned
for Xi, where Xi is the parent of Xj Keypoints of this algorithm are as follows:

Step-(2), CSP is arc consistent so the assignment of values in step (3)

requires no backtracking.
Step-(2), the arc consistency is applied in reverse order to ensure the
consistency of arcs that are processed already.

General constraint graphs can be reduced to trees on two ways. They are:

(a) Removing nodes - Cutset conditioning

(b) Collapsing nodes together - Tree decomposition.

(a) Removing nodes - Cutset conditioning

Assign values to some variables so that the remaining variables form a

tree.

CSE – Panimalar Engineering College 65

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Delete the value assigned variable from the list and from the domains of
the other variables any values that are inconsistent with the value chosen
for the variable.

This works for binary CSP's and not suitable for higher order constraints.

The remaining problem (tree structure) is solved in linear order time

variables.

Example: In the constraint graph of map coloring problem, the region SA is

assigned with a value and it is removed to make the problem in the form of tree
structure, then it is solvable in linear time

The original constraint graph

The constraint graph after the removal of SA

If the value chosen for the variable to be deleted for tree structure is
wrong, then the following algorithm is executed.

(i) Choose a subset S from VARIABLES[CSP] such that the constraint

graph becomes a tree after removal of S-cycle cutset.

(ii) For each variable on S with assignment satisfies all constraints on S.

CSE – Panimalar Engineering College 66

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

* Remove from the deomains of the remaining variables any values that
are inconsistent with the assignment for S.
* If the remaining CSP has a solution, return it with the assignment for S.

(b) Collapsing nodes together-Tree decomposition

Construction of tree decomposition the constraint graph is divided into a

set of subproblems, solved independently and the resulting solutions are
combined.

Works well, when the subproblem is small.

Requirements of this method are:

 Every variable in the base problem should appear in atleast one of

the subproblem.
 If the binary constraint is exist, then the same constraint must
appear in atleast one of the subproblem. .
 If the variable appears in two subproblems in the tree, it must
appear in every subproblem along the path connecting those
subproblems, that is the variable should be assigned with same
value and constraint in every subproblem.

A tree decompositon of the constraint graph

Solution

 If any subproblem has no solution then the entire problem has no

solution.
 If all the sub problems are solvable then a global solution is achieved.

Adversarial Search

CSE – Panimalar Engineering College 67

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Competitive environments, in which the agent’s goals are in conflict, give rise to
adversarial search problems-often known as games.

In our terminology, games means deterministic, fully observable environments in

which there are two agents whose actions must alternate and in which the utility
values at the end of the game are always equal and opposite. For example, if
one player wins a game of chess (+1), the other player necessarily loses (-1).

There are two types of games

1. Perfect Information ( Example : chess, checkers)

2. Imperfect Information ( Example : Bridge, Backgammon)

In game playing to select the next state, search technique is required. Game
playing itself is considered as a type of search problems. But, how to reduce the
search time to make on a move from one state to another state.

The pruning technique allows us to ignore positions of the search tree that
make no difference to the final choice.

Heuristic evaluation function allows us to find the utility (win, loss, and
draw) of a state without doing a complete search.

Optimal Decisions in Games

A Game with two players - Max and Min.

 Max, makes a move first by choosing a high value and take turns moving
until the game is over
 Min, makes a move as a opponent and tries to minimize the max player
score, until the game is over.

At the end of the game (goal state or time), points are awarded to the winner.

The components of game playing are :

Initial state - Which includes the board position and an indication of whose
move and identifies the player to the move.
Successor function - which returns a list of (move, state) pairs, each
indicating a legal move and the resulting state.

Terminal test - Which determines the end state of the game. States where the
game has ended are called terminal states.

CSE – Panimalar Engineering College 68

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Utility function (also called an objective function or payoff function), - which

gives a numeric value for the terminal states. In chess, the outcome is a win,
loss, or draw, with values +1, -1, or 0. Some games have a wider , variety of
possible outcomes; the payoffs in backgammon range from +192 to -192.

The initial state and the legal moves for each side define the game tree for the
game

Example : Tic – Tac – Toe (Noughts and Crosses)

From the initial state, MAX has nine possible moves. Play alternates between
MAX'S placing an X and MIN'S placing an O until we reach leaf nodes
corresponding to ter.mina1 states such that one player has three in a row or all
the squares are filled.

Initial State : Initial Board Position

Successor Function : Max placing X’s in the empty square

Min placing O’s in the empty square

Goal State : We have three different types of goal state, any one to be
reached.
i) If the O’s are placed in one column, one row (or) in the diagonal
continuously then, it is a goal state of min player. (Won by Min Player)
ii) If the X’s are placed in one column, one row (or) in the diagonal
continuously then, it is a goal state of min player. (Won by Max Player)
iii) If the all the nine squares are filled by either X or O and there is no
win condition by Max and Min player then it is a state of Draw between
two players.
Some terminal states

Won by Min Players

X O X
O
O

CSE – Panimalar Engineering College 69

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Won by Max Players

X O X
X
X 0 O

Draw between two Players

X O X
O O X
X O O

Utility function

Win = 1 Draw =0 Loss = -1

A (partial) search tree for the game of tic-tac-toe

Optimal strategies

In a normal search problem, the optimal solution would be a sequence of moves

leading to a goal state-a terminal state that is a win. In a game, an optimal
strategy leads to outcomes at least as good as any other strategy when one is
playing an infallible opponent

Given a game tree, the optimal strategy can be determined by examining the
minimax value of each node, which we write as MINIMAX- VALUE(n).

CSE – Panimalar Engineering College 70

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

MINIMAX- VALUE(n) =

UTILITY(n) if n is a terminal state

Maxs  successors(n) MAX-VALUE(s) if n is a MAX node
Mins  Successors(n) MAX-VALUE(s) if n is a MIN node

Even a simple game like tic-tac-toe is too complex for us to draw the entire game
tree.

The possible moves for MAX at the root node are labeled al, a2, and a3. The
possible replies to a1 for MIN are b1, b2, b3, and so on.
A Two Ply Game Tree

 - Moves by Max Player

 - Moves by Min Player

 The terminal nodes show the utility values for MAX;

 The other nodes are labeled with their minimax values.
 MAX'S best move at the root is al, because it leads to the successor
with the highest minimax value
 MIN'S best reply is bl, because it leads to the successor with the lowest
minimax value.
 The root node is a MAX node; its successors have minimax values 3, 2,
and 2; so it has a minimax value of 3.
 The first MIN node, labeled B, has three successors with values 3, 12,
and 8, so its minimax value is 33

The minimax algorithm

The minimax algorithm computes the minimax decision from the current
state. It uses a simple recursive computation of the minimax values of each
successor state, directly implementing the defining equations. The recursion
proceeds all the way down to the leaves of the tree, and then the minimax
values are backed up through the tree as the recursion unwinds.

CSE – Panimalar Engineering College 71

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

For Example

The algorithm first recurses down to the three bottom left nodes, and uses the
UTILITY function on them to discover that their values are 3, 12, and 8
respectively. Then it takes the minimum of these values, 3, and returns it as the
backed-up value of node B. A similar process gives the backed up values of 2 for
C and 2 for D. Finally, we take the maximum of 3,2, and 2 to get the backed-up
value of 3 for the root node.

An algorithm for minimax decision

function MINIMAX-DECISION (state) returns an action

inputs: state, current state in game
v <- MAX-VALUE(state)
return the action in SUCCESSORS(state) with value v

function MAX-VALUE(state) returns a utility value

if TERMINAL-TEST(state) then return UTILITY(state)
v<- - 
for a, s in SUCCESSORS(state) do
v <- MAX(v, MIN-VALUE(s))
return v

function MIN-VALUE(state) returns a utility value

if TERMINAL-TEST(state) then return UTILITY(state)
V <- 
for a, s in SUCCESSORS(state)do'
v <- MIN(v, MAX-VALUE(s))
return v

 Generate the whole game tree, all the way down to the terminal state.
 Apply the utility function to each terminal state to get its value.

CSE – Panimalar Engineering College 72

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

 Use utility functions of the terminal state one level higher than the current
value to determine Max or Min value.
 Minimax decision maximizes the utility under the assumption that the
opponent will play perfectly to minimize the max player score.

Complexity : If the maximum depth of the tree is m, and there are b legal
moves at each point then the time complexity of the minimax algorithm is O(bm).
This algorithm is a depth first search, therefore the space requirements are linear
in m and b. For real games the calculation of time complexity is impossible,
however this algorithm will be a good basis for game playing.

Completeness : It the tree is finite, then it is complete.

Optimality : It is optimal when played against an optimal opponent

ALPHA - BETA PRUNING

Pruning - The process of eliminating a branch of the search tree from

consideration without examining is called pruning.

The two parameters of pruning technique are:

Alpha (  ) : Best choice for the value of MAX along the path (or) lower bound
on the value that on maximizing node may be ultimately assigned.

Beta (  ) : Best choice for the value of MIN along the path (or) upper bound
on the value that a minimizing node may be ultimately assigned.

Alpha - Beta pruning : The  and  values are applied to a minimax tree, it
returns the same move as minimax, but prunes away branches that cannot
possibly influence the final decision is called Alpha - Beta pruning (or) Cutoff
Consider again the two-ply game tree from

Let the two unevaluated successors of node C have values x and y and let z be
the minimum of x and y. The value of the root node is given by

CSE – Panimalar Engineering College 73

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

MINIMAX-VALUE(ROOT)=max((min(3,12,8),min(2,x,y),min(l4,5,2 ))
= max(3, min(2, x, y), 2)
= max(3, z, 2) where z <=2
= 3.

In other words, the value of the root and hence the minimax decision are
independent of the values of the pruned leaves x and y.

Stages in the calculation of the optimal decision for the game tree

(a) The first leaf below B has the value 3. Hence, B, which is a MIN node, has a
value of at most 3

(b) The second leaf below B has a value of 12; MIN would avoid this move, so
the value of B is still at most 3

c) The third leaf below B has a value of 8; we have seen all B's successors, so
the value of B is exactly 3. Now, we can infer that the value of the root is at least
3, because MAX has a choice worth 3 at the root.

(d) The first leaf below C has the value 2. Hence, C, which is a MIN node, has a
value of at most 2. But we know that B is worth 3, so MAX would never choose
C. Therefore, there is no point in looking at the other successors of C. This is an
example of alpha-beta pruning.

CSE – Panimalar Engineering College 74

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

(e) The first leaf below D has the value 14, so D is worth atmost 14. This is still
higher than MAX'S best alternative (i.e., 3), so we need to keep exploring D's
successors. Notice also that we now have bounds on all of the successors of the
root, so the root's value is also at most 14.

(f) The second successor of D is worth 5, so again we need to keep exploring.

The third successor is worth 2, so now D is worth exactly 2. MAX'S decision at
the root is to move to B, giving a value of 3.

The alpha-beta search algorithm

function ALPHA-BETA-SEARCH (returns an ) action

inputs: state, current state in game
v<- M A X -V A L U E (State,-  , +  )
return the action in SUCCESSORS(state) with value v

CSE – Panimalar Engineering College 75

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

function MAX - VALUE ( state,  ,  ) returns  utility

value
inputs: state, current state in game
 , the value of the best alternative for MAX along the
path to state
 , the value of the best alternative for MIN along the
path to state
if TERMINAL-TEST(state) then return) UTILITY( s t a t e
)
v<- - 
for  , S in S U C C E S S O R S (state) do
v <- MAX(v,MIN-VALUE(S,  ,  ))
if v >=  then return v
 <- MAx(  , v)
return v

function MIN - VALUE ( state,  ,  ) returns  utility

value
inputs: state, current state in game
 , the value of the best alternative for MAX along the
path to state
 , the value of the best alternative for MIN along the
path to state
if TERMINAL-TEST(state) then return) UTILITY( s t a t e
)
v<- + 
for  , S in S U C C E S S O R S (state) do
v <- MIN(v,MAX-VALUE(S,  ,  ))
if v <=  then return v
 <- MIN(  , v)
return v

Effectiveness of Alpha – Beta Pruning

Alpha - Beta pruning algorithm needs to examine only O(bd/2) nodes to pick the
best move, instead of O(bd) with minimax algorithm, that is effective branching
factor is
 instead of b.

CSE – Panimalar Engineering College 76

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Imperfect Real Time Decisions

The minimax algorithm generates the entire game search space, whereas the
alpha-beta algorithm allows us to prune large parts of it. However, alpha-beta
still has to search all the way to terminal states for at least a portion of the
search space. This depth is usually not practical, because moves must be made
in a reasonable amount of time-typically a few minutes at most.

Shannon’s proposed instead that programs should cut off the search earlier and
apply a heuristic evaluation function to states in the search, effectively turning
non terminal nodes into terminal leaves.

 The utility function is replaced by an Evaluation function

 The terminal test is replaced by a Cut-off test

1. Evaluation function

Example: Chess Problem

In chess problem each material (Queen, Pawn, etc) has its own value that is
called as material value. From this depends on the move the evaluation function
is calculated and it is applied to the search tree.

This suggests that the evaluation function should be specified by the rules of
probability.

For example If player A has a 100% chance of winning then its evaluation
function is 1.0 and if player A has a 50% chance of winning, 25% of losing and
25% of being a draw then the probability is calculated as; 1x0.50 -lxO.25 +
OxO.25 = 0.25.

As per this example is concerned player A is rated higher than player B. The
material value, of each piece can be calculated independently with-out
considering other pieces in the board is also called as one kind of evaluation
function and it is named as weighted linear function. It can be expressed as

Eval(s) = w1f1(s) + w2f2(s) + w3f3(s)..... + wnfn (s)

w - Weights of the pieces (1 for Pawn, 3 for Bishop etc)
f - A numeric value which represents the numbers of each kind of piece on the
board.

CSE – Panimalar Engineering College 77

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

2. Cut - off test

To perform a cut-off test, an evaluation function, should be applied to positions

that are quiescent, that is a position that will not swing in bad value for long time
in the search tree is known as waiting for quiescence.

Quiescence search - A search which is restricted to consider only certain types

of moves, such as capture moves, that will quickly resolve the uncertainties in
the position.

Horizon problem - When the program is facing a move by the opponent that
causes serious damage and is ultimately unavoidable

Example:
1. Beginning of the search - one ply

2. This diagram shows the situation of horizon problem that is when one level is
generated from B, it causes bad value for B

3. When one more successor level is generated from E and F and situation
comes down and the value of B is retained as a good move. The time B is waited
for this situation is called waiting for quiescence.

CSE – Panimalar Engineering College 78

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

Games That Include An Element Of Chance

Backgammon Game

Backgammon is a typical game that combines luck and skill. Dice are rolled at
the beginning of a player's turn to determine the legal moves.

Goal State

The goal of the game is to move all one's pieces off the board. White moves
clockwise toward 25, and black moves counterclockwise toward 0

Successor Function or Operator

Move to any position except where two or more of the opponent pieces.
If it moves to a position with one opponent piece it is captured and again it has
to start from Beginning

Task : In the position shown, White has rolled 6-5. So Find out the legal moves
for the set of the dice thrown as 6 and 5.

Solution :

There are Four legal moves. They are

 (5-11,5-10)
 (5-11, 19-24)

CSE – Panimalar Engineering College 79

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

 (10-16,5-10)
 (5-11,11-16)

A game tree in backgammon must include chance nodes in addition to MAX

and MIN nodes. Chance nodes are shown as circles. The branches leading from
each chance node denote the possible dice rolls, and each is labeled with the roll
and the chance that it will occur. There are 36 ways to roll two dice, each equally
likely; but because a 6-5 is the same as a 5-6, there are only 21 distinct rolls.
The six doubles (1-1 through 6-6) have a 1/36 chance of coming up, the other
15 distinct rolls a 1/18 chance each.

The resulting positions do not have definite minimax values. Instead, we have to
only calculate the expected value, where the expectation is taken over all the
possible dice rolls that could occur.

Terminal nodes and MAX and MIN nodes (for which the dice roll is known) work
exactly the same way as before; chance nodes are evaluated by taking the
weighted average of the values resulting from all possible dice rolls, that is,

EXPECTIMINIMAX(n)=

UTILITY( n ) if n is a terminal state

Max s  successors(n) EXPECTIMINIMAX(S) if n is a MAX node
Min s  successors(n) EXPECTIMINIMAX(S) if n is a MIN node
 s  successors(n) P(s).EXPECTIMINIMAX(S) if n is a chance node

CSE – Panimalar Engineering College 80

Download Useful Materials @ www.rejinpaul.com

 www.rejinpaul.com
CS8691 – Artificial Intelligence

where the successor function for a chance node n simply augments the state of
n with each possible dice roll to produce each successor s and P(s) is the
probability that that dice roll occurs.

Card games

Card games are interesting for many reasons besides their connection with
gambling.

Imagine two players, MAX and MIN, playing some practice hands of four-card
two handed bridge with all the cards showing.

The hands are as follows, with MAX to play first:

MAX : 6, 6, 9, 8

MIN : 2, 4, 10 , 5

Suppose that MAX leads wiht 9. MIN must now follow suit, playing either with
10 or 5 . MIN plays with 10 and wins the trick.

MIN goes next turn leads the with 2. MAX has no spades (and so cannot win
the trick) and therefore must throw away some card. The obvious choice is the
6 because the other two remaining cards are winners.

Now, whichever card MIN leads for the next trick, MAX will win both remaining
tricks and the game will be tied at two tricks each.

CSE – Panimalar Engineering College 81

Download Useful Materials @ www.rejinpaul.com