CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

UNIT II

PROBLEM SOLVING METHODS

Problem solving Methods - Search Strategies- Uninformed - Informed -

Heuristics - Local Search Algorithms and Optimization Problems - Searching
with Partial Observations – Constraint Satisfaction Problems – Constraint
Propagation - Backtracking Search - Game Playing – Optimal Decisions in
Games – Alpha - Beta Pruning - Stochastic Games

PROBLEM-SOLVING AGENTS
One kind of goal-based agent is called a problem-solving agent. Problem-solving agents use atomic
representations, where the states of the world are considered as wholes, with no internal structure
visible to the problem solving algorithms.

Steps performed by Problem-solving agent

Step 1: Goal Formulation: It is the first and simplest step in problem-solving. It organizes
the steps/sequence required to formulate one goal out of multiple goals as well as actions to
achieve that goal. Goal formulation is based on the current situation and the agent's
performance measure (discussed below).

Step 2: Problem Formulation: It is the most important step of problem-solving which

decides what actions should be taken to achieve the formulated goal. There are following five
components involved in problem formulation:

• Initial State: It is the starting state or initial step of the agent towards its goal.
• Actions: It is the description of the possible actions available to the agent.
• Transition Model: It describes what each action does.
• Goal Test: It determines if the given state is a goal state.
• Path cost: It assigns a numeric cost to each path that follows the goal. The
problem-solving agent selects a cost function, which reflects its performance measure.
Remember, an optimal solution has the lowest path cost among all the solutions.

Note: Initial state, actions, and transition model together define the state-space of the
problem implicitly. State-space of a problem is a set of all states which can be reached from
the initial state followed by any sequence of actions. The state-space forms a directed map or
graph where nodes are the states, links between the nodes are actions, and the path is a
sequence of states connected by the sequence of actions.

Step 3: Search: It identifies all the best possible sequence of actions to reach the goal state
from the current state. It takes a problem as an input and returns solution as its
output.
Step 4: Solution: It finds the best algorithm out of various algorithms, which may be proven
as the best optimal solution.
Step 5: Execution: It executes the best optimal solution from the searching algorithms to

1
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

reach the goal state from the current state.

WELL DEFINED PROBLEMS AND SOLUTIONS

Example Problems

Basically, there are two types of problem approaches:

1. Toy Problem: It can also be called a puzzle-like problem which can be used as a
way to explain a more general problem-solving technique. It is a concise and
exact description of the problem which is used by the researchers to compare the
performance of algorithms. Vacuum cleaner world, 8 Puzzle Problem, Eight queens
puzzle are some examples of Toy problems in AI.

2. Real-world Problem: It is real-world based problems which require solutions. Unlike a

toy problem, it does not depend on descriptions, but we can have a general formulation
2
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

of the problem. Route-Finding, Touring Problems, TSP(Travelling Salesperson

Problem), VLSI layout, Robot navigation, Automatic assembly sequencing, Internet
Searching, Protein Design are some examples of real-world problems in AI.

Some Toy Problems

EXAMPLE 1: Vacuum cleaner world

The first example we examine is the vacuum world. This can be formulated as a problem as follows:

States: The state is determined by both the agent location and the dirt locations. The agent is in one of

two locations, each of which might or might not contain dirt. Thus, there are 2 × 2 2= 8

possible world states. A larger environment with n locations has n ・ 2n states.

Initial state: Any state can be designated as the initial state.

Actions: In this simple environment, each state has just three actions: Left, Right, and Suck. Larger

environments might also include Up and Down.

Transition model: The actions have their expected effects, except that moving Left in the leftmost

square, moving Right in the rightmost square, and Sucking in a clean square have no effect.

Goal test: This checks whether all the squares are clean.

Path cost: Each step costs 1, so the path cost is the number of steps in the path.

3
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

EXAMPLE 2 - 8 Puzzle Problem

 Here, we have a 3x3 matrix with movable tiles numbered from 1 to 8 with a blank
space. The tile adjacent to the blank space can slide into that space. The objective is to
reach a specified goal state similar to the goal state, as shown in the below figure.
 In the figure, our task is to convert the current state into goal state by sliding digits
into the blank space.

The standard formulation is as follows:

States: A state description specifies the location of each of the eight tiles and the blank in one of the

nine squares.

Initial state: Any state can be designated as the initial state. Note that any given goal can be reached

from exactly half of the possible initial states (Exercise 3.4).

Actions: The simplest formulation defines the actions as movements of the blank space Left, Right,

Up, or Down. Different subsets of these are possible depending on where the blank is.

Transition model: Given a state and action, this returns the resulting state; for example, if we apply

Left to the start state in Figure 3.4, the resulting state has the 5 and the blank switched.

Goal test: This checks whether the state matches the goal configuration shown in Figure 3.4.

Path cost: Each step costs 1, so the path cost is the number of steps in the path.

4
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

EXAMPLE 3: Eight queens puzzle

The problem here is to place eight chess queens on an 8*8 chessboard in a way that
no queens will threaten the other. If two queens will come in the same row,
column, or diagonal one will attack another.

Consider the Incremental formulation for this problem: It starts from an empty
state and the operator expands a queen at each step.
Following are the steps involved in this formulation:

 States: Arranging 0 to 8 queens on the chessboard.

 Initial State: An empty chessboard

 Actions: Adding a queen to any of the empty boxes on the chessboard.

 Transition model: Returns the new state of the chessboard with the queen
added in a box.

 Goal test: Checks whether 8-queens are kept on the chessboard without any
possibility of attack.

 Path cost: Path costs are neglected because only final states are counted.

5
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Real – world problems

6
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Problem Solving Methods

1. Solving problem by Searching
 Uninformed Search Strategies
 Informed ( Heuristic) Search Strategies

Terminologies in Search Algorithms

 Search: A step-by-step procedure to solve a given search problem. Three
main factors of a search problem:
o Search Space: The set of possible solutions a system may have.
o Start state: The state where the agent starts the search.
o Goal test: A function that monitors the current state and returns
whether the goal is achieved or not.
 Search Tree: The representation of the search problem in the form of a tree.
The root of the tree is the initial state of the problem.
 Actions: The description of all actions to the agent.
 Transition model: Used to represent the description of the action.
 Solution: Sequence of actions that leads from the initial node to the final
node.
 Path Cost: A function that assigns a numeric cost to each path.
 Optimal Solution: The solution with the lowest cost.
 Problem Space: The environment in which the search takes place.
 Depth of a problem: Length of the shortest path from the initial state to the
final state.

7
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

The general TREE-SEARCH algorithm is shown informally in Figure

Properties of Search Algorithms / Measuring Problem-Solving

Performance
There are four essential properties for search algorithms to compare the
efficiencies.

UNINFORMED SEARCH / BRUTE-FORCE / BLIND SEARCH

ALGORITHMS

The uninformed search algorithms:

` It means that the strategies have no additional information about states beyond that
provided in the problem definition. It only knows how to traverse and how to distinguish
between a leaf node and goal node and hence it is operated in a brute-force way and so it is
also called brute-force algorithms. It searches every node without any prior knowledge and
is hence also called blind search algorithms.

Uninformed search algorithms can be divided into six main types:

8
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

1. BREADTH-FIRST SEARCH:
It is the most common searching strategy to traverse a tree or a graph. It uses a breadthwise
searching procedure and hence it is called breadth-first search. The process starts from the
root node of the tree and expands to all the successor nodes at the first level before moving to
the next level nodes. The queue data structure that works on the First In First Out (FIFO)
concept is used to implement this method. As it returns a solution (if it exists) it can be said
as a complete algorithm.

The searching procedure starts from the root node A. Here the goal node is G. from node A it
will traverse in an order of A-B-C-D-G. it will traverse level-wise.

9
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Advantages Disadvantages

 Provides a solution if any exist.  Increased memory requirement since

each level of the tree must be saved
 A minimal solution with the least in the memory to expand the next.
number of steps is provided when
there exists more than one solution  Time requirement is more if the
for a given problem. solution is far from the root node.

2. DEPTH-FIRST SEARCH:
It is a process similar to that of BFS. The searching procedure starts from the root node and
follows each path to the greatest depth and then backtracks before entering the next path. The
stack data structure that works on the concept of Last In First Out (LIFO) is used to
implement the procedure.
Example:

10
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

The searching procedure starts from the root node A. Here the goal node is G. From node A it
will traverse in an order of A-B-D-G. it will traverse depth-wise.

Advantage Disadvantage

 Less space requirement because it  The algorithm may go in an infinite

stores nodes linearly. loop.
 The time requirement is less than  Possibility of re-occurrence of states.
BFS.

3. DEPTH-LIMITED SEARCH:
It is much similar to a depth-first search with a predetermined limit. The main disadvantage
of DFS is the infinite path, this can be solved with a Depth-limited search. Here the node at
the limit is treated like it has no successor nodes further. It can be called an extended and
refined version of DFS.

Depth-limited search will terminate with two conditions.

 Standard failure value: Indicates that the given problem doesn’t have any solution

 Cutoff failure value: indicates that there is no solution for the given problem within
the given limit.

Example :

11
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

The searching starts from node A. The limit for the procedure is set as l=2. Our goal state is
node G. The process is like A-B-D-E-C-F-G.

Advantage Disadvantage

 Memory efficient  Incompleteness

 Not optimal for problems with more
than one solution.

4. UNIFORM-COST SEARCH ALGORITHM:

It is used to traverse a weighted tree or graph. It is different from both DFS and BFS. Here
the cost is considered as the factor. There is more than one path to reach the goal. The path
with the least cost is selected. For traversing the path an increased order of cost is selected. If
the cost is the same for each transition then it can be said to be similar to that of BFS.

12
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Example.

If the starting node is A and the goal to reach is G, then it will traverse A-C-G. The cost will
be 3

Advantage Disadvantage

13
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

 Optimal because the least cost path is  May be stuck in an infinite loop
chosen. because the path cost is only
considered.

5. ITERATIVE DEEPENING DEPTH-FIRST SEARCH:

It is a combination of both DFS and BFS. It will find the best depth limit and the limit is
gradually increased until a goal is found. The algorithm performs a depth-first search to a
certain depth limit and this depth limit is increased after each iteration until the goal is
reached. The fastness of BFS and the memory efficiency of DFS are combined within this
iterative deepening depth-first search. With large search space and unknown depth of the goal
node, this can be more useful.

Example:

1’st Iteration------A

2’nd Iteration------A,B,C

3’rd Iteration------A,B,D,E,C,F,G

4’th Iteration------A,B,D,H,I,E,C,F,G

The goal node is found at the 4th iteration.

14
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Advantage Disadvantage

 Fast search  Previous works are repeated.

 Memory efficiency

6. BIDIRECTIONAL SEARCH ALGORITHM:

It performs two searches simultaneously. One from the starting end is called forward-search
and the other from the end is called backward search. One single graph is replaced with two
small subgraphs one from the initial vertex and the other from the final vertex. The searching
procedure stops when these two graphs intersect each other. It can use search techniques such
as BFS, DFS, DLS, etc.

Example:

The algorithm terminates at node I where the two sub-graphs meet.

Advantage Disadvantage

15
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

 Fast  Implementation is difficult

 Less memory  The goal state must be known.

Comparing uninformed search strategies

HEURISTIC SEARCH STRATEGIES / INFORMED SEARCH ALGORITHM

Definition:

An informed search strategy uses problem-specific knowledge beyond the definition of the
problem itself and it can find solutions more efficiently than an uninformed strategy.

Types:
– Greedy best-first search
– A* search

(i)Expand the node closest to the goal state using estimated cost as the evaluation function is
called greedy best first search.

(ii) Expand the node on the least cost solution path using estimated cost and actual cost as the
evaluation function is called A*search

Greedy Best-First Search

Greedy best-first search algorithm always selects the path which appears best at that moment.
It is the combination of depth-first search and breadth-first search algorithms. It uses the
heuristic function and search. Best-first search allows us to take the advantages of both

16
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

algorithms. With the help of best-first search, at each step, we can choose the most promising
node. In the best first search algorithm, we expand the node which is closest to the goal node
and the closest cost is estimated by heuristic function, i.e.

f(n)= g(n)
Where, h(n)= estimated cost from node n to the goal.
The greedy best first algorithm is implemented by the priority queue. We use the straight-line
distance heuristic:
hSLD(n) = straight line distance from node n to goal node as evaluation function.

Properties of greedy best-first search

• Complete? No – can get stuck in loops, e.g., Iasi -> Neamt -> Iasi -> Neamt
• Time? O(b m), (in worst case) but a good heuristic can give dramatic improvement (m is
max depth of search space).
m
• Space? O(b ) -- keeps all nodes in memory.
• Optimal? No (not guaranteed to render lowest cost solution).

Advantages:
 Best first search can switch between BFS and DFS by gaining the advantages of both
the algorithms.
 This algorithm is more efficient than BFS and DFS algorithms.
Disadvantages:
 It can behave as an unguided depth-first search in the worst case scenario.
 It can get stuck in a loop as DFS.
 This algorithm is not optimal.

17
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Example 1 : Finding the path from one node to another node

Heuristic function : A good heuristic function for Path/ Route-finding problems is Straight-
Line Distance to the goal and it is denoted as hSLD(n).

Solution :
From the given graph and estimated cost, the goal state is identified as B from A. Apply the
evaluation function h(n) to find a path from A to B

18
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

From F, goal state B is reached. Therefore the path from A to B using greedy search is A - S -
F - B = 450 (i.e) (140 + 99 + 211)

19
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Example 2 : Route Finding Problem

Find a path from Arad to Bucharest

Heuristic function : A good heuristic function for route-finding problems is Straight-Line

Distance to the goal and it is denoted as hSLD(n).

Solution :
From the given graph and estimated cost, the goal state is identified as B u c h a r e s t from
Arad. Apply the evaluation function h (n) to find a path from Arad to Bucharest.

20
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

21
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

22
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

23
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

The first node to be expanded from Arad will be Sibiu, because it is closer to Bucharest than
either Zerind or Timisoara.
The next node to be expanded will be Fagaras, because it is closest. Fagaras in turn generates
Bucharest, which is the goal.

For this particular problem, greedy best-first search using hSLD finds a solution without ever
expanding a node that is not on the solution path; hence, its search cost is minimal. It is not
optimal, however: the path via Sibiu and Fagaras to Bucharest is 32 kilometers longer than
the path through Rimnicu Vilcea and Pitesti. This shows why the algorithm is called
"greedy'-at each step it tries to get as close to the goal as it can.

24
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

A* SEARCH

( MINIMIZING THE TOTAL ESTIMATED SOLUTION COST)

A* search is the most commonly known form of best-first search. It uses heuristic function
h(n), and cost to reach the node n from the start state g(n). A* search algorithm finds the
shortest path through the search space using the heuristic function. This search algorithm
expands less search tree and provides optimal result faster. A* algorithm uses g(n)+h(n)
instead of g(n).

Heuristic function : A good heuristic function for route-finding problems is Straight-Line

Distance to the goal and it is denoted as hSLD(n).

Properties of A*
• Complete: Yes (unless there are infinitely many nodes with f ≤ f(G) ).
• Time: Exponential.
• Space: Keeps all nodes in memory, so also exponential.
• Optimal: Yes (provided h admissible or consistent).
• Optimally Efficient: Yes (no algorithm with the same heuristic is guaranteed to expand
fewer nodes).
Advantages:
o A* search algorithm is the best algorithm than other search algorithms.
o A* search algorithm is optimal and complete.
o This algorithm can solve very complex problems.

Disadvantages:
o It does not always produce the shortest path as it mostly based on heuristics and
approximation.

25
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

o A* search algorithm has some complexity issues.

o The main drawback of A* is memory requirement as it keeps all generated nodes in
the memory, so it is not practical for various large-scale problems.

Example 1 : Finding the path from one node A to another node B

Solution:
From the given graph and estimated cost, the goal state is identified as B from A. Apply the
evaluation function to find a path from A to B.

f(n) = g(n) +h(n)

26
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

27
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

From P, goal state B is reached. Therefore the path from A to B using A* search is A – S - R
- P -B : 418 (ie) {140 + 80 + 97 + 101), that the path cost is less than Greedy search path cost.
Time and space complexity: Time complexity depends on the heuristic function and the
admissible heuristic value. Space complexity remains in the exponential order.

28
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Example 2 : Route Finding Problem

Find a path from Arad to Bucharest

29
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

30
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

31
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

32
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

33
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

34
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

35
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

HEURISTIC FUNCTIONS

Heuristic Functions:

An informed search makes use of heuristic functions in order to reach the goal node in a more
prominent way. Therefore, there are several pathways in a search tree to reach the goal node
from the current node. A good heuristic function is determined by its efficiency. More is the
information about the problem, more is the processing time.

Some toy problems, such as 8-puzzle, 8-queen, tic-tac-toe, etc., can be solved more
efficiently with the help of a heuristic function. Let’s see how:

Consider the following 8-puzzle problem where we have a start state and a goal state. Our
task is to slide the tiles of the current/start state and place it in an order followed in the goal
state. There can be four moves either left, right, up, or down. There can be several ways to
convert the current/start state to the goal state, but, we can use a heuristic function h(n) to
solve the problem more efficiently.

EXAMPLE 1:

Task : Find the shortest solution using heuristic function that never over estimates the
number of steps to the goal.

Solution : To perform the given task two candidates are required, which are named as h1 and
h2

h1 = the number of misplaced tiles.

All of the eight tiles are out of position in the above figure, so the start state would have

hl = 8.

hl is an admissible heuristic, because it is clear that any tile that is out of place must be moved
at least once.

36
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

h2 = the sum of the distances of the tiles from their goal positions.

Because tiles cannot move along diagonals, the distance we will count is the sum of the
horizontal and vertical distances. This is called as the city block distance or Manhattan
distance. h2 is also admissible, because any move can do is move one tile one step closer to
the goal. Tiles 1 to 8 in the start state give a Manhattan distance of

h2=3+1+2+2+2+3+3+2=18.

True solution cost is h1 + h2 = 26

EXAMPLE 2

h1=7

h2 = 2 + 3 + 3 + 2 + 4 + 2 + 0 + 2 = 18

True Solution Cost is h1 + h2 = 25

37
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

EXAMPLE 3

A heuristic function for the 8-puzzle problem is defined below:

h(n)=Number of tiles out of position.

So, there is total of three tiles out of position i.e., 6,5 and 4. Do not count the empty tile
present in the goal state.

h(n)=3. Now, we require to minimize the value of h(n) =0.

Construct a state-space tree to minimize the h(n) value to 0, as shown below:

38
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

It is seen from the above state space tree that the goal state is minimized from h(n)=3 to
h(n)=0. However, we can create and use several heuristic functions as per the requirement. It
is also clear from the above example that a heuristic function h(n) can be defined as the
information required to solve a given problem more efficiently. The information can be
related to the nature of the state, cost of transforming from one state to another, goal
node characteristics, etc., which is expressed as a heuristic function.

39
 CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

LOCAL SEARCH AND OPTIMIZATION PROBLEMS

Local search algorithms keep track of a single current state and generally move only to
neighbors of that state.

In addition to finding goals, local search algorithms are useful for solving pure optimization
problems, in which the aim is to find the best state according to an objective function.

For example, in the 8-queens problem, what matters is the final configuration of queens, not
the order in which they are added.

The local search problem is explained with the state space land scape. A landscape has:

Some of the local search algorithms are:

1. Hill climbing search (Greedy Local Search)

2. Simulated annealing
3. Local beam search
4. Genetic Algorithm (GA)

1. HILL CLIMBING SEARCH (GREEDY LOCAL SEARCH)

o Hill climbing algorithm is a local search algorithm which continuously moves in the
direction of increasing elevation/value to find the peak of the mountain or best solution to

40
 CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

the problem. It terminates when it reaches a peak value where no neighbor has a higher
value.
o Hill climbing algorithm is a technique which is used for optimizing the mathematical
problems. One of the widely discussed examples of Hill climbing algorithm is Traveling-
salesman Problem in which we need to minimize the distance traveled by the salesman.
o It is also called greedy local search as it only looks to its good immediate neighbor state and
not beyond that.
o A node of hill climbing algorithm has two components which are state and value.
o Hill Climbing is mostly used when a good heuristic is available.
o In this algorithm, we don't need to maintain and handle the search tree or graph as it only
keeps a single current state.

Features of Hill Climbing:

o Greedy approach: Hill-climbing algorithm search moves in the direction which
optimizes the cost.
o No backtracking: It does not backtrack the search space, as it does not remember the
previous states.

State-space Diagram for Hill Climbing:

41
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Different regions in the state space landscape:

Local Maximum: Local maximum is a state which is better than its neighbor states, but there
is also another state which is higher than it.

Global Maximum: Global maximum is the best possible state of state space landscape. It has
the highest value of objective function.

Current state: It is a state in a landscape diagram where an agent is currently present.

Flat local maximum: It is a flat space in the landscape where all the neighbor states of
current states have the same value.

Shoulder: It is a plateau region which has an uphill edge.

Types of Hill Climbing

A. Simple Hill climbing (examine neighbor node – one by one)

B. Steepest-Ascent Hill climbing (examine neighbor node – all)

C. Stochastic hill climbing (examine neighbor node – random)

A. SIMPLE HILL CLIMBING:

It examines the neighboring nodes one by one and selects the first neighboring node which
optimizes the current cost as the next node.

42
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Algorithm for Simple Hill climbing :

Step 1 : Evaluate the initial state. If it is a goal state then stop and return success. Otherwise,
make initial state as current state.
Step 2 : Loop until the solution state is found or there are no new operators present which can
be applied to the current state.
a) Select a state that has not been yet applied to the current state and apply it to produce a
new state.
b) Perform these to evaluate new state
i. If the current state is a goal state, then stop and return success.
ii. If it is better than the current state, then make it current state and proceed further.
iii. If it is not better than the current state, then continue in the loop until a solution is found.
Step 3 : Exit.

B. STEEPEST-ASCENT HILL CLIMBING

It first examines all the neighboring nodes and then selects the node closest to the solution
state as the next node.

Algorithm for Steepest Ascent Hill climbing :

Step 1 : Evaluate the initial state. If it is a goal state then stop and return success. Otherwise,
make initial state as current state.
Step 2 : Repeat these steps until a solution is found or current state does not change
a) Select a state that has not been yet applied to the current state.
b) Initialize a new ‘best state’ equal to current state and apply it to produce a new
state.
c) Perform these to evaluate new state
i. If the current state is a goal state, then stop and return success.
ii. If it is better than best state, then make it best state else continue loop with
another new state.
d) Make best state as current state and go to Step 2: b) part.
Step 3 : Exit

43
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

C. STOCHASTIC HILL CLIMBING

It does not examine all the neighboring nodes before deciding which node to select. It just
selects a neighboring node at random and decides (based on the amount of improvement in
that neighbor) whether to move to that neighbor or to examine another.

Algorithm for Stochastic Hill Climbing

Step 1: Evaluate the initial state. If it is a goal state then stop and return success. Otherwise,
make the initial state the current state.
Step 2: Repeat these steps until a solution is found or the current state does not change.
a) Select a state that has not been yet applied to the current state.
b) Apply successor function to the current state and generate all the neighbor states.
c) Among the generated neighbor states which are better than the current state choose a
state randomly (or based on some probability function).
d) If the chosen state is the goal state, then return success, else make it the current state
and repeat step 2: b) part.
Step 3: Exit.

Hill-climbing search algorithm

44
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Problems in Hill Climbing Algorithm:

1. Local Maximum: A local maximum is a peak state in the landscape which is better than
each of its neighboring states, but there is another state also present which is higher than the
local maximum.

Solution: Backtracking technique can be a solution of the local maximum in state

space landscape. Create a list of the promising path so that the algorithm can
backtrack the search space and explore other paths as well.

2. Plateau: A plateau is the flat area of the search space in which all the neighbor states of
the current state contains the same value, because of this algorithm does not find any best
direction to move. A hill-climbing search might be lost in the plateau area.

Solution: The solution for the plateau is to take big steps or very little steps while
searching, to solve the problem. Randomly select a state which is far away from the
current state so it is possible that the algorithm could find non-plateau region.

3. Ridges: A ridge is a special form of the local maximum. It has an area which is higher than
its surrounding areas, but itself has a slope, and cannot be reached in a single move.

Solution: With the use of bidirectional search, or by moving in different directions,

we can improve this problem.

45
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Applications of hill climbing algorithm

1. Travelling salesman problem (also called the travelling salesperson problem or TSP
2. Marketing
3. Robotics
4. Job Scheduling

2. SIMULATED ANNEALING SEARCH

 An algorithm which combines hill climbing with random walk to yield both efficiency
and completeness.
 In metallurgy, annealing is the process used to temper or harden metals and glass by
heating them to a high temperature and then gradually cooling them.
 When the search stops at the local maxima, we will allow the search to take some
down Hill steps to escape the local maxima by allowing some "bad" moves but
gradually decrease their size and frequency.
 The node is selected randomly and it checks whether it is a best move or not.
 If the move improves the situation, it is executed.
 E variable is introduced to calculate the probability of worsened.
 A Second parameter T is introduced to determine the probability.

46
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

The simulated annealing search algorithm

Property of simulated annealing search

T decreases slowly enough then simulated annealing search will find a global optimum with
probability approaching one.

Applications
 VLSI layout
 Airline scheduling

3. LOCAL BEAM SEARCH

 Local beam search is a variation of beam search which is a path based algorithm.
 It uses K states and generates successors for K states in parallel instead of one state
and its successors in sequence.
 The useful information is passed among the K parallel threads.
 The sequence of steps to perform local beam search is given below:
o Keep track of K states rather than just one.
o Start with K randomly generated states.
o At each iteration, all the successors of all K states are generated.

47
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

o If anyone is a goal state stop; else select the K best successors from the
complete list and repeat.
 This search will suffer from lack of diversity among K states.
 Therefore a variant named as stochastic beam search selects K successors at random,
with the probability of choosing a given successor being an increasing function of its
value.
4. GENETIC ALGORITHMS (GA)
A genetic algorithm (or GA) is a variant of stochastic beam search in which successor states
are generated by combining two parent states, rather than by modifying a single state.

A genetic algorithm is used to solve complicated problems with a greater number of variables
& possible outcomes/solutions. The combinations of different solutions are passed through
the algorithm to find the best solutions.

GA begins with a set of k randomly generated states, called the population.

Each state, or individual, is represented as a string over a finite alphabet.

For Example an 8 queen’s state could be represented as 8 digits, each in the range from 1 to
8.

Initial population: K randomly generated states of 8 queen problem

Individual (or) state: Each string in the initial population is individual (or) state. In one state,
the position of the queen of each column is represented.

Example: The state with the value 24748552 is derived as follows:

The Initial Population (Four randomly selected States) are :

48
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Evaluation function (or) Fitness function: A function that returns higher values for better
State. For 8 queens problem the number of non-attacking pairs of queens is defined as fitness
function.

Minimum fitness value is 0

Maximum fitness value is : 8*7/2 = 28 for a solution

The values of the four states are 24, 23, 20, and 11.
The probability of being chosen for reproduction is directly proportional to the fitness score,
which is denoted as percentage.

24 / (24+23+20+11) = 31%
23 / (24+23+20+11) = 29%
20 / (24+23+20+11) = 26%
11 / (24+23+20+11) = 14%

Selection : A random choice of two pairs is selected for reproduction, by considering the
probability of fitness function of each state. In the example one state is chosen twice
probability of 29%) and the another one state is not chosen (Probability of 14%)

Cross over: Each pair to be mated, a crossover point is randomly chosen. For the first pair
the crossover point is chosen after 3 digits and after 5 digits for the second pair.

49
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

offspring : Offspring is created by crossing over the parent strings in the crossover point.
That is, the first child of the first pair gets the first 3 digits from the first parent and the
remaining digits from the second parent. Similarly the second child of the first pair gets the
first 3 digits from the second parent and the remaining digits from the first parent.

Mutation : Each location is subject to random mutation with a small independent

probability. One digit was mutated in the first, third, and fourth offspring.

50
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

The initial population in (a) is ranked by the fitness function in (b), resulting in pairs for
mating in (c). They produce offspring in (d), which are subject to mutation in(e).

The sequence of steps to perform GA is summarized below:

Start with K randomly generated states population

O's and 1's)

51
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

es.

CONSTRAINT SATISFACTION PROBLEMS

We have seen so many techniques like Local search, Adversarial search to solve different
problems. The objective of every problem-solving technique is one, i.e., to find a solution to
reach the goal. Although, in adversarial search and local search, there were no constraints on
the agents while solving the problems and reaching to its solutions.

Definition: CSP (CONSTRAINT SATISFACTION PROBLEM)

It is a Type of problem-solving technique. By the name, it is understood that constraint

satisfaction means solving a problem under certain constraints or rules. Constraint
satisfaction is a technique where a problem is solved when its values satisfy certain
constraints or rules of the problem. Such type of technique leads to a deeper understanding of
the problem structure as well as its complexity.

Constraint satisfaction depends on three components, namely:

 X: It is a set of variables.
 D: It is a set of domains where the variables reside. There is a specific domain for
each variable.[values]
 C: It is a set of constraints which are followed by the set of variables.

The constraint value consists of a pair of {scope, rel}. The scope is a tuple of variables which
participate in the constraint and rel is a relation which includes a list of values which the
variables can take to satisfy the constraints of the problem.

CSP Examples
Constraint satisfaction includes those problems which contains some constraints while
solving the problem. CSP includes the following problems:

 Graph Coloring: The problem where the constraint is that no adjacent sides can have
the same color.

52
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

 Sudoku Playing: The gameplay where the constraint is that no number from 0-9 can
be repeated in the same row or column.

 n-queen problem: In n-queen problem, the constraint is that no queen should be

placed either diagonally, in the same row or column.

 Cryptarithmetic Problem: This problem has one most important constraint that is,
we cannot assign a different digit to the same character. All digits should contain a
unique alphabet.

53
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

EXAMPLE : DEFINING MAP COLORING PROBLEM

54
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

55
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

CONSTRAINT PROPAGATION / INFERENCE IN CSP

https://www.youtube.com/watch?v=leRtaG-0NLk

In local state-spaces, the choice is only one, i.e., to search for a solution. But in CSP, we have
two choices either:

 We can search for a solution or

 We can perform a special type of inference called constraint propagation.

Constraint propagation is a special type of inference which helps in reducing the legal
number of values for the variables. The idea behind constraint propagation is local
consistency.
In local consistency, variables are treated as nodes, and each binary constraint is treated as
an arc in the given problem. There are following local consistencies which are discussed
below:

 Node Consistency: A single variable is said to be node consistent if all the values in
the variable’s domain satisfy the unary constraints on the variables.
 Arc Consistency: A variable is arc consistent if every value in its domain satisfies the
binary constraints of the variables.
 Path Consistency: When the evaluation of a set of two variable with respect to a
third variable can be extended over another variable, satisfying all the binary
constraints. It is similar to arc consistency.
 k-consistency: This type of consistency is used to define the notion of stronger forms
of propagation. Here, we examine the k-consistency of the variables.

56
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

57
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

AC-3 algorithm:

AC-3 maintains a queue of arcs which initially contains all the arcs in the CSP. AC-3
then pops off an arbitrary arc (Xi, Xj) from the queue and makes Xi arc-consistent with
respect to Xj. If this leaves Di unchanged, just moves on to the next arc; But if this
revises Di, then add to the queue all arcs (Xk, Xi) where Xk is a neighbor of Xi. If Di is
revised down to nothing, then the whole CSP has no consistent solution, return failure;
Otherwise, keep checking, trying to remove values from the domains of variables until
no more arcs are in the queue. The result is an arc-consistent CSP that have the same
solutions as the original one but have smaller domains.

58
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

59
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

BACKTRACKING SEARCH FOR CSP’s

https://www.youtube.com/watch?v=ytE2rzh1XpQ

Backtracking search, a form of depth-first search, is commonly used for solving CSPs.

Backtracking search: A depth-first search that chooses values for one variable at a
time and backtracks when a variable has no legal values left to assign.

Backtracking algorithm repeatedly chooses an unassigned variable, and then tries all
values in the domain of that variable in turn, trying to find a solution. If an
inconsistency is detected, then BACKTRACK returns failure, causing the previous
call to try another value.
BACKTRACKING-SARCH keeps only a single representation of a state and alters
that representation rather than creating a new ones.

60
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

MAP COLORING PROBLEM

61
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

62
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

63
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

64
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

GAME THEORY / ADVERSARIAL SEARCH

o Searches in which two or more players with conflicting goals are trying to explore the
same search space for the solution, are called adversarial searches, often known as
Games.

Types of games in AI

o Perfect information: A game with the perfect information is that in which agents can
look into the complete board. Agents have all the information about the game, and
they can see each other moves also. Examples are Chess, Checkers, Go, etc.

65
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

o Imperfect information: If in a game agents do not have all information about the
game and not aware with what's going on, such type of games are called the game
with imperfect information, such as tic-tac-toe, Battleship, blind, Bridge, etc.
o Deterministic games: Deterministic games are those games which follow a strict
pattern and set of rules for the games, and there is no randomness associated with
them. Examples are chess, Checkers, Go, tic-tac-toe, etc.
o Non-deterministic games: Non-deterministic are those games which have various
unpredictable events and has a factor of chance or luck. This factor of chance or luck
is introduced by either dice or cards. These are random, and each action response is
not fixed. Such games are also called as stochastic games. Example: Backgammon,
Monopoly, Poker, etc.

Zero-Sum Game

o Zero-sum games are adversarial search which involves pure competition.

o In Zero-sum game each agent's gain or loss of utility is exactly balanced by the losses
or gains of utility of another agent.
o One player of the game try to maximize one single value, while other player tries to
minimize it.
o Each move by one player in the game is called as ply.
o Chess and tic-tac-toe are examples of a Zero-sum game.

OPTIMAL DECISIONS IN GAMES

o A game can be defined as a type of search problem in AI which can be formalized of

the following elements:
 Initial state: It specifies how the game is set up at the start.
 Player(s): It specifies which player has moved in the state space.
 Action(s): It returns the set of legal moves in state space.
 Result(s, a): It is the transition model, which specifies the result of moves in
the state space.
 Terminal-Test(s): Terminal test is true if the game is over, else it is false at
any case. The state where the game ends is called terminal states.

66
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Utility Function / Objective Function / Payoff function: A utility function gives the final
numeric value for a game that ends in terminal state. For Chess, the outcomes are a win, loss,
or draw and its payoff values are +1, -1 or 0.

GAME TREE : The initial and the legal moves for each side define the GAME TREE for
the game. A game tree is a tree where nodes of the tree are the game states and
Edges of the tree are the moves by players. Game tree involves initial state,
actions function, and result Function.

THE MINIMAX ALGORITHM

Example Game Tree: Tic-Tac-Toe game tree or Mini-max algorithm

Minimax algorithm is a recursive or backtracking algorithm which is used in decision-
making and game theory. It provides an optimal move for the player assuming that opponent
is also playing optimally. MiniMax algorithm is mostly used for game playing in AI Such as
Chess, Checkers, tic-tac-toe, go, and various two-players game.

 There are two players MAX and MIN.

 Players have an alternate turn and start with MAX.
 MAX maximizes the result of the game tree
 MIN minimizes the result.
 Both Players of the game are opponent of each other, where MAX will select the
maximized value and MIN will select the minimized value.
 The minimax algorithm performs a depth-first search algorithm for the exploration
of the complete game tree.
 The minimax algorithm proceeds all the way down to the terminal node of the tree,
then backtrack the tree as the recursion.
 So in the game tree, we have a layer of Max, a layer of MIN, and each layer is called
as Ply. Max place x, then MIN puts o to prevent Max from winning, and this game
continues until the terminal node.
 In this either MIN wins, MAX wins, or it's a draw.

The following figure is showing part of the game-tree for tic-tac-toe game. Following are
some key points of the game:

67
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Even a simple game like tic-tac-toe is too complex for us to draw the entire game tree.
The possible moves for MAX at the root node are labeled al, a2, and a3. The possible replies
to a1 for MIN are b1, b2, b3, and so on.

A Two Ply Game Tree

68
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

THE OPTIMAL STRATEGY:

In a given game tree, the optimal strategy can be determined from the minimax value of each
node, which can be written as MINIMAX(n). MAX prefer to move to a state of maximum
value and MIN prefer to move to a state of minimum value then:

An algorithm for minimax decision

Complexity : If the maximum depth of the tree is m, and there are b legal moves at each
point then the time complexity of the minimax algorithm is O(bm). This algorithm is a depth
first search, therefore the space requirements are linear in m and b. For real games the
calculation of time complexity is impossible, however this algorithm will be a good basis for
game playing.

69
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Completeness : It the tree is finite, then it is complete.

Optimality : It is optimal when played against an optimal opponent

ALPHA–BETA PRUNING

o Alpha-beta pruning is a modified version of the minimax algorithm. It is an

optimization technique for the minimax algorithm.
o Pruning - The process of eliminating a branch of the search tree from consideration
without examining is called pruning.
o This involves two threshold parameter Alpha and beta for future expansion, so it is
called alpha-beta pruning. It is also called as Alpha-Beta Algorithm.
o Alpha-beta pruning can be applied at any depth of a tree, and sometimes it not only
prune the tree leaves but also entire sub-tree.
o The two-parameter can be defined as:
a. Alpha: The best (highest-value) choice we have found so far at any point
along the path of Maximizer. The initial value of alpha is -∞.
b. Beta: The best (lowest-value) choice we have found so far at any point along
the path of Minimizer. The initial value of beta is +∞.
The Alpha-beta pruning removes all the nodes which are not really affecting the final
decision and makes the algorithm fast.

Condition for Alpha-beta pruning:

The main condition which required for alpha-beta pruning is:

α >= β
Key points about alpha-beta pruning:
o The Max player will only update the value of alpha.
o The Min player will only update the value of beta.
o While backtracking the tree, the node values will be passed to upper nodes instead of
values of alpha and beta.
o We will only pass the alpha, beta values to the child nodes.

Working of Alpha-Beta Pruning:

Let's take an example of two-player search tree to understand the working of Alpha-beta
pruning

70
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Step 1: At the first step the, Max player will start first move from node A where α= -∞ and
β= +∞, these value of alpha and beta passed down to node B where again α= -∞ and β= +∞,
and Node B passes the same value to its child D.

Step 2: At Node D, the value of α will be calculated as its turn for Max. The value of
α is compared with firstly 2 and then 3, and the max (2, 3) = 3 will be the value of α at
node D and node value will also 3.

Step 3: Now algorithm backtrack to node B, where the value of β will change as this
is a turn of Min, Now β= +∞, will compare with the available subsequent nodes value,
i.e. min (∞, 3) = 3, hence at node B now α= -∞, and β= 3.

71
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

In the next step, algorithm traverse the next successor of Node B which is node E, and the
values of α= -∞, and β= 3 will also be passed.

Step 4: At node E, Max will take its turn, and the value of alpha will change. The current
value of alpha will be compared with 5, so max (-∞, 5) = 5, hence at node E α= 5 and β= 3,
where α>=β, so the right successor of E will be pruned, and algorithm will not traverse it, and
the value at node E will be 5.

72
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Step 5: At next step, algorithm again backtrack the tree, from node B to node A. At node A,
the value of alpha will be changed the maximum available value is 3 as max (-∞, 3)= 3, and
β= +∞, these two values now passes to right successor of A which is Node C.

At node C, α=3 and β= +∞, and the same values will be passed on to node F.

Step 6: At node F, again the value of α will be compared with left child which is 0, and
max(3,0)= 3, and then compared with right child which is 1, and max(3,1)= 3 still α remains
3, but the node value of F will become 1.

Step 7: Node F returns the node value 1 to node C, at C α= 3 and β= +∞, here the value of
beta will be changed, it will compare with 1 so min (∞, 1) = 1. Now at C, α=3 and β= 1, and
again it satisfies the condition α>=β, so the next child of C which is G will be pruned, and the
algorithm will not compute the entire sub-tree G.

73
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Step 8: C now returns the value of 1 to A here the best value for A is max (3, 1) = 3.
Following is the final game tree which is the showing the nodes which are computed
and nodes which has never computed. Hence the optimal value for the maximizer is 3
for this example.

74
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Algorithm for Alpha-Beta Search

75
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Example 2:

Stages in the calculation of the optimal decision for the game tree

76
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

STOCHASTIC GAMES / (GAMES THAT INCLUDE AN ELEMENT OF CHANCE)

Definition:

Example 1 - Backgammon Game

Backgammon is a typical game that combines luck and skill. Dice are rolled at the beginning
of a player's turn to determine the legal moves. In the backgammon position of Figure 5.10,
for example, White has rolled a 6–5 and has four possible moves.

77
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Goal State
The goal of the game is to move all one's pieces off the board. White moves clockwise
toward 25, and black moves counterclockwise toward 0.

Successor Function or Operator

Move to any position except where two or more of the opponent pieces.
If it moves to a position with one opponent piece it is captured and again it has to start from
Beginning

Task : In the position shown, White has rolled 6-5. So Find out the legal moves for the set of
the dice thrown as 6 and 5.

Solution :
There are Four legal moves. They are
(5-11,5-10)
(5-11, 19-24)
(10-16,5-10)
(5-11,11-16)

Although White knows what his or her own legal moves are, White does not know what
Black is going to roll and thus does not know what Black’s legal moves will be. That means

78
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

White cannot construct a standard game tree of the sort we saw in chess and tic-tac-toe. A
game tree in backgammon must include chance nodes in addition to MAX and MIN nodes.
Chance nodes are shown as circles. The branches leading from each chance node denote the
possible dice rolls, and each is labelled with the roll and the chance that it will occur. There
are 36 ways to roll two dice, each equally likely; but because a 6-5 is the same as a 5-6, there
are only 21 distinct rolls. The six doubles (1-1 through 6-6) have a 1/36 chance of coming up,
the other 15 distinct rolls a 1/18 chance each.

The next step is to understand how to make correct decisions. Obviously, we still
want to pick the move that leads to the best position. However, positions do not have definite
minimax values. Instead, we can only calculate the expected value of a position: the average
over all possible outcomes of the chance nodes.

This leads us to generalize the minimax value for deterministic games to an

expectiminimax value for games with chance nodes. Terminal nodes and MAX and MIN
nodes (for which the dice roll is known) work exactly the same way as before. For chance
nodes we compute the expected value, which is the sum of the value over all outcomes,
weighted by the probability of each chance action:

where r represents a possible dice roll (or other chance event) and RESULT(s, r) is the same
state as s, with the additional fact that the result of the dice roll is r.

79
CS8691 ARTIFICIAL INTELLIGENCE UNIT - 2

Example 2 – Card Game

80