ARTIFICIAL INTELLIGENCE

UNIT - I

Chapter-I: Problem Solving by Search –II: Problem-Solving Agents, Searching for Solutions,
Uninformed Search Strategies: Breadth-first search, Uniform cost search, Depth-first search,
Iterative deepening Depth-first search, Bidirectional search, Informed (Heuristic) Search
Strategies: Greedy best-first search, A* search, Heuristic Functions, Beyond Classical Search:
Hill-climbing search, Simulated annealing search, Local Search in Continuous Spaces, Searching
with Non-Deterministic Actions, Searching with Partial Observations, Online Search Agents and
Unknown Environment.

SEARCH ALGORITHMS IN ARTIFICIAL INTELLIGENCE

 Search algorithms are one of the most important areas of Artificial Intelligence. The following will
explain all about the search algorithms in AI.
Problem-solving agents:
 In Artificial Intelligence, Search techniques are universal problem-solving methods.
 Rational agents or Problem-solving agents in AI mostly use these search strategies or algorithms
to solve a specific problem and provide the best result.
 Problem-solving agents are the goal-based agents and use atomic representation.
 Various problem-solving search algorithms are discussed here.

Search Algorithm Terminologies:

o Search: Searching is a step-by-step procedure to solve a search-problem in given search space. A
search problem can have three main factors:
1. Search Space: Search space represents a set of possible solutions, which a system may have.
2. Start State: It is a state from where agent begins the search.
3. Goal test: It is a function which observe the current state and returns whether the goal state is
achieved or not.
o Search tree: A tree representation of search problem is called Search tree. The root of the search tree
is the root node which is corresponding to the initial state.
o Actions: It gives the description of all the available actions to the agent.
o Transition model: A description of what each action do, can be represented as a transition model.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 o Path Cost: It is a function which assigns a numeric cost to each path.
o Solution: It is an action sequence which leads from the start node to the goal node.
o Optimal Solution: If a solution has the lowest cost among all solutions.

Properties of Search Algorithms:

Following are the four essential properties of search algorithms to compare the efficiency of these algorithms:

 Completeness: A search algorithm is said to be complete if it guarantees to return a solution if at least

any solution exists for any random input.
 Optimality: If a solution found for an algorithm is guaranteed to be the best solution (lowest path
cost) among all other solutions, then such a solution is said to be an optimal solution.
 Time Complexity: Time complexity is a measure of time for an algorithm to complete its task.
 Space Complexity: It is the maximum storage space required at any point during the search, as the
complexity of the problem.

TYPES OF SEARCH ALGORITHMS

 Based on the search problems the search algorithms can be classified into uninformed (Blind search)
search and informed search (Heuristic search) algorithms.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Uninformed/Blind Search:
 The uninformed search does not contain any domain knowledge such as closeness, the location of
the goal.
 It operates in a brute-force way as it only includes information about how to traverse the tree and
how to identify leaf and goal nodes.
 Uninformed search applies a way in which search tree is searched without any information about the
search space like initial state operators and test for the goal, so it is also called blind search. It
examines each node of the tree until it achieves the goal node.
 It can be divided into six main types:
1. Breadth-first Search
2. Depth-first Search
3. Depth-limited Search
4. Iterative deepening depth-first search
5. Uniform cost search
6. Bidirectional Search

Informed Search
 Informed search algorithms use domain knowledge.
 In an informed search, problem information is available which can guide the search.
 Informed search strategies can find a solution more efficiently than an uninformed search strategy.
Informed search is also called a Heuristic search.
 A heuristic is a way which might not always be guaranteed for best solutions but guaranteed to find a
good solution in reasonable time.
 Informed search can solve much complex problem which could not be solved in another way. An
example of informed search algorithms is a traveling salesman problem.
 The following are the types of informed search algorithms:
1. Best First Search Algorithm (Greedy search)
2. A* Search Algorithm

UNINFORMED SEARCH ALGORITHMS

 Uninformed search is a class of general-purpose search algorithms which operates in brute force-way.
Uninformed search algorithms do not have additional information about state or search space other
than how to traverse the tree, so it is also called blind search.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Following are the various types of uninformed search algorithms:
1. Breadth-first Search
2. Depth-first Search
3. Depth-limited Search
4. Iterative deepening depth-first search
5. Uniform cost search
6. Bidirectional Search

1. Breadth-first Search:
 Breadth-first search is the most common search strategy for traversing a tree or graph. This
algorithm searches breadthwise in a tree or graph, so it is called breadth-first search.
 BFS algorithm starts searching from the root node of the tree and expands all successor nodes at the
current level before moving to nodes of next level.
 The breadth-first search algorithm is an example of a general-graph search algorithm. Breadth-first
search implemented using FIFO queue data structure.
 As breadth-first search algorithm do the traversal level by level, it will also called as level-order
traversal.

Advantages:
 BFS will provide a solution if any solution exists.
 If there are more than one solution for a given problem, then BFS will provide the minimal solution
which requires the least number of steps.

Disadvantages:
 It requires lots of memory since each level of the tree must be saved into memory to expand the next
level.
 BFS needs lots of time if the solution is far away from the root node.

Example:
 In the below tree structure, the traversing of the tree is shown using BFS algorithm from the root
node S to goal node K.
 BFS search algorithm traverse in layers, so it will follow the path which is shown by the arrow, and
the traversed path will be:

SABCDGHEFIK
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  Time Complexity: Time Complexity of BFS algorithm can be obtained by the number of nodes
traversed in BFS until the shallowest Node.
Where the d= depth of shallowest solution and b is a node at every state.
T(b) = 1+b2+b3+.......+ bd= O(bd)

 Space Complexity: Space complexity of BFS algorithm is given by the Memory size of frontier
which is O(bd).
 Completeness: BFS is complete, which means if the shallowest goal node is at some finite depth,
then BFS will find a solution.
 Optimality: BFS is optimal if path cost is a non-decreasing function of the depth of the node.

2. Depth-first Search
 Depth-first search is a recursive algorithm for traversing a tree or graph data structure.
 It is called the depth-first search because it starts from the root node and follows each path to its
greatest depth node before moving to the next path.
 DFS uses a stack data structure (LIFO) for its implementation.
 The process of the DFS algorithm is also like the BFS algorithm.

Advantage:
 DFS requires very less memory as it only needs to store a stack of the nodes on the path from root
node to the current node.
 It takes less time to reach to the goal node than BFS algorithm (if it traverses in the right path).
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
Disadvantage:

 There is the possibility that many states keep re-occurring, and there is no guarantee of finding the
solution.
 DFS algorithm goes for deep down searching and sometimes it may go to the infinite loop.

Example:

 In the below search tree, the flow of depth-first search is shown, and it will follow the order as:
Pre-order (RootLeftRight)
In-order (LeftRootRight)
Post-order (LeftRightRoot)

 It will start searching from root node S, and traverse A, then B, then D and E, after traversing E, it will
backtrack the tree as E has no other successor and still goal node is not found.
 After backtracking it will traverse node C and then G, and here it will terminate as it found goal node.
 There are three different ways in which DFS can be performed pre-order, in-order and post-order.
 Hence, the solution will be:
Pre-order (RootLeftRight): SABDECG
In-order (LeftRootRight): DBEACG
Post-order (LeftRightRoot): DEBG

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Completeness: DFS search algorithm is complete within finite state space as it will expand every
node within a limited search tree. But if the search space becomes infinite DFS will become in
complete.
 Time Complexity: Time complexity of DFS will be equivalent to the node traversed by the
algorithm. It is given by:
T(n)= 1+ n2+ n3 +.........+ nm=O(nm)

Where, m= maximum depth of any node and this can be much larger than d (Shallowest solution
depth)
 Space Complexity: DFS algorithm needs to store only single path from the root node, hence space
complexity of DFS is equivalent to the size of the fringe set, which is O(bm).
 Optimal: DFS search algorithm is non-optimal, as it may generate many steps or high cost to reach
to the goal node.

3. Depth-Limited Search Algorithm:

 A depth-limited search algorithm is like depth-first search with a predetermined limit.
 Depth-limited search can solve the drawback of the infinite path in the Depth-first search.
 In this algorithm, the node at the depth limit will treat as it has no successor nodes further.
 Depth-limited search can be terminated with two Conditions of failure:
o Standard failure value: It indicates that problem does not have any solution.

o Cutoff failure value: It defines no solution for the problem within a given depth limit.

Advantages:

 Depth-limited search is Memory efficient.

Disadvantages:
 Depth-limited search also has a disadvantage of incompleteness.
 It may not be optimal if the problem has more than one solution.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Example:

Solution: Assume l (predetermined limit) as 2 and apply the algorithm.

 As it is same as DFS but with a predetermined limit, it will also have three different ways of
searching (Pre-Order, In-Order and Post-Order).
 Hence, the solution will be:

Pre-order (RootLeftRight): SACDBIJ

In-order (LeftRootRight): CADSIBJ
Post-order (LeftRightRoot): CDAIJ

 Completeness: DLS search algorithm is complete if the solution is above the depth-limit.
 Time Complexity: Time complexity of DLS algorithm is O(bℓ).
 Space Complexity: Space complexity of DLS algorithm is O(b×ℓ).
 Optimal: Depth-limited search can be viewed as a special case of DFS, and it is also not optimal
even if ℓ>d.

4. Uniform-cost Search Algorithm:

 Uniform-cost search is a searching algorithm used for traversing a weighted tree or graph.
 This algorithm comes into play when a different cost is available for each edge.
 The primary goal of the uniform-cost search is to find a path to the goal node which has the lowest
cumulative cost.
 Uniform-cost search expands nodes according to their path costs form the root node. It can be used
to solve any graph/tree where the optimal cost is in demand.
 A uniform-cost search algorithm is implemented by the priority queue. It gives maximum priority to
the lowest cumulative cost.
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  Uniform cost search is equivalent to BFS algorithm if the path cost of all edges is the same.

Advantages:
 Uniform cost search is optimal because at every state the path with the least cost is chosen.

Disadvantages:
 It does not care about the number of steps involve in searching and only concerned about path cost.
Due to which this algorithm may be stuck in an infinite loop.

Example:

 Completeness: Uniform-cost search is complete, such as if there is a solution, UCS will find it.
 Time Complexity: Let C* is Cost of the optimal solution, and ε is each step to get closer to the goal
node. Then the number of steps is = C*/ε+1. Here +1 is taken, as it starts from state 0 and end to C*/ε.
Hence, the worst-case time complexity of Uniform-cost search is O(b1 + [C*/ε]).
 Space Complexity: The same logic is for space complexity so, the worst-case space complexity of
Uniform-cost search is O(b1 + [C*/ε]).
 Optimal: Uniform-cost search is always optimal as it only selects a path with the lowest path cost.
5. Iterative deepening depth-first Search:
 The iterative deepening algorithm is a combination of DFS and BFS algorithms. This search algorithm
finds out the best depth limit and does it by gradually increasing the limit until a goal is found.
 This algorithm performs depth-first search up to a certain "depth limit", and it keeps increasing the
depth limit after each iteration until the goal node is found.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  This Search algorithm combines the benefits of Breadth-first search's fast search and depth-first
search's memory efficiency.
 The iterative search algorithm is useful uninformed search when search space is large, and depth of
goal node is unknown.
Advantages:
 It combines the benefits of BFS and DFS search algorithm in terms of fast search and memory
efficiency.
Disadvantages:
 The main drawback of IDDFS is that it repeats all the work of the previous phase.
Example:
 Following tree structure is showing the iterative deepening depth-first search. IDDFS algorithm
performs various iterations until it does not find the goal node.
 The iteration performed by the algorithm is given as:

1'st Iteration  A
2'nd Iteration  A, B, C
3'rd Iteration  A, B, D, E, C, F, G
In the third iteration, the algorithm will find the goal node.

 Completeness: This algorithm is complete is if the branching factor is finite.

 Time Complexity: Let us suppose b is the branching factor and depth is d then the worst-case time
complexity is O(bd).
 Space Complexity: The space complexity of IDDFS will be O(bd).
 Optimal: IDDFS algorithm is optimal if path cost is a non- decreasing function of the depth of the
node.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
6. Bidirectional Search Algorithm:
 Bidirectional search algorithm runs two simultaneous searches, one form initial state called as
forward-search and other from goal node called as backward-search, to find the goal node.
 Bidirectional search replaces one single search graph with two small subgraphs in which one starts
the search from an initial vertex and other starts from goal vertex. The search stops when these two
graphs intersect each other.
 Bidirectional search can use search techniques such as BFS, DFS, DLS, etc.

Advantages:
 Bidirectional search is fast.
 Bidirectional search requires less memory.

Disadvantages:
 Implementation of the bidirectional search tree is difficult.
 In bidirectional search, one should know the goal state in advance.

Example:

 In the below search tree, bidirectional search algorithm is applied.

 This algorithm divides one graph/tree into two sub-graphs.
 It starts traversing from node 1 in the forward direction and starts from goal node 16 in the backward
direction.
 The algorithm terminates at node 9 where two searches meet.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Completeness: Bidirectional Search is complete if we use BFS in both searches.
 Time Complexity: Time complexity of bidirectional search using BFS is O(bd).
 Space Complexity: Space complexity of bidirectional search is O(bd).
 Optimal: Bidirectional search is Optimal.

INFORMED SEARCH ALGORITHMS

 The uninformed search algorithms which looked through search space for all possible solutions of the
problem without having any additional knowledge about search space.
 But informed search algorithm contains an array of knowledge such as how far we are from the goal, path
cost, how to reach to goal node, etc.
 This knowledge help agents to explore less to the search space and find more efficiently the goal node.
 The informed search algorithm is more useful for large search space. Informed search algorithm uses
the idea of heuristic, so it is also called Heuristic search.

Heuristics function [What, Why and How]:

 Heuristic is a function which is used in Informed Search, and it finds the most promising path.
 It takes the current state of the agent as its input and produces the estimation of how close agent is
from the goal.
 The heuristic method, however, might not always give the best solution, but it guaranteed to find a
good solution in reasonable time.
 The heuristic method is a technique designed to solve a problem quickly.
 Heuristic function estimates how close a state is to the goal. It is represented by h(n), and it
calculates the cost of an optimal path between the pair of states. The value of the heuristic function is
always positive.
 Through out AI we use heuristic function / heuristic value in many situations, hence a better
understanding of this is required.
 In uninformed search algorithms we do blind search, i.e., we search all possible states for goal state.
Once reaching goal state the search will stop.

Solving 8-Puzzle Problem (Without Heuristics Search)

 The eight-puzzle problem by the name of N puzzle problem or sliding puzzle problem, that consists
of N tiles (N+1 titles with an empty tile) where N can be 8, 15, 24 and so on.
 In our example N = 8. (i.e., square root of (8+1) = 3 rows and 3 columns).

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  In the same way, if we have N = 15, 24, then they have Row and columns as follow (square root of
(N+1) rows and square root of (N+1) columns).
 That is if N=15 than number of rows and columns= 4, and if N= 24 number of rows and columns= 5.
 In these types of problems, we have given an initial state or initial configuration (Start state) and a
Goal state or Goal Configuration.
 Here We are solving a problem of 8 puzzle that is a 3x3 matrix.

Solution:
 The puzzle can be solved by moving the tiles one by one in the single empty space and thus
achieving the Goal state.
Rules of solving puzzle
 Instead of moving the tiles in the empty space we can visualize moving the empty space in
place of the tile.
 The empty space can only move in four directions (Movement of empty space).
1. Up
2. Down
3. Right or
4. Left
 The empty space cannot move diagonally and can take only one step at a time.
All possible move of an Empty tile

O- Position total possible moves are (2)

X - position total possible moves are (3) and
#-position total possible moves are (4)

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Let us solve the problem without Heuristic Search that is Uninformed Search or Blind Search
(Breadth First Search)
Breath First Search to solve Eight puzzle problem.

Time complexity: In worst case time complexity in BFS is O(bd) know as order of b raise to power
d. In this case it is (320).
b – branch factor
d – depth factor
The problem in uninformed search algorithms is the time complexity is growing exponentially.
8 – Puggle Problem 15 – Puggle Problem 24 – Puggle Problem
(Time complexity worst case) (Time complexity worst case) (Time complexity worst case)
O(bd) O(bd) O(bd)
320 – Possible States 1013 – Possible States 1024 – Possible States

 Let us say chess problem where the branch factor may go upto 32 and depth factor upto 85 to
90. And the possible states 3285 to 90 (it is a very huge number to imagine).

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  From the above it is clear that the time complexity will grow exponentially. These types of
problems are called as Non-Polynomial (NP) problems. And in NP problem the time
complexity will be more.
 The advantage of NP problems is that, the solution is guaranteed.
 To solve the NP problems in Polynomial time, the heuristic function is used.
Calculating Heuristic value:
There are different methods to calculate the heuristic value. Some basic types are discussed here:
i. Euclidean Distance (straight line distance): This method is generally applied when the nodes
to be considered are points in Euclidean Space. i.e., for the points with x, y coordinate values.

Then the Euclidean space will be:

ii. Manhattan Distance: It is generally used when vertical or horizontal distance needs to be
found. Mostly the Manhattan distance will be problem specific.
 In the 8-puzzle problem it will be the total number of moves to make the tiles in their right
positions.
Example:

Tile# 1 2 3 4 5 6 7 8 Total
No of moves required to put the 0 0 0 1 1 0 0 1 3
tile in its right position

 So, the Manhattan distance for the above state will be 3.

 Calculating the number of misplaced tiles is also an approach for finding the Manhattan
distance.
Example:

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Tile# 1 2 3 4 5 6 7 8 Total
Misplaced? (Y/N) N N N Y Y N N Y 3

 So, the Manhattan distance for the above state will be 3.

Admissibility of the heuristic function is given as:

h(n) <= h*(n)

 Here h(n) is heuristic cost, and h*(n) is the estimated cost. Hence heuristic cost should be less than or
equal to the estimated cost.

Pure Heuristic Search:

 Pure heuristic search is the simplest form of heuristic search algorithms. It expands nodes based on
their heuristic value h(n).
 It maintains two lists, OPEN and CLOSED list. In the CLOSED list, it places those nodes which
have already expanded and in the OPEN list, it places nodes which have yet not been expanded.
 On each iteration, each node n with the lowest heuristic value is expanded and generates all its
successors and n is placed to the closed list. The algorithm continues unit a goal state is found.
 In the informed search we will discuss two main algorithms which are given below:
1. Best First Search Algorithm (Greedy search)
2. A* Search Algorithm

1. Best-first Search Algorithm (Greedy Search):

 Greedy best-first search algorithm always selects the path which appears best at that moment. It is
the combination of depth-first search and breadth-first search algorithms. It uses the heuristic
function and search.
 Best-first search allows us to take the advantages of both algorithms. With the help of best-first
search, at each step, the most promising node can be selected.
 In the best first search algorithm, the node which is closest to the goal node can be expanded and
the closest cost is estimated by heuristic function, i.e.
f(n)= h(n).

Were, h(n)= estimated cost from node n to the goal.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  To implement such a graph-search procedure, we will need to use two lists of nodes;
o OPEN – nodes that have been generated and have had the heuristic function applied to them,
but which have not yet been examined (i.e., had their successors generated).
o CLOSED – nodes that have already been examined. We need to keep these nodes in memory
if we want to search a graph rather than a tree, since whenever a new node is generated, we
need to check whether it has been generated before.

Best first search algorithm:

o Step 1: Place the starting node into the OPEN list.

o Step 2: If the OPEN list is empty, Stop and return failure.
o Step 3: Remove the node n, from the OPEN list which has the lowest value of h(n) and places it
in the CLOSED list.
o Step 4: Expand the node n, and generate the successors of node n.
o Step 5: Check each successor of node n, and find whether any node is a goal node or not. If any
successor node is goal node, then return success and terminate the search, else proceed to Step 6.
o Step 6: For each successor node, algorithm checks for evaluation function f(n), and then check if
the node has been in either OPEN or CLOSED list. If the node has not been in both lists, then add
it to the OPEN list.
o Step 7: Return to Step 2.

Advantages:
 Best first search can switch between BFS and DFS by gaining the advantages of both the algorithms.
 This algorithm is more efficient than BFS and DFS algorithms.

Disadvantages:
 It can behave as an unguided depth-first search in the worst-case scenario.
 It can get stuck in a loop as DFS.
 This algorithm is not optimal.

Example – 1:
 Consider the below search problem, and we will traverse it using greedy best-first search.
 At each iteration, each node is expanded using evaluation function f(n)=h(n), which is given in the
below table.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  In this search example, two lists which are OPEN and CLOSED Lists are used.
 Following are the iteration for traversing the above example.

Expand the nodes of S and put in the CLOSED list

Initialization: Open [S], Closed []
Iteration 1: Open [B, A], Closed [S]

Iteration 2: Open [F, E, A], Closed [S, B]

: Open [E, A], Closed [S, B, F]

Iteration 3: Open [G, I, E, A], Closed [S, B, F]

: Open [I, E, A], Closed [S, B, F, G]
Hence the final solution path will be: SBFG

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Time Complexity: The worst-case time complexity of Greedy best first search is O(bm).
 Space Complexity: The worst-case space complexity of Greedy best first search is O(bm). Where, m
is the maximum depth of the search space.
 Complete: Greedy best-first search is also incomplete, even if the given state space is finite.
 Optimal: Greedy best first search algorithm is not optimal.
Example – 2: In the following example C is the initial or source node and L and Z are goal nodes.

Step – 1: OPEN: [C]

CLOSED: [-]
Step – 2: OPEN: [T, O, E, B, P]
(Keep all the nodes in the OPEN in ascending order of their heuristic values)
CLOSED: [C]
Step – 3: OPEN: [O, E, B, P]
(Keep all the nodes in the OPEN in ascending order of their heuristic values)
CLOSED: [C, T]
As T is neither the goal state nor it is having any successors to examine, so examine the very next node in
OPEN list, which is O.
Step – 4: OPEN: [I, E, B, P, N]
(Keep all the nodes in the OPEN in ascending order of their heuristic values)
CLOSED: [C, T, O]
Step – 5: OPEN: [Z, E, B, P, N]
(Keep all the nodes in the OPEN in ascending order of their heuristic values)
CLOSED: [C, T, O, I]
Step – 6: Next node from is Z which is a goal node. So, stop the search.
The goal is found, and the final path is COIZ.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
2. A* Search Algorithm:
 A* search is the most well-known form of best-first search. It uses heuristic function h(n), and cost to
reach the node n from the start state g(n). It has combined features of uniform cost search (UCS) and
greedy best-first search, by which it solves the problem efficiently.
 A* search algorithm finds the shortest path through the search space using the heuristic function. This
search algorithm expands less search tree and provides optimal result faster.
 A* algorithm is like uniform cost search (UCS) except that it uses g(n)+h(n) instead of g(n).
 In A* search algorithm, we use heuristic search as well as the cost to reach the node n. Hence, we can
combine both costs as following, and this sum is called as a fitness number.

Algorithm of A* search:

Step1: Place the starting node in the OPEN list.

Step 2: Check if the OPEN list is empty or not, if the list is empty then return failure and stops.
Step 3: Select the node from the OPEN list which has the smallest value of evaluation function
(g+h), if node n is goal node, then return success and stop, otherwise
Step 4: Expand node n and generate all of its successors and put n into the closed list. For each
successor n', check whether n' is already in the OPEN or CLOSED list, if not then
compute evaluation function for n' and place into Open list.
Step 5: Else if node n' is already in OPEN and CLOSED, then it should be attached to the back
pointer which reflects the lowest g(n') value.
Step 6: Return to Step 2.

Advantages:
 A* search algorithm is the best algorithm than other search algorithms.
 A* search algorithm is optimal and complete.
 This algorithm can solve very complex problems.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Disadvantages:
 It does not always produce the shortest path as it mostly based on heuristics and approximation.
 A* search algorithm has some complexity issues.
 The main drawback of A* is memory requirement as it keeps all generated nodes in the memory, so it
is not practical for various large-scale problems.

Example – 1:

 In this example, the given graph will be traversed using the A* algorithm.
 The heuristic value of all states is given in the below table so, calculate the f(n) of each state using
the formula f(n)= g(n) + h(n), where g(n) is the cost to reach any node from start state.
 Here OPEN and CLOSED list will be used.

Solution:

Initialization: {(S, 5)}

Iteration – 1: {(SA, 4), (SG, 10)}
Iteration – 2: {(SAC, 4), (SAB, 7), (SG, 10)}
Iteration – 3: {(SACG, 6), (SACD, 11), (SAB, 7), (SG, 10)}
Iteration – 4: will give the result, as SACG it provides the optimal path with cost 6.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Points to remember:
 A* algorithm returns the path which occurred first, and it does not search for all remaining paths.
 The efficiency of A* algorithm depends on the quality of heuristic.
 A* algorithm expands all nodes which satisfy the condition f(n)<="" li="">

 Complete: A* algorithm is complete as long as:

o Branching factor is finite.
o Cost at every action is fixed.
 Optimal: A* search algorithm is optimal if it follows below two conditions:
 Admissible: the first condition requires for optimality is that h(n) should be an admissible heuristic
for A* tree search. An admissible heuristic is optimistic in nature.
 Consistency: Second required condition is consistency for only A* graph-search.
If the heuristic function is admissible, then A* tree search will always find the least cost path.
 Time Complexity: The time complexity of A* search algorithm depends on heuristic function, and
the number of nodes expanded is exponential to the depth of solution d. So, the time complexity is
O(bd), where b is the branching factor.
 Space Complexity: The space complexity of A* search algorithm is O(bd)

Example - 2

Example – 3
Given an initial state of a 8-puzzle problem and final state to be reached-

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Find the most cost-effective path to reach the final state from initial state using A* Algorithm.
Consider g(n) = Depth of node and h(n) = Number of misplaced tiles.
Solution:
 A* Algorithm maintains a tree of paths originating at the initial state.
 It extends those paths one edge at a time.
 It continues until final state is reached.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Example – 4:
Consider the following graph-

 The numbers written on edges represent the distance between the nodes.
 The numbers written on nodes represent the heuristic value.
 Find the most cost-effective path to reach from start state A to final state J using A* Algorithm.

Solution:
Step-01: We start with node A.
Node B and Node F can be reached from node A.
A* Algorithm calculates f(B) and f(F).
 f(B) = 6 + 8 = 14
 f(F) = 3 + 6 = 9
Since f(F) < f(B), so it decides to go to node F.
Path- A → F
Step-02: Node G and Node H can be reached from node F.
A* Algorithm calculates f(G) and f(H).
 f(G) = (3+1) + 5 = 9
 f(H) = (3+7) + 3 = 13
Since f(G) < f(H), so it decides to go to node G.
Path- A → F → G
Step-03: Node I can be reached from node G.
A* Algorithm calculates f(I).
 f(I) = (3+1+3) + 1 = 8
It decides to go to node I.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Path- A → F → G → I
Step-04: Node E, Node H and Node J can be reached from node I.
A* Algorithm calculates f(E), f(H) and f(J).
 f(E) = (3+1+3+5) + 3 = 15
 f(H) = (3+1+3+2) + 3 = 12
 f(J) = (3+1+3+3) + 0 = 10
Since f(J) is least, so it decides to go to node J.
Path- A → F → G → I → J
This is the required shortest path from node A to node J.

Example – 5:
Consider the following graph-

 The numbers written on edges represent the distance between the nodes.
 The numbers written on nodes represent the heuristic value.
 Find the most cost-effective path to reach from start state S to final state G using A* Algorithm.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Solution:
Step-1:

Here f(A) < f(D), so we explore node A.

Step-2:

Here in all leaf nodes f(B) is having the minimum fitness value, so we explore node B.
Step-3:

Here in all leaf nodes f(D), which is direct successor of initial node S is having the minimum fitness
value, so we explore node D (in that direction).
Step-4:

Here in all leaf nodes f(E) (in the direction SDE), is having the minimum fitness value, so we
explore node E (in that direction: SDE).

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Step-5:

Here in all leaf nodes f(F) (in the direction SDEF), is having the minimum fitness value, so we
explore node F (in that direction: SDEF).

Here we reached the goal state in the direction (SDEFG)

BEYOND CLASSICAL SEARCH:

Hill Climbing Algorithm in Artificial Intelligence:

o Hill climbing algorithm is a local search algorithm which continuously moves in the direction of
increasing elevation/value to find the peak of the mountain or best solution to the problem. It
terminates when it reaches a peak value where no neighbor has a higher value.
o Hill climbing algorithm is a technique which is used for optimizing the mathematical problems. One
of the widely discussed examples of Hill climbing algorithm is Traveling-salesman Problem in
which we need to minimize the distance traveled by the salesman.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 o It is also called greedy local search as it only looks to its good immediate neighbor state and not
beyond that.
o A node of hill climbing algorithm has two components which are state and value.
o Hill Climbing is mostly used when a good heuristic is available.
o In this algorithm, we do not need to maintain and handle the search tree or graph as it only keeps a
single current state.

Features of Hill Climbing:

Following are some main features of Hill Climbing Algorithm:

o Generate and Test variant: Hill Climbing is the variant of Generate and Test method. The
Generate and Test method produce feedback which helps to decide which direction to move in the
search space.
o Greedy approach: Hill-climbing algorithm search moves in the direction which optimizes the cost.
o No backtracking: It does not backtrack the search space, as it does not remember the previous
states.
State-space Diagram for Hill Climbing:

 The state-space landscape is a graphical representation of the hill-climbing algorithm which is

showing a graph between various states of algorithm and Objective function/Cost.
 On Y-axis we have taken the function which can be an objective function or cost function, and state-
space on the x-axis. If the function on Y-axis is cost then, the goal of search is to find the global
minimum and local minimum.
 If the function of Y-axis is Objective function, then the goal of the search is to find the global
maximum and local maximum.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Different regions in the state space landscape:

 Local Maximum: Local maximum is a state which is better than its neighbor states, but there is also
another state which is higher than it.
 Global Maximum: Global maximum is the best possible state of state space landscape. It has the
highest value of objective function.
 Current state: It is a state in a landscape diagram where an agent is currently present.
 Flat local maximum: It is a flat space in the landscape where all the neighbor states of current states
have the same value.
 Shoulder: It is a plateau region which has an uphill edge.

Types of Hill Climbing Algorithm:

1. Simple hill Climbing:
2. Steepest-Ascent hill-climbing:
3. Stochastic hill Climbing:

1. Simple Hill Climbing:

 Simple hill climbing is the simplest way to implement a hill climbing algorithm. It only evaluates
the neighbor node state at a time and selects the first one which optimizes current cost and set
it as a current state.
 It only checks it's one successor state, and if it finds better than the current state, then move else be
in the same state.
 This algorithm has the following features:

o Less time consuming

o Less optimal solution and the solution is not guaranteed.

Algorithm for Simple Hill Climbing:

o Step 1: Evaluate the initial state, if it is goal state then return success and Stop.
o Step 2: Loop Until a solution is found or there is no new operator left to apply.
o Step 3: Select and apply an operator to the current state.
o Step 4: Check new state:
1. If it is goal state, then return success and quit.
2. Else if it is better than the current state then assign new state as a current state.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 3. Else if not better than the current state, then return to step2.
o Step 5: Exit.

2. Steepest-Ascent hill climbing:

 The steepest-Ascent algorithm is a variation of simple hill climbing algorithm.

 This algorithm examines all the neighboring nodes of the current state and selects one neighbor node
which is closest to the goal state.
 This algorithm consumes more time as it searches for multiple neighbors.

Algorithm for Steepest-Ascent hill climbing:

o Step 1: Evaluate the initial state, if it is goal state then return success and stop, else make current state as initial
state.
o Step 2: Loop until a solution is found or the current state does not change.
1. Let SUCC be a state such that any successor of the current state will be better than it.
2. For each operator that applies to the current state:
I. Apply the new operator and generate a new state.
II. Evaluate the new state.
III. If it is goal state, then return it and quit, else compare it to the SUCC.
IV. If it is better than SUCC, then set new state as SUCC.
V. If the SUCC is better than the current state, then set current state to SUCC.

o Step 3: Exit.

3. Stochastic hill climbing:

 Stochastic hill climbing does not examine for all its neighbor before moving.
 Rather, this search algorithm selects one neighbor node at random and decides whether to choose it as
a current state or examine another state.

Problems in Hill Climbing Algorithm:

1. Local Maximum: A local maximum is a peak state in the landscape, which is better than each of its
neighboring states, but there is another state also present which is higher than the local maximum.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Solution: Backtracking technique can be a solution of the local maximum in state space landscape.
Create a list of the promising path so that the algorithm can backtrack the search space and explore
other paths as well.

2. Plateau: A plateau is the flat area of the search space in which all the neighbor states of the current
state contain the same value, because of this algorithm does not find any best direction to move. A
hill-climbing search might be lost in the plateau area.

Solution: The solution for the plateau is to take big steps or little steps while searching, to solve the
problem. Randomly select a state which is far away from the current state so it is possible that the
algorithm could find non-plateau region.

3. Ridges: A ridge is a special form of the local maximum. It has an area which is higher than its
surrounding areas, but itself has a slope, and cannot be reached in a single move.

Solution: With the use of bidirectional search, or by moving in different directions, we can improve
this problem.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
SIMULATED ANNEALING:

 A hill-climbing algorithm which never makes a move towards a lower value guaranteed to be
incomplete because it can get stuck on a local maximum. And if algorithm applies a random walk,
by moving a successor, then it may complete but not efficient. Simulated Annealing is an algorithm
which yields both efficiency and completeness.
 In mechanical term Annealing is a process of hardening a metal or glass to a high temperature then
cooling gradually, so this allows the metal to reach a low-energy crystalline state.
 The same process is used in simulated annealing in which the algorithm picks a random move,
instead of picking the best move. If the random move improves the state, then it follows the same
path. Otherwise, the algorithm follows the path which has a probability of less than 1 or it moves
downhill and chooses another path.
 In every simulated annealing example, a random new point is generated. The distance between the
current point and the new point has a basis of the probability distribution on the scale of the
proportion of temperature.

Implement Simulated Annealing

There are a set of steps that are performed for simulated annealing in AI. These steps can be summarized as
follows:

 Simulated annealing creates a trial point randomly. The algorithm selects the distance between the
current point and the trial point by a probability distribution. The scale of such distribution is
temperature. With the annealing function trial, point distribution distance is set. To keep the
boundaries intact, the trial point is shifted gradually.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  The Simulated Annealing formula then determines if the new point is better than the older or not. If
the new point is better, it is made as a next point, while if the new point is worse, it can still be
accepted depending upon the simulated annealing acceptance function.
 A systematic algorithm gradually reduces the temperature selecting the best point that gets generated
in the process.
 For lowering the values, the annealing parameters are set, raising and reducing the temperatures. The
simulated annealing parameters are based on the values of the probable gradients of every dimension
of the objective.
 The simulated annealing is concluded when it reaches the lowest minima or any of the specific
stopping criteria.

Stopping Criteria of Simulated annealing

Some of the conditions that are considered as the basis to stop the simulated-annealing are as follows:

 The simulated-annealing performs until the value of the objective function goes lesser than the
tolerance function.
 The default value of iterations in simulated-annealing is INF. This can be set to any positive integer
as well. When the algorithm exceeds the iteration value, it stops.
 The annealing concludes when the maximum number of evaluations is achieved. The default value
of such evaluations is 3000 * number of variables.
 The default value of maximum time is Inf, We can set a constant value here and when that is
reached, the algorithm stops.
 When the best objective function value is lesser than the limit of the objective it concludes. The
default value of such an objective function is -Inf.

Simulated Annealing Worked Example

To understand how simulated-annealing works, one can take the example of a traveling salesman. The
solution can be created by applying any of the language selections.

Let us understand the problem and the solution with simulated-annealing applications.

 At the onset, a city class needs to be created to specify several destinations the travelling salesman
would visit.
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  After that, a class has to be created that keeps track of the cities.
 Then a class is created that models the tour of the travelling salesman.
 With all the different classes and the information in hand, a simulated-annealing algorithm is
created.
 Thus with the types of optimization problems, a relatively simpler algorithm is created, and the
solution is sought.

Simulated annealing vs hill-climbing methods

 There is a huge difference between hill-climbing and simulated-annealing considering the way they
are applied, and the results are achieved.
 Simulated-annealing is believed to be a modification or an advanced version of hill-climbing
methods.
 Hill climbing achieves optimum value by tracking the current state of the neighborhood.
 Simulated-annealing achieves the objective by selecting the bad move once a while.
 A global optimum solution is guaranteed with simulated-annealing, while such a guarantee is not
assured with hill climbing or descent.

Local beam search

 Keeping just one node in memory might seem to be an extreme reaction to the problem of memory
limitations.
 The local beam search algorithm keeps track of k states rather than just one.
 It begins with k randomly generated states.
 At each step, all the successors of all k states are generated.
 If anyone is a goal, the algorithm halts. Otherwise, it selects the k best successors from the complete
list and repeats.
 At first sight, a local beam search with k states might seem to be nothing more than running k
random restarts in parallel instead of in sequence.
 In fact, the two algorithms are quite different.
 In a random-restart search, each search process runs independently of the others.
 In a local beam search, useful information is passed among the parallel search threads.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Genetic algorithms

 A genetic algorithm (or GA) is a variant of stochastic beam search in which successor states are
generated by combining two parent states rather than by modifying a single state.
 The analogy to natural selection is the same as in stochastic beam search, except that now we are
dealing with genetic rather than non-genetic reproduction. Like beam searches, GAs begin with a set
of k randomly generated states, called the population.
 Each state, or individual, is represented as a string over a finite alphabet—most commonly, a string
of 0s and 1s. For example, an 8-queens state must specify the positions of 8 queens, each in a
column of 8 squares, and so requires 8 × log2 8 = 24 bits.
 Alternatively, the state could be represented as 8 digits, each in the range from 1 to 8. (We
demonstrate later that the two encodings behave differently.) The following figure shows a
population of four 8-digit strings representing 8-queens states.
 The production of the next generation of states is also shown in the figure(a). In (b), each state is
rated by the objective function, or (in GA terminology) the fitness function.
 A fitness function should return higher values for better states, so, for the 8-queens problem we use
the number of non-attacking pairs of queens, which has a value of 28 for a solution.
 The values of the four states are 24, 23, 20, and 11. In this particular variant of the genetic algorithm,
the probability of being chosen for reproducing is directly proportional to the fitness score, and the
percentages are shown next to the raw scores.
 In (c), two pairs are selected at random for reproduction, in accordance with the probabilities in (b).
Notice that one individual is selected twice and one not at all. For each pair to be mated, a crossover
point is chosen randomly from the positions in the string. In Figure 4.6, the crossover points are after
the third digit in the first pair and after the fifth digit in the second pair.
 In (d), the offspring themselves are created by crossing over the parent strings at the crossover point.
For example, the first child of the first pair gets the first three digits from the first parent and the
remaining digits from the second parent, whereas the second child gets the first three digits from the
second parent and the rest from the first parent. The 8-queens states involved in this reproduction
step are shown in the figure.
 The example shows that when two parent states are quite different, the crossover operation can
produce a state that is a long way from either parent state. It is often the case that the population is
quite diverse early on in the process, so crossover (like simulated annealing) frequently takes large

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 steps in the state space early in the search process and smaller steps later on when most individuals
are quite similar.

 Finally, in (e), each location is subject to random mutation with a small independent probability. One
digit was mutated in the first, third, and fourth offspring. In the 8-queens problem, this corresponds
to choosing a queen at random and moving it to a random square in its column.
 In practice, genetic algorithms have had a widespread impact on optimization problems, such as
circuit layout and job-shop scheduling. At present, it is not clear whether the appeal of genetic
algorithms arises from their performance or from their esthetically pleasing origins in the theory of
evolution. Much work remains to be done to identify the conditions under which genetic algorithms
perform well.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
LOCAL SEARCH IN CONTINUOUS SPACES
 Environments can be discrete or continuous, pointing out that most real-world environments are
continuous.
 This section provides a very brief introduction to some local search techniques for finding optimal
solutions in continuous spaces.
 To perform local search in continuous state spaces, we need techniques from calculus. The main
technique to find a minimum is called gradient descent (or gradient ascent to find the maximum).

What is optimization?
 Optimization refers to the task of minimizing or maximizing an objective function f(x) w.r.t weights.

What is Gradient Descent algorithm?

 Gradient descent is a first-order iterative optimization algorithm for finding a local minimum of a
differentiable function.
 Gradient Descent is an optimization Algorithm for finding/Updating weights/Coefficients of the
Algorithms.
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  Gradient descent algorithm is an iterative process that takes us to the minimum of a function. It is
used to minimize a cost function (Error).
 It is a first-order optimization algorithm. This means it only takes into account the first derivative
when performing the updates on the parameters.
 On each iteration, we update the parameters in the opposite direction of the gradient of the objective
function with respect to the parameters where the gradient gives the direction of the steepest ascent.
 The size of the step we take on each iteration to reach the local minimum is determined by the
learning rate α. Therefore, we follow the direction of the slope downhill until we reach a local
minimum.
 If we want a local maximum value, at that time use gradient ascent algorithm (Which will work
exactly opposite to gradient descent).

 In the above figure at point c, the slope of the gradient will be negative and at point a, the slope of
the gradient will be positive.
 When the slope is exactly zero, at that time we can say it is a minimum cost function. It is simply
that the derivative of the cost function will be zero at minimum point.

 Gradient is nothing but the partial derivative of the cost function, simply find the first derivative of
the cost function to get the minimum and maximum points. And to calculate the new weight, simply
use the below equation.
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  The –ve sign is used to find the new weight in case of gradient descent and the +ve sign is used to
the new weight in case of gradient ascent.

Gradient Descent

 What is the gradient of a function f(x)?

- Usually written as

- ∇f(x) (the gradient itself) represents the direction of the steepest slope

- |∇f(x)| (the magnitude of the gradient) tells you how big the steepest slope is

 Suppose we want to find a local minimum of a function f(x). We use the gradient descent rule:

x←x – α∇f(x)

 Suppose we want to find a local maximum of a function f(x). We use the gradient ascent rule:

x←x + α∇f(x)

 If α is too large
 Gradient descent overshoots the optimum point
 If α is too small
 Gradient descent requires too many steps and will take a very long time to converge
α is the learning rate, which is usually a small number like 0.05

Multivariate Gradient Descent

What happens if your function is multivariate ex., f(x1,x2,x3)?

 Then

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  The gradient descent rule becomes:

SEARCHING WITH NONDETERMINISTIC ACTIONS

 When the environment is fully observable and deterministic then the agent knows what the effects of
each action are. In such case the percepts provide no new information after each action. Percepts
become useful in partially observable or nondeterministic environments
 In a partially observable environment, the percept helps narrow down the set of possible states and
make it easier to achieve the goal.
 When the environment is nondeterministic, percepts tell the agent which of the possible outcomes of
its actions has actually occurred.
 The solution to a problem is not a sequence but a strategy that specifies what to do depending on
what percepts are received.

Non deterministic Environment

Example: Vacuum World (actions = {left, right, suck})

 Modify the Vacuum World to Erratic Vacuum World which is nondeterministic

 In the erratic vacuum case, the result of an action can vary.
 When sucking a dirty square, it cleans it and sometimes cleans up dirt in an adjacent square.
 When sucking a clean square, it sometimes deposits dirt on the carpet.
 For such problems we need to generalization the state-space model

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Generalization of State-Space Model
 In deterministic environment the transition model is defined by a RESULT function that returns a
single state.
 In nondeterministic case RESULTS function returns a set of possible outcome states
 Generalize the transition function to return a set of possible outcomes.
 oldf: S x A→S newf: S x A → 2s
 For example, in the erratic vacuum world, the Suck action in state 1 leads to a state in the set {5, 7}
 Generalize the solution to a contingency plan.
 [Suck, if State = 5 then [Right, Suck] else [ ]].
 Generalize the search tree to an AND-OR search tree.

AND – OR search trees

 Branching introduced by the agent's own choices in each state results in OR nodes.
 Branching introduced by the environment’s choice of outcome for each action results in AND nodes.
 A solution for an AND-OR search problem is a subtree that
 has a goal node at every leaf,
 specifies one action at each of its OR nodes
 includes every outcome branch at each of its AND nodes.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
AND – OR search trees (Algorithm)

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Best-first search is what the AO* algorithm does.
 The AO* method divides any given difficult problem into a smaller group of problems that are
then resolved using the AND-OR graph concept.
 AND OR graphs are specialized graphs that are used in problems that can be divided into smaller
problems.
 The AND side of the graph represents a set of tasks that must be completed to achieve the main goal,
while the OR side of the graph represents different methods for accomplishing the same main goal.

 In the above figure, the buying of a car may be broken down into smaller problems or tasks that can
be accomplished to achieve the main goal in the above figure, which is an example of a simple
AND-OR graph.
 The other task is to either steal a car that will help us accomplish the main goal or use your own
money to purchase a car that will accomplish the main goal.
 The AND symbol is used to indicate the AND part of the graphs, which refers to the need that all
sub problems containing the AND to be resolved before the preceding node or issue may be finished.
 The start state and the target state are already known in the knowledge-based search strategy known
as the AO* algorithm, and the best path is identified by heuristics.
 The informed search technique considerably reduces the algorithm’s time complexity.

Working of AO* algorithm:

The evaluation function in AO* looks like this:
f(n) = g(n) + h(n)
f(n) = Actual cost + Estimated cost
here, f(n) = The actual cost of traversal.
g(n) = the cost from the initial node to the current node.
h(n) = estimated cost from the current node to the goal state.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Difference between the A* Algorithm and AO* algorithm
 A* algorithm and AO* algorithm both works on the best first search.
 They are both informed search and works on given heuristics values.
 A* always gives the optimal solution but AO* doesn’t guarantee to give the optimal solution.
 Once AO* got a solution doesn’t explore all possible paths but A* explores all paths.
 When compared to the A* algorithm, the AO* algorithm uses less memory.
 Opposite to the A* algorithm, the AO* algorithm cannot go into an endless loop.

Example:

Here in the above example below the Node which is given is the heuristic value i.e h(n). Edge length is
considered as 1.

Step – 1:
With help of f(n) = g(n) + h(n) evaluation function,
Start from node A,
f(A⇢B) = g(B) + h(B)
=1 + 5 ……here g(n)=1 is
taken by default for path cost
=6
f(A⇢C+D) = g(c) + h(c) + g(d) + h(d)
=1+2+1+4 ……here we have
added C & D because they are in AND
=8
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
 So, by calculation A⇢B path is chosen which is
the minimum path, i.e f(A⇢B)

Step – 2:
According to the answer of step 1, explore node B
Here the value of E & F are calculated as follows.
f(B⇢E) = g(e) + h(e)
f(B⇢E) = 1 + 7 = 8
f(B⇢f) = g(f) + h(f)
f(B⇢f) = 1 + 9 = 10
o So, by above calculation B⇢E path is chosen
which is minimum path, i.e f(B⇢E) because
B's heuristic value is different from its actual
value The heuristic is updated and the
minimum cost path is selected. The
minimum value in our situation is 8.
o Therefore, the heuristic for A must be
updated due to the change in B's heuristic.
o So we need to calculate it again.
f(A⇢B) = g(B) + updated h(B)
=1+8
=9
We have Updated all values in the above tree.

Step – 3:

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 By comparing f(A⇢B) & f(A⇢C+D)
f(A⇢C+D) is shown to be smaller. i.e 8 < 9
Now explore f(A⇢C+D) So, the current node is
C.
f(C⇢G) = g(g) + h(g)
f(C⇢G) = 1 + 3 = 4

f(C⇢H+I) = g(h) + h(h) + g(i) + h(i)

f(C⇢H+I) = 1 + 0 + 1 + 0 = 2
(here we have added H & I because they are in
AND)
f(C⇢H+I) is selected as the path with the lowest cost and the heuristic is also left unchanged because it
matches the actual cost. Paths H & I are solved because the heuristic for those paths is 0, but Path A⇢D
needs to be calculated because it has an AND.
f(D⇢J) = g(j) + h(j)
f(D⇢J) = 1 + 0 = 1
The heuristic of node D needs to be updated to 1.
f(A⇢C+D) = g(c) + h(c) + g(d) + h(d)
=1+2+1+1=5

 As we can see that path f(A⇢C+D) is get solved and this tree has become a solved tree now.
 In simple words, the main flow of this algorithm is that we have to find firstly level 1st heuristic
value and then level 2nd and after that update the values with going upward means towards the root
node.
 In the above tree diagram, we have updated all the values.

Partially observable environment

 Partial observability: The agent's percepts do not suffice to pin down the exact state
 In such environment an action may lead to one of several possible outcomes even if the environment
is deterministic.
 To solve partially observable problems the concept of the "belief state" is used.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  The belief state represents the agent's current belief about the possible physical states it might be in,
given the sequence of actions and percepts up to that point
 The belief state concept can be studied in two different scenarios;
o Searching with no observation
o Searching with observations

Searching with no observation

 Sensor-less or conformant agents: percepts provide no information at all
 They are surprisingly useful, because they don't rely on sensors working properly
 Sensor-less Vacuum world (geography known, not location & dirt distribution)
o Initial state could be any element of the set {1, 2, 3, 4, 5, 6, 7, 8}
o Action right will cause it to be in one of the states {2, 4, 6, 8}
o The action sequence [Right, Suck] will always end up in one of the states {4, 8}.
o Sequence [Right, Suck, Left, Suck] is guaranteed to reach the goal state 7 irrespective of the
start state
 To solve sensor-less problems, space of belief-states is searched rather than physical
 Believe-state space is fully observable as it is always known to the agent

 Suppose the underlying physical problem P is defined by ACTIONSp, RESULTp, GOAL-TESTp,

and STEP-COSTp.
 Belief states: If P has N states, then the sensor-less problem has up to 2N states
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  Initial state: Typically the set of all states in P
 Actions: If a belief state b={s1, s2}, and ACTIONSp(s1) ≠ ACTIONSp(s2); legal actions?
o Take union of all the actions, given that illegal action has no effect on the
environment
o Otherwise intersection, i.e. the set of actions legal in all the states (illegal action
might be the end of the world)
 Transition model: The agent doesn’t know which state in the belief state is the right one; so
as far as it knows, it might get to any of the states resulting from applying the action to one of
the physical states in the belief state.
o with deterministic actions, the set of states that might be reached is

o with non-deterministic actions

 Goal test: The agent wants a plan that is sure to work, which means that a belief state
satisfies the goal only if all the physical states in it satisfy GOAL-TESTP . The agent may
accidentally achieve the goal earlier, but it won’t know that it has done so.
 Path cost: Same action can have different costs in different states of a belief state, an
assumption can make the formulation easy, i.e. the cost of an action is the same in all states

Searching with observation

 In partial observations, usually several states could have produced any given percept
o The percept [A, Dirty] is produced by state 3 as well as by state 1 in local-sensing vacuum
(where only a position sensor and a local dirt sensor but no global dirt sensor)
o In such case the initial belief state for the local-sensing vacuum world will be {1,3}
o ACTIONS, STEP-COST, and GOAL-TEST are same as for sensor-less problems

 Transition model is a bit different and consists of three stages

o Prediction: given the action a in belief state b, b'= PREDICT(b, a), same as for sensor-less
o Observation prediction: set of percepts that could be observed in the predicted belief state

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 POSSIBLE-PERCEPTS(b) = {o: o = PERCEPT(s) and s ∈ b'}

o Update: for each possible percept, the belief state that would result from the percept

bo= UPDATE(b, o)={s: o =PERCEPT(s) and s ∈ b} updated belief state bo can be no larger
than the predicted belief state b`;

 Putting these three stages together, to get the transition model

RESULTS(b, a)={bo : bo= UPDATE(PREDICT(b, a), o) and
POSSIBLE-PERCEPTS(PREDICT(b, a))}
 The next belief state is achieved a given action a the subsequent possible percepts
 Figure a, shows transition model for local-sensor vacuum world
 Figure b, shows transition model for slippery vacuum world

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Solving partially observable problems
 Given the formulation discussed previously, the AND-OR search algorithm can be applied directly
to derive a solution
 The solution is the conditional plan that tests the belief state rather than the actual state
 The first level of the AND-OR search tree for a problem in the local-sensing vacuum is

Online Search Problems

 The offline search algorithms compute a complete solution before setting foot in the real world and
then execute the solution
 Online search takes an action, then it observes the environment and computes the next suitable for
dynamic or semi-dynamic domains where there is a penalty for sitting around and computing too
long
 Also useful for
o Unknown environments
o Non-deterministic domains
 Examples
o Web search
o Autonomous vehicle
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
Online Search Agents And Unknown Environments

 An agent is anything that can perceive its environment through sensors and acts upon that
environment through Actuators(effectors).
 Offline Search Agents- The agents who compute a complete solution before setting foot in the real
world and then execute the solution is called Offline search Agents.
 Online Search Agents- The online search works on the concept of interleave computation and
action.
 In the real world, Online Search Agent first takes an action, then it observes the environment and
computes the next action.
 Online search is a good idea in unknown environments, where the agent does not know what states
exist or what its actions do.

Online Search Agents and Unknown Environments

Online search problem – Assume deterministic (If the next state is completely determined by the current
action of the agent, then the environment is deterministic.), fully observable environment.
Agent only knows:
 Actions(s) - list of actions allowed in state s.
 Step-cost function c(s, a, s') - cannot be used until agent knows that s' is the outcome of doing a.
 Goal-Test(s) - In particular, it doesn't know Result (s, a) except by actually being in s and doing a,
then observing s'.
s = state
a = action
s' = state after doing action a
c = Step cost function

 In this Maze, the agent start with 'S' state i.e. (1, 1) and then leads to (1, 2), likewise use up leading it
then achieves goal state 'G’.
 The going down will lead it to (1, 1) i.e. start state.
 The degree of achieving goal state fast can be improved.
Competitive ratio = Online cost/Best cost.
Online path cost: total cost of the path that the agent actually travels.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 Best cost: cost of the shortest path "if it knew the search space in advance" Competitive ratio
means shortest path. The competitive ratio must be very small.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]