Artificial Intelligence Notes

UNIT – I :
Problem solving by search -I: Introduction to AI , Intelligent Agents.
Problem solving by search -II: Problem-Solving Agents,
Introduction:
Artificial Intelligence is concerned with the design of intelligence in an artificial device
or machine to solve by logically, reasonably within time without error accurately,
efficiently to provide solution with respective problem .

The term was coined by John McCarthy in 1956. It is the ability to acquire,
understand and apply the knowledge to achieve goals in the world.

AI is the study of the mental faculties through the use of computational models and
AI is the study of intellectual/mental processes as computational processes.
AI program will demonstrate a high level of intelligence to a degree that equals or
exceeds the intelligence required of a human in performing some task.
AI is unique, sharing borders with Mathematics, Computer Science, Philosophy,
Psychology, Biology, Cognitive Science and many others. Although there is no clear
definition of AI or even Intelligence, it can be described as an attempt to build machines
that like humans can think and act, able to learn and use knowledge to solve problems
on their own.

Foundations of Artificial Intelligence:

• Philosophy : Ex..., foundational issues (can a machine think?), issues of

knowledge and believe, mutual knowledge
• Psychology and Cognitive Science : Ex..., problem solving skills
• Neuro-Science : Ex..., brain architecture
• Computer Science And Engineering : Ex..., complexity theory, algorithms, logic
and inference, programming languages, and system building.
• Mathematics and Physics : Ex..., statistical modeling, continuous mathematics,
Statistical Physics, and Complex Systems.

Sub Areas of AI:

1) Game Playing: -Deep Blue Chess program beat world champion Gary Kasparov
2) Speech Recognition:- The 1990s has seen significant advances in speech recognition so that
limited systems are now successful.
3) Computer Vision:- Face recognition programs in use by banks, government, etc. 4) Expert
Systems:- Application-specific systems that rely on obtaining the knowledge of 5)
Mathematical Theorem Proving: Use inference methods to prove new theorems. 6) Natural
Language Understanding : AltaVista's translation of web pages. Translation of Caterpillar
Truck manuals into 20 languages.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 7) Scheduling and Planning : Automatic scheduling for manufacturing.
8) Artificial Neural Networks.
9) Machine Learning.

Uses or Applications of AI:

Education:
AI can automate grading so that the tutor can have more time to teach. AI chatbot can
communicate with students as a teaching assistant.
AI in the future can be work as a personal virtual tutor for students, which will be accessible
easily at any time and any place.

Manufacturing :
Some Automotive industries are using AI to provide virtual assistant to their user for better
performance. Such as Tesla has introduced Tesla Bot, an intelligent virtual assistant. Various
Industries are currently working for developing self-driven cars which can make your journey more
safe and secure.

Travel & Transport:

AI is becoming highly demanding for travel industries. AI is capable of doing various travel
related works such as from making travel arrangement to suggesting the hotels, flights, and best
routes to the customers Marketing or E-commerce:
AI is providing a competitive edge to the e-commerce industry, and it is becoming more
demanding in the e-commerce business. AI is helping shoppers to discover associated products
with recommended size, color, or even brand.
Finance:
AI and finance industries are the best matches for each other. The finance industry is implementing
automation, chatbot, adaptive intelligence, algorithm trading, and machine learning into financial
processes. Astronomy:
Artificial Intelligence can be very useful to solve complex universe problems. AI technology can be
helpful for understanding the universe such as how it works, origin, etc.
Healthcare:
Healthcare Industries are applying AI to make a better and faster diagnosis than humans. AI
can help doctors with diagnoses and can inform when patients are worsening so that medical help
can reach to the patient before hospitalization.
Gaming:
AI can be used for gaming purpose. The AI machines can play strategic games like chess, where
the machine needs to think of a large number of possible places.
Data Security:
The security of data is crucial for every company and cyber-attacks are growing very rapidly
in the digital world. AI can be used to make your data more safe and secure., AI Platform is used to
determine software bug and cyber-attacks in a better way.
Social Media:
AI can organize and manage massive amounts of data. AI can analyze lots of data to identify
the latest trends, hashtag, and requirement of different users in Social Media sites such as
Facebook, Twitter.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 Robotics:
Artificial Intelligence has a remarkable role in Robotics. Usually, general robots are
programmed such that they can perform some repetitive task, but with the help of AI, we can
create intelligent robots which can perform tasks with their own experiences without
preprogrammed.
Customer service or Entertainment:
We are currently using some AI based applications in our daily life with some entertainment
services such as Netflix or Amazon. With the help of ML/AI algorithms, these services show the
recommendations for programs or shows.
Agriculture:
Agriculture is applying AI as agriculture robotics, solid and crop monitoring, predictive
analysis. AI in agriculture can be very helpful for farmers.

Building of AI Systems:
1) Perception:
Intelligent biological systems are physically embodied in the world and experience the world
through their sensors (senses).
For an autonomous vehicle, input might be images from a camera and range information from
a rangefinder.
2) Reasoning :
Inference, decision-making, classification from what is sensed and what the internal "model"
is of the world. Includes areas of knowledge representation, problem solving, decision theory,
planning, game theory, machine learning, uncertainty reasoning, etc.
3) Action:
Biological systems interact within their environment by actuation, speech, etc. All behavior is
centered around actions in the world. Includes areas of robot actuation, natural language
generation, and speech synthesis.

Intelligent Systems:
In order to design intelligent systems, it is important to categorize them into 4 categories such as.
1. Systems that think like humans
2. Systems that think rationally
3. Systems that behave like humans
4. Systems that behave rationally

1.Systems That Think Like Humans

Requires a model for human cognition. Precise enough models allow simulation by computers.
• Focus is not just on behavior and I/O, but looks like reasoning process.
• Goal is not just to produce human-like behavior but to produce a sequence of steps of the
reasoning process, similar to the steps followed by a human in solving the same task.

2.Systems That Think Rationally :

The study of mental faculties through the use of computational models; that it is, the study of
computations that make it possible to perceive reason and act.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 • Focus is on inference mechanisms that are probably correct and guarantee an optimal
solution.
• Goal is to formalize the reasoning process as a system of logical rules and procedures of
inference.
• Develop systems of representation to allow inferences to be like “Socrates is a man. All men
are mortal. Therefore Socrates is mortal”

3.Systems That Behave Like Humans:

The art of creating machines that perform functions requiring intelligence when performed by
people; that it is the study of, how to make computers do things which, at the moment, people do
better.
• Focus is on action, and not intelligent behavior centered around the representation of the
world.
.
4.Systems that behave rationally:
Tries to explain and emulate intelligent behavior in terms of computational process; that it is
concerned with the automation of the intelligence.
• Focus is on systems that act sufficiently if not optimally in all situations.
• Goal is to develop systems that are rational and sufficient

Intelligent Agents:

Agents in Artificial Intelligence:

An AI system can be defined as the study of the rational agent and its environment.
The agents sense the environment through sensors and act on their environment through
actuators. An AI agent can have mental properties such as knowledge, belief, intention,
etc.
An agent can be anything that perceive its environment through sensors and act
upon that environment through actuators. An Agent runs in the cycle of perceiving,
thinking, and acting. An agent can be:

1. Human-Agent: A Human Agent has eyes, ears, and other organs which work for
sensors and hand, legs, vocal tract work for actuators.
2. Robotic Agent: A Robotic Agent can have cameras, infrared range finder, NLP for
sensors and various motors for actuators.
3. Software Agent: Software Agent can have keystrokes, file contents as sensory
input and act on those inputs and display output on the screen.
Hence the world around us is full of agents such as thermostat, cellphone, camera, and
even we are also agents.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 Effectors

Sensor: Sensor is a device which detects the change in the environment and sends the
information to other electronic devices. An agent observes its environment through
sensors.
Actuators: Actuators are the component of machines that converts energy into motion.
The actuators are only responsible for moving and controlling a system. An actuator can
be an electric motor, gears, rails, etc.

Effectors: Effectors are the devices which affect the environment. Effectors can be legs,
wheels, arms, fingers, wings, fins, and display screen.

An intelligent agent is an autonomous entity which act upon an environment using

sensors and actuators for achieving goals. An intelligent agent may learn from the
environment to achieve their goals.
Following are the main four rules for an AI agent:

Rule 1: An AI agent must have the ability to perceive the environment.

Rule 2: The observation must be used to make decisions.
Rule 3: Decision should result in an action.
Rule 4: The action taken by an AI agent must be a rational action.

Structure of an AI Agent
The task of AI is to design an agent program which implements the agent function.
The structure of an intelligent agent is a combination of architecture and agent program. It
can be viewed as:

Agent = Architecture + Agent program

Types of Classes of AI Agents:

AI notes Btech(AIML) 3-2: by V. Ravindranath

 Agents can be grouped into Five Classes Based On Their Degree of perceived
intelligence and capability. All these agents can improve their performance and generate
better action over the time. These are given below:
1. Simple Reflex Agent
2. Model-based reflex agent
3. Goal-based agents
4. Utility-based agent
5. Learning agent.

1. Simple Reflex agent:

• The Simple reflex agents are the simplest agents. These agents take decisions
on the basis of the current percepts and ignore the rest of the percept history.
• These agents only succeed in the fully observable environment.
• The Simple reflex agent works on Condition-action rule, which means it maps
the current state to action. Such as a Room Cleaner agent, it works only if there
is dirt in the room.
• Problems for the simple reflex agent design approach:

• They have very limited intelligence

• They do not have knowledge of non-perceptual parts of the current state
• Mostly too big to generate and to store.
• Not adaptive to changes in the environment.

2. Model-based reflex agent

• The Model-based agent can work in a partially observable environment, and track the
situation.
• A model-based agent has 2 important factors:
1) Model: It is knowledge about "how things happen in the world," so it is called a
Model-based agent.
2) Internal State: It is a representation of the current state based on percept
history.
• These agents have the model, "which is knowledge of the world" and based on the
model they perform actions.
• Updating the agent state requires information about:
• How the world evolves
• How the agent's action affects the world.
3.Goal-based agents:

• The knowledge of the current state environment is not always sufficient to decide for an
agent to what to do. needs to know its goal which describes desirable situations.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 • Goal-based agents expand the capabilities of the model-based agent by having the
"goal" information.
• They choose an action, so that they can achieve the goal.
• These agents may have to consider a long sequence of possible actions before
deciding whether the goal is achieved or not. Such considerations of different scenario
are called searching and planning, which makes an agent proactive.

4. Utility-based agents:
These agents are similar to the goal-based agent but provide an extra component of utility
measurement which makes them different by providing a measure of success at a given
state.
• Utility-based agent act based not only goals but also the best way to achieve the goal.
• The Utility-based agent is useful when there are multiple possible alternatives, and an
agent has to choose in order to perform the best action.
• The utility function maps each state to a real number to check how efficiently each
action achieves the goals.

5. Learning Agents
• A learning agent in AI is the type of agent which can learn from its past experiences,
or it has learning capabilities.
• It starts to act with basic knowledge and then able to act and adapt automatically
through learning.
• A learning agent has mainly four conceptual components, which are:
a. Learning element: It is responsible for making improvements by learning
from environment
b. Critic: Learning element takes feedback from critic which describes that how
well the agent is doing with respect to a fixed performance standard.
c. Performance element: It is responsible for selecting external action
d. Problem generator: This component is responsible for suggesting actions that will
lead to new and informative experiences.

Problem Solving Agents:

In Artificial Intelligence, Search techniques are universal problem-solving methods.
Rational agents or Problem-solving agents in AI mostly used these search strategies or
algorithms to solve a specific problem and provide the best result. Problem-solving agents
are the goal-based agents and use atomic representation. In this topic, we will learn
various problem-solving search algorithms.

There are basically three types of problem in artificial intelligence:

1. Ignorable: In which solution steps can be ignored.
2. Recoverable: In which solution steps can be undone.
3. Irrecoverable: Solution steps cannot be undo.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 Steps of Algorithm of Problem-Solving In AI:
The problem of AI is directly associated with the nature of humans and their activities.
So we need a number of finite steps to solve a problem which makes human easy works.
These are the following steps which require to solve a problem :
• Problem definition: Detailed specification of inputs and acceptable system
solutions.
• Problem analysis: Analyse the problem thoroughly.
• Knowledge Representation: collect detailed information about the problem and
define all possible techniques.
• Problem-solving: Selection of best techniques.

Components to formulate the associated problem:

• Initial State: This state requires an initial state for the problem which starts the AI
agent towards a specified goal. In this state new methods also initialize problem
domain solving by a specific class.
• search: Searching is a step by step procedure to solve a search-problem in a given
search space. A search problem can have three main factors:
1. Search Space: Search space represents a set of possible solutions, which a
system may have.
2. Start State: It is a state from where agent begins the search.
3. Goal test: It is a function which observe the current state and returns
whether the goal state is achieved or not.
• Search tree: A tree representation of search problem is called Search tree. The root
of the search tree is the root node which is corresponding to the initial state.
• Action: This stage of problem formulation works with function with a specific class
taken from the initial state and all possible actions done in this stage.
• Transition: This stage of problem formulation integrates the actual action done by
the previous action stage and collects the final stage to forward it to their next stage.
• Goal test: This stage determines that the specified goal achieved by the integrated
transition model or not, whenever the goal achieves stop the action and forward into
the next stage to determines the cost to achieve the goal.
• Path costing: This component of problem-solving numerical assigned what will be
the cost to achieve the goal. It requires all hardware software and human working
cost.

Properties of Search Algorithms:

There are 4 essential properties of search algorithms to compare the efficiency of
these algorithms:
1. Completeness: A search algorithm is said to be complete if it guarantees to return
a solution if at least any solution exists for any random input.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 2. Optimality: If a solution found for an algorithm is guaranteed to be the best solution
(lowest path cost) among all other solutions, then such a solution for is said to be an
optimal solution.
3. Time Complexity: Time complexity is a measure of time for an algorithm to
complete its task.
4. Space Complexity: It is the maximum storage space required at any point during
the search, as the complexity of the problem.

Types Of Search Algorithms :

• Based on the search problems we can classify the search algorithms into
uninformed (Blind search) search and informed search (Heuristic search)
algorithms.

AI notes Btech(AIML) 3-2: by V. Ravindranath

 Uninformed Search Strategies:

The search does not contain any domain knowledge such as closeness, the
location of the goal. It operates in a brute-force way as it only includes information about how to
traverse the tree and how to identify leaf and goal nodes is called Uninformed Search.
Uninformed search applies a way in which search tree is searched without any
information about the search space like initial state operators and test for the goal, so it
is also called Blind Search. It examines each node of the tree until it achieves the goal
node. It Can Be Divided Into 6 Main Types:

1. Breadth-first Search
2. Depth-first Search
3. Depth-limited Search
4. Iterative deepening depth-first search
5. Uniform cost search
6. Bidirectional Search

1. Breadth-first Search:
Breadth-first search is the most common search strategy for traversing a tree
or graph. This algorithm searches breadthwise in a tree or graph, so it is
called breadth- first search.
BFS algorithm starts searching from the root node of the tree and expands
all successor node at the current level before moving to nodes of next level.
The breadth-first search algorithm is an example of a general-graph search
algorithm.
Breadth-first search implemented using FIFO queue data structure.
Advantages:
BFS will provide a solution if any solution exists.
If there are more than one solutions for a given problem, then BFS will provide
the minimal solution which requires the least number of steps.
Disadvantages:
* It requires lots of memory since each level of the tree must be saved into
memory to expand the next level.
BFS needs lots of time if the solution is far away from the root node.
Algorithm of Bread first search:
1. Create a variable called NODE-LIST and set it to initial state
2. Until a goal state is found or NODE-LIST is empty do
a. Remove the first element from NODE-LIST and call it E.
If NODE-LIST was empty, Quit
b. For each way that each rule can match the state described in E
do:
i. Apply the rule to generate a new state
ii. If the new state is a Goal state, quit and return this state
iii. Otherwise, add the new state to the end of NODE-LIST

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath

 Example:
In the below tree structure, we have shown the traversing of the tree using BFS
algorithm from the root node S to goal node K. BFS search algorithm traverse in
layers,so it will follow the path which is shown by the dotted arrow, and the traversed
path will be:

Path to goal : S A B C D G H E F I K

BFS search Properties:

Time Complexity: T (b) = 1+b2+b3+. ..... + bd= O (bd)
Where the d= depth of shallowest solution and b is a node at every state.

Space Complexity: the Memory size of frontier which is O(bd).

Completeness: BFS is complete, which means if the shallowest goal node is
atsome finite depth, then BFS will find a solution.
Optimality: It is optimal if path cost is a non-decreasing function of the depth of
the node.

2. Depth-First Search:
The search starts from the root node and follows each path to its greatest depth
node before moving to the next path , recursively for traversing a tree or graph data
structure is called the Depth-First Search.

DFS uses a stack data structure for its implementation.

The process of the DFS algorithm is similar to the BFS algorithm.
Note: Backtracking is an algorithm technique for finding all possible solutions using
recursion.

Advantage:
* DFS requires very less memory as it only needs to store a stack of the nodes on
the path from root node to the current node.
* It takes less time to reach to the goal node than BFS algorithm (if it traverses in
the right path).
Disadvantages:
* There is the possibility that many states keep re-occurring, and there is no
guarantee of finding the solution.

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath

 *DFS algorithm goes for deep down searching and sometime it may go to the
infinite loop.
Algorithm:
1. Create a variable called NODE-LIST and set it to initial state
2. Until a goal state is found or NODE-LIST is empty do
a.Remove the first element from NODE-LIST and call it E. If NODE-LIST was empty,
quit
b. For each way that each rule can match the state described in E do:
i. Apply the rule to generate a new state
ii. If the new state is a goal state, quit and return this state
iii. Otherwise, add the new state in front of NODE-LIS

Example:: Root node -->Left node -------- > right node.

Path to Goal : S—A—B---D—E—C—G

Step 1: Initially fringe contains only the node for S. FRINGE: S

Step 2: S is removed from fringe. S is expanded and its children A and H are put in
front of fringe.FRINGE: A ,H
Step 3: Node A is removed from fringe, and A is expanded its children B and C are
pushed in front of fringe. FRINGE: B,C
Step 4: Node B is removed from fringe, B is Expanded D and E are pushed in front of
fringe. FRINGE: D, E, C
Step 5: D and E not has child node so it backtrack traverse node C, C is removed from
fringe and expanded Its child G is pushed in front of fringe. FRINGE: G
Step 6: Node G is expanded and found to be a goal node.

It will start searching from root node S, and traverse A, then B, then D and E,
after traversing E, it will backtrack the tree as E has no other successor and still goal
node is not found. After backtracking it will traverse node C and then G, and here it will
terminate as it found goal node.

DFS Search Properties:

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath
 Completeness: DFS search algorithm is complete within finite state space as it will
expand every node within a limited search tree.
Time Complexity: It is equivalent to the node traversed by the algorithm. It is given
by: T(n)= 1+ n2+ n3 +.........+ nm=O(nm), Where, m= maximum >d
Space Complexity: It is equivalent to the size of the fringe set, which is O(bm).
Optimal: DFS is Non-Optimal, as it may generate a large number of steps or high
cost to reach to the goal node.

3. Depth-Limited Search Algorithm:

A depth-limited search algorithm is similar to depth-first search with a
predetermined limit. Depth-limited search can solve the drawback of the infinite path in
the Depth-first search. In this algorithm, the node at the depth limit will treat as it has no
successor nodes further.
Depth-limited search can be terminated with two Conditions of failure:
 Standard failure value: It indicates that problem does not have any solution.
 Cut off failure value: It defines no solution for the problem within a given depth
limit.

Advantages: Depth-limited search is Memory efficient.

Disadvantages:
 Depth-limited search also has a disadvantage of incompleteness.
 It may not be optimal if the problem has more than one solution.

Example:Path to Goal: S—A—C—D—B—I—G

DLS Search Properties :

Completeness: DLS search algorithm is complete if the solution is above the
depth-limit.
Time Complexity: Time complexity of DLS algorithm is O(bℓ).
Space Complexity: Space complexity of DLS algorithm is O(b×ℓ).
Optimal: Depth-limited search can be viewed as a special case of DFS, and it
isalso not optimal even if ℓ>d.

4. Uniform-cost Search Algorithm:

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath
 Uniform-cost search is a searching algorithm used for traversing a weighted tree
or graph. This algorithm comes into play when a different cost is available for each
edge. The primary goal of the uniform-cost search is to find a path to the goal node
which has the lowest cumulative cost.
Uniform-cost search expands nodes according to their path costs form the root
node. It can be used to solve any graph/tree where the optimal cost is in demand.
It implemented by the priority queue. It gives maximum priority to the lowest
cumulative cost.
Uniform cost search is equivalent to BFS algorithm if the path cost of all edges is
the same.

Advantages:
 Uniform cost search is optimal because at every state the path with the least
cost is chosen.
Disadvantages:
 It does not care about the number of steps involve in searching and only
concerned about path cost. Due to which this algorithm may be stuck in an
infinite loop.
Example:

Step 1: Start At Root Node S

Step 2 : Expand S , It Has 2 Nodes A, B Check A, B Not Goal Nodes.
Step 3: Select Node A , It Has Low Cost Than B. Expand A.
Step 4: A Has Child Nodes C, D Are Not Goal Nodes , Select D Has Less Cost Than C,
Expand D.
Step 5: D Has Child Nodes F , G. G Is Goal Node Stop Search.

Path to Goal: S—A—D—G

Properties :
Completeness: It is complete, such as if there is a solution, UCS will find it.
Time Complexity: Let C* is Cost of the optimal solution, and ε is each step to get
closer to the goal node. Then the number of steps is = C*/ε+1. worst-case time
complexity O(b1 + [C*/ε])/.
Space Complexity: The worst-case space complexity O(b1 + [C*/ε]).
Optimal: UCS is always optimal as it only selects a path with the lowest path cost.

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath

 5. Iterative Deepening Depth-First Search:
The Iterative Deepening Algorithm is a combination of DFS and BFS algorithms
to find out the best depth limit and does it by gradually increasing thelimit until a goal is
found.
This algorithm performs depth-first search up to a certain "depth limit", andit
keeps increasing the depth limit after each iteration until the goal node is found.
It combines the benefits of Breadth-first search's fast search and Depth-First
Search's memory efficiency.It is useful when search space is large, and depth of goal
node is unknown.

Advantages: It combines the benefits of BFS and DFS search algorithm in terms of fast
searchand memory efficiency.
Disadvantages: The main drawback of IDDFS is that it repeats all the work of
theprevious phase.
Algorithm:

Bool IDDFS(src, target, max_depth)

for limit from 0 to max_depth
if DLS(src, target, limit) == true
return true
return false

bool DLS(src, target, limit)

if (src == target)
return true;

// If reached the maximum depth,

// stop recursing.
if (limit <= 0)
return false;

for each adjacent t i of src

if DLS(i, target, limit?1)
return true
return false
Example:
Following tree structure is showing the iterative deepening depth-first search.
IDDFS algorithm performs various iterations until it does not find the goal node. The
iteration performed by the algorithm is given as:

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath

 1'st Iteration-----> A level-0
2'nd Iteration----> A, B, C level-1
3'rd Iteration------>A, B, D, E, C, F, G level-2
4'th Iteration------>A, B, D, H, I, E, C, F, K, G level-3
In the fourth iteration, the algorithm will find the goal node.

Path to goal: A—B—D—H—I—E—C—F—K--G

Properties :
Completeness: This algorithm is complete is ifthe branching factor is finite.
Time Complexity: b is the branching factor and depth is d then the worst-case
time complexity is O(bd).
Space Complexity:The space complexity of IDDFS will be O(bd).
Optimal: IDDFS algorithm is optimal if path cost is a non- decreasing function of
the depth of the node.

6. Bidirectional Search Algorithm:

Bidirectional search algorithm runs two simultaneous searches, one form initial
state called as forward-search and other from goal node called as backward-search, to
find the goal node.
Bidirectional search replaces one single search graph with two small sub graphs in
which one starts the search from an initial vertex and other starts from goal vertex. The
search stops when these two graphs intersect each other.
Bidirectional search can use search techniques such as BFS, DFS, DLS, etc.

It starts traversing from Initial node in the forward direction to GoalIt

starts from Goal Node in the backward direction to Initial Node
Both Intersect at Node where two searches meet and terminate search.
Advantages:
 Bidirectional search is Fast.
 Bidirectional search requires less memory
Disadvantages:

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath

  Implementation of the Bidirectional Search tree is difficult.
 In bidirectional search, one should know the goal state in advance.

Example:
In the below search tree, Bidirectional Search Algorithm is applied. This algorithm
divides one graph/tree into two sub-graphs.
It starts traversing from node 1 in the forward direction to goal
It starts from goal node 16 in the backward direction to node
Both Intersect at node 9 where two searches meet and terminate search.

Step 1: A is the initial node and 1 is the goal node, and 9 is the intersection node.

Step 2: Start searching simultaneously From Start To Goal Node and backward
fromGoal To Start Node.

Step 3: search forwardly 1 4 8 9 and search backwardly 9 10 12 16

intersect at node 9, then the searching stops.

Path to Goal: 1 4 8 9 10 12 16

Properties :
Completeness: Bidirectional Search is complete if we use BFS in both searches.
Time Complexity: Time complexity of bidirectional search using BFS is O(bd).
Space Complexity: Space complexity of bidirectional search is O(bd).
Optimal: Bidirectional search is Optimal.

B.Tech-CSE(AIML)-3-2- AI-Notes prepared by V. Ravindranath

 Informed Search Algorithms
The uninformed search algorithms which looked through search space for all
possible solutions of the problem without having any additional knowledge about search
space. It contains an array of knowledge such as how far we are from the goal, path cost,
how to reach to goal node, etc.

This knowledge help agents to explore less to the search space and find more
efficiently the goal node.
The informed search algorithm is more useful for large search space.
Informed search algorithm uses the idea of heuristic, so it is also called Heuristic
search.

Heuristics function: Heuristic is a function which is used in Informed Search, and it finds
the most promising path. It takes the current state of the agent as its input and produces
the estimation of how close agent is from the goal. The heuristic method, however, might
not always give the best solution, but it guaranteed to find a good solution in reasonable
time. Heuristic function estimates how close a state is to the goal.
It is represented by h(n), and it calculates the cost of an optimal path between the
pair of states. The value of the heuristic function is always positive.
Admissibility of the heuristic function is given as:
h(n) <= h*(n)

h(n) is heuristic cost.

h*(n) is the estimated cost.

Pure Heuristic Search: Pure heuristic search is the simplest form of heuristic search
algorithms. It expands nodes based on their heuristic value h(n). It maintains two lists,
OPEN and CLOSED list. In the CLOSED list, it places those nodes which have already
expanded and in the OPEN list, it places nodes which have yet not been expanded.

On each iteration, each node n with the lowest heuristic value is expanded and
generates all its successors and n is placed to the closed list. The algorithm continues unit
a goal state is found.

In the informed search we will discuss two main algorithms which are given below:

1. Best First Search Algorithm(Greedy search)

2. A* Search Algorithm

1. Best-first Search Algorithm (Greedy Search):

Greedy best-first search algorithm always selects the path which appears best
atthat moment. It is the combination of depth-first search and breadth-first search
algorithms. It uses the heuristic function and search.

Best-first search allows us to take the advantages of both algorithms. With the help
of best-first search, at each step, we can choose the most promising node. In the best
first search algorithm, we expand the node which is closest to the goal node and the
closest cost is estimated by heuristic function, i.e.
1. f(n)= g(n).

AI Notes prepared by Ravindranath

Were, h(n)= estimated cost from node n to the goal.
The greedy best first algorithm is implemented by the priority queue.

Best first search algorithm:

Step 1: Place the starting node into the OPEN list.
Step 2: If the OPEN list is empty, Stop and return failure.
Step 3: Remove the node n, from the OPEN list which has the lowest value
ofh(n), and places it in the CLOSED list.
Step 4: Expand the node n, and generate the successors of node n.
Step 5: Check each successor of node n, and find whether any node is a
goal node or not. If any successor node is goal node, then return success and
terminate the search, else proceed to Step 6.
Step 6: For each successor node, algorithm checks for evaluation function f(n),
and then check if the node has been in either OPEN or CLOSED list. If the
nodehas not been in both list, then add it to the OPEN list.
Step 7: Return to Step 2.

Advantages:
Best first search can switch between BFS and DFS by gaining the
advantagesof both the algorithms.
This algorithm is more efficient than BFS and DFS algorithms.

Disadvantages:
It can behave as an unguided depth-first search in the worst case scenario.
It can get stuck in a loop as DFS.
This algorithm is not optimal.

Example: Consider the below search problem, and we will traverse it using greedy best-
first search. At each iteration, each node is expanded using evaluation function f(n)=h(n)
, which is given in the below table.

In this search example, we are using two lists which are OPEN and CLOSED Lists.
Following are the iteration for traversing the above example.

AI Notes prepared by Ravindranath

Expand the nodes of S and put in the CLOSED list

Initialization: Open [A, B], Closed [S]

Iteration 1: Open [A], Closed [S, B]
Iteration 2: Open [E, F, A], Closed [S, B]
: Open [E, A], Closed [S, B, F]
Iteration 3: Open [I, G, E, A], Closed [S, B, F]
: Open [I, E, A], Closed [S, B, F, G]
PATH : S----> B----->F --- > G
Properties :

Time Complexity: The worst case time complexity of Greedy best first search is O(bm).
Space Complexity: The worst case space complexity of Greedy best first search is
O(bm). Where, m is the maximum depth of the search space.
Complete: Greedy best-first search is also incomplete, even if the given state space is
finite.
Optimal: Greedy best first search algorithm is not optimal.

2. A* Search Algorithm:

A* search is the most commonly known form of best-first search. It uses heuristic
function h(n), and cost to reach the node n from the start state g(n). It has combined
features of UCS and greedy best-first search, by which it solve the problem efficiently. A*
search algorithm finds the shortest path through the search space using the heuristic
function. This search algorithm expands less search tree and provides optimal result
faster. A* algorithm is similar to UCS except that it uses g(n)+h(n) instead of g(n).

In A* search algorithm, we use search heuristic as well as the cost to reach the
node. Hence we can combine both costs as following, and this sum is called as a fitness
number.

AI Notes prepared by Ravindranath

 At each point in the search space, only those node is expanded which have the lowest
value of f(n), and the algorithm terminates when the goal node is found.

Algorithm of A* search:

Step1: Place the starting node in the OPEN list.

Step 2: Check if the OPEN list is empty or not, if the list is empty then return failure and
stops.
Step 3: Select the node from the OPEN list which has the smallest value of evaluation
function (g+h), if node n is goal node then return success and stop, otherwise
Step 4: Expand node n and generate all of its successors, and put n into the closed list.
For each successor n', check whether n' is already in the OPEN or CLOSED list, if
not then compute evaluation function for n' and place into Open list.
Step 5: Else if node n' is already in OPEN and CLOSED, then it should be attached to the
back pointer which reflects the lowest g(n') value.
Step 6: Return to Step 2.

Advantages:
 A* search algorithm is the best algorithm than other search algorithms.
 A* search algorithm is optimal and complete.
 This algorithm can solve very complex problems.

Disadvantages:
 It does not always produce the shortest path as it mostly based on
heuristicsand approximation.
 A* search algorithm has some complexity issues.
 The main drawback of A* is memory requirement as it keeps all generated
nodes in the memory, so it is not practical for various large-scale problems.

Example:
In this example, we will traverse the given graph using the A* algorithm. The
heuristic value of all states is given in the below table so we will calculate the f(n) of each
state using the formula f(n)= g(n) + h(n), where g(n) is the cost to reach any node from
start state.
Here we will use OPEN and CLOSED list.

AI Notes prepared by Ravindranath

 Solution:

Initialization: {(S, 5)}

Iteration1: {(S--> A, 4), (S-->G, 10)}
Iteration2: {(S--> A-->C, 4), (S--> A-->B, 7), (S-->G, 10)}
Iteration3: {(S--> A-->C--->G, 6), (S--> A-->C--->D, 11), (S--> A-->B, 7), (S-->G, 10)}
Iteration 4 will give the final result, as S--->A--->C--->G it provides the optimal path with
cost 6.

Points to remember:
 A* algorithm returns the path which occurred first, and it does not search for
all remaining paths.
 The efficiency of A* algorithm depends on the quality of heuristic.
 A* algorithm expands all nodes which satisfy the condition f(n)<="" li="">

Properties :

Complete: A* algorithm is complete as long as: Branching factor is finite. Cost at every
action is fixed.
Optimal: A* search algorithm is optimal if it follows below two conditions:

Admissible: the first condition requires for optimality is that h(n) should be
an admissible heuristic for A* tree search. An admissible heuristic is optimistic
in nature.
Consistency: Second required condition is consistency for only A* graph-search.
If the heuristic function is admissible, then A* tree search will always find the least
costpath.
AI Notes prepared by Ravindranath
Time Complexity: The time complexity of A* search algorithm depends on heuristic
function, and the number of nodes expanded is exponential to the depth of solution d. So
the time complexity is O(b^d), where b is the branching factor.

Space Complexity: The space complexity of A* search algorithm is O(b^d)

Beyond Classical Search:

Hill-climbing search, Simulated annealing search, Local Search in Continuous

Spaces, Searching with Non-Deterministic Actions, Searching with Partial Observations,
Online Search Agents and Unknown Environment .

Hill Climbing Search in Artificial Intelligence:

 Hill climbing algorithm is a local search algorithm which continuously moves in
the direction of increasing elevation/value to find the peak of the mountain or
bestsolution to the problem. It terminates when it reaches a peak value where no
neighbor has a higher value is called Hill Climbing Search.
 Hill climbing algorithm is a technique which is used for optimizing the
mathematical problems.
Example : Hill climbing algorithm is Traveling-salesman Problem in which
we need to minimize the distance traveled by the salesman.
 It is also called greedy local search as it only looks to its good immediate
neighbor state and not beyond that.
 A node of hill climbing algorithm has two components which are state and value.
 Hill Climbing is mostly used when a good heuristic is available.
 In this algorithm, we don't need to maintain and handle the search tree or
graphas it only keeps a single current state.

Features of Hill Climbing:

 Generate and Test variant: Hill Climbing is the variant of Generate and Test
method. The Generate and Test method produce feedback which helps to
decidewhich direction to move in the search space.
 Greedy approach: Hill-climbing algorithm search moves in the direction which
optimizes the cost.
 No backtracking: It does not backtrack the search space, as it does not
remember the previous states.

State-space Diagram for Hill Climbing:

The state-space landscape is a graphical representation of the hill-climbing
algorithm which is showing a graph between various states of algorithm and Objective
function/Cost.
On Y-axis we have taken the function which can be an objective function or
costfunction, and state-space on the X-Axis. If the function on Y-axis is cost then, the
goal of search is to find the global minimum and local minimum. If the function of Y-
axis is Objective function, then the goal of the search is to find the global maximum
and local maximum.

AI Notes prepared by Ravindranath

Different regions in the state space landscape:

Local Maximum: Local maximum is a state which is better than its neighbor
states, but there is also another state which is higher than it.
Global Maximum: Global maximum is the best possible state of state space
landscape. It has the highest value of objective function.
Current state: It is a state in a landscape diagram where an agent is
currentlypresent.
Flat local maximum: It is a flat space in the landscape where all the neighbor
states of current states have the same value.
Shoulder: It is a plateau region which has an uphill edge.

Types of Hill Climbing Algorithm:

1. Simple Hill Climbing:
2. Steepest-Ascent Hill-Climbing:
3. Stochastic Hill Climbing:

1. Simple Hill Climbing:

Simple hill climbing is the simplest way to implement a hill climbing algorithm. It
only evaluates the neighbor node state at a time and selects the first one which
optimizes current cost and set it as a current state. It only checks it's one successor
state, and if it finds better than the current state, then move else be in the same state.
This algorithm has the following features:
 Less time consuming
 Less optimal solution and the solution is not guaranteed

AI Notes prepared by Ravindranath

Algorithm for Simple Hill Climbing:
 Step 1: Evaluate the initial state, if it is goal state then return success
andStop.
 Step 2: Loop Until a solution is found or there is no new operator left to apply.
 Step 3: Select and apply an operator to the current state.
 Step 4: Check new state:
1. If It Is Goal State, Then Return Success And Quit.
2. Else If It Is Better Than The Current State Then Assign New State
AsA Current State.
3. Else If Not Better Than The Current State, Then Return To Step2.
Step 5: Exit.

2. Steepest-Ascent Hill Climbing:

The steepest-Ascent algorithm is a variation of simple hill climbing algorithm.
This algorithm examines all the neighboring nodes of the current state and selects one
neighbor node which is closest to the goal state. This algorithm consumes more time as
it searches for multiple neighbors

Algorithm for Steepest-Ascent hill climbing:

 Step 1: Evaluate the initial state, if it is goal state then return success
andstop, else make current state as initial state.
 Step 2: Loop until a solution is found or the current state does not change.
 Step 3: Let SUCC be a state such that any successor of the current state
will be better than it. For each operator that applies to the current state:
 Step 4: Apply the new operator and generate a new state. Evaluate the
newstate.
 If it is goal state, then return it and quit, else compare it to the SUCC.
 If it is better than SUCC, then set new state as SUCC.
 If the SUCC is better than the current state, then set current state
toSUCC.
Step 5: Exit.

3. Stochastic hill climbing:

Stochastic hill climbing does not examine for all its neighbor before moving.
Rather, this search algorithm selects one neighbor node at random and decides
whether to choose it as a current state or examine another state.
Problems in Hill Climbing Algorithm:
1. Local maximum
2. Plateau
3. Ridges
1. Local Maximum: A local maximum is a peak state in the landscape which is
better than each of its neighboring states, but there is another state also present which
is higher than the local maximum.

Solution: Backtracking technique can be a solution of the local maximum in state

space landscape. Create a list of the promising path so that the algorithm can backtrack

AI Notes prepared by Ravindranath

the search space and explore other paths as well.

2. Plateau: A plateau is the flat area of the search space in which all the neighbor
states of the current state contains the same value, because of this algorithm does not
find any best direction to move. A hill-climbing search might be lost in the plateau
area.Solution: The solution for the plateau is to take big steps or very little steps while
searching, to solve the problem. Randomly select a state which is far away from the
current state so it is possible that the algorithm could find non-plateau region.

3. Ridges: A ridge is a special form of the local maximum. It has an area which is
higher than its surrounding areas, but itself has a slope, and cannot be reached in
asingle move.
Solution: With the use of bidirectional search, or by moving in different directions,
wecan improve this problem.

Simulated Annealing Search:

A hill-climbing algorithm which never makes a move towards a lower value
guaranteed to be incomplete because it can get stuck on a local maximum. And if
algorithm applies a random walk, by moving a successor, then it may complete but not
efficient. Simulated Annealing is an algorithm which yields both efficiency and
completeness.
In mechanical term Annealing is a process of hardening a metal or glass to a
high temperature then cooling gradually, so this allows the metal to reach a low-energy
crystalline state. The same process is used in simulated annealing in which the
algorithm picks a random move, instead of picking the best move.

Function: SIMULATED-ANNEALING(Problem, Schedule) returns a solution state

Inputs : Problem, a problem
Schedule, a mapping from time to temperature
Local Variables : Current, a node
AI Notes prepared by Ravindranath
 Next, a node
T, a “temperature” controlling the probability of downward steps
Current = MAKE-NODE(INITIAL-STATE[Problem])
For t = 1 to ∞ do
T = Schedule[t]
If T = 0 then return Current
Next = a randomly selected successor of Current
ΛE = VALUE[Next] – VALUE[Current]
if ΛE > 0 then
Current = Next
else
Current = Next only with probability exp(-ΛE/T

Local search in continuous spaces

The states are defined by n variables (defined by an n-dimensional vector of
variables x.)
Gradient of the objective function ▽f df/dxi

Local expression for the gradient

Can perform steepest-ascent hill climbing by updating the current state according
tothe formula x ← x + α▽f(x) , αis the step size (a small constant).
If the objection function f is not available in a differentiable form, use empirical gradient
search.
Line search: tries to overcome the dilemma of adjusting α by extending the current
gradient direction—repeatedly doubling αuntil f starts to decrease again. The point at
which this occurs becomes the new current state.
In cases that the objective function is not differentiable: we calculate an empirical
gradient by evaluating the response to small increments and decrements in each
coordinate. Empirical gradient search is the same as steepest-ascent hill climbing in a
discretized version of the state space.

Newton-Raphson method:

Hf(x) is the Hessian matrix of second derivatives, Hij =

Constrained optimization: An optimization problem is constrained if solutions must

satisfy some hard constraints on the values of the variables. (e.g. : linear programming
problems)

Many local search methods apply also to problems in continuous spaces.

Linear programming and convex optimization problems obey certain restrictions on
theshape of the state space and the nature of the objective function, and admit
polynomial-time algorithms that are oftenly extremely efficient in practice.

Searching With Nondeterministic Actions:

When the environment is either partially observable or nondeterministic (or
both), the future percepts cannot be determined in advance, and the agent’s future
AI Notes prepared by Ravindranath
actions will depend on those future percepts.

Nondeterministic problems:
Transition model is defined by RESULTS function that returns a set of possible
outcome states;
Solution is not a sequence but a contingency plan (strategy),
e.g.
Suck, if State = 5 then [Right, Suck] else []];
In nondeterministic environments, agents can apply AND-OR search to
generate contingent plans that reach the goal regardless of which outcomes occur
during execution.

AND-OR Search Trees:

OR nodes: In a deterministic environment, the only branching is introduced by

the agent’s own choices in each state, we call these nodes OR nodes.

AND nodes: In a nondeterministic environment, branching is also introduced by

the environment’s choice of outcome for each action, we call these nodes AND nodes.

AND-OR tree: OR nodes and AND nodes alternate. States nodes are OR
nodes where some action must be chosen. At the AND nodes (shown as circles),
every outcome must be handled.
A solution for an AND-OR search problem is a subtree that
1) has a goal node at every leaf.
2) specifies one action at each of its OR nodes.
3) includes every outcome branch at each of its AND nodes.
Solution path nodes: 1, 5, 6, 8

Algorithm :A recursive, depth-first algorithm for AND-OR graph search

AI Notes prepared by Ravindranath

 Cyclic solution: keep trying until it works. We can express this solution by adding a
label to denote some portion of the plan and using the label later instead of repeating
the plan itself. E.g.: [Suck, L1: Right, if State = 5 then L1 else Suck]. (or “while State =
5 do Right”).

Searching With Partial Observations:

Belief state: The agent's current belief about the possible physical states it
might be in, given the sequence of actions and percepts up to that point.

Standard search algorithms can be applied directly to belief-state space to

solve sensorless problems, and belief-state AND-OR search can solve general
partially observable problems. Incremental algorithms that construct solutions state-by-
state within a belief state are often more efficient.

Introduce belief state: represents agent’s current belief about possible

physical states It could be in, given sequenc of actions and
percepts up to that point in 3 ways:
1. Searching with no observation
2. Searching with observations
3. Solving partially observable problems

1. Searching With No Observation:

When the agent's precepts provide no information at all, we have a sensor
less problem. To solve sensor less problems, we search in the space of belief states
rather than physical states. In belief-state space, the problem is fully observable, the
solution is always a sequence of actions.

AI Notes prepared by Ravindranath

Belief-state problem can be defined by: (The underlying physical problem P is
defined by ACTIONSP, RESULTP, GOAL-TESTP and STEP-COSTP)

Belief states: Contains every possible set of physical states. If P has N states, the
sensor less problem has up to 2^N states

Initial states: Typically the set of all states in P.

Actions:
a. If illegal actions have no effect on the environment, take the union of all
theactions in any of the physical states in the current belief b: ACTIONS(b)
=US€BRESULTSP(s,a)
b. If illegal actions are extremely dangerous, take the intersection.

Transition model:
For deterministic actions,
b' = RESULT(b,a) = {s' : s' = RESULTP(s,a) and s∈b}. (b' is never larger than b).

For nondeterministic actions,

b' = RESULT(b,a) = {s' : s' = RESULTP(s,a) and s∈b} = (b' may be larger
thanb)
The process of generating the new belief state after the action is called the
prediction step.
Goal test: A belief state satisfies the goal only if all the physical states in
itsatisfyGOAL-TESTP.

Path cost: If an action sequence is a solution for a belief state b, it is also a solution
for any subset of b. Hence, we can discard a path reaching the superset if the subset
has already been generated. Conversely, if the superset has already been generated
and found to be solvable, then any subset is guaranteed to be solvable.

Solution: Represent the belief state by some more compact description;

Avoid the standard search algorithm, develop incremental belief state search
algorithms instead.
Incremental belief-state search: Find one solution that works for all the states,
typically able to detect failure quickly.

2. Searching with Observations:

When observations are partial, The ACTIONS, STEP-COST, and GOAL-TEST
are constructed form the underlying physical problem just as for sensor less problems.
Transition model: We can think of transitions from one belief state to the next
for a particular action as occurring in 3 stages.

The prediction stage is the same as for sensor less problem, given the action a
in belief state b, the predicted belief state is
The observation prediction stage determines the set of percepts o that could be
observed in the predicted belief state:

AI Notes prepared by Ravindranath

 The update stage determines, for each possible percept, the belief state that
would result from the percept. The new belief state bo is the set of states in bˆ that
could have produced the percept:

In conclusion:
RESULTS(b,a) = { bo : bo = UPDATE(PREDICT(b,a),o) and o∈POSSIBLE-
PERCEPTS(PREDICT(b,a))}

Search algorithm return a conditional plan that test the belief state rather than
the actual state.

3. Solving partially observable problems:

It is similar to the simple problem-solving agent (formulates a problem, calls a
search algorithm, executes the solution).
Main difference:

1) The solution will be a conditional plan rather than a sequence.

2) The agent will need to maintain its belief state as it performs actions and receives
percepts.

Given an initial state b, an action a, and a percept o, the new belief state is
b’ = UPDATE(PREDICT(b, a), o). //recursive state estimator
Sate estimation: a.k.a. monitoring or filtering, a core function of intelligent system in
partially observable environments—maintaining one’s belief state.

Online search Agents:

Online search is a necessary idea for unknown environments. Online search
agent interleaves computation and action: first it takes an action, then it observes the
environment and computes the next action.

Online search problem

Assume a deterministic and fully observable environment, the agent only knows:
 ACTION(s): returns a list of actions allowed in states;
 c(s, a, s’): The step-cost function, cannot be used until the agent knows
thats’is the outcome;
 GOAL-TEST(s).
 The agent cannot determine RESULT(s, a) except by actually being in s
anddoing a.

AI Notes prepared by Ravindranath

 The agent might have access to an admissible heuristic function h(s) that estimates
the distance from the current state to a goal state.
Competitive ratio: The cost (the total path cost of the path that the agent actually
travels) / the actual shortest path (the path cost of the path the agent would follow if it
knew the search space in advance). The competitive ratio is expected to be as small
as possible.
In some case the best achievable competitive ratio is infinite, e.g. some actions are
irreversible and might reach a dead-end state. No algorithm can avoid dead ends in all
state spaces.

Safely explorable: some goal state is reachable from every reachable state. E.g.
state spaces with reversible actions such as mazes and 8-puzzles.
No bounded competitive ratio can be guaranteed even in safely explorable
environments if there are paths of unbounded cost.

2. Online Search Agents

ONLINE-DFS-AGENT works only in state spaces where the actions are reversible.

RESULT: a table the agent stores its map, RESULT[s, a] records the state resulting
from executing action a in state s.
Whenever an action from the current sate has not been explored, the agent tries that
action.
When the agent has tried all the actions in a state, the agent in an online
searchbacktracks physically (in a depth-first search, means going back to the state
from which the agent most recently entered the current sate).

3. Online local search:

Exploration problems arise when the agent has no idea about the state and
actions of its environment.
For safely explorable environments, online search agents can build a map and
find a goal if one exists. Updating heuristic estimates from experience provides an
effective method to escape from local minima.

AI Notes prepared by Ravindranath

Random walk: Because online hill-climbing search cannot use restart(because the
agent cannot transport itself to a new state), can use random walk instead. A random
walk simply selects at random one of the available actions from the current state,
preference can be given to actions that have not yet been tried.

Basic idea of online agent: Random walk will eventually find a goal or complete its
exploration if the space is finite, but can be very slow. A more effective approach is to
store a “current best estimate” H(s) of the cost to reach the goal from each state that
has been visited. H(s) starts out being the heuristic estimate h(s) and is updated as the
agent gains experience in the state space.
If the agent is stuck in a flat local minimum, the agent will follow what seems
tobe the best path to the goal given the current cost estimates for its neighbours.
The estimate cost to reach the goal through a neighbor s’ is the cost to get to s’ plus
the estimated cost to get to a goal from there, that is, c(s, a, s') + H(s').

LRTA*: learning real-time A*. It builds a map of the environment in the result table,
update the cost estimate for the state it has just left and then chooses the “apparently
best” move according to its current cost estimates.

Actions that have not yet been tried in a state s are always assumed to lead
immediately to the goal with the least possible cost (a.k.a.h(s)), this optimism
underuncertainty encourages the agent to explore new, possible promising paths.

AI Notes prepared by Ravindranath