Artificial Intelligence

Department of Artificial Intelligence and Data Science

Guru Nanak Institute of Technology
Course Objectives
• To learn the distinction between optimal reasoning Vs. human like
reasoning
• To understand the concepts of state space representation, exhaustive
search, heuristic search together with the time and space
complexities.
• To learn different knowledge representation techniques.
• To understand the applications of AI, namely game playing, theorem
proving, and machine learning.
Course Outcomes
At end of this course, student should be able to
• Understand the basic concepts of AI and Select a search algorithm to
solve a problem
• Solve any problem with constraint satisfaction
• Solve knowledge representation problems using first order logic
• Analyze various planning techniques
• Understand supervised and unsupervised learning concepts
Syllabus
UNIT - I

Problem Solving by Search-I: Introduction to AI, Intelligent Agents

Problem Solving by Search –II: Problem-Solving Agents, Searching for
Solutions, Uninformed Search Strategies: Breadth-first search, Uniform cost
search, Depth-first search, Iterative deepening Depth-first search,
Bidirectional search, Informed (Heuristic) Search Strategies: Greedy best-first
search, A* search, Heuristic Functions, Beyond Classical Search: Hill-climbing
search, Simulated annealing search, Local Search in Continuous Spaces,
Searching with Non-Deterministic Actions, Searching with Partial
Observations, Online Search Agents and Unknown Environment .
Syllabus contd…
• UNIT – II
• Problem Solving by Search-II – Continuation
• Adversarial Search: Games, Optimal Decisions in Games, Alpha–Beta
Pruning, Imperfect Real-Time Decisions.
• Constraint Satisfaction Problems: Defining Constraint Satisfaction
Problems, Constraint Propagation, Backtracking Search for CSPs, Local
Search for CSPs, The Structure of Problems.
Syllabus contd…
• UNIT – III
• Propositional Logic, Logic and Knowledge Representation, Inference in
First Order Logic

• UNIT – IV
• Planning: Classical Planning: Definition of Classical Planning,
Algorithms for Planning with State-Space Search, Planning Graphs,
other Classical Planning Approaches, Analysis of Planning approaches.
Planning and Acting in the Real World
Syllabus contd…
• UNIT – V
• Uncertain knowledge and Learning, Learning: Forms of Learning,
Supervised Learning, Learning Decision Trees, Knowledge in Learning
Text Books and References
• TEXT BOOK:
• Stuart Russell and Peter Norvig, Artificial Intelligence A Modern Approach,
3rd Edition, 2010, Pearson Education.

• REFERENCE BOOKS:
• E. Rich and K. Knight, Artificial Intelligence, 3rd Edition, 2017 , McGraw Hill
Education
• Patrick Henny Winston, Artificial Intelligence, 3rd Edition., 2002, Pearson
Education.
• Shivani Goel, Artificial Intelligence, 2013, Pearson Education.
• Dan W. Patterson, Artificial Intelligence and Expert systems, 1995, Pearson
Education
History of AI
• Maturation of Artificial Intelligence (1943-1952)
• Year 1943 The first work which is now recognized as AI was
done by Warren McCulloch and Walter pits in 1943. They
proposed a model of artificial neurons.
• Year 1949 Donald Hebb demonstrated an updating rule for
modifying the connection strength between neurons. His rule is
now called Hebbian learning.
• Year 1950 The Alan Turing who was an English mathematician
and pioneered Machine learning in 1950. Alan Turing
publishes "Computing Machinery and Intelligence" in which he
proposed a test. The test can check the machine's ability to
exhibit intelligent behavior equivalent to human intelligence,
called a Turing test.
History of AI
• The birth of Artificial Intelligence (1952-1956)
• Year 1955 An Allen Newell and Herbert A. Simon created
the "first artificial intelligence program"Which was named
as "Logic Theorist". This program had proved 38 of 52
Mathematics theorems, and find new and more elegant
proofs for some theorems.
• Year 1956 The word "Artificial Intelligence" first adopted by
American Computer scientist John McCarthy at the
Dartmouth Conference. For the first time, AI coined as an
academic field.
• At that time high-level computer languages such as
FORTRAN, LISP, or COBOL were invented. And the
enthusiasm for AI was very high at that time.
History of AI
• The golden years-Early enthusiasm (1956-1974)
• Year 1966 The researchers emphasized developing
algorithms which can solve mathematical problems. Joseph
Weizenbaum created the first chatbot in 1966, which was
named as ELIZA.
• Year 1972 The first intelligent humanoid robot was built in
Japan which was named as WABOT1.
History of AI
• The first AI winter (1974-1980)
• The duration between years 1974 to 1980 was the first AI
winter duration. AI winter refers to the time period where
computer scientist dealt with a severe shortage of funding
from government for AI researches.
• During AI winters, an interest of publicity on artificial
intelligence was decreased.
History of AI
• A boom of AI (1980-1987)
• Year 1980 After AI winter duration, AI came back with
"Expert System". Expert systems were programmed that
emulate the decision-making ability of a human expert.
• In the Year 1980, the first national conference of the
American Association of Artificial Intelligence was held at
Stanford University.
History of AI
• The second AI winter (1987-1993)
• The duration between the years 1987 to 1993 was the
second AI Winter duration.
• Again Investors and government stopped in funding for AI
research as due to high cost but not efficient result. The
expert system such as XCON was very costly.
History of AI
• The emergence of intelligent agents (1993-2011)
• Year 1997 In the year 1997, IBM Deep Blue beats world
chess champion, Gary Kasparov, and became the first
computer to beat a world chess champion.
• Year 2002 for the first time, AI entered the home in the
form of Roomba, a vacuum cleaner.
• Year 2006 AI came in the Business world till the year 2006.
Companies like Facebook, Twitter, and Netflix also started
using AI.
History of AI
• Deep learning, big data and artificial general intelligence (2011-present)
• Year 2011 In the year 2011, IBM's Watson won jeopardy, a quiz show,
where it had to solve the complex questions as well as riddles. Watson
had proved that it could understand natural language and can solve
tricky questions quickly.
• Year 2012 Google has launched an Android app feature "Google
now", which was able to provide information to the user as a
prediction.
• Year 2014 In the year 2014, Chatbot "Eugene Goostman" won a
competition in the infamous "Turing test."
• Year 2018 The "Project Debater" from IBM debated on complex
topics with two master debaters and also performed extremely well.
• Google has demonstrated an AI program "Duplex" which was a virtual
assistant and which had taken hairdresser appointment on call, and
lady on other side didn't notice that she was talking with the machine.
History of AI
Intelligent Agents
Outline
• Agents and environments
• Rationality
• PEAS (Performance measure, Environment, Actuators, Sensors)
• Environment types
• Agent types

Artificial Intelligence a modern approach 19

Agents
• An agent is anything that can be viewed as perceiving its environment
through sensors and acting upon that environment through actuators

• Human agent:
– eyes, ears, and other organs for sensors;
– hands, legs, mouth, and other body parts for actuators

• Robotic agent:
– cameras and infrared range finders for sensors
– various motors for actuators

Artificial Intelligence a modern approach 20

Agents and environments

• The agent function maps from percept histories to actions:

[f: P* 🡪 A]

• The agent program runs on the physical architecture to

produce f
• agent = architecture + program
Artificial Intelligence a modern approach 21
Vaccum-cleaner world

• Percepts: location and contents, e.g., [A,Dirty]

• Actions: Left, Right, Suck, NoOp
• Agent’s function 🡪 look-up table
• For many agents this is a very large table

Artificial Intelligence a modern approach 22

Rational agents
• Rationality
– Performance measuring success
– Agents prior knowledge of environment
– Actions that agent can perform
– Agent’s percept sequence to date

• Rational Agent: For each possible percept sequence, a rational agent should
select an action that is expected to maximize its performance measure, given
the evidence provided by the percept sequence and whatever built-in
knowledge the agent has.

Artificial Intelligence a modern approach 23

Rationality
• Rational is different from omniscience
• Percepts may not supply all relevant information
• E.g., in card game, don’t know cards of others.

• Rational is different from being perfect

• Rationality maximizes expected outcome while perfection maximizes actual
outcome.

Artificial Intelligence a modern approach 24

Autonomy in Agents
The autonomy of an agent is the extent to which its
behaviour is determined by its own experience,
rather than knowledge of designer.

• Extremes
• No autonomy – ignores environment/data
• Complete autonomy – must act randomly/no program
• Example: baby learning to crawl
• Ideal: design agents to have some autonomy
• Possibly become more autonomous with experience
PEAS
• PEAS: Performance measure, Environment, Actuators,
Sensors

• Must first specify the setting for intelligent agent design

• Consider, e.g., the task of designing an automated taxi
driver:
– Performance measure: Safe, fast, legal, comfortable trip,
maximize profits
– Environment: Roads, other traffic, pedestrians, customers
– Actuators: Steering wheel, accelerator, brake, signal, horn
– Sensors: Cameras, sonar, speedometer, GPS, odometer,
engine sensors, keyboard

Artificial Intelligence a modern approach 26

PEAS
• Agent: Part-picking robot
• Performance measure: Percentage of parts in correct bins
• Environment: Conveyor belt with parts, bins
• Actuators: Jointed arm and hand
• Sensors: Camera, joint angle sensors

Artificial Intelligence a modern approach 27

PEAS
• Agent: Interactive English tutor
• Performance measure: Maximize student's score on test
• Environment: Set of students
• Actuators: Screen display (exercises, suggestions, corrections)
• Sensors: Keyboard

Artificial Intelligence a modern approach 28

Environment types
• Fully observable (vs. partially observable)
• Deterministic (vs. stochastic)
• Episodic (vs. sequential)
• Static (vs. dynamic)
• Discrete (vs. continuous)
• Single agent (vs. multiagent):

Artificial Intelligence a modern approach 29

Fully observable (vs. partially observable)

• Is everything an agent requires to choose its actions available to it via

its sensors? Perfect or Full information.
• If so, the environment is fully accessible
• If not, parts of the environment are inaccessible
• Agent must make informed guesses about world.
• In decision theory: perfect information vs. imperfect information.

Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Fully Partially Partially Partially Fully Fully

Artificial Intelligence a modern approach 30

Deterministic (vs. stochastic)
• Does the change in world state
• Depend only on current state and agent’s action?
• Non-deterministic environments
• Have aspects beyond the control of the agent
• Utility functions have to guess at changes in world

Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Deterministic Stochastic Stochastic Stochastic Stochastic Deterministic

Artificial Intelligence a modern approach 31

Episodic (vs. sequential):

• Is the choice of current action

• Dependent on previous actions?
• If not, then the environment is episodic
• In non-episodic environments:
• Agent has to plan ahead:
• Current choice will affect future actions

Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Sequential Sequential Sequential Sequential Episodic Episodic

Artificial Intelligence a modern approach 32

 Static (vs. dynamic):
• Static environments don’t change
• While the agent is deliberating over what to do
• Dynamic environments do change
• So agent should/could consult the world when choosing actions
• Alternatively: anticipate the change during deliberation OR make decision
very fast
• Semidynamic: If the environment itself does not change with
the passage of time but the agent's performance score does.

Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Static Static Static Dynamic Dynamic Semi

Another example: off-line route planning vs. on-board navigation system

Artificial Intelligence a modern approach 33
Discrete (vs. continuous)

• A limited number of distinct, clearly defined percepts and actions vs. a range of
values (continuous)

Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Discrete Discrete Discrete Conti Conti Conti

Artificial Intelligence a modern approach 34

 Single agent (vs. multiagent)
• An agent operating by itself in an environment or there are many
agents working together

Cross Word Poker Backgammon Taxi driver Part picking robot Image analysis
Single Multi Multi Multi Single Single

Artificial Intelligence a modern approach 35

Summary.
Observable Deterministic Episodic Static Discrete Agents

Cross Word Fully Deterministic Sequential Static Discrete Single

Poker Fully Stochastic Sequential Static Discrete Multi

Backgammon Partially Stochastic Sequential Static Discrete Multi

Taxi driver Partially Multi

Stochastic Sequential Dynamic Conti

Part picking robot Partially Stochastic Episodic Dynamic Conti Single

Image analysis Fully Deterministic Episodic Semi Conti Single

Artificial Intelligence a modern approach

THE STRUCTURE OF AGENTS
• The job of AI is to design an agent program that implements the agent function the mapping from percepts to
actions. We assume this program will run on some sort of computing device with physical sensors and
actuators—we call this the architecture:
• agent = architecture + program .

Artificial Intelligence a modern approach 37

Agent types
• Four basic types in order of increasing generality:
• Simple reflex agents
• Reflex agents with state/model
• Goal-based agents
• Utility-based agents
• All these can be turned into learning agents

Artificial Intelligence a modern approach 38

Simple reflex agents

Artificial Intelligence a modern approach 39

Simple reflex agents
• Simple but very limited intelligence. A
simple reflex agent. It acts according to a
rule whose condition matches the current state, as defined by the
percept.
• Action does not depend on percept history, only on current percept.
• Therefore no memory requirements.
• Infinite loops
• Suppose vacuum cleaner does not observe location. What do you do given location = clean?
Left of A or right on B -> infinite loop.
• Fly buzzing around window or light.

Artificial Intelligence a modern approach 40

Model-based reflex agents
⚫ Know how world evolves
⚪ Overtaking car gets closer from
behind
⚫ How agents actions affect the
world
⚪ Wheel turned clockwise takes you
right

⚫ Model base agents update their

state

Artificial Intelligence a modern approach 41

Model-based reflex agents
• A model-based reflex agent. It keeps track of the current state of the world, using an internal model. It then
chooses an action in the same way as the reflex agent.
• Internal state that depends on the percept history and thereby reflects at least some of the unobserved aspects
of the current state.
• For the braking problem, the internal state is not too extensive—just the previous frame from the camera, allowing the agent to
detect when two red lights at the edge of the vehicle go on or off simultaneously.
• Updating this internal state information as time goes by requires two kinds of knowledge to be encoded in the
agent program.
• First, we need some information about how the world evolves independently of the agent—for example, that
an overtaking car generally will be closer behind than it was a moment ago.
• Second, we need some information about how the agent’s own actions affect the world—for example, that
when the agent turns the steering wheel clockwise, the car turns to the right, or that after driving for five
minutes northbound on the freeway, one is usually about five miles north of where one was five minutes ago

Artificial Intelligence a modern approach 42

Goal-based agents
• knowing state and environment? Enough?
– Taxi can go left, right, straight
• Have a goal
• A destination to get to
• Uses knowledge about a goal to guide its actions
• E.g., Search, planning

Artificial Intelligence a modern approach 43

Goal-based agents

• Reflex agent breaks when it sees brake lights. Goal based agent
reasons
– Brake light -> car in front is stopping -> I should stop -> I should use brake

Artificial Intelligence a modern approach 44

Utility-based agents
• Goals are not always enough
• Many action sequences get taxi to destination
• Consider other things. How fast, how safe…..
• A utility function maps a state onto a real number which describes the associated
degree of “happiness”, “goodness”, “success”.
• Where does the utility measure come from?
• Economics: money.
• Biology: number of offspring.
• Your life?

Artificial Intelligence a modern approach 45

Utility-based agents

Artificial Intelligence a modern approach 46

Learning agents
⚫ Performance element is
what was previously the
whole agent
⚫ Input sensor
⚫ Output action
⚫ Learning element
⚫ Modifies performance
element.

Artificial Intelligence a modern approach 47

Learning agents
⚫ Critic: how the agent is
doing
⚫ Input: checkmate?
⚫ Fixed

⚫ Problem generator
⚫ Tries to solve the problem
differently instead of
optimizing.
⚫ Suggests exploring new
actions -> new problems.

Artificial Intelligence a modern approach 48

Learning agents(Taxi driver)
• Performance element
• How it currently drives
• Taxi driver Makes quick left turn across 3 lanes
• Critics observe shocking language by passenger and other drivers and
informs bad action
• Learning element tries to modify performance elements for future
• Problem generator suggests experiment out something called Brakes
on different Road conditions
• Exploration vs. Exploitation
• Learning experience can be costly in the short run
• shocking language from other drivers
• Less tip
• Fewer passengers

Artificial Intelligence a modern approach 49

Problem Solving agents
Problem-solving agent
• When the correct action to take is not immediately obvious, an agent
may need to to plan ahead: to consider a sequence of actions that
form a path to a goal state. Such an agent is called a problem-solving
agent, and the computational process it undertakes is called search.
Example: Romania
• On Holiday in Romania currently in Arad.
• Flight leaves to Bucharest
• The agent can follow this four-phase problem-solving process:
• Goal formulation: The agent adopts the goal of reaching Bucharest.
• Problem formulation: The agent devises a description of the states and
actions necessary to reach the goal
• Search: Before taking any action in the real world, the agent simulates
sequences of actions in its model, searching until it finds a sequence of
actions that reaches the goal. Such a sequence is called a solution.
• Execution: The agent can now execute the actions in the solution, one at a
time.
A simplified road map of part of Romania,
with road distances in miles.
Search problems and solutions
• A search problem can be defined formally as follows:
• A set of possible states that the environment can be in. We call this the state
space.
• The initial state that the agent starts in. For example: Arad.
• A set of one or more goal states. Sometimes there is one goal state (e.g.,
Bucharest), sometimes there is a small set of alternative goal states, and
sometimes the goal is defined by a property that applies to many states
(potentially an infinite number). For example, in a vacuum-cleaner world, the
goal might be to have no dirt in any location, regardless of any other facts
about the state. We can account for all three of these possibilities by
specifying an IS-GOAL method for a problem
Search problems and solutions
• The actions available to the agent. Given a state s, ACTIONS(s) returns a finite
set of actions that can be executed in s. We say that each of these actions is
applicable in s.
• An example:
• ACTIONS(Arad) = {ToSibiu, ToTimisoara, ToZerind}.
• A transition model, which describes what each action does. RESULT(s, a) returns
the state that results from doing action a in state s. For example,
• RESULT(Arad, ToZerind) = Zerind .
Search problems and solutions
• An action cost function, denoted by ACTION-COST(s,a, s′) when we are
programming Action cost function or c(s,a, s′) when we are doing math, that
gives the numeric cost of applying action a in state s to reach state s′. A
problem-solving agent should use a cost function that reflects its own
performance measure.
• for example, for route-finding agents, the cost of an action might be the length in miles ,
or it might be the time it takes to complete the action.
Example problem: Vacuum world
• The vacuum world can be formulated as a grid world problem as follows:
• States: A state of the world says which objects are in which cells. For the vacuum world, the
objects are the agent and any dirt. In the simple two-cell version, the agent can be in either of
the two cells, and each call can either contain dirt or not, so there are2 · 2 · 2 = 8 states
• Initial state: Any state can be designated as the initial state.
• Actions: In the two-cell world we defined three actions: Suck, move Left, and move Right. In a
two-dimensional multi-cell world we need more movement actions. We could add Upward and
Downward, giving us four absolute movement actions, or we could switch to egocentric
actions, defined relative to the viewpoint of the agent—for example, Forward, Backward,
TurnRight, and TurnLeft.
• Transition model: Suck removes any dirt from the agent’s cell; Forward moves the agent ahead
one cell in the direction it is facing, unless it hits a wall, in which case the action has no effect.
Backward moves the agent in the opposite direction, while TurnRight and TurnLeft change the
direction it is facing by 90◦.
• Goal states: The states in which every cell is clean.
• Action cost: Each action costs 1.
The state-space graph for the two-cell
vacuum world.
Example: Sliding-tile puzzle
• The standard formulation of the 8 puzzle is as follows:
• States: A state description specifies the location of each of the tiles.
• Initial state: Any state can be designated as the initial state. Note that a parity
property
• Actions: While in the physical world it is a tile that slides, the simplest way of
describing an action is to think of the blank space moving Left, Right, Up, or Down.
If the blank is at an edge or corner then not all actions will be applicable.
• Transition model: Maps a state and action to a resulting state; for example, if we
apply Left to the start state in Figure 3.3, the resulting state has the 5 and the
blank switched.
• Goal state: Although any state could be the goal, we typically specify a state with
then numbers in order
• Action cost: Each action costs 1.
Example: Sliding-tile puzzle
Search algorithm
• A search algorithm takes a search problem as input and returns a
solution, or an indication of Search algorithm failure.
• We consider algorithms that superimpose a search tree over the state
space graph, forming various paths from the initial state, trying to find
a path that reaches a goal state.
• Each node in the search tree corresponds to a state in the state space
and the edges Node in the search tree correspond to actions. The
root of the treecorresponds to the initial state of the problem.
Best- First-Search
Best- First-Search
• Best-first search, chooses a node, n, with minimum value of some
Best-first search evaluation function, f (n).
• On each iteration we choose Evaluation function a node on the
frontier with minimum f (n) value, return it if its state is a goal state,
and otherwise apply EXPAND to generate child nodes.
• Each child node is added to the frontier if it has not been reached
before, or is re-added if it is now being reached with a path that has a
lower path cost than any previous path.
• The algorithm returns either an indication of failure, or a node that
represents a path to a goal.
Search data structures
• A node in the tree is represented by a data structure with four
components:
• node.STATE: the state to which the node corresponds;
• node.PARENT: the node in the tree that generated this node;
• node.ACTION: the action that was applied to the parent’s state to
generate this node;
• node.PATH-COST: the total cost of the path from the initial state to this
node.
Search data structures
The operations on a frontier are:
• IS-EMPTY(frontier) returns true only if there are no nodes in the
frontier.
• POP(frontier) removes the top node from the frontier and returns it.
• TOP(frontier) returns (but does not remove) the top node of the
frontier.
• ADD(node, frontier) inserts node into its proper place in the queue.
Search data structures
Three kinds of queues are used in search algorithms:
• A priority queue first pops the node with the minimum cost according
to some evaluation function, f . It is used in best-first search.
• A FIFO queue or first-in-first-out queue first pops the node that was
added to the queue first; we shall see it is used in breadth-first
search.
• A LIFO queue or last-in-first-out queue (also known as a stack) pops
first the most recently added node; we shall see it is used in
depth-first search.
Measuring problem-solving performance
We can evaluate an algorithm’s performance in four ways:
• Completeness: Is the algorithm guaranteed to find a solution when there is
one, and to Completeness correctly report failure when there is not?
• Cost optimality: Does it find a solution with the lowest path cost of all
solutions?
• Time complexity: How long does it take to find a solution? This can be
measured in Time complexity seconds, or more abstractly by the number of
states and actions considered.
• Space complexity: How much memory is needed to perform the search?
• Systematic: a search algorithm must be systematic in the way it explores an
infinite Systematic state space, making sure it can eventually reach any
state that is connected to the initial state
Uninformed search strategies
• An uninformed search algorithm is given no clue about how close a
state is to the goal(s).
• Uninformed search methods have access only to the problem
definition. Algorithms build a search tree in an attempt to find a
solution. Algorithms differ based on which node they expand first:
• For example, consider our agent in Arad with the goal of reaching
Bucharest. An uninformed agent with no knowledge of Romanian
geography has no clue whether going to Zerind or Sibiu is a better
first step.
Breadth-first search
• When all actions have the same cost, an appropriate strategy is
breadth-first search, in which the root node is expanded first, then all the
successors of the root node are expanded next, then their successors, and
so on.
• This is a systematic search strategy that is therefore complete even on
infinite state spaces.
• Breadth-first search always finds a solution with a minimal number of
actions, because when it is generating nodes at depth d, it has already
generated all the nodes at depth d − 1, so if one of them were a solution, it
would have been found. That means it is cost-optimal for problems where
all actions have the same cost, but not for problems that don’t have that
property
Breadth-first search
Breadth-first search algorithm
Dijkstra’s algorithm or uniform-cost search
• When actions have different costs, an obvious choice is to use
best-first search where the evaluation function is the cost of the path
from the root to the current node. This is called Dijkstra’s algorithm
by the theoretical computer science community, and uniform-cost
search Uniform-cost search by the AI community
Dijkstra’s algorithm: Example
Uniform-cost search
• Uniform-cost search is complete and is cost-optimal, because the first
solution it finds will have a cost that is at least as low as the cost of
any other node in the frontier. Uniform cost search considers all paths
systematically in order of increasing cost, never getting caught going
down a single infinite path
Depth-first search
• Depth-first search always expands the deepest node in the frontier
first. It could be implemented as a call to BEST-FIRST-SEARCH where
the evaluation function f is the negative of the depth.
• It is implemented as a tree-likecsearch that does not keep a table of
reached states.
Depth-first search
• Depth-first search is not cost-optimal; it returns the first solution it
finds, even if it is not cheapest.
• In cyclic state spaces it can get stuck in an infinite loop; therefore
some implementations of depth-first search check each new node for
cycles.
• Infinite state spaces, depth first search is not systematic: it can get
stuck going down an infinite path, even if there are no cycles. Thus,
depth-first search is incomplete.
• A depth-first tree-like search takes time proportional to the number
of states, and has memory complexity of only O(bm), where b is the
branching factor and m is the maximum depth of the tree.
Depth-first search
Depth-limited search
• To keep depth-first search from wandering down an infinite path, we
can use depth-limited Depth-limited search, a version of depth-first
search in which we supply a depth limit, ℓ, and treat all nodes at
depth ℓ as if they had no successors.
• The time complexity is O(bl) and the space complexity is O(bℓ).
• Unfortunately, if we make a poor choice for ℓ the algorithm will fail to
reach the solution, making it incomplete again.
Iterative deepening search
• Iterative deepening search solves the problem of picking a good value
for ℓ by trying search all values: first 0, then 1, then 2, and so
on—until either a solution is found, or the depth limited search
returns the failure value rather than the cutoff value.
• Iterative deepening combines many of the benefits of depth-first and
breadth-first search.
• Like depth-first search, its memory requirements are modest: O(bd) when
there is a solution, or O(bm) on finite state spaces with no solution.
• Like breadth-first search, iterative deepening is optimal for problems where
all actions have the same cost, and is complete on finite acyclic state spaces,
or on any finite state space when we check nodes for cycles all the way up
the path.
• The time complexity is O(bd ) when there is a solution, or O(bm) when
there is none.
• Each iteration of iterative deepening search generates a new level, in
the same way that breadth first search does, but breadth-first does
this by storing all nodes in memory, while iterative deepening does it
by repeating the previous levels, thereby saving memory at the cost
of more time.
• In general, iterative deepening is the preferred uninformed search
method when the search state space is larger than can fit in memory
and the depth of the solution is not known.
Iterative deepening and depth-limited
tree-like search.
Iterative deepening and depth-limited
tree-like search.
• Iterative deepening repeatedly applies depth-limited search with increasing
limits.
• It returns one of three different types of values:
• either a solution node;
• or failure, when it has exhausted all nodes and proved there is no solution at any
depth;
• or cutoff , to mean there might be a solution at a deeper depth than ℓ.
• This is a tree-like search algorithm that does not keep track of reached
states, and thus uses much less memory than best-first search, but runs the
risk of visiting the same state multiple times on different paths.
• Also, if the IS-CYCLE check does not check all cycles, then the algorithm may
get caught in a loop.
Bidirectional search
• A bidirectional search simultaneously searches forward from the
initial state and backwards from the goal state(s), hoping that the two
searches will meet. The motivation is that bd/2 + bd/2 is much less than
bd (e.g., 50,000 times less when b=d=10).
Bidirectional best-first search
Bidirectional best-first search
• Bidirectional best-first search keeps two frontiers and two tables of
reached states.
• When a path in one frontier reaches a state that was also reached in
the other half of the search, the two paths are joined (by the function
JOIN-NODES) to form a solution.
• The first solution we get is not guaranteed to be the best; the
function TERMINATED determines when to stop looking for new
solutions.
Comparing uninformed search algorithms
Informed search
• Informed search is one that uses domain-specific hints about the
location of goals.
• It can find solutions more efficiently than an uninformed strategy.
• The hints come in the form of a heuristic function, denoted h(n).
• h(n) = estimated cost of the cheapest path from the state at node n to
a goal state.
• Informed search methods have access to a heuristic function h(n) that
estimates the cost of a solution from n. They may have access to
additional information such as pattern databases with solution costs.
Greedy best-first search
• Greedy best-first search expands nodes with minimal h(n). It is not
optimal but is often efficient.
Roadmap of Romania with distance in miles
between cities
Values of hSLD
—straight-line distances to Bucharest.
Stages in a greedy best-first tree-like search for Bucharest with the straight-line
distance heuristic hSLD
Stages in a greedy best-first tree-like search for Bucharest with the straight-line
distance heuristic hSLD
Stages in a greedy best-first tree-like search for Bucharest with the straight-line
distance heuristic hSLD
Stages in a greedy best-first tree-like search for Bucharest with the straight-line
distance heuristic hSLD
 ∗
A search
• A∗ search expands nodes with minimal f (n) = g(n)+h(n).
• A∗ is complete and optimal, provided that h(n) is admissible.
• A key property is admissibility: an admissible heuristic is one that
never overestimates the cost to reach a goal.
• An admissible heuristic is therefore optimistic.
Stages in an A∗ search for Bucharest.
Stages in an A∗ search for Bucharest.
Stages in an A∗ search for Bucharest.
Stages in an A∗ search for Bucharest.
Stages in an A∗ search for Bucharest.
Stages in an A∗ search for Bucharest.
IDA*
DF Branch & Bound
Example DF B & B
Heuristic function
Dominance
Relaxed Problems
Local Search
Path vs. State optimization
• Previous lecture: Path to the goal is solution to previous problem
• Systematic exploration of the search state
• This lecture: a state is solution to problem
• For some problems path is irrelevant
• E.g. 8-queens
• Different algorithms can be used
• Depth first branch and bound
• Local search
Satisfaction vs. optimization
Local Search and Optimization
Example n queens
Hill Climbing Search: 8 queens problem

Need to convert to optimization problem

H=number of pairs of queens attacking that are each other
H=17 for above state.
Search space
Hill Climbing Search: 8 queens problem
Trivial algorithms
Hill Climbing(Greedy Local Search)
max verion
function HILL-CLIMBING(problem) returns a state that is a local maximum
current←problem.INITIAL
while true do
neighbor←a highest-valued successor state of current
If VALUE(neighbor) ≤ VALUE(current) then return current
current←neighbor
• The hill-climbing search algorithm, which is the most basic local search technique.
• At each step the current node is replaced by the best neighbor
Hill climbing Search
A one-dimensional state-space landscape in which elevation
corresponds to the objective function.
Hill climbing
Simulated annealing
• In metallurgy, annealing Simulated annealing is the process used to
temper or harden metals and glass by heating them to a high
temperature and then gradually cooling them, thus allowing the
material to reach a low-energy crystalline state.
Simulated annealing
Simulated annealing
Simulated annealing
• The overall structure of the simulated-annealing algorithm is similar to hill
climbing. Instead of picking the best move, however, it picks a random
move.
• If the move improves the situation, it is always accepted.
• Otherwise, the algorithm accepts the move with some probability less than
1. The probability decreases exponentially with the “badness” of the
move—the amount DE by which the evaluation is worsened. The
probability also decreases as the “temperature” T goes down: “bad” moves
are more likely to be allowed at the start when T is high, and they become
more unlikely as T decreases. If the schedule
• lowers
∆E/T
T to 0 slowly enough, then a property of the Boltzmann distribution,
e , is that all the probability is concentrated on the global maxima, which
the algorithm will find with probability approaching 1.
Application of Simulated Annealing
Local Beam Search
Local Beam Search
Genetic Algorithm
Example: 8 Queens problem
Example: 8 Queens problem
Example: 8 Queens problem
 Searching in continuous statespaces (brieFly...)
Observation: so far, states have been discrete “moves” apart
• Each “move” corresponds to an “atomic action” (can’t do a half-action! 1/16 action
• But the real world is generally a continuous space!
• What if we want to plan in real world space, rather than logical space?

From researchGate.net

Katieluethgeospatial.blogspot.com
 Searching Continuous spaces

Example: Suppose we want to site three airports in Romania:

• 6-D state space defined by (x1, y2), (x2, y2), (x3, y3)
• objective function f (x1, y2, x2, y2, x3, y3) = sum of squared distances from each city
to nearest airport (six dimensional search space)

Approaches:
Discretization methods turn continuous space into discrete space
• e.g., empirical gradient search considers ±δ change in each coordinate
• If you make δ small enough, you get needed accuracy

Gradient methods actually compute a gradient vector as a continuous fn.

⎞
⎛
⎮∂ f ∂f ∂f ∂f ∂ f ∂ f ⎮
∇f ⎮
, , , , , ⎮
⎛
= ∂x1 ∂y1 ∂x2 ∂y2 ∂x3 ∂y3 ⎠

to increase/reduce f , e.g., by x ← x + α ∇
f (x)

Summary: interesting area, highly complex

 Searching with Non-deterministic actions

• So far: fully-observable, deterministic worlds.

– Agent knows exact state. All actions always produce one outcome.
– Unrealistic?

• Real world = partially observable, non-deterministic

– Percepts become useful: can tell agent which action occurred
– Goal: not a simple action sequence, but contingency plan

• Example: Vacuum world, v2.0

– Suck(p1, dirty)= (p1,clean)
and sometimes (p2, clean)
– Suck(p1, clean)= sometimes (p1,dirty)

– If start state=1, solution=

[Suck, if(state=5) then [right,suck] ]
 AND-OR trees to represent non-determinism

• Need a different kind of search tree

– When search agent chooses an action: OR node
• Agent can specifically choose one action or another to include in plan.
• In Ch3 : trees with only OR nodes.

– Non-deterministic action= there may be several possible outcomes

• Plan being developed must cover all possible outcomes
• AND node: because must plan down all branches too.

• Search space is an AND-OR tree

– Alternating OR and AND layers
– Find solution= search this tree using same methods from Ch3.

• Solution in a non-deterministic search space

– Not simple action sequence
– Solution= subtree within search tree with:
• Goal node at each leaf (plan covers all contingencies)
• One action at each OR node
• A branch at AND nodes, representing all possible outcomes

• Execution of a solution = essentially “action, case-stmt, action, case-stmt”.

AND-OR SEARCH ALGORITHM
 Non-deterministic search trees

• Start state = 1

• One solution:
1. Suck,
2. if(state=5) then
[right,suck] ]

• What about the “loop”

leaves?
– Dead end?
– Discarded?
 Non-determinism: Actions that
fail
• Action failure is often a non-deterministic
outcome
– Creates a cycle in the search tree

• If no successful solution (plan) without a

cycle:
– May return a solution that contains a
cycle
– Represents retrying the action

• Infinite loop in plan execution?

– Depends on environment
• Action guaranteed to succeed
eventually?
– In practice: can limit loops
• Plan no longer complete (could fail)
 Searching with Partial Observations

• Previously: Percept gives full picture of state

– eg. Whole chess board, whole boggle board, entire robot maze

• Partial Observation: incomplete glimpse of current state

– Agent’s percept: zero <= percept < full state
– Consequence: we don’t always know exactly what state we’re in.

• Concept of believe state

– set of all possible states agent could be in.

• Find a solution (action sequence) that the leads to goal

– Actions applied to a believe state 🡪 new believe state based on union of that
action applied to all real states within believe state
 Conformant (sensorless) search

• Worst possible case: percept= null.Blind!

– Actually quite useful: finds plan that works regardless of sensor failure

• Plan:
– Build a belief state space based on the real state space
– Search that state space using the usual search techniques!

• Belief state space:

– Believe states: Power-set(real states).
• Huge! All possible combinations! N physical states = 2N believe states!
• Usually: only small subset actually reachable!

– Initial State: All states in world

• No sensor input = no idea what state I’m really in.
• So I “believe” I might be in any of them.
 Conformant (sensorless) search
• Belief state space (cont.):
– Actions: basically same actions as in physical space.
• For simplicity: Assume that illegal actions have no effect
• Example: Move(left, p1) = p1 if p1 is the left edge of the board.
• Can adapt for contexts in which illegal actions are fatal (more complex).

– Transitions (applying actions):

• Essentially take Union of action applied to all physical states in belief state
• Example: b={s1,s2,s3), then action(b) = Union( action(s1), action(s2),action(s3) )
• If non-deterministic actions: just Union the set of states that each action produces.

– Goal Test: Plan must work regardless!

• Believe state is goal iff all physical states it contains are goals!

– Path cost: tricky

• What if a given action has different costs of different physical states?
• Assume for now: all actions = same cost in all physical states.

• With this framework:

– can *automatically* construct belief space from any physical space
– Now simply search belief space using standard algos.
 Conformant (sensorless) search: Example
space

Start!

• Belief state space for the super simple vacuum world

Goal states
• Observations:
– Only 12 reachable states. Versus 28= 256 possible belief states
– State space still gets huge very fast! 🡪 seldom feasible in practice
– We need sensors! 🡪 Reduce state space greatly!
 Searching with Observations (percepts)
• Obviously: must state what percepts are available

– Specify what part of “state” is observable at each percept

– Ex: Vacuum knows position in room, plus if local square dirty

• But no info about rest of squares/space.
• In state 1, Percept = [A, dirty]
• If sensing non-deterministic 🡪 could return a set of possible percepts 🡪
multiple possible belief states

• So now transitions are:

– Predict: apply action to each physical
states in belief state to get new belief
state
• Like sensorless
– Observe: gather possible
percepts
• Or percepts, if non-det.
– Update: filter belief state based on
percepts
 Example: partial percepts

• Initial percept = [A, dirty]

• Partial observation = partial certainty
– Percept could have been produced by several states (1...or 3)
– Predict: Apply Action 🡪 new belief state
– Observe: Consider possible percepts in new b-state
– Update: New percepts then prune belief space
• Percepts (may) rule out some physical states in the belief state.
• Generates successor options in tree

– Look! Updated belief states no larger than parents!!

• Observations can only help reduce uncertainty 🡪 much better than sensorless state
space explosion!
 Searching/acting in partially observable worlds
• Searching for goal = find viable plan
– Use same standard search techniques
• Nodes, actions, successors
• Dynamically generate AND-OR tree
• Goal = subtree where all leaves are goal states
– Just like sensorless...but pruned by percepts!

• Action! An agent to execute the plan you find

– Execute the conditional plan that was produced
• Branches at each place where multiple percepts possible.
• Agent tests its actual percept at branch points 🡪 follows branch
• Maintains its current belief state as it goes
 Online Search

• So far: Considered “offline” search problem

– Works “offline” 🡪 searches to compute a whole plan...before ever acting
– Even with percepts 🡪 gets HUGE fast in real world
• Lots of possible actions, lots of possible percepts...plus non-det.

• Online search
– Idea: Search as you go. Interleave search + action
– Pro: actual percepts prune huge subtrees of search space @ each move
– Con: plan ahead less 🡪 don’t foresee problems
• Best case = wasted effort. Reverse actions and re-plan
• Worst case: not reversible actions. Stuck!

• Online search only possible method in some worlds

– Agent doesn’t know what states exist (exploration problem)
– Agent doesn’t know what effect actions have (discovery learning)
– Possibly: do online search for awhile
• until learn enough to do more predictive search
 The nature of active online search

• Executing online search = algorithm for planning/acting

– Very different than offline search algos!
– Offline: search virtually for a plan in constructed search space...
• Can use any search algorithm, e.g., A* with strong h(n)
• A* can expand any node it wants on the frontier (jump around)

– Online agent: Agent literally is in some place!

• Agent is at one node (state) on frontier of search tree
• Can’t just jump around to other states...must plan from current state.
• (Modified) Depth first algorithms are ideal candidates!

– Heuristic functions remain critical!

• H(n) tells depth first which of the successors to explore!
• Admissibility remains relevant too: want to explore likely optimal paths first
• Real agent = real results. At some point I find the goal
– Can compare actual path cost to that predicted at each state by H(n)
– Competitive Ratio: Actual path cost/predicted cost. Lower is better.
– Could also be basis for developing (learning!) improved H(n) over time.
 Online Local Search for Agents

• What if search space is very bushy?

– Even IDS version of depth-first are too costly
– Tight time constraints could also limit search time
• Can use our other tool for local search!
– Hill-climbing (and variants)

• Problem: agents in in the physical world, operating

– Random restart methods for avoiding local minima are problematic
• Can’t just move robot back to start all the time!
– Random Walk approaches (highly stochastic hill-climbing) can work
– Will eventually wander across the goal place/state.

• Random walk + memory can be helpful

– Chooses random moves but…
– remembers where it’s been, and updates costs along the way
– Effect: can “rock” its way out of local minima to continue search
 Online Local Search for Agents

• Result: Learning Real-time A*

(LRTA*)

• Idea: memory = update the h(n) for nodes you’ve visited

– When stuck use: h(n) = cost(n 🡪 best neighbor) + h(neighbor)
– Update the h(n) to reflect this. If you ever go back there, h(n) is higher
– You “fill in” the local minimum as you cycle a few times. Then escape...

• LRTA* 🡪 many variants; vary in selecting next action and updating rules