Chapter 3

Searching and Planning

Outline

A. Solving Problems by Searching

B. Beyond Classical Search
C. Adversarial Search
D. Constraint Satisfaction Problems
A. Solving problems by searching

 When the correct action to take is not immediately obvious,

an agent may need to plan ahead: to consider a sequence of
actions that form a path to a goal state.
 Such an agent is called a problem-solving agent, and the
computational process it undertakes is called search.
 Problem-solving agents use atomic representations.
 Agents that use factored or structured representations of
states are called planning agents.
 Episodic, single-agent, fully observable, deterministic, static,
discrete, and known environments are considered here.
 i. Problem-Solving agents

 Figure 1: Simplified road map of part of Romania, road distances in miles.

Problem-Solving agents…
 Suppose the agent is currently in the city of Arad and has a
nonrefundable ticket to fly out of Bucharest the following day.
 The agent observes street signs and sees that there are three
roads leading out of Arad: one toward Sibiu, one to Timisoara,
and one to Zerind.
 None of these are the goal, so unless the agent is familiar with
the geography of Romania, it will not know which road to
follow.
 If the agent has no additional information—that is, if the
environment is unknown—then the agent can do no better than
to execute one of the actions at random.
Problem-Solving agents…
 In this approach, we will assume our agents always have access to
information about the world, such as the map in Figure 1.
 With that information, the agent can follow this four-phase problem-
solving process:
1. GOAL FORMULATION: The agent adopts the goal of reaching Bucharest.
Goals organize behavior by limiting the objectives and hence the actions
to be considered.
2. PROBLEM FORMULATION: The agent devises a description of the states
and actions necessary to reach the goal—an abstract model of the
relevant part of the world.
 For our agent, one good model is to consider the actions of traveling
from one city to an adjacent city, and therefore the only fact about the
state of the world that will change due to an action is the current city.
Problem-Solving agents…

 SEARCH: Before taking any action in the real world, the agent
simulates sequences of actions in its model, searching until it
finds a sequence of actions that reaches the goal - a solution.
 The agent might have to simulate multiple sequences that
do not reach the goal, but eventually it will find a solution
(such as going from Arad to Sibiu to Fagaras to Bucharest),
or it will find that no solution is possible.
 EXECUTION: The agent can now execute the actions in the
solution, one at a time.
Problem-Solving agents…
 It is an important property that in a fully observable,
deterministic, known environment, the solution to any problem
is a fixed sequence of actions: drive to Sibiu, then Fagaras,
then Bucharest.
 If the model is correct, then once the agent has found a
solution, it can ignore its percepts while it is executing the
actions—closing its eyes, so to speak—because the solution is
guaranteed to lead to the goal.
 Control theorists call this an open-loop system: ignoring the
percepts breaks the loop between agent and environment.
 If there is a chance that the model is incorrect, or the
environment is nondeterministic, then the agent would be safer
using a closed-loop approach that monitors the percepts.
Problem-Solving agents…

 In partially observable or nondeterministic environments,

a solution would be a branching strategy that recommends
different future actions depending on what percepts
arrive.
 For example, the agent might plan to drive from Arad to
Sibiu but might need a contingency plan in case it arrives
in Zerind by accident or finds a sign saying “Drum Închis”
(Road Closed).
Search problems and solutions

A search problem can be defined formally as follows:

 A set of possible states that the environment can be in. We
call this the state space.
 The initial state that the agent starts in. For example: Arad.
 A set of one or more goal states. Sometimes there is one goal
state (e.g., Bucharest), sometimes there is a small set of
alternative goal states, and sometimes the goal is defined by a
property that applies to many states (potentially an infinite
number).
Search problems and solutions…

 The actions available to the agent. Given a state s, ACTIONS(s)

returns a finite set of actions that can be executed in s.
 We say that each of these actions is applicable in s. For
example:
ACTIONS(Arad) = {ToSibiu, ToTimisoara, ToZerind}
 A transition model, which describes what each action does.
RESULT(s, a) returns the state that results from doing action a
in state s. For example,
RESULT(Arad, ToZerind) = Zerind
Search problems and solutions…

 An action cost function, denoted by ACTION-COST(s, a, s′)

when we are programming or c(s, a, s′) when we are doing
math, that gives the numeric cost of applying action in state
s to reach state s’.
 A problem-solving agent should use a cost function that
reflects its own performance measure;
 For example, for route-finding agents, the cost of an action
might be the length in miles, or it might be the time it takes
to complete the action.
Search problems and solutions…

 A sequence of actions forms a path, and a solution is a path

from the initial state to a goal state.
 We assume that action costs are additive; that is, the total
cost of a path is the sum of the individual action costs.
 An optimal solution has the lowest path cost among all
solutions. In this section, we assume that all action costs will
be positive, to avoid certain complications.
 The state space can be represented as a graph in which the
vertices are states and the directed edges between them are
actions.
Formulating problems

 Our formulation of the problem of getting to Bucharest is a

model—an abstract mathematical description—and not the
real thing.
 All other considerations are left out of our model because
they are irrelevant to the problem of finding a route to
Bucharest.
 The process of removing detail from a representation is
called abstraction.
 A good problem formulation has the right level of detail.
Formulating problems…

 Can we be more precise about the appropriate level of

abstraction?
 The abstraction is valid if we can elaborate any abstract
solution into a solution in the more detailed world;
 The abstraction is useful if carrying out each of the actions in
the solution is easier than the original problem;
 In our case, the action “drive from Arad to Sibiu” can be
carried out without further search or planning by a driver
with average skill.
 The choice of a good abstraction thus involves removing as
much detail as possible while retaining validity and ensuring
that the abstract actions are easy to carry out.
 Were it not for the ability to construct useful abstractions,
intelligent agents would be completely swamped by the real
world.
ii. Example problems

 A standardized problem is intended to illustrate or exercise

various problem-solving methods.
 It can be given a concise, exact description and hence is
suitable as a benchmark for researchers to compare the
performance of algorithms.
 A real-world problem, such as robot navigation, is one whose
solutions people actually use, and whose formulation is
idiosyncratic, not standardized, because, for example, each
robot has different sensors that produce different data.
Standardized problems

 A grid world problem is a two-dimensional rectangular

array of square cells in which agents can move from cell to
cell.
 Typically the agent can move to any obstacle-free adjacent
cell — horizontally or vertically and in some problems
diagonally.
 Cells can contain objects, which the agent can pick up,
push, or otherwise act upon; a wall or other impassible
obstacle in a cell prevents an agent from moving into that
cell.
Grid world

 Figure 2: The state-space graph for the two-cell vacuum world. There are 8
states and three actions for each state: L = Left, R = Right, S = Suck
Grid world…

 States: A state of the world says which objects are in which

cells.
 For the vacuum world, the objects are the agent and any dirt.
 In the simple two-cell version, the agent can be in either of
the two cells, and each call can either contain dirt or not, so
there are 2 · 2 · 2 = 8 states (see Figure 2).
 In general, a vacuum environment with n cells has n·2^n
states.
 Initial state: Any state can be designated as the initial state.
Grid world…

 Actions: In the two-cell world we defined three actions:

Suck, move Left, and move Right.
 In a two-dimensional multi-cell world we need more
movement actions.
 We could add Upward and Downward, giving us four
absolute movement actions, or
 We could switch to egocentric actions, defined relative to
the viewpoint of the agent—for example, Forward,
Backward, TurnRight, and TurnLeft.
Grid world…

 Transition model:
 Suck removes any dirt from the agent’s cell;
 Forward moves the agent ahead one cell in the direction it is
facing, unless it hits a wall, in which case the action has no
effect.
 Backward moves the agent in the opposite direction, while
TurnRight and TurnLeft change the direction it is facing by 90◦.
 Goal states: The states in which every cell is clean.
 Action cost: Each action costs 1.
Real-world problems

 We have already seen how the route-finding problem is

defined in terms of specified locations and transitions along
edges between them.
 Route-finding algorithms are used in a variety of applications.
 Some, such as Web sites and in-car systems that provide
driving directions, are relatively straightforward extensions of
the Romania example.
Real-world problems…

 The main complications are varying costs due to traffic-

dependent delays, and rerouting due to road closures.
 Others, such as routing video streams in computer networks,
military operations planning, and airline travel-planning
systems, involve much more complex specifications.
 Consider the airline travel problems that must be solved by a
travel-planning Web site:
Real-world problems…

 States: Each state obviously includes a location (e.g., an

airport) and the current time.
 Furthermore, because the cost of an action (a flight segment)
may depend on previous segments, their fare bases, and their
status as domestic or international, the state must record
extra information about these “historical” aspects.
 Initial state: The user’s home airport.
Real-world problems…

 Actions: Take any flight from the current location, in any

seat class, leaving after the current time, leaving enough
time for within-airport transfer if needed.
 Transition model: The state resulting from taking a flight
will have the flight’s destination as the new location and
the flight’s arrival time as the new time.
Real-world problems…

 Goal state: A destination city. Sometimes the goal can be

more complex, such as “arrive at the destination on a
nonstop flight.”
 Action cost: A combination of monetary cost, waiting time,
flight time, customs and immigration procedures, seat
quality, time of day, type of airplane, frequent-flyer reward
points, and so on.
Real-world problems…
 Touring problems describe a set of locations that must be
visited, rather than a single goal destination.
 The traveling salesperson problem (TSP) is a touring problem in
which every city on a map must be visited.
 The aim is to find a tour with cost < C (or in the (TSP)
optimization version, to find a tour with the lowest cost
possible).
 An enormous amount of effort has been expended to improve the
capabilities of TSP algorithms.
 The algorithms can also be extended to handle fleets of vehicles.
 For example, a search and optimization algorithm for routing
school buses in Boston saved $5 million, cut traffic and air
pollution, and saved time for drivers and students (Bertsimas et
al., 2019).
Real-world problems…

 A VLSI layout problem requires positioning millions of

components and connections on a chip to minimize area,
minimize circuit delays, minimize stray capacitances, and
maximize manufacturing yield.
 The layout problem comes after the logical design phase and is
usually split into two parts: cell layout and channel routing.
 In cell layout, the primitive components of the circuit are
grouped into cells, each of which performs some recognized
function.
Real-world problems…

 Each cell has a fixed footprint (size and shape) and requires a
certain number of connections to each of the other cells.
 The aim is to place the cells on the chip so that they do not
overlap and so that there is room for the connecting wires to
be placed between the cells.
 Channel routing finds a specific route for each wire through
the gaps between the cells.
 These search problems are extremely complex, but definitely
worth solving.
Real-world problems…

 Robot navigation is a generalization of the route-finding

problem described earlier.
 Rather than following distinct paths (such as the roads in
Romania), a robot can roam around, in effect making its own
paths.
 For a circular robot moving on a flat surface, the space is
essentially two-dimensional.
 When the robot has arms and legs that must also be
controlled, the search space becomes many-dimensional—one
dimension for each joint angle.
Real-world problems…

 Advanced techniques are required just to make the

essentially continuous search space finite.
 In addition to the complexity of the problem, real robots
must also deal with errors in their sensor readings and
motor controls, with partial observability, and with other
agents that might alter the environment.
Real-world problems…

 Automatic assembly sequencing of complex objects (such as

electric motors) by a robot has been standard industry
practice since the 1970s.
 Algorithms first find a feasible assembly sequence and then
work to optimize the process.
 Minimizing the amount of manual human labor on the
assembly line can produce significant savings in time and cost.
 In assembly problems, the aim is to find an order in which to
assemble the parts of some object.
Real-world problems…

 If the wrong order is chosen, there will be no way to add some

part later in the sequence without undoing some of the work
already done.
 Checking an action in the sequence for feasibility is a
difficult geometrical search problem closely related to robot
navigation.
 Thus, the generation of legal actions is the expensive part of
assembly sequencing.
 Any practical algorithm must avoid exploring all but a tiny
fraction of the state space.
Real-world problems…

 One important assembly problem is protein design, in

which the goal is to find a sequence of amino acids that
will fold into a three-dimensional protein with the right
properties to cure some disease.