Automated Planning

Supervisor: Asia Mahdi

By: Noor Sabah Ail AlSharify

Introduction and Overview

Overview
What is AI Planning?
 A Conceptual Model for Planning

 Restricting Assumptions

 A Running Example: Dock-Worker

Robots

Automated Planning: Introduction and Overview 2

Defining AI Planning
 planning:
• explicit deliberation process that chooses and
organizes actions by anticipating their
outcomes
• aims at achieving some pre-stated objectives
 AI planning:
• computational study of this deliberation
process

Automated Planning: Introduction and Overview 3

Why Study Planning in AI?
 scientific goal of AI:
understand intelligence
• planning is an important component of
rational (intelligent) behaviour

 engineering goal of AI:

build intelligent entities
• build planning software for choosing and
organizing actions for autonomous intelligent
machines

Automated Planning: Introduction and Overview 4

Domain-Specific vs.
Domain-Independent Planning
 domain-specific planning: use specific
representations and techniques adapted to each
problem
• important domains: path and motion planning, perception
planning, manipulation planning, communication planning
 domain-independent planning: use generic
representations and techniques
• exploit commonalities to all forms of planning
• leads to general understanding of planning

 domain-independent planning
complements domain-specific planning

Automated Planning: Introduction and Overview 5

Overview
 What is AI Planning?
A Conceptual Model for Planning
 Restricting Assumptions
 A Running Example: Dock-Worker
Robots

Automated Planning: Introduction and Overview 6

Why a Conceptual Model?
 conceptual model: theoretical device for
describing the elements of a problem
 good for:
• explaining basic concepts
• clarifying assumptions
• analyzing requirements
• proving semantic properties
 not good for:
• efficient algorithms and computational concerns

Automated Planning: Introduction and Overview 7

Conceptual Model for Planning:
State-Transition Systems
 A state-transition system is a 4-tuple
Σ =(S,A,E,γ), where:
• S = { s1,s2,…} is a finite or recursively enumerable set
of states;
• A = {a1,a2,…} is a finite or recursively enumerable set
of actions;
• E = { e1,e2,…} is a finite or recursively enumerable
set of events; and
• γ : S×(AE)→2S is a state transition function.
 if aA and γ(s,a)   then a is applicable in s
 applying a in s will take the system to sγ (s,a(
Automated Planning: Introduction and Overview 8
State-Transition Systems as
Graphs

 A state-transition system Σ =
(S,A,E,γ) can be represented by a
directed labelled graph G = (NG,EG)
where:
• the nodes correspond to the states in S, i.e.
NG=S ; and
• there is an arc from sNG to sNG,i.e. s→s
EG,with label u(AE) if and only if sγ (s,a).

Automated Planning: Introduction and Overview 9

State-Transition Graph Example:
Missionaries and Cannibals
c1 c1
c2 c2

m1 m1 m1 m1
c2 c2
c1 c1

c1 c1

c1 c1

m2 m2

m1

c1

Automated Planning: Introduction and Overview 10

Objectives and Plans
 state-transition system:
• describes all ways in which a system may evolve
 plan:
• a structure that gives appropriate actions to apply in
order to achieve some objective when starting from a
given state
 types of objective:
• goal state sg or set of goal states Sg
• satisfy some conditions over the sequence of states
• optimize utility function attached to states
• task to be performed

Automated Planning: Introduction and Overview 11

 Planning and Plan Execution
Description of Σ
 planner:
• given: description of Σ, initial
Initial State state, objective
Objectives
Planner • generate: plan that achieves
objective
Plan
 controller:
Controller • given: plan, current state
(observation function: η:S→O(
Observations Actions • generate: action
System Σ  state-transition system:
• evolves as actions are executed
and events occur.
Events

Automated Planning: Introduction and Overview 12

Overview
 What is AI Planning?
 A Conceptual Model for Planning
Restricting Assumptions
 A Running Example: Dock-Worker
Robots

Automated Planning: Introduction and Overview 13

A0: Finite Σ
 Assumption A0
• system Σ has a finite set of states
 Relaxing A0
• why?
• to describe actions that construct or bring new
objects into the world
• to handle numerical state variables
• issues:
• decidability and termination of planners

Automated Planning: Introduction and Overview 14

A1: Fully Observable Σ
 Assumption A1
• system Σ is fully observable, i.e. η is the identity
function
 Relaxing A1
• why?
• to handle states in which not every aspect is or can be
known
• issues:
• if η(s)=o, η-1(o) usually more than one state (ambiguity(
• determining the successor state

Automated Planning: Introduction and Overview 15

A2: Deterministic Σ
 Assumption A2
• system Σ is deterministic, i.e. for all sS, uAE:
| γ (s,u)|1
• short form: γ (s,u)=s for γ (s,u)={s}
 Relaxing A2
• why?
• to plan with actions that may have multiple alternative
outcomes
• issues:
• controller has to observe actual outcomes of actions
• solution plan may include conditional and iterative
constructs

Automated Planning: Introduction and Overview 16

A3: Static Σ
 Assumption A3
• system Σ is static, i.e. E=
• short form: Σ =(S,A,γ) for Σ =(S,A,,γ(
Relaxing A3
• why?
• to model a world in which events can occur
• issues:
• world becomes nondeterministic from the point of
view of the planner (same issues)

Automated Planning: Introduction and Overview 17

A4: Restricted Goals
 Assumption A4
• the planner handles only restricted goals that are
given as an explicit goal state sg or set of goal states Sg
 Relaxing A4
• why?
• to handle constraints on states and plans, utility
functions, or tasks
• issues:
• representation and reasoning over constraints, utility,
and tasks

Automated Planning: Introduction and Overview 18

A5: Sequential Plans
 Assumption A5
• a solution plan is a linearly ordered finite sequence of
actions
 Relaxing A5
• why?
• to handle dynamic systems (see A3: static Σ(
• to create different types of plans
• issues:
• must not shift problem to the controller
• reasoning about (more complex) data structures

Automated Planning: Introduction and Overview 19

A6: Implicit Time
 Assumption A6
• actions and events have no duration in state transition
systems
 Relaxing A6
• why?
• to handle action duration, concurrency, and deadlines
• issues:
• representation of and reasoning about time
• controller must wait for effects of actions to occur

Automated Planning: Introduction and Overview 20

A7: Offline Planning
 Assumption A7
• planner is not concerned with changes of Σ
while it is planning
 Relaxing A7
• why?
• to drive a system towards some objectives
• issues:
• check whether the current plan remains valid
• if needed, revise current plan or re-plan

Automated Planning: Introduction and Overview 21

The Restricted Model
 restricted model: make assumptions A0-A7

 Given a planning problem P=(Σ,si,Sg) where

• Σ = (S,A,γ) is a state transition system,
• siS is the initial state, and
• Sg  S is a set of goal states,
 find a sequence of actions 〈 a1,a2,…,ak〉
• corresponding to a sequence of state transitions
〈si,s1,…,sk〉 such that
• s1= γ(si,a1), s2= γ(s1,a2),…, sk= γ(sk-1,ak), and skSg.
Automated Planning: Introduction and Overview 22
Overview
 What is AI Planning?
 A Conceptual Model for Planning
 Restricting Assumptions
A Running Example: Dock-Worker
Robots

Automated Planning: Introduction and Overview 23

The Dock-Worker Robots (DWR(
Domain
 aim: have one example to
illustrate planning procedures
and techniques
 informal description:
• harbour with several locations
(docks), docked ships, storage
areas for containers, and
parking areas for trucks and
trains
• cranes to load and unload ships
etc., and robot carts to move
containers around

Automated Planning: Introduction and Overview 24

 DWR Example State
crane
1k 2k

cc cf
container pallet
container cd ce
ca cd
1r
1l 2l
Pile (p1 and q1( robot pile (p2 and q2, both empty(

location

Automated Planning: Introduction and Overview 25

Actions in the DWR Domain
 move robot r from location l to some adjacent
and unoccupied location lʼ
 take container c with empty crane k from the
top of pile p, all located at the same location l
 put down container c held by crane k on top of
pile p, all located at location l
 load container c held by crane k onto
unloaded robot r, all located at location l
 unload container c with empty crane k from
loaded robot r, all located at location l

Automated Planning: Introduction and Overview 26

 State-Transition Systems: Graph
Example
crane crane crane
s0 move1 s2 s5

cont. cont. t
palle t pall pallet cont.
robot e
robot robot
location2 move2
location1 location1 location2 location1 location2

take put take put move1 move2

crane crane crane

s1 move1 s3 load s4
cont. cont.

pallet pallet pcont.

alle
robot robot t
robot
move2
location2 location1 unload
location1 location2 location1 location2

Automated Planning: Introduction and Overview 27

State-Space Search and
the STRIPS Planner

Searching for a Path through

a Graph of Nodes
Representing World States
Classical Representations
 propositional representation
• world state is set of propositions
• action consists of precondition propositions,
propositions to be added and removed
 STRIPS representation
• like propositional representation, but first-order literals
instead of propositions
 state-variable representation
• state is tuple of state variables {x1,…,xn{
• action is partial function over states

State-Space Search and the STRIPS Planner 29

Overview
The STRIPS Representation
 The Planning Domain Definition Language
(PDDL(
 Problem-Solving by Search
 Heuristic Search
 Forward State-Space Search
 Backward State-Space Search
 The STRIPS Planner

State-Space Search and the STRIPS Planner 30

STRIPS Planning Domains:
Restricted State-Transition Systems

 A restricted state-transition system is a triple

Σ=)S,A,γ), where:
• S={s1,s2,…{ is a set of states;
• A={a1,a2,…{ is a set of actions;
• γ:S×A→S is a state transition function.
 defining STRIPS planning domains:
• define STRIPS states
• define STRIPS actions
• define the state transition function

State-Space Search and the STRIPS Planner 31

States in the STRIPS
Representation
 Let L be a first-order language with finitely
many predicate symbols, finitely many
constant symbols, and no function symbols.
 A state in a STRIPS planning domain is a set
of ground atoms of L.
• )ground) atom p holds in state s iff p∈s
• s satisfies a set of )ground) literals g (denoted s ⊧ g) if:
• every positive literal in g is in s and
• every negative literal in g is not in s.

State-Space Search and the STRIPS Planner 32

 DWR Example: STRIPS States
state = {attached(p1,loc1),
attached(p2,loc1), in(c1,p1),
in(c3,p1), top(c3,p1),
on(c3,c1), on(c1,pallet), in(c2,p2), crane1

top(c2,p2), on(c2,pallet), c2

belong(crane1,loc1), c3
p2 r1
c1
empty(crane1), adjacent(loc1,loc2), pallet
loc2
loc1
adjacent(loc2, loc1), at(r1,loc2),
occupied(loc2), unloaded(r1)}

State-Space Search and the STRIPS Planner 33

Fluent Relations
 Predicates that represent relations, the
truth value of which can change from
state to state, are called a fluent or
flexible relations.
• example: at
 A state-invariant predicate is called a
rigid relation.
• example: adjacent
State-Space Search and the STRIPS Planner 34
Operators and Actions in
STRIPS Planning Domains
 A planning operator in a STRIPS planning
domain is a triple
o = (name(o), precond(o), effects(o)) where:
• the name of the operator name(o) is a syntactic
expression of the form n(x1,…,xk) where n is a
(unique) symbol and x1,…,xk are all the variables that
appear in o, and
• the preconditions precond(o) and the effects effects(o)
of the operator are sets of literals.
 An action in a STRIPS planning domain is a
ground instance of a planning operator.

State-Space Search and the STRIPS Planner 35

DWR Example: STRIPS
Operators
 move(r,l,m)
• precond: adjacent(l,m), at(r,l), ¬occupied(m)
• effects: at(r,m), occupied(m), ¬occupied(l), ¬at(r,l)
 load(k,l,c,r)
• precond: belong(k,l), holding(k,c), at(r,l),
unloaded(r)
• effects: empty(k), ¬holding(k,c), loaded(r,c),
¬unloaded(r)
 put(k,l,c,d,p)
• precond: belong(k,l), attached(p,l), holding(k,c), top(d,p)
• effects: ¬holding(k,c), empty(k), in(c,p), top(c,p), on(c,d),
¬top(d,p)
State-Space Search and the STRIPS Planner 36
Applicability and State
Transitions
 Let L be a set of literals.
• L+ is the set of atoms that are positive literals in L and
• L- is the set of all atoms whose negations are in L.
 Let a be an action and s a state. Then a is
applicable in s iff:
• precond+(a) ⊆s ; and
• precond-(a) ⋂ s = }{ .
 The state transition function γ for an applicable
action a in state s is defined as:
• γ(s,a) = (s ‒ effects-(a)) ∪ effects+(a(
State-Space Search and the STRIPS Planner 37
STRIPS Planning Domains
 Let L be a function-free first-order language. A
STRIPS planning domain on L is a restricted
state-transition system Σ=)S,A,γ) such that:
• S is a set of STRIPS states, i.e. sets of ground atoms
• A is a set of ground instances of some STRIPS
planning operators O
• γ : S×A→S where
• γ(s,a)=(s - effects-(a)) ∪ effects+(a) if a is applicable in s
• γ(s,a)=undefined otherwise
• S is closed under γ

State-Space Search and the STRIPS Planner 38

STRIPS Planning Problems
 A STRIPS planning problem is a triple
P=)Σ,si,g) where:
• Σ=)S,A,γ) is a STRIPS planning domain on
some first-order language L
• si∈S is the initial state
• g is a set of ground literals describing the
goal such that the set of goal states is: Sg=
}s∈S | s satisfies g {

State-Space Search and the STRIPS Planner 39

DWR Example: STRIPS Planning
Problem
 Σ: STRIPS planning domain for DWR domain
 Si : any state
• example: s0 = {attached(pile,loc1), crane
s0
in(cont,pile), top(cont,pile),
on(cont,pallet), belong(crane,loc1), Cont.
empty(crane), adjacent(loc1,loc2), pallet
robot
adjacent )loc2,loc1), at(robot,loc2), loc1 loc2
occupied(loc2), unloaded(robot({
 g : any subset of L crane
s5
• example: g = {¬unloaded(robot(,
at(robot,loc2)}, i.e. Sg={s5 { pallet cont.
robot
location1 location2

State-Space Search and the STRIPS Planner 40

Statement of a STRIPS Planning
Problem
 A statement of a STRIPS planning
problem is a triple P=(O,si,g) where:
• O is a set of planning operators in an
appropriate STRIPS planning domain Σ=
(S,A,γ) on L
• si is the initial state in an appropriate
STRIPS planning problem P=(Σ,si,g(
• g is a goal (set of ground literals) in the same
STRIPS planning problem P

State-Space Search and the STRIPS Planner 41

Classical Plans
 A plan is any sequence of actions π =〈 a,1
,…ak〉 , where k ≥ . 0
• The length of plan π is ¦π|=k, the number of actions.
• If π1=〈 a1,…,ak〉 and π2=〈 aʼ1,…,aʼj〉 are plans,
then their concatenation is the plan π1 ∙ π2 = 〈 a, 1
,…ak ,aʼ1,…,aʼj 〉 .
• The extended state transition function for plans is
defined as follows:
• γ(s,π)=s if k=0 (π is empty(
• γ(s,π)=γ(γ(s,a1),〈 a2,…,ak〉 ) if k >0 and a1 applicable in s
• γ(s,π)=undefined otherwise

State-Space Search and the STRIPS Planner 42

Classical Solutions
 Let P=)Σ,si,g) be a planning problem. A
plan π is a solution for P if γ(si , π)
satisfies g .
• A solution πis redundant if there is a proper
subsequence of πis also a solution for P.
• πis minimal if no other solution for P contains
fewer actions than π.

State-Space Search and the STRIPS Planner 43

DWR Example: Solution Plan

plan π1 =
〈 move(robot,loc2,loc1),
take(crane,loc1,cont,pallet,pile),
load(crane,loc1,cont,robot),
move(robot,loc1,loc2) 〉
 ¦π1|=4
 π1 is a minimal, non-redundant solution
State-Space Search and the STRIPS Planner 44
Overview
 The STRIPS Representation
The Planning Domain Definition Language
(PDDL(
 Problem-Solving by Search
 Heuristic Search
 Forward State-Space Search
 Backward State-Space Search
 The STRIPS Planner

State-Space Search and the STRIPS Planner 45

Overview
 The STRIPS Representation
 The Planning Domain Definition Language
(PDDL(
Problem-Solving by Search
 Heuristic Search
 Forward State-Space Search
 Backward State-Space Search
 The STRIPS Planner

State-Space Search and the STRIPS Planner 46

Search Problems
 initial state
 set of possible actions/applicability conditions
• successor function: state  set of <action, state>
• successor function + initial state = state space
• path (solution(
 goal
• goal state or goal test function
 path cost function
• for optimality
• assumption: path cost = sum of step costs

State-Space Search and the STRIPS Planner 47

Missionaries and Cannibals:
Initial State and Actions
 initial state:  5possible actions:
• all missionaries, all • one missionary crossing
cannibals, and the • one cannibal crossing
boat are on the left • two missionaries
bank crossing
• two cannibals crossing
• one missionary and one
cannibal crossing

State-Space Search and the STRIPS Planner 48

Missionaries and Cannibals:
Successor Function
state set of <action, state>
(L:3m,3c,b-R:0m,0c)  {<2c, (L:3m,1c-R:0m,2c,b)>,
<1m1c, (L:2m,2c-R:1m,1c,b)>,
<1c, (L:3m,2c-R:0m,1c,b)>}
(L:3m,1c-R:0m,2c,b)  {<2c, (L:3m,3c,b-R:0m,0c)>,
<1c, (L:3m,2c,b-R:0m,1c)>}
(L:2m,2c-R:1m,1c,b)  {<1m1c, (L:3m,3c,b-R:0m,0c)>,
<1m, (L:3m,2c,b-R:0m,1c)>}

State-Space Search and the STRIPS Planner 49

Missionaries and Cannibals:
State Space
1c 1c
2c 2c

1m 1m 1m 1m
1c 2c 2c 1c

1c 1c

1c 1c

2m 2m

1m
1c

State-Space Search and the STRIPS Planner 50

Missionaries and Cannibals:
Goal State and Path Cost
 goal state:  path cost
• all missionaries, all • step cost: 1 for each
cannibals, and the crossing
boat are on the right • path cost :number of
bank crossings = length of path
 solution path:
• 4 optimal solutions
• cost: 11

State-Space Search and the STRIPS Planner 51

Real-World Problem:
Touring in Romania
Oradea
71
Neamt
Zerind
151 87
75 Iasi

Arad 140
Sibiu Fagaras 92

118 99
80 Vaslui
Rimnicu Vilcea
Timisoara 97 211
Pitesti 142

111
Lugoj 101 85 98 Hirsova
70 146 Bucharest Urziceni
86
Mehadia 138
90
75 Eforie
Dobreta 120 Giurgiu
Craiova

State-Space Search and the STRIPS Planner 52

Touring Romania:
Search Problem Definition
 initial state:
• In(Arad(
 possible Actions:
• DriveTo(Zerind), DriveTo(Sibiu), DriveTo(Timisoara),
etc.
 goal state:
• In(Bucharest(
 step cost:
• distances between cities

State-Space Search and the STRIPS Planner 53

 Search Trees
 search tree: tree structure defined by initial
state and successor function
 Touring Romania (partial search tree(:
In(Arad(

In(Zerind( In(Sibi( In(Timisoara(

In(Arad( In(Oradea( In(Fagaras( In(Rimnicu Vilcea(

In(Sibiu( In(Bucharest(

State-Space Search and the STRIPS Planner 54

Search Nodes
 search nodes: the nodes in the search tree
 data structure:
• State : a state in the state space
• parent node : the immediate predecessor in the
search tree
• Action : the action that, performed in the parent nodeʼs
state, leads to this nodeʼs state
• path cost : the total cost of the path leading to this
node
• Depth : the depth of this node in the search tree

State-Space Search and the STRIPS Planner 55

 Fringe Nodes
in Touring Romania Example

fringe nodes: nodes that have not been

expanded

In(Arad(

In(Zerind( In(Sibiu( In(Timisoara(

In(Arad( In(Oradea( In(Fagaras( In(Rimnicu Vilcea(

In(Sibiu( In(Bucharest(

State-Space Search and the STRIPS Planner 56

Search (Control) Strategy
 search or control strategy: an effective
method for scheduling the application of the
successor function to expand nodes
• selects the next node to be expanded from the fringe
• determines the order in which nodes are expanded
• aim: produce a goal state as quickly as possible
 examples:
• LIFO/FIFO-queue for fringe nodes
• alphabetical ordering

State-Space Search and the STRIPS Planner 57

General Tree Search Algorithm
function treeSearch(problem, strategy)
fringe  { new
searchNode(problem.initialState({
loop
if empty(fringe) then return failure
node  selectFrom(fringe, strategy(
if problem.goalTest(node.state) then
return pathTo(node(
fringe  fringe + expand(problem, node(

State-Space Search and the STRIPS Planner 58

 General Search Algorithm:
Touring Romania Example

In(Arad(

In(Zerind( In(Sibiu( In(Timisoara(

In(Arad( In(Oradea( In(Fagaras( In(Rimnicu Vilcea(

In(Sibiu( In(Bucharest(
fringe
selected

State-Space Search and the STRIPS Planner 59

Uninformed vs. Informed Search
 uninformed search (blind search(
• no additional information about states beyond
problem definition
• only goal states and non-goal states can be
distinguished
 informed search (heuristic search(
• additional information about how “promising”
a state is available

State-Space Search and the STRIPS Planner 60

Breadth-First Search:
Missionaries and Cannibals

depth = 0
depth = 1
depth = 2
3
=

depth
State-Space Search and the STRIPS Planner 61
Depth-First Search:
Missionaries and Cannibals

depth = 0
depth = 1
depth = 2
3
=

depth
State-Space Search and the STRIPS Planner 62
Iterative Deepening Search
 strategy:
• based on depth-limited (depth-first) search
• repeat search with gradually increasing depth
limit until a goal state is found
 implementation:
for depth  0 to ∞ do
result  depthLimitedSearch(problem, depth(
if result ≠ cutoff then return result

State-Space Search and the STRIPS Planner 63

Discovering Repeated States:
Potential Savings
 sometimes repeated states are unavoidable,
resulting in infinite search trees
 checking for repeated states:
• infinite search tree ⇒finite search tree
• finite search tree ⇒exponential reduction
state space graph

state space graph

search tree

State-Space Search and the STRIPS Planner 64

Overview
 The STRIPS Representation
 The Planning Domain Definition Language
(PDDL(
 Problem-Solving by Search
Heuristic Search
 Forward State-Space Search
 Backward State-Space Search
 The STRIPS Planner

State-Space Search and the STRIPS Planner 65

Uniform-Cost Search
 an instance of the general tree search or
graph search algorithm
• strategy: select next node based on an
evaluation function f : state space → ℝ
• select node with lowest value f(n(
 implementation:
selectFrom(fringe, strategy(
• priority queue: maintains fringe in ascending
order of f-values

State-Space Search and the STRIPS Planner 66

Heuristic Functions
 heuristic function h : state space → ℝ
 h ( n ) = estimated cost of the cheapest
path from node n to a goal node
 if n is a goal node then h ( n ) must be 0
 heuristic function encodes problem-
specific knowledge in a problem-
independent way

State-Space Search and the STRIPS Planner 67

Greedy Best-First Search
 use heuristic function as evaluation
function: f (n) = h (n(
• always expands the node that is closest to the
goal node
• eats the largest chunk out of the remaining
distance, hence, “greedy”

State-Space Search and the STRIPS Planner 68

Touring in Romania: Heuristic
 hSLD(n) = straight-line distance to Bucharest

Arad 366 Hirsova 151 Rimnicu 193

Bucharest 0 Iasi 226 Vilcea
Craiova 160 Lugoj 244 Sibiu 253
Dobreta 242 Mehadia 241 Timisoara 329
Eforie 161 Neamt 234 Urziceni 80
Fagaras 176 Oradea 380 Vaslui 199
Giurgiu 77 Pitesti 100 Zerind 374

State-Space Search and the STRIPS Planner 69

Greediness
 greediness is susceptible to false starts

 repeated states may lead to infinite oscillation

initial state
goal state

State-Space Search and the STRIPS Planner 70

A* Search
 Uniform-cost search where
f(n) = h(n) + g(n(
• h(n) the heuristic function (as before(
• g(n) the cost to reach the node n
 evaluation function:
f(n) = estimated cost of the cheapest
solution through n
 A* search is optimal if h(n) is admissible
State-Space Search and the STRIPS Planner 71
Admissible Heuristics
A heuristic h(n) is admissible if it never
overestimates the distance from n to the nearest
goal node.

 example: hSLD
 A* search: If h(n) is admissible then f(n) never
overestimates the true cost of a solution
through n.

State-Space Search and the STRIPS Planner 72

 A* Search:
Touring Romania
fringe

d=0
Arad (366)
selected

d=1
Zerind Sibiu Timisoara
(449) (393) (447)

Arad Oradea

d=2
(646) (671)
Fagaras Rimnicu Vilcea
(415) (413)

d=3
Sibiu Bucharest Craiova Pitesti Sibiu
(591) (450) (526) (417) (553)

d=4
Bucharest Craiova Rimnicu Vilcea
(418) (615) (607)

State-Space Search and the STRIPS Planner 73

Optimality of A* (Tree Search(

Theorem:
A* using tree search is optimal if the
heuristic h(n) is admissible.

State-Space Search and the STRIPS Planner 74

A*: Optimally Efficient
 A* is optimally efficient for a given
heuristic function:
no other optimal algorithm is guaranteed
to expand fewer nodes than A* .
 any algorithm that does not expand all
nodes with f (n) < C * runs the risk of
missing the optimal solution

State-Space Search and the STRIPS Planner 75

A* and Exponential Space
 A* has worst case time and space
complexity of O (bl (
 exponential growth of the fringe is
normal
• exponential time complexity may be
acceptable
• exponential space complexity will exhaust any
computerʼs resources all too quickly

State-Space Search and the STRIPS Planner 76

Overview
 The STRIPS Representation
 The Planning Domain Definition Language
(PDDL(
 Problem-Solving by Search
 Heuristic Search
Forward State-Space Search
 Backward State-Space Search
 The STRIPS Planner

State-Space Search and the STRIPS Planner 77

State-Space Search
 idea: apply standard search algorithms
(breadth-first, depth-first, A*, etc.) to
planning problem:
• search space is subset of state space
• nodes correspond to world states
• arcs correspond to state transitions
• path in the search space corresponds to plan

State-Space Search and the STRIPS Planner 78

 DWR Example: State Space
crane crane crane
s0 s2 s5

cont. cont. t
palle pall pallet cont.
robot e
robot robot
location1 location2 location1 location2 location1 location2

crane crane crane

s1 s3 s4
cont. cont.

pallet pallet pcont.

alle
robot robot t
robot

location1 location2 location1 location2 location1 location2

State-Space Search and the STRIPS Planner 79

Search Problems
 initial state
 set of possible actions/applicability conditions
• successor function: state  set of <action, state<
• successor function + initial state = state space
• path (solution(
 goal
• goal state or goal test function
 path cost function
• for optimality
• assumption: path cost = sum of step costs

State-Space Search and the STRIPS Planner 80

State-Space Planning as a
Search Problem
 given: statement of a planning problem
P=(O,si,g(
 define the search problem as follows:
• initial state: si
• goal test for state s: s satisfies g
• path cost function for plan π: ¦π|
• successor function for state s: Γ( s (

State-Space Search and the STRIPS Planner 81

Reachable Successor States
 The successor function Γm:2S→2S for a
STRIPS domain Σ=)S,A,γ) is defined as:
• Γ(s)={γ(s,a) ¦ a∈A and a applicable in s } for s∈S
• Γ(}s1,…,sn})= ∪(k∈[1,n])Γ(sk(
• Γ0({s1,…,sn})= { s 1 ,…, s n { s1,…,sn∈S
• Γm({s1,…,sn})= Γ(Γm-1({s1,…,sn{ ))
 The transitive closure of Γdefines the set of all
reachable states:
• Γ>(s)= ∪(k∈[0 , ∞](Γk ({s{ ( for s∈S

State-Space Search and the STRIPS Planner 82

Solution Existence

 Proposition: A STRIPS planning

problem P=)Σ,si,g) (and a statement of
such a problem P=(O,si,g) ) has a
solution iff Sg ⋂ Γ>({si{ ( ≠} {

State-Space Search and the STRIPS Planner 83

Forward State-Space Search
Algorithm
function fwdSearch(O,si,g(
state  si
plan  〈 〉
loop
if state.satisfies(g) then return plan
applicables 
}ground instances from O applicable in state{
if applicables.isEmpty() then return failure
action  applicables.chooseOne()
state  γ (state,action(
plan  plan ∙ 〈 action〉

State-Space Search and the STRIPS Planner 84

 DWR Example: Forward Search
initial state: plan = goal state:
crane
s0 take(crane,loc1,cont,pallet,pile) crane s5
move(robot,loc2,loc1)
cont. load(crane,loc1,cont,robot)
pall pallet cont.
e robot move(robot,loc1,loc2 ( robot
loc1 loc2 loc1 loc2

crane crane crane

s1 s3 s4
cont. cont.

pallet pallet alle

pcont.
robot t
robot
robot
loc1 loc2 loc1 loc2 loc1 loc2

State-Space Search and the STRIPS Planner 85

Finding Applicable Actions:
Algorithm
function addApplicables(A, op, precs, σ, s(
if precs+.isEmpty() then
for every np in precs - do
if s.falsifies)σ)np)) then return
A.add)σ)op((
else
pp  precs+.chooseOne)(
for every sp in s do
σʼ  σ.extend)sp, pp(
if σ ʼ.isValid() then
addApplicables(A, op, (precs - pp), σʼ,s(

State-Space Search and the STRIPS Planner 86

Properties of Forward Search
 Proposition: fwdSearch is sound, i.e. if the function
returns a plan as a solution then this plan is indeed a
solution.
• proof idea: show (by induction) state=γ(si,plan) at the
beginning of each iteration of the loop

 Proposition: fwdSearch is complete, i.e. if there exists

solution plan then there is an execution trace of the
function that will return this solution plan.
• proof idea: show (by induction) there is an execution trace
for which plan is a prefix of the sought plan

State-Space Search and the STRIPS Planner 87

Making Forward Search
Deterministic
 idea: use depth-first search
• problem: infinite branches
• solution: prune repeated states
 pruning: cutting off search below certain
nodes
• safe pruning: guaranteed not to prune every solution
• strongly safe pruning: guaranteed not to prune every
optimal solution
• example: prune below nodes that have a
predecessor that is an equal state (no repeated
states(

State-Space Search and the STRIPS Planner 88

Overview
 The STRIPS Representation
 The Planning Domain Definition Language
(PDDL(
 Problem-Solving by Search
 Heuristic Search
 Forward State-Space Search
Backward State-Space Search
 The STRIPS Planner

State-Space Search and the STRIPS Planner 89

The Problem with Forward
Search
 number of actions applicable in any
given state is usually very large
 branching factor is very large
 forward search for plans with more than
a few steps not feasible

 idea: search backwards from the goal

 problem: many goal states

State-Space Search and the STRIPS Planner 90

Relevance and Regression Sets
 Let P=)Σ,si,g) be a STRIPS planning
problem. An action a∈A is relevant for
g if
• g ⋂ effects(a) ≠{} and
• g+ ⋂ effects-(a) = {} and g - ⋂ effects+(a(= }{ .
 The regression set of g for a relevant
action a∈A is:

• γ-1(g,a)=(g -effects(a)) ∪ precond( a(

State-Space Search and the STRIPS Planner 91
Regression Function
 The regression function Γ-m for a STRIPS
domain Σ=)S,A,γ) on L is defined as:
• Γ-1(g)={γ -1(g,a) | a∈A is relevant for g} for g∈2L
• Γ0({g1,…,gn})= {g1,…,gn{
• Γ-1({g1,…,gn})= ∪(k∈[1,n])Γ-1(gk( g1,…,gn∈2L
• Γ-m({g1,…,gn})= Γ-1)Γ-(m-1)({g1,…,gn{( (
 The transitive closure of Γ-1 defines the set of
all regression sets:
• Γ<(g)= ∪(k∈[0,∞](Γ-k({g{( for g∈2L

State-Space Search and the STRIPS Planner 92

State-Space Planning as a
Search Problem
 given: statement of a planning problem
P=(O,si,g(
 define the search problem as follows:
• initial search state: g
• goal test for state s: si satisfies s
• path cost function for plan π: ¦π|
• successor function for state s: Γ-1(s(

State-Space Search and the STRIPS Planner 93

Solution Existence
 Proposition: A propositional planning
problem P=)Σ,si,g) (and a statement of
such a problem P=(O,si,g) ) has a
solution iff ∃s∈Γ< ( {g} ) : si satisfies s.

State-Space Search and the STRIPS Planner 94

Ground Backward State-Space
Search Algorithm
function groundBwdSearch(O,si,g(
subgoal  g
plan  〈 〉
loop
if si.satisfies(subgoal) then return plan
applicables 
}ground instances from O relevant for subgoal {
if applicables.isEmpty() then return failure
action  applicables.chooseOne()
subgoal  γ -1(subgoal, action(
plan  〈 action〉 ∙ plan

State-Space Search and the STRIPS Planner 95

 DWR Example: Backward
Search
:initial state plan = :goal state
crane
s0 take(crane,loc1,cont,pallet,pile) crane s5
move(robot,loc2,loc1)
cont. t load(crane,loc1,cont,robot)
pall pallet cont.
e robot move(robot,loc1,loc)2 robot
loc1 loc2 loc1 loc2

crane crane
crane
s1 s3 s4
cont.
cont.

pallet alle
pcont.
pallet
robot robott
robot
loc1 loc1 loc2 loc1 loc2
loc2

State-Space Search and the STRIPS Planner 96

Example: Regression with
Operators
 goal: at(robot,loc1)
 operator: move(r,l,m(
• precond: adjacent(l,m), at(r,l), ¬occupied(m(
• effects: at(r,m), occupied(m), ¬occupied(l), ¬at(r,l (
 actions: move(robot,l,loc1(
• l =?
• many options increase branching factor

 lifted backward search: use partially

instantiated operators instead of actions

State-Space Search and the STRIPS Planner 97

Lifted Backward State-Space
Search Algorithm
function liftedBwdSearch(O,si,g(
subgoal  g
plan  〈 〉
loop
if ∃σ:si.satisfies)σ)subgoal)) then return σ)plan(
applicables 
})o,σ) | o∈Oand σ)o) relevant for subgoal}
if applicables.isEmpty() then return failure
action  applicables.chooseOne)(
subgoal  γ -1(σ)subgoal), σ)o((
plan  σ)〈 action〉 ) ∙ σ)plan(

State-Space Search and the STRIPS Planner 98

 DWR Example: Lifted Backward
Search
crane
s0
 liftedBwdSearch)
}move(r,l,m)}, s0, {at(robot,loc1)} )
cont.
pall t
e robot  ∃σ:si.satisfies)σ)subgoal:)) no
loc1 loc2

 initial state: s0 = {attached(pile,loc1),  applicables =

in(cont,pile), top(cont,pile), })move(r1,l1,m1),{r1←robot,
on(cont,pallet), belong(crane,loc1), m1←loc1})}
empty(crane), adjacent(loc1,loc2),
adjacent(loc2,loc1), at(robot,loc2),  subgoal =
occupied(loc2), unloaded(robot(} }adjacent(l1,loc1), at(robot,l1 )
 operator:move(r,l,m( ¬occupied(loc1){
• precond: adjacent(l,m), at(r,l), ¬occupied
(m(  Plan => move(robot,l1,loc1)>
• effects: at(r,m), occupied(m), ¬occupied 1
(l), ¬at(r,l(
 ∃σ:si. satisfies)σ)subgoal)): yes
σ = { l1 ←loc1 }

State-Space Search and the STRIPS Planner 99

Properties of Backward Search
 Proposition: liftedBwdSearch is sound, i.e. if the function
returns a plan as a solution then this plan is indeed a
solution.
• proof idea: show (by induction) subgaol=γ -1(g,plan) at the
beginning of each iteration of the loop

 Proposition: liftedBwdSearch is complete, i.e. if there

exists solution plan then there is an execution trace of the
function that will return this solution plan.
• proof idea: show (by induction) there is an execution trace
for which plan is a suffix of the sought plan

State-Space Search and the STRIPS Planner 100

Avoiding Repeated States
 search space:
• let gi and gk be sub-goals where gi is an
ancestor of gk in the search tree
• let σ be a substitution such that σ)gi) ⊆gk
 pruning:
• then we can prune all nodes below gk

State-Space Search and the STRIPS Planner 101

Overview
 The STRIPS Representation
 The Planning Domain Definition Language
(PDDL(
 Problem-Solving by Search
 Heuristic Search
 Forward State-Space Search
 Backward State-Space Search
The STRIPS Planner

State-Space Search and the STRIPS Planner 102

Problems with Backward Search
 state space still too large to search
efficiently
 STRIPS idea:
• only work on preconditions of the last operator
added to the plan
• if the current state satisfies all of an
operatorʼs preconditions, commit to this
operator

State-Space Search and the STRIPS Planner 103

Ground-STRIPS Algorithm
function groundStrips(O,s,g)
plan 〈 〉
loop
if s.satisfies(g) then return plan
applicables 
}ground instances from O relevant for g-s{
if applicables.isEmpty() then return failure
action  applicables.chooseOne)(
subplan  groundStrips(O,s,action.preconditions)((
if subplan = failure then return failure
s  γ(s, subplan . 〈 action 〉 (
plan  plan . subplan .〈 action 〉 .

State-Space Search and the STRIPS Planner 104

Problems with STRIPS
 STRIPS is incomplete:
• cannot find solution for some problems, e.g.
interchanging the values of two variables
• cannot find optimal solution for others, e.g. Sussman
anomaly:
A

C B

A B C
Table Table
State-Space Search and the STRIPS Planner 105
STRIPS and the Sussman
Anomaly (1)
 achieve on(A,B(
• put C from A onto table C A
• put A onto B A B C B

 achieve on(B,C(
• put A from B onto table A B

• put B onto C C B A C

 re-achieve on(A,B( A
• put A onto B A
B
C
B
C

State-Space Search and the STRIPS Planner 106

STRIPS and the Sussman
Anomaly (2)
B
 achieve on(B,C( C C
• put B onto C A B A
 achieve on(A,B(
• put B from C onto table
B
C A
• put C from A onto table A B C
• put A onto B
 re-achieve on(B,C(
• put A from B onto table A B

• put B onto C
B C A C

 re-achieve on(A,B( A
• put A onto B B B
A C C

State-Space Search and the STRIPS Planner 107

Interleaving Plans for an
Optimal Solution
 shortest solution  shortest solution for
achieving on(A,B(: on(A,B) and on(B,C(:
put C from A onto table

put A onto B

 shortest solution
achieving on(B,C(:
put B onto C

State-Space Search and the STRIPS Planner 108

Overview
 The STRIPS Representation
 The Planning Domain Definition Language
(PDDL(
 Problem-Solving by Search
 Heuristic Search
 Forward State-Space Search
 Backward State-Space Search
The STRIPS Planner

State-Space Search and the STRIPS Planner 109