PLANNING

Problems in Problem Solving

• The problem solving agent can be overwhelmed by
irrelevant action
• Finding a good heuristic function
• Problem solver might be inefficient because it cannot
take advantage of problem decomposition.
• Problems with search agent
– Too many actions and too many states to consider
– Heuristic function can only choose among states and can’t
eliminate actions from consideration; so which action should be
taken?
– Agent is forced to consider actions starting from the initial state.
 Solutions to the problems
• Planner is free to add actions to the plan
wherever they are needed
– So it can make obvious and important decisions first,
others later
• Most parts of the world are independent of each
other (nearly decomposable) -> so we can solve
it independently (Div&Conq)
• Open up the representation of states, goals,
actions
– States and goals are represented by sets of
sentences; actions by logical descriptions of
preconditions and effects -> direct connections
between states and actions
 PLANNING
• In which we see how an agent can take advantage of the
structure of a problem to construct complex plans of
action
• The task of coming up with a sequence of actions that
will achieve a goal is planning
• Problem-solving agents are able to plan ahead before
acting
• They are different in representing goals, states, and
actions, and in ways of constructing action sequences.
 Basic representations

• STRIPS language for efficient planning

– States are conjunctions of function-free ground literals
At(P1,JFK)^At(P2,SFO)^Plane(P1)^Plane(P2)^...
– Goals are conjunctions of literals, contain variables
• At(C1,JFK)^At(C2,SFO)
– Implicit representations of states in planning - only
changes are tracked.
– Actions in SRIPS have three components: action,
precondition, and effect
 Planning in STRIP language
• Action(Fly(p, from, to),
PRECOND:At(p,from)^Plane(p)^Airport(from)^Airport(to)
EFFECT:!At(p,from)^At(p,to))
• The above is an action schema
• An action schema consists of 3 parts
– Action name and parameter
– Precondition – a conjunction of function-free positive
literals stating what must be true in a state before the
action can be executed
– Effect – a conjunction of function-free literals describing
how the state changes when the action is executed.
Planning with State-Space Search
• Two approaches to search
– An example of Fly
– Forward state-space search -progression planning
• Starting with the initial state, actions, goal test
– Backward state-space search – regression planning
• Starting with the goal state, applying only relevant
actions
• For actions that achieve some desired literal, the
actions should not undo any desired literals
(Consistency)
 Continued…
• A plan is complete if every precondition of every
step is achieved by some other step.
– A step achieves a condition if the condition is
one of the effects of the step.
– The causal link to protect a precond
• A plan is consistent if there are no contradictions
in the ordering or binding constraints.
– Ordering is transitive
 Example: blocks world (Sussman anomaly)

Initial:
Goal: A

C B

A B C

State I: on-table(A)^ on(C,A) ^on-table(B)^(clear B) ^ (clear C)

Goal: on(A,B)^ on(B,C)

Nets Of Action Hierarchies

on(a, b)
S J
on(b, c)

clear(a)
S J puton(a, b)
clear(b)
S J
clear(b)
S J puton(b, c)
clear(c)
Nets Of Action Hierarchies

on(a, b)
S J
on(b, c)

clear(a)
S J puton(a, b)
clear(b)
S J
clear(b)
S J puton(b, c)
clear(c)
Resolve conflicts ‘critic’:

clear(a)
S J puton(a, b)
clear(b)
S J
clear(b)
S J puton(b, c)
clear(c)

clear(a)
S J puton(a, b)
clear(b)
S J
clear(b)
S J puton(b, c)
clear(c)
 clear(a)
S J puton(a, b)
clear(b)
S J
clear(b)
S J puton(b, c)
clear(c)

clear(a)
J puton(a, b)
S
clear(b)
S J puton(b, c)
clear(c)
 clear(a)
J puton(a, b)
S
clear(b)
S J puton(b, c)
clear(c)

clear(c) puton(c, X)
J puton(a, b)
S
clear(b)
S J puton(b, c)
clear(c)
 Final plan

clear(c) puton(c, X)
J puton(b, c) puton(a, b)
S
clear(b)
 Partial-order planning
• It’s nondeterministic
• It starts with a minimal partial plan.
• It satisfies one precond at a time.
• POP is a regression planner.
• POP is sound and complete.
 Heuristics for POP

• POP does not represent states directly, so

it is harder to estimate how far a POP is
from achieving a goal
• Heuristics are needed to choose which
plan to refine
• Heuristic 1: to count the number of distinct
open preconditions
• Heuristic 2: the most-constrained-variable
Total-Order vs Partial-Order Plans
Plan Generation: Search space of plans

Partial-Order Planning (POP)

• Nodes are partial plans
• Arcs/Transitions are plan refinements
• Solution is a node (not a path).

Principle of “Least commitment”

• e.g. do not commit to an order of actions until it
is required
 Partial Plan Representation
• Plan = (A, O, L), where
– A: set of actions in the plan
– O: temporal orderings between actions (a < b)
– L: causal links linking actions via a literal

• Causal Link: Ap Q Ac
Action Ac (consumer) has precondition Q that is
established in the plan by Ap (producer).
(clear b)
move-a-from-b-to-table move-c-from-d-to-b
 POP
POP is sound and complete
• POP Plan is a solution if:
– All preconditions are supported (by causal links), i.e.,
no open conditions.
– No threats
– Consistent temporal ordering
• By construction, the POP algorithm reaches a
solution plan
POP example: Sussman Anomaly

A0
(on C A) (on-table A) (on-table B) (clear C) (clear B)

(on A B) (on B C)
Ainf
Work on open precondition (on B C) and
(clear B)

A0
(on C A) (on-table A) (on-table B) (clear C) (clear B)

(clear B) (clear C) (on-table B)

A1: move B from Table to C
-(on-table B) -(clear C) (on B C)

(on A B) (on B C)
Ainf
 Work on open precondition (on A B)

A0
(on C A) (on-table A) (on-table B) (clear C) (clear B)

(clear B) (clear C) (on-table B)

(clear A) (clear B) (on-table A) A1: move B from Table to C
A2: move A from Table to B -(on-table B) -(clear C) (on B C)
-(on-table A) -(clear B) (on A B)

(on A B) (on B C)
Ainf
 Protect causal links

A0
(on C A) (on-table A) (on-table B) (clear C) (clear B)

(A0, (clear B), A1) theatened by A2

--> Promotion
(clear B) (clear C) (on-table B)
(clear A) (clear B) (on-table A) A1: move B from Table to C
A2: move A from Table to B -(on-table B) -(clear C) (on B C)
-(on-table A) -(clear B) (on A B)

(on A B) (on B C)
Ainf
 Work on open precondition (clear A) and
protect links

A0
(on C A) (on-table A) (on-table B) (clear C) (clear B)

(on C A) (clear C)
A3: move C from A to Table
-(on C A) (on-table C) (clear A)
(clear B) (clear C) (on-table B)
(clear A) (clear B) (on-table A) A1: move B from Table to C
A2: move A from Table to B -(on-table B) -(clear C) (on B C)
-(on-table A) -(clear B) (on A B)

(on A B) (on B C)
Ainf
 Final plan

A0
(on C A) (on-table A) (on-table B) (clear C) (clear B)

(on C A) (clear C)
A3: move C from A to Table
-(on C A) (on-table C) (clear A)
(clear B) (clear C) (on-table B)
(clear A) (clear B) (on-table A) A1: move B from Table to C
A2: move A from Table to B -(on-table B) -(clear C) (on B C)
-(on-table A) -(clear B) (on A B)

(on A B) (on B C)
Ainf
 Summary

• Planning uses more flexible representations of

state, actions, goals, and plans
• STRIPS - a specific language that connects states
to actions (action, precond, effect); there are other
languages like ADL (action description lang)
• Search through the plan space - regression
• Least commitment: partial ordering & instantiation
• Causal links: protect rewriting & detect conflicts
early
• Hierarchical Task Network (HTN)
Planning
Hierarchical Decomposition
Task Reduction
 HTN Planning
• Capture hierarchical structure of the planning
domain
• Planning domain contains non-primitive actions
and schemas for reducing them
• Reduction schemas:
– given by the designer
– express preferred ways to accomplish a task
 HTN Planning Algorithm
Problem reduction:
• Decompose tasks into subtasks
• Handle constraints
• Resolve interactions
• If necessary, backtrack and try other
decompositions
 Basic HTN Procedure
1. Input a planning problem P
2. If P contains only primitive tasks, then resolve
the conflicts and return the result. If the
conflicts cannot be resolved, return failure
3. Choose a non-primitive task t in P
4. Choose an expansion for t
5. Replace t with the expansion
6. Find interactions among tasks in P and
suggest ways to handle them. Choose one.
7. Go to 2
 Planning Graphs
• Construct a graph that encodes constraints on
possible plans
• Use this “planning graph” to constrain search for
a valid plan:
– If valid plan exists, it’s a subgraph of the planning
graph
• Planning graph can be built for each problem in
polynomial time
Problem handled by GraphPlan*
• Pure STRIPS operators:
– conjunctive preconditions
– no negated preconditions
– no conditional effects
– no universal effects
• Finds “shortest parallel plan”
• Sound, complete and will terminate with failure if
there is no plan.
 Planning graph
• Directed, leveled graph
– 2 types of nodes:
• Proposition: P
• Action: A
– 3 types of edges (between levels)
• Precondition: P -> A
• Add: A -> P
• Delete: A -> P
• Proposition and action levels alternate
• Action level includes actions whose preconditions are
satisfied in previous level plus no-op actions (to solve
frame problem).
Planning graph

…
…
…
Constructing the planning graph
• Level P1: all literals from the initial state
• Add an action in level Ai if all its preconditions
are present in level Pi
• Add a precondition in level Pi if it is the effect of
some action in level Ai-1 (including no-ops)
• Maintain a set of exclusion relations to eliminate
incompatible propositions and actions (thus
reducing the graph size)

P1 A1 P2 A2 … Pn-1 An-1 Pn
 Mutual Exclusion relations
• Two actions (or literals) are mutually exclusive
(mutex) at some stage if no valid plan could
contain both.
• Two actions are mutex if:
– Interference: one clobbers others’ effect or
precondition
– Competing needs: mutex preconditions
• Two propositions are mutex if:
– All ways of achieving them are mutex
 GraphPlan algorithm
• Grow the planning graph (PG) until all goals are
reachable and not mutex. (If PG levels off first,
fail)
• Search the PG for a valid plan
• If not found, add a level to the PG and try again
 Searching for a solution plan
• Backward chain on the planning graph
• Achieve goals level by level
• At level k, pick a subset of non-mutex actions to achieve
current goals. Their preconditions become the goals for
k-1 level.
• Build goal subset by picking each goal and choosing an
action to add. Use one already selected if possible. Do
forward checking on remaining goals (backtrack if can’t
pick non-mutex action)
 Planning & Execution

Initial State: Move(x y) Goal:

pre: clear(x) ^ clear(y) ^ on(x z) On(C, D)
eff: on(x y) ^ clear(z) ^ On(D, B)
on(x z) ^ clear(y)
Plan ready to start execution

but genie intervenes: moves D to B !

New state of the world:

Updated plan:
But it actually was a helpful interference:
• Can link to on(D B) from current state
• Move(D B) is now redundant
Now the agent can execute move(C D) to achieve the goal
Unfortunately our agent is clumsy and
drops C onto A instead of D
The new current state looks like:

And the updated plan is:

 Keep planning to satisfy open condition on(C D)

Resulting plan:
 Fortunately, this time execution works:

The plan is finally completed:

• Goals achieved
• No threats
• No unexecuted step “flaws”
Conditional Planning
 Planning and information gathering

• UWL representation language

• SENSP algorithm

• Conference Discussion of: Oren Etzioni, Steve

Hanks, Daniel Weld, Denise Draper, Neal Lesh,
Mike Williamson (1992) "An Approach to
Planning with Incomplete Information".
Proceedings of the 3rd International on
Principles of Knowledge Representation and
Reasoning