On the Use of Integer Programming Models in AI Planning

Thomas Vossen Michael Ball Amnon Lotem Dana Nau

Robert H. Smith School of Business Department of Computer Science
and Institute for Systems Research and Institute for Systems Research
University of Maryland University of Maryland
College Park, MD 20742 USA College Park, MD 20742 USA
{tvossen, mball}@rhsmith.umd.edu {lotem, nau}@cs.umd.edu

Abstract (IP), which has a rich history in OR, to a classical AI

problem, AI planning.
Recent research has shown the promise of using The possibility of using OR techniques in AI plan-
propositional reasoning and search to solve Al ning has not received much attention so far. Bylander
planning problems. In this paper, we further 1997] uses Linear Programming as a heuristic for non-
explore this area by applying Integer Program- inear planning; Bockmayr and Dimopoulos [1998] de-
ming to solve AI planning problems. The ap- scribe domain-dependent IP models for specific problem
plication of Integer Programming to AI plan- domains; and Kautz and Walser [1999], who use IP for-
ning has a potentially significant advantage, mulations for planning problems with resources, action
as it allows quite naturally for the incorpo- costs, and complex objective functions. However, theirs
ration of numerical constraints and objectives is the only work we know of besides ours.
into the planning domain. Moreover, the ap-
One potential advantage of using Integer Program-
plication of Integer Programming to AI plan-
ming for AI planning is that IP formulations quite natu-
ning addresses one of the challenges in propo-
rally allow the incorporation of numeric constraints and
sitional reasoning posed by Kautz and Selman,
objectives into planning domains(for example, see Kautz
who conjectured that the principal technique
and Walser [1999]). The use of numerical constraints and
used to solve Integer Programs—the linear pro-
objectives is not addressed adequately in most existing
gramming (LP) relaxation—is not useful when
AI planning systems, but it is critical in real-world plan-
applied to propositional search.
ning [Nau et al, 1998].
We discuss various IP formulations for the class One difficulty indeveloping Integer Programming for-
of planning problems based on STRIPS-style mulations for AI planning is that the performance of the
planning operators. Our main objective is to resulting IP will depend critically on how AI planning
show that a carefully chosen IP formulation sig- problems are formulated as Integer Programs. The pur-
nificantly improves the "strength" of the LP re- pose of this paper is therefore to develop good domain-
laxation, and that the resultant LPs are useful independent IP formulations for AI planning. In par-
in solving the IP and the associated planning ticular, we discuss various IP formulations for the class
problems. Our results clearly show the impor- of action-based planning problems using STRIPS-style
tance of choosing the "right" representation, operators. Our main objective is to show that a care-
and more generally the promise of using Inte- fully chosen IP formulation significantly improves the
ger Programming techniques in the AI planning "strength" of the LP relaxation, so that it can provide
domain. useful guidance in solving the problem.
Our IP formulations are principally derived from work
by Kautz and Selman [1996], which showed that planning
1 Introduction problems can be efficiently solved by general proposi-
Although some of the application areas addressed in the tional satifiablity algorithms. As such, the use of Integer
fields of Artificial Intelligence (AI) and Operations Re- Programming also addresses one of the challenges posed
search (OR) are very similar (e.g., planning, scheduling), in the paper "Ten Challenges in Propositional Reason-
the techniques that are used to solve these problems are ing and Search," by Selman et al. [1997]. Specifically,
oftentimes substantially different. Therefore, it seems the challenge that we address concerns the development
only natural that recent research in the interface between of IP models and methods for propositional reasoning.
AI and OR has focused on comparing the relative mer- Selman et al. mention that the basic technique used
its of the techniques and tools that are used in these to solve integer programs—that is, the Linear Program-
areas. In this paper, we further explore the interface ming (LP) relaxation of the problem—does not appear
between AI and OR by applying Integer Programming to be useful for satisfiability problems, since it usually

304 CHALLENGE PAPERS

sets all variables (modulo unit propagation) to the value formulations for a given problem. One formulation is
^ and therefore does not guide the selection of variables stronger than another if the feasible region of the LP re-
in solving the problem. laxation more closely approximates the integer program
Our results are as follows: (see [Wolsey, 1998] for more details on this concept).
• In our experiments, IP formulations derived directly Stronger formulations are more likely to yield integer so-
from SAT encodings proposed by Kautz and Selman lutions and produce objective function values closer to
[1996] performed rather poorly—but an alternative the values of the integer program.
IP formulation that we call the "state-change for- In the remainder of this section we discuss two IP for-
mulation" was competitive with BlackBox using the mulations for STRIPS-style planning problems. In order
systematic Satz solver [Kautz and Selman, 1998], to express these formulations, we first introduce the fol-
in terms of the number of nodes expanded in the lowing sets:
search space. Since the brandling rule that is used • F, the set of fluents, that is, the set of all instanti-
in systematic satisfiability algorithms is an impor- ated predicates;
tant factor in reducing the size of the search tree,
this indicates that the LP relaxation does guide the • A, the set of actions, that is, the set of all instanti-
selection of variables in solving the problem. ated operators;

• Like Graphplan-based planners such as Blackbox, • represents the set of fluents that hold
state-change formulation is guaranteed to find plans initially;
that have optimal values for the number of time • represents the set of fluents that have to
steps in the plan. However, this formulation also to hold in the goal state.
takes into account the number of actions required We assume that the number of time steps in the plan, t,
by the plans (i.e., the plan length); and the number is given. Furthermore, we introduce the sets
of actions obtained using the state-change IP for-
mulation was usually much less than the number of • represents the set of
actions in the plans obtained by BlackBox. actions which have fluent / as a precondition;
The organization of this paper is as follows. In Section • represents the set of
2, we discuss the various IP formulations of the planning actions which have fluent / as an add effect;
problem. Next, Section 3 provides experimental results • represents the set of
for these formulations, and a comparison with the sys- actions that delete fluent /.
tematic satisfiability solver. We conclude in Section 4
with a brief discussion of issues that arise in using IP 2.1 SATPLAN-based IP Formulations
techniques, and of plans for future work. Initially, our IP formulations were motivated by the well
known SATPLAN encodings, as discussed in [Kautz and
2 Integer Programming Formulations Selman, 1996]. In SATPLAN, the problem of determin-
The most effective current approach for solving general ing whether a plan exists, given a fixed number of time
integer programs involves the use of branch and bound steps, is expressed as a satisfiability problem.
employing a linear programming (LP) relaxation. Thus, It is well known that satisfiability problems can be
the key to the effectiveness of using integer programming expressed as integer linear programs, (see for instance
to solve planning problems will lie in the effectiveness [Blair et a/., 1986) or [Hooker, 1988]). Usually, this is
of the LP relaxation in improving the underlying tree done by converting the clauses in the CNF representation
search. The LP relaxation is typically solved at every of the satisfiability problem to 0-1 linear inequalities. For
node in the search tree. Search can be terminated at instance, the clause
a node 1) if LP relaxation value indicates that further
search could only uncover solutions with objective func-
tion values inferior to the best known, 2) if the LP is is equivalent to the 0—1 inequality
infeasible, which in turn implies the integer program is
infeasible and 3) if the LP yields an integer solution.
Since for planning problems, the objective function is
only of secondary consideration, 1) will have little value. Our first formulation consisted of this conversion for
On the other hand 2) and 3) can be quite useful in im- the SATPLAN encodings that are based on GraphPlan
proving search performance for planning problems. In [Blum and Furst, 1996], i.e., we allow for parallel actions
particular, if the initial LP solves integer then no search and the propagation of fluents using the "no-op" opera-
is necessary. Another role the LP relaxation plays is that tor. The resulting formulation is summarized as follows.
it provides information useful in deciding which variables Variables For all , we have fluent
to branch on. variables, which are defined as
A key issue in the performance of integer programming
algorithms is the "strength" of the formulation. In gen-
eral, there can be many equivalent integer programming

VOSSEN, BALL, LOTEM, AND NAU 305

 For all we have action variables, not in the second. It should be noted that the ability to
which are defined as detect clique inequalities is available in most of today's
commercial solvers.
Secondly, we did not restrict all variables to be 0-1 in-
tegers. Specifically, the integrality of the fluent variables
was relaxed, that is, the constraints {0,1}
We remark that the action variables include the "no-
op" maintain operators from GraphPlan for each were replaced by . This is possible because
time step and fact, which simply has that fact both the integrality of these variables is implied by the in-
as a precondition and as an add effect. "no-op" tegrality of the action variables. We remark that as a
actions are necessary to propagate the fluent values. consequence, none of the fluent variables will be selected
in the branch and bound tree.
C o n s t r a i n t s The constraints are separated into differ-
ent classes, which can be outlined as follows:
2.2 An Alternative Formulation
• I n i t i a l / G o a l S t a t e C o n s t r a i n t s These con-
straints set the requirements on the initial and We now describe an alternative formulation of the plan-
final period, i.e. ning problem, which we shall refer to as the "state-
change formulation". The differences with respect to
the formulation described in the previous section are
twofold. First of all, the original fluent variables are
"compiled away" and suitably defined "state change"
variables are introduced instead. As we will see, this
results in a stronger representation of the exclusion con-
• P r e c o n d i t i o n C o n s t r a i n t s Actions should straints. Secondly, we more explicitly restrict the pos-
imply their preconditions, which is expressed sible propagation of fluents through "no-op"-actions, so
as follows. as to reduce the number of equivalent feasible solutions.
Before giving this formulation, we again first define the
variables. The action variables are the same as before,
i.e.,
• B a c k w a r d C h a i n i n g C o n s t r a i n t s Backward
chaining is expressed as

for all Now however, the "no-op"

• Exclusiveness C o n s t r a i n t s Actions conflict actions are not included, but represented separately by
if one deletes a precondition or add effect of the variables , for a l l . . . , t .
other. The exclusiveness of conflicting actions In order to express the possible state changes, we in-
is expressed as
troduce auxiliary variables
which are defined logically as

for all _ , . . . , t , and all a, a! for which there

exist such that and
add f.
O b j e c t i v e F u n c t i o n The objective function was set to
minimize the number of actions in the plan. It
should be noted that in theory we could have chosen
any objective function, since the constraints guar-
antee a feasible solution. In practice however, the
choice of an objective function can significantly im-
pact performance. Informally, = 1 if and only if an action is exe-
cuted in period i that has / as a precondition but does
In addition, we made the following two modifications
not delete it. We note that the execution of such an ac-
in the formulation. First of all, we used the notion of
tion at a given time step implicitly asserts that the value
clique inequalities to strengthen the formulation. The
basic idea behind this is that the inequalities fluent / is propagated. Similarly, if and only
can be replaced by if an action is executed in period i that has / both as a
a single inequality . This leads to a precondition and a delete effect. if and only if
formulation which is not only more compact but also an action is executed in period i that has / as an add
stronger, in the sense that the fractional solution effect but not as a precondition.
is feasible in the first set of inequalities, but The logical interpretation of these variables is repre-

306 CHALLENGE PAPERS

sented i n the I P formulation b y the following constraints: 3 Experimental Results

We tested the IP formulations on a variety of planning

problems from the Blackbox software distribution, and
compared the results w i t h those obtained by Blackbox
using the systematic Satz solver. The integer programs
were solved using Cplex 6.0, a widely used L P / I P solver.
In solving the integer programs, we used all of Cplex's
default settings, except the following: the initial LP op-
t i m u m was obtained by solving the dual problem, and
the variable selection strategy used was "pseudo-reduced
cost". In addition, the solver was terminated as soon as
a feasible integer solution was found. A l l problems were
run on a Sun Ultra workstation.
for all , t. The equality in the defi- The results are shown in Table 1. "Nodes" represents
nition of follows from the fact that all actions the number of nodes visited in the branch and bound
that have / b o t h as a precondition and as an add effect procedure, and "iterations" the number of simplex iter-
are mutually exclusive. As a consequence these variables ations performed. A l l times are in seconds. It should be
can in fact be substituted out, although for reasons of noted that, both for the IP formulations and BlackBox,
clarity we shall not do so here. The remaining exclusive- the results shown are for the problem of finding a fea-
ness constraints can easily be expressed in terms of the sible solution given the number of time steps (i.e., t is
auxiliary variables, by stating that is mutually known in advance and given).
As shown in Table 1, the state-change formulation led
exclusive w i t h . However,
to a significant improvement in performance. Whereas
in order to strengthen the formulation we furthermore
the SATPLAN-based IP formulation solved only the
assert t h a t is mutually exclusive w i t h and
smallest problems, the state-change formulations solved
. Informally, this means that a fluent can only all, and required both fewer nodes and less computa-
be propagated at a time step if no action that adds it is tion time. While the systematic BlackBox solver usu-
executed. The resulting constraints are as follows. ally required less time than the state-change formula-
tion, both BlackBox and the state-change formulation
explored similar numbers of nodes. Moreover, the Black-
Box/Satz did not find a feasible solution to the "bw-
x
large.b" blocks-world problem, while the state-change
formulation did find a solution, using only 28 nodes.
for all It should be noted that the introduction of auxiliary
The backward chaining requirements can also be ex- variables can possibly introduce a large number of vari-
pressed in terms of the auxiliary variables. Since, all ables and constraints. However,we found that the size
auxiliary variable t h a t assert the precondition of a fact of the formulation was significantly reduced by standard
/ at a certain time step and IP preprocessing (similar to the use of Graphplan as a
preprocessing tool in BlackBox). For example, while the
are mutually exclusive, we have the following initial formulation of the problem "rocket.a" had 27744
constraint variables and 40018 constraints, preprocessing reduced
this to 1573 variables and 3007 constraints. Similar re-
ductions in size were also obtained for the other prob-
lems.
A further indication of the strength of respective for-
mulations can be found by examining the value of the LP
relaxations. Since the objective function that is used is
to minimize the number of actions in the plan, the value
of the LP relaxation may also viewed as a lower bound
on the number of actions required in the plan. The re-
sults for the SATPLAN-based and the state-change for-
mulation are shown in Table 2. In almost all cases,
the state-change formulation has a much higher lower
bound, which indicates that its formulation is indeed
The objective function is again set to minimize the much stronger.
number of actions. Also, the integrality requirement of In the S A T P L A N and Graphplan framework, the par-
the auxiliary variables variables was again relaxed, as it allel length, (i.e., the number of time steps) of plans is
is implied by the integrality of the action variables. minimized. The IP formulation follows this framework,

VOSSEN, BALL, LOTEM, AND NAU 307

 - denotes that no plan was found after 10 hours of computation time.
* denotes that the node limit of 2500 was reached without finding a feasible integer solution.

efficient planning. In our results, the number of

nodes expanded in the search space was typically
small, and comparable to a systematic satisfiablity
solver. This indicates that the LP relaxation gave
significant guidance in the selection of variables in
solving planning problems.

so it also is guaranteed to minimize the number of time • Second, IP models may provide a natural means
steps. In addition, the IP formulation also explicitly uses of incorporating numeric constraints and objec-
minimization of the number of actions in the objective tives into the planning formulation. This capability
function. Since we set the solver to terminate as soon would be important in many application domains,
as the first feasible integer solution was found, the IP but it is not available in most existing approaches
solutions were not guaranteed to minimize the possible to AI planning. It should be noted, however, that
number of actions. Still, we found that in most cases, the the way in which numeric constraints will be repre-
IP formulation found plans with a significantly smaller sented may have a significant influence on the per-
total number of actions than those obtained by Black- formance, much in the same way as we saw with
Box. This is shown in Table 3. the various IP formulations. Therefore, the devel-
opment of strong IP representations that capture
common numeric constraints that arise in the plan-
4 Conclusions ning domain is an issue for further research.
Although Selman et al. [1997] reported difficulty in mak-
We would like to emphasize that so far our main concern
ing effective use of IP techniques for propositional rea-
has been the development of different IP formulations,
soning in general, our results suggest that IP techniques
rather than improving the efficiency of the LP relaxation
may potentially work well for AI planning problems, for
itself. While we believe that the state-change formula-
the following reasons.
tion is reasonably strong, solving the LP relation at each
• First, the IP formulation has the potential to do node is still sometimes computationally expensive. One

308 CHALLENGE PAPERS

of the main reasons for this, we believe, is the degener-
acy of the LP relaxation (a condition that can cause the
LP solver to execute many non-productive iterations).
Therefore, we are currently also investigating techniques
to resolve this degeneracy, as well as further strength-
ening of the IP formulation. In particular, we want to
investigate the use of constraint and column generation
techniques.

References
[Blair et a/., 1986] Blair, C.E., Jeroslow, R.G., and J.K.
Lowe. 1986. Some results and experiments in program-
ming techniques for propositional reasoning. Comput-
ers and Operations Research 13:633-645.
[BockMayr and Dimopoulos, 1998] Alexander
Bockmayr and Yanis Dimopoulos. 1998. Mixed Integer
Programming Models for Planning Problems. CP'98
Workshop on Constraint Problem Reformulation.
[Bylander, 1997] T. Bylander. 1997. A Linear Program-
ming Heuristic for Optimal Planning. Proc. AAAI-97.
[Blum and Furst, 1997] A. L. Blum and M. L. Furst.
1997. Fast Planning Through Planning Graph Analy-
sis. Artificial Intelligence, 90(l-2):281-300.
[Hooker, 1988] J. N. Hooker. 1988. A quantitative ap-
proach to logical inference. Decision Support Systems
4:45-69.
(Kautz et al, 1996] Henry Kautz, David McAllester,
and Bart Selman. 1996. Encoding plans in proposi-
tional logic. Proc. KR-96.
[Kautz and Selman, 1996] Henry Kautz and Bart Sel-
man. 1996. Pushing the envelope: Planning, propo-
sitional logic, and stochastic search. Proc. AAAI-96.
[Kautz and Selman, 1998] Henry Kautz and Bart Sel-
man. 1998. B L A C K B O X : A New Approach to the
Application of Theorem Proving to Problem Solving.
Working notes of the Workshop on Planning as Com-
binatorial Search, held in conjunction with A1PS-98,
Pittsburgh, PA, 1998.
[Kautz and Walser, 1999] Henry Kautz and Joachim P.
Walser. 1999. State-space Planning by Integer Opti-
mization. Proc. AAAI-99.
[Nau et al, 1998] D. S. Nau, S. J. Smith and Kutluhan
Erol. 1998. Control strategies in H T N planning: the-
ory versus practice. In AAAI-98/IAAI-98 Proceedings,
1127-1133, 1998.
[Selman et al, 1997] B. Selman, H. Kautz, and D.
McAllester. 1997. Ten challenges in propositional
reasoning and search. In Proc. Fifteenth Interna-
tional Joint Conf. Artificial Intelligence (IJCAI-97),
Nagoya, Japan.
[Wolsey, 1998] L. Wolsey, Integer Programming, 1998,
John Wiley, New York.