c

ELSEVIER
Science of

Science of Computer Programming

Computer Programming 30 (1998) 83-I 18

Constraint

Newt on programming over nonlinear

constraints

Pascal Van Hentenryck a,*, Laurent Michel, F&d&-ic Benhamoub

aBrmw
b Umkrsit~ Urziocrsity, Box 1910, Providence. RI 02912. USA dOrl&ns. rue L&ward de Vinci. BP 6759, F-45067 Orl&ns. Frmce

Abstract

This paper is an introduction to Newton, a constraint programming language over nonlinear real constraints. Newton originates from an effort to reconcile the declarative nature of constraint logic programming (CLP) languages over intervals with advanced interval tcchniqucs dcvclopcd in numerical analysis, such as the interval Newton method. Its key conceptual idea is to introduce the notion of box-consistency, which approximates arc-consistency, a notion well-known in artificial intelligence. Box-consistency achieves an effective pruning at a reasonable computation cost and generalizes some traditional interval operators. Newton has been applied to numerous applications in science and engineering. including nonlinear equation-solving, unconstrained optimization, and constrained optimization. It is competitive with continuation methods on their equation-solving benchmarks and outperforms the interval-based m&hods WC arc aware of on optimization problems. 0 1998 Elsevier Science B.V.
Key\~wd.v:

Constraint

programming;

Nonlinear

programming;

Interval reasoning

1. Introduction
Many mechanics) or finding These finding plications. (Intervals) oped applications require in science finding value and engineering solutions of a nonlinear due to their issues language (e.g., function involved designed the chemistry, subject robotics, economics, constraints. (i.e., they (i.e., of apof CLP devel-

all isolated

to a system

of nonlinear

real constraints

the mmlmum

to nonlinear complexity

problems

are difficult

inherent

computational

are NP-hard) Newton

and due to the numerical or the global from is a constraint It originates languages, analysis, programming an attempt

to guarantee termination. to support declarative interval

correctness this class nature methods

all solutions

optimum)

and to ensure to reconcile Newton

such as BNR-Prolog, such as the interval

with advanced

in numerical

method.

* Corresponding author.
0167-6423/98/$19.00 @ PZZ SOl67-6423(97)00008-7

1998 Elsevier Science B.V. All rights

reserved

84

P. Van Hentenryck et al. IScience of Computer Programminy 30 (19981 83-118

Traditionally, tive constraints approximations gence. Complex advantage inconvenience

CLP(Intervals)

languages

were designed

in terms of simple primi-

(e.g. a&(x, y,z), mult(x, y,~), cos(x,z), . . .) on which they apply simple of arc-consistency [ 17, 191, a notion well-known in artificial intelliconstraints are simply rewritten into a set of primitive semantics constraints. The The is the elegant operational of the language. is relatively

of this methodology

is that convergence

may be slow and the pruning

weak due

to the decomposition process, making this approach unpractical on many applications. By contrast, interval research in numerical analysis has focused, among other things, on producing reliable and reasonably fast methods to solve the above applications (e.g., [4,6,7,9, ll-14,20,26,33]). Many of these techniques use ideas behind Newton root finding method, exploit properties such as differentiability, and define various pruning operators, many of which extend the seminal work of Krawczyk [ 141. These algorithms can often be viewed as an iteration of two steps, constraint propagation and splitting, although they are rarely presented this way and it is not always clear what the constraint propagation step computes. The key contribution of Newton is the notion of box-consistency, an approximation of arc-consistency which produces an effective tradeoff between precision and pruning. Box-consistency is parametrized by an interval extension operator for the constraint and can be instantiated to produce various narrowing operators. In particular, boxconsistency on the Taylor extension of the constraint produces a generalization of the Hansen-Segupta operator [7], well-known in interval methods. In addition, boxconsistency on the natural extension produces narrowing operators which are more effective when the algorithm is not near a solution. Newton has been applied to numerous applications. It has been shown to be competitive on constraint-solving benchmarks with state-of-the-art continuation methods (e.g., [25,37]) and to outperform traditional interval methods [36]. In addition, Newton has been applied to many benchmarks in global optimization (unconstrained and constrained optimization), outperforming interval methods we are aware of. This paper is an introduction to Newton, a specification of its main functionalities, and a description of its performance on a number of benchmarks. The introduction illustrates the behaviour of Newton on a number of representative examples from univariate equation solving to multivariate constrained optimization. The specification contains static definitions of the main functionalities of Newton (e.g., constraint solving and optimization), as well as hints on how they are actually implemented. The performance results describe the efficiency of Newton on a number of traditional benchmarks from numerical analysis. The rest of this paper is organized as follows. Section 2 describes the functionality of Newton on a number of applications to give readers an informal understanding of the language. Section 3 describes the language, i.e., its syntax and its semantics. The language is not fully covered. Rather we focus on the main features and we avoid entering into too much detail which would only be of interest to advanced users of the language. Section 4 describes the experimental results of Newton. Section 5 describes related work, while Section 6 concludes the paper.

P. Van Hentenryck

et al. IScience

of Computer

Programminy

30 (19981 83-118

85

2. A short tour of Newton The purpose of this section is to illustrate the constraint-solving capabilities of

Newton. We present examples which are simple to state and to read but readers should keep in mind that many problems require a complex phase of constraint generation that we do not discuss here. This part is of course performed in Newton as well, since Newton is in fact a superset of Prolog, but presents little interest. We begin with a simple program to compute the square root of a number. squareRoot(X,Root) :constraint [ Root >= 0 , Root-2 The predicate ?squareRoot (X,Root)

= X 1.

holds if Root is the square root of X. The query

squareRoot(2,Root)

returns the solution Root in [1.41421356237306428127,1.41421356237312623172].

This simple program illustrates one of the fundamental aspects of Newton: intervals. Newton associates an interval with each variable and it is the role of constraints to narrow down these intervals to the required precision. Consider now the problem of finding a solution x4 + .Yy2 1 = in the interval
[0, 11. The Newton

to the equation

program

simpleRoot constraint returns the solution X in

(X)

:[ X in

[O, 11 , X-4 + X-2 = 1 1 .

[0.78615137475949814493,0.786151381235926316481. simpleRoot (X). The same problem in the interval

[-1, 11 has

when queried with ?two solutions.

The Newton program (X) : [ X in .

which finds them both is as follows:

simpleRoots constraint split(X)

C-1,11

, X-4 + X-2 = 1 I ,

The second goal of the clause is a nondeterministic predicate which splits the interval of X into smaller intervals until a solution to the constraint system is found. This goal is needed, since the constraint solver of Newton is incomplete. If omitted, the program would return
X in

[-0.78615138123592631648,0.78615138123592631648]
i.e., the returned intervals have a width

All examples in this section assume a default 8 digit precision, smaller than lOW*.

86

P. Van Henterryck

et al. /Science

of Computer

Programming

30 (1998)

83-118

for the query ?-

simpleRoots

(XI. split

(X) simply splits the interval associated with nondeterministically. and constraint More precisely, solving then produces

X into two parts and adds the resulting immediately X in the solution [-0.78615138123592631648,-0

constraint

Newton first assumes that X is in the left interval

786151377599601852701.

On backtracking, Newton assumes that X is in the right interval second solution that was shown previously.

and generates

the

This last example illustrates the traditional style of constraint logic programs: prune and branch. Constraints are used to prune the intervals associated with variables, while the branching nondeterministically splits an interval into two parts, whenever no more pruning is possible. To conclude this set of univariate examples, consider the search for the zeros of three related polynomials, an example which was used in a wellknown textbook to illustrate the difficulty of nonlinear constraint solving. The Newton programs for these functions are simply rootsF1 (X> : constraint [ X in [-10-S, IO-81 , x-4 - 12 * x-3 + 47 * x-2
I,

60 * X = 0

split(X) rootsF2

(X> : constraint C X in C-10-8, IO-81 , x-4 - 12 * x-3 + 47 * x-2 split (XI.

60 * X = -24

J,

rootsF3 (X) : constraint [ X in [-10~8,10~83 , X-4 - 12 * X-3 + 47 * X-2 - 60 * X = -24.1

J,

split(X) The query ?X in

rootsFl(X)

returns the four solutions

X in X in X in

[-0.00000000000000122569,0.00000000000000014497~; [2.99999999999997202237,3.000000000000162980751; [3.99999999999988498089,4.00000000000130828682~; [4.99999999999923971927,5.000000000002645883521.

P. Van Hentenryck et al. IScience oj Computer Proyrumminy 30 (1998)

83-118

81

The query ?X in X in

rootsF2(X)

returns the two solutions

[0.88830577862955317769,0.88830577912592989521~; [0.99999999976380837818,1.00000000000002575718~. the absence of zeros. The above examples and we consider is as follows: first the intersection of

fails, indicating The query ?- rootsFS(X) take a couple of milliseconds in Newton. We now turn to multivariate examples a circle and of a parabola. circleAndParabola(L) L = [X1,X21, constraint [ x1-2 + x2-2 = 1, x1-2 - x2 = 0 1, listsplit (L) .

The Newton program :-

It follows the structure of the previous programs, except that the predicate is used instead of split, since there are several variables. If the predicate is omitted, the program returns the box X in Y in Splitting X in Y in [-1.00000000029311686412,1.00000000029311686412~ [-0.00000000000000000000,1.00000000029311686412]. once on variable Xl leads directly to the first solution

listsplit listsplit

[-0.78615137775742405247,-0.78615137775742249814] [0.61803398874989357025,0.61803398874989623480]. Newton produces the second solution

On backtracking, X in Y in

[0.78615137775742249814,0.78615137775742405247] [0.61803398874989345923,0.61803398874989623480].

A more interesting application is robot kinematics, where the goal is to find the angles for the joints of a robot arm so that the robot hand ends up in a specified position. A Newton program, solving a problem given in [9], is depicted in Fig. 1. Once again, the program follows the simple structure that we have encountered so far. It takes about 15 s to find all solutions on a SUN Spare-IO workstation. Nonlinear constraint techniques are also of primary importance to solve chemical equilibrium systems. Our next example is an equilibrium system to describe the propulsion of propane in air and is taken from [ 181. The Newton program which finds the solution in about 5 s is depicted in Fig. 2. The above programs guarantee that any solution to the constraint system are located in at least one box returned by Newton. Hence, we can safely conclude (modulo implementation bugs) that there is no solution when Newton fails. However, there is no guarantee in the above programs that a box actually contains a solution. The

88

P. Vun Hmtmryk

et al. IScience

of Computer

Progranlming

30 (19981 83-118

robotAngles :L = [X1,X2,X3,X4,X5,X6,X7,X8,X9,XlO,Xll,Xl2], constraint [ x1-2 + x2-2 = I, X3-2 + X4-2 = 1, X5-2 + X6-2 = 1, X7-2 + X8-2 = 1, x9-2 + x10-2 = 1, x11-2 + x12-2 = 1, X12*(X4+X6+X8)-(X10*X11*(X3+X5+X7)) = 0.4077, X2 * X9 * (X4 + X6 + X8) + Xl * X10 = 1.9115, x9 * (X3 + x5 + X7) = 1.9791, X2 * (3 * X4 + 2 * X6 + X8) = 4.0616, Xl * (3 * X4 + 2 * X6 + X8) = 1.7172, 3 * x3 + 2 * x5 + x7 = 3.9701 I, listsplit CL).

Fig I. A Newton program for robot kinematics.

problem of existence and unicity of solutions is in fact a difficult problem, which is an active research area in interval methods. Newton provides a facility to obtain safe boxes, i.e., boxes that are guaranteed to contain a solution. The proof of existence is obtained numerically during the computation. For instance, the first univariate polynomial problem rootsF1 described (S) :previously can be stated as follows to obtain safe boxes:

S is-a-Sbox_of [ X in [-10^8,10^81 , X-4 - 12*X-3 f 47*X-2

- 60*X = 0

1.
The query ?X X X X in in in in rootsFl(X) returns the four solutions

[-0.00000000000000122569,0.00000000000000014497]; [2.99999999999997202237,3.00000000000016298075~; [3.99999999999988498089,4.00000000000130828682~; [4.99999999999923971927,5.00000000000264588352~.

indicating that there is a solution in each box. The proof of existence cannot always be obtained, even if the box actually contains a solution. Newton does not return the box as a solution in this case. For all examples described so far, there is no difficulty in obtaining safe boxes. Note also that no splitting goal is necessary when safe boxes are requested, since Newton needs to control the splitting process

P. Van Henterrryck

et al. IScience

qf Computer

Proyrumminy

30

( 1998) 83-118

89

equilibrium(L) :L = [Yl,Y2,Y3,Y4,Y5], R = 10, R5 = 0.193, R6 = 0.002597, R7 = 0.003448, R8 = 0.00001799 * 0.025, R9 = 0.0002155, RIO = 0.00003846 * 0.025, constraint Yl in Y2 in Y3 in Y4 in Y5 in [ [O,lO-81, [0,10"8], [O,lO-81, [O,iO-81, [0,10^8],

sqrt(40)*(3*(2*Yl*Y2 + Yl + Y2*Y3^2 + R8*Y2) R*Y5 + 6*RlO*Y2-2) + 3*(R7*Y2*Y3 + R9*Y2*Y4) = 0, sqrt(40)*(3*(2*Y2*Y3-2+ 2*R5*Y3-2) - 8*Y5) + 3*(R6*Y3 + R7*Y2*Y3) = 0, 3*R9*Y2*Y4 + sqrt(40)*(6*Y4-2 - 4*R*Y5) = 0, sqrt(40)*(Yl*Y2 + Yl + RlO*Y2-2 + Y2*Y3-2 + R8*Y2 + R5*Y3-2 + Y4-2 - 1) + R6*Y3 + R7*Y2*Y3 + R9*Y2*Y4 =o Y5 = Y1*(Y2 + 1)

I,
listsplit(

Fig. A Newtonprogram the 2. for combustion of propane inair when a safe box is obtained. For symmetry, Newton also provides a construction for obtaining boxes which are not necessarily safe and the above program can simply be written as follows with this facility: rootsF1 (S) : S is_a_Cbox_of [ X in [-10~8,10~81 , X-4 - 12*X-3 + 47 * X-2 - 60*X = 0

1.
The C stands for canonical and its meaning will become clear later on.

90

P. Van Herrtenryck er al. IScience

qf Computer Proqramminy 30 (1998) 83-118

minimization(Min,Boxes) :minof ( cos(2*Xl+l)+ 2*cos(3*Xl+2)+ 3*cos (4*x1+3)+ 4*cos (5*X1+4)+ 5*cos(6*Xl+5)) * cos(2*X2+1)+ ( 2*cos (3*X2+2)+ 33~0s (4*X2+3)+ 4*cos(5*X2+4)+ 5*cos(6*X2+5)) in c-10,101 isin Min for Boxes.

Fig. 3. A Newton program

for unconstrained

optimization.

In addition to solving systems of nonlinear constraints, Newton has also been instrumental in solving global optimization problems. The solving of global optimization problems are in fact very similar to solving systems of constraints and Newton employs mostly the same techniques to solve them. Our first example in this area is an unconstrained minimization, i.e., the search for the minimum value of a nonlinear function. A typical problem taken from [ 161 is given by the minimization of the function

fj icos((i + l)& k=t j=,

+ i).

Restricting attention to the range [-lo, lo] for the variables, to solve this problem is depicted in Fig. 3. For variable Min, Newton returns the interval

a simple Newton program

[-186.7309211408969247,-186.73089666225865811 which bounds the value of the global optimum and, for variable Boxes, a list of the global solutions. There are 18 global optima in this problem, one of them being Xl in X2 in [5.4828642061208122,5.4828642089241431] [4.8580568744302264,4.8580568843370485].

P. Vun Hentenryck et al. IScience qf Chputer

Proyramnhg

30 (1998)

83-118

91

minimization(Min,Boxes) :Bl = 4.97, B2 = -1.88, B3 = -29.08, B4 = -78.02, minof 4.3*X1 + 31.8*X2 + 63.3*X3 + 15.8*X4 + 68.5*X5 + 4.7*X6 subject-to [ Xl in [0,0.311, X2 in [0,0.046], X3 in [0,0.0681, X4 in [0,0.042], X5 in [0,0.028], X6 in [0,0.0134], 17.1*X1 + 38.2*X2 + 204.2*X3 + 212.3*X4 + 623.4*X5 + 1495.5*X6 - 169*X1*X3 - 3580*X3*X5 - 3810*X4*X5 18500*X4*X6 - 24300*X5*X6 >= Bl, 17.9*X1 + 36.8*X2 + 113.9*X3 + 169.7*X4 + 337.8*X5 + 1385.2*X6 - 139*X1*X3 - 2450*X4*X5 - 16600*X4*X6 17200*X5*X6 >= B2, -273*X2 - 70*X4 - 819*X5 + 26000*X4*X5 >= B3, 159.9*X1 - 311*X2 + 587*X4 + 391*X5 + 2198*X6 - 14000*X1+X6 >= B4 ] isin Min for Boxes.

Fig. 4. A Newton program

for constrained

optimization.

Newton takes about 20s to solve this problem. Note that Newton returns the global optimum, not a local optimum. Newton has also solved a variety of constrained optimization problems. A beautiful example taken from [8] is shown in Fig. 4. Newton solves this problem in about 2s. Newton has two primitives for constrained optimization, minof and safe-minof. They differ on the way they compute the upper bound to the minimum value (and hence the optimum solutions). In minof, the upper bound is computed using canonical boxes which make the constraint system box-consistent. As a consequence, the result may not be an upper-approximation of the minimum value, since these boxes are not guaranteed to contain solutions. In contrast, safeminof uses safe boxes of the constraint system to compute the upper bound which is then a conservative (but

92

P. Vun Henrenryck et (11. IScience of Comnputer Progrunminy

30 (1998)

83-118

possibly

poor) approximation

of the upper bound.

This is explained

in more detail in

Section 3.6.

3. The constraint language Newton This section describes the specification of Newton in more detail. It describes successively its syntax, its constraint solver, its splitting operations, its facilities to obtain safe and canonical boxes, its primitives for unconstrained and constrained optimization, and some pragmatic issues. 3.1. The syntas oj Newton an outline of the syntax of Newton. The figure is not complete

Fig. 5 describes

and omits certain features of the language but it is representative of the kernel of the implementation. Newton is a superset of Prolog. A program is a set of clauses, each of which is either composed of a head or of a head and a body. A body is either a constraint, a goal, or a conjunction of two bodies. A constraint is either a single basic constraint or list of basic constraints. Basic constraints are equations and inequalities over expressions as well as membership constraints which are abbreviations for a conjunction of two inequalities. Expressions are constructed from variables, floating-point numbers, multiplication, addition, subtraction, exponentiation by a natural number, sine and cosine, and the logarithm and exponentiation functions. A goal is either an atom, or one of the facilities available in the language for safe boxes and optimization. 3.2. Constraint We now turn solz+q to the study of nonlinear constraint solving in Newton. Since

Newton is a superset of Prolog, it also includes equations over finite trees but these are not covered in this article. The rest of this section is organized as follows. Section 3.2.1 briefly reviews some fundamental concepts of interval arithmetic, Section 3.2.2 discusses the representation of constraints, Section 3.2.3 introduces the notion of boxconsistency, Section 3.2.4 describes several interval extensions on which Newton applies box-consistency, Section 3.2.5 describes the specification of the constraint solver. The internal details of the solver are not covered here. Instead, we focus on a high-level static specification of the language. The reader interested in the details of the solver can consult [36]. 3.2.1. Interval arithmetic We consider RX = RU infinity symbols and the consider a finite subset g the floating-point numbers

{-cc, co} the set of real numbers extended with the two natural extension of the relation < to this set. We also of Y? containing -c~,oo,O. In practice, .9 corresponds to used in the implementation.

(Program) ::= (Clauses)* (Clauses ) ::= (Head). (Head) :- (Body). (Head) ::= (Body) ::= (Constraint) (Goal) (Body) > (Body) (Constraint) (Atom)

::=

(c) E(C)1 (c) : :=

(Expr) >= (Expr) <= (Expr) = (Expr) in (Expr) : := (Expr) (Expr) (Expr) (Range)

War)
(Float) (Expr) * (Expr) (Expr) + (Expr) (Expr) - (Expr) (Expr) ^ (Nat) - (Expr) sin((Expr)) cos((Expr)) log((Expr)) exp((Expr)) (Range) : := [(Expr) , (Expr)l (Goal) ::= (Atom) (Var) is_a_Sbox_of (Constraint) (Var) is_a_Cbox_of (Constraint) minof (Expr) in (Range) is-in (Var) for (Var) minof (Expr) subject-to (Constraint) is-in (Var) for (Var) safeminof (Expr) subject-to (Constraint) is-in (Var) for (Var) Fig. An outline the syntax Newton 5. of of

94

P. Van Henrenryk

et al. IScience of Computer Programming 30 (1998)

83-118

Definition

1 (Interval).

An interval

[1, U] with 1, u E .9 is the set of real numbers

{YE!R 16r6u). The set of intervals Definition is denoted by Y and is ordered by set inclusion. 2 and hull). Let S be a subset of 8. The enclosure of S, denoted

2 (Enclosure

by ?? or OS, is the smallest interval I such that 5 C I. We often write r instead of {r} for r E R The interval hull of II and 12, denoted by II H I,, is defined as box{lr u 12). We denote real numbers by the letters r, v, a, 6, c, d, s-numbers by the letters I, m, u, intervals by the letter I, real functions by the letters f,g and interval functions (e.g., functions of signature 9 + S) by the letters F, G, all possibly subscripted. We use I+ (resp. I-) to denote the smallest (resp. largest) F-number strictly greater (resp. smaller) than the F-number 1. To capture outward rounding, we use 1~1 (resp. 1~1) to return the smallest (resp. largest) F-number greater (resp. smaller) or equal to the real number Y. We also use Z to denote a box (Q, . . . , rn). A canonical interval is an interval of A canonical box is a tuple of canonical intervals. .,1/ is the set of natural numbers. Finally, we use (II,. . . ,I,) and Y to denote a tuple the form [1, I] or of the form [I, I+]. 2 is the set of rational numbers and the following notations.

ul) = I, right( [1,u] ) = u,

lef([l, center([a, b]) = L(a + b)/2j The fundamental Definition iff when a # - (x: and b # IX. arithmetic is the notion of interval extension extension. of f: P + !R concept of interval extension).

3 (Interval

F: 9 -3 .Y is an interval

VI~...I,E~:~, An interval relation iff VI, . . . I,ES:[

EII ,..., r,,EI,+f(q

,..., r,)EF(I,

,..., In). of a relation c : W + Boo1

C : 9 + Boo1 is an interval extension

3rlEI1,...,

+,EI,c(rl,..., @ defined as

r,)]*C(Il,...,

I,).

Example 4. The interval

function

[a~~hl~[a~.W=[La~ +azJ,[h +hll

is an interval as I, -I2 is an interval extension (IlnZ2#0) extension of the equality relation on real numbers. of addition of real numbers. The interval relation defined

2 Our intervals

are usually called floating-point

intervals

in the literature

P. Van Hentenryck et al. IScience of Computer Programming 30 (1998) 83-118

95

It is important addition a function

to stress that a real function the interval function possible

(resp. relation) interval

can be extended interval

in many of while

ways. For instance,

@ is the most precise containing

extension

(i.e., it returns the smallest always returning we assume

all real results)

[-co, co] would be the least accurate. fixed interval extensions for the basic real operators of + is defined by @) (for instance, the interval extension

In the following,

+, -, x and exponentiation

and the basic real relations =, 3. In addition, we overload the real symbols and use them for their interval extensions. Finally, we denote relations by the letter c possibly subscripted, interval relations by the letter C possibly and relations are used as synonyms in this paper. subscripted. Note that constraints

3.2.2. Constraint representations It is well-known that different computer representations of a real function produce different results when evaluated with floating-point numbers on a computer. As a consequence, the way constraints are written may have an impact on the behaviour on the algorithm. We will abuse notation by denoting functions (resp. constraints) and their representations by the same symbol. In this section, real variables in constraints will be taken from a finite (but arbitrary large) set {xi,. . .,x,}. Similar conventions apply to interval functions and interval constraints. Interval variables will be taken from a finite (but arbitrary large) set {Xi,. ,Xn} and interval constraints by the letter C. For simplicity of exposition, we restrict attention to equations. It is straightforward to generalize our results to inequalities. For convenience, we also assume in the rest of this section that all constraints are defined over variables xi,. ,x,.

3.2.3. Box consistency Box consistency [l] is an approximation

of arc consistency,

a notion well-known

in

artificial intelligence [ 171 which states a simple local condition on a constraint c and the set of possible values for each of its variables, say Di, . . . , D,. Informally speaking, a constraint of c. Definition 5 (Projection constraint). A projection constraint (c,i) is a pair of a constraint c and an index i (1 <i <n). Projection constraints are denoted by the letter p, possibly subscripted. Example 6. Consider the constraint 1, 2) are projection constraints. Definition . . ..D.) iff 7 (Arc consistency). xy +xz = 1. Both (XT+ XI = 1 , 1) and (XT+x5 = c is arc-consistent if none of the Di can be reduced by using projections

A projection

constraint

(c, i) is arc-consistent

wrt (01,

Di C {ri 13rl E DI,. . . ,rj_l

lDi_~,ri+l E Di+l ,..., r,ED,

: C(Y)}.

96

P. Vun Hentrruyk

et al. IScience

qf Computer

Programming

30 (1998)

83-118

A constraint consistent

c is arc-consistent

wrt (01,. . . . Dn) if each of its projections .Y is arc-consistent wrt (Q, . . . ,Dn).

is arc-

wrt (01,. . . ,D,,). A system of constraints in Y is arc-consistent

wrt (0, , . . , Dn)

if each constraint

Example 8. Let c be the constraint

x: +xs = 1. c is arc-consistent since, for instance,

wrt ([ - 1, 11.[ - 1, 11) there is no value ~1

but is not arc-consistent wrt ([-1. l],[-2,2]) for xl in [-1, l] such that r: + 2 = 1. Given some initial domains

(07. . . . , 0,). an arc consistency algorithm computes the largest domains (01,. ,D,,) included in (Dy,. . . , 0,) such that all constraints are arcconsistent. These domains always exist and are unique. Enforcing arc consistency is very effective on discrete combinatorial problems. However, it cannot be computed in general when working with real numbers and polynomial constraints. Moreover, simple approximations to take into account numerical accuracy are very expensive to compute (the exact complexity is an open problem). For instance, a simple approximation of arc consistency consists in working with intervals and approximating the set computed by arc consistency to return an interval, i.e., Zi=U(Zin{ rjI3ri EZ1,...,rj_i EZi-l,...,Tj+l EZi+l,I;,EZn
1 C(Y)}).

This condition, used in systems like [28,2], is easily enforced on simple constraints such as XI =x2 +x3, XI =_YZ - x3, x1 =.XZ x x3 but it is also computationally very expensive for complex constraints with multiple occurrences of the same variables. Moreover, decomposing complex constraints into simple constraints entails a substantial loss in pruning, making this approach unpractical on many applications. See [l] for experimental results on this approach and their comparison with the approach presented in this paper. The notion of box consistency introduced in [l] is a coarser approximation of arc consistency which provides a much better trade-off between efficiency and pruning. It consists in replacing the existential quantification in the above condition by the evaluation of an interval extension of the constraint variables. Since there are many interval extensions box consistency in terms of interval constraints. on the intervals of the existential for a single constraint, we define

Definition 9 (Interd projection construint). An interval projection constraint (C, i) is the association of an interval constraint C and of an index i (1 <i <n). Interval projection constraints are denoted by the letter P, possibly interval subscripted. constraint (C, i) is box-

Definition 10 (BOX consistency). An consistent wrt Z = (Ii,. . . , In) iff C(Z I,..., zj_].[1,1+],1.
z+l,...,L) A

projection

c(~I,...,~i-I,[u-,ul,~,+l,

. . . . L),

where I = left(I;) and u = rigId( An interval constraint is box-consistent wrt Z if each of its projections is box-consistent wrt I. A system of interval constraints is

P. Van Henrenryck et al. IScience of Computer Proyramrniny 30 (1998)

83-118

97

box-consistent wrt I. Intuitively

wrt

Z iff each

interval

constraint

in the system

is box-consistent

speaking,

the above condition interval to zi-1,r,,li+1,... constraint

states that the ith interval cannot be pruned obtained by replacing all variables but X;: Note also that the

further using the unary by their intervals, above condition

since the boundaries is equivalent

satisfy the unary constraint.

II=o{

rj EZ, 1C(ZI

)...,

,r,,>,

which shows clearly that box consistency is an approximation of arc consistency. The difference between arc consistency and box consistency appears essentially when there are multiple occurrences of the same variable. Example Il. Consider the constraint xi +.Q -xi = 0. The constraint is not arc-consistent

since there is no value ~1 for XI which satisfies ~1 + 1 - Y! = 0. wrt (r-l, ll.]-l,ll) On the other hand, the interval constraint Xl + X2 - Xl = 0 is box-consistent wrt since ([-l,l]+[-l,-l+]-[-l,l])n[O,O] and ([-l,l]+[l-,l]([-l,ll.[-1, ll), [- 1, 11) n [O,O]are non-empty.

3.2.4. Interval extensions for bos consistency Box consistency strongly depends on the interval extensions chosen for the constraints and different interval extensions can produce very different (often incomparable) tradeoffs between pruning and the computational complexity of the solver. In this section, we consider three extensions used in Newton: the natural interval extension, the distributed interval extension, and the Taylor interval extension. Natural interval extension. The simplest extension of a function (resp. of a con-

straint) is its natural interval extension. Informally speaking, it consists in replacing each number by the smallest interval enclosing it, each real variable by an interval variable, each real operation by its fixed interval extension and each real relation by its fixed interval extension. In the following, if f (resp. c) is a real function constraint), we denote by f (resp. 2) its natural extension. Example 12 (NaturuZ interval extension). The natural interval extension tion xl (x2 + x3) is the interval function Xi (XI +X3). The natural interval the constraint xi (x2 + x3 ) = 0 is the interval constraint Xr (X2 + X3 ) 2 0. (resp.

of the funcextension of

The advantage of this extension is that it preserves the way constraints are written and hence users of the system can choose constraint representations particularly appropriate for the problem at hand. A very nice application where this extension is fundamental is the More-Cosnard discretization of a nonlinear integral equation described in the experimental results. Using the natural extension allows users to minimize the problem of dependency of interval arithmetic and hence to increase precision.

98

P. lfun Hentenr~ck et al. IScience of Computer Progranminy

30 (19981 83-118

Distributed interval e.xtension. The second interval not preserve the way constraints are written straints. The key advantage of this extension

extension

used by Newton does form of the conto enforce

but uses a distributed

is that it allows the algorithm

box consistency by applying the interval Newton method on univariate real functions. The real functions are derived from univariate interval constraints obtained by replacing all but one variable by their intervals. As a consequence, applying box consistency will be particularly efficient, although the pruning may be weaker than for the natural extension due to the dependency problem of interval arithmetics. 3 Intuitively, the distributed interval extension should be viewed as a way to speed up the computation of box consistency on the natural extension. However, it may happen that it gives more precision than the natural extension if users are not careful in stating their constraints. Definition 13 (Distributed forwz). A constraint c in (simplified) sum of products form ml + . . + nzk = 0, where each monomial mi is of the form c$ . . . x? with q E _2 and e, E .V, is said to be in distributed form. 4 Definition 14 (Distributed interval extension). The distributed interval extension of a function f (resp. constraint c) is the natural extension of its distributed form. The distributed interval extension of a function f (resp. of a constraint c) is denoted by .r (resp. ?;). Example 15 (Distributed interval extension). The distributed interval extension of the function x1(x2 + x3) is the interval function Xl& +X,X3. The distributed interval extension of the constraint x1(x2 +x3) = 0 is the interval constraint X,X, +X,X3 =o. Taylor interval estension. The last interval extension we introduce is based on the Taylor expansion around a point. This extension is an example of centered forms which are interval extensions introduced by Moore [20] and studied by many authors, since they have important properties. The Taylor interval extension of a constraint is parametrized by the intervals for the variables in the constraint. It also assumes that the constraint which it is applied to is of the form f = 0 where f denotes a function which has continuous partial derivatives. Given these assumptions, the key idea behind the extension is to apply a Taylor expansion of the function around the center of the box and to bound the rest of the series using the box. Definition 16 (Taylor interval extension). Let c be a constraint

f = 0, f be a function

with continuous partial derivatives, Z be a box (II.. ,I,,), and mi be the center of Ii. The Taylor interval extension of c wrt I, denoted by c(), is the interval constraint

3 Note that it is not always necessary to go through the distributed form to obtain the above property Newton adopts it for simplicity. 4 The distributed version can easily be turned into a canonical representation for constraints.

but

P. Van Hentenryk

et al. IScience of Computer Progranminy

30 (19981 83-118

99

In the current version of our system, the partial derivatives using automatic differentiation [30].

are computed numerically

3.2.5. Specijication of the constraint solver We are now in position to specify the behaviour

of the constraint

solver of Newton. extensions

Informally speaking, Newton enforces box consistency of the constraints without removing any solution. Definition 17. Let Y be a system box-consistent wrt Z if of constraints

on the three interval

{cl,. . . , c,} and Z be a box. Y is

{G,. . . ,Fn} u (6,. . . ,c7,} u {cp,. is box-consistent wrt I.

. . ,c;}

Specification 1 (Constraint solving). Let Y be a system of constraints. The constraint solver of Newton returns a box Z which satisfies the following two properties: _ Y is box-consistent wrt Z; ~ if (ri,. . ,rn) is a solution of Y, (ri,. . . ,1;2) E I.

Note that there may be more than one box Z satisfying the above conditions due to the nature of the Taylor extension. However, any of these boxes is sound by definition and which box is actually returned is left as an implementation issue. 3.3. Splitting As should be clear from the above description, the constraint solver in Newton is incomplete. To solve practical applications, it is necessary to enhance the constraint solver with nondeterministic choices. Newton supports splitting operations which can be specified as follows. this process by providing two

Specification 2 (Splitting operations). Procedure split (t> holds iff t is a canonical interval. Procedure listSplit ( Et 1, . . . , trill > holds iff (tr , . . , tn) is a canonical box. In its current implementation, Newton does not simply enumerate all the canonical

intervals. Rather, it performs a binary search, splitting the intervals in two parts until canonical boxes are obtained. This implementation gives naturally rise to a branch and prune algorithm, which alternates a constraint propagation phase and nondeterministic choices. In the case of listsplit, Newton uses a simple round-robin strategy to split the box. 3.4. Sujk boxes Newton provides facilities to guarantee that boxes returned as answers be safe, i.e., that they actually contain a unique solution to the constraint system. Verifying that a

100

P. Van Hentenryck et al. IScience of Computer Programming 30 (1998)

83.-118

box Z contains

a solution

to an inequality

,f 3 0 is relatively

easy. It is sufficient

to

evaluate f over Z, producing an interval [I, u]. The constraint is guaranteed to be satisfied by a point in the box if 13 0. Determining if a box provably satisfies an equation or a system of equations in interval analysis. is obviously more difficult and this is an active research area currently used in Newton are described in [36]. boxes in Newton. The techniques

We now specify the facilities Specification 3

to obtain safe and canonical boxes). The procedure

(Safe and canonical System

Box is-a-Sbox-of holds iff System procedure Box is_a_Cbox_of holds iff System

is box-consistent

wrt Box and Box is a safe box of System.

The

System wrt Box and Box is a canonical box.

is box-consistent

3.5.

&constrained

optimization optimization in Newton. Informally speaking, the

We now turn to unconstrained procedure minof F in Range is-in

Min for

Solutions

holds if Min is the (global) minimum value of F (which is assumed to be in Range) and Solutions is a list of all (global) minima. The precise specification requires however a number of definitions. Definition 18 (Interval minimum). Let K be the set of canonical value of a function f, denoted boxes of the form by imin(f), is an

(4 , . . . ,I,). The interval-minimum interval [I, u] such that I = $; left&Z)) and

u = r;l;lKn right(f^(Z)). box Z such that f(Z) The procedure n imin(

An interval-minimum Specification minof 4 F in

of f is a canonical

f )#8.

(Unconstrained Range is-in

minimization). Min for

Boxes

holds iff _ Min is the interval-minimum value of F; - Boxes is a list of interval-minima of F in range of F belongs to at least one box of Boxes.

such that each (global)

minimum

P. Vun Hrntenryck

et al. IScience

of Computer

Proqrunmirq

30 (1998)

83-118

101

The current minima.

implementation

of Newton

is based on a depth-first function

branch

and bound

which uses partial derivatives

of the objective

to speed up the search for the

3.6. Constrained

optimkation

Constrained optimization in Newton looks superficially the same as unconstrained optimization. However, special care should be exercised when guaranteed bounds are required. Informally speaking, the procedure minof F subject-to Constraints is-in Min for Solutions

holds if Min is the (global) minimum value of F satisfying the constraint Constraints and Solutions is a list of all (global) minima. The precise specification requires number of definitions, closely related to those of the previous section.

Definition 19 (Zntervd minimum). Let J be a function, Y be a system of constraints, and let Kb be the set of canonical boxes I such that 9 is box-consistent wrt I. The interval-minimum value of a function f wrt 9, denoted by imin(f, 9). is an interval [1, U] such that 1 =mi; left(f(Z)) h of and u=~miright(~(Z)).

An interval-minimum

f wrtY is a canonical box Z of & such that

f(Z )n imi~z(f,Y)#@ Specification minof 5 (Constrained minimization). Constraints The procedure is-in Min for Boxes

F subject-to

holds iff - Min is inrin(F,Constraints); - Boxes is a list of interval-minima of F wrt Constraints of F wrt Constraints belongs such that each (global) minimum to at least one box of Boxes.

The above procedure does not necessarily provide guaranteed bounds on the minimum value of the function wrt the constraints. The problem comes from the fact that box-consistent canonical boxes are not guaranteed to contain any solution to the constraint system. As a consequence, the upper bound which is returned may not be an upper approximation of the minimum value. To remedy this problem, Newton provides a facility to obtain guaranteed bounds as well. The specification is slightly more involved. Definition 20 (Safe interval minimum). Let f be a function, 9 be a system of constraints, & be the set of canonical boxes Z such that Y is box-consistent wrt I, and

102

P. Van He?ltenryck

et al. IScience

of Computer

Programming

30 (1998)

83-118

KS be the set of safe boxes Z, such that Y is box-consistent

wrt Z,. The safe interval-

minimum value of a function such that I= Z$I left&Z b A conservative )) and

f wrtY, denoted by smin(f, Y), is an interval [i, u]

u = z$ of

vi,ght(f^(Z )). box Z of Kb such that

interval-minimum

f wrt9 is a canonical

f(z)nsmin(f,.Y)#0.
The main difference is of course the computation safe boxes instead of canonical boxes. Specification safearinof 6 (Safe constrained F subject-to minimization). Constraints of the upper bound, which uses

The procedure is-in Min for Boxes

holds iff _ Min is smin(F,Constraints); _ Boxes is a list of conservative interval-minima of F wrt Constraints such that each (global) minimum of F wrt Constraints belongs to at least one box of Boxes. The current implementation conditions to find the minima. of Newton enforces box-consistency on the Fritz-John

3.7. Pragnzatics Newton has also a number of pragmas that affect the semantics of the programs.

Perhaps the most important one is the fact that users can specify the precision of the system, since they are rarely interested in canonical boxes. In practice, a precision of, say, lo- may be acceptable and users can specify that Newton returns boxes whose width is smaller than 10p8. The semantics of the language essentially remains the same, except that the system returns boxes of the required precision. These boxes covered all the boxes that would be returned otherwise. Another feature of Newton is that the constraint solver may use (and typically uses) linear combinations of constraints to improve pruning. Hence, box consistency is applied to a system which is different from the system specified by the program.

4. Experimental

results of Newton on traditional benchunconstrained optimization, and

This section describes some experimental results marks. It considers successively equation solving, constrained optimization.

P. Van Hentenryck et al. IScience oj Computer Programming 30 (1998)

83-118

103

4.1. Equation

sohing

This section reports experimental

results of Newton on a variety of standard bench-

marks for equation solving. The benchmarks were taken from papers on numerical analysis [23], interval analysis [7,9,22], and continuation methods [37,25,24, 181. We also compare Newton with a traditional interval method using the Hansen-Seguptas operator, range testing, and branching. This method uses the same implementation technology as Newton and is denoted by HRB in the following. 5 Finally, we compare Newton with a state-of-the-art continuation method [37], denoted by CONT in the following. Note that all results given in this section were obtained by running Newton on a Sun Spare 10 workstation to obtain all solutions. In addition, the final intervals must have widths smaller than 10P8. The results are summarized in Table 1. For each benchmark, we give the number of variables (2:). the total degree of the system (d), the initial range for the variables, and the results of each method in seconds. Note that the times for the continuation method are on a DEC 5000/200. A space in a column means that the result is not available for the method. A question mark means that the method does not terminate in a reasonable time (>l hour). The rest of this section describes each benchmark. Note that Newton solves Broyden, More-Cosnard, and interval benchmarks il, i2, i3 and i5 without backtracking (contrary to most interval methods we know of). Newton is essentially linear on Broyden and quadratic on More-Cosnard. Broyden bunded functions. This is a traditional benchmark of interval techniques and was used, for instance, in [6]. It consists in finding the zeros of the functions fi(Xl,....
~,~)=xi(2 + 5x?)+

1 - C x,j(l +x,)
,SJ,

(1 Gidn),

where Ji = { j /j # i & max( 1, i - 5) < j < min(n, i + 1)}. One of the interesting features of this benchmark is that it is easy to scale up to an arbitrary dimension and hence provides a good basis to compare various methods. Discretization of u nonlinear inteyrul equation. This example comes from [23] and is also a standard benchmark for nonlinear equation solving . It consists in finding the root of the functions fk (xi,. . .x, ) ( 1 d k d m) defined as 1 xk+2(m+1)

(l-&t/(,+$+1)3+Q
J=l

5
j=k+l

(1-tj)(xj+$+1)3

1
,

where t/ =jh and h = l/(m + 1). These functions come from the discretization of a nonlinear integral equation, giving a constraint system denser than the sparse constraint system for the Broyden banded functions (i.e., many more coefficients are non-zeros). The variables xi were given initial domains [-4.51 as in [31].
5 Some interval methods such as [6] are more sophisticated than HRB but the sophistication aims at speeding up the computation near a solution. Our main contribution is completely orthogonal and aims at speeding up the computation when far from a solution and hence comparing it to HRB is meaningful.

104

P. Vun Hentenryck

et al. IScience

of Computer

Proyromminy

30 (1996) 83-118

Table I Summary of the experimental results on equatiotl solving Benchmarks Broyden Broyden Broyden Broyden MorB-Cosnard Mor&Cosnard More-Cosnard MorB-Cosnard il i2 i3 i4 i5 kin1 kin2 eco eco eco eco eco eco combustion chemistry neuro neuro t 10 20 320 320 20 40 80 80 10 20 20 10 10 12 8 4 5 6 7 8 9 10 5 6 6 d 30 320 330 330 30 340 3x0 380 30 30 30 6O 110 4608 256 18 54 162 486 1458 4374 96 108 1024 1024 Range Newton 1.65 4.25 113.71 143.40 24.49 192.81 1752.64 1735.09 0.06 0.30 0.31 13.94 0.08 14.24 353.06 0.60 3.35 22.53 127.65 915.24 8600.28 9.94 6.32 0.91 172.71 HRB 18.23 ? ? CONT

[-I.11

[-1,ll
[-I.11 [-lo*, 1081 E-4.51 [-4.51 [-4.51 [-108,0] [-2.21 [-1.21 [-2.21 i-1,

968.25

? ?
14.28 1821.23 5640.80 445.28 33.58 1630.08 4730.34 2.44 29.88 7 ?

II [-I,11
[-108% 1081 [-10X, 10x1 [-108,108] [-lo*, 10x1 [-10X, 1081 [-10X, lo] [-108,108] [-10X. lo*] [-10X. 10x1 [O. 1OS] [-IO. lo] [- 1000, IOOO]

35.61 1.13 5.87 50.18 991.45

? 28.84 ?

57.40 56.55 5.02 5.02

Intemal arithmetic benchmarks. These are traditional benchmarks from interval arithmetic papers [21,9]. Benchmark ii is the following set of equations: 0 =x1 - 0.25428722 0 =x2 - 0.37842197 0 =x3 - 0.27162577 0=x4 - 0.19807914 - 0.18324757~~~~~~. - O.l6275449x,~,~x~, - 0.16955071 xIx2.xIo, - O.l5585316x,.x,x~, - 0.19950920.x,x6x3, - 0.18922793 x8x5x1(), - 0.21180486xzx5xs, - 0.17081208x1x7x6, - O.1961274Ox,ox6x8,

0 =x5 - 0.44166728 0 =x6 - 0.14654113 0=x7 - 0.42937161

0 =x8 - 0.07056438 0 =x9 - 0.34504906

0 = xl0 - 0.4265 1102 - 0.21466544 x4x8x1,

P. Van Hentenryck et al. IScience qf Computer Proyranminy

30 (1998)

83%II8

105

with initial intervals

[-2,2].

Benchmark

i2 is the set of equations

0 =x1 - 0.24863995 0 =x2 - 0.87528587 0 =x3 - 0.23939835 0=x4 - 0.47620128

- 0.19594124 x,x,(,x,& - 0.05612619x,sxsx,,, - 0.20177810xlox7x~,, - 0.16497518xl~x15x1, - 0.20198178~~x9x~~, - 0.15724045 xl(,x1sxI,, - 0.12384342x,2x13x15, - 0.18180253x19x15x1s, - 0.21241045 x13x2x17, - 0.16522613 x12x9x13, - 0.17221383 x16x17x8, - 0.2355625 1 x14x1Ix4,

0 =x5 - 0.24711044 0 =x6 - 0.33565227 0 =x7 - 0.13128974 0 =X8 - 0.45937304 0 =x9 - 0.46896600 0 =xlo - 0.57596835 0 =xll - 0.56896263

0 = xlz - 0.70561396

0 = xl3 - 0.596425 12 - 0.24475 135 x7x16x20, 0 =x14 - 0.46588640 0 = xl5 - 0.10607114 - 0.21790395 s13x3x10, - 0.20920602 x,x~x,o,

0 = xl6 - 0.265 16898 - 0.2 1037773 xJx19x9, 0

=x17 -

0.20436664

- 0.19838792~20~10~13, - 0.18114505x~x13xs, - 0.04417537 x7x13x16, - 0.17949149x,x3xll, as i2 but

O=x18 - 0.56003141 0 =x19

-

0.92894617

0 =xlo - 0.57001682

with initial intervals [-1,2]. Benchmark i3 has the same set of equations has initial intervals [-2,2]. Benchmark i4 has the set of equations 0 = x; - 0.25428722 0 =x; 0 =xf - 0.37842197 - 0.27162577 - 0.18324757 x,;-xfx;, - 0.16275449xf~;?~x& - 0.16955071 xfx;~;~, - 0.155853 16 x:x:x& - 0.19950920x;x~x~.

0 =x4 - 0.19807914 0 =xf - 0.44166728

106

P. Van Hentenryck

et ul. ISciemv

of Computer

Proyrumminy

30 (1998)

83-118

0=x; - 0.14654113 - 0.18922793 x;x;x&,

0 =x; 0 =x; 0 =x; - 0.42937161 - 0.07056438 - 0.34504906 - 0.21180486x;x;x;, - 0.17081208 x;x;x& - 0.19612740&x~x;,

0 = xx - 0.4265 1102 - 0.2 1466544 x:x%x;, and initial Benchmark intervals [- 1,1]. The number of solutions i5 has the following set of equations: - 0.18324757 x:x:x; - 0.16275449 x;x;~x; - 0.16955071 x;x;x$, - 0.15585316~;~;~; + xix;, + x;,,x;, +x;x;s, +xfx;. must be a multiple of 1024.

0 = xl - 0.25428722 0 =x2 - 0.37842197 0=x3 - 0.27162577

0 =x4 - 0.19807914 0=_y5 - 0.44166728 0 =x6 - 0.14654113 0=x7 - 0.42937161

- O.l995092Ox~.x,3x~ +x:x;, - 0.18922793 x;x;x& - 0.21180486x;x5~s - 0.17081208 x:x+x; - 0.19612740 x;~x;x; + x;x;~, +x:x;, + x;x;, +x:x;, + xix;,

0 =x8 - 0.07056438 0 =x9 - 0.34504906

0 = xl0 - 0.4265 1102 - 0.21466544 x;x;x; and initial intervals

Kinrnmtics

[-1, 11. Application kin1 comes from robotics and describes the

applications.

inverse kinematics of an elbow manipulator [9]. It consists of a sparse system with 12 variables and the set of equations is as follows: szc5s6 - s3c5s6 ClC2SS J-2.95 + + CIC3S5 + + s4c5s(j + c2c6 + c3c6 + c4c(, =

0.4077,

ClC4S5 =

SIC5

1.9115,

s3s5

s4s5

1.9791,
c1c2 + clc3 + clc2

c~cz +
S1c2

c1c3

cic4

=4.0616,
=

+s1c3

~1~4

slc2

slc3

slc2

1.7172,

S? +S3 s;+c;=l

+s4

+s2

+s3

+s2=3.9701,

(l,<i<6).

The second benchmark, denoted by kin2, is from [24] and describes the inverse po_ sition problem for a Six-revolute-joint problem in mechanics. The equations which

P. Van Hetltenq*ck et al. IScier~e oj Computer Proyranvniny 30 (1998)

Table 2 Coefficients the inversekinematicsexample for -0.249150680 +I.609135400 +0.279423430 +I.434801600 +o.ooooooooo +0.400263840 -0.800527680 +o.ooooooooo +0.074052388 -0.083050031 -0.386159610 -0.755266030 +0.504201680 -1.091628700 +o.ooooooooo +0.049207290 +0.049207290 f0.125016350 -0.686607360 -0.119228120 -0.719940470 -0.432419270 +o.ooooooooo +o.ooooooooo -0.864838550 -0.037157270 +0.035436896 +0.085383482 +o.ooooooooo -0.039251967 +0.000000000 -0.432419270 +o.ooooooooo +0.013873010 -0.635550070 -0.115719920 -0.666404480 +0.110362110 +0.290702030 +I.258776700 -0.629388360 +0.581404060 +0.195946620 -1.228034200 +o.ooooooooo -0.079034221 +0.026387877 -0.057131430 -1.162808100 +1.258776700 t2.162575000

83-118

107

+I.48947730 +0.23062341 +I.32810730 -0.25864503 +I.16517200 -0.26908494 +0.53816987 +0.58258598 -0.20816985 +2.68683200 -0.69910317 f0.35744413 f1.24991170 +I.46773600 +I.16517200 +I.07633970 -0.69686809

describe

a denser constraint 1 =o,

system are as follows (1 <i <4):

x;+x;+, -

+U9iXl

+ UlOi.X2+ UlljX3 + UI~~X~ + U13iX5 + U14iX6 + u,~~X, + U,6ixg + u,7l =O,

where the coefficients

ukl are given in Table 2. In both examples, The following

the initial intervals

were given as [-log, lo]. An economics modelling application. a difficult economic modelling problem For a given dimension n, the problem

example is taken from [25]. It is dimensions.

that can be scaled up to arbitrary can be stated as the system

(
n-l

n-k-l

Xl, +

C
i=l

XiXi+h

-x,1 -

Ck=O

( l<k<n-l),

and the constants can be chosen 1,. , n ~ 1 but the results are scale. Combustion application. This sents a combustion problem for

at random. Our experimental results use the constants essentially the same for other constants in the same problem is also from Morgans book [25] and reprea temperature of 3000. The problem is described by

108

P. Van Hentenryck et al. IScience of Computer Programming 30 (1998) 83-118

the following

sparse systems of equations:

lop5,

~2 +2x6 +x9 +2X10=

x3 +x8 =

3 x1O-5,

x1+x3+2x5+2xs+x~+x,~=5x10-~,
x4 +

2x7 = 10-5,

0.5140437 x10-7x5 =xf, 0.1006932 x 1o-%6 = 2x;, 0.7816278 x lo-15x7 =x;, 0.1496236 x lo-%s =x1x3,

0.6194411 x 1O-7x9 =x1x2, 0.2089296 x 10-4x,o =x1x;, which is typical of chemical equilibrium systems. Chemical equilibrium application. This problem a chemical
R=

originates

from [ 181 and describes

equilibrium 10,

system. The set of equations

is as follows:

R5 = 0.193, R6 = 0.002597/&, R7 = 0.003448/&i

R8 = 0.00001799/40

R9 = 0.0002 155/v%
R,o = 0.00003846/40,

XIX? +x1 -3x5=0,

2x1x2 + XI f x2x: + R8x2 - Rx5 + 2Rto~i -k R7Xz~3 + R9~2~4 = 0, 2x2.x: f2R5x: - 8x5 +&x3
+R7x2x3 10,

R~X~XJ -I2x: - 4Rx5 = 0,

x1x2 + XI + &ox;

+x2x;

+ R8x2 + R5x; +x;

- 1 + R6x3 + R7x2x3 + R9x2xd = 0,

and all xis must be positive. A neurophysiology application. This example illustrates The application is from neurophysiology [37] and consists the limitations of Newton. of the fdowing system of

P. Van Henten&

et (11. IScience of ComputevPrograrrming (1998) 83-118 30

109

equations:

xf+x;= 1,
x; +x;= x,x: +x6x: 1, =Lr,

xgx: + xgx; = C?. x,x,x;

x,x:x,

+x,x:x,
+ x,x;x,

=
=

c3,

c-4.

No initial intervals random. 4.2.

Unconstrainrd

for the variables

were given and the constants

ci can be chosen at

optimization

Table 3 describes the results of Newton on unconstrained optimization. The benchmarks were taken mainly from [16, 10,32,34] and, for each of them, we give the number of variables, the range of the variables, the CPU time, and the number of splits. The experimental results once again exhibit a number of interesting facts. Newton is able to solve problems such as Levy5 and Levy6 in essentially linear time in the number of variables. Newton solves the problems Ratz25, Ratz27, and Rat2210 without splitting. These problems were used in [32] to study splitting strategies. Finally, Newton does not exhibit the behaviour of traditional interval methods on problems such as the Rosenbrock function. The performance of the traditional interval methods degrades substantially when the initial intervals are large, while Newton can be used with arbitrarily large intervals in these cases without degrading the performance. Note that our results are better or comparable to other interval methods that we are aware of [5,32], although the comparison is more difficult since we have not reimplemented in detail. their algorithms. We now describe each benchmark Hump is the three-hump camel function f(x, ,x2) = 12.x: - 6.3x; + xf + 6x2(x2 - x1 ). Levy1 is the function j(x) =x6 - 15x4 + 27x + 250. Levy2 is the function f(x) = - I$ i cos[(i + 1)x + i].
Levy3

is the function fi &icos((i+ &I i=l

1)xk +i).

f(xr,x2)=

110

P. Van Hentenryck et al. IScience of Computer Programming 30 (1998)

83-118

Table 3 Summary

of the experimental

results on unconstrained

optimization Time 0.17 0.09 0.61 16.14 2.13 4.35 15.27 59.08 235.22 4.29 14.86 64.11 372.39 2.50 3.31 5.52 23.29 0.24 0.11 6.69 0.03 0.33 1.19 2.86 4.44 7.42 9.13 10.04 189.56 Splits 3 2 4 30 7

Benchmarks

Range

HOP Levy1 Levy2 Levy3 Levy4 Levy5 Levy5 Levy5 Levy5 Levy6 Levy6 Levy6 Levy6 Beale Beale Beale Beale Schwef ell Booth Powell Schwef e13 Rosenbrock Ratzl Ratz25 Ratz27 Ratz210 Ratz3 Morel More2

2
1 2 2 10 20 40 80 10 20 40 80 2 2 2 2 3 2 4 2 2 5 4 4 4 6 3 4

[-IO, 1081 [-IO, 101

[-IO, IO] [-10, lo] [-10. lo] [-lo. lo] [- 10. lo] [-10. IO] [-10, lo] [-IO, IO] [-10, lo] [-IO, IO] [-IO, lo] [-4.5,4.5] [- 102,102] [- 104,104] [-lo, 101 [-IO. 101 [-IO. 101 [-IO, 201 [-lo, IO] [-lo, 101 [-500,600] to, IO] [O. 101 [O. 101 LO,11 L-4,41 [-25,251

1 3 12 31 61 0 0 267 0 10 0 0 0 0 2 5 32

This function has 760 local minima Levy4 is the related function fi 5 icos((i
k=l i=l

and 18 global minima

in [ - lO,lO].

+ 1)xk + i) + (x1 + 1.42513)* + (x2 + 0.80032). f(x1,. . ,x,) defined as + lOsin(xyi+~)*) + (yn - 1)2

Levy5 is the function sin(nyl)* where vi=

n-l + C (yi - l)*(l

i=l

1 + (Xi - 1)/4

(1 <i<IZ).

Levy6 is the function

n-1

f(xl, . . . ,x,*) defined as

+ sin(27u,)).

sin(37rxL)2 + C (Xi - l)*(l + 10sin(37LX:+1)2) + (In - l)(l

i=l

P. Van Hentenryck et al. IScience of Computer Programming 30 (1998)

83-118

111

For n = 2, this problem Beale is the function

has 900 local minima of local minima defined as f(xi,x2)

when variables

range over [- 10, lo].

For 12= 3 and n = 4, the number

goes up to 2700 and 7 1000.

Schwefell

is the function
i=l

f(Xl7X2yXj)=C3(XI

-Xf)

+ (Xi -

1)2.

Booth

is the function

f(Xi,X2) = (x, + 2x2 - 7)2 + (2x, +x2 - 5)2. Schwefel2 is the function
= 5 F,?, k=l

j-(X1,X2,X3)

where Fk = exp(-O.lkXi)
/tk =

- exp(-O.lkx2)

- AkX3,

exp( -0. lk) - exp( -k).

Powell

is the function

(X,+10*X2)2+5*(X~-X4)2+(X2-2*Xj)4+10*(X,-X4)4.

It is a difficult problem at the solution point. Schwefel3

for interval

methods

in general,

since the Hessian

is singular

is the function
[(XI -xi)+(l

f(x,,x2,.\-3,=~

i=2

-d21.

Rosenbrock j(x,,xz)

is the function = 100(x2 - Xi)2 + (x, - l)? box for this

Interval methods are usually very dependent on the size of the initial problem, although this is not the case for Newton. Ratzl is the function f(Xl,..., Ratz25,
.0x1,...,

x5) = 5 x;/400
i=l

- n 5 COS(Xi/&) + 1.
i=l

Ratz27

and Ratz210

are generated 1

from the function

xm=-z$

(X-Ai)(x-A;)T+Ci

112

P. Van Hentenryck et al. IScience oj Computer Programmhg

30 (19981 83-118

for m equal to 5,7 and 10, respectively. follows: 4 1 8 6 A= 3 2 5 8 6 7 Ratz3 4 1 8 6 7 9 5 1 2 3.6 4 1 8 6 3 2 3 8 6 7 4 1 8 6 7 9 3 1 2 3.6 and C=

The matrix A and the vector c are defined as 0.1 0.2 0.2 0.4 0.4 0.6 0.3 0.7 0.5 0.5 /

is the function
4

f(x~,...,xe) where

= -Cciexp i=l

17

3.5 10 10

0.05

1.7 8 17 0.1

8 14 8 14

P=

0.1312 0.2329 0.2348 0.4047

0.1696 0.4135 0.1451 0.8828

0.5569 0.8307 0.3522 0.8732

0.0124 0.3736 0.2883 0.5743

0.8283 0.1004 0.3047 0.1091

0.5886 0.9991 0.6650 0.0381

Morel

is the function
15 = C
i=l

f(x1,...,~3)

(xl

exp(-0.5x2(tj

-x312)

yi),

where ti = (8 - i)/2 and

1,15 2,14 3,13 4,12 5,ll 6,lO 7,9 8

0.0009 0.0044 0.0175 0.0540 0.1295 0.2420 0.3521 0.3989

P. Van Hentenryck et al. I Science of Computer Programming 30 (1998)

83-118

113

Table 4 Summary

of the experimental 6 6 6 6 7 8 10

results on constrained c 16 16 16 16 18 22 28

optimization Time 2.59 2.64 103.16 51.59 53.71 926.72 4410.26 Splits 10 11 230 226 131 149 7296

Benchmarks h95 h96 h97 h98 hlO0 h106 h113

More2

is the function
f(xl-.

. .3X4) =

fJ i=l

[(XI + t;X2 -

eXp(ti))2

+ (X3 +X4

Sin(ti) -

COS(ti))2]2,

where ti = i/5. 4.3. Constrained optimization

We now turn to constrained optimization problems which are, in general, very difficult to solve. Table 4 summarizes some of our computation results on some of the toughest problems from [8]. We give the number of variables in the initial statement (u), the number of constraints (c), the CPU time, and the number of splits. Note that, for a problem with n variables and m constraints, the system generates a constraint problem involving n + m variables when using the Fritz-John conditions. There are almost no experimental results on constrained optimization methods for interval methods, so that a comparison with related work is difficult. Problems h95, h96, h97, h98 are instantiations in Fig. 6 for the following values of bi: i
1

of the generic problem

described

b(95) 4.97 -1.88 -29.08 -78.02

h100,

b(96) 4.97 -1.88 -69.08 -118.02 hi06

b(97) 32.97 25.12 -29.08 -78.02

b(98) 32.97 25.12 - 124.08 -173.03

2 3 4 Problems

and h113 are depicted in Figs. 7-9.

5. Related work The introduction of a relational form of interval arithmetic in logic programming has been proposed by Cleat-y in [3]. These ideas have been developed and made popular

114

P. Van Hentenrqck

et al. /Science

of Computer

Proyrammirq

30 (1998)

83-118

min 4.3x1 + 31.8x2 + 63.3x3 + 15.8x4 + 68.5x5 + 4.7~~ subject to 0 i XI 5 0.31, 0 5 x2 < 0.046, 0 5 x3 5 0.068 0 < ~4 < 0.042, 0 5 r5 < 0.028, 0 < x6 < 0.0134 17.1x1 + 38.2x2 + 204.2~ + 212.3x4 + 623.4~~ + 1495.526 - 169xlx33580x3x5 - 3810x4x5 - 1850024X1j - 243002526 > bl 17.9x1 + 36.8x2 + 113.9x3 + 169.7x4 + 337.8~ 2450x4~~ - 16600X4x6 - 172OOx5xg 1 b* -273x2 - 70x4 - 819x5 + 26000c4x5 2 b3, 2 b4 + 1385.226 - 139x1x3-

159.9x1 - 311x2 + 587x,, + 391x5 + 2198x6 - 14000x1xs

Fig. 6. The constrained

optimization

problem

for h95,

h96,

h97,

h98.

min (Xl lo)2 + 5(x, -

12)2 + x; + 3(x4 - 11)2 + 10x; + 7x; + x$ - 4X@-

10x6 - 8x7 subject to

-10

x1,~2,x3,x4,x5,x6,xj

<

10

127 - 2x; - 3x; - x3 - 4x7 - 5x5 > 0 282 - 7x1 - 3x2 - 10x; - x4 + x5 _> o 196 - 23al - x; - 6x; + 8x7 2 0 -4x: - x; + 321x2 - 2x; - 5x6 + 11x7 2 0.

Fig. 7. The constrained

optimization

problem

for h100.

by the CLP system BNR-Prolog [29] and generalized to constraint solving over discrete quantities in its successor CLP(BNR) [27,2]. Many other systems (e.g. [ 15,351) have been developed on similar principles. The key idea behind CLP(Intervals) languages is to let users state arbitrary constraints over reals and to narrow down the set of possible values for the variables using various approximations of arc-consistency. In addition, combining the constraint solver with splitting operations allows these systems to isolate narrow regions which may contain solutions to sets of constraints. Traditionally, CLP(Intervals) languages were designed in terms of simple primitive constraints (e.g. add(a,y,~), muZt(x,y,~), cos(x,z), . . .) on which they apply approx-

P. )/LIn Hentenrwk

et al. I Science of Computer Proyranvning 30 (1998)

83-118

115

min 21+52+q subject to

0.0025(~ +x6) < 1 0.0025(-q +q +a,) < 1 0.01(-x5 +zg) < 1 lOOq -z1z6+833.33252x4 -83333.333 < 0 x2x4 -zzx7 -1250~ +1250x5 < 0 Z~IC:, -x3xX -2500x:, +1250000 5 0

Fig. 8. The constrained

optimization

problem

for h106.

min ZT + x; + II * Icy2 - 14 * x1 - 16 * x;? (23- lo)*+ + 4*(x4 - 5)2+(xg - 3)2+2*(ze - 1)2+5 *cc;+ 7* (28- 11)2+ 2 * (29- lo)2+(q0 - 7)2f45 to

subject

105-4*2~-5*x~+3*2~-9*x*>0 -10*x1 +8*22+17*x7-2*x~ > 0 8*~I-2*s2-55~zs+2~x10+12~o -3*(x1 - 2)2- 4 *(x2 - 3)2-2*x; + 7*2q + 120 > 0 -5*x: -8*x2 - (x3- 6)2+2*x4+40 2 0 -0.5*(x1 - 8)2-2*(x2 - 4)2-3*x; +Qj +30 2 0 -xi - 2 *(X2 - 2)2+2*X1 *X2 - 14*Xj+6*zs 2 0 3*x1 -6*x2 - 12 *(x9 -8)2+7*x1O > 0 x:+x; +x1 *x2 -14*x1 - 16*z2 +(x3 - 10)2+ 4*(x4 - 5)2+ (x5- 3)2+2 *(x,j 1)2+5*X;+ 7* (za- 11)'+ 2 *(x9 - lo)2+(x10 - 7)2+45 5 200

Fig. 9. The constrained

optimization

problem

for h113.

imations of arc consistency. Complex constraints are simply rewritten into a set of primitive constraints. The advantage of this methodology is the elegant operational semantics of the language. The inconvenience is that convergence may be slow and the pruning is relatively weak due to the decomposition process.

116

P. Im Hentenr.vck

et ul. IScience

of Computer

Proyramnziny

30 (19981

83-118

Our main goal was to design new approximations use of existing interval the pruning it nicely attempt of interval Newton methods

of arc-consistency

that could make

methods. The main problem was the difficulty in characterizing in a declarative operator way (in order to introduce emerged as an languages) Newton and box consistency

in the above programming to generalize the traditional

to make sure that the bounds

of the interval were locally consistent. Subsequent research made us realize that box consistency is independent of the traditional operator and can be enforced even if the functions are not continuous or differentiable. In addition, the value of applying box consistency on several extensions became clear. On the one hand, box consistency on the Taylor extension generalizes interval methods based on Gauss-Seidel iterations and enables us to capture nicely the Hansen-Seguptas operator. On the other hand, box consistency on the natural and distributed extensions is really orthogonal to the pruning obtained from the Taylor expansion, producing a particularly effective algorithm. It is interesting to note that the idea of using approximations of arc consistency was

also used independently by Hong and Stahl [9], who were also exposed to research on Constraint Logic Programming. Their use of projections is however quite different from ours. The key idea is to work with a set of boxes and to use projections to split a box into several subboxes by isolating all zeros of a projection. This gives an algorithm of a very different nature which cannot easily be characterized as a branch & prune algorithm since constraints are used to branch. Our approach seems to be more effective in practice, since their use of projections may generate many subboxes that may all need to be pruned away later on, implying much redundant work. Our approach postpones the branching until no pruning takes place and generates only subboxes when they are strictly necessary to progress. It is also very interesting to report that, on all benchmarks that we tested, the projection never contained more than two zeros. This seems to indicate that searching for all zeros may not be worthwhile in most cases and that box consistency may be the right trade-off here.

6. Conclusion This paper has presented constraints. application Newton, a constraint programming language over nonlinear

Newton uses intervals to cope with the two fundamental area: numerical accuracy and computational complexity.

problems of this Numerical accu-

racy is dealt with by evaluating every operation on intervals instead of on floating-point numbers. The computational difficulty of the problem is addressed by associating intervals with variables and by using constraints to reduce these intervals. The interval reduction is performed using a novel consistency notion, box consistency, which can be applied to various interval extensions to produce well-known and novel pruning operators. Newton uses box consistency to solve constraint systems as well as unconstrained and constrained optimization problems. Experimental results on numerous benchmarks from numerical analysis, and comparison with other tools, show the effectiveness of Newton.

P. Vm Hmtenr~d

rt al. I Science of Corrprter

Proyrmminy

30 (199s)

83-l 18

117

Acknowledgements The equation-solving algorithm was developed jointly with D. McAllester and

D. Kapur [36]. Special thanks to A. Colmerauer ful comments and suggestions, work was supported

and B. Le Charlier for his invitation Investigator Award

for several helpto CCP95. This with matching

and to P. Codognet Young

in part by the Office of Naval Research

under grant ONR Grant

NOOO14-94-I-1153 and an NSF National funds of Hewlett-Packard.

References
[I] F. Benhamou. D. McAllester, P. Van Hentenryck, CLP(intervals) revisited, in: Proc. Internat. Symp. on Logic Programming (ILPS-94). Ithaca. NY, November 1994, pp. 1244138. [2] F. Benhamou, W. Older, Applying interval arithmetic to real, integer and Boolean constraints. J. Logic Programming, 1997, to appear. [3] J.G. Cleat-y, Logical arithmetic, Future Generation Comput. Systems 2 (1987) 125149. [4] R. Hammer, M. Hocks, M. Kulisch, D. Ratz, Numerical Toolbox for Verified Computing I ~ Basic Numerical Problems, Theory. Algorithms, and PASCAL-XSC Programs, Springer, Heidelberg, 1993. [5] E. Hansen, Global Optimization Using Interval Analysis, Marcel Dekker, New York, 1992. [6] E.R. Hansen, R.I. Greenberg. An interval Newton method, Appl. Math. Comput. 12 (1983) 89-98. [7] E.R. Hansen, S. Sengupta, Bounding solutions of systems of equations using interval analysis, BIT 21 (1981) 203-211. [8] W. Hock, K. Schittkowski. Test Examples for Nonlinear Programming Codes, Lecture Notes in Economics and Mathematical Systems, Springer. Berlin. 198 1. [9] H. Hong, V. Stahl, Safe starting regions by fixed points and tightening, Computing 53 (3-4) (1994) 323-335. [IO] J. More, B. Garbow, K. Hillstrom. Testing unconstrained optimization software, ACM Trans. Math. Software 7 (1) (1981) 17-41. [I I] R.B. Kearfott, Preconditioners for the interval GausssSeidel Method, SIAM J. Numer. Anal. 27 (1990). [12] R.B. Kearfott, A review of preconditioners for the interval Gauss-Seidel method. Interval Comput. 1 1 (1991) 59-85. [13] R.B. Kearfott, A review of techniques in the verified solution of constrained global optimization problems, to appear. [ 141 R. Krawczyk, Newton-Algorithmen zur Bestimmung von Nullstellen mit Fehlerschranken, Computing 4 (1969) 187-201. [15] J.H.M. Lee, M.H. van Emden, Interval computation as deduction in CHIP, J. Logic Programming 16 (34) (1993) 255-276. [16] A.V. Levy, A. Montalvo, The tunnelling algorithm for the global minimization of functions, SIAM J. Sci. Statist. Comput. 6 (1) ( 1985) 15-29. [17] A.K. Mackworth, Consistency in networks of relations, Artificial Intelligence 8 (1) (1977) 99--l 18. [ 181 K. Meintjes, A.P. Morgan, Chemical equilibrium systems as numerical test problems, ACM Trans. Math. Software 16 (1990) 1433151. [19] U. Montanari. Networks of constraints: Fundamental properties and applications to picture processing, Inform. Sci. 7 (2) (1974) 95-132. [20] R.E. Moore. Interval Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1966. [21] R.E. Moore, Methods and Applications of Interval Analysis. SIAM, Philadelphia, 1979. [22] R.E. Moore. S.T. Jones, Safe starting regions for iterative methods, SIAM J. Numer. Anal. 14 ( 1977) 1051-1065. [23] J.J. More, M.Y. Cosnard, Numerical solution of nonlinear equations. ACM Trans. Math. Software 5 (1979) 64-85. [24] A.P. Morgan. Computing all solutions to polynomial systems using homotopy continuation. Appl. Math. Comput. 24 (1987) 115-138.

II8

P. Van Hentenryck

et al. IScience

of Computer

Progmnming

30 (19981 83-118

[25] A.P. Morgan. Solving Polynomial Systems Using Continuation for Scientific and Engineering Problems, Prentice-Hall, Englewood Cliffs, NJ. 1987. [26] A. Neumaier, Interval Methods for Systems of Equations, PHI Series in Computer Science, Cambridge University Press, Cambridge, 1990. [27] W. Older, F. Benhamou, Programming in CLP(BNR), PPCP93, Newport, RI (USA), 1993. [28] W. Older, A. Vellino, Extending prolog with constraint arithmetics on real intervals, in: Canadian Conf. on Computer & Electrical Engineering, Ottawa, 1990. [29] W. Older, A. Vellino, Constraint arithmetic on real intervals, in: Constraint Logic Programming: Selected Research, The MIT Press, Cambridge. MA, 1993. [30] L.B. Rail. Automatic Differentiation: Techniques and Applications, Lectures Notes in Computer Science, Springer. New York, 1981. [31] H. Ratschek, J. Rokne, New Computer Methods for Global Optimization, Ellis Horwood Limited, Chichester. 1988. [32] D. Ratz, Box-splitting strategies for the interval Gauss-Seidel step in a global optimization method. Computing 53 (34) (1994) 337-353. [33] SM. Rump, Verification methods for dense and sparse systems of equations, in: J. Herzberger (Ed.), Topics in Validated Computations. Elsevier, Amsterdam, 1988, pp. 217-231. [34] H. Schwefel, Numerical Optimization of Computer Models, Wiley, New York, 1981. [35] Cl. Sidebottom, W. Havens, Hierarchical arc consistency applied to numeric processing in constraint logic programming, Computat. Intelligence 8 (4) (1992). [36] P. Van Hentenryck, D. McAllister, D. Kapur, Soiving polynomial systems using a branch and prune approach, SIAM J. Numer. Anal. 34 (2) (1997). [37] J. Verschelde, P. Verlinden, R. Cools, Homotopies exploiting Newton polytopes for solving sparse polynomial systems, SIAM J. Numer. Anal, 31 (3) (1994) 915-930.