Constraint-Based 3D-Object Layout using a Genetic Algorithm

Stéphane Sanchez Olivier Le Roux Hervé Luga Véronique Gaildrat

Department of Computer Science IRIT

UPS, 118 route de Narbonne, 31062 Toulouse Cedex 4, France
Phone. (33) 05 61 55 83 29 Fax. (33) 05 61 55 62 58
E-mail: { sanchez, leroux, luga, gaildrat }@irit.fr

Abstract
Object layout in a large 3D-scene or in a complex virtual environment is a time-consuming and tedious
task. In order to assist the user in this task, we present a general-purposed constraint-based system. The
underlying constraint solver is built on a genetic algorithm. It is able to process a complex set of
constraints, including geometric and pseudo-physics ones. Moreover, universal quantifiers and Boolean
combination of constraints are allowed to make the layout description easier. Finally, to get realistic
scenes, objects can take any orientations, not only isothetic ones.

Keywords: non-isothetic 3D-object layout, genetic algorithm, constraint solving.

1 Introduction Unfortunately, using such family of algorithms

leads to difficulties to obtain realistic scenes
Realistic object layout in a large 3D-scene is because of exhaustive enumeration; hard
currently a time-consuming task. In this paper we constraints are usually added to get a satisfying
study and experiment a genetic algorithm as core solution. Another approach is to consider the
of a constraint-based generation system. Once problem as a numeric optimization one. The
validated, this tool will be included in our formulation can be linear [Vries et al. 00] or non-
declarative modelling platform DEM²ONS 1 linear [Donikian et al. 93]. However, strict
[Kwaiter et al. 97]. restrictions (especially on NLP4) must be set on
the constraint system to ensure satisfying results.
By now, most of the placement tools take into A third approach consists in using metaheuristics
account only isothetic 2 objects. This such as local search (simulated annealing, tabu
simplification allows minimizing size of the search) or evolutionist strategies [Masui 92]
search space and thus computation time. [Vassilas et al. 02].
Excepting some systems (e.g. [Xu et al. 02]) that
choose to drastically simplify the allowed In the following we present some reasons to
constraints, no one is interactive. In fact, the choose genetic algorithm for our problem:
classical problem is known to be NP-complete - huge search space (each object has seven
(i.e. there are no complete tractable algorithms to degrees of freedom);
solve it). In complex cases, only stochastic - some constraints, in their algebraic
heuristics give suitable results. However, in a formulation, are not differentiable (e.g. the
classical 3D environment, the problem is widely non-overlapping constraint);
under-constrained and some systems generate - object layout must be as realistic as possible: a
interesting results using standard CSP3 algorithms stochastic process is well suited;
[Charman 95] [Kwaiter et al. 97] [Bonnefoi et al. - universal quantifiers and Boolean combination
99] [Ruchaud et al. 02]. of constraints must be used to improve
expressiveness in description of layout
1
problem;
DEM²ONS = Declarative Multimodal ModeliNg System
2
isothetic = parallel and/or orthogonal to world coordinate
system
3 4
CSP = Constraint Satisfaction Problem NLP = Non Linear Programming
- some descriptions involve hard problems 2.2 Solver requirements
including cyclic constraint graph;
- finally, our object layout system has to manage To be used as a valuable generation tool the
optimization problems (like “add two more solver must have some features.
chairs around the table”). It must solve topological constraints (“the plate is
on the table”), metrics constraints (“the chair is at
This paper is organized as follows. Part 2 one meter of the desk”), angular constraints
introduces our approach. We describe the inner (“dispatch uniformly three chairs around the
architecture of the system: objects, domains and table”), orientation constraints (“the sofa is in
constraint modeling. The genetic algorithm is front of the TV”), and size constraints (“the table
also presented. In part 3, we give some is 1.80 meter long”).
experimental results on classical problems. Part 4 The resulting scene must be realistic: 3D-objects
discusses on advantages and drawbacks of our with any orientations, no overlapping and respect
approach. We also compare our system with of gravity constraints.
related ones. Finally part 5 concludes this study. Constraint's hierarchy is used to express user’s
preferences and to solve over-constrained
2 Our constraint-based system problems.
The solver must be able to treat cyclic constraint
2.1 A brief overview of Genetic Algorithm graph.
Disjunctive constraints (“set the chair against A
In 1975, J. H. Holland introduced genetic wall”, “set the waste basket to the right OR the
algorithms as a new technique that can be applied left of the desk”) and negative ones (“the
to solve a wide variety of problems [Holland 75] wardrobe is NOT in front of the window”) must
and D. Goldberg presented first real applications be managed.
that made them popular in 1989 [Goldberg 89].
J. R. Koza [Koza 92] has given a good definition 2.3 Problem modeling
of genetic algorithm:
"The genetic algorithm is a highly parallel In this section, we present our problem modeling.
technique that transforms a population of This stage is decisive and determines the
individual objects, each with an associated fitness performance of the system.
value, into a new generation of the population
using the Darwinian principle of reproduction In our evolutionist approach:
and survival of the fittest and naturally occurring - placement space (i.e. domains) and objects
genetic operations such as crossover already in the scene represent the environment
(recombination) and mutation. Each individual in of the population of genetic algorithm;
the population represents a possible solution to a - objects to set represent genetic components of
given problem." individuals;
- combination of constraints represents the
pressure to apply on individuals in order to
make the genetic algorithm converge toward a
suitable solution.

2.3.1 Domains and objects

The placement space, in which new objects are
added, is a limited three-dimensional space,
oriented around the height axis.

To be realistic, objects in the scene are identified

Figure 1: Main loop of a genetic algorithm with a hierarchical set of bounding boxes.
Usually, when an object is put on shelves, it must
be on an inner shelf and not on top of them.
Each bounding box is defined using three kinds Where object target is the object the user intends to
of parameters: put in the scene relatively to object landmark in
- position: location of the box in the world relation with spatial preposition S_P.
coordinate space. A variable of position is
made up of three one-dimensional values. The user can give any geometric constraint either
- orientation: an angle in radians. Usually the in a positive way or in a negative one (operator
three Euler angles (roll, pitch, head or yaw) are NOT).
used. To minimize computation time, we Four kinds of geometric constraints are
choose to maintain only head (rotation around considered:
the height axis). This choice is justified by - zone: “the chair is on the LEFT of the table”,
usual object orientation in real world. “put the vase ON the shelf”;
- size: a one-dimensional parameter along an - orientation: “the sofa is FACING the TV”.
axis. Three size parameters are used to define - distance: “the monitor is AT 50 CM of the
the dimensions of a box. desk chair”;
- size: “the table is 2 METERS LONG”.
Remark: to lower processing time and to achieve And each one is defined by a set of basic
correct convergence of genetic algorithm, the mathematic and/or geometric properties.
objects to set in the scene are defined only by For example, a zone constraint is described as a
their global bounding box. Consequently, if the volume made of a control polygon as base and a
user wants to set a table in the scene and a stool set of allowed heights:
under it, he must first set the table in place and
then set the stool relatively to hierarchical
structure of the table. Nevertheless, ten chairs and
associated plates and forks can be set around the
table at one time.

2.3.2 The constraints

The constraints used, mainly inspired of the ones
proposed by P. Charman [Charman 95], are low-
level ones: they each describe a local layout
problem that the solver can easily process in
order to generate a suitable solution. However,
these constraints can be combined to create high-
level properties. Figure 2: Control polygon of zone constraint

We consider two kinds of constraints: the first

ones, we name physical constraints, ensure
coherence and realism of the generated scene.
The others, called geometric constraints, will be
combined to express the problem.
The physical constraints consist in two main
constraints:
- non-overlapping avoids collisions between
objects ;
- gravity invalidates every object that is not on
the ground or balanced on another valid object.
The geometric constraints aim to describe each a
simple and elementary spatial problem. They
usually use the Vandeloise's terminology: (with
Figure 3: Set of allowed heights of zone
exception of size constraint as seen below):
constraint
object target S_P (parameters) object landmark
2.4 Solving the problem - “the plate on the table facing one of the two
chairs but not on the vase” (cf. Figure 5):
The layout object problem to solve consists of C1 = Plate ZONE(on) table
two associated parts: C2 = Plate ORIENTATION(pi) chair#1
- a genome that represents the objects to set in C3 = Plate ZONE(in_front) chair#1
the scene, C4 = Plate ORIENTATION(pi) chair#2
- associated with a combination of constraints. C5 = Plate ZONE(in_front) chair#2
C6 = Plate NOT ZONE(ON) vase
2.4.1 The genome
2.4.3 Evaluation of individuals
An individual of the genetic population represents
a potential solution to the problem. The This step is critical to the solving process. Indeed,
evaluation of its genome (using a fitness convergence of the genetic algorithm towards a
function) allows estimating the quality of the valuable solution depends only on the quality of
adaptation of the individual to the current the evaluation process: the fitness function
problem. applied to individuals must express, the most
The genome consists of at least one chromosome. exactly, how they satisfy the set of constraints
Each chromosome corresponds to a set of they are associated with. Besides, results of
equivalent objects (i.e. objects related to the same fitness functions associated with the different
size constraint) and it is associated to one low-level geometric constraints must be
combination of constraints. Each gene of the homogeneous in order to combine them (using
chromosome represents a unique object to set in Boolean tree) without introducing an unwanted
the scene. hierarchy during the evaluation process.

2.4.2 Setting the object layout problem

As said, each problem can be described with a
combination of geometric constraints. The
combination is made up of a Boolean tree with
constraints as leaves and using the following
node operators: AND, OR, XOR and NOT.
Using a Boolean tree with these four operators
allows many combinations of heterogeneous
constraints and a really natural way of expressing
problems. We could note that a tree structure
facilitates introduction of a priority factor, at leaf
level, in order to create a constraints hierarchy.

- “sofa facing TV at 2 meters of the TV” (cf. Figure 5: Constraint tree #2

Figure 4):
C1 = Sofa ZONE(in_front) TV
The evaluation of both physical constraints (non-
C2 = Sofa ORIENTATION(pi) TV
overlapping and gravity) validates, or invalidates,
C3 = Sofa DISTANCE (2.0)TV
a gene of the individual (i.e. an object to set in the
scene). Each fitness function, matching with one
geometric constraint, is applied to each valid gene
and uses a method that returns a percentage of
satisfaction (in order to maintain homogeneity).
For example, in case of positive zone constraint
the evaluating process will be:
Figure 4: Constraint tree #1
Figure 6: Fitness of zone constraint Figure 9: Fitness of zone constraint

Figure 10: Fitness of zone constraint

Figure 7: Fitness of zone constraint

The results of fitness functions are processed

according to the node operators and to the priority
factors so as to obtain the satisfaction value of
each gene. For example, the fitness value fg of the
gene corresponding to the plate in the following
problem, "plate on the table facing one of the two
chairs but not on the vase", will be:

f g = AND ( prioC1 × C1,

OR( AND ( prioC 2 × C 2,
prioC 3 × C 3 ),
AND ( prioC 4 × C 4,
prioC 5 × C 5 ) ),
Figure 8: Fitness of zone constraint prioC 6 × NOT (C 6) )

With Ci the fitness value of gene g for

constraint Ci,
prioCi the priority factor of constraint Ci
Then, the fitness value fch of each chromosome of The reference computer is a Pentium III 1.2
the individual will be a simple average value: GHz. The solver is compiled with Java SDK 1.4.
All the computation times given for the following

∑f
problems are average values of 10 consecutive
g runs.
nbGenes
f ch = - "20 boxes on the table but not on a box already
nbGenes on the table":

At last, the final fitness value F is obtained with

an average of fitness values of chromosome or, if
we need to add a hierarchy between simultaneous
problems, with a weighted sum:

∑ prio × f
nbChromosome s
ch ch

F=
∑ prio
nbchromosome s
ch

3 Experimental results
In order to evaluate how the solver computes
different kinds of problem, we use the following
3D-scene as reference: Figure 12: Computation time = 52.1 s

- "20 books correctly arranged on first shelf of

shelves ":

Figure 11: The reference scene

For information, the solver has built this scene,

composed of a wardrobe, a table with a box on it, Figure 13: Computation time = 58.4 s
a stool, a sofa and shelves, in about 18 seconds.
- "8 chairs around the table, more or less facing 4 Discussion
it":
Our set of constraints allows describing many
basic problems {on, against, near, facing, above,
in, parallel to, etc.}) but also more complex ones
using Boolean trees of constraints.

Furthermore, in our tests:

- All the under-constrained problems have been
solved.
- For each of the over-constrained problems, the
solver has generated a suitable solution.
- The only problems that are not solved (no
convergence of the genetic algorithm) are the
ones with no possible solution (for example,
"the chair in front of and behind the table").
- All the cyclic problems have been successfully
processed.
- Negative problems are solved just as positive
ones: the solver makes no difference.
Figure 14: Computation time = 12.7 s - Constraints hierarchies have been correctly
solved.
- "4 objects around the box on the table,
Nevertheless, many improvements are still
cyclically oriented at 90° from the previous
possible:
one":
- The computation times are erratic from one
problem to another. They do not seem to be
directly correlated with number of objects
and/or number of constraints. A study on
convergence of the algorithm seems necessary.
- Evaluation of individuals using a weighted
sum does not allow the solver to generate a
diversified set of solutions. The use of multi-
criteria methods, as MOGA [Fonseca et al.
95], might correct, at least partially, this
drawback.

Currently, there is no standard benchmark on

non-isothetic 3D-object layout problems.
Consequently, results produced by our system
cannot be easily compared with related works. To
give some useful informations, the next table
provides a qualitative comparison of some similar
projects.
Figure 15: Computation time = 2.1 s
 DEM²ONS MultiFormes MultiFormes MultiCAD DEM²ONS
Project EAAS
ORANOS 4 5 GA GA
Year 1993-1995 1997-1998 1999 2002 2002 2000-2003
[Vassilas et al.
Main reference [Charman 95] [Kwaiter 97] [Bonnefoi 99] [Ruchaud 02]
02]
This paper
Approach constructive constructive constructive constructive iterative iterative
Is result NO NO NO NO
YES YES
guaranteed? (discretization) (discretization) (stochastic) (stochastic)
Is algorithm
YES YES YES YES NO NO
sound?
Objects / rectangles 3D bounding 3D bounding 3D vertices 3D bounding 3D bounding
degrees of boxes boxes boxes boxes
freedom 4 9 6 3 6 7
any around the
Orientation 0° or 90° isothetic only isothetic only any isothetic only
height axis
Boolean A tree using
combination of AND AND AND AND AND AND, OR, XOR,
constraints and NOT
Hierarchical
NO YES NO NO NO YES
constraints
Relaxation if
over- NO Partial NO NO YES, implicit YES, implicit
constrained?
S-CSP DHN-CSP CSP CSP GA GA
FC or RFL dynamic and incremental BT or FC
+ hierarchical FC or PL + About 20 About 300
CBJ numerical (multi- dynamic individuals individuals
(optional) pre-filtering process) heuristic
+ + + (FFP based) constraint domain pre-
filtering incremental BJ programming filtering
Algorithm (to generate
AC-SG FC or RFL + +
(derived + AC5-filtering the initial classical genetic
from AC3) dynamic + population) algorithm
+ heuristic dynamic + (see section 2)
dynamic (FFP based) heuristic classical
heuristic (FFP based) genetic
(FFP based) algorithm

CSP = Constraint Satisfaction Problem; S = Spatial; DHN=Dynamic Hierarchical Numerical;

BT = Backtracking; FC = Forward Checking; RFL = Real-Full-Lookahead; PL = Partial Lookahead;
BJ = Backjumping; CBJ = Conflict directed Backjumping; FFP = Fail First Principle;
AC = Arc Consistency.
Table 1: Comparative synthesis of our study with related works

Here are summarized advantages and drawbacks - very complex problems can be formulated: in
of our system. our system, universal quantifier and Boolean
combination of constraints are allowed;
Advantages: - fuzzy description are naturally taken into
- the system is general-purpose, easy to account;
understand and easy to extend; - when the system encounters an over-
- maximum computation time allowed can be constrained problem, a suitable solution is
defined by the user; proposed according to the weight of
- unlike traditional constructive approaches (e.g. constraints;
CSP), a partial solution can be visualized and - the system can both deal with satisfaction and
selected at any moment; optimization problems.
- as it is a stochastic process, final scene is very
realistic;
Drawbacks: section 3) are promising. However, particular
- as all metaheuristics techniques, simulation benchmarks are needed to allow quantitative
parameters (population size, mutation rate, comparisons between different approaches.
etc.) are problem-dependant; several tries can In the future, we plan to include this tool in our
be necessary to get good results; DEM²ONS [Kwaiter et al. 97] declarative
- global convergence is not guaranteed; platform, to compare it more precisely with
- inconsistency of a description cannot be others existing constraint solvers and others
proved; metaheuristics (e.g. local search).
- the algorithm is not sound: scenes which are However, to get an easy-to-use object-layout
not solution can sometimes be generated. software, the description of the scene can be
made easier by the use of semantic and functional
5 Conclusion features associated with the objects. So, the user
can obtain a complex and realistic scene without
In this paper we have presented a constraint- giving all details.
based system for non-isothetic 3D-object layout
built on a genetic algorithm. The first results (see

Figure 16: Example of a realistic 3D-scene (88 objects) generated with our tool (ray tracing rendering)
References [Koza 92] J.R. Koza. Genetic Programming. On
the Programming of Computers by Means of
Natural Selection. The Mit Press, 1992.
[Bonnefoi et al. 99] P-F Bonnefoi, D. Plemenos.
Object Oriented Constraint Satisfaction for [Kwaiter et al. 97] G. Kwaiter, V. Gaildrat, R.
Hierarchical Declarative Scene Modeling. Caubet. DEM²ONS: A High Level Declarative
WSCG'99, Plzen, Czech Republic, February 2- Modeller for 3D Graphics Applications. In
12, 1999. Proceedings of the International Conference
on Imaging Science Systems and Technology,
[Charman 93] P. Charman. Solving Space CISST’97, pp 149-154, Las Vegas, June 30-
Planning Problems using Constraint July 3, 1997.
Technology. Technical report CS 57/93,
Institute of Cybernetics - Estonian Academy [Masui 92] T. Masui. Graphic Object Layout
of Sciences, 1993. with Interactive Genetic Algorithms.
Proceedings of the 1992 IEEE Workshop on
[Charman 95] P. Charman. Gestion de contraintes Visual Languages, pp 74-80, September 1992.
géométriques pour l'aide à l'aménagement
spatial. Thèse de doctorat. Ecole Nationale des [Ruchaud et al. 02] W. Ruchaud, D. Plemenos.
Ponts et Chaussées. Novembre 1995. MultiFormes: a declarative modeller as a 3D
scene sketching tool. International Conference
[Donikian et al. 93] S. Donikian, G. Hégron. A ICCVG'2002, Zakopane (Poland), September
Declarative Design Method for 3D Scene 25-29, 2002.
Sketch Modeling. In Eurographics'93, Vol.12
(33), pp 223-236, 1993. [Vries et al. 00] B. de Vries, A.J. Jessurun,
R.H.M.C Kelleners. Using 3D geometric
[Fonseca et al. 95] C.M. Fonseca, P.J. Fleming. constraints in architectural design support
An Overview of Evolutionary Algorithms in systems. WSCG'00, 2000.
Multiobjective Optimization. Evolutionary
Computation 3, 1 (Spring), 1-16. 1995. [Vassilas et al. 02] N. Vassilas, G. Miaoulis, D.
Chronopoulos, E. Konstantinidis, I. Ravani, D.
[Goldberg 89] D.E. Goldberg. Genetic algorithms Makris, D. Plemenos. MultiCAD-GA: A
for search, optimization, and machine System for the Design of 3D Forms Based on
learning. Addison-Wesley, Reading, Genetic Algorithms and Human Evaluation.
Massachusetts, 1989. SETN 2002, pp 203-214, 2002.
[Holland 92] J.H. Holland. Adaptation in Natural [Xu et al. 02] K. Xu, J. Stewart, E. Fiume.
and Artificial Systems. An Introductory Constraint-based Automatic Placement for
Analysis with Applications to Biology, Scene Composition, Graphics Interface '02,
Control and Artificial Intelligence (Second May 2002.
ed.). MIT Press, Cambridge, Massachusetts,
1992.