Constraint Satisfaction Problems

Adrian Groza

Department of Computer Science

Technical University of Cluj-Napoca
12 Nov 2014
Constraint Reasoning Constraint Programming and Backtracking

Outline

1 Constraint Reasoning

2 Constraint Programming and Backtracking

Systematic Search Methods
Improving backtracking efficiency
Constraint Reasoning Constraint Programming and Backtracking

Example
Constraints programming represents one of the
closest approaches computer science has made to
the Holy Grail of programming: the user states the
problem, the computer solves it. Eugene C. Freuder,
Inaugural issue of the Constraints Journal, 1997.
Place numbers 1 through 8 on nodes
1 each number appears exactly once
2 no connected nodes have consecutive numbers
Constraint Reasoning Constraint Programming and Backtracking

Heuristic Search
”To succeed, try first where you are most likely to fail.”
”Deal with hard cases first: they can only get more difficult if you put them off.”

Which nodes are hardest to number?

Guess a value, but be prepared to
backtrack
Which are the least constraining values to
use?
Symmetry means we don’t need to
consider: 8 1
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation I

1 each number appears exactly once

2 no connected nodes have consecutive numbers

We can now eliminate many values for other nodes

Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation II

1 each number appears exactly once

2 no connected nodes have consecutive numbers

We can now eliminate many values for other nodes

Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation III

1 each number appears exactly once

2 no connected nodes have consecutive numbers

By simmetry
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation IV

1 each number appears exactly once

2 no connected nodes have consecutive numbers
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation V

1 each number appears exactly once

2 no connected nodes have consecutive numbers

By symmetry
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation VI

1 each number appears exactly once

2 no connected nodes have consecutive numbers
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation VII

1 each number appears exactly once

2 no connected nodes have consecutive numbers
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation VIII

1 each number appears exactly once

2 no connected nodes have consecutive numbers

And propagate
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation IX

1 each number appears exactly once

2 no connected nodes have consecutive numbers

Guess a value, but be prepared to backtrack

Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation X

1 each number appears exactly once

2 no connected nodes have consecutive numbers

And propagate
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation XI

1 each number appears exactly once

2 no connected nodes have consecutive numbers

More propagation?
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation XII

1 each number appears exactly once

2 no connected nodes have consecutive numbers
Constraint Reasoning Constraint Programming and Backtracking

Inference & constraint propagation XIII

1 each number appears exactly once

2 no connected nodes have consecutive numbers

A solution
Constraint Reasoning Constraint Programming and Backtracking

Constraint programming methodology

1 Model problem
specify in terms of constraints on acceptable solutions:
constraint satisfaction problem
define/choose constraint model: variables, domains,
constraints
2 Solve model
define/choose algorithm
define/choose heuristics
3 Verify and analyze solution
Constraint Reasoning Constraint Programming and Backtracking

Constraints Properties
A logical relation among several unknowns (variables)
May specify partial information: X > 2, ”the circle is inside
the square”
Non-directional: two variables X, Y can be used to infer a
constraint on X given a constraint on Y and vice versa:
X=Y+2
Declarative: specify what relationship must hold without
specifying a computational procedure to enforce that
relationship
Additive: the order of imposition of constraints does not
matter, all that matters, at the end is that the conjunction of
constraints is in effect
Rarely independent: typically constraints in the constraint
store share variables.
Constraint Reasoning Constraint Programming and Backtracking

Constraint satisfaction problem (CSP)

A CSP is defined by
a set of variables: X , Y , Z ,...
a domain of values for each variable: X :: {1, 2}, Y :: {1, 2},
Z :: {1, 2}
a set of constraints between variables: X = Y , X 6= Z ,
Y >Z
A solution is an assignment of a value to each variable that
satisfies the constraints: X = 2, Y = 2, Z = 1
Given a CSP
Determine whether it has a solution or not (satisfiability)
Find any solution
Find all solutions
Find an optimal solution, given some cost function
Constraint Reasoning Constraint Programming and Backtracking

Give me some examples

Puzzle
Map coloring
N-queen problem
Sudoku
Cryptarithmetic problem
Constraint Reasoning Constraint Programming and Backtracking

Constraint Model For Puzzle

variables
v1 , ..., v8
domains
{1, ..., 8}
constraints
|v1 − v2 | = 6 1
|v1 − v3 | = 6 1
..
. |v7 − v8 | =
6 1
alldifferent(v1 , ..., v8 )
Constraint Reasoning Constraint Programming and Backtracking

Constraint Model For Graph Coloring

Given k colors, does there exist

a coloring of the nodes such
that adjacent nodes are
assigned different colors
variables
v1 , v2 , v3 , v4 , v5
domains
{yellow, grey, blue}
constraints
vi 6= vj if vi and vj are
adjacent
Constraint Reasoning Constraint Programming and Backtracking

Constraint Model For N-queens

Place n-queens on an n x n
board so that no pair of queens
attacks each other variables
x1 , x2 , x3 , x4
domains
{1, 2, 3, 4}
constraints
xi 6= xj and
|xi − xj | =
6 |i − j|
a solution
x1 ← 2, x2 ← 4, x3 ← 1, x4 ← 3
Constraint Reasoning Constraint Programming and Backtracking

Modern Art Accident

Constraint Reasoning Constraint Programming and Backtracking

Crypto-arithmetic problem

More Constraints?
S 6= 0, M 6= 0
M = 1, S = 9, O = 0,
N + R > 10
N = E + 1, E in 2..7
Constraints
N in 3..8, D, R, Y in 2..8
[S,E,N,D,M,O,R,Y] in 0..9
R in 3..8
alldifferent([S,E,N,D,M,O,R,Y])
N + R = 1E → E < 7
1000*S + 100*E + 10*N +
D+1000*M + 100*O + 10*R + R = 8 from N + R > 10,
E = 10000*M + 1000*O + N =E +1
100*N + 10*E + Y E in 2..6, N in 3..7
Constraint Reasoning Constraint Programming and Backtracking

Crypto-arithmetic problem

E+D = Y+10*C1
C1+N+R = E+10*C2
C2+E+O = N+10*C3
C3+S+M = 10*M+O
E,N,D,O,R,Y::{0,..,9}
S,M::{1,..,9}
C1,C2,C3::{0,1}
Constraint Reasoning Constraint Programming and Backtracking

Send Most Money

SEND+MOST=MONEY
Money should be maximal
Constraint Reasoning Constraint Programming and Backtracking

Example problem: Job-shop scheduling I

Car assembly - 15 tasks: install axles (front and back), affix all
four wheels (right and left, front and back), tighten nuts for each
wheel, affix hubcaps, and inspect the final assembly, in
maximum 30 minutes.
Variables
X = {AxleF , AxleB , WheelRF , WheelLF , WheelRB , WheelLB , NutsRF ,
NutsLF , NutsRB , NutsLB , CapRF , CapLF , CapRB , CapLB , Inspect}
Precedence constraints: T1 + d1 ≤ T2
Axles have to be in place before the wheels are put on, and it
takes 10 minutes to install an axle,
AxleF + 10 ≤ WheelRF AxleF + 10 ≤ WheelLF
AxleB + 10 ≤ WheelRB AxleB + 10 ≤ WheelLB
Constraint Reasoning Constraint Programming and Backtracking

Example problem: Job-shop scheduling II

For each wheel, we must affix the wheel (which takes 1
minute), then tighten the nuts (2 minutes)
WheelRF + 1 ≤ NutsRF NutsRF + 2 ≤ CapRF
WheelLF + 1 ≤ NutsLF NutsLF + 2 ≤ CapLF
WheelRB + 1 ≤ NutsRB NutsRB + 2 ≤ CapRB
WheelLB + 1 ≤ NutsLB NutsLB + 2 ≤ CapLB
4 our workers to install wheels, but they have to share one tool
that helps put the axle in place.
Disjunctive constraint to say that AxleF and AxleB must not
overlap in time:
(AxleF + 10 ≤ AxleB ) or (AxleB + 10 ≤ AxleF )
The inspection comes last and takes 3 minutes:
∀X , X 6= Inspect, X + dX ≤ Inspect
All assembly done in 30 minute: Di =
{1, 2, 3, ..., 27}
Constraint Reasoning Constraint Programming and Backtracking

Global constraints
Involves an arbitrary number of constraints (but not necessarily all variables).

Alldiff (A1,A2,A3,A4,A5,A6, A7, A8, A9)

Alldiff (B1,B2,B3,B4,B5,B6,B7,B8,B9)
...
Alldiff (A1,B1,C1,D1,E1, F1,G1,H1, I1)
Alldiff (A2,B2,C2,D2,E2, F2,G2,H2, I2)
...
Constraint Reasoning Constraint Programming and Backtracking

Global constraints

At most contraint- resource contraints

Let P1,..., P4 numbers of personnel assigned to each of four tasks.
No more than 10 personnel are assigned in total:
Atmost(10, P1, P2, P3, P4).
Assume D = {3, 4, 5, 6} - detect inconsistency by checking the sum of the
minimum values of the current domains;
Assume D = {2, 3, 4, 5, 6} enforce consistency by deleting the maximum
value of any domain if it is not consistent with the minimum values of the
other domains
Bounds propagation

Example (Airline-scheduling problem)

Two flights, F1 and F2 , for which the planes have capacities 165 and 385,
respectively: D1 = [0, 165] and D2 = [0, 385]. F1 + F2 = 420

D1 = [35, 165] and D2 = [255, 385]

Constraint Reasoning Constraint Programming and Backtracking

Outline

1 Constraint Reasoning

2 Constraint Programming and Backtracking

Systematic Search Methods
Improving backtracking efficiency
Constraint Reasoning Constraint Programming and Backtracking

Systematic Search Methods

Constraints, an Ultimate Anti NP-Hard weapon?

Most problems that the constraint programming concerns
belong to the group that conventional programming techniques
find hardest
Example
Schedule the loading and unloading of
10 ships using only 5 berths.

510 (about 10 million) alternatives in the worst case: 3hours

10 berths and 20 ships: 300 million years
No need to explore all alternatives:
Some berths are too small for some ships
Cannot load two ships in the same berth, at the same time
You don’t need an optimal schedule: break the problem in two
parts - 6 hours
Constraint Reasoning Constraint Programming and Backtracking

Systematic Search Methods

Systematic Search Methods

exploring the solution space

complete and sound
Complete- guarantees finding a solution or proving its
non-existence
Sound- guarantees finding only valid solutions
efficiency issues
Generate and Test (GT)
Backtracking (BT)
 Constraint Reasoning Constraint Programming and Backtracking

Systematic Search Methods

Systematic Search Methods

Problem: X :: {1, 2}, Y :: {1, 2}, Z :: {1, 2}, X = Y , X 6= Z , Y > Z

Backtracking
The basic uninformed algorithm for CSP.
Generate and Test Incrementally extends a partial solution
towards a complete solution
Probably the most general
problem solving method Algorithm:
Algorithm: Build a partial solution: a partial
consistent assignment
1 generate labelling
Extend consistently the partial
2 test satisfaction
solution: one new assigned variable
Drawbacks: blind generator, late each time
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

1 Which variable should be assigned next?

2 In what order should its values be tried?
3 Can we detect inevitable failure early?
4 Can we take advantage of problem structure?
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Minimum remaining values

Choose the variable with the fewest legal values (or most
constrained variable or “fail-first” heuristic)
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Degree heuristic

Choose the variable with the most constraints on remaining

variables
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Least constraining value

Given a variable, choose the least constraining value: the one

that rules out the fewest values in the remaining variables

Combining these heuristics makes 1000 queens feasible

Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Constraint propagation

Arc consistency : X → Y is consistent iff for

every value x of X there is some allowed y
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Constraint propagation

Arc consistency : X → Y is consistent iff for

every value x of X there is some allowed y
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Weaknesses of Backtracking
Thrashing: throws away the reason of the conflict
Example: A, B, C, D, E :: 1..10, A > E: tries all the assignments for B,C,D
before finding that A 6= 1
The chronological backtracking backtracks to the Queen 5 and it finds
another column for this queen (column H). However, it is still impossible to
place the Queen 6
Solution: backjumping (jump to the source of the failure)
The closest queen that can resolve the conflict is Queen 4 because then
there is a chance that column D can be used for Queen 6.
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Weaknesses of Bactkracking
Redundant Work
Unnecessary constraint checks are repeated
Even if the conflicting values of variables they are not remembered
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Weaknesses of Backtracking

Late detection of conflict

Constraint violation is discovered only when the values are known
Example: A, B, C, D, E :: 1..10, A = 3 ∗ E
The fact that A > 2 is discovered when labelling E
Solution: forward checking - forward check of constraints
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Forward Checking
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

Local Search for CSP

Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency

1 Which variable should be assigned next?

2 In what order should its values be tried?
3 Can we detect inevitable failure early?
4 Can we take advantage of problem structure?
Constraint Reasoning Constraint Programming and Backtracking

Improving backtracking efficiency