ECE 566

Constraint Satisfaction Problems

and Techniques

1
 Outline
(1) Constraint Satisfaction problems.
1.0 Definitions, Constraint network representation, features of CSP’s.
(2) CSP problem solving
2.1 Problem reduction
2.2 Consistency, Satisfiability
2.3 Solution Strategies
2.4 Search orders in CSP.
(3) Interval Constraint Satisfaction Problems.
3.1 Interval arithmetic.
3.2 Constraints with Integer, Real and Interval values.
3.3 Interval Constraint networks.
3.4 Waltz Filtering algorithm.

2
What is a Constraint Satisfaction Problem(CSP)?
A CSP consists of :
•Variables - a finite number.
•Domain - finite or infinite domain.
•Constraints - Restricting what values variables can simultaneously
take.
Example: 8 - Queens problem
Variables: The eight queens’ positions.
Domain: The Chessboard squares.
Constraints: No queen attacks the other.
Goal: To find an “Assignment” of variables satisfying all the
constraints.
A CSP is also referred to as a “Consistent labeling problem”

3
Simple Classification based on the types of variables
and constraints:
(1) Discrete CSP’s (logical CSP’s). e.g. : A planning problem
Variables are symbolic variables, Constraints are logical
constraints.
(2)Numerical CSP’s. e.g. : a linear programming problem
Variables are numerical, constraints: functions,equations etc.

4
 Formal definition of a CSP:

CSP is a triple (Z,D,C)

Z: Set of variables.
Z = { x1,x2,….xn}

D: Z -> set of objects of any type.

D maps variables Z to abstract objects in the real world.
For each variable:
Dxi = The set of possible values for variable xi.
(domain of xi )

C: Set of Constraints. (Set of sets of compound labels)

5
 Example of a simple CSP.
B
A C Map Coloring Problem.
Colors given:Red,Blue,Green.

Z = A,B,C. (Variables)

D i = Red,Blue,Green. (For each variable) , i = A,B,C.

Domain of the variables.

C = No two adjacent regions have the same color.

= A != B, A != C, B != C.
(Set of constraints)
6
Why CSP’s?
- Many Design and Engineering problems can be formulated
as CSP’s.
- Algorithms using special features of a CSP
are available for effective solution.
Examples of problems that can be modeled as CSP’s:
• Machine Vision.(Waltz)
• Job-Shop Scheduling.(Prosser,Fox etc.)
• Graph problems.(Haddock)
• Temporal and Spatial Reasoning.(Tsang)
• Electrical Circuit Design.(Sussman,Steele)
• Mechanical Design.(Serrano,Nudel,Navinchandra etc.)
• Diagnostic Reasoning.(Pearl)
• Constraint programming.(Uses CSP concepts)

7
Constraint Network:
It is a graphical representation of a CSP where Nodes
represent variables and Arcs represent constraints.

Binary CSP - Constraint graph.

General CSP - Constraint Hyper-graph.
Constraint Graph: <Green>
B Constraints: A
Adjacent colors
A different.
C
A (Green only)
B (An color) A ≠B A ≠C
C (Blue or Green)

Map Coloring Problem. B B ≠C C

Colors that can be used:
Red , Blue , Green. <Red,Blue,Green> <Blue,Green>

8
Label: variable-value pair assigning a value to the variable.
Notation: L = <x,a>
Example: L1 = < A,Red >
Here variable “A” is assigned the value “Red”.

9
Compound label: Simultaneous assignment of values to a set of
variables.
Notation: CL = (<x1,v1> <x2,v2>. . . . . . . <xn,vn>)
Example: CL1=(<A,Red>,<B,Green>,<C,Blue>).

Here a simultaneous assignment is made to three variables A, B

and C simultaneously.

k- Compound label: A Compound label assigning values to k

variables simultaneously.

Example: CL1 above is a 3- compound label.

10
Constraints: Sets of legal compound labels for sets of variables.
(Restrictions on what values the variables can simultaneously take.)

Notation: Cs - S may be a variable or a set of variables.

Constraints may be defined with equations, with predicates, with
computational procedures (functions), or with sets of legal
compound labels

Example :
A constraint in a discrete CSP.
for A not equal B in a map coloring problem assume that domain
of A = <Red> and that of B = < Red, Green, Blue>.
we can conceptually visualize this as a set of all legal compound
labels
CAB= ( (<A,Red>,<B,Green>), (<A,Red>,<B,Blue>) )
11
Unary Constraints: Constraints on only one variable.
Example: Variable A not equal to “Red”

Binary Constraints: relationship between two variables.

Ex: A is not equal to B

Constraint Expression(CE) of a set of variables S: Set of all the

constraints on the set S and constraints on all subsets of S.
Notation: CE(S)
Example: Let a set S = { A,B,C}, Then CE(S) includes all the
Unary, binary and ternary constraints.
CE(S) = { CA , CB, CC, CAB, CAC, CBC, CABC }

12
• General CSP: A CSP which is not limited to Binary
relationships.
• Lot of research in CSP literature refers to Binary CSP’s.
• (Any CSP can be converted to a Binary CSP)

13
Satisfiability: Shows us that the problem is solvable and a
solution exists.

k-satisfiability:A k-compound label CL satisfies a constraint

expression CE if and only if CL satisfies all constraints in CE.
In the above example if a 3-compound label is
CL= (< A, Red>, < B, Blue>, <C, Green>) then CL 3-satisfies
CE(S) if and only if it satisfies all the unary, binary and
ternary constraint in CE(S).

14
k-satisfiable CSP: For all subsets of k-variables,We can find legal
values for all the k-variables.
(Note that the total number of variables in the CSP may be n > k).
Example: Consider a CSP with three variables A, B and C.
It is 2-satisfiable if all constraints are satisfied for all sets of 2
variables, i.e. Constraints on AB, AC, and BC.

Satisfiable CSP: A CSP which has n-variables is “Satisfiable” if it

is n-satisfiable (Means that a “solution tuple” exists).
Solution tuple for a CSP: A solution tuple for a CSP is a compound
label for all the variables satisfying all the constraints.
Example: for a 3- variable map-coloring problem
solution tuple = (< A, Red>, <B, Blue>, <C, Green>)

15
Types of solution for a CSP:
• One solution.
• All solutions.
• Optimum solutions.

16
Types of solution for a CSP:
• One solution.
• All solutions.
• Optimum solutions.
Features of CSP’s:
CSP’s can be solved as search problems.
• In discrete CSP’s: Size of the search space is finite.
Number of leaves in the search tree, L = | DX1| . | DX2|........ | DXn|
where | DXi| = Size of the domain of each variable.
Number of internal nodes: 1+ 6 | DX1| . | DX2|........ | DXn|
Ordering affects number of nodes.

17
Types of solution for a CSP:
• One solution.
• All solutions.
• Optimum solutions.
Features of CSP’s:
CSP’s can be solved as search problems.
• In discrete CSP’s: Size of the search space is finite.
Number of leaves in the search tree, L = | DX1| . | DX2|........ | DXn|
where | DXi| = Size of the domain of each variable.
Number of internal nodes: 1+ 6 | DX1| . | DX2|........ | DXn|
Ordering affects number of nodes.
• Depth of the tree is fixed.

18
Types of solution for a CSP:
• One solution.
• All solutions.
• Optimum solutions.
Features of CSP’s:
CSP’s can be solved as search problems.
• In discrete CSP’s: Size of the search space is finite.
Number of leaves in the search tree, L = | DX1| . | DX2|........ | DXn|
where | DXi| = Size of the domain of each variable.
Number of internal nodes: 1+ 6 | DX1| . | DX2|........ | DXn|
Ordering affects number of nodes.
• Depth of the tree is fixed.
• Subtrees in the search tree are similar.

19
 Features of CSP’s
Search space
The CSP
<Green>
A= Green
A

A ≠B A ≠C B= Green B= Red
B= Blue

B B ≠C C

C= Green C= Green
<Red,Blue,Green> <Blue,Green>
C= Green

C= Blue C= Blue
C= Blue

The number of leaves = | DA| . | DB|. | DC| = 1 x 3 x2 =6.

Number of internal nodes: 1+ 6 | DX1| . | DX2|........ |DXn|= 1 + 1 +1 x 3 +1 x 3 x 2 =11

Ordering: A,B,C.

20
 Features of CSP’s, Ordering affects the size of search space.
Search space

A= Green

The order with ascending order of domain size

gives the least number of internal nodes.
This is the least for the given problem.
C= Green C= Blue

B= Green B= Red B= Blue B= Green B= Red B= Blue

Number of internal nodes: 1+ 6 | DX1| . | DX2|........ |DXn|= 1 + 1 +1 x 2 +1 x 3 x 2 =10

Ordering: A,C,B.
Note: with this ordering the number of internal nodes is 10, In previous case it was 11. But the
number of leaves does not change.

21
Example of a general search strategy
The simplest algorithm for solving a search problem
is the Chronological Backtracking.

22
Solution using a general search strategy
The simplest algorithm for solving a search problem
is the Chronological Backtracking.
The algorithm works as follows:
(1) Pick a variable from the set of variables.
(2) Assign a value for the variable from the domain to
form a compound label. If the compound label satisfies all
constraints, go to (3), else backtrack and pick another value
If no value can be picked, then problem has no solution.
(3) Check if all the variables are labeled. If yes,Then the
compound label is the solution tuple. If no, then go to (1).
23
 Example of Chronological Back tracking:
<Green>
Unlabeled pick value compound label
A
(1) {A, B, C} A Green {<A,Green>}
No constraints violated , Pick another variable.
A ≠B A ≠C (2) {B,C} B Green {<A,Green>, <B,Green>}
Constraint violated, Backtrack and Pick another value for B

≠C
(3) {B,C} B Red {<A,Green>, <B,Red>}
B B C
No constraints violated , Pick another variable.
<Red,Blue,Green> <Blue,Green> (4) {C} C Green {<A,Green>, <B,Red>, <C,Green>}
Constraint violated, Backtrack and Pick another value for C.
(5) {C} C Blue {<A,Green>, <B,Red>, <C,Blue>}

No constraints violated and no more variables unlabeled.

So the compound label is the solution.

24
Backtrack free search: If a solution can be found using the
backtracking algorithm without any backtracking.

Definition: A search is backtrack free in an ordering if for

every variable that is to be labeled , we can find a value for it
compatible with all the variables assigned before.

In the previous example we saw there were instances where

backtracking occurred.(Steps 2 and 4).

The temporal complexity of the algorithm is O(ean) and the spatial

complexity is O(log n) where n is the number of variables, e is the

number of constraints and a is the domain size.

25
Problems with the simple backtracking algorithm:
Inefficient: More operations than necessary.
Reasons:
(1) Local view of the problem.(Trashing behavior)
In the algorithm, attention is focussed on the present variable and not the
problem as a whole.Consider n variables x1 ….xn. Suppose we have assigned
values for variables x1 to x n-1. Suppose that we assign a certain value to xn
that is incompatible with the first variable x1 ,the algorithm will backtrack to

the previous variable xn-1 and try out all values, and so on till it goes back
and assigns a different value for x1.Many backtracks avoidable - If
knowledge that x1 is causing problem is available.

26
(2) No learning property: Consider variables xi, xj and xn. Assume that when
we assign values,we first assign values to xi ,then to xj and then to xn.If some
value of the domain in xn is incompatible with xj, then this will be detected when
we first assign values to xj, and try to assign values to xn. Assume that due to
some backtrack we have to go back to xi, Now when we assign some value to xj,
we also have to assign values to xn , the algorithm does not remember that a
certain value of xn was incompatible with a value of xj, the algorithm checks for
the particular value in xn that was causing the problem, again.

Both these concepts can be easily understood by the example which follows.

27
 Example:
Consider a simple CSP consisting of the three variables x,y,z whose domains are respectively
Dx={4,5,6} Dy ={1,2} and Dz={2,3}
The Constraints given are x+y+z > 10 x+z > 8 and y+z >4
Consider the trace of a backtracking algorithm:
We assume that the back-tracking
Step x y z Cxyz Cyz Cxz
1 4 1 2 No No No algorithm chooses the variables in
2 1 3 No No No this order i.e x, y and z.
3 2 2 No No No
4 2 3 No Yes No
5 5 1 2 No No No
C xyz denotes constraints on x,y and
6 1 3 No No No z and similarly Cxz and Cyz denote
7 2 2 No No No
8 2 3 No Yes No
constraints on X,Z and Y,Z
9 6 1 2 No No No respectively.
10 1 3 No No No
11 2 2 No No No “No” signifies that constraint is not
12 2 3 Yes Yes Yes
satisfied.
(1) For local view observe that we can only satisfy the constraint x+z > 8 only if x > 5 , But the backtracking
algorithm doesn’t use this fact, backtracks 10 times before it finds the solution.
(2) For the no learning property consider the constraint y + z > 4. Note steps 1 to 4, we discovered that only the
value of y=2 and z =3 will satisfy this property. But when we backtracked to x and changed x to 5, we never use
this fact and carry out the backtracks for y and z (Steps 5 to 7) and discover this again.

28
 Overview of Problem reduction:
• Transform original CSP to new equivalent problem.
(Equivalent problem: A problem with same sets of variables and solution
tuples as the original problem).
• Problem may be easier to solve.
• Easier to recognize insoluble problems.
A preprocessing step, It is rare that a solution can be obtained by only problem
reduction.
Reduction process:
• Modify Domains: Remove redundant values.
Redundant - Don’t appear in “any” solution tuple.
• Tighten Constraints: Fewer compound labels satisfy constraints , i.e. remove
redundant compound labels.

29
Ex:

(1) DX={1,2,3,4,5}. If constraints are x < 4 and x < 5.

Value 4,5 are redundant. Also we note that the Constraint x< 5 is
redundant.

(2) Example for redundant compound label: Consider three variables, A,B
and C. Assume that binary constraints exist between each of these
variables.
Consider a compound label (<A,a>,<C,c>) satisfying constraints on A
and C,
This can be a redundant compound label, If we cannot assign any value
b to B such that (<A,a>,<B,b>) or (<B,b>,<C,c>) can be satisfied.

30
Gains from reduction:
• Reduction in search space.
• Avoiding repeatedly searching futile sub-trees.
• Detect insoluble problems.

How to detect Redundant values?

By using the Consistency techniques.

Minimal Problem:
• No Redundant Values in the domain and No redundant
Compound labels in the constraints.
• Reduction to a Minimal problem is a NP-hard problem.

31
Fundamental Concepts in CSP:
How to detect redundant values and compound labels for problem
reduction ?
• Consistency Techniques
-defined such that presence of redundant values and
redundant compound labels falsifies consistency
requirements. Neither a Sufficient nor a Necessary condition
for a problem to be solvable.

32
 Consistency:
1 - Consistency: Every value in every domain satisfies Unary
Constraints on the variable.

k - Consistency: For all (k-1) compound labels satisfying the

constraints we can add an additional label to form a k-compound
label satisfying all the relevant constraints.

33
 Example for k-consistency:

(R,G,B) A C (R,G,B)
B≠C
C≠ E E (R,G,B)
A≠B C≠D
D≠E
B D
(R,G,B) B≠C (R,G,B)

(R,G,B) are the values that the variables can take, red, green and Blue.
We can show that this problem is 5-consistent, i.e for all 4-compound
labels satisfying all the constraints, we can add another label to get a 5-
compound label satisfying all the constraints.

34
 K-consistency continued.

The table below shows the 4-compound labels(values) for the variables A,B,
C and D, It shows that we can select a value for E in the domain to make a 5-
compound label satisfying all the constraints.Thus it is 5-consistent.
The table shows the situation when A takes the value Blue, we can show that the
same situation applies when A takes Red or Green, and also when we consider the
other variables and assign alternate values to them.

A B C D E
(Variables)
Blue Red Red Green Blue
Blue Red Green Blue Red
Blue Red Red Blue Green
Blue Green Red Blue Green
Blue Green Green Red Blue
Blue Green Green Blue Red

35
 k - Consistency does not imply k-1 Consistency.
B {r,b}
A≠B B≠C 3-consistent CSP but
not 2-consistent.

A C
{r} {r}

In the above example: problem is 3- consistent because for the 2-compound

labels which are consistent (<A,r>,<C,r>), (<A,r>,<B,b>) (<B,b>,<C,r>)
We can add another label(<B,b>, <C,r> and <A,r> respectively for the above 2-
consistent labels.) to get a 3-compound label satisfying the constraints.However it
is not 2-consistent( we can never use <B,r> to get a 2-compound label satisfying
the constraints).(Recall definition of 2-consistency.)

Strong k-Consistency : CSP is 1,2,3........... k-Consistent

36
 Special Consistency definitions for Binary CSP’s:

•Node Consistency:Each possible node value satisfies the constraint

involving only that variable.(Same as 1-Consistency).

•Arc Consistency: Consider an arc(x,y),The arc is arc-consistent if

for every value of x in its domain,We can find a value for y
satisfying the constraint

•A CSP is arc-consistent if all its arcs are arc-consistent.

(Same as 2- Consistency)
Node consistency and Arc consistency can be used to
remove redundant values in the domain.

37
Path Consistency:

X2 X3
X1 . .. ... ..... ..
Xn

For a compound label (<x1,v1>,<xn,vn>) satisfying all constraints on

x1 and xn, we can find values for all the variables (x2, x3…... ,xn-1) in
the path satisfying the binary constraint for each arc in the path.A
CSP is path-consistent if all its paths are path-consistent.
Directional Arc and Path Consistency:
Here the variables are ordered.

38
(Removal of Redundant Values and Constraints)
Node Consistency: removes redundant values.
Arc Consistency: stronger condition than node consistency
removes more redundant values.
Path Consistency: stronger condition than Arc consistency
not only removes redundant values, but also redundant
compound labels.
Why do reduction at all?
The time complexity of reduction is less in many cases than
searching and this offers good benefits in problem solving over
using just searching without problem reduction.
39
 Problem Reduction Cost Savings
Remove redundant values from domains,tighten constraints without
losing solution tuples.
Problem Reduction Equivalent problem
Efforts for complete reduction
Total cost, R+S+Overhead

Computational
problem reduction cost(R)
Cost

Search cost(S)

Amount of problem reduction

40
 Example to show Problem reduction:

<Green> <Green>
A A
Applying Problem reduction

A ≠B A ≠C A ≠B A ≠C

B B ≠C C B B ≠C C

<Red,Blue,Green> <Blue,Green> <Red> <Blue>

Redundant values in variable domain

removed

41
 How search efficiency is increased by problem reduction?
The Original problem Original Search space

<Green>
A= Green
A

A ≠B A ≠C B= Green B= Red
B= Blue

B B ≠C C

C= Green C= Green
<Red,Blue,Green> <Blue,Green>
C= Green

C= Blue C= Blue
C= Blue

42
 The problem after reduction
Reduced Search space
<Green>
A A= Green

A ≠B A ≠C
B= Green B= Red
B= Blue

B B ≠C C

<Red> <Blue>
C= Green C= Green

Redundant values in variable domain C= Green

removed C= Blue
C= Blue C= Blue

The bold lines indicate the new- search space,the

dotted lines are pruned.

The reduced problem here is backtrack free

43
 An Algorithm for Arc-Consistency achievement

An arc ( Vi, Vj) can be made consistent by simply deleting values from the domain of Di (Vi) for which
we cannot find consistent values in the domain of the variable Vj.The following algorithm which does
that will be used as a part of the algorithm for Arc-consistency.
Algorithm REVISE

procedure REVISE( (Vi , Vj ), (Z,D,C) )

DELETE < - false;
for each X in Di do
if there is no such Vj in Dj such that ( X, Vj ) is consistent,
then
delete X from Di ;
DELETE <- true;
endif ;
endfor ;
return DELETE;
end REVISE

We have to note that when we revise the domain of a variable Vi,then each
previously revised arc (Vk, Vi) has to be revised since some of the values in the
domain of Vk may no longer be compatible, since we may have deleted some
values from Vi .
44
 Example: Consider this problem which was discussed earlier, B and C
are initially consistent, But when we make A and C consistent by
<Green>
deleting Green from the domain of C, B and C become inconsistent
A
because we do not have a value for C when B is assigned value Blue.

A ≠B A ≠C Here’s an algorithm called AC-3 which uses algorithm

REVISE( ) to achieve arc-consistency in an entire graph.

B B ≠C C procedure AC-3 (Z,D,C)

Q <- { (Vi ,Vj )∈ arcs(G), i ≠ j };
<Red,Blue,Green> <Blue,Green> while Q not empty
select and delete any arc ( Vk ,Vm ) from Q.
if (REVISE( (Vk ,Vm),(Z,D,C)) then
Q ∪ { (Vi ,Vk) such that (Vi ,Vk)∈ arcs(G), i ≠ k, i ≠ m }
endif;
endwhile;
end AC-3

We note that we use the Revise( (Vi, Vj,) , (Z,D,C) ) procedure defined earlier
in this algorithm.
45
Working of the AC-3 algorithm

A list of arcs Q is maintained, We remove one arc ( Vk, Vm) from the list and
choose a value X in the domain of Vk , and see if there exist values for Vm such
that the constraint on this arc is satisfied, If no such value is available, we
delete X from the Dk. The above steps are carried out by the REVISE algorithm.
When any values in the domain of Vk are changed, we include into the list Q,
arcs that may be possibly affected by this deletion, i.e. All the arcs that are
connected to Vk , (Vi,Vk ) are added.

Various revisions of the above algorithm are available, These are called AC-1,
AC-2 and AC-4. Similarly for Path-Consistency we have PC-1, PC-2, PC-3
and PC-4. [Foundations of Constraint Satisfaction, E.Tsang, 1993]

46
 <Green>
It is easy to verify that using the above
A
algorithm modifies the original problem
to the arc-consistent problem shown
A ≠B A ≠C here. Here we can now solve the problem
without any back-tracking, But this is not
so in general. (Arc-consistency is not a
B B ≠C C strong enough condition to always
eliminate back-tracking)
<Red> <Blue>

47
Comparison between the various algorithms:

Algorithms: (T-Time complexity and S-Space complexity)

(1) Node Consistency algorithm- just removes values not-satisfying
unary constraints. T=O(an), S= O(an).
(2) Arc Consistency algorithms(achieves node consistency also)
AC-1,AC-2, AC-3, AC-4 (improvements in temporal efficiency).
AC-1, T=O(a3ne), S= O(e+an).
AC-3, T=O(a3e), S= O(e+an).
AC-4, T=O(a2e), S= O(a2e). a = domain size.
(3) Path Consistency algorithms: e = number of constraints.
PC-1, T=O(a5n5), S= O(n3a2). n = number of variables.
PC-2, T=O(a5n3), S= O(n3+n2a2).
PC-4, T=O(a3n3), S= O(n3a3).

48
Relationship between Consistency and Satisfiabilty:
• k-Consistency is insufficient to guarantee satisfiability in a CSP
with more than k-variables.
• Path Consistency is not a necessary condition for satisfiability.

Satisfiability Theorem:
A CSP which is 1-satisfiable and strong k-consistent is k-satisfiable
for all k.

Theorem due to Freuder: (Condition for back-track free search).

A Binary CSP is backtrack free if the constraint graph forms a tree
and both node and arc consistency are achieved.

49