Constraint Satisfaction Problems:

A Fast General Survey

CIS 391 - Intro to AI

1
CIS 391 - Intro to AI
2
Constraint satisfaction problems
 What is a CSP?
• Finite set of variables X1, X2, …, Xn
• Nonempty domain of possible values for each variable
D1, D2, … Dd
• Finite set of constraints C1, C2, …, Cm
• Each constraint Ci limits the values that variables can
take, e.g., X1 ≠ X2
 A state is defined as an assignment of values to
some or all variables.
 Consistent assignment: assignment does not violate
the constraints.

CIS 391 - Intro to AI

3
Constraint satisfaction problems
 An assignment is complete when every variable is
assigned a value.
 A solution to a CSP is a complete assignment that
satisfies all constraints.
 Some CSPs require a solution that maximizes an
objective function.
 Applications:
• Scheduling problems
—Job shop scheduling
—Scheduling the Hubble Space Telescope
• Floor planning for VLSI
• Map coloring

CIS 391 - Intro to AI

 Contd…

CIS 391 - Intro to AI

5
 Example

CIS 391 - Intro to AI

6
 Example: Map-Coloring

 Variables: WA, NT, Q, NSW, V, SA, T

 Domains: Di = {red,green,blue}
 Constraints: adjacent regions must have different colors
• e.g., WA ≠ NT
—So (WA,NT) must be in {(red,green),(red,blue),(green,red), …}

CIS 391 - Intro to AI

7
 Example: Map-Coloring

Solutions are complete and consistent assignments,

• e.g., WA = red, NT = green,Q = red,NSW = green,
V = red,SA = blue,T = green

CIS 391 - Intro to AI

8
 CSP as search Problems

CIS 391 - Intro to AI

9
 Type of Variable constraints

CIS 391 - Intro to AI

10
 Constraint graph
 Binary CSP: each constraint relates
NT  Q
two variables N
T
WA

NT  S A
 Constraint graph:
• nodes are variables WA
 SA
• arcs are constraints

 CSP benefits
• Standard representation pattern: variables with
values
• Generic goal and successor functions
• Generic heuristics (no domain specific expertise).

 Graph can be used to simplify search.

— e.g. Tasmania is an independent subproblem.

CIS 391 - Intro to AI

11
 Varieties of CSPs
 Discrete variables
• finite domains:
—n variables, domain size d  O(dn) complete assignments
—e.g., Boolean CSPs, includes Boolean satisfiability (NP-complete)
—Line Drawing Interpretation
• infinite domains:
—integers, strings, etc.
—e.g., job scheduling, variables are start/end days for each job
—need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3

 Continuous variables
• e.g., start/end times for Hubble Space Telescope observations
• linear constraints solvable in polynomial time by linear
programming

CIS 391 - Intro to AI

12
 Varieties of constraints
 Unary constraints involve a single variable,
• e.g., SA ≠ green

 Binary constraints involve pairs of variables,

• e.g., SA ≠ WA

 Higher-order constraints involve 3 or more variables

• e.g., crypt-arithmetic column constraints

 Preference (soft constraints) e.g. red is better than

green can be represented by a cost for each variable
assignment
• Constrained optimization problems.

CIS 391 - Intro to AI

13
 Contd..

CIS 391 - Intro to AI

14
 Example: Cryptarithmetic
X3 X2 X1

 Variables: F T U W R O,
X1 X2 X3
 Domain: {0,1,2,3,4,5,6,7,8,9}
 Constraints: Alldiff (F,T,U,W,R,O)
• O + O = R + 10 · X1
• X1 + W + W = U + 10 · X2
• X2 + T + T = O + 10 · X3
• X3 = F, T ≠ 0, F ≠ 0
CIS 391 - Intro to AI
15
 Real-world CSPs

 Assignment problems
 e.g., who teaches what class

 Timetabling problems
 e.g., which class is offered when and where?

 Transportation scheduling
 Factory scheduling
 Notice that many real-world problems involve real-
valued variables
 CSP as a standard search problem

 A CSP can easily expressed as a standard

search problem, as we have seen
• Initial State: the empty assignment {}.
• Successor function: Assign value to unassigned
variable provided that there is not conflict.
• Goal test: the current assignment is complete.
• Path cost: a constant cost for every step.
 Solution is found at depth n , for n variables
• Hence depth first search can be used

CIS 391 - Intro to AI

17
 Backtracking search
 Note that variable assignments are commutative
• Eg [ step 1: WA = red; step 2: NT = green ]
equivalent to [ step 1: NT = green; step 2: WA = red ]

 Only need to consider assignments to a single variable at

each node
b = d and there are dn leaves

 Depth-first search for CSPs with single-variable assignments

is called backtracking search

 Backtracking search is the basic uninformed algorithm for

CSPs

 Can solve n-queens for n ≈ 25

CIS 391 - Intro to AI

18
•
 Backtracking search (for notes)
function BACKTRACKING-SEARCH(csp) return a solution or failure
return RECURSIVE-BACKTRACKING({} , csp)

function RECURSIVE-BACKTRACKING(assignment, csp) return a solution or failure

if assignment is complete then return assignment
var  SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)
for each value in ORDER-DOMAIN-VALUES(var, assignment, csp) do
if value is consistent with assignment according to CONSTRAINTS[csp]
then
add {var=value} to assignment
result  RECURSIVE-BACTRACKING(assignment, csp)
if result  failure then return result
remove {var=value} from assignment
return failure

CIS 391 - Intro to AI

19
 Backtracking example

CIS 391 - Intro to AI

20
 Backtracking example

CIS 391 - Intro to AI

21
 Backtracking example

CIS 391 - Intro to AI

22
 Backtracking example

CIS 391 - Intro to AI

23
 Improving backtracking efficiency
 General-purpose methods can give huge gains
in speed:
• Which variable should be assigned next?
• In what order should its values be tried?
• Can we detect inevitable failure early?
 Heuristics:
1. Most constrained variable
2. Most constraining variable
3. Least constraining value
4. Forward checking

CIS 391 - Intro to AI

24
Heuristic 1: Most constrained variable
 Most constrained variable:
choose the variable with the fewest legal values

 a.k.a. minimum remaining values (MRV) heuristic

 Select the variable with the smallest number of
remaining
 choices:
 X := argminX2X |values(X, A, C)|

CIS 391 - Intro to AI

25
Heuristic 2: Most constraining variable
 Tie-breaker among most constrained variables
 Most constraining variable:
• choose the variable with the most constraints on remaining
variables

CIS 391 - Intro to AI

26
Heuristic 3: Least constraining value
 Given a variable, choose the least constraining
value:
• the one that rules out the fewest values in the remaining
variables

 Combining these heuristics makes 1000 queens

feasible

CIS 391 - Intro to AI

27
 Forward Checking

CIS 391 - Intro to AI

28
 Heuristic 4: Forward checking
 Idea:
• Keep track of remaining legal values for unassigned
variables
• Terminate search when any variable has no legal values

• Edge & Arc consistency are variants

CIS 391 - Intro to AI

29
Forward checking
 Idea:
• Keep track of remaining legal values for unassigned
variables
• Terminate search when any variable has no legal values

CIS 391 - Intro to AI

30
Forward checking
 Idea:
• Keep track of remaining legal values for unassigned
variables
• Terminate search when any variable has no legal values

CIS 391 - Intro to AI

31
Forward checking
 Idea:
• Keep track of remaining legal values for unassigned variables
• Terminate search when any variable has no legal values

 A Step toward AC-3: The most efficient algorithm

CIS 391 - Intro to AI

32
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

(From B.J. Dorr, U of Md, CMSC 421)

CIS 391 - Intro to AI
33
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

CIS 391 - Intro to AI

34
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }

CIS 391 - Intro to AI

35
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }

CIS 391 - Intro to AI

36
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
Backtrack!!
{ , , , } { ,2,3, }
!
CIS 391 - Intro to AI
37
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

CIS 391 - Intro to AI

38
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}

CIS 391 - Intro to AI

39
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}

CIS 391 - Intro to AI

40
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }

CIS 391 - Intro to AI

41
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }

CIS 391 - Intro to AI

42
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }

CIS 391 - Intro to AI

43
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }

CIS 391 - Intro to AI

44
Constraint propagation
 Forward checking propagates information from assigned to
unassigned variables, but doesn't provide early detection for all
failures:

 NT and SA cannot both be blue!

 Constraint propagation repeatedly enforces constraints locally

CIS 391 - Intro to AI

45
 Arc Consistency

CIS 391 - Intro to AI

46
 Arc consistency

 Simplest form of propagation makes each arc

consistent
 X Y is consistent iff
 for every value x of X there is some allowed y
 Arc consistency

 Simplest form of propagation makes each arc

consistent
 X Y is consistent iff
 for every value x of X there is some allowed y
 Arc consistency

 Simplest form of propagation makes each arc

consistent
 X Y is consistent iff
 for every value x of X there is some allowed y

 If X loses a value, neighbors of X need to be rechecked

 Arc consistency

 Simplest form of propagation makes each arc consistent

 X Y is consistent iff
 for every value x of X there is some allowed y

 If X loses a value, neighbors of X need to be rechecked

 Arc consistency detects failure earlier than forward
checking
 Can be run as a preprocessor or after each assignment
Arc consistency: The General Case
 Simplest form of propagation makes each arc consistent
 X Y is consistent iff
for every value x of X there is some allowed y

CIS 391 - Intro to AI

51
 General version of AC-3
function AC-3(csp) return the CSP, possibly with reduced domains
inputs: csp, a binary csp with variables {X1, X2, …, Xn}
local variables: queue, a queue of arcs initially the arcs in csp

while queue is not empty do

(Xi, Xj)  REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then Typo on your
for each Xk in NEIGHBORS[Xi ] do
version fixed
add (Xk, Xj) to queue

function REMOVE-INCONSISTENT-VALUES(Xi, Xj) return true iff we remove a value

removed  false
for each x in DOMAIN[Xi] do
if no value y in DOMAIN[Xi] allows (x,y) to satisfy the constraints between Xi and
Xj
then delete x from DOMAIN[Xi]; removed  true
return removed
Time complexity: O(n2d3)
CIS 391 - Intro to AI
52
Local search for CSPs
 Hill-climbing, simulated annealing typically work
with "complete" states, i.e., all variables assigned

 To apply to CSPs:
• allow states with unsatisfied constraints
• operators reassign variable values

 Variable selection: randomly select any conflicted

variable

 Value selection by min-conflicts heuristic:

• choose value that violates the fewest constraints
• i.e., hill-climb with h(n) = total number of violated constraints

CIS 391 - Intro to AI

53
Example: n-queens
 States: 4 queens in 4 columns (44 = 256 states)
 Actions: move queen in column
 Goal test: no attacks
 Evaluation: h(n) = number of attacks

 Given random initial state, AC-3 can solve n-

queens in almost constant time for arbitrary n with
high probability (e.g., n = 10,000,000)

CIS 391 - Intro to AI

54
Real-world CSPs
 Assignment problems
• e.g., who teaches what class
 Timetabling problems
• e.g., which class is offered when and where?
 Transportation scheduling
 Factory scheduling

 Notice that many real-world problems involve

real-valued variables

CIS 391 - Intro to AI

55