Constraint Satisfaction Problems

22MDC45 Artificial Intelligence

Unit 3 -1

1
CSP
∙ Use of Factored representation for each state: a
set of variables, each of which has a value.
∙ A problem is solved when each variable has a
value that satisfies all the constraints on the
variable.
∙ A problem described this way is called a
constraint satisfaction problem, or CSP.
∙ CSPsearch algorithms take advantage of the
structure of states and use general-purpose
rather than problem-specific heuristics to enable
the solution of complex problems.

2
Constraint satisfaction problems
∙ What is a CSP?
• Finite set of variables V1, V2, …, Vn
• Nonempty domain of possible values for each variable
DV1, DV2, … DVn
• Finite set of constraints C1, C2, …, Cm
• Each constraint Ci limits the values that variables can
take, e.g., V1 ≠ V2
∙ A state is an assignment of values to some or all
variables.
∙ Consistent assignment: assignment does not
violate the constraints.

3
Constraint satisfaction problems
∙ An assignment is complete when every variable
has a value.
∙ A solution to a CSP is a complete assignment
that satisfies all constraints.
∙ Some CSPs require a solution that maximizes an
objective function.
∙ Applications:
• Scheduling the Hubble Space Telescope,
• Floor planning for VLSI,
• Map coloring,
• Cryptography

4
Example: Map-Coloring

∙ Variables: WA, NT, Q, NSW, V, SA, T

∙ Domains: Di = {red,green,blue}
∙ Constraints: adjacent regions must have different colors
• e.g., WA ≠ NT
— So (WA,NT) must be in {(red,green),(red,blue),(green,red), …}

5
∙ X ={WA,NT,Q,NSW,V,SA,T}.
∙ C = {SA≠WA,SA≠NT,SA≠Q,SA≠NSW,SA≠V,
WA≠NT,NT≠Q,Q≠NSW,NSW≠V}.
∙ SA≠ WA is a shortcut for (SA,WA),SA≠ WA, where
SA≠WA can be fully enumerated in turn as

{(red,green),(red,blue),(green,red),(green,blue),(bl
ue,red),(blue,green)}
• There are many possible solutions to this
problem, such as {WA=red,NT =green,Q=red,NSW
=green,V =red,SA=blue,T =red }.

6
Example: Map-Coloring

Solutions are complete and consistent assignments,

• e.g., WA = red, NT = green,Q = red,NSW = green,
V = red,SA = blue,T = green

7
Constraint graph
∙ Binary CSP: each constraint relates
two variables

∙ Constraint graph:
• nodes are variables
• arcs are constraints

∙ CSP benefits
• Standard representation pattern
• Generic goal and successor functions
• Generic heuristics (no domain specific expertise).

∙ Graph can be used to simplify search.

— e.g. Tasmania is an independent subproblem.

8
Why formulate a problem as a CSP?
∙ CSPs yield a natural representation for a wide
variety of problems; if you already have a
CSP-solving system, it is often easier to solve a
problem using it than to design a custom solution
using another search technique
∙ CSP solvers can be faster than state-space
searchers because the CSP solver can quickly
eliminate large swatches of the search space.
∙ Eg: 3^5 =243 assignments for the five
neighboring variables; with constraint
propagation we never have to consider blue as a
value, so we have only 2^5 =32 assignments to
look at, a reduction of 87%.
9
Varieties of CSPs
∙ Discrete variables
• finite domains:
— n variables, domain size d O(dn) complete assignments
— e.g., Boolean CSPs, includes Boolean satisfiability (NP-complete)
• infinite domains:
— integers, strings, etc.
— e.g., job scheduling, variables are start/end days for each job
— need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3

∙ Continuous variables
• e.g., start/end times for Hubble Space Telescope observations
• linear constraints solvable in polynomial time by linear
programming

10
 Varieties of constraints
∙ Unary constraints involve a single variable,
• e.g., SA ≠ green
∙ Binary constraints involve pairs of variables,
• e.g., SA ≠ WA
∙ Higher-order constraints involve 3 or more variables
• e.g., asserting that the value of Y is between X and Z, with the
ternary constraint Between(X,Y,Z).
∙ Global Constriants involve an arbitrary number of
variables
• e.g., AllDiff in Sudoku.
∙ Preference (soft constraints) e.g. red is better than
green can be represented by a cost for each variable
assignment => Constrained optimization problems.

11
Example: Cryptarithmetic

∙ Variables: FTUWRO
X1 X 2 X3
∙ Domain: {0,1,2,3,4,5,6,7,8,9}
∙ Constraints: Alldiff (F,T,U,W,R,O)
• O + O = R + 10 · X1
• X1 + W + W = U + 10 · X2
• X2 + T + T = O + 10 · X3
• X3 = F, T ≠ 0, F ≠ 0

12
CSP as a standard search problem

∙ A CSP can easily be expressed as a standard

search problem.
• Initial State: the empty assignment {}.
• Operators: Assign value to unassigned variable provided
that there is no conflict.
• Goal test: assignment consistent and complete.
• Path cost: constant cost for every step.
• Solution is found at depth n, for n variables
• Hence depth first search can be used

13
Backtracking search
∙ Variable assignments are commutative,
• Eg [ WA = red then NT = green ]
equivalent to [ NT = green then WA = red ]

∙ Only need to consider assignments to a single variable at

each node
b = d and there are dn leaves

∙ Depth-first search for CSPs with single-variable

assignments is called backtracking search

∙ Backtracking search basic uninformed algorithm for CSPs

∙ Can solve n-queens for n ≈ 25

14
 Backtracking search
function BACKTRACKING-SEARCH(csp) % returns a solution or failure
return RECURSIVE-BACKTRACKING({} , csp)

function RECURSIVE-BACKTRACKING(assignment, csp) % returns a solution or failure

if assignment is complete then return assignment
var ← SELECT-UNASSIGNED-VARIABLE(VARIABLES[csp],assignment,csp)
for each value in ORDER-DOMAIN-VALUES(var, assignment, csp) do
if value is consistent with assignment according to CONSTRAINTS[csp] then
add {var=value} to assignment
result ← RECURSIVE-BACKTRACKING(assignment, csp)
if result ≠ failure then return result
remove {var=value} from assignment
return failure

15
Backtracking example

16
Backtracking example

17
Backtracking example

18
Backtracking example

19
Improving backtracking efficiency

General-purpose methods can give huge speed gains:

• Which variable should be assigned next?

• In what order should its values be tried?
• Can we detect inevitable failure early?

20
Choice: Most constrained variable

∙ Most constrained variable:

choose the variable with the fewest legal values

∙ a.k.a. minimum remaining values (MRV) heuristic.

“fail-first” heuristic

21
Most constraining variable

∙ Tie-breaker among most constrained variables:

degree heuristic
∙ Most constraining variable:
• choose the variable with the most constraints on remaining
variables

22
Order: Least constraining value
∙ Given a variable, choose the least constraining
value:
• the one that rules out the fewest values in the remaining
variables

∙ Combining these heuristics makes 1000 queens

feasible

23
Forward Checking
∙ Idea:
• Keep track of remaining legal values for unassigned
variables
• Terminate search when any variable has no legal values

24
Forward checking
∙ Idea:
• Keep track of remaining legal values for unassigned
variables
• Terminate search when any variable has no legal values

25
Forward checking
∙ Idea:
• Keep track of remaining legal values for unassigned
variables
• Terminate search when any variable has no legal values

26
Forward checking
∙ Idea:
• Keep track of remaining legal values for unassigned
variables
• Terminate search when any variable has no legal values

∙ No more value for SA: backtrack

27
 Constraint propagation

∙ Forward checking propagates information from

assigned to unassigned variables, but doesn't provide
early detection for all failures

∙ NT and SA cannot both be blue!

∙ Constraint propagation repeatedly enforces
constraints locally
 Arc consistency

∙ Simplest form of propagation makes each pair of

variables consistent:
• X Y is consistent iff for every value of X there is some allowed
value of Y

Consist
ent!
 Arc consistency

∙ Simplest form of propagation makes each pair of

variables consistent:
• X Y is consistent iff for every value of X there is some allowed
value of Y
 Arc consistency

∙ Simplest form of propagation makes each pair of

variables consistent:
• X Y is consistent iff for every value of X there is some allowed
value of Y
• When checking X Y, throw out any values of X for which there isn’t
an allowed value of Y

∙ If X loses a value, all pairs Z X need to be rechecked

 Arc consistency

∙ Simplest form of propagation makes each pair of

variables consistent:
• X Y is consistent iff for every value of X there is some allowed
value of Y
• When checking X Y, throw out any values of X for which there isn’t
an allowed value of Y

∙ If X loses a value, all pairs Z X need to be rechecked

 Arc consistency

∙ Simplest form of propagation makes each pair of

variables consistent:
• X Y is consistent iff for every value of X there is some allowed
value of Y
• When checking X Y, throw out any values of X for which there isn’t
an allowed value of Y

∙ If X loses a value, all pairs Z X need to be rechecked

 Arc consistency

∙ Simplest form of propagation makes each pair of

variables consistent:
• X Y is consistent iff for every value of X there is some allowed
value of Y
• When checking X Y, throw out any values of X for which there isn’t
an allowed value of Y
 Arc consistency

∙ Simplest form of propagation makes each pair of

variables consistent:
• X Y is consistent iff for every value of X there is some allowed
value of Y
• When checking X Y, throw out any values of X for which there isn’t
an allowed value of Y

∙ Arc consistency detects failure earlier than forward

checking
∙ Can be run before or after each assignment
 Arc Consistency Algorithm AC-3
function AC-3(csp) % returns the CSP, possibly with reduced domains
inputs: csp, a binary csp with variables {X1, X2, … , Xn}
local variables: queue, a queue of arcs initially the arcs in csp
while queue is not empty do
(Xi, Xj) ← REMOVE-FIRST(queue)
if REMOVE-INCONSISTENT-VALUES(Xi, Xj) then
for each Xk in NEIGHBORS[Xi ] do
add (Xk, Xi) to queue

function REMOVE-INCONSISTENT-VALUES(Xi, Xj) % returns true iff a value is removed

removed ← false
for each x in DOMAIN[Xi] do
if no value y in DOMAIN[Xj] allows (x,y) to satisfy the constraints between Xi and Xj
then delete x from DOMAIN[Xi]; removed ← true
return removed

Time complexity: O(n2d3)

36
Arc consistency algorithm
AC-3
Does arc consistency always detect the
lack of a solution?

A D
B
A D
C

∙ There exist stronger notions of consistency

(path consistency, k-consistency), but we
won’t worry about them
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

[4-Queens slides copied from B.J. Dorr]

39
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

40
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }

41
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }

42
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , ,3,4}
1
2
3
4
X3 X4
{ , , , } { ,2, , }

43
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{ ,2, ,4} { ,2,3, }

44
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{ ,2, , } { , ,3, }

45
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{ ,2, , } { , ,3, }

46
Example: 4-Queens Problem

X1 X2
1 2 3 4 {1,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{ ,2, , } { , , , }

47
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} {1,2,3,4}
1
2
3
4
X3 X4
{1,2,3,4} {1,2,3,4}

48
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}

49
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, ,3, } {1, ,3,4}

50
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }

51
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } {1, ,3, }

52
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }

53
Example: 4-Queens Problem

X1 X2
1 2 3 4 { ,2,3,4} { , , ,4}
1
2
3
4
X3 X4
{1, , , } { , ,3, }

54
Intelligent Backtracking
∙ Chronological backtracking - the most recent decision
point is revisited
∙ Suppose we have generated the partial assignment
{Q=red,NSW =green,V =blue,T =red}
When we try the next variable, SA, we see that every
value violates a constraint. We back up to T and try a
new color – this cannot possibly resolve the problem
with SA
∙ A more intelligent approach to backtracking is to
backtrack to a variable that might fix the problem—a
variable that was responsible for making one of the
possible values of SA impossible.

55
Backjumping
∙ We will keep track of a set of assignments that are in conflict
with some value for SA.
∙ The set (in this case {Q=red,NSW =green,V =blue,}), is called
the conflict set for SA.
∙ The backjumping method backtracks to the most recent
assignment in the conflict set; in this case, backjumping would
jump over Tasmania and try a new value for V .
∙ A backjumping algorithm that uses conflict sets defined in this
way is called conflict-directed backjumping.
∙ Constraint learning is the idea of finding a minimum set of
variables from the conflict set that causes the problem. This set
of variables, along with their corresponding values, is called a
no-good.

56
Local Search for CSPs

∙ Hill-climbing methods typically work with "complete"

states, i.e., all variables assigned

∙ To apply to CSPs:

• allow states with unsatisfied constraints

• operators reassign variable values

∙ Variable selection: randomly select any conflicted variable

∙ Value selection by min-conflicts heuristic:

• choose value that violates the fewest constraints

• i.e., hill-climb with h(n) = number of violated constraints

57
Example: n-queens
∙ States: 4 queens in 4 columns (44 = 256 states)
∙ Actions: move queen in column
∙ Goal test: no attacks
∙ Evaluation: h(n) = number of attacks

∙ Given random initial state, we can solve n-queens for large n

with high probability

58
Real-world CSPs

∙ Assignment problems
• e.g., who teaches what class
∙ Timetabling problems
• e.g., which class is offered when and where?
∙ Transportation scheduling
∙ Factory scheduling
∙ Notice that many real-world problems involve
real-valued variables

59
Summary
∙ CSPs are a special kind of problem:

• states defined by values of a fixed set of variables

• goal test defined by constraints on variable values

∙ Backtracking = depth-first search with one variable

assigned per node
∙ Variable ordering and value selection heuristics help
significantly
∙ Forward checking prevents assignments that guarantee
later failure
∙ Constraint propagation (e.g., arc consistency) additionally
constrains values and detects inconsistencies
∙ Iterative min-conflicts is usually effective in practice

60