Constraint Satisfaction

Problems!
Các bài toán thỏa mãn ràng buộc #
 Outline!
• Constraint Satisfaction Problems (CSP)#
• Backtracking search for CSPs#
• Local search for CSPs#

Phạm Bảo Sơn 2

 Constraint satisfaction problems
(CSPs)!
• Standard search problem:#
– state is a "black box“ – any data structure that supports
successor function, heuristic function, and goal test#
• CSP:#
– state is defined by variables Xi with values from domain Di#
– goal test is a set of constraints specifying allowable
combinations of values for subsets of variables.#
– Aim is to find an assignment of Xi from domain Di in such a
way that none of the constraints are violated.#
• Simple example of a formal representation language#
• Allows useful general-purpose algorithms with more
power than standard search algorithms#
#
Phạm Bảo Sơn 3
 Example: Map-Coloring!

• Variables WA, NT, Q, NSW, V, SA, T #

• Domains Di = {red,green,blue}#
• Constraints: adjacent regions must have different colors#
#
• e.g., WA ≠ NT, or (WA,NT) in {(red,green),(red,blue),(green,red),
(green,blue),(blue,red),(blue,green)}#
#
Phạm Bảo Sơn 4
 Example: Map-Coloring!

• Solutions are complete and consistent

assignments, e.g., WA = red, NT = green,Q =
red,NSW = green,V = red,SA = blue,T =
green#
Phạm Bảo Sơn 5
#
 Example: n-queens puzzle!
• Assume one queen in each
column.#
• Variables Q1, ..Qn.#
• Domains Di={1,..,n}#
• Constraints#
• Qi ≠ Qj (cannot be in the same
row)#
• |Qi-Qj| ≠ |i-j| ( or same diagonal)#

Phạm Bảo Sơn 6

Example Sudoku!

Phạm Bảo Sơn 7

 Real-world CSPs!
• Assignment problems (e.g. who teaches what
class)#
• Timetabling problems (e.g. which class is
offered when and where?)#
• Hardware configuration#
• Transport scheduling#
• Factory scheduling#

Phạm Bảo Sơn 8

 Constraint graph!
• Binary CSP: each constraint relates two variables#
• Constraint graph: nodes are variables, arcs are
constraints#
#

Phạm Bảo Sơn 9

 Varieties of constraints!
• Unary constraints involve a single variable, #
– e.g., SA ≠ green#
• Binary constraints involve pairs of variables,#
– e.g., SA ≠ WA#
• Higher-order constraints involve 3 or more
variables,#
– e.g., cryptarithmetic column constraints#
• Soft constraints (preferences)#
– 11am lecture is better than 8am lecture#
# Phạm Bảo Sơn 10
 Example: Cryptarithmetic!

• Variables: F T U W R O X1 X2 X3!
• Domains: {0,1,2,3,4,5,6,7,8,9}#
• Constraints: Alldiff (F,T,U,W,R,O)#
– O + O = R + 10 · X1!
– X1 + W + W = U + 10 · X2!
– X2 + T + T = O + 10 · X3!
– X3 = F, T ≠ 0, F ≠ 0#
#
Phạm Bảo Sơn 11
 Standard search formulation (incremental)!
Let's start with the straightforward approach, then fix it#
States are defined by the values assigned so far#
• Initial state: the empty assignment { }#
• Successor function: assign a value to an unassigned variable
that does not conflict with current assignment#
à fail if no legal assignments#
• Goal test: the current assignment is complete#
1. This is the same for all CSPs#
2. Every solution appears at depth n with n variables
à use depth-first search#
3. Path is irrelevant, so can also use complete-state formulation#
4. b = (n - l )d at depth l, hence n! · dn leaves (d: number of
variable values)#

Phạm Bảo Sơn 12

 Backtracking search!
• Variable assignments are commutative, i.e.,#
[ WA = red then NT = green ] same as [ NT = green
then WA = red ]#
• Only need to consider assignments to a single
variable at each node#
à b = d and there are dn leaves#
• Depth-first search for CSPs with single-variable
assignments is called backtracking search#
• Backtracking search is the basic uninformed
algorithm for CSPs#
• Can solve n-queens for n ≈ 25#
#
Phạm Bảo Sơn 13
Backtracking search!

Phạm Bảo Sơn 14

Backtracking example!

Phạm Bảo Sơn 15

Backtracking example!

Phạm Bảo Sơn 16

Backtracking example!

Phạm Bảo Sơn 17

Backtracking example!

Phạm Bảo Sơn 18

 Improving backtracking
efficiency!
• General-purpose methods can give
huge gains in speed:#
– Which variable should be assigned next?#
– In what order should its values be tried?#
– Can we detect inevitable failure early?#
#

Phạm Bảo Sơn 19

 Most constrained variable
Biến bị ràng buộc nhiều nhất!
• Most constrained variable: choose the
variable with the fewest legal values#
#
#
• a.k.a. minimum remaining values (MRV)
heuristic#
#

Phạm Bảo Sơn 20

 Most constraining variable
Biến ràng buộc nhiều nhất!
• Tie-breaker among most constrained
variables#
• Most constraining variable (degree heuristic):#
– choose the variable with the most constraints on
remaining variables#
#

Phạm Bảo Sơn 21

 Least constraining value
Giá trị ràng buộc ít nhất!
• Given a variable, choose the least
constraining value:#
– the one that rules out the fewest values in the
remaining variables#
#

#
• Combining these heuristics makes 1000
queens feasible#
# Phạm Bảo Sơn 22
 Forward checking
Kiểm tra trước!
• Idea: #
– Keep track of remaining legal values for unassigned
variables#
– Terminate search when any variable has no legal values#
#

Phạm Bảo Sơn 23

 Forward checking!
• Idea: #
– Keep track of remaining legal values for unassigned
variables#
– Terminate search when any variable has no legal values#
#

Phạm Bảo Sơn 24

 Forward checking!
• Idea: #
– Keep track of remaining legal values for unassigned
variables#
– Terminate search when any variable has no legal values#
#

Phạm Bảo Sơn 25

 Forward checking!
• Idea: #
– Keep track of remaining legal values for unassigned
variables#
– Terminate search when any variable has no legal values#
#

Phạm Bảo Sơn 26

 Constraint propagation!
• Forward checking propagates information from
assigned to unassigned variables, but doesn't provide
early detection for all failures:#
#

• NT and SA cannot both be blue!#

• Constraint propagation repeatedly enforces
constraints locally#
# Phạm Bảo Sơn 27
 Arc consistency
!
• Simplest form of propagation makes each arc
consistent#
• X àY is consistent iff#
for every value x of X there is some allowed y#
#

Phạm Bảo Sơn 28

 Arc consistency!
• Simplest form of propagation makes each arc
consistent#
• X àY is consistent iff#
for every value x of X there is some allowed y#
#

Phạm Bảo Sơn 29

 Arc consistency!
• Simplest form of propagation makes each arc
consistent#
• X àY is consistent iff#
for every value x of X there is some allowed y#
#
#
#
#
#
• If X loses a value, neighbors of X need to be
rechecked#
# Phạm Bảo Sơn 30
 Arc consistency!
• Simplest form of propagation makes each arc consistent#
• X àY is consistent iff#
for every value x of X there is some allowed y#
#
#
#
#
#

• If X loses a value, neighbors of X need to be rechecked#

• Arc consistency detects failure earlier than forward checking#
• Can be run as a preprocessor or after each assignment#
# Phạm Bảo Sơn 31
#
#
 Arc consistency algorithm
AC-3!

• Time complexity: O(n2d3)#

# Phạm Bảo Sơn 32
 Special constraints!
• Arc-consistency does miss some cases#
• Example: #
– {WA=red, NSW=red}#
– AC-3: Domain for SA, NT, Q : {green, blue}#
– Alldiff constraint is violated as number of
values is less than number of variables.#

Phạm Bảo Sơn 33

 Local search for CSPs!
• Local search or iterative improvement.#
• Hill-climbing, simulated annealing typically work with "complete"
states, i.e., all variables assigned#
• To apply to CSPs:#
– allow states with unsatisfied constraints#
– operators reassign variable values#
• Variable selection: randomly select any conflicted variable#
• Value selection by min-conflicts (mâu thuẫn ít nhất) heuristic:#
– choose value that violates the fewest constraints#
– i.e., hill-climb with h(n) = total number of violated constraints#
#

Phạm Bảo Sơn 34

 Example: 4-Queens!
• States: 4 queens in 4 columns (44 = 256 states)#
• Actions: move queen in column#
• Goal test: no attacks#
• Evaluation: h(n) = number of attacks#
#

Phạm Bảo Sơn 35

 Phase transition in CSP’s!
• Given random initial state, can solve n-queens in almost
constant time for arbitrary n with high probability (e.g., n =
10,000,000)#
• In general, randomly-generated CSP tend to be easy if there are
very few or very many constraints. They become extra hard in a
narrow range of the ratio:#
#

Phạm Bảo Sơn 36

 Flat regions and local
optima!

• Sometimes, have to go sideways or even backwards

in order to make progress towards the actual solution.#

Phạm Bảo Sơn 37

 Simulated Annealing!
• Stochastic hill climbing based on difference between
evaluation of previous state (h0) and new state (h1).#
• If h1 < h0, definitely make the change.#
• Otherwise, make the change with probability:#
# #e-(h1-h0)/T ,T is a “temperature” parameter#
• Reduces to ordinary hill climbing when T=0.#
• Become totally random search as T-> ∞#
• We gradually decrease the value of T during the
search. #

Phạm Bảo Sơn 38

 Summary!
• CSPs are a special kind of problem:#
– states defined by values of a fixed set of variables#
– goal test defined by constraints on variable values#
• Backtracking = depth-first search with one variable assigned per
node#
• Variable ordering and value selection heuristics help
significantly#
• Forward checking prevents assignments that guarantee later
failure#
• Constraint propagation (e.g., arc consistency) does additional
work to constrain values and detect inconsistencies#
• Iterative min-conflicts is usually effective in practice#
• Simulated Annealing can help to escape from local optima.#
#

Phạm Bảo Sơn 39

 References!
• Artificial Intelligence, A modern
approach. Chapter 5.#

Phạm Bảo Sơn 40