Introduction to

Artificial Intelligence
Chapter 2: Solving Problems
by Searching (7)
Constraint Satisfaction Problems
Nguyễn Hải Minh, Ph.D
[email protected]

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 1

Outline
1. Constraint Satisfaction Problems
(CSP)
2. Constraint Satisfaction Problem as
a Search
3. Constraint Propagation: Inference
in CSPs

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 2

Constraint Satisfaction Problem

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 3

What is a Constraint Satisfaction Problem?
❑A problem with constraints!
o Time constraints
o Budget constraints
❑Few constraints, handle them manually
❑Many constraints, we need computers
❑Here we will only touch on problems that have
finite domain variables.
o This means that the domains are a finite set of
integers, as opposed to a real-valued domain that
would include an infinite number of real-values
between two bounds.

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 4

Solving Constrain Satisfaction Problem
❑Like many other AI problems, CSP’s are solved
using search
❑Unlike other AI problems, CSP’s exhibit a
standard structure
❑Knowledge about this structure (as heuristics)
can be incorporated in the solution process

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 5

State-space search
State-space

Formalize
Problem
…

Each state is atomic,

indivisible

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 6

 Constraint Satisfaction Problem
Problem Variable Value + Contraints
X1

Variable Value + Contraints

X2

…
Each state is
Variable Value + Contraints
factored represented
Xn
Contraint Satisfaction Problem

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 7

Constraint satisfaction problems (CSPs)

❑Standard search problem:

o state is a “black box” – any data structure that
supports successor function, heuristic function, and
goal test
❑CSP:
o state is defined by variables Xi with values from
domain Di
o goal test is a set of constraints C specifying allowable
combinations of values for subsets of variables

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 8

Constraint satisfaction problems (CSPs)
❑An assignment is complete when every variable is
mentioned.
❑Consistent assignment
o Each assignment (a state change or step in a search) of a value to
a variable must be consistent: it must not violate any of the
constraints.
❑A solution to a CSP is a complete assignment that satisfies
all constraints.
❑Some CSPs require a solution that maximizes an objective
function.
❑CSP benefits
o Standard representation pattern
o Generic goal and successor functions
o Generic heuristics (no domain specific expertise)
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 9
 Example: Map-Coloring

❑Variables WA, NT, Q, NSW, V, SA, T

❑Domains Di = {red, green, blue}
❑Constraints: adjacent regions must have different colors
o e.g., WA ≠ NT, or (WA,NT) in {(red,green), (red,blue), (green,red),
(green,blue), (blue,red), (blue,green)}

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 10

 Example: Map-Coloring

❑Solutions are complete and consistent assignments

o e.g., WA = red, NT = green, Q = red, NSW = green, V = red,
SA = blue, T = green

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 11

Aside: Famous Graph Coloring Porblem
❑More general problem than map
coloring
❑Planar graph = graph in the 2d
plane with no edge crossings
❑Guthrie’s conjecture (1852)
o Every planar graph can be colored
with 4 colors or less
→ Proved (using a computer) in 1977
(Appel and Haken)

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 12

 Real-world CSPs
❑Operations Research (scheduling, timetabling)
❑Bioinformatics (DNA sequencing)
❑Electrical engineering (circuit layout-ing)
❑Telecommunications
❑Scheduling the time of observations on the Hubble Space
Telescope
❑Airline schedules
❑Cryptography
❑Computer vision -> image interpretation
→ Notice that many real-world problems involve real-valued
variables
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 13
Varieties of CSPs
❑Discrete variables
o finite domains:
• n variables, domain size d → O(dn) complete assignments
• e.g., N-Queens
o infinite domains:
• integers, strings, etc.
• e.g., job scheduling, variables are start/end days for each
job
• need a constraint language, e.g., StartJob1 + 5 ≤ StartJob3

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 15

Varieties of CSPs
❑Continuous variables
o e.g., start/end times for Hubble Space Telescope
observations
o linear constraints solvable in polynomial time by
linear programming

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 16

Varieties of constraints
❑Unary constraints involve a single variable,
o e.g., SA ≠ green

❑Binary constraints involve pairs of variables,

o e.g., SA ≠ WA

❑Higher-order constraints involve 3 or more

variables,
o e.g., Professors A, B, and C cannot be on a committee
together
o Y = D + E or D + E - 10
o Can always be represented by multiple binary constraints

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 17

Varieties of constraints
❑Inequality constraints on continuous
variables
o endjob1 + 5 ≤ startjob3
❑Preference (soft constraints)
o e.g. red is better than green often can be
represented by a cost for each variable
assignment
o combination of optimization with CSPs
❑the Alldifferent constraint forces all the
variables it touches to have different values
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 18
Constraint graph
❑Constraint graph: nodes are variables, arcs are
constraints

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 19

Example: 4-Queens Problem
❑Variables: 𝑄1, 𝑄2, 𝑄3, 𝑄4
❑Domains: D = {1,2,3,4}
❑Constraints:
o 𝑄𝑖 ≠ 𝑄𝑗 (cannot be in the same row)
o 𝑄𝑖 − 𝑄𝑗 ≠ 𝑖 − 𝑗 (cannot be in the
same diagonal)

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 20

 Example: Cryptarithmetic
❑Variables: 𝐹 𝑇 𝑈 𝑊 𝑅 𝑂 𝐶1 𝐶2 𝐶3
❑Domains: {0,1,2,3,4,5,6,7,8,9}
❑Constraints:
o Alldiff(𝐹, 𝑇, 𝑈, 𝑊, 𝑅, 𝑂)
o 𝐶3 = 𝐹, 𝑇 ≠ 0, 𝐹 ≠ 0
o …

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 21

CSP as a Search
Backtracking search
Forward checking

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 22

Standard search formulation (incremental)
❑Let's start with the straightforward approach, then fix it
❑States are defined by the values assigned so far
o Initial state: the empty assignment { }
o Successor function: assign a value to an unassigned variable that
does not conflict with current assignment
→ fail if no legal assignments
o Goal test: the current assignment is complete
❑This is the same for all CSPs
o Every solution appears at depth n with n variables
→ use depth-first search
o If using BFS: b = (n - l )d at depth l, n! · dn leaves even though
there are only dn complete assignments!

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 23

Backtracking search
❑Variable assignments are commutative,
o i.e., [ WA = red then NT = green ] same as [ NT = green
then WA = red]
❑Only need to consider assignments to a single
variable at each node
→b = d and there are dn leaves
❑Depth-first search for CSPs with single-variable
assignments is called backtracking search
❑Backtracking search is the basic uninformed
algorithm for CSPs
o Can solve n-queens for n ≈ 25

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 24

Backtracking search

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 25

Backtracking example

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 26

Backtracking example

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 27

Backtracking example

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 28

Backtracking example

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 29

Improving backtracking efficiency
❑General-purpose methods can give huge
gains in speed:
o Which variable should be assigned next?
o In what order should its values be tried?
o Can we detect inevitable failure early?

❑Using Heuristic Rules:

o Minimum Remaining Value (MRV)
o Degree Heuristic (DH)
o Least Constraining Value (LCV)
o ...
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 30
Most constrained variable
❑Heuristic Rule:
o choose the variable with the fewest legal values
• E.g., will immediately detect failure if X has no legal values

❑a.k.a. minimum remaining values (MRV) heuristic

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 31

Most constraining variable
❑Tie-breaker among most constrained variables
❑Most constraining variable:
o choose the variable with the most constraints on
remaining variables (most edges in graph)

❑a.k.a. Degree Heuristic (DH)

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 32

Least constraining value
❑Least constraining value heuristic:
o Given a variable, choose the least constraining value:
• the one that rules out the fewest values in the remaining
variables
• leaves the maximum flexibility for subsequent variable
assignments

❑Combining these heuristics makes 1000 queens

feasible
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 33
Forward checking
❑Idea:
o Keep track of remaining legal values for unassigned variables
o Terminate search when any variable has no legal values

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 34

Forward checking
❑Idea:
o Keep track of remaining legal values for unassigned variables
o Terminate search when any variable has no legal values

✓ Assign {WA=red}
✓ Effects on other variables connected by constraints to WA
– NT can no longer be red
– SA can no longer be red

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 35

Forward checking
❑Idea:
o Keep track of remaining legal values for unassigned variables
o Terminate search when any variable has no legal values

✓ Assign {Q=green}
✓ Effects on other variables connected by constraints to Q
• NT can no longer be green
→ MRV heuristic would automatically
• SA can no longer be green
• NSW can no longer be green select NT or SA next
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 36
Forward checking
❑Idea:
FC has detected that partial assignment is inconsistent with
o Keep track of remaining legal values for unassigned variables
the constraints and backtracking can occur.
o Terminate search when any variable has no legal values

✓ Assign {V=blue}
✓ Effects on other variables connected by constraints to V
• NSW can no longer be blue
• SA is empty
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 37
Constraint Propagation:
Inference in CSPs

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 38

Constraint propagation
❑Forward checking propagates information from assigned
to unassigned variables. How about constraints between
two unassigned states?

o NT and SA cannot both be blue!

o Constraint propagation repeatedly enforces constraints locally

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 39

Inference in CSPs
❑In regular state-space search: Search only
❑In CSPs: Search or Inference
❑Constraint propagation:
o Using constraints to reduce number of legal values for a
variable, which in turn can reduce the legal values for
another variable, and so on.
o Can be intertwined with search, or a preprocessing step
o Sometimes this preprocessing can solve the whole problem
❑Idea: local consistency
o Node: variable
o Arc: binary constraint
→ Enforcing local consistency will eliminate inconsistent values

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 40

Arc consistency
❑An arc X →Y is consistent iff for every value x of X there is
some allowed y

Consider state of search after WA and Q are assigned:

• SA → NSW is consistent if SA=blue and NSW=red

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 41

Arc consistency
❑An arc X →Y is consistent iff for every value x of X there is
some allowed y

NSW → SA is consistent if
NSW=red and SA=blue
NSW=blue and SA=???
Arc can be made consistent by removing blue from NSW

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 42

Arc consistency
❑An arc X →Y is consistent iff for every value x of X there is
some allowed y

❑If X loses a value, neighbors of X need to be rechecked

Continue to propagate constraints….
– Check V → NSW
– Not consistent for V = red
– Remove red from V
7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 43
Arc consistency
❑An arc X →Y is consistent iff for every value x of X there is
some allowed y

❑If X loses a value, neighbors of X need to be rechecked

❑Arc consistency detects failure earlier than forward
checking

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 44

Arc consistency vs Forward checking
❑Given a constraint CXY between two variables X
and Y, for any value of X, there is a consistent
value that can be chosen for Y such that CXY is
satisfied, and visa versa.
❑Thus, unlike forward checking, arc consistency is
directed and is checked in both directions for two
connected variables.
→ This makes it stronger than forward checking.

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 45

Arc consistency
❑Can be run as a preprocessor or after each assignment
o Or as preprocessing before search starts
❑AC must be run repeatedly until no inconsistency
remains
❑Trade-off
o Requires some overhead to do, but generally more effective than
direct search
o It can eliminate large (inconsistent) parts of the state space more
effectively than search can
❑Need a systematic method for arc-checking
o If X loses a value, neighbors of X need to be rechecked:
o i.e. incoming arcs can become inconsistent again (outgoing arcs
will stay consistent).

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 46

Arc consistency algorithm AC-3

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS

Time complexity: O(cd3)
47
Local search for CSPs
❑Hill-climbing, simulated annealing typically work with
"complete" states, i.e., all variables assigned
❑To apply to CSPs:
o allow states with unsatisfied constraints
o operators reassign variable values
❑Variable selection: randomly select any conflicted
variable
❑Value selection by min-conflicts heuristic:
o choose value that violates the fewest constraints
o i.e., hill-climb with h(n) = total number of violated constraints

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 48

MIN-CONFLICT Algorithm

Count the number of constraints violated by a particular value,

given the rest of the current assignment

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 49

Example: 4-Queens
❑States: 4 queens in 4 columns (44 = 256 states)
❑Actions: move queen in column
❑Goal test: no attacks
❑Evaluation: h(n) = number of attacks

❑Given random initial state, can solve n-queens in almost constant

time for arbitrary n with high probability (e.g., n = 10,000,000)

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 50

 Summary
❑CSPs are a special kind of problem:
o states defined by values of a fixed set of variables
o goal test defined by constraints on variable values
❑Backtracking = depth-first search with one variable assigned
per node
❑Variable ordering and value selection heuristics help
significantly
❑Forward checking prevents assignments that guarantee later
failure
❑Constraint propagation (e.g., arc consistency) does additional
work to constrain values and detect inconsistencies
❑Iterative min-conflicts is usually effective in practice

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 51

Next week
❑Individual Assignment 3 (I3)
❑Chapter 3: Knowledge
Representation and Reasoning
o Propositional Logic
o First Order Logic

7/12/2020 Nguyễn Hải Minh @ FIT - HCMUS 52