UNIT IV

Constraint satisfaction problem:

Defining Constraint Satisfaction Problems,
Constraint Propagation: Inference in CSPs,
Backtracking Search for CSPs,
Local Search for CSPs,
The Structure of Problems.

22/01/2024 PRNCSE 1
 CONSTRAINT
SATISFACTION PROBLEMS
• Till now we have seen that problems can be solved by searching in a
space of states. These states can be evaluated by domain-specific
heuristics and tested to see whether they are goal states.

• In search algorithm point of view however, each state is atomic, or

indivisible—a black box with no internal structure.

• This chapter describes a way to solve a wide variety of problems more

efficiently.

22/01/2024 PRNCSE 2
• We use a factored representation for each state: a set of variables,
each of which has a value.

• A problem is solved when each variable has a value that satisfies all
the constraints on the variable.

• A problem described this way is called a constraint satisfaction

problem, or CSP.

22/01/2024 PRNCSE 3
• CSP search algorithms take advantage of the structure of states and
use general-purpose rather than problem-specific heuristics to enable
the solution of complex problems.

• The main idea is to eliminate large portions of the search space all at
once by identifying variable/value combinations that violate the
constraints.

22/01/2024 PRNCSE 4
 DEFINING CONSTRAINT SATISFACTION
PROBLEMS
• A constraint satisfaction problem consists of three components, X,D,
and C:
X is a set of variables, {X1, . . . ,Xn}.
D is a set of domains, {D1, . . . ,Dn}, one for each variable.
C is a set of constraints that specify allowable combinations of values.
• Each domain Di consists of a set of allowable values, {v1, . . . , vk} for
variable Xi.
• Each constraint Ci consists of a pair <scope, rel> , where scope is a tuple
of variables that participate in the constraint and rel is a relation that
defines the values that those variables can take on

22/01/2024 PRNCSE 5
• A relation can be represented as an explicit list of all tuples of values
that satisfy the constraint, or as an abstract relation that supports two
operations: testing if a tuple is a member of the relation and
enumerating the members of the relation.

• For example, if X1 and X2 both have the domain {A,B}, then the
constraint saying the two variables must have different values can be
written as (X1,X2), [(A,B), (B,A)] or as [(X1,X2),X1 != X2.]

22/01/2024 PRNCSE 6
• To solve a CSP, we need to define a state space and the notion of a
solution.
• Each state in a CSP is defined by an assignment of values to some or
all of the variables, {Xi =vi,Xj = vj , . . .}.
• An assignment that does not violate any constraints is called a
consistent or legal assignment.
• A complete assignment is one in which every variable is assigned, and
a solution to a CSP is a consistent, complete assignment.
• A partial assignment is one that assigns values to only some of the
variables.
22/01/2024 PRNCSE 7
 Example problem: Map coloring

• Consider map of Australia showing each of its states and territories

22/01/2024 PRNCSE 8
• We are given the task of coloring each region either red, green, or
blue in such a way that no neighboring regions have the same color.

• To formulate this as a CSP, we define the variables to be the regions

X = {WA,NT,Q,NSW, V,SA, T} .

• The domain of each variable is the set Di = {red , green, blue}.

• The constraints require neighboring regions to have distinct colors.

22/01/2024 PRNCSE 9
• Since there are nine places where regions border, there are nine
constraints:
• C = {SA != WA,SA != NT,SA != Q,SA != NSW,SA != V, WA != NT,NT != Q,
Q != NSW,NSW! = V } .
• Here we are using abbreviations, SA != WA is a shortcut for
(SA,WA),SA != WA, where SA != WA can be fully enumerated in turn as
{(red , green), (red , blue), (green, red ), (green, blue), (blue, red ),
(blue, green)} .
• There are many possible solutions to this problem, such as
• {WA=red ,NT =green, Q=red ,NSW =green, V =red ,SA=blue, T =red }.

22/01/2024 PRNCSE 10
• It can be helpful to visualize a CSP as a constraint graph

• The nodes of the graph correspond to variables of the problem, and a

link connects any two variables that participate in a constraint.

22/01/2024 PRNCSE 11
• Why formulate a problem as a CSP?

• One reason is that the CSPs yield a natural representation for a wide
variety of problems.

• If you already have a CSP-solving system, it is often easier to solve a

problem using it than to design a custom solution using another
search technique.

• In addition, CSP solvers can be faster than state-space searchers

because the CSP solver can quickly eliminate large swatches of the
search space.
22/01/2024 PRNCSE 12
• For example, once we have chosen {SA=blue} in the Australia
problem, we can conclude that none of the five neighboring variables
can take on the value blue.

• Without taking advantage of constraint propagation, a search

procedure would have to consider 35 =243 assignments for the five
neighboring variables, with constraint propagation we never have to
consider blue as a value, so we have only 25 =32 assignments to look
at, a reduction of 87%.

22/01/2024 PRNCSE 13
• In regular state-space search we can only ask: is this specific state a
goal? No? What about this one?
• With CSPs, once we find out that a partial assignment is not a
solution, we can immediately discard further refinements of the
partial assignment.
• Furthermore, we can see why the assignment is not a solution and
which variables violate a constraint, so we can focus attention on the
variables that matter.
• As a result, many problems that are intractable for regular state-space
search can be solved quickly when formulated as a CSP.

22/01/2024 PRNCSE 14
 Example problem: Job-shop scheduling
• Factories have the problem of scheduling a day’s worth of jobs,
subject to various constraints.
• In practice, many of these problems are solved with CSP techniques.
• Consider the problem of scheduling the assembly of a car.
• The whole job is composed of tasks, and we can model each task as a
variable, where the value of each variable is the time that the task
starts, expressed as an integer number of minutes.
• Constraints can assert that one task must occur before another.

22/01/2024 PRNCSE 15
• for example, a wheel must be installed before the hubcap is put on
and that only so many tasks can go on at once.
• Constraints can also specify that a task takes a certain amount of time
to complete.
• We consider a small part of the car assembly, consisting of 15 tasks:
install axles (front and back), affix all four wheels (right and left, front
and back), tighten nuts for each wheel, affix hubcaps, and inspect the
final assembly.
• We can represent the tasks with 15 variables:

22/01/2024 PRNCSE 16
• The value of each variable is the time that the task starts.
• Next we represent precedence constraints between individual tasks.

• Whenever a task T1 must occur before task T2 PRECEDENCE , and task

T1 takes duration d1 to complete, we add an arithmetic constraint of
the form T1 + d1 ≤ T2

• In our example, the axles have to be in place before the wheels are
put on, and it takes 10 minutes to install an axle, so we write

22/01/2024 PRNCSE 17
• Next we say that, for each wheel, we must affix the wheel (which
takes 1 minute), then tighten the nuts (2 minutes), and finally attach
the hubcap (1 minute, but not represented yet):

22/01/2024 PRNCSE 18
• Suppose we have four workers to install wheels, but they have to
share one tool that helps put the axle in place. We need a disjunctive
constraint to say that AxleF and AxleB must not overlap in time, either
one comes first or the other does:
(AxleF + 10 ≤ AxleB) or (AxleB + 10 ≤ AxleF ) .

• This looks like a more complicated constraint, combining arithmetic

and logic.

• But it still reduces to a set of pairs of values that AxleF and AxleB can
take on.

22/01/2024 PRNCSE 19
• We also need to assert that the inspection comes last and takes 3
minutes.
• For every variable except Inspect we add a constraint of the form
X +dX ≤ Inspect .
• Finally, suppose there is a requirement to get the whole assembly
done in 30 minutes. We can achieve that by limiting the domain of all
variables:
Di = {1, 2, 3, . . . , 27} .
• This particular problem is trivial to solve, but CSPs have been applied
to job-shop scheduling problems like this with thousands of variables.
• In some cases, there are complicated constraints that are difficult to
specify in the CSP formalism, and more advanced planning techniques
are used
22/01/2024 PRNCSE 20
 Variations on the CSP formalism
• The simplest kind of CSP involves variables that have discrete, finite
domains.

• Map-coloring problems and scheduling with time limits are both of

this kind.

• The 8-queens problem can also be viewed as a finite-domain CSP,

where the variables Q1, . . . ,Q8 are the positions of each queen in
columns 1, . . . , 8 and each variable has the domain Di = {1, 2, 3, 4, 5,
6, 7, 8}.

22/01/2024 PRNCSE 21
• A discrete domain can be infinite, such as the set of integers or
strings. (If we didn’t put a deadline on the job-scheduling problem,
there would be an infinite number of start times for each variable.)
• With infinite domains, it is no longer possible to describe constraints
by enumerating all allowed combinations of values.
• Instead, a constraint language must be used that understands
constraints such as T1 + d1 ≤ T2 directly, without enumerating the set
of pairs of allowable values for (T1, T2).
• Special solution algorithms exist for linear constraints on integer
variables—that is, constraints, such as the one just given, in which
each variable appears only in linear form.
• It can be shown that no algorithm exists for solving general nonlinear
constraints on integer variables.
22/01/2024 PRNCSE 22
• Constraint satisfaction problems with continuous domains are common in
the real world and are widely studied in the field of operations research.
• For example, the scheduling of experiments on the Hubble Space
Telescope requires very precise timing of observations, the start and finish
of each observation and maneuver are continuous-valued variables that
must obey a variety of astronomical, precedence, and power constraints.
• The best-known category of continuous-domain CSPs is that of linear
programming problems, where constraints must be linear equalities or
inequalities.
• Linear programming problems can be solved in time polynomial in the
number of variables.
• Problems with different types of constraints and objective functions have
also been studied—quadratic programming, second-order conic
programming, and so on.
22/01/2024 PRNCSE 23
• In addition to examining the types of variables that can appear in CSPs,
it is useful to look at the types of constraints.

• Unary constraint: Restricts the value of a single variable. Ex: Map-

coloring problem <(SA),SA != green> South Australians won’t tolerate
the color green

• Binary constraint: Relates two variables. Ex: SA != NSW. A binary CSP is

one with only binary constraints, it can be represented as a constraint
graph.

• Higher-order constraints: Asserting that the value of Y is between X

and Z, with the ternary constraint between (X, Y,Z).
22/01/2024 PRNCSE 24
• A constraint involving arbitrary number of variables is called a global
constraint.(need not involve all the variables in a problem)

• One of the most common global constraints is Alldiff , which says that all of
the variables involved in the constraint must have different values.

• Ex:
Sudoku problems, all variables in a row or column must satisfy an Alldiff
constraint.
Cryptarithmetic puzzles, each letter stands for a distinct digit, the aim is
to find a substitution of digits for letters such that the resulting sum is
arithmetically correct, with the added restriction that no leading zeroes
are allowed
22/01/2024 PRNCSE 25
 Cryptarithmetic Problem
• Cryptarithmetic Problem is a type of csp where the game is about
digits and its unique replacement either with alphabets or other
symbols.
• The rules or constraints on a cryptarithmetic problem are as follows:
• There should be a unique digit to be replaced with a unique alphabet.
• The result should satisfy the predefined arithmetic rules, i.e., 2+2 =4, nothing
else.
• Digits should be from 0-9 only.
• There should be only one carry forward, while performing the addition
operation on a problem.
• The problem can be solved from both sides, i.e., lefthand side (L.H.S), or
righthand side (R.H.S)

22/01/2024 PRNCSE 26
• Given a cryptarithmetic problem, i.e., S E N D + M O R E = M O N E Y

• Starting from the left hand side (L.H.S) , the terms are S and M.
• Assign a digit which could give a satisfactory result.
• Let’s assign S->9 and M->1.
• Hence, we get a satisfactory result by adding up the terms and got an
assignment for O as O->0 as well.

22/01/2024 PRNCSE 27
• Now, move ahead to the next terms E and O to get N as its output.

• Adding E and O, which means 5+0=0, which is not possible

because according to cryptarithmetic constraints, we cannot assign
the same digit to two letters. So, we need to think more and assign
some other value.

22/01/2024 PRNCSE 28
• When we will solve further, we will get one carry, so after applying it,
the answer will be satisfied.
• Further, adding the next two terms N and R we get,

• But, we have already assigned E->5. Thus, the above result does not
satisfy the values because we are getting a different value for E. So,
we need to think more.
• Again, after solving the whole problem, we will get a carryover on
this term, so our answer will be satisfied.
22/01/2024 PRNCSE 29
 Again, on adding the last two terms,
i.e., the rightmost terms D and E,
we get Y as its result.

Keeping all the constraints in mind, the final resultant is as

follows:

S E N D M O R Y
9 5 6 7 1 0 8 2

22/01/2024 PRNCSE 30
 Global constraint Alldiff (F, T,U,W,R,O)

The addition constraints on the four columns of the

puzzle can be written as the following n-ary constraints:
O + O = R + 10 ・ C10
C10 +W +W = U +10 ・ C100
C100 + T + T = O + 10 ・ C1000
C1000 = F
where C10, C100, and C1000 are auxiliary variables
representing the digit carried over into the tens,
hundreds, or thousands column.

A hypergraph consists of ordinary nodes(the circles

in the figure) and hypernodes (the squares),
which represent n-ary constraints.
22/01/2024 PRNCSE 31
• Every finite-domain constraint can be reduced to a set of binary
constraints if enough auxiliary variables are introduced.

• So we could transform any CSP into one with only binary constraints.
This makes the algorithms simpler.

• Another way to convert an n-ary CSP to a binary one is the dual graph
transformation.

• By creating a new graph in which there will be one variable for each
constraint in the original graph and one binary constraint for each pair
of constraints in the original graph that share variables.

22/01/2024 PRNCSE 32
• For example, if the original graph has variables {X, Y, Z} and constraints
<(X, Y, Z),C1> and <(X, Y ),C2> then the dual graph would have variables
{C1,C2} with the binary constraint <(X, Y ),R1>, where (X, Y ) are the
shared variables and R1 is a new relation that defines the constraint
between the shared variables, as specified by the original C1 and C2.
• Two reasons why we might prefer a global constraint such as Alldiff
rather than a set of binary constraints.
• easier and less error-prone to write the problem description
• possible to design special-purpose inference algorithms for global
constraints that are not available for a set of more primitive
constraints.

22/01/2024 PRNCSE 33
• The constraints we have described so far have all been absolute
constraints, violation of which rules out a potential solution.
• Many real-world CSPs include preference constraints indicating which
solutions are preferred.
• For example, in a university class-scheduling problem there are
absolute constraints that no professor can teach two classes at the
same time.
• But we also may allow preference constraints: Prof. R might prefer
teaching in the morning, whereas Prof. N prefers teaching in the
afternoon.
• A schedule that has Prof. R teaching at 2 p.m. would still be an
allowable solution (unless Prof. R happens to be the department
chair) but would not be an optimal one.
22/01/2024 PRNCSE 34
• Preference constraints can often be encoded as costs on individual
variable assignments.

• For example, assigning an afternoon slot for Prof. R costs 2 points

against the overall objective function, whereas a morning slot costs 1.

• With this formulation, CSPs with preferences can be solved with

optimization search methods, either path-based or local.

• We call such a problem a constraint optimization problem, or COP.

Linear programming problems do this kind of optimization.

22/01/2024 PRNCSE 35
 CONSTRAINT PROPAGATION:
INFERENCE IN CSPS
• In regular state-space search, an algorithm can do only one thing i.e.,
search.

• In CSPs there is a choice: an algorithm can search (choose a new

variable assignment from several possibilities) or do a specific type of
inference called constraint propagation

• Using constraints can reduce the number of legal values for a variable,
which in turn can reduce the legal values for another variable, and so
on.

22/01/2024 PRNCSE 36
• Constraint propagation may be intertwined with search, or it may be
done as a pre-processing step, before search starts.

• Sometimes this pre-processing can solve the whole problem, so no

search is required at all.

• The key idea is local consistency. If we treat each variable as a node in

a graph and each binary constraint as an arc, then the process of
enforcing local consistency in each part of the graph causes
inconsistent values to be eliminated throughout the graph. There are
different types of local consistency.

22/01/2024 PRNCSE 37
 Node-consistent
• A single variable (a node in the CSP network) is node-consistent if all
the values in the variable’s domain satisfy the variable’s unary
constraint.
• We say that a network is node-consistent if every variable in the
network is node-consistent.
• For example, in the variant of the Australia map-coloring problem
where South Australians dislike green, the variable SA starts with
domain {red , green, blue}, and we can make it node consistent by
eliminating green, leaving SA with the reduced domain {red , blue}.

22/01/2024 PRNCSE 38
 Arc-consistency

• A variable in a CSP is arc-consistent if every value in its domain satisfies

the variable’s binary constraints.

• Xi is arc-consistent with respect to another variable Xj if for every value in

the current domain Di there is some value in the domain Dj that satisfies
the binary constraint on the arc (Xi, Xj).

• A network is arc-consistent if every variable is arc-consistent with every

other variable.

• Arc consistency tightens down the domains (unary constraint) using the
arcs (binary constraints).
22/01/2024 PRNCSE 39
• For example, consider the constraint Y = X2 where the domain of both
X and Y is the set of digits.

• We can write this constraint explicitly as

• <(X, Y ), {(0, 0), (1, 1), (2, 4), (3, 9))}>

• To make X arc-consistent with respect to Y , we reduce X’s domain to

{0, 1, 2, 3}.

• If we also make Y arc-consistent with respect to X, then Y ’s domain

becomes {0, 1, 4, 9} and the whole CSP is arc-consistent.

22/01/2024 PRNCSE 40
• Arc consistency can do nothing for the Australia map-coloring
problem.

• Consider the following inequality constraint on (SA,WA): {(red ,

green), (red , blue), (green, red ), (green, blue), (blue, red ), (blue,
green)}

• No matter what value you choose for SA (or for WA), there is a valid
value for the other variable.

• So applying arc consistency has no effect on the domains of either

variable.
22/01/2024 PRNCSE 41
• The most popular algorithm for arc consistency is called AC-3.

22/01/2024 PRNCSE 42
• AC-3 maintains a queue of arcs which initially contains all the arcs in the CSP.
• AC-3 then pops off an arbitrary arc (Xi, Xj) from the queue and makes Xi arc-
consistent with respect to Xj.
• If this leaves Di unchanged, just moves on to the next arc;
• But if this revises Di, then add to the queue all arcs (Xk, Xi) where Xk is a
neighbor of Xi.
• If Di is revised down to nothing, then the whole CSP has no consistent
solution, return failure;
• Otherwise, keep checking, trying to remove values from the domains of
variables until no more arcs are in the queue.
• The result is an arc-consistent CSP that have the same solutions as the
original one but have smaller domains.

22/01/2024 PRNCSE 43
 The complexity of AC-3

• Assume a CSP with n variables, each with domain size at most d, and

with c binary constraints (arcs). Checking consistency of an arc can be

done in O(d2) time, total worst-case time is O(cd3).

22/01/2024 PRNCSE 44
 Path consistency

• A two-variable set {Xi, Xj} is path-consistent with respect to a third

variable Xm if, for every assignment {Xi = a, Xj = b} consistent with the
constraint on {Xi, Xj}, there is an assignment to Xm that satisfies the
constraints on {Xi, Xm} and {Xm, Xj}.

• Path consistency tightens the binary constraints by using implicit

constraints that are inferred by looking at triples of variables.

22/01/2024 PRNCSE 45
 K-consistency
• A CSP is k-consistent if, for any set of k-1 variables and for any
consistent assignment to those variables, a consistent value can
always be assigned to any kth variable.

• 1-consistency = node consistency;

• 2-consisency = arc consistency;
• 3-consistensy = path consistency.

22/01/2024 PRNCSE 46
 Global constraints
• Global constraint is one involving an arbitrary number of variables
(but not necessarily all variables).

• Global constraints occur frequently in real problems and can be

handled by special-purpose algorithms that are more efficient than
the general-purpose methods.

• For example, the Alldiff constraint says that all the variables involved
must have distinct values (as in the cryptarithmetic problem and
Sudoku puzzles)

22/01/2024 PRNCSE 47
 Resource Constraint
• Another important higher-order constraint is the resource constraint,
sometimes called the atmost constraint.
• For example, in a scheduling problem, let P1, . . . , P4 denote the numbers of
personnel assigned to each of four tasks. The constraint that no more than
10 personnel are assigned in total is written as Atmost(10, P1, P2, P3, P4).
• We can detect an inconsistency simply by checking the sum of the minimum
values of the current domains.
• for example, if each variable has the domain {3, 4, 5, 6}, the Atmost
constraint cannot be satisfied.
• We can also enforce consistency by deleting the maximum value of any
domain if it is not consistent with the minimum values of the other domains.
• Thus, if each variable in our example has the domain {2, 3, 4, 5, 6}, the
values 5 and 6 can be deleted from each
22/01/2024 PRNCSE
domain. 48
• For large resource-limited problems with integer values—such as
logistical problems involving moving thousands of people in hundreds
of vehicles—it is usually not possible to represent the domain of each
variable as a large set of integers and gradually reduce that set by
consistency-checking methods.

• Instead, domains are represented by upper and lower bounds and are
managed by bounds propagation

22/01/2024 PRNCSE 49
• For example, in an airline-scheduling problem, let’s suppose there are two
flights, F1 and F2, for which the planes have capacities 165 and 385,
respectively.
• The initial domains for the numbers of passengers on each flight are then
D1 = [0, 165] and D2 = [0, 385] .
• Now suppose we have the additional constraint that the two flights
together must carry 420 people: F1 + F2 = 420.
• Propagating bounds constraints, we reduce the domains to
D1 = [35, 165] and D2 = [255, 385] .
• We say that a CSP is bounds consistent if for every variable X, and for both
the lower-bound and upper-bound values of X, there exists some value of Y
that satisfies the constraint between X and Y for every variable Y .
• This kind of bounds propagation is widely used in practical constraint
problems.

22/01/2024 PRNCSE 50
 Sudoku example
• Home work

22/01/2024 PRNCSE 51
 Backtracking Search for CSPs
• Sudoku problems are designed to be solved by inference over constraints.
• But many other CSPs cannot be solved by inference.
• Here we look at backtracking search algorithms that work on partial
assignments.
• We could apply a standard depth-limited search. A state would be a
partial assignment, and an action would be adding var = value to the
assignment.
• But for a CSP with n variables of domain size d:
• the branching factor at the top level is nd because any of d values can be assigned
to any of n variables.
• At the next level, the branching factor is (n − 1)d, and so on for n levels.
• We generate a tree with n!*dn leaves, even though there are only d n possible
complete assignments.
22/01/2024 PRNCSE 52
• A crucial property common to all CSPs: commutativity.
• A problem is commutative if the order of application of any given set
of actions has no effect on the outcome.
• CSPs are commutative because when assigning values to variables, we
reach the same partial assignment regardless of order.
• Therefore, we need only consider a single variable at each node in the
search tree.
• For example, at the root node of a search tree for coloring the map of
Australia, we might make a choice between SA=red, SA=green, and
SA=blue, but we would never choose between SA=red and WA=blue.
• With this restriction, the number of leaves is dn, as we would hope.

22/01/2024 PRNCSE 53
 A simple backtracking algorithm for CSP.
The algorithm is modelled on the
recursive depth-first search.
By varying the functions
SELECT-UNASSIGNED-VARIABLE and
ORDER-DOMAIN-VALUES, we can
implement the general-purpose heuristics.
The function INFERENCE can optionally be
used to impose arc-, path-, or k-
consistency, as desired.
If a value choice leads to failure(either by
INFERENCE or by BACKTRACK), then value
assignments (including those made by
INFERENCE) are removed from the current
assignment and a new value is tried.

22/01/2024 PRNCSE 54
 • The term backtracking search is used for a depth-first search that
chooses values for one variable at a time and backtracks when a
variable has no legal values left to assign.
• It repeatedly chooses an unassigned variable, and then tries all values
in the domain of that variable in turn, trying to find a solution.
• If an inconsistency is detected, then BACKTRACK returns failure,
causing the previous call to try another value.

22/01/2024 PRNCSE 55
• To solve CSPs efficiently without domain-specific knowledge, address
following questions:

1. Which variable should be assigned next (SELECT-UNASSIGNED-

VARIABLE), and in what order should its values be tried (ORDER-
DOMAIN-VALUES)?

2. What inferences should be performed at each step in the search

(INFERENCE)?

3. When the search arrives at an assignment that violates a

constraint, can the search avoid repeating this failure?

22/01/2024 PRNCSE 56
 SELECT-UNASSIGNED-VARIABLE
Variable selection—fail-first
• Minimum-remaining-values (MRV) heuristic: The idea of choosing
the variable with the fewest “legal” value.
• A.k.a. “most constrained variable” or “fail-first” heuristic, it picks a
variable that is most likely to cause a failure soon thereby pruning the
search tree.
• If some variable X has no legal values left, the MRV heuristic will
select X and failure will be detected immediately avoiding pointless
searches through other variables.
• E.g. After the assignment for WA=red and NT=green, there is only one
possible value for SA, so it makes sense to assign SA=blue next rather
than assigning Q.
22/01/2024 PRNCSE 57
• Degree heuristic: The degree heuristic attempts to reduce the
branching factor on future choices by selecting the variable that is
involved in the largest number of constraints on other unassigned
variables. [useful tie-breaker]

• e.g. SA is the variable with highest degree 5; the other variables have
degree 2 or 3; T has degree 0.

22/01/2024 PRNCSE 58
 ORDER-DOMAIN-VALUES
Value selection—fail-last
• If we are trying to find all the solution to a problem (not just the first
one), then the ordering does not matter.
• Least-constraining-value heuristic: prefers the value that rules out
the fewest choice for the neighboring variables in the constraint
graph. (Try to leave the maximum flexibility for subsequent variable
assignments.)
• e.g. We have generated the partial assignment with WA=red and
NT=green and that our next choice is for Q. Blue would be a bad
choice because it eliminates the last legal value left for Q’s neighbor,
SA, therefore prefers red to blue.

22/01/2024 PRNCSE 59
• The minimum-remaining-values and degree heuristic
are domain-independent methods for deciding which
variable to choose next in a backtracking search.
The least-constraining-value heuristic helps in
deciding which value to try first for a given variable.

22/01/2024 PRNCSE 60
 INFERENCE

• Forward checking: [One of the simplest forms of inference.]

Whenever a variable X is assigned, the forward-checking process
establishes arc consistency for it.

• For each unassigned variable Y that is connected to X by a constraint,

delete from Y’s domain any value that is inconsistent with the value
chosen for X.

• There is no reason to do forward checking if we have already done arc

consistency as a preprocessing step.
22/01/2024 PRNCSE 61
22/01/2024 PRNCSE 62
• Advantage: For many problems the search will be more effective if we
combine the MRV heuristic with forward checking.

• Disadvantage: Forward checking only makes the current variable arc-

consistent, but doesn’t look ahead and make all the other variables
arc-consistent.

22/01/2024 PRNCSE 63
• MAC (Maintaining Arc Consistency) algorithm:
• [More powerful than forward checking, detect this inconsistency.]
• After a variable Xi is assigned a value, the INFERENCE procedure calls
AC-3, but instead of a queue of all arcs in the CSP, we start with only
the arcs(Xj, Xi) for all Xj that are unassigned variables that are
neighbors of Xi.
• From there, AC-3 does constraint propagation in the usual way, and if
any variable has its domain reduced to the empty set, the call to AC-3
fails and we know to backtrack immediately.

22/01/2024 PRNCSE 64
 Intelligent backtracking
• Chronological backtracking:
• In BACKGRACKING-SEARCH when a branch of the search fails, back up
to the preceding variable and try a different value for it. (The most
recent decision point is revisited.)
• E.g. Suppose we have generated the partial assignment {Q=red,
NSW=green, V=blue, T=red}.
• When we try the next variable SA, we see every value violates a
constraint.
• We back up to T and try a new color, it cannot resolve the problem.

22/01/2024 PRNCSE 65
• Backtrack to a variable that was responsible for making one of the
possible values of the next variable (e.g. SA) impossible.
• Conflict set for a variable: A set of assignments that are in conflict
with some value for that variable.
(e.g. The set {Q=red, NSW=green, V=blue} is the conflict set for SA.)

• Backjumping method: Backtracks to the most recent assignment in

the conflict set.
(e.g. backjumping would jump over T and try a new value for V.)

22/01/2024 PRNCSE 66
• Forward checking can supply the conflict set with no extra work.

• Whenever forward checking based on an assignment X=x deletes a

value from Y’s domain, add X=x to Y’s conflict set.

• If the last value is deleted from Y’s domain, the assignment in the
conflict set of Y are added to the conflict set of X.

• Infact, every branch pruned by backjumping is also pruned by forward

checking.

• Hence simple backjumping is redundant in a forward-checking search

or in a search that uses stronger consistency checking (such as MAC).
22/01/2024 PRNCSE 67
 Conflict-directed backjumping
• consider the partial assignment which is proved to be inconsistent:
{WA=red, NSW=red}.
• We try T=red next and then assign NT, Q, V, SA, no assignment can
work for these last 4 variables.
• Eventually we run out of value to try at NT, but simple backjumping
cannot work because NT doesn’t have a complete conflict set of
preceding variables that caused to fail.

22/01/2024 PRNCSE 68
• The set {WA, NSW} is a deeper notion of the conflict set for NT,
caused NT together with any subsequent variables to have no
consistent solution. So the algorithm should backtrack to NSW and
skip over T.

• A backjumping algorithm that uses conflict sets defined in this way is

called conflict-direct backjumping.

22/01/2024 PRNCSE 69
• When a variable’s domain becomes empty, the “terminal” failure
occurs, that variable has a standard conflict set.

• Let Xj be the current variable, let conf(Xj) be its conflict set. If every
possible value for Xj fails, backjump to the most recent
variable Xi in conf(Xj), and set
• conf(Xi) ← conf(Xi)∪conf(Xj) – {Xi}.

• The conflict set for an variable means, there is no solution from that
variable onward, given the preceding assignment to the conflict set.

22/01/2024 PRNCSE 70
• E.g.
• assign WA, NSW, T, NT, Q, V, SA.
• SA fails, and its conflict set is {WA, NT, Q}. (standard conflict set)
• Backjump to Q, its conflict set is {NT, NSW}∪{WA,NT,Q}-{Q} = {WA, NT,
NSW}.
• Backtrack to NT, its conflict set is {WA}∪{WA,NT,NSW}-{NT} = {WA,
NSW}.
• Hence the algorithm backjump to NSW. (over T)

22/01/2024 PRNCSE 71
• After backjumping from a contradiction, how to avoid running into
the same problem again:
• Constraint learning: The idea of finding a minimum set of variables
from the conflict set that causes the problem. This set of variables,
along with their corresponding values, is called a no-good. We then
record the no-good, either by adding a new constraint to the CSP or
by keeping a separate cache of no-goods.
•
• Backtracking occurs when no legal assignment can be found for a
variable. Conflict-directed backjumping backtracks directly to the
source of the problem.

22/01/2024 PRNCSE 72
 Local search for CSPs

• Local search algorithms for CSPs use a complete-state formulation:

the initial state assigns a value to every variable, and the search
change the value of one variable at a time.

• The min-conflicts heuristic: In choosing a new value for a variable,

select the value that results in the minimum number of conflicts with
other variables.

22/01/2024 PRNCSE 73
22/01/2024 PRNCSE 74
22/01/2024 PRNCSE 75
• The landscape of a CSP under the min-conflicts heuristic usually has a
series of plateau.

• Simulated annealing and Plateau search (i.e. allowing sideways moves

to another state with the same score) can help local search find its
way off the plateau.

• This wandering on the plateau can be directed with tabu search:

keeping a small list of recently visited states and forbidding the
algorithm to return to those states.

22/01/2024 PRNCSE 76
• Constraint weighting: a technique that can help concentrate
the search on the important constraints.
• Each constraint is given a numeric weight Wi, initially all 1.
• At each step, the algorithm chooses a variable/value pair to
change that will result in the lowest total weight of all
violated constraints.
• The weights are then adjusted by incrementing the weight of
each constraint that is violated by the current assignment.
• Local search can be used in an online setting when the
problem changes, this is particularly important in scheduling
problems.
22/01/2024 PRNCSE 77
 The structure of problem
• Here we examine ways in which the structure of the problem, as represented by
the constraint graph, can be used to find solutions quickly.
• The only way we can possibly hope to deal with the real world is to decompose it
into many sub-problems.
• Observing the constraint graph for Australia one fact stands out: Tasmania is not
connected to the mainland, so coloring them both are independent sub-
problems—any solution for the mainland combined with any solution for
Tasmania yields a solution for the whole map.
• Independence can be ascertained simply by finding connected components of
the constraint graph.

22/01/2024 PRNCSE 78
• If assignment Si is a solution of CSPi, then πiSi is a solution of πi CSPi.

• Suppose each CSPi has c variables from the total of n variables, where c is a
constant. Then there are n/c sub-problems, each of which takes at most dc
work to solve, where d is the size of the domain.

• Hence, the total work is O(dcn/c), which is linear in n.

• But without the decomposition, the total work is O(dn), which is exponential in
n.

• Dividing a Boolean CSP with 80 variables into four sub-problems reduces the
worst-case solution time from the lifetime of the universe down to less than a
second.
22/01/2024 PRNCSE 79
• Completely independent sub-problems are delicious, then, but rare.
• Some other graph structures are also easy to solve.
• For example, a constraint graph is a tree when any two variables are
connected by only one path.
• We show that any tree-structured CSP can be solved in time linear in
the number of variables.
• The key is a new notion of consistency, called directed arc consistency
or DAC.
• A CSP is defined to be directed arc-consistent under an ordering of
variables X1,X2, . . . ,Xn if and only if every Xi is arc-consistent with each
Xj for j > i.

22/01/2024 PRNCSE 80
• To solve a tree-structured CSP, first pick any variable to be the root of
the tree, and choose an ordering of the variables such that each
variable appears after its parent in the tree.
• Such an ordering is called a topological sort.

22/01/2024 PRNCSE 81
• Any tree with n nodes has n−1 arcs, so we can make this graph
directed arc-consistent in O(n) steps, each of which must compare up
to d possible domain values for two variables, for a total time of
O(nd2).
• Once we have a directed arc-consistent graph, we can just march
down the list of variables and choose any remaining value.
• Since each link from a parent to its child is arc consistent, we know
that for any value we choose for the parent, there will be a valid value
left to choose for the child.
• That means we won’t have to backtrack, and we can move linearly
through the variables

22/01/2024 PRNCSE 82
22/01/2024 PRNCSE 83
There are 2 primary ways to reduce more general constraint graphs to
trees:
1. Based on removing nodes
2. Based on collapsing nodes together

22/01/2024 PRNCSE 84
 Based on removing nodes
• For Example delete SA from the graph by fixing a value for SA and
deleting from the domains of other variables any values that are
inconsistent with the value chosen for SA.

22/01/2024 PRNCSE 85
• The general algorithm:
• Choose a subset S of the CSP’s variables such that the constraint
graph becomes a tree after removal of S.
• S is called a cycle cutset.
• For each possible assignment to the variables in S that satisfies all
constraints on S,
(a) remove from the domain of the remaining variables any values
that are inconsistent with the assignment for S, and
(b) If the remaining CSP has a solution, return it together with the
assignment for S.
• Time complexity: O(dc·(n-c)d2), c is the size of the cycle cut set.
• Cutset conditioning: The overall algorithmic approach of efficient
approximation algorithms is to find the smallest cycle cutset.
22/01/2024 PRNCSE 86
 Based on collapsing nodes together
• Tree decomposition: Construct a tree decomposition of the
constraint graph into a set of connected sub-problems, each sub-
problem is solved independently, and the resulting solutions are then
combined.

22/01/2024 PRNCSE 87
 A tree decomposition must satisfy 3 requirements:

• Every variable in the original problem appears in at least one of the

sub-problems.

• If 2 variables are connected by a constraint in the original problem,

they must appear together (along with the constraint) in at least one
of the sub-problems.

• If a variable appears in 2 sub-problems in the tree, it must appear in

every sub-problem along the path connecting those sub-problems.

22/01/2024 PRNCSE 88
• We solve each sub-problem independently.

• If any one has no solution, the entire problem has no solution.

• If we can solve all the sub-problems, then construct a global solution

as follows:
• First, view each sub-problem as a “mega-variable” whose domain
is the set of all solutions for the sub-problem.
• Then, solve the constraints connecting the sub-problems using the
efficient algorithm for trees.

22/01/2024 PRNCSE 89
• A given constraint graph admits many tree decomposition.

• In choosing a decomposition, the aim is to make the sub-problems as small as

possible.

Tree width:

• The tree width of a tree decomposition of a graph is one less than the size of
the largest sub-problems.

• The tree width of the graph itself is the minimum tree width among all its tree
decompositions.

• Time complexity: O(ndw+1), w is the tree width of the graph.

22/01/2024 PRNCSE 90
• The complexity of solving a CSP is strongly related to the structure of
its constraint graph.

• Tree-structured problems can be solved in linear time.

• Cutset conditioning can reduce a general CSP to a tree-structured one

and is quite efficient if a small cutset can be found.

• Tree decomposition techniques transform the CSP into a tree of sub-

problems and are efficient if the tree width of constraint graph is
small.
22/01/2024 PRNCSE 91
 The structure in the values of variables
• By introducing a symmetry-breaking constraint, we can break
the value symmetry and reduce the search space by a factor of n!.
• E.g: Consider the map-coloring problems with n colors, for every
consistent solution, there is actually a set of n! solutions formed by
permuting the color names.(value symmetry)
• On the Australia map, WA, NT and SA must all have different colors,
so there are 3!=6 ways to assign.
• We can impose an arbitrary ordering constraint NT<SA<WA that
requires the 3 values to be in alphabetical order.
• This constraint ensures that only one of the n! solution is possible:
{NT=blue, SA=green, WA=red}. (symmetry-breaking constraint)
22/01/2024 PRNCSE 92