ARTIFICIAL INTELLIGENCE

UNIT - II

Chapter-II: Constraint Satisfaction Problems: Defining Constraint Satisfaction Problems,

Constraint Propagation, Backtracking Search for CSPs, Local Search for CSPs, The Structure of
Problems.

CONSTRAINT SATISFACTION PROBLEMS (CSP’S)

 We have seen many techniques like Local search, Adversarial search to solve different problems.
The objective of every problem-solving technique is to find a solution to reach the goal. Although, in
adversarial search and local search, there were no constraints on the agents while solving the
problems and reaching to its solutions.
 This section focusses another type of problem-solving technique known as Constraint satisfaction
technique. By the name, it is understood that constraint satisfaction means solving a problem under
certain constraints or rules.
 Constraint satisfaction is a technique where a problem is solved when its values satisfy certain
constraints or rules of the problem. Such type of technique leads to a deeper understanding of the
problem structure as well as its complexity.
 A constraint satisfaction problem (CSP) is a problem that requires its solution within some
limitations or conditions also known as constraints.
CSPs represent a state with a set of variable/value pairs and represent the conditions for a
solution by a set of constraints on the variables. Many important real-world problems
can be described as CSPs.

 CSP (constraint satisfaction problem): Use a factored representation (a set of variables, each of
which has a value) for each state, a problem that is solved when each variable has a value that
satisfies all the constraints on the variable is called a CSP.
 It consists of the following:
o X: A finite set of variables which stores the solution (V = {V1, V2, V3,....., Vn})
o D: A set of discrete values known as domain from which the solution is picked (D = {D1, D2,
D3,.....,Dn})
Each domain Di consists of a set of allowable values, {v1, …, vk} for variable Xi.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 o C: A finite set of constraints (C = {C1, C2, C3,......, Cn})
 In constraint satisfaction, domains are the spaces where the variables reside, following the problem
specific constraints.
 The above mentioned are the three main elements of a constraint satisfaction technique. The
constraint value consists of a pair of {scope, rel}.
 The scope is a tuple of variables which participate in the constraint and rel is a relation which
includes a list of values which the variables can take to satisfy the constraints of the problem.

Popular Problems with CSP

The following are some of the popular problems that can be solved using CSP:
 CryptArithmetic (Coding alphabets to numbers.)
 n-Queen (In an n-queen problem, n queens should be placed in an nXn matrix such that no queen
shares the same row, column or diagonal.)
 Map Coloring (coloring different regions of map, ensuring no adjacent regions have the same color)
 Crossword (everyday puzzles appearing in newspapers)
 Sudoku (a number grid)
 Latin Square Problem

An assignment of values to a variable can be done in three ways:

 Consistent or Legal Assignment: An assignment which does not violate any constraint or rule is
called Consistent or legal assignment.
 Complete Assignment: An assignment where every variable is assigned with a value, and the
solution to the CSP remains consistent. Such assignment is known as Complete assignment.
 Partial Assignment: An assignment which assigns values to some of the variables only. Such types
of assignments are called Partial assignments.

Types of Domains in CSP

There are two types of domains which are used by the variables:
 Discrete Domain: It is an infinite domain which can have one state for multiple variables. For
example, a start state can be allocated infinite times for each variable.
 Finite Domain: It is a finite domain which can have continuous states describing one domain for
one specific variable. It is also called a continuous domain.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
Constraint Types in CSP
With respect to the variables, basically there are following types of constraints:
 Unary Constraints: It is the simplest type of constraints that restricts the value of a single variable.
 Binary Constraints: It is the constraint type which relates two variables. A value x2 will contain a
value which lies between x1 and x3.
 Global Constraints: It is the constraint type which involves an arbitrary number of variables.

Some special types of solution algorithms are used to solve the following types of constraints:
 Linear Constraints: These types of constraints are commonly used in linear programming where
each variable containing an integer value exists in linear form only.
 Non-linear Constraints: These types of constraints are used in non-linear programming where each
variable (an integer value) exists in a non-linear form.
Note: A special constraint which works in real-world is known as Preference constraint.

Solving Constraint Satisfaction Problems

The requirements to solve a constraint satisfaction problem (CSP) is:
 A state-space
 The notion of the solution.
A state in state-space is defined by assigning values to some or all variables such as
{X1=v1, X2=v2, and so on…}.
Example: Map colouring

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
To formulate a CSP:
 Define the variables to be the regions X = {WA, NT, Q, NSW, V, SA, T}.
 The domain of each variable is the set Di = {red, green, blue}.
 The constraints is C = {SA≠WA, SAW≠NT, SA≠Q, SA≠NSW, SA≠V, WA≠NT, NT≠Q, Q≠NSW,
NSW≠V}. (SA≠WA is a shortcut for < (SA, WA), SA≠WA>.)
 Constraint graph: The nodes of the graph correspond to variables of the problem, and a link connects
to any two variables that participate in a constraint.

Advantage of formulating a problem as a CSP:

1) CSPs yield a natural representation for a wide variety of problems;
2) CSP solvers can be faster than state-space searchers because the CSP solver can quickly eliminate
large swatches of the search space;
3) With CSP, once we find out that a partial assignment is not a solution, we can immediately discard
further refinements of the partial assignment.
4) We can see why a assignment is not a solution—which variables violate a constraint.

CONSTRAINT PROPAGATION
 In local state-spaces, the choice is only one, i.e., to search for a solution. But in CSP, we have two
choices either:
 We can search for a solution or
 We can perform a special type of inference called constraint propagation.

 A number of inference techniques use the constraints to infer which variable/value pairs are
consistent and which are not. These include node, arc, path, and k-consistent.
 constraint propagation: Using the constraints to reduce the number of legal values for a variable,
which in turn can reduce the legal values for another variable, and so on.
 local consistency: If we treat each variable as a node in a graph and each binary constraint as an arc,
then the process of enforcing local consistency in each part of the graph causes inconsistent values to
be eliminated throughout the graph.
There are different types of local consistency:
 Node Consistency: A single variable is said to be node consistent if all the values in the variable’s
domain satisfy the unary constraints. A network is node-consistent if every variable in the network is
node-consistent.
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  Arc Consistency: A variable is arc consistent if every value in its domain satisfies the binary
constraints. Xi is arc-consistent with respect to another variable Xj if for every value in the current
domain Di there is some value in the domain Dj that satisfies the binary constraint on the arc (Xi,
Xj). A network is arc-consistent if every variable is arc-consistent with every other variable.
 Path Consistency: A two-variable set {Xi, Xj} is path-consistent with respect to a third variable Xm
if, for every assignment {Xi = a, Xj = b} consistent with the constraint on {Xi, Xj}, there is an
assignment to Xm that satisfies the constraints on {Xi, Xm} and {Xm, Xj}.
 k-consistency: A CSP is k-consistent if, for any set of k-1 variables and for any consistent
assignment to those variables, a consistent value can always be assigned to any kth variable.

GLOBAL CONSTRAINTS
 A global constraint is one involving an arbitrary number of variables (but not necessarily all
variables). Global constraints can be handled by special-purpose algorithms that are more efficient
than general-purpose methods.
1) Inconsistency detection for Alldiff constraints

 A simple algorithm: First remove any variable in the constraint that has a singleton domain, and
delete that variable’s value from the domains of the remaining variables. Repeat as long as there
are singleton variables. If at any point an empty domain is produced or there are more vairables
than domain values left, then an inconsistency has been detected.
 A simple consistency procedure for a higher-order constraint is sometimes more effective than
applying arc consistency to an equivalent set of binary constrains.
2) Inconsistency detection for resource constraint (the atmost constraint)

 We can detect an inconsistency simply by checking the sum of the minimum of the current
domains;
e.g. Atmost(10, P1, P2, P3, P4): no more than 10 personnel are assigned in total.
If each variable has the domain {3, 4, 5, 6}, the Atmost constraint cannot be satisfied.
 We can enforce consistency by deleting the maximum value of any domain if it is not consistent
with the minimum values of the other domains.
e.g. If each variable in the example has the domain {2, 3, 4, 5, 6}, the values 5 and 6 can be
deleted from each domain.
3) Inconsistency detection for bounds consistent

 For large resource-limited problems with integer values, domains are represented by upper and
lower bounds and are managed by bounds propagation.
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
 e.g. suppose there are two flights F1 and F2 in an airline-scheduling problem, for which the
planes have capacities 165 and 385, respectively. The initial domains for the numbers of
passengers on each flight are D1 = [0, 165] and D2 = [0, 385].
 Now suppose we have the additional constraint that the two flight together must carry 420
people: F1 + F2 = 420. Propagating bounds constraints, we reduce the domains to D1 = [35, 165]
and D2 = [255, 385].
 A CSP is bounds consistent if for every variable X, and for both the lower-bound and upper-
bound values of X, there exists some value of Y that satisfies the constraint between X and Y for
every variable Y.

BACKTRACKING SEARCH FOR CSPs

 Backtracking search, a form of depth-first search, is commonly used for solving CSPs. Inference can
be interwoven with search.
 Commutativity: CSPs are all commutative. A problem is commutative if the order of application of
any given set of actions has no effect on the outcome.
 Backtracking search: A depth-first search that chooses values for one variable at a time and
backtracks when a variable has no legal values left to assign.
 Backtracking algorithm repeatedly chooses an unassigned variable, and then tries all values in the
domain of that variable in turn, trying to find a solution. If an inconsistency is detected, then
BACKTRACK returns failure, causing the previous call to try another value.
 There is no need to supply BACKTRACKING-SEARCH with a domain-specific initial state, action
function, transition model, or goal test.
 BACKTRACKING-SARCH keeps only a single representation of a state and alters that
representation rather than creating new ones.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
To solve CSPs efficiently without domain-specific knowledge, address following questions:
1) function SELECT-UNASSIGNED-VARIABLE: which variable should be assigned next?
function ORDER-DOMAIN-VALUES: in what order should its values be tried?
2) function INFERENCE: what inferences should be performed at each step in the search?
3) When the search arrives at an assignment that violates a constraint, can the search avoid repeating
this failure?
1. Variable and value ordering

SELECT-UNASSIGNED-VARIABLE
Variable selection—fail-first
 Minimum-remaining-values (MRV) heuristic: The idea of choosing the variable with the fewest
“legal” value. A.k.a. “most constrained variable” or “fail-first” heuristic, it picks a variable that is
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
 most likely to cause a failure soon thereby pruning the search tree. If some variable X has no legal
values left, the MRV heuristic will select X and failure will be detected immediately—avoiding
pointless searches through other variables.
e.g. After the assignment for WA=red and NT=green, there is only one possible value for SA, so it
makes sense to assign SA=blue next rather than assigning Q.
 Degree heuristic: The degree heuristic attempts to reduce the branching factor on future choices by
selecting the variable that is involved in the largest number of constraints on other unassigned
variables. [useful tie-breaker]
e.g. SA is the variable with highest degree 5; the other variables have degree 2 or 3; T has degree 0.

ORDER-DOMAIN-VALUES
Value selection—fail-last
 If we are trying to find all the solution to a problem (not just the first one), then the ordering does not
matter.
 Least-constraining-value heuristic: prefers the value that rules out the fewest choice for the
neighboring variables in the constraint graph. (Try to leave the maximum flexibility for
subsequent variable assignments.)
e.g. We have generated the partial assignment with WA=red and NT=green and that our next choice
is for Q. Blue would be a bad choice because it eliminates the last legal value left for Q’s neighbor,
SA, therefore prefers red to blue.
 The minimum-remaining-values and degree heuristic are domain-independent methods for
deciding which variable to choose next in a backtracking search. The least-constraining-
value heuristic helps in deciding which value to try first for a given variable.

2. Interleaving search and inference

INFERENCE
 forward checking: [One of the simplest forms of inference.] Whenever a variable X is assigned, the
forward-checking process establishes arc consistency for it: for each unassigned variable Y that is
connected to X by a constraint, delete from Y’s domain any value that is inconsistent with the value
chosen for X.
 There is no reason to do forward checking if we have already done arc consistency as a pre-
processing step.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Advantage: For many problems the search will be more effective if we combine the MRV heuristic
with forward checking.
 Disadvantage: Forward checking only makes the current variable arc-consistent, but doesn’t look
ahead and make all the other variables arc-consistent.
 MAC (Maintaining Arc Consistency) algorithm: [More powerful than forward checking, detect
this inconsistency.] After a variable Xi is assigned a value, the INFERENCE procedure calls AC-3,
but instead of a queue of all arcs in the CSP, we start with only the arcs(Xj, Xi) for all Xj that are
unassigned variables that are neighbours of Xi. From there, AC-3 does constraint propagation in the
usual way, and if any variable has its domain reduced to the empty set, the call to AC-3 fails and we
know to backtrack immediately.

3. Intelligent backtracking

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Chronological backtracking: The BACKGRACKING-SEARCH in Fig 6.5. When a branch of the
search fails, back up to the preceding variable and try a different value for it. (The most recent
decision point is revisited.)
e.g. Suppose we have generated the partial assignment {Q=red, NSW=green, V=blue, T=red}. When
we try the next variable SA, we see every value violates a constraint. We back up to T and try a new
color, it cannot resolve the problem.
 Intelligent backtracking: Backtrack to a variable that was responsible for making one of the
possible values of the next variable (e.g. SA) impossible.
 Conflict set for a variable: A set of assignments that are in conflict with some value for that
variable. (e.g. The set {Q=red, NSW=green, V=blue} is the conflict set for SA.)
 Back-jumping method: Backtracks to the most recent assignment in the conflict set. (e.g. back-
jumping would jump over T and try a new value for V.)

Forward checking can supply the conflict set with no extra work.
 Whenever forward checking based on an assignment X=x deletes a value from Y’s domain, add X=x
to Y’s conflict set;
 If the last value is deleted from Y’s domain, the assignment in the conflict set of Y are added to the
conflict set of X.
 In fact, every branch pruned by back-jumping is also pruned by forward checking. Hence simple
back-jumping is redundant in a forward-checking search or in a search that uses stronger consistency
checking (such as MAC).

 Conflict-directed back-jumping: e.g. consider the partial assignment which is proved to be

inconsistent: {WA=red, NSW=red}. We try T=red next and then assign NT, Q, V, SA, no
assignment can work for these last 4 variables.
 Eventually we run out of value to try at NT, but simple back-jumping cannot work because NT
doesn’t have a complete conflict set of preceding variables that caused to fail.
 The set {WA, NSW} is a deeper notion of the conflict set for NT, caused NT together with any
subsequent variables to have no consistent solution. So the algorithm should backtrack to NSW and
skip over T.
 A back-jumping algorithm that uses conflict sets defined in this way is called conflict-direct back-
jumping.
How to Compute:
Mr. Mohammed Afzal, Asst. Professor in AIML
Mob: +91-8179700193, Email: [email protected]
  When a variable’s domain becomes empty, the “terminal” failure occurs, that variable has a standard
conflict set.
 Let Xj be the current variable, let conf(Xj) be its conflict set. If every possible value for Xj fails,
backjump to the most recent variable Xi in conf(Xj), and set
 conf(Xi) ← conf(Xi)∪conf(Xj) – {Xi}.
 The conflict set for an variable means, there is no solution from that variable onward, given the
preceding assignment to the conflict set.
e.g. assign WA, NSW, T, NT, Q, V, SA.
SA fails, and its conflict set is {WA, NT, Q}. (Standard conflict set)
Back-jump to Q, its conflict set is {NT, NSW}∪{WA,NT,Q}-{Q} = {WA, NT, NSW}.
Backtrack to NT, its conflict set is {WA}∪{WA,NT,NSW}-{NT} = {WA, NSW}.
Hence the algorithm back-jump to NSW. (over T)
After back-jumping from a contradiction, how to avoid running into the same problem again:
 Constraint learning: The idea of finding a minimum set of variables from the conflict set that
causes the problem. This set of variables, along with their corresponding values, is called a no-good.
We then record the no-good, either by adding a new constraint to the CSP or by keeping a separate
cache of no-goods.
 Backtracking occurs when no legal assignment can be found for a variable. Conflict-directed back-
jumping backtracks directly to the source of the problem.

LOCAL SEARCH FOR CSPs

 Local search algorithms for CSPs use a complete-state formulation: the initial state assigns a value to
every variable, and the search change the value of one variable at a time.
 The min-conflicts heuristic: In choosing a new value for a variable, select the value that results in
the minimum number of conflicts with other variables.
 Local search techniques can be used in local search for CSPs.
 The landscape of a CSP under the mini-conflicts heuristic usually has a series of plateau. Simulated
annealing and Plateau search (i.e. allowing sideways moves to another state with the same score) can
help local search find its way off the plateau. This wandering on the plateau can be directed
with tabu search: keeping a small list of recently visited states and forbidding the algorithm to
return to those states.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  Constraint weighting: a technique that can help concentrate the search on the important constraints.
 Each constraint is given a numeric weight Wi, initially all 1.
 At each step, the algorithm chooses a variable/value pair to change that will result in the lowest total
weight of all violated constraints.
 The weights are then adjusted by incrementing the weight of each constraint that is violated by the
current assignment.
 Local search can be used in an online setting when the problem changes, this is particularly
important in scheduling problems.

THE STRUCTURE OF PROBLEMS

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
1. The structure of constraint graph
 The structure of the problem as represented by the constraint graph can be used to find solution
quickly.
e.g. The problem can be decomposed into 2 independent subproblems: Colouring T and colouring
the mainland.
 Tree: A constraint graph is a tree when any two variables are connected by only one path.
 Directed arc consistency (DAC): A CSP is defined to be directed arc-consistent under an ordering
of variables X1, X2, … , Xn if and only if every Xi is arc-consistent with each Xj for j>i.
 By using DAC, any tree-structured CSP can be solved in time linear in the number of variables.
How to solve a tree-structure CSP:
 Pick any variable to be the root of the tree;
 Choose an ordering of the variable such that each variable appears after its parent in the tree.
(Topological sort)
 Any tree with n nodes has n-1 arcs, so we can make this graph directed arc-consistent in O(n) steps,
each of which must compare up to d possible domain values for 2 variables, for a total time of
O(nd2).
 Once we have a directed arc-consistent graph, we can just march down the list of variables and
choose any remaining value.
 Since each link from a parent to its child is arc consistent, we won’t have to backtrack, and can move
linearly through the variables.

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
  There are 2 primary ways to reduce more general constraint graphs to trees:
a) Based on removing nodes

 e.g. We can delete SA from the graph by fixing a value for SA and deleting from the
domains of other variables any values that are inconsistent with the value chosen for SA.
The general algorithm:
 Choose a subset S of the CSP’s variables such that the constraint graph becomes a tree
after removal of S. S is called a cycle cutset.
 For each possible assignment to the variables in S that satisfies all constraints on S,

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 (a) remove from the domain of the remaining variables any values that are inconsistent
with the assignment for S, and
(b) If the remaining CSP has a solution, return it together with the assignment for S.
 Time complexity: O(dcꞏ(n-c)d2), c is the size of the cycle cut set.
 Cutset conditioning: The overall algorithmic approach of efficient approximation
algorithms to find the smallest cycle cutset.
b) Based on collapsing nodes together
 Tree decomposition: construct a tree decomposition of the constraint graph into a set
of connected subproblems, each subproblem is solved independently, and the resulting
solutions are then combined.

A tree decomposition must satisfy 3 requirements:

Every variable in the original problem appears in at least one of the subproblems.
If 2 variables are connected by a constraint in the original problem, they must appear together
(along with the constraint) in at least one of the subproblems.
If a variable appears in 2 subproblems in the tree, it must appear in every subproblem along
the path connecting those subproblems.

We solve each subproblem independently.

 If anyone has no solution, the entire problem has no solution.
 If we can solve all the subproblems, then construct a global solution as follows:

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]
 o First, view each subproblem as a “mega-variable” whose domain is the set of all
solutions for the subproblem.
o Then, solve the constraints connecting the subproblems using the efficient algorithm
for trees.

A given constraint graph admits many tree decompositions;

 In choosing a decomposition, the aim is to make the subproblems as small as possible.
Tree width:
The tree width of a tree decomposition of a graph is one less than the size of the largest subproblems.
The tree width of the graph itself is the minimum tree width among all its tree decompositions.
Time complexity: O(ndw+1), w is the tree width of the graph.

The complexity of solving a CSP is strongly related to the structure of its constraint graph. Tree-
structured problems can be solved in linear time. Cutset conditioning can reduce a general CSP to a
tree-structured one and is quite efficient if a small cutset can be found. Tree
decomposition techniques transform the CSP into a tree of subproblems and are efficient if the tree
width of constraint graph is small.
2. The structure in the values of variables
 By introducing a symmetry-breaking constraint, we can break the value symmetry and reduce the
search space by a factor of n!.
 e.g. Consider the map-colouring problems with n colours, for every consistent solution, there is
actually a set of n! solutions formed by permuting the colour names. (value symmetry)
 On the Australia map, WA, NT and SA must all have different colours, so there are 3!=6 ways to
assign.
 We can impose an arbitrary ordering constraint NT<SA<WA that requires the 3 values to be in
alphabetical order. This constraint ensures that only one of the n! solution is possible: {NT=blue,
SA=green, WA=red}. (Symmetry-breaking constraint)

Mr. Mohammed Afzal, Asst. Professor in AIML

Mob: +91-8179700193, Email: [email protected]