Lecture 5: Constraint Satisfaction Problem (CSP)

Heshaam Faili [email protected] University of Tehran

Constraint Satisfaction Problems

Many problems can be formulated as finding a solution, possibly optimal, that satisfies a set of constraints.

built a time table for the next semester with the minimum number of conflicts schedule air crew assignments so that pilots dont fly too many hours, have evenly-spaced breaks, find a 0/1 assignment to a propositional formula 8-queen problem, crypto arithmetic
2

CSP as a search problem (1)

State:

a set of variables x1, x2, ,xn variable domains D1, D2 Dn state vector (x1, x2, x3, xn) with instantiated and uninstantiated variables

Rule: constraints relating the variables

xi xj, xi = 1, min f(x1, x2, ,xn)

Start state: vector of uninstantiated variables Goal state: instantiated vector that satisfies all the constraints
3

Examples of CSP formulation

Time-table schedule
Mon Tue Wed Thu Fri 8 9 10 11

CSP and Constraint Graph

CSP as a search problem (2)

Simplest form: discrete and finite domain For n variables and domain size d the possible assignments are O(dn) Boolean CSP: domains are boolean, like 3SAT , special case of NPcomplete Infinite domain: no exact solution can be found

Continuous domains:

Linear constraints Non-linear constraints

Constraints:

The best-known category is linear programming Constraints are linear inequalities forming a convex region Unary: involve only one variable: can be eliminated by remove any value violate the constraint from the domain of that variable Binary: involves two variables (like 8-queen) Higher-order : like crypt arithmetic

Higher-order Constraints

Can be represented by a constraints hyper-graph every higher-order, finite-domain constraint can be reduced to a set of binary constraints if enough auxiliary variables are introduced, prove it!

Backtracking in CSP

For n variables with d value in each domain

Commutativity: if the order of application of any given set of actions has no effect on the outcome. CSPs are commutative

Branching factor for level 1 is nd, level 2 is (n-1)d, level 3 is (n-2)d, Tree has n!dn leaves while there are only dn possible assignments

all CSP search algorithms generate successors by considering possible assignments for only a single variable at each node in the search tree In root node of Australia region coloring: we choice between SA = red, SA = green, and SA = blue, and we never choose between SA = red and WA = blue. The number of leaves are dn

CSP as a search problem

Search strategy
1. pick an uninstantiated variable, pick a value from its domain, check if all constraints are satisfied 2. if constraints are satisfied, continue search recursively 3. else, backtrack: go back to previous decision and make another choice

Size of search tree:

bd

d = n, number of variables b = max Di when only one variable at a time is

Backtracking Search

10

CSP search example

11

CSP search example

M=1

N=2

S32D
E=3

10R3 1023Y Conflict!

CSP problem characteristics (1)

Special case of a search problem Domains can be discrete or continuous The order in which the constraints are satisfied does not matter. The order in which variables and their values are picked makes a big difference!

13

CSP problem characteristics (2)

The goal test is decomposed into a set of constraints on variables, rather than a single black box When sets of variables are independent (no constraints between them) the problem is decomposable and subproblems can be solved independently. At each step we must check for consistency. We need constraint propagation methods!

14

CSP problem characteristics (3)

Answering the following questions are important key:

Which variable should be assigned next, and in what order should its values be tried? What are the implications of the current variable assignments for the other unassigned variables? When a path failsthat is, a state is reached in which a variable has no legal values can the search avoid repeating this failure in subsequent paths?
15

Comparison of various CSP algorithms

16

Variable ordering

Pick values for the minimum remaining values (MRV) or most constrained variables or first fail principle

choosing the variable with the fewest "legal" value those with smaller domains and which appear in more constraints. The rationale: those leave fewer choices and focus the search

Improve the search from 3 til 3000 times better than simple
17

Variable ordering

MRV doesnt help in choosing the first variable

Degree heuristic: attempts to reduce the BF in feature by selecting the variables that is involved in the largest number of constraints on other unassigned variables

SA on region coloring example

18

Value ordering

least-constraining-value: prefers the value that rules out the fewest choices for the neighboring variables in the constraint In the previous coloring problem:

WA = red and NT = green, Q is better to assign red than blue

if we are trying to find all the solutions to a problem, not just the first one, then the ordering does not matter because we have to consider every value anyway.
19

Propagating information through constraints (1)

Forward checking: delete values from domains as the assignments are made. When a domain becomes empty, abort.

Often more efficient than backtracking

20

Forward checking example

21

Propagating information through constraints (2)

Forward Checking: doesn't detect all of inconsistencies.

WA = red, Q=green => NT = SA = blue (this inconsistency doesnt detected)

Arc-consistency: a state is arc-consistent if every variable has a value in its domain that is consistent with each of the constraints of that variable. This is a form of constraint propagation. Arc-consistency provides a fast method of constraint propagation that is substantially stronger than forward checking. arc-consistency is directed arc in constraint graph (SA, NSW) is consistent while (NSW,SA) is not consistent 22 in the third row of Figure 5.6

Arc-Consistency

AC-3 can not find harder inconsistency (WA=red, NSW=red) Need more concepts

23

Maintaining arc consistency (MAC)

Arc-consistency can be applied after every assignment during search: MAC Complexity for binary CSP

Binary CSP has at most O(n2) arcs Each arc (Xi,Xk) can be inserted on the agenda only d times: Xi has at most d values to delete Checking consistency of an arc can be done in O(d2) Total O(n2d3)
24

CSP terminology

Label <xi, a> Compound Label { <xi, a> } K-Constraint Modeling 8-Queen problem:

Q1,Q2, ,Q8 in [a,b,,h] Qx Qy , |Qx-Qy| |x-y|

25

Important Concepts

K-Satisfiable K-Consistency

K=1: node consistency 1-Consistent: node consistent K>1: for each K-1 satisfiable variable set can add 2-consistent: arc-consistent the Kth variable with consistent value and no 3-consistent: path consistent change in the previous K-1 variable values

K-strongly consistent

A {red, green} C{red} B{red}

3-Consistent But not 2-Consistent

26

Handling special constraints

some problem-specific constraints are used much in different problems

Use the following idea to develop a method:

AllDiff constraints says that all variables should be different If n variables with m values and n>m, the constraints should not be satisfied
First, remove any variable in the constraint that has a singleton domain, and delete that variable's value from the domains of the remaining variables. Repeat as long as there are singleton variables. If at any point an empty domain is produced or there are more variables than domain values left, then an inconsistency has been detected.

By using the above method, the inconsistency (WA=red, NSW=red) can be detected. Because all SA, NT and Q has domain {green, blue}.

3 variables and 2 values : inconsistency

27

Handling special constraints

Resource constraints: the most important higherorder constraints

For example, let's suppose there are two flights, 271 and 272, for which the planes have capacities 165 and 385, respectively. The initial domains for the numbers of passengers on each flight are then

Checking the sum of minimum values of current domains (in each step of procedure)

Now suppose we have the additional constraint that the two flights together must carry 420 people: Flight271 + Flight272 [420,420]. Propagating bounds constraints, we reduce the domains to

Flight271 [0,165] and Flight272 [0, 385]

Flight271 [35,165] and Flight272 [255, 385]

28

CSP heuristics: example

M=1

C1 = 1 C2 + S + 1 = O

N=2 E=3

C2 + S + 1 = 1 C3 + E + 0 = N C4 + N + R = E

Conflict is detected as soon as the assignement is made

29

Graph of variables dependencies

U
30

Intelligent backtracking: looking backward

Chronological backtracking: Simple BT More intelligent: use conflict set

Backjumping method: backtrack to the most recent

conflict set for variable X is the set of previously assigned variables that are connected to X by constraints.

variable in conflict set

Is Backjumping equal to forward checking?

Backjumping occurs when every value in a domain is in conflict with the current assignment; but forward checking detects this event and prevents the search from ever reaching such a node! every branch pruned by backjumping is also pruned by forward checking.
31

Min-conflicts heuristics

Select the value that results in the minimum number of conflicts with other variables Surprisingly effective for many CSPs

Independent of problem size Roughly speaking, n-queens is easy for local search because solutions are densely distributed throughout the state space.

32

Min-conflicts heuristics

33

Min-conflict strategy on 8queens

34

Important concept (constraint graph)

Graph Order Node width Graph Order width Graph Width Value ordering for CSP search in the minimum Graph order width If the level of strong consistency is greater than the width of constraint graph, the search for solution tuple is backtrack free
35

Structure of the problem

Connected component in constraint graph is same to independence sub problem For subproblem CSPi

For tree constraint graph:

c variables there are n/c problems Each takes dc works Total: O(dcn/c) E.g; boolean CSP with n=80, c=20 decrease the time from lifetime of world to less than a second Only one path between any two variables tree-structured CSP can be solved in time linear in the number of variables.

36

Algorithm for tree-structured CSP

1. Choose any variable as the root of the tree, and order the variables from the root to the leaves in such a way that every node's parent in the tree precedes it in the ordering.

2. For j from n down to 2, apply arc consistency to the arc (Xi,Xj), where Xi is the parent of Xj, removing values from DOMAIN[Xj] as necessary. 3. For j from 1 to n, assign any value for Xj consistent with the value assigned for Xi, where Xj is the parent of Xj.

Label the variables X1,..., Xn in order. Now, every variable except the root has exactly one parent variable.

Complexity : O(nd2)
37

tree-structured CSP

38

More constrained CSP

constraint graphs can be reduced to trees

based on removing nodes

Delete SA in constraint graph of Australia by fixing the value

based on collapsing nodes

39

Converting more general CG into tree

40

More general algorithm to change CG into tree

1. Choose a subset S from VARIABLES[csp] such that the constraint graph becomes a tree after removal of S. S is called a cycle cutset. 2. For each possible assignment to the variables in S, that satisfies all constraints on S

(a) remove from the domains of the remaining variables any values that are inconsistent with the assignment for S (b) If the remaining CSP has a solution, return it together with the assignment for S.
41

More general algorithm to change CG into tree

For Cycle cutset (size = c) => O(dc.(n-c)d2)

If the graph is "nearly a tree" then c will be small and the savings over straight backtracking will be huge. In the worst case, c =(n 2)

Finding the smallest cycle cutset is NP-hard but several efficient approximation algorithms are known for this task.

42

Second approach on changing more general CG into tree (based on collapsing nodes )

Tree decomposition: should satisfies the following:

Every variable in the original problem appears in at least one of the sub-problems. If two variables are connected by a constraint in the original problem, they must appear together (along with the constraint) in at least one of the sub-problems. If a variable appears in two sub-problems in the tree, it must appear in every sub-problem along the path connecting those sub-problems. (SA in following example)
43

Tree decomposition

44

Solving the tree decomposition

Using the concept of mega-variables Each sub-problem can be a megavariable Tree width: size of the largest subproblem For graph with w width: problem can be solved in O(ndw+1)
45

CSP problems at large

An area of great economic importance

scheduling, production, etc. integer programming multiobjective optimization

Closely related to optimization problems

Large body of literature in different areas: AI, CS, Operations Research, Mathematics,
46

?
47