Constraint Satisfaction Problems: Version 2 CSE IIT, Kharagpur
Constraint Satisfaction Problems: Version 2 CSE IIT, Kharagpur
com
www.jwjobs.net
Module 4
Constraint satisfaction problems
Version 2 CSE IIT, Kharagpur www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
Lesson 9
Constraint satisfaction problems - I
Version 2 CSE IIT, Kharagpur www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
Note that the 6 queens puzzle has, interestingly, fewer solutions than the 5 queens puzzle! Example 2: A crossword puzzle: We are to complete the puzzle
1 2 3 4 5 +---+---+---+---+---+ | 1 | | 2 | | 3 | +---+---+---+---+---+ | # | # | | # | | +---+---+---+---+---+ | # | 4 | | 5 | | +---+---+---+---+---+ | 6 | # | 7 | | | +---+---+---+---+---+ | 8 | | | | | +---+---+---+---+---+ | | # | # | | # | +---+---+---+---+---+ Given the list AFT ALE EEL HEEL HIKE HOSES KEEL KNOT of words: LASER LEE LINE SAILS SHEET STEER TIE
1 2 3 4 5 6
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
The numbers 1,2,3,4,5,6,7,8 in the crossword puzzle correspond to the words that will start at those locations. Example 3: A map coloring problem: We are given a map, i.e. a planar graph, and we are told to color it using k colors, so that no two neighboring countries have the same color. Example of a four color map is shown below:
The four color theorem states that given any plane separated into regions, such as a political map of the countries of a state, the regions may be colored using no more than four colors in such a way that no two adjacent regions receive the same color. Two regions are called adjacent if they share a border segment, not just a point. Each region must be contiguous: that is, it may not consist of separate sections like such real countries as Angola, Azerbaijan, and the United States. It is obvious that three colors are inadequate: this applies already to the map with one region surrounded by three other regions (even though with an even number of surrounding countries three colors are enough) and it is not at all difficult to prove that five colors are sufficient to color a map. The four color theorem was the first major theorem to be proved using a computer, and the proof is not accepted by all mathematicians because it would be infeasible for a human to verify by hand. Ultimately, one has to have faith in the correctness of the compiler and hardware executing the program used for the proof. The lack of Version 2 CSE IIT, Kharagpur
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
mathematical elegance was another factor, and to paraphrase comments of the time, "a good mathematical proof is like a poem this is a telephone directory!" Example 4: The Boolean satisfiability problem (SAT) is a decision problem considered in complexity theory. An instance of the problem is a Boolean expression written using only AND, OR, NOT, variables, and parentheses. The question is: given the expression, is there some assignment of TRUE and FALSE values to the variables that will make the entire expression true? In mathematics, a formula of propositional logic is said to be satisfiable if truth-values can be assigned to its variables in a way that makes the formula true. The class of satisfiable propositional formulas is NP-complete. The propositional satisfiability problem (SAT), which decides whether or not a given propositional formula is satisfiable, is of central importance in various areas of computer science, including theoretical computer science, algorithmics, artificial intelligence, hardware design and verification. The problem can be significantly restricted while still remaining NP-complete. By applying De Morgan's laws, we can assume that NOT operators are only applied directly to variables, not expressions; we refer to either a variable or its negation as a literal. For example, both x1 and not(x2) are literals, the first a positive literal and the second a negative literal. If we OR together a group of literals, we get a clause, such as (x1 or not(x2)). Finally, let us consider formulas that are a conjunction (AND) of clauses. We call this form conjunctive normal form. Determining whether a formula in this form is satisfiable is still NP-complete, even if each clause is limited to at most three literals. This last problem is called 3CNFSAT, 3SAT, or 3-satisfiability. On the other hand, if we restrict each clause to at most two literals, the resulting problem, 2SAT, is in P. The same holds if every clause is a Horn clause; that is, it contains at most one positive literal. Example 5: A cryptarithmetic problem: In the following pattern
S E N D M O R E ========= M O N E Y
we have to replace each letter by a distinct digit so that the resulting sum is correct. All these examples and other real life problems like time table scheduling, transport scheduling, floor planning etc are instances of the same pattern, captured by the following definition: A Constraint Satisfaction Problem (CSP) is characterized by:
a set of variables {x1, x2, .., xn}, for each variable xi a domain Di with the possible values for that variable, and
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
a set of constraints, i.e. relations, that are assumed to hold between the values of the variables. [These relations can be given intentionally, i.e. as a formula, or extensionally, i.e. as a set, or procedurally, i.e. with an appropriate generating or recognising function.] We will only consider constraints involving one or two variables.
The constraint satisfaction problem is to find, for each i from 1 to n, a value in Di for xi so that all constraints are satisfied. A CSP can easily be stated as a sentence in first order logic, of the form:
(exist x1)..(exist xn) (D1(x1) & .. Dn(xn) => C1..Cm)
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
VARIABLE | STARTING CELL | DOMAIN ================================================ 1ACROSS | 1 | {HOSES, LASER, SAILS, SHEET, STEER} 4ACROSS | 4 | {HEEL, HIKE, KEEL, KNOT, LINE} 7ACROSS | 7 | {AFT, ALE, EEL, LEE, TIE} 8ACROSS | 8 | {HOSES, LASER, SAILS, SHEET, STEER} 2DOWN | 2 | {HOSES, LASER, SAILS, SHEET, STEER} 3DOWN | 3 | {HOSES, LASER, SAILS, SHEET, STEER} 5DOWN | 5 | {HEEL, HIKE, KEEL, KNOT, LINE} 6DOWN | 6 | {AFT, ALE, EEL, LEE, TIE}
The domain of each variable is the list of words that may be the value of that variable. So, variable 1ACROSS requires words with five letters, 2DOWN requires words with five letters, 3DOWN requires words with four letters, etc. Note that since each domain has 5 elements and there are 8 variables, the total number of states to consider in a naive approach is 58 = 390,625. The constraints are all binary constraints:
1ACROSS[3] equal to the first 1ACROSS[5] 4ACROSS[2] 4ACROSS[3] 4ACROSS[4] 7ACROSS[1] 7ACROSS[2] 7ACROSS[3] 8ACROSS[1] 8ACROSS[3] 8ACROSS[4] 8ACROSS[5] = 2DOWN[1] i.e. the third letter of 1ACROSS must be letter of 2DOWN = 3DOWN[1] = 2DOWN[3] = 5DOWN[1] = 3DOWN[3] = 2DOWN[4] = 5DOWN[2] = 3DOWN[4] = 6DOWN[2] = 2DOWN[5] = 5DOWN[3] = 3DOWN[5]
1A
2D
4A
3D
8A 7A 5D
6D
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
4.4.2 Backtracking
We order the variables in some fashion, trying to place first the variables that are more highly constrained or with smaller ranges. This order has a great impact on the efficiency of solution algorithms and is examined elsewhere. We start assigning values to variables. We check constraint satisfaction at the earliest possible time and extend an assignment if the constraints involving the currently bound variables are satisfied. Example 2 Revisited: In our crossword puzzle we may order the variables as follows: 1ACROSS, 2DOWN, 3DOWN, 4ACROSS, 7ACROSS, 5DOWN, 8ACROSS, 6DOWN. Then we start the assignments:
1ACROSS 2DOWN 2DOWN[1] <- HOSES <- HOSES => failure, 1ACROSS[3] not equal to
<- LASER => failure <- SAILS 3DOWN <- HOSES => failure <- LASER => failure <- SAILS 4ACROSS <- HEEL => failure <- HIKE => failure <- KEEL => failure <- KNOT => failure <- LINE => failure, backtrack 3DOWN <- SHEET 4ACROSS <- HEEL 7ACROSS <- AFT => failure ................................
What we have shown is called Chronological Backtracking, whereby variables are unbound in the inverse order to the the order used when they were bound. Dependency Directed Backtracking instead recognizes the cause of failure and backtracks to one of the causes of failure and skips over the intermediate variables that did not cause the failure.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
The following is an easy way to do dependency directed backtracking. We keep track at each variable of the variables that precede it in the backtracking order and to which it is connected directly in the constraint graph. Then, when instantiation fails at a variable, backtracking goes in order to these variables skipping over all other intermediate variables. Notice then that we will backtrack at a variable up to as many times as there are preceding neighbors. [This number is called the width of the variable.] The time complexity of the backtracking algorithm grows when it has to backtrack often. Consequently there is a real gain when the variables are ordered so as to minimize their largest width.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
Arc (Vi,Vj) is arc consistent if for every value x the current domain of Vi there is some value y in the domain of Vj such that Vi=x and Vj=y is permitted by the binary constraint between Vi and Vj. Note, that the concept of arc-consistency is directional, i.e., if an arc (Vi,Vj) is consistent, than it does not automatically mean that (Vj,Vi) is also consistent. Clearly, an arc (Vi,Vj) can be made consistent by simply deleting those values from the domain of Vi for which there does not exist corresponding value in the domain of Dj such that the binary constraint between Vi and Vj is satisfied (note, that deleting of such values does not eliminate any solution of the original CSP). The following algorithm does precisely that.
Algorithm REVISE
procedure REVISE(Vi,Vj) DELETE <- false; for each X in Di do if there is no such Y in Dj such that (X,Y) is consistent, then delete X from Di; DELETE <- true; endif; endfor; return DELETE; end REVISE
To make every arc of the constraint graph consistent, it is not sufficient to execute REVISE for each arc just once. Once REVISE reduces the domain of some variable Vi, then each previously revised arc (Vj,Vi) has to be revised again, because some of the members of the domain of Vj may no longer be compatible with any remaining members of the revised domain of Vi. The following algorithm, known as AC-1, does precisely that.
Algorithm AC-1
procedure AC-1 Q <- {(Vi,Vj) in arcs(G),i#j}; repeat CHANGE <- false; for each (Vi,Vj) in Q do CHANGE <- REVISE(Vi,Vj) or CHANGE; endfor until not(CHANGE) end AC-1
This algorithm is not very efficient because the succesfull revision of even one arc in some iteration forces all the arcs to be revised again in the next iteration, even though only a small number of them are really affected by this revision. Visibly, the only arcs affected by the reduction of the domain of Vk are the arcs (Vi,Vk). Also, if we revise the Version 2 CSE IIT, Kharagpur
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
arc (Vk,Vm) and the domain of Vk is reduced, it is not necessary to re-revise the arc (Vm,Vk) because non of the elements deleted from the domain of Vk provided support for any value in the current domain of Vm. The following variation of arc consistency algorithm, called AC-3, removes this drawback of AC-1 and performs re-revision only for those arcs that are possibly affected by a previous revision.
Algorithm AC-3
procedure AC-3 Q <- {(Vi,Vj) in arcs(G),i#j}; while not Q empty select and delete any arc (Vk,Vm) from Q; if REVISE(Vk,Vm) then Q <- Q union {(Vi,Vk) such that (Vi,Vk) in arcs(G),i#k,i#m} endif endwhile end AC-3
When the algorithm AC-3 revises the edge for the second time it re-tests many pairs of values which are already known (from the previous iteration) to be consistent or inconsistent respectively and which are not affected by the reduction of the domain. As this is a source of potential inefficiency, the algorithm AC-4 was introduced to refine handling of edges (constraints). The algorithm works with indiviual pairs of values as the following example shows.
Example:
First, the algorithm AC-4 initializes its internal structures which are used to remember pairs of consistent (inconsistent) values of incidental variables (nodes) - structure Si,a. This initialization also counts "supporting" values from the domain of incindental variable - structure counter(i,j),a - and it removes those values which have no support. Once the value is removed from the domain, the algorithm adds the pair <Variable,Value> to the list Q for re-revision of affected values of corresponding variables.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
Algorithm INITIALIZE
procedure INITIALIZE Q <- {}; S <- {}; % initialize each element of structure S for each (Vi,Vj) in arcs(G) do % (Vi,Vj) and (Vj,Vi) are same elements for each a in Di do total <- 0; for each b in Dj do if (a,b) is consistent according to the constraint (Vi,Vj) then total <- total+1; Sj,b <- Sj,b union {<i,a>}; endif endfor; counter[(i,j),a] <- total; if counter[(i,j),a]=0 then delete a from Di; Q <- Q union {<i,a>}; endif; endfor; endfor; return Q; end INITIALIZE
After the initialization, the algorithm AC-4 performs re-revision only for those pairs of values of incindental variables that are affected by a previous revision.
Algorithm AC-4
procedure AC-4 Q <- INITIALIZE; while not Q empty select and delete any pair <j,b> from Q; for each <i,a> from Sj,b do counter[(i,j),a] <- counter[(i,j),a] - 1; if counter[(i,j),a]=0 & a is still in Di then delete a from Di; Q <- Q union {<i,a>}; endif endfor endwhile end AC-4
Both algorithms, AC-3 and AC-4, belong to the most widely used algorithms for maintaining arc consistency. It should be also noted that there exist other algorithms AC5, AC-6, AC-7 etc. but their are not used as frequently as AC-3 or AC-4. Maintaining arc consistency removes many inconsistencies from the constraint graph but is any (complete) instantiation of variables from current (reduced) domains a solution to the CSP? If the domain size of each variable becomes one, then the CSP has exactly one solution which is obtained by assigning to each variable the only possible value in its Version 2 CSE IIT, Kharagpur
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
domain. Otherwise, the answer is no in general. The following example shows such a case where the constraint graph is arc consistent, domains are not empty but there is still no solution satisfying all constraints.
Example:
This constraint graph is arc consistent but there is no solution that satisfies all the constraints.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
is also exponential. If the graph is (strongly) K-consistent for K<n, then in general, backtracking cannot be avoided, i.e., there still exist inconsistent values.
Forward checking detects the inconsistency earlier than simple backtracking and thus it allows branches of the search tree that will lead to failure to be pruned earlier than with simple backtracking. This reduces the search tree and (hopefully) the overall amount of work done. But it should be noted that forward checking does more work when each assignment is added to the current partial solution.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
Forward checking is almost always a much better choice than simple backtracking.
www.jntuworld.com
www.jntuworld.com
www.jwjobs.net
Look ahead prunes the search tree further more than forward checking but, again, it should be noted that look ahead does even more work when each assignment is added to the current partial solution than forward checking.
More constraint propagation at each node will result in the search tree containing fewer nodes, but the overall cost may be higher, as the processing at each node will be more expensive. In one extreme, obtaining strong n-consistency for the original problem would completely eliminate the need for search, but as mentioned before, this is usually more expensive than simple backtracking. Actually, in some cases even the full look ahead may be more expensive than simple backtracking. That is the reason why forward checking and simple backtracking are still used in applications.
www.jntuworld.com