CZ3005

Artificial Intelligence

Constraint satisfaction and

adversarial search

Asst/P Hanwang Zhang

https://personal.ntu.edu.sg/hanwangzhang/
Email: [email protected]
Office: N4-02c-87
Lesson Outline

• Constraint Satisfaction
• Adversarial search (Game Playing)
Constraint Satisfaction Problem (CSP)
Goal: discover some state that satisfies a given set of
constraints
Example: Sudoku Example: Minesweeper
              1 3
      7         6
      5   9      
       
R
    9    
R
1   6            
            2    
7 4           5  
  8         4    
        1        
Examples: Real-world CSPs
• Assignment problems
• e.g. who teaches what class
• Timetabling problems
• e.g. which class is offered when and where?
• Hardware configuration
• Transportation scheduling
• Factory scheduling
• Floor-planning
CSP
State
• defined by variables with values from domain
Example: 8-queens
• Variables: locations of each of the eight queens
• Values: squares on the board

Goal test
• a set of constraints specifying allowable combinations of values for subsets of
variables
Example: 8-queens
R
• Goal test: No two queens in the same row, column or diagonal
Example: Cryptarithmetic Puzzle

S E N D
+ M O R E
M O N E Y

 Variables: D, E, M, N, O, R, S, Y
 Value ∈ Domain: {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
 Constraints
 Y = D + E or Y = D + E - 10, etc
 D ≠ E, D ≠ M, D ≠ N, etc.
 M ≠ 0, S ≠ 0 (unary constraints: concern the value of a
single variable)
Example: Map Colouring
Colour a map so that no adjacent parts have the same colour

• Variables: Countries Ci
• Domains: {Red, Blue, Green}
• Contraints: C1 ≠ C2, C1 ≠ C5, etc.
• binary constraints
Some Definitions

• A state of the problem is defined by an assignment of values

to some or all of the variables.
• An assignment that does not violate any constraints is called
a consistent or legal assignment.
• A solution to a CSP is an assignment with every variable
given a value (complete) and the assignment satisfies all the
constraints.
Applying Standard Search

• States: defined by the values assigned so far

• Initial state: all variables unassigned
• Actions: assign a value to an unassigned variable
• Goal test: all variables assigned, no constraints violated
Applying Standard Search
Question: How to represent constraints?
Answer: Explicitly (e.g., D ≠ E)

Example
• Row the 1st queen occupies: ∈ {1, 2, 3, 4, 5, 6, 7, 8 }
(similarly, for )
R

• No-attack constraint for and :

{ <1, 3> , <1, 4>, <1, 5>, … , <2, 4>, <2, 5>, …}

Implicitly: use a function to test for constraint satisfaction

Explicit or Implicit?
Backtracking Search
Backtracking search: Do not waste time searching when constraints have
already been violated

Initial State • Before generating

successors, check for
constraint violations
S1 Go back to the last decision
point • If yes, backtrack to try
something else
S2 X
Something goes wrong
Example (4-Queens)
       
       
       
       

       
       
       
       
       
       
       
       

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
           
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
       
                                                               
                                               
               

                                                                       
                                                                       
                                                                       
                                                                       

                                                       
                                                       
                                                       
                                                       
Heuristics for CSPs
Plain backtracking is an uninformed algorithm!!
More intelligent search that takes into consideration
• Which variable to assign next
• What order of the values to try for each variable
• Implications of current variable assignments for the other unassigned
variables
• forward checking and constraint propagation

Constraint propagation: propagating the implications of a constraint

on one variable onto other variables
Example (4-Queens) without Constraint Propagation
       
       
       
       

       
       
       
       
       
       
       
       

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                           
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
                       
                                                               
                                                               

                                                                       
                                                                       
                                                                       
                                                                       

                                                       
                                                       
                                                       
                                                       
Search Tree of 4-Queens with Constraint
Propagation
       
       
       
       

               
               
               
               

       
               
       
               
       
               
       
               

               
               
               
               
Example (Map Colouring)

NT
Northern
Territory Q
Queensland WA
Western
Australia South
SA NSW
Australia
New South
Wales
V
Victoria

T
Tasmania
Example (Map Colouring)...

NT
Q
WA
SA
NSW
V

T
WA NT Q NSW V SA T
Example (Map Colouring)...

WA NT Q NSW V SA T
Example (Map Colouring)...

WA NT Q NSW V SA T
Example (Map Colouring)...

WA NT Q NSW V SA T
Search Tree of 4-Queens with Constraint
Propagation
       
       
       
       

               
               
               
               

       
               
       
               
       
               
       
               

               
               
               
               
Most Constraining Variable

Example: map colouring

To reduce the branching factor on future choices by selecting the variable that is
involved in the largest number of constraints on unassigned variables.
Least Constraining Value

Allow 1 value
for SA

Allow 0 values
for SA

Choose the value that leaves maximum flexibility for subsequent variable
assignments
Min-Conflicts Heuristic (8-queens)
• A local heuristic search method for solving CSPs
• Given an initial assignment, selects a variable in the scope of a violated
constraint and assigns it to the value that minimises the number of violated
constraints

2 3
2 3
1
2 2
3 3
1 2
2 3
0
Games as Search Problems
Abstraction
• Ideal representation of real world problems
• e.g. board games, chess, go, etc. as an abstraction of war games
• Perfect information, i.e. fully observable
• Accurate formulation: state space representation

Uncertainty
• Account for the existence of hostile agents (players)
• Other agents acting so as to diminish the agent's well-being
• Uncertainty (about other agents' actions):
• not due to the effect of non-deterministic actions
• not due to randomness
 Contingency problem
Games as Search Problems...
Complexity
• Games are abstract but not simple
• e.g. chess: average branching factor = 35, game length > 50
 complexity = (only for legal moves)

• Games are usually time limited

• Complete search (for the optimal solution) not possible
 uncertainty on actions desirability
• Search efficiency is crucial
Types of Games

Deterministic Chance

Perfect Chess, Backgammon,

information Checkers, Monopoly
Go, Othello
Imperfect Bridge, Poker,
information Scrabble, Nuclear war

In this course, we only focus on Perfect information

• each player has complete information about his opponent's position and
about the choices available to him
Game as a Search Problem
• Initial state: initial board configuration and indication of who makes the first
move
• Operators: legal moves
• Terminal test: determines when the game is over
• states where the game has ended: terminal states
• Utility function (payoff function): returns a numeric score to quantify the
outcome of a game

Example: Chess
R
Win (+1), loss(-1) or draw (0)
Game Tree for Tic-Tac-Toe
     
MAX(X)      
     

X       X       X                                    
MIN(O)                   X       X       X                  
                                    X       X       X

X O   X   O X    
      O    
MAX(X)      
                 
…
• The utility value of the terminal state
X O X X O   X O   is from the point of view of MAX
MIN(O)       X       X   …
                 

… … … …
• MAX uses the search tree to
determine the best move
X O X X O X X O X
TERMINAL …
  O X O O X   X  
  O   X X O X O O

Utility -1 0 +1
A Toy Min-Max Example
A Toy Min-Max Example

Step 1:
Expand the whole game tree.
Initialize each node to –Inf for max
+Inf for min
A Toy Min-Max Example

Step 2:
Update the value from btm to top

e.g., for node D:

We want to winmax the value (-1, 4)
A Toy Min-Max Example

Step 2:
Update the value from btm to top

e.g., for node D:

I want to winmax the value (-1, 4)

e.g., for node B:

You want me to lose  min the value (4, 6)
A Toy Min-Max Example
Minimax Search Strategy

Search strategy
• Find a sequence of moves that leads to a terminal state (goal)

Minimax search strategy

• Maximise one's own utility and minimise the opponent's
• Assumption is that the opponent does the same
Minimax Search Strategy
3-step process
1. Generate the entire game tree down to terminal states

2. Calculate utility
a) Assess the utility of each terminal state
b) Determine the best utility of the parents of the terminal state
c) Repeat the process for their parents until the root is reached

3. Select the best move (i.e. the move with the highest utility value)
Perfect Decisions by Minimax Algorithm
Perfect decisions: no time limit is Example
imposed
• generate the complete MAX 3
search tree 𝐴1
𝐴2
𝐴3

Two players: MAX and MIN MIN

R
3 2 2

• Choose move with best

achievable payoff against best 𝐴11 𝐴12 𝐴13 𝐴21 𝐴 22 𝐴23 𝐴31 𝐴32 𝐴33
play
• MAX tries to max the utility, 12 14

assuming that MIN will try to

min it
Othello 4
• A player can place a new piece in a position if there
exists at least one straight (horizontal, vertical, or
diagonal) occupied line between the new piece and
       
another piece of the same kind, with one or more
 
contiguous pieces from the opponent player between
X O  
them
 
O X  

       
• After placing the new piece, the pieces from the
opponent player will be captured and become the
pieces from the same Player

• The player with the most pieces on the board wins

`X' plays first
       

X considers the game now   X O  

  O X  

       

       
O considers the game now   X O  
  X X  
  X    
                       

X considers the game now   O     X O     X O  

  X     X     X  

                       
Game Tree Othello 4
       
  X O  
MAX(X)   O X  
       

       
  X O  
MIN(O)   X X  
  X    

                       
MAX(X) O O O     X O     X O  
  X X     O X     X O  
  X     O X       X O  

X         X       X     X   X      
O X O   O X O   O O X   O O X   X X   X X X X O   X O  
  X X     X X     X X     X X     X O     X X     X X X   X X  
  X       X       X       X       X O     X O     X O     X X X

    X  
X O   X X   X X X
X X X     O X     O X  
MIN(O) O X     O X     O X    
Imperfect Decisions

For chess, branching factor ≈ 35, each player typically makes 50 moves  for the
complete game tree, need to examine positions
Time/space requirements  complete game tree search is intractable 
impractical to make perfect decisions
Modifications to minimax algorithm
1. replace utility function by an estimated desirability of the position
• Evaluation function
2. partial tree search
• E.g., depth limit
• Replace terminal test by a cut-off test
Evaluation Functions
Returns an estimate of the expected utility of the game from a given position

Black: to move White: to move

White: slightly better Black: winning
Evaluation Functions...
Requirements
 Computation is efficient
 Agrees with utility function on terminal states
 Accurately reflects the chances of winning
Trade off between accuracy and efficiency

Example (Chess)
Define features
 e.g. , (number of white queens) – (number of black queens)

Use a weighted sum of these features R

 Need to learn the weight

Evaluation Functions for Othello 4
 A corner of the board is one of the most important positions. A piece
at the corner can help capture other pieces from the opponent
player
 A square at the border is also more important than any position in
the middle of the board

Heuristics for `X' is proposed as follows:

 For any non-terminal game state, the evaluation function is
computed as
3( - ) + 2( - ) + ( - )
where is the number of X's at corners,
is the number of X's at the border (excluding corners) ,
is the number of X's in the middle of the grid,
, and are the number of O's at the corners, the border and the middle of the board
Evaluation Functions for Othello 4 ...
       
  X O  
MAX(X)   O X  
       

       
  X O  
MIN(O)   X X   3( - ) + 2( - ) + ( - )
  X    

                       
MAX(X) O O O     X O     X O  
  X X     O X     X O  
  X     O X       X O  

X         X       X     X   X      
O X O   O X O   O O X   O O X   X X   X X X X O   X O  
  X X     X X     X X     X X     X O     X X     X X X   X X  
  X       X       X       X       X O     X O     X O     X X X

5 4 4 5     X   5 6 4 9
X O   X X   X X X
X X X     O X     O X  
MIN(O) O X     O X     O X    
3 3 3
Minimax Search for Othello 4
       
  X O  
MAX(X) 3   O X  
       

       
  X O  
MIN(O) 3   X X  
  X    

                       
MAX(X) O O O     X O     X O  
5   X X   3   O X   9   X O  
  X     O X       X O  

X         X       X     X   X      
O X O   O X O   O O X   O O X   X X   X X X X O   X O  
  X X     X X     X X     X X     X O     X X     X X X   X X  
  X       X       X       X       X O     X O     X O     X X X

5 4 4 5     X   5 6 4 9
X O   X X   X X X
X X X     O X     O X  
O X     O X     O X    
MIN(O) 3 3 3
Quiescent Search
The evaluation function should only be applied to quiescent positions
– positions that are not likely to have large variations in evaluation in the near future
Example (Chess)
Positions in which favorable captures can be made

Black: to move
White: about to lose

Expansion of non-quiescent positions until quiescent positions are reached

Revisit: Alpha Go

#turns

#Moves = 250150

#stone positions
Another Example: Monte Carlo Search Tree (MCST)
MCST Key Steps
#Wins/#Games
MCST for Go is very expensive & slow

Why? The deeper you move, the wider you are


#number of simulations is exploded

Solution:
1. For selection/expansion, no longer stats
(xx/xx) but f = g+h
2. For simulation, no longer real self-play but f =
g+h
 MCST: Update f = g() + h()

Policy Network Value Network f f

𝑔 () h( )
f f f f f f

f f f f f f f f

f f f f f f f f

No real play,
just an estimation