` Introduction to

Artificial
Intelligence
Dr Lekshmi S Nair
Live 13
 Game Playing

Why do AI researchers study game playing?

1. It’s a good reasoning problem, formal and nontrivial.

2. Direct comparison with humans and other computer

programs is easy.

2
 What Kinds of Games?
Mainly games of strategy with the following
characteristics:

1. Sequence of moves to play

2. Rules that specify possible moves
3. Rules that specify a payment for each
move
4. Objective is to maximize your payment
3
 Games vs. Search Problems

• Unpredictable opponent  specifying a

move for every possible opponent reply

• Time limits  unlikely to find goal, must

approximate

4
 Two-Player Game
Opponent’s Move

Generate New Position

Game yes
Over?
no
Generate Successors

Evaluate Successors

Move to Highest-Valued Successor

no Game yes
Over?
5
 Game Tree (2-player,
Deterministic, Turns)
computer’s
turn

opponent’s
turn

computer’s The computer is Max.

turn
The opponent is Min.
opponent’s
turn

leaf nodes At the leaf nodes, the

are evaluated utility function
is employed. Big value
6
means good, small is bad.
 Mini-Max Terminology
• utility function: the function applied to leaf nodes
• backed-up value
– of a max-position: the value of its largest successor
– of a min-position: the value of its smallest successor
• minimax procedure: search down several levels;
at the bottom level apply the utility function,
back-up values all the way up to the root node,
and that node selects the move.

7
 Minimax
• Perfect play for deterministic games
• Idea: choose move to position with highest minimax
value
= best achievable payoff against best play
• E.g., 2-ply game:

8
 Example 1
Use the Minimax algorithm to compute the minimax value at each node for the game
tree below

9
 Example 1
Use the Minimax algorithm to compute the minimax value at each node for the game
tree below

10
 Minimax Strategy
• Why do we take the min value every other
level of the tree?

• These nodes represent the opponent’s

choice of move.

• The computer assumes that the human

will choose that move that is of least value
to the computer.
11
Minimax algorithm

12
 Tic Tac Toe
• Let p be a position in the game
• Define the utility function f(p) by
– f(p) =
• largest positive number if p is a win for computer
• smallest negative number if p is a win for opponent
• RCDC – RCDO
– where RCDC is number of rows, columns and
diagonals in which computer could still win
– and RCDO is number of rows, columns and diagonals
in which opponent could still win.

13
 Sample Evaluations
• X = Computer; O = Opponent

O O O X
X X X

X O X O
rows rows
cols cols
diags diags
14
 Minimax is done depth-first
max

min

max

leaf
2 5 1

15
 Properties of Minimax
• Complete? Yes (if tree is finite)
• Optimal? Yes (against an optimal opponent)
• Time complexity? O(bm)
• Space complexity? O(bm) (depth-first exploration)

• For chess, b ≈ 35, m ≈100 for "reasonable" games

 exact solution completely infeasible

Need to speed it up.

16
 Alpha-Beta Procedure
• The alpha-beta procedure can speed up a
depth-first minimax search.
• Alpha: a lower bound on the value that a
max node may ultimately be assigned
v>

• Beta: an upper bound on the value that a

minimizing node may ultimately be
assigned
v<
17
α-β pruning example

18
α-β pruning example
=3

alpha cutoff

19
α-β pruning example

20
α-β pruning example

21
α-β pruning example

22
 Alpha Cutoff
>3 =3

8 10

What happens here? Is there an alpha cutoff?

23
 Beta Cutoff

=4 <4

4 >8

8  cutoff

24
 Alpha-Beta Pruning
max

min

max

eval 5 2 10 11 1 2 2 8 6 5 12 4 3 25 2

25
 Properties of α-β
• Pruning does not affect final result. This means that it
gets the exact same result as does full minimax.

• Good move ordering improves effectiveness of pruning

• With "perfect ordering," time complexity = O(bm/2)

 doubles depth of search

• A simple example of the value of reasoning about which

computations are relevant (a form of metareasoning)

26
The α-β algorithm

cutoff

27
The α-β algorithm

cutoff

28
When do we get alpha cutoffs?

100 ...

< 100 < 100

29
 Shallow Search Techniques
1. limited search for a few levels

2. reorder the level-1 sucessors

3. proceed with - minimax search

30
 Additional Refinements
• Waiting for Quiescence: continue the search
until no drastic change occurs from one level to
the next.

• Secondary Search: after choosing a move,

search a few more levels beneath it to be sure it
still looks good.

• Book Moves: for some parts of the game

(especially initial and end moves), keep a
catalog of best moves to make.
31
 Evaluation functions
• For chess/checkers, typically linear weighted sum of
features
Eval(s) = w1 f1(s) + w2 f2(s) + … + wn fn(s)

e.g., w1 = 9 with
f1(s) = (number of white queens) – (number of black
queens), etc.

32
 Example: Samuel’s Checker-
Playing Program
• It uses a linear evaluation function
f(n) = a1x1(n) + a2x2(n) + ... + amxm(n)

For example: f = 6K + 4M + U
– K = King Advantage
– M = Man Advantage
– U = Undenied Mobility Advantage (number of
moves that Max has that Min can’t jump after)
33
 Samuel’s Checker Player
• In learning mode

– Computer acts as 2 players: A and B

– A adjusts its coefficients after every move
– B uses the static utility function
– If A wins, its function is given to B

34
 Samuel’s Checker Player
• How does A change its function?
1. Coefficent replacement
(node ) = backed-up value(node) – initial value(node)
if > 0 then terms that contributed
positively are given more weight and terms that
contributed negatively get less weight
if < 0 then terms that contributed
negatively are given more weight and terms that
contributed positively get less weight

35
 Samuel’s Checker Player
• How does A change its function?
2. Term Replacement
38 terms altogether
16 used in the utility function at any one time

Terms that consistently correlate low with the

function value are removed and added to the end of
the term queue.

They are replaced by terms from the front of the

term queue.

36
 Kalah
P’s holes

6 6 6 6 6 6
KP Kp
0 counterclockwise 0

6 6 6 6 6 6

p’s holes
To move, pick up all the stones in one of your holes, and
put one stone in each hole, starting at the next one,
including your Kalah and skipping the opponent’s Kalah. 37
 Kalah
• If the last stone lands in your Kalah, you get
another turn.
• If the last stone lands in your empty hole, take
all the stones from your opponent’s hole directly
across from it and put them in your Kalah.
• If all of your holes become empty, the opponent
keeps the rest of the stones.
• The winner is the player who has the most
stones in his Kalah at the end of the game.

38
 Cutting off Search
MinimaxCutoff is identical to MinimaxValue except
1. Terminal? is replaced by Cutoff?
2. Utility is replaced by Eval

Does it work in practice?

bm = 106, b=35  m=4
4-ply lookahead is a hopeless chess player!
– 4-ply ≈ human novice
– 8-ply ≈ typical PC, human master
– 12-ply ≈ Deep Blue, Kasparov

39
Deterministic Games in Practice
• Checkers: Chinook ended 40-year-reign of human world champion
Marion Tinsley in 1994. Used a precomputed endgame database
defining perfect play for all positions involving 8 or fewer pieces on
the board, a total of 444 billion positions.

• Chess: Deep Blue defeated human world champion Garry Kasparov

in a six-game match in 1997. Deep Blue searches 200 million
positions per second, uses very sophisticated evaluation, and
undisclosed methods for extending some lines of search up to 40
ply.

• Othello: human champions refuse to compete against computers,

who are too good.

• Go: human champions refuse to compete against computers, who

are too bad. In Go, b > 300, so most programs use pattern
knowledge bases to suggest plausible moves.

40
 Games of Chance
• What about games that involve chance,
such as
– rolling dice
– picking a card
• Use three kinds of nodes:
– max nodes    min
– min nodes chance
– chance nodes max

41
 Games of Chance

c chance node with

max children

d1 di dk

S(c,di)

expectimax(c) = ∑P(di) max(backed-up-value(s))

i s in S(c,di)

expectimin(c’) = ∑P(di) min(backed-up-value(s))

i s in S(c,di) 42
 Example Tree with Chance
max

chance
.4 .6
min   1.2
chance
.4 .6 .4 .6
max

leaf 3 5 1 4 1 2 4 5

43
 Complexity
• Instead of O(bm), it is O(bmnm) where n is
the number of chance outcomes.

• Since the complexity is higher (both time

and space), we cannot search as deeply.

• Pruning algorithms may be applied.

44
 Summary
• Games are fun to work on!

• They illustrate several important points about AI.

• Perfection is unattainable  must approximate.

• Game playing programs have shown the world

what AI can do.

45