COMPSCI 204 – Intro to Artificial Intelligence

Games: Minimax and Alpha-Beta

Slides were adapted from those by Dan Klein and Pieter Abbeel for CS188 Intro to AI
at UC Berkeley (ai.berkeley.edu)
Table of Contents

• History / Overview
• Minimax for Zero-Sum Games
• α-β Pruning
• Finite lookahead and evaluation
Game Playing State-of-the-Art
• Checkers: 1950: First computer player.
1994: First computer champion: Chinook
ended 40-year-reign of human champion
Marion Tinsley using complete 8-piece
endgame.

• A bit like 跳棋? English for this?

Game Playing State-of-the-Art
• Chess: 1997: Deep Blue defeats
human champion Gary Kasparov in a
six-game match. Deep Blue
examined 200M positions per
second, used very sophisticated
evaluation and undisclosed methods
for extending some lines of search up
to 40 ply. Current programs are even
better, if less historic.

• Chinese chess 象棋 (Xiangqi)?

► In Oct 2022, SenseRobot, a physical
artificial intelligence-powered robot,
made Chinese chess history recently
when it beat two professional human
rivals, during a livestreamed event to
an audience of 850,000.
Game Playing State-of-the-Art
• Go: 2016: Alpha GO defeats human champion, a Korean go player,
Lee Sedol (李世石). Uses Monte Carlo Tree Search!
Game Playing State-of-the-Art
• Level of computer/AI playing the games
Behavior and Computation

• What are the computations the pacman (agent)

must have to behave like its playing?
Adversarial Games
Types of Games
• Many different kinds of
games!
► Arcade games
► Role playing games
► Strategy games
► Adventure games
► Actions games
► Puzzle games
► Racing games
Types of Games
• Many different kinds of games!

• Axes:
► Deterministic or stochastic?
• Chess, LoL, GTA?
► One, two, or more players?
► Zero sum?
► Perfect information (can you see the state)?
• Go, Poker, Bridge, Majhong

• Need algorithms for calculating a strategy (policy) which

recommends a move from each state
Deterministic Games
• Many possible formalizations, one is:
► States: S (start at s0)
► Players: P={1...N} (usually take turns)
► Actions: A (may depend on player / state)
► Transition Function: (S, A) → S’
► Terminal Test: S → {True, False}
► Terminal Utilities: (State, Player) → R

• Solution for a player is a policy: S → A

Zero-Sum Games

• Zero-Sum Games • General Games

► Agents have opposite utilities (values on ► Agents have independent utilities
outcomes) (values on outcomes)
► Lets us think of a single value that one ► Cooperation, indifference, competition,
maximizes and the other minimizes and more are all possible
► Adversarial, pure competition ► More later on non-zero-sum games
Adversarial Search
Single-Agent Trees
• Utility:
► 10 for eating a pellet, and -1 for each move
► Big triangles represent successors (child nodes afterwards)

2 0 … 2 6 … 4 6
Value of a State
• Value of a state:
► The best achievable
outcome (utility) from that
state Non-Terminal
States:

2 0 … 2 6 … 4 6 Terminal States:
Adversarial Game Trees

-20 -8 … -18 -5 … -10 +4 -20 +8

Tic-Tac-Toe Game Tree
Minimax Values
States Under Agent’s States Under Opponent’s
Control: Control:

-8 -5 -10 +8

Terminal States:
(Assuming the two turns of play, and the
numbers are for illustration only)
Adversarial Search (Minimax)
• Deterministic, zero-sum games:
Minimax values:
► Tic-tac-toe, chess, checkers
computed recursively
► One player maximizes result

► The other minimizes result

5 max

• Minimax search:
2 5 min
► A state-space search tree

► Players alternate turns

► Compute each node’s minimax 8 2 5 6

value: the best achievable utility
Terminal values:
against a rational (optimal)
part of the game
adversary
• Assumes all future moves will be optimal
• => rational against a rational player
 Minimax Implementation (Dispatch)
def value(state):
if the state is a terminal state: return the state’s utility
if the next agent is MAX: return max-value(state)
if the next agent is MIN: return min-value(state)

def max-value(state):
initialize v = -∞ def min-value(state):
for each successor of state: initialize v = +∞
v = max(v, min-value(successor)) for each successor of state:
return v v = min(v, max-value(successor))
return v
Minimax Example
• Up-triangle: MAX player
• Down-triangle: MIN player

3 12 8 2 4 6 14 5 2
Minimax Properties
• The search is optimal if both players act rationally (being a
perfect player). What if not?
max

min

10 10 9 100
Minimax Efficiency
• How efficient is minimax?
► Just like (exhaustive) DFS
► Time: O(bm)
► Space: O(bm)

• Example: For chess, b  35, m  100

► Exact solution is completely infeasible
► But, do we need to explore the whole tree?
Resource Limits
Game Tree Pruning
Minimax Example

3 12 8 2 4 6 14 5 2
Minimax Pruning

3 12 8 2 14 5 2
Alpha-Beta Pruning
• General configuration (MIN version)
► We’re computing the MIN-VALUE at some node n
► We’re looping over n’s children MAX
► n’s estimate of the childrens’ min is dropping
► Who cares about n’s value? MAX MIN a

► Let α be the best value that MAX can get so far at any
choice point along the current path from the root
► If n’s value becomes worse than α, MAX will avoid it, MAX
so we can prune n’s other children (it’s already bad
enough that it won’t be played) MIN n

• Pruning children of MAX node is symmetric

► Let β be the best value that MIN can get so far at any
choice point along the current path from the root
Alpha-Beta Example
α = best option so far from any
MAX node on this path

α =3 α =3
3

3 12 8 2 14 5 2

• The order of generation matters: more

pruning is possible if good moves come first
Alpha-Beta Implementation

α: MAX’s best option on path to root

β: MIN’s best option on path to root

def max-value(state, α, β): def min-value(state , α, β):

initialize v = -∞ initialize v = +∞
for each successor of state: for each successor of state:
v = max(v, value(successor, α, β)) v = min(v, value(successor, α, β))
if v ≥ β return v if v ≤ α return v
α = max(α, v) β = min(β, v)
return v return v
Alpha-Beta Pruning Properties
• This pruning has no effect on minimax value
computed for the root!

• Values of intermediate nodes might be wrong

► Important: children of the root may have the wrong value
► So the most naïve version won’t let you do action max
selection

min
• Good child ordering improves effectiveness of
pruning

• With “perfect ordering”: 10 10 0

► Time complexity drops to O(bm/2)
► Doubles solvable depth!
► Full search of, e.g. chess, is still hopeless…

• This is a simple example of metareasoning

(computing about what to compute)
Alpha-Beta Quiz
Alpha-Beta Quiz 2
Resource Limits
Resource Limits
• Problem: In realistic games, cannot search to leaves!
► Simply just because of too many options/steps forward
► Big branch factor!
Resource Limits
• Problem: In realistic games, cannot search to
leaves!
► Simply just because of too many options/steps forward
4 max
• Solution: Depth-limited search -2 4 min
► Instead, search only to a limited depth in the tree
► Replace terminal utilities with an evaluation function -1 -2 4 9
for non-terminal positions

• Example:
► Suppose we have 100 seconds, can explore 10K
nodes / sec
► So can check 1M nodes per move
► - reaches about depth 8 – decent chess program

• Guarantee of optimal play is gone

? ? ? ?
• More plies makes a BIG difference
► ‘ply’ here means how many steps/depths you may search,
• ‘layer’: 2 plies mean each player takes one turn.
Example of depth 2 search
• Two steps looking ahead
► Give scores
► Which option to step? How to tell?
Evaluation Functions
Evaluation Functions
• Evaluation functions score non-terminals in depth-limited search

• Ideal function: returns the actual minimax value of the position

• In practice: typically weighted linear sum of features (i.e., factors to
consider():

► f1, f2, … are features/factors: how about for pacman?

Depth Matters

• Evaluation functions are always

imperfect
• The deeper in the tree the
evaluation function is buried, the
less the quality of the evaluation
function matters
• An important example of the
tradeoff between complexity of
features and complexity of
computation
Summary
• Games are decision problems with  2 agents
► Huge variety of issues and phenomena depending on details of
interactions and payoffs
• For zero-sum games, optimal decisions defined by minimax
► Simple extension to n-player “rotating” max with vectors of utilities
► Implementable as a depth-first traversal of the game tree
► Time complexity O(bm), space complexity O(bm)
• Alpha-beta pruning
► Preserves optimal choice at the root
► Alpha/beta values keep track of best obtainable values from any max/min
nodes on path from root to current node
► Time complexity drops to O(bm/2) with ideal node ordering
• Exact solution is impossible even for “small” games like chess