Introduction to Artificial Intelligence

Session 04 - AI for Games Playing

Mahesh Madhushan
B.Sc. in ICT (RUSL), B.Eng. (Hons) in Software Engineering (UK), M.Sc. in IT (UOP)
 Why a course on AI in and for Games?
Artificial Intelligence is at the very core of modern game design and
development. It plays and tests games, it supports the design of
games and it analyses the way we play them. AI techniques used for
these tasks include expert domain-knowledge systems, search and
optimization, computational intelligence, and machine learning.
This session aims to introduce students to the theory and practice
of game artificial intelligence.
Artificial Intelligence
◦ Making computers able to do things which currently only humans can
do
Artificial Intelligence in Games
◦ Making computers able to do things which currently only humans can
do in games
What do Humans Do with Games?
Play them
Study them
Build content for them
◦ levels, maps, art,
◦ characters, missions…

Design and develop them

Do marketing
Make a statement
Make money!
Why is game playing so interesting from an AI
point of view?
Game Playing is harder then common searching
◦ The search space is too large for complete searches

We are facing an unpredictable opponent

◦ Games are adversarial search problems
◦ Solution is strategy, contingency plan.
◦ There are time limits

Game playing is considered to be an intelligent activity.

Two-Person games
How do we think when we play e.g. Chess?
◦ If I move my queen there, then my opponent has to move his knight there and
then can I move my pawn there and check mate.

We are making some assumptions

◦ We want our best move
◦ The opponent wants his best move
◦ The opponent has the same information as we
◦ Our opponent wants to win
History of AI in Games

Milestones in AI Game Playing:

◦ Early Attempts: Tic-Tac-Toe, Checkers
◦ IBM's Deep Blue vs. Garry Kasparov (Chess, 1997)
◦ IBM's Watson vs. Jeopardy Champions (Jeopardy!, 2011)
◦ Google's AlphaGo vs. Lee Sedol (Go, 2016)
 Games helps us design better AI…
…and AI can help us make better games ...
…but not necessarily through game agents!
How then?
 “Games cannot be
dissociated from emotion
and learning…”
Why use AI to Play Games?
Playing to win vs playing for experience
◦ For experience: human-like, “fun”, believable,
predictable...?
Playing in the player role vs. playing in a non-player role
AI in Game Playing
AI in game playing involves developing algorithms and strategies for
computers to play games autonomously or assist human players.
Game playing serves as a benchmark for testing AI techniques and
algorithms due to its complexity and strategic decision-making nature.
Types of games suitable for AI: Games with clear rules, a defined goal,
and a search space that can be explored using AI algorithms.
Benefits of using AI for game playing: Enhanced gameplay experience,
more challenging opponents, development of new and innovative
games, and an improved understanding of game strategies.
Games and AI Areas
•Perception
•Signal Processing
•Machine Learning
•Artificial Psychology
•Planning and Search
•KR and Reasoning
•Natural language processing
•Navigation
Number of Players
Single-player games: Challenging players with adaptive AI opponents.
◦ e.g. puzzles and time-trial racing

Two-player
◦ e.g. Chess, Checkers and Spacewar!

Multiplayer games: Providing fair and engaging experiences for all

players.
◦ e.g. League of Legends (Riot Games, 2009), the Mario Kart (Nintendo,
1992–2014) series and the online modes of most FPS games.
 Two Player Games with Game Trees
Game trees are used to represent two-player games.
Alternate moves in the game are represented by alternate levels in the
tree (plies).
Nodes in the tree represent positions.
Edges between nodes represent moves.
Leaf nodes represent won, lost or drawn positions.
For most games, the game tree can be enormous.
Game Trees

This is an example of a partial game tree

for the game tic-tac-toe.
Even for this simple game, the game tree
is very large.
Assumptions
In talking about game playing systems, we make a
number of assumptions:
The opponent is rational – will play to win.
The game is zero-sum – if one player wins, the other
loses.
Usually, the two players have complete knowledge of the
game. For games such as poker, this is clearly not true.
 Two-Player Games with Complete Trees
We can use search algorithms to write “intelligent” programs that
play games against a human opponent.
Just consider this extremely simple (and not very exciting) game: .
(1st version of NIM)
• At the beginning of the game, there are seven coins
on a table.
• Player 1 makes the first move, then player 2, then
player 1 again, and so on.
• One move consists of removing 1, 2, or 3 coins.
• The player who removes all remaining coins wins.
 Two-Player Games with Complete Trees
Let us assume that the computer has the first move. Then, the game
can be described as a series of decisions, where the first decision is
made by the computer, the second one by the human, the third one
by the computer, and so on, until all coins are gone.
The computer wants to make decisions that guarantee its victory (in
this simple game).
The underlying assumption is that the human always finds the
optimal move.
Two-Player Games with Complete Trees
C 7

1 2 3
H 6 5 4

1 2 1 1 2 3
C 5 4 4 3 2 1

1 1 2 3 1 2 3 3 2 1
H 4 3 2 1 3 2 1 C C C

1 2 3 3 2 1 3 2 1
C 3 2 1 H H H H H H

3 2 1
H C C C
Search Algorithms in Game Playing
Explore possible moves and their consequences in the game tree.
Evaluate the best move based on a given heuristic or evaluation function.
Examples of Search Algorithms:
◦ Minimax: Evaluates all possible moves and chooses the one that maximizes
the agent's score while minimizing the opponent's score (most favorable future
state).
◦ Alpha-Beta pruning: Improves efficiency by discarding branches unlikely to
contribute to the optimal outcome.
◦ Monte Carlo Tree Search: A probabilistic search algorithm that simulates
games repeatedly to estimate the value of different moves.
Minimax Algorithm
Decision-making algorithm used in two-player zero-sum games.
Maximizes the minimum gain and minimizes the maximum loss for each
player.
Steps:
◦ Generate the game tree to a certain depth.
◦ Apply the minimax algorithm recursively to evaluate each possible move.
◦ Choose the move that leads to the highest minimax value for the maximizing
player.

Example: Tic-Tac-Toe, Chess, Checkers.

Games as Search
Two players: MAX and MIN

MAX moves first and they take turns until the game is over
◦ Winner gets reward, loser gets penalty.
◦ “Zero sum” means the sum of the reward and the penalty is a constant.

E.g., win (+1), lose (-1), and draw (0) in tic-tac-toe.

E.g., win (+1), lose (0), and draw (1/2) in chess.

MAX uses search tree to determine next move.

So, in choosing, Max will choose best move to get highest points, assuming Min will
choose best move to get lowest points
Two-Player Games with Complete Trees
So the computer will start the game by taking three coins and is guaranteed to
win the gameThe most practical way of implementing such an algorithm is the
Minimax procedure:
• Call the two players MIN and MAX.
• Mark each leaf of the search tree with -1, if it
shows a victory of MIN, and with 1, if it shows a
victory of MAX.
• Propagate these values up the tree using the rules:
◦ If the parent state is a MAX node, give it the maximum value among its
children.
◦ If the parent state is a MIN node, give it the minimum value among its
children.
Two-Player Games with Complete Trees
C (max) 7
1

H (min) 6 5 4
-1 -1 1

C (max) 5 4 4 3 2 1
1 -1 -1 1 1 1

H (min) 4 3 2 1 3 2 1 C1 C
1 C
1
1 -1 -1 -1 -1 -1 -1

C (max) 3 2 1 -1
H -1
H -1
H -1
H -1
H -1H
1 1 1

H (min) 1
C C
1 C
1
 MiniMax
Example 2 of MiniMax – 2nd version of NIM
◦The rules of NIM are as follows
◦ We start with a number of sticks in a group
◦ On each move, a player must divide a group into two smaller
groups
◦ Groups of one or two sticks can’t be divided
◦ The last player to make a legal move wins
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
NIM Search tree
6
Max moves
1-5 2-4 3-3

Min moves
1-1-4 1-2-3 2-2-2

Max moves
1-1-1-3 1-1-2-2

Min moves
1-1-1-1-2
An optimal procedure: The Min-Max
method

Designed to find the optimal strategy for Max and find best move:

• 1. Generate the whole game tree, down to the leaves.

• 2. Apply utility (payoff) function to each leaf.

• 3. Back-up values from leaves through branch nodes:

• a Max node computes the Max of its child values
• a Min node computes the Min of its child values

• 4. At root: choose the move leading to the child of highest value.

Game Trees
Two-Ply Game Tree
Two-Ply Game Tree
Two-Ply Game Tree
Minimax maximizes the utility for the worst-case outcome for max

The minimax decision

Minimax – Animated Example

Max 3 6 The computer can

obtain 6 by
choosing the right
6 hand edge from the
Min 5 3
first node.

Max 1 3 6 0 7
5

5 2 1 3 6 2 0 7
 MinMax
Max’s turn
5 Max
Would like the “9” points (the maximum)
But if choose left branch, Min will choose
move to get 3 3 4 5 Min
→ left branch has a value of 3
If choose right, Min can choose any one of 5,
6 or 7 (will choose 5, the minimum)
3 9 4 5 6 7 Max
→ right branch has a value of 5
Right branch is largest (the maximum) so
choose that move
MinMax Max

Min

Max

Min

Max’s turn
Circles represent Max, Squares represent Min
Values inside represent the value the MinMax algorithm
Red arrows represent the chosen move
Numbers on left represent tree depth
Blue arrow is the chosen move
Two-Player Games with Complete Trees
The previous example shows how we can use the Minimax procedure
to determine the computer’s best move.
It also shows how we can apply depth-first search and a variant of
backtracking to prune the search tree.
Before we formalize the idea for pruning, let us move on to more
interesting games.
For such games, it is impossible to check every possible sequence of
moves. The computer player then only looks ahead a certain number
of moves and estimates the chance of winning after each possible
sequence.
Two-Player Games
Therefore, we need to define a static evaluation function e(p) that tells
the computer how favorable the current game position p is from its
perspective.
In other words, e(p) will assume large values if a position is likely to
result in a win for the computer, and low values if it predicts its defeat.
In any given situation, the computer will make a move that guarantees
a maximum value for e(p) after a certain number of moves.
For this purpose, we can use the Minimax procedure with a specific
maximum search depth (ply-depth k for k moves of each player).
 Two-Player Games
For example, let us consider Tic-Tac-Toe (although it would still be
possible to search the complete game tree for this game).
What would be a suitable evaluation function for this game?
We could use the number of lines that are still open for the
computer (X) minus the ones that are still open for its opponent
(O).
Two-Player Games
X O O X
O X X O
X
e(p) = 8 – 8 = 0 e(p) = 6 – 2 = 4 e(p) = 2 – 2 = 0

shows the weak-ness of

this e(p)
How about these?

O O X X X
X O O O
X X
e(p) = e(p) = -
Two-Player Games
Applying MiniMax to tic-tac-toe
The static heuristic evaluation function
Backup Values
 MinMax and Chess
With full tree, can determine best possible move
However, full tree impossible for some games! Ex: Chess
◦ At a given time, chess has ~ 35 legal moves. Exponential growth:
◦ 35 at one ply, 352 = 1225 at two plies … 356 = 2 billion and 3510 = 2 quadrillion
◦ Games can last 40 moves (or more), so 3540 … Stars in universe: ~ 228
For large games (Chess) can’t see end of the game. Must estimate winning or losing
from top portion
◦ Evaluate() function to guess end given board
◦ A numeric value, much smaller than victory (ie- Checkmate for Max will be one
million, for Min minus one million)
So, computer’s strength at chess comes from:
◦ How deep can search
◦ How well can evaluate a board position
◦ (In some sense, like a human – a chess grand master can evaluate board better
and can look further ahead)
Questions?

Thank You