ARTIFICIAL INTELLIGENCE - II

By Sonal Gupta
OIMT Faculty
 HILL-CLIMBING SEARCH/ALGORITHM

• Hill climbing is an iterative improvement algorithm that is similar to greedy

best-first search, except that backtracking is not permitted.
• At each step in the search, a single node is chosen to follow.
• The criterion for the node to follow is that it’s the best state for the current
state.
• Local Search Algo,
• greedy approach,
• no backtrack
 SIMPLE HILL CLIMBING

• It examines the neighboring nodes one by one and selects the first neighboring node
which optimizes the current cost as the next node.
• Evaluate the initial state. If it is a goal state then stop and return success.
Otherwise, make the initial state as the current state.
• Loop until a solution is found or until there are no new operators left to be
applied in the current state:
• Select an operator that has not yet been applied to the current state and apply it to
produce a new state.
• Evaluate the new state.
• If it is a goal state, then return it and quit
• If it is not a goal state but it is better than the current state, then make it the
current state.
• If it is not better than the current state, then continue in the loop.
SIMPLE HILL CLIMBING
SIMPLE HILL CLIMBING
SIMPLE HILL CLIMBING
SIMPLE HILL CLIMBING
 SIMPLE HILL CLIMBING

Ref: https://www.youtube.com/watch?v=2SlO34_VsY4
 STEEPEST-ASCENT HILL CLIMBING

• A useful variation on simple hill climbing considers all the moves from the
current state and select, the best one as the next state.
• Notice that this contrasts with the basic method in which the first state that is
better than the current state is selected.
• Evaluate the initial state. If it is also a goal state, then return it and quit.
Otherwise, continue with the initial state as the current state.
• Loop until a solution is found or until a complete iteration produces no change
to current state:
• Let SUCC be a state such that any possible successor of the current state will be
better than SUCC.
 STEEPEST-ASCENT HILL CLIMBING

• For each operator that applies to the current state do:

• Apply the operator and generate a new state.
• Evaluate the new state. If it is a goal state, then return
it and quit. If not, compare it to SUCC. If it is better,
then set SUCC to this state. If it is not better, leave
SUCC alone.
• If the SUCC is better than current state, then set
current state to SUCC.

Ref: https://www.youtube.com/watch?v=LQbBGpiw1Pc
 SIMULATED ANNEALING HILL CLIMBING

• In contrast to Hill Climbing, Simulated Annealing (SA) allows

downward steps as well.
• Annealing is a process in metallurgy where metals are slowly
cooled to make them reach a state of low energy where they
are very strong.
• In SA, the algorithm come back a few steps and then evaluates
the further steps.
• This helps us reach the global maxima.
• SA checks all the neighbors.
SIMULATED ANNEALING HILL CLIMBING
 SIMULATED ANNEALING HILL CLIMBING

Simulated Annealing Hill Climbing

Annealing Schedule is maintained to Not maintained

keep track of the return path

Moves to worst states may be accepted Not accepted

Best state found so far is maintained Not maintained

 AND-OR GRAPHS

• The AND-OR graph (or tree), is useful for representing the solution of problems that
can be solved by decomposing them into a set of smaller problems, all of which must
then be solved.
• This decomposition, or reduction, generates arcs that we call AND arcs.
• One AND arc may point to any number of successor nodes, all of which must be solved
in order for the arc to point to a solution.
• Just as in an OR graph, several arcs may emerge from a single node, indicating a variety
of ways in which the original problem might be solved.
• This is why the structure is called not simply an AND graph but rather an AND-OR
graph.

Source: Artificial Intelligence

by Kevin Knight
GAME TREE
 GAME TREE

• To solve games using AI, we will introduce the concept of a game tree.
• The different states of the game are represented by nodes in the game tree,
very similar to the above planning problems. The idea is just slightly different.
• In the game tree, the nodes are arranged in levels that correspond to each
player’s turns in the game so that the “root” node of the tree (usually depicted
at the top of the diagram) is the beginning position in the game.
• In tic-tac-toe, this would be the empty grid with no Xs or Os played yet.
• Under root, on the second level, there are the possible states that can result
from the first player’s moves, be it X or O.
• We call these nodes the “children” of the root node.

Source: https://course.elementsofai.com/2/3
 TWO-PLAYER GAMES

• Two-player games are games in which two players compete against each other.
• These are also known as zero-sum games.
• The goal then in playing a two-player game is choosing a move that maximizes
the score of the player and/or minimizes the score of the competing player.
• A zero-sum game is one in which the gain of one player is balanced exactly by
the loss of the other player.
• Consider the two-player game Tic-Tac-Toe. Players alternate moves, and as
each move is made, the possible moves are constrained.
TWO-PLAYER GAMES
 TWO-PLAYER GAMES

• At each node in the tree (a possible move) a value defining the goodness of
the move toward the player winning the game can be provided.
• So at a given node, the child nodes (possible moves from this state in the
game) each have an attribute defining the relative goodness of the move.
• A static evaluation function (that measure the goodness of a move) is used to
determine the value of a given move from a given game state.
• The evaluation function identifies the relative value of a successor move from
the list of possible moves as a measure of the move quality toward winning the
game.
• Game Tree – Tree representation of a game
 TWO-PLAYER GAMES

• The static evaluation function is defined as x’s possibility to win – o’s possibility
to win. Max – x, Min – o
 THE MINIMAX ALGORITHM

• The minimax algorithm is a useful method for simple two-player games.

• It is a method for selecting the best move given an alternating game where
each player opposes the other working toward a mutually exclusive goal.
• Each player knows the moves that are possible given a current game state, so
for each move, all subsequent moves can be discovered.
• At each node in the tree (possible move) a value defining the goodness of the
move toward the player winning the game can be provided.
• Minimax is a depth-first search algorithm that maintains a minimum or a
maximum value for successor nodes at each node that has children.
 THE MINIMAX ALGORITHM

• Upon reaching a leaf node, the value of the node is calculated using an
evaluation (or utility) function.
• For our move, we’ll use the maximum value as our determiner for the best
move to make.
• For our opponent, the minimum value is used.
• At each layer of the tree, the child nodes area is scanned and depending on
whose move is to come, the maximum value is kept (in the case of our move),
or the minimum value is kept (in the case of the opponent’s move).
• Since these values are propagated up in an alternating fashion, we maximize the
minimum, or minimize the maximum.
• In other words, we assume that each player makes the move next that benefits
them the most.
 THE MINIMAX ALGORITHM

• To demonstrate this approach, Figure shows the end-game

for a particular Tic-Tac-Toe board configuration.
• Both X and O have played three turns, and now it’s X’s
turn.
• We traverse this tree in depth-first order, and upon
reaching either a win/lose or draw position, we set the
score for the board.
• We’ll use a simple representation here, -1 representing a
loss, 0 for a draw, and 1 for a win.
• The boards with bold lines define the win/loss/ draw boards
where the score is evaluated.
• When all leaf nodes have been evaluated, the node values
can be propagated up based on the current player.
 THE MINIMAX ALGORITHM

• At layer 2 in the game tree, it’s O’s turn, so we minimize the

children and score the parent with the smallest value.
• At the far left portion of the game tree, the values 0 and 1
are present, so 0 is kept (the minimum) and stored in the
parent.
• At layer 1 in the tree, we’re looking at the maximum, so out
of node scores 0, -1, and -1, we keep 0 and store this at the
parent (the root node of our game tree).
• With the scores having been propagated to the root, we
can now make the best move possible.
• Since it’s our move, we’re maximizing, so we look for the
node with the largest score (the left-most node with a
value of 0), and we take this position.
 THE MINIMAX ALGORITHM

• Our opponent (who is minimizing) then chooses the

minimum node value (left-most node in tree depth 2).
• This leaves us with our final move, resulting in a draw.
• Note that in a game where perfect information is available
to each player, and no mistakes are made, the end-result will
always be a draw.
• HW – Play a game of tic-tac-toe with this scoring
mechanism and see that it always draws
 THE MINIMAX ALGORITHM

• Minimax is a kind of backtracking algorithm that is used in decision making and

game theory to find the optimal move for a player, assuming that your
opponent also plays optimally.
• It is widely used in two player turn-based games such as Tic-Tac-Toe,
Backgammon, Mancala, Chess, etc.
• In Minimax the two players are called maximizer and minimizer. The maximizer
tries to get the highest score possible while the minimizer tries to do the
opposite and get the lowest score possible.
• Every board state has a value associated with it.
• In a given state if the maximizer has upper hand then, the score of the board
will tend to be some positive value.
 THE MINIMAX ALGORITHM

• If the minimizer has the upper hand in that board state then it
will tend to be some negative value.
• The values of the board are calculated by some heuristics
which are unique for every type of game.
• Example:
• Consider a game which has 4 final states and paths to reach
final state are from root to 4 leaves of a perfect binary tree as
shown below.
• Assume you are the maximizing player and you get the first
chance to move, i.e., you are at the root and your opponent at
next level.
• Which move you would make as a maximizing player
considering that your opponent also plays optimally?
 THE MINIMAX ALGORITHM

• Since this is a backtracking based algorithm, it tries all possible

moves, then backtracks and makes a decision.
• Maximizer goes LEFT: It is now the minimizers turn. The
minimizer now has a choice between 3 and 5. Being the
minimizer it will definitely choose the least among both, that is
3
• Maximizer goes RIGHT: It is now the minimizers turn. The
minimizer now has a choice between 2 and 9. He will choose 2
as it is the least among the two values.
• Being the maximizer you would choose the larger value that is
3. Hence the optimal move for the maximizer is to go LEFT
and the optimal value is 3.
 THE MINIMAX ALGORITHM

• Now the game tree looks like shown--

• The tree shows two possible scores when maximizer makes
left and right moves.
• Note: Even though there is a value of 9 on the right subtree, the
minimizer will never pick that.We must always assume that our
opponent plays optimally.
 CHESS GAME PLAY

• Chess is an interesting test-bed for intelligent applications because the game is

rich and complex with a massive search space.
• Chess is a game of perfect information, unlike games such as Poker, where not
all of the information is known to each player.
• Both players see the same Chess board and know all moves that are possible
for both players.
• Chess programs are commonly made up of three modules.
• The first is a move generator which analyzes the current board and identifies
the legal moves that can be made.
 CHESS GAME PLAY

• The second module is the evaluation function, which computes a relative utility
for a given board configuration (how good a given board is compared to
others for a given board configuration).
• The final module is the search algorithm which must efficiently iterate through
the available board configurations, given a move, and decide which path to take
through the tree to select the next move to make.
• In the 1990s IBM’s Deep Blue successfully defeated Gary Kasparov, who at the
time was the world Chess champion.