Local Search Algorithms and Optimization Problems

Hill Climbing Algorithm in Artificial Intelligence

o Hill climbing algorithm is a local search algorithm which continuously moves in the
direction of increasing elevation/value to find the peak of the mountain or best
solution to the problem. It terminates when it reaches a peak value where no neighbor
has a higher value.
o Hill climbing algorithm is a technique which is used for optimizing the mathematical
problems. One of the widely discussed examples of Hill climbing algorithm is
Traveling-salesman Problem in which we need to minimize the distance traveled by
the salesman.
o It is also called greedy local search as it only looks to its good immediate neighbor
state and not beyond that.
o A node of hill climbing algorithm has two components which are state and value.
o Hill Climbing is mostly used when a good heuristic is available.
o In this algorithm, we don't need to maintain and handle the search tree or graph as it
only keeps a single current state.

Features of Hill Climbing:

Following are some main features of Hill Climbing Algorithm:

o Generate and Test variant: Hill Climbing is the variant of Generate and Test
method. The Generate and Test method produce feedback which helps to decide
which direction to move in the search space.
o Greedy approach: Hill-climbing algorithm search moves in the direction which
optimizes the cost.
o No backtracking: It does not backtrack the search space, as it does not remember the
previous states.

State-space Diagram for Hill Climbing:

The state-space landscape is a graphical representation of the hill-climbing algorithm which

is showing a graph between various states of algorithm and Objective function/Cost.

On Y-axis we have taken the function which can be an objective function or cost function,
and state-space on the x-axis. If the function on Y-axis is cost then, the goal of search is to
find the global minimum and local minimum. If the function of Y-axis is Objective function,
then the goal of the search is to find the global maximum and local maximum.

1
Different regions in the state space landscape:

Local Maximum: Local maximum is a state which is better than its neighbor states, but there
is also another state which is higher than it.

Global Maximum: Global maximum is the best possible state of state space landscape. It has
the highest value of objective function.

Current state: It is a state in a landscape diagram where an agent is currently present.

Flat local maximum: It is a flat space in the landscape where all the neighbor states of
current states have the same value.

Shoulder: It is a plateau region which has an uphill edge.

Types of Hill Climbing Algorithm:

o Simple hill Climbing:

o Steepest-Ascent hill-climbing:
o Stochastic hill Climbing:

1. Simple Hill Climbing:

Simple hill climbing is the simplest way to implement a hill climbing algorithm. It only
evaluates the neighbor node state at a time and selects the first one which optimizes
current cost and set it as a current state. It only checks it's one successor state, and if it
finds better than the current state, then move else be in the same state. This algorithm has the
following features:

2
 o Less time consuming
o Less optimal solution and the solution is not guaranteed

Algorithm for Simple Hill Climbing:

o Step 1: Evaluate the initial state, if it is goal state then return success and Stop.
o Step 2: Loop Until a solution is found or there is no new operator left to apply.
o Step 3: Select and apply an operator to the current state.
o Step 4: Check new state:

a. If it is goal state, then return success and quit.

b. Else if it is better than the current state then assign new state as a current state.
c. Else if not better than the current state, then return to step2.
o Step 5: Exit.

2. Steepest-Ascent hill climbing:

The steepest-Ascent algorithm is a variation of simple hill climbing algorithm. This algorithm
examines all the neighboring nodes of the current state and selects one neighbor node which
is closest to the goal state. This algorithm consumes more time as it searches for multiple
neighbors

Algorithm for Steepest-Ascent hill climbing:

o Step 1: Evaluate the initial state, if it is goal state then return success and stop, else
make current state as initial state.
o Step 2: Loop until a solution is found or the current state does not change.

a. Let SUCC be a state such that any successor of the current state will be better
than it.
b. For each operator that applies to the current state:

a. Apply the new operator and generate a new state.

b. Evaluate the new state.
c. If it is goal state, then return it and quit, else compare it to the SUCC.
d. If it is better than SUCC, then set new state as SUCC.
e. If the SUCC is better than the current state, then set current state to
SUCC.

3
 o Step 5: Exit.

3. Stochastic hill climbing:

Stochastic hill climbing does not examine for all its neighbor before moving. Rather, this
search algorithm selects one neighbor node at random and decides whether to choose it as a
current state or examine another state.

Problems in Hill Climbing Algorithm:

1. Local Maximum: A local maximum is a peak state in the landscape which is better than
each of its neighboring states, but there is another state also present which is higher than the
local maximum.

Solution: Backtracking technique can be a solution of the local maximum in state space
landscape. Create a list of the promising path so that the algorithm can backtrack the search
space and explore other paths as well.

2. Plateau: A plateau is the flat area of the search space in which all the neighbor states of
the current state contains the same value, because of this algorithm does not find any best
direction to move. A hill-climbing search might be lost in the plateau area.

Solution: The solution for the plateau is to take big steps or very little steps while searching,
to solve the problem. Randomly select a state which is far away from the current state so it is
possible that the algorithm could find non-plateau region.

4
3. Ridges: A ridge is a special form of the local maximum. It has an area which is higher than
its surrounding areas, but itself has a slope, and cannot be reached in a single move.

Solution: With the use of bidirectional search, or by moving in different directions, we can
improve this problem.

Simulated Annealing:

A hill-climbing algorithm which never makes a move towards a lower value guaranteed to be
incomplete because it can get stuck on a local maximum. And if algorithm applies a random
walk, by moving a successor, then it may complete but not efficient. Simulated Annealing is
an algorithm which yields both efficiency and completeness.

In mechanical term Annealing is a process of hardening a metal or glass to a high

temperature then cooling gradually, so this allows the metal to reach a low-energy crystalline
state. The same process is used in simulated annealing in which the algorithm picks a random
move, instead of picking the best move. If the random move improves the state, then it
follows the same path. Otherwise, the algorithm follows the path which has a probability of
less than 1 or it moves downhill and chooses another path.

Simulated Annealing

5
Simulated annealing is similar to the hill climbing algorithm. It works on the current
situation. It picks a random move instead of picking the best move. If the move leads to the
improvement of the current situation, it is always accepted as a step towards the solution
state, else it accepts the move having a probability less than 1. This search technique was first
used in 1980 to solve VLSI layout problems. It is also applied for factory scheduling and
other large optimization tasks.

Local Beam Search

Local beam search is quite different from random-restart search. It keeps track of k states
instead of just one. It selects k randomly generated states, and expand them at each step. If
any state is a goal state, the search stops with success. Else it selects the best k successors
from the complete list and repeats the same process. In random-restart search where each
search process runs independently, but in local beam search, the necessary information is
shared between the parallel search processes.

Disadvantages of Local Beam search

 This search can suffer from a lack of diversity among the k states.

 It is an expensive version of hill climbing search.

Note: A variant of Local Beam Search is Stochastic Beam Search which selects k successors
at random rather than choosing the best k successors.

Constraint Satisfaction Problems in Artificial Intelligence

We have seen so many techniques like Local search, Adversarial search to solve different
problems. The objective of every problem-solving technique is one, i.e., to find a solution to
reach the goal. Although, in adversarial search and local search, there were no constraints on
the agents while solving the problems and reaching to its solutions.

In this section, we will discuss another type of problem-solving technique known as

Constraint satisfaction technique. By the name, it is understood that constraint satisfaction
means solving a problem under certain constraints or rules.

Constraint satisfaction is a technique where a problem is solved when its values satisfy
certain constraints or rules of the problem. Such type of technique leads to a deeper
understanding of the problem structure as well as its complexity.

Constraint satisfaction depends on three components, namely:

 X: It is a set of variables.

 D: It is a set of domains where the variables reside. There is a specific domain for
each variable.

 C: It is a set of constraints which are followed by the set of variables.

6
In constraint satisfaction, domains are the spaces where the variables reside, following the
problem specific constraints. These are the three main elements of a constraint satisfaction
technique. The constraint value consists of a pair of {scope, rel}. The scope is a tuple of
variables which participate in the constraint and rel is a relation which includes a list of
values which the variables can take to satisfy the constraints of the problem.

Solving Constraint Satisfaction Problems

The requirements to solve a constraint satisfaction problem (CSP) is:

 A state-space

 The notion of the solution.

A state in state-space is defined by assigning values to some or all variables such as

{X1=v1, X2=v2, and so on…}.

An assignment of values to a variable can be done in three ways:

 Consistent or Legal Assignment: An assignment which does not violate any constraint
or rule is called Consistent or legal assignment.

 Complete Assignment: An assignment where every variable is assigned with a value,

and the solution to the CSP remains consistent. Such assignment is known as
Complete assignment.

 Partial Assignment: An assignment which assigns values to some of the variables

only. Such type of assignments are called Partial assignments.

Types of Domains in CSP

There are following two types of domains which are used by the variables :

 Discrete Domain: It is an infinite domain which can have one state for multiple
variables. For example, a start state can be allocated infinite times for each variable.

 Finite Domain: It is a finite domain which can have continuous states describing one
domain for one specific variable. It is also called a continuous domain.

Constraint Types in CSP

With respect to the variables, basically there are following types of constraints:

 Unary Constraints: It is the simplest type of constraints that restricts the value of a
single variable.

 Binary Constraints: It is the constraint type which relates two variables. A

value x2 will contain a value which lies between x1 and x3.

7
  Global Constraints: It is the constraint type which involves an arbitrary number of
variables.

Some special types of solution algorithms are used to solve the following types of constraints:

 Linear Constraints: These type of constraints are commonly used in linear

programming where each variable containing an integer value exists in linear form
only.

 Non-linear Constraints: These type of constraints are used in non-linear programming

where each variable (an integer value) exists in a non-linear form.

Note: A special constraint which works in real-world is known as Preference constraint.

Constraint Propagation

In local state-spaces, the choice is only one, i.e., to search for a solution. But in CSP, we have
two choices either:

 We can search for a solution or

 We can perform a special type of inference called constraint propagation.

Constraint propagation is a special type of inference which helps in reducing the legal
number of values for the variables. The idea behind constraint propagation is local
consistency.

In local consistency, variables are treated as nodes, and each binary constraint is treated as
an arc in the given problem. There are following local consistencies which are discussed
below:

 Node Consistency: A single variable is said to be node consistent if all the values in
the variable’s domain satisfy the unary constraints on the variables.

 Arc Consistency: A variable is arc consistent if every value in its domain satisfies the
binary constraints of the variables.

 Path Consistency: When the evaluation of a set of two variable with respect to a third
variable can be extended over another variable, satisfying all the binary constraints. It
is similar to arc consistency.

 k-consistency: This type of consistency is used to define the notion of stronger forms
of propagation. Here, we examine the k-consistency of the variables.

CSP Problems

Constraint satisfaction includes those problems which contains some constraints while
solving the problem. CSP includes the following problems:

8
  Graph Coloring: The problem where the constraint is that no adjacent sides can have
the same color.

 Sudoku Playing: The gameplay where the constraint is that no number from 0-9 can
be repeated in the same row or column.

 n-queen problem: In n-queen problem, the constraint is that no queen should be

placed either diagonally, in the same row or column.

Note: The n-queen problem is already discussed in Problem-solving in AI section.

 Crossword: In crossword problem, the constraint is that there should be the correct
formation of the words, and it should be meaningful.

9
  Latin square Problem: In this game, the task is to search the pattern which is occurring
several times in the game. They may be shuffled but will contain the same digits.

 Cryptarithmetic Problem: This problem has one most important constraint that is, we
cannot assign a different digit to the same character. All digits should contain a
unique alphabet.

Cryptarithmetic Problem

10
Cryptarithmetic Problem is a type of constraint satisfaction problem where the game is about
digits and its unique replacement either with alphabets or other symbols. In cryptarithmetic
problem, the digits (0-9) get substituted by some possible alphabets or symbols. The task in
cryptarithmetic problem is to substitute each digit with an alphabet to get the result
arithmetically correct.

We can perform all the arithmetic operations on a given cryptarithmetic problem.

The rules or constraints on a cryptarithmetic problem are as follows:

 There should be a unique digit to be replaced with a unique alphabet.

 The result should satisfy the predefined arithmetic rules, i.e., 2+2 =4, nothing else.

 Digits should be from 0-9 only.

 There should be only one carry forward, while performing the addition operation on a
problem.

 The problem can be solved from both sides, i.e., lefthand side (L.H.S), or righthand
side (R.H.S)

Let’s understand the cryptarithmetic problem as well its constraints better with the help of an
example:

 Given a cryptarithmetic problem, i.e., S E N D + M O R E = M O N E Y

In this example, add both terms S E N D and M O R E to bring M O N E Y as a result.

Follow the below steps to understand the given problem by breaking it into its subparts:

 Starting from the left hand side (L.H.S) , the terms are S and M. Assign a digit which
could give a satisfactory result. Let’s assign S->9 and M->1.

Hence, we get a satisfactory result by adding up the terms and got an assignment for O as O-
>0 as well.

11
  Now, move ahead to the next terms E and O to get N as its output.

Adding E and O, which means 5+0=0, which is not possible because according to
cryptarithmetic constraints, we cannot assign the same digit to two letters. So, we need to
think more and assign some other value.

Note: When we will solve further, we will get one carry, so after applying it, the answer will
be satisfied.

 Further, adding the next two terms N and R we get,

But, we have already assigned E->5. Thus, the above result does not satisfy the values

because we are getting a different value for E. So, we need to think more.

Again, after solving the whole problem, we will get a carryover on this term, so our answer
will be satisfied.

12
 where 1 will be carry forward to the above term

Let’s move ahead.

 Again, on adding the last two terms, i.e., the rightmost terms D and E, we get Y as its
result.

where 1 will be carry forward to the above term

 Keeping all the constraints in mind, the final resultant is as follows:

 Below is the representation of the assignment of the digits to the alphabets.

13
More examples of cryptarithmatic problems can be:

14
Constraint propagation

Constraint propagation is the process of communicating the domain reduction of a decision

variable to all of the constraints that are stated over this variable. This process can result in
more domain reductions. These domain reductions, in turn, are communicated to the
appropriate constraints. This process continues until no more variable domains can be
reduced or when a domain becomes empty and a failure occurs. An empty domain during the
initial constraint propagation means that the model has no solution.

Example for constraint propagation

For example, consider the decision variables y with an initial domain [0..10], z with an initial
domain [0..10] and t with an initial domain [0..1], and the constraints

y + 5*z <= 4
t != z
t != y

15
over these three variables.

The domain reduction of the constraint y + 5*z <= 4 reduces the domain of y to [0..4]
and z to [0]. The variable z is thus fixed to a single value. Constraint propagation invokes
domain reduction of every constraint involving z. Domain reduction is invoked again for the
constraint y + 5*z <= 4, but the variable domains cannot be reduced further. Domain
reduction of the constraint t != z is invoked again, and because z is fixed to 0, the constraint
removes the value 0 from the domain of t. The variable t is now fixed to the value 1, and
constraint propagation invokes domain reduction of every constraint involving t, namely t !=
z and t != y. The constraint that can reduce domains further is t != y. Domain reduction
removes the value 1 from the domain of y.

Constraint propagation is performed on constraints involving y; however, no more domain

reduction can be achieved and the final domains are:

 y = [0 2..4],
 z = [0] and
 t = [1].
 Backtracking

 In this topic, we will learn about the backtracking, which is a very important skill set
to solve recursive solutions. Recursive functions are those that calls itself more than
once. Consider an example of Palindrome:

 Initially, the function isPalindrome(S, 0, 8) is called once with the parameters

isPalindrome(S, 1, 7). The recursive call isPalindrome(S, 1, 7) is called once with the
parameters isPalindrome(S, 2, 6).

 Backtracking is one of the techniques that can be used to solve the problem. We can
write the algorithm using this strategy. It uses the Brute force search to solve the
problem, and the brute force search says that for the given problem, we try to make all
the possible solutions and pick out the best solution from all the desired solutions.
This rule is also followed in dynamic programming, but dynamic programming is
used for solving optimization problems. In contrast, backtracking is not used in
solving optimization problems. Backtracking is used when we have multiple
solutions, and we require all those solutions.

 Backtracking name itself suggests that we are going back and coming forward; if it
satisfies the condition, then return success, else we go back again. It is used to solve a
problem in which a sequence of objects is chosen from a specified set so that the
sequence satisfies some criteria.

When to use a Backtracking algorithm?

When we have multiple choices, then we make the decisions from the available choices. In
the following cases, we need to use the backtracking algorithm:

o A piece of sufficient information is not available to make the best choice, so we use
the backtracking strategy to try out all the possible solutions.

16
 o Each decision leads to a new set of choices. Then again, we backtrack to make new
decisions. In this case, we need to use the backtracking strategy.

How does Backtracking work?

Backtracking is a systematic method of trying out various sequences of decisions until you
find out that works. Let's understand through an example.

We start with a start node. First, we move to node A. Since it is not a feasible solution so we
move to the next node, i.e., B. B is also not a feasible solution, and it is a dead-end so we
backtrack from node B to node A.

Suppose another path exists from node A to node C. So, we move from node A to node C. It
is also a dead-end, so again backtrack from node C to node A. We move from node A to the
starting node.

Now we will check any other path exists from the starting node. So, we move from start node
to the node D. Since it is not a feasible solution so we move from node D to node E. The
node E is also not a feasible solution. It is a dead end so we backtrack from node E to node D.

17
Suppose another path exists from node D to node F. So, we move from node D to node F.
Since it is not a feasible solution and it's a dead-end, we check for another path from node F.

Suppose there is another path exists from the node F to node G so move from node F to node
G. The node G is a success node.

18
The terms related to the backtracking are:

o Live node: The nodes that can be further generated are known as live nodes.
o E node: The nodes whose children are being generated and become a success node.
o Success node: The node is said to be a success node if it provides a feasible solution.
o Dead node: The node which cannot be further generated and also does not provide a
feasible solution is known as a dead node.

Many problems can be solved by backtracking strategy, and that problems satisfy complex
set of constraints, and these constraints are of two types:

o Implicit constraint: It is a rule in which how each element in a tuple is related.

o Explicit constraint: The rules that restrict each element to be chosen from the given
set.

Applications of Backtracking

o N-queen problem
o Sum of subset problem
o Graph coloring
o Hamiliton cycle

19
Difference between the Backtracking and Recursion

Recursion is a technique that calls the same function again and again until you reach the base
case. Backtracking is an algorithm that finds all the possible solutions and selects the desired
solution from the given set of solutions.

Recursive Maze Algorithm

Recursive Maze Algorithm is one of the best examples for backtracking algorithms.
Recursive Maze Algorithm is one of the possible solutions for solving the maze.

Maze

The maze is an area surrounded by walls; in between, we have a path from starting point to
ending position. We have to start from the starting point and travel towards from ending
point.

Principle of Maze

As explained above, in the maze we have to travel from starting point to ending point. The
problem is to choose the path. If we find any dead-end before ending point, we have to
backtrack and move the direction. The direction for traversing is North, East, West, and
South. We have to continue "move and backtrack" until we reach the final point.

Consider that we are having a two-dimensional maze cell [WIDTH] [HEIGHT]. Here
cell [x] [y] = 1 denotes wall and cell [x] [y] = 0 denotes free cell in the particular location x, y
in the maze. The directions we can change in the array are North, East, West, and South. The
first step is to make the boundary of the two - dimensional array as one so that we won't go
out of the maze and usually reside inside the maze at any time.1.3K

Example Maze

1 1 1 1 1 1 1

1 0 0 0 1 1 1

1 1 1 0 1 1 1

1 1 1 0 0 0 1

1 1 1 1 1 0 1

1 1 1 1 1 1 1

20
 JavaNow start changing from the starting position (since the boundary is filled by 1) and find
the next free cell then turn to the next free cell and so on. If we grasp a dead-end, we have to
backtrack and make the cells in the path as 1 (wall). Continue the same process till the final
point is reached.
Hamiltonian Circuit Problems

Given a graph G = (V, E) we have to find the Hamiltonian Circuit using Backtracking
approach. We start our search from any arbitrary vertex say 'a.' This vertex 'a' becomes the
root of our implicit tree. The first element of our partial solution is the first intermediate
vertex of the Hamiltonian Cycle that is to be constructed. The next adjacent vertex is selected
by alphabetical order. If at any stage any arbitrary vertex makes a cycle with any vertex other
than vertex 'a' then we say that dead end is reached. In this case, we backtrack one step, and
again the search begins by selecting another vertex and backtrack the element from the
partial; solution must be removed. The search using backtracking is successful if a
Hamiltonian Cycle is obtained.

Example: Consider a graph G = (V, E) shown in fig. we have to find a Hamiltonian circuit
using Backtracking method.

Solution: Firstly, we start our search with vertex 'a.' this vertex 'a' becomes the root of our
implicit tree.

Next, we choose vertex 'b' adjacent to 'a' as it comes first in lexicographical order (b, c, d).

21
Next, we select 'c' adjacent to 'b.'

Next, we select 'd' adjacent to 'c.'

Next, we select 'e' adjacent to 'd.'

22
Next, we select vertex 'f' adjacent to 'e.' The vertex adjacent to 'f' is d and e, but they have
already visited. Thus, we get the dead end, and we backtrack one step and remove the vertex
'f' from partial solution.

From backtracking, the vertex adjacent to 'e' is b, c, d, and f from which vertex 'f' has already
been checked, and b, c, d have already visited. So, again we backtrack one step. Now, the
vertex adjacent to d are e, f from which e has already been checked, and adjacent of 'f' are d
and e. If 'e' vertex, revisited them we get a dead state. So again we backtrack one step.

Now, adjacent to c is 'e' and adjacent to 'e' is 'f' and adjacent to 'f' is 'd' and adjacent to 'd' is 'a.'
Here, we get the Hamiltonian Cycle as all the vertex other than the start vertex 'a' is visited
only once. (a - b - c - e - f -d - a).

23
Again Backtrack

24
Here we have generated one Hamiltonian circuit, but another Hamiltonian circuit can also be
obtained by considering another vertex.

Subset Sum Problem

It is one of the most important problems in complexity theory. The problem is given an A set
of integers a1, a2,…., an upto n integers. The question arises that is there a non-empty subset
such that the sum of the subset is given as M integer?. For example, the set is given as [5, 2,
1, 3, 9], and the sum of the subset is 9; the answer is YES as the sum of the subset [5, 3, 1] is
equal to 9. This is an NP-complete problem again. It is the special case of knapsack

Let's understand this problem through an example.

problem.

We have a set of 5 integers given below:

N = 4, -2, 2, 3, 1

We want to find out the subset whose sum is equal to 5. There are many solutions to this
problem.
25
 The naïve approach, i.e., brute-force search generates all the possible subsets of the original
array, i.e., there are 2n possible states. Here the running time complexity would be
exponential. Then, we consider all these subsets in O(N) linear running time and checks
whether the sum of the items is M or not.

The dynamic programming has pseudo-polynomial running time.

Statement: Given a set of positive integers, and a value sum, determine that the sum of the
subset of a given set is equal to the given sum.

Or

Given an array of integers and a sum, the task is to have all subsets of given array with sum
equal to the given sum.

1. Example 1:
2. Input: set[] = {4, 16, 5, 23, 12}, sum = 9
3. Output = true
4. Subset {4, 5} has the sum equal to 9.
5.
6. Example 2:
7. Input: set[] = {2, 3, 5, 6, 8, 10}, sum = 10
8. Output = true
9. There are three possible subsets that have the sum equal to 10.
10. Subset1: {5, 2, 3}
11. Subset2: {2, 8}
12. Subset3: {10}

There are two ways of solving the subset problem:

o Recursion
o Dynamic programming

Method 1: Recursion

Before knowing about the recursive approach, we should know about two things in a subset
which are given below:

o Include: Here include means that we are selecting the element from the array.
o Exclude: Here, exclude means that we are rejecting the element from the array.

To implement the recursive approach, we consider the following two cases:

26
 o Now we consider the first element and now the required sum is equal to the difference
between the target sum and value of first element. The number of elements is equal to
the difference between the total elements and 1.
o Leave the 'first' element and now the required sum = target sum. The number of
elements is equal to the difference between the total elements and 1.

Let's understand that how can we solve the problem using recursion. Consider the
array which is given below:

arr = [3, 4, 5, 2]

sum = 9

result = []

In the above example, we have taken an array, and the empty array named result that stores
all the values whose resultant sum is equal to 9.

First element in an array is 3. There are two scenarios:

o First scenario is select. The sum is equal to the target sum - value of first element,
i.e., 9 - 3 = 6 and the first element, i.e., 3 gets stored in the result array, i.e., result[].
o Second scenario is reject. The array arr contains the elements 4, 5, 2, i.e., arr = [4, 5,
2] and sum would be same as 9 as we are rejecting the element 3. The result[] array
would remain empty.

Now we perform the same select and reject operation on element 4 as it is the
first element of the array now.
o Select the element 4 from the array. Since we are selecting 4 from the array so array
arr would contain the elements 5, 2, i.e., arr = [5, 2]. The sum is equal to the 6-4 = 2
and the element 4 gets stored in the result arr. The result[] = {3, 4}.
o Reject the element 4 from the array. Since we are rejecting the 4 from the array so
array arr would contain the elements 5, 2, i.e., arr = [5, 2]. The sum would remain

27
 same as 6 and the result array would be same as previous, i.e., {3}.

Now we perform the select and reject operation on element 5.

o Select the element 5 from the array. Since we are selecting 5 from the array so array
arr would contain the elements 2, i.e., arr = [2]. The sum is equal to the 2 - 5 equals to
-3 and the element 5 gets stored in the result arr. The result[] = {3, 4, 5}.
o Reject the element 5 from the array. Since we are rejecting 5 from the array so array
arr would contain the element 2, i.e., arr = [2]. The sum would remain same as
previous, i.e., 6 and the result array would be same as previous, i.e., {3, 4}.

If we observe S-5, we can see that the sum is negative that returns false. It means that there is
no further subset available in the set.

Consider R-5. It also has two scenarios:

28
 o Select the element 2 from the array. Once the element 2 gets selected, the array
becomes empty, i.e., arr[] = " ". The sum would be 2-2 equals to 0 and the element 2
gets stored in the result array. The result[] = [3, 4, 2].
o Reject the element 2 from the array. Once the element 2 gets rejected, the array
becomes empty, i.e., arr[] = " ". The sum would be same as previous, i.e., 2 and the
result array would also be same as previous, i.e., [3, 4].

Consider R-4. It has two scenarios:

o Select the element 5 from the array. Since we are selecting 5 from the array so array
arr would contain the elements 2, i.e., arr = [2]. The sum would be 6-5 equals to 1 and
the element 5 gets stored in the result array. The result[] = [3, 5].
o Reject the element 5 from the array. Since we are rejecting 5 from the array so array
arr would contain the element 2, i.e., arr = [2]. The sum would remain same as
previous, i.e., 6 and the result array would be same as previous, i.e., {3}.

29
Consider S-5. It has two scenarios:

o Select the element 2 from the array. Since we are selecting 2 from the array so array
arr would be empty, i.e., arr = " ". The sum would be 1-2 equals to -1 and the element
2 gets stored in the result array. The result[] = [3, 5, 2].
o Reject the element 2 from the array. Since we are rejecting 2 from the array so array
arr would become empty. The sum would remain same as previous, i.e., 1 and the
result array would be same as previous, i.e., {3, 5}.

30
Consider R-5. It has two scenarios:

o Select the element 2 from the array. Since we are selecting 2 from the array so array
arr would be empty, i.e., arr = " ". The sum would be 6-2 equals to 4 and the element
2 gets stored in the result array. The result[] = [3, 2].
o Reject the element 2 from the array. Since we are rejecting 2 from the array so array
arr would become empty. The sum would remain same as previous, i.e., 6 and the
result array would be same as previous, i.e., {3}.

31
 Similarly, we get the reject case, i.e., R-3 as shown as below:

Following are the base conditions:

1. if sum == 0
2. return true
3. if sum < 0
4. return false
5. if (arr[] && sum!= 0)
6. return false

When we apply the base conditions on the above tree, then we will find two subsets given
below:

S1 = {3, 4, 2}

S2 = {4, 5}

Implementation

1. def subset_sum(arr, res, sum)

2. if sum ==0
3. return true
4. if sum < 0
5. return false
6. if len(arr) == 0 and sum!= 0
7. return false

32
8. arr.pop(0);
9. if len(arr) > 0
10. res.append(arr[0])
11. select = subset_sum(arr, sum-arr[0], res)
12. reject = subset_sum(arr, res, sum)
13. return reject or sum

Method 2: Dynamic Programming

Let A be an array or set which contains 'n' non-negative integers. Find a subset 'x' of set 'A'
such that the sum of all the elements of x is equal to w where x is another input (sum).

For example:

A = {1, 2, 5, 9, 4}

Sum(w) = 18

Now we have to find out the subset from the given set whose sum is equal to 18. Here we
will use the dynamic programming approach to solve the subset sum problem.

Example:

A = [2, 3, 5, 7, 10]

Sum(w) = 14

First, we create a table. The column contains the values from 0 to 14 while row contains the
elements of the given set shown as below:

In the below table:

i: It represents the rows. Rows represents the elements.

j: It represents the columns. Columns represent the sum.

Consider the element 2. We will use 1 as a true value and 0 as a false value. The value 1
comes under 0 and 2 columns shown as below:

33
Here i=1, a[i] =2

Note: The rule for filling each column is given below:

Required sum = j - element
A[i][j] = A[i-1][required sum]

When j= 1

Required sum = 1 - 2 = -1; Since the sum is negative so put 0 under the column 1 as shown
in the above table.

When j= 2

Required sum = 2 - 2 = 0; Since the value of sum is zero so we put 1 under the column 2 as
shown in the above table.

We put 0 under the columns whose sum is greater than 2 as we cannot make sum more than 2
from the element 2.

Consider the element 3.

Here i = 2, a[i] = 3

The columns whose sum is less than 3 will have the same values as the previous columns.

When j = 3, sum[j] = 3

Required sum = 3 -3 = 0; since the sum is zero so we put 1 under the column 3 as shown in
the above table.

When j = 4; sum[j] = 4

34
Required sum = 4 - 3 = 1; Since the sum is 1 so we move to the previous row, i.e., i=1 and
j=1. The value at a[1][1] is 0 so we put 0 at a[2][4].

When j = 5, sum[j] = 5

Required sum = 5 -3 = 2; The value of sum is 2 so the value at a[1][2] equals to 1. Therefore,
the value at a[2][5] would be 1.

When j = 6, sum[j] = 6

Required sum = 6 -3 = 3; The value of sum is 3 so the value at a[1][3] equals to 0. Therefore,
the value at a[2][6] would be 0.

When j = 7, sum[7] = 7

Required sum = 7 - 3 = 4; The value of sum is 4 so the value at a[1][4] equals to 0. Therefore,
the value at a[2][7] would be 0.

In this way, we get the value 0 from the columns 8 to 14.

Consider the element 5.

Here i=3, a[i] = 5

The columns whose sum is less than 5 will have the same values as the previous columns.

When j =5, sum[j] = 5

Required sum = 5-5 = 0; Since the value of sum is 0; therefore, the value at a[2][5] equals to
1.

When j = 6, sum[j] = 6

Required sum = 6-5 = 1; the value of sum is 1 so the value at a[2][1] equals to 0; therefore,
the value at a[3][6] equals to 0.

When j=7, sum[j] = 7

Required sum = 7-5 = 2; the value of sum is 2 so the value at a[2][2] equals to 1; therefore,
the value at a[3][7] equals to 1.

When j=8, sum[j] = 8

35
Required sum = 8-5 = 3; the value of sum is 3 so the value at a[2][3] equals to 1; therefore,
the value at a[3][8] equals to 1.

When j=9, sum[j] =9

Required sum= 9-5 = 4; the value of sum is 4 so the value at a[2][4] equals to 0; therefore the
value at a[3][9] equals to 0.

In this way, we get the values from the columns 10 to 14.

Consider the element 7.

Here i=4, a[i] =7

The columns whose sum is less than 7 will have the same values as the previous columns.

When j=9, sum[j] = 9

Required sum = 9 - 7 = 2; the value of sum is 2 so the value at a[3][2] equals to 1; therefore
the value at a[4][9] equals to 1.

When j=10, sum[j] = 10

Required sum = 10 - 7= 3; the value of sum is 3 so the value at a[3][3] equals to 1; therefore,
the value at a[4][10] equals to 1.

When j=11, sum[j] =11

Required sum = 11-7 = 4; the value of sum is 4 so the value at a[3][4] equals to 0; therefore,
the value at a[4][11] equals to 0.

When j=12, sum[j] = 12

Required sum = 12-7 = 5; the value of sum is 5 so the value at a[3][5] equals to 1; therefore,
the value at a[4][12] equals to 1.

When j=13, sum[j] =13

Required sum= 13 - 7 = 6; the value of sum is 6 so the value at a[3][6] equals to 0; therefore,
the value at a[4][13] equals to 0.

36
 When j=14, sum[j] = 14

Required sum = 14 - 7 = 7; the value of sum is 7 so the value at a[3][7] equals to 1; therefore,
the value at a[4][14] equals to 1.

Consider the element 10

Here i=5, a[i] = 10

The columns whose sum is less than 10 will have the same values as the previous columns.

When j = 10, sum[j] = 10

Required sum = 10 - 10 = 0; the value of sum is 0 so the value at a[4][0] equals to 1;

therefore, the value at a[5][10] equals to 1.

When j = 11, sum[j] = 11

Required sum = 11 - 10 = 1; the value of sum is 1 so the value at a[4][1] equals to 0;

therefore, the value at a[5][11] equals to 0.

When j=12, sum[j] = 12

Required sum = 12-10 = 2; the value of sum is 2 so the value at a[4][2] equals to 1; therefore,
the value at a[5][12] equals to 1.

When j=13, sum[j] = 13

Required sum = 13 - 10 = 3; the value of sum is 3 so the value at a[4][3] equals to 1;

therefore, the value at a[5][13] equals to 1.

To determine whether the above given problem contains the subset or not, we need to check
the last row and the last column. If the value is 1 which means that there would exist atleast
one subset.

1. We have basically followed three conditions where we write 1 in the cell of the table:
2. • A[i] = j
3. • A[i-1][j] = 1
4. • A[i-1][j-A[i]] = 1
N-Queens Problem

37
N - Queens problem is to place n - queens in such a manner on an n x n chessboard that no
queens attack each other by being in the same row, column or diagonal.

It can be seen that for n =1, the problem has a trivial solution, and no solution exists for n =2
and n =3. So first we will consider the 4 queens problem and then generate it to n - queens
problem.

Given a 4 x 4 chessboard and number the rows and column of the chessboard 1 through 4.

Since, we have to place 4 queens such as q 1 q2 q3 and q4 on the chessboard, such that no two
queens attack each other. In such a conditional each queen must be placed on a different row,
i.e., we put queen "i" on row "i."

Now, we place queen q1 in the very first acceptable position (1, 1). Next, we put queen q 2 so
that both these queens do not attack each other. We find that if we place q 2 in column 1 and 2,
then the dead end is encountered. Thus the first acceptable position for q 2 in column 3, i.e. (2,
3) but then no position is left for placing queen 'q 3' safely. So we backtrack one step and place
the queen 'q2' in (2, 4), the next best possible solution. Then we obtain the position for placing
'q3' which is (3, 2). But later this position also leads to a dead end, and no place is found
where 'q4' can be placed safely. Then we have to backtrack till 'q 1' and place it to (1, 2) and
then all other queens are placed safely by moving q 2 to (2, 4), q3 to (3, 1) and q4 to (4, 3). That
is, we get the solution (2, 4, 1, 3). This is one possible solution for the 4-queens problem. For
another possible solution, the whole method is repeated for all partial solutions. The other
solutions for 4 - queens problems is (3, 1, 4, 2) i.e.

38
The implicit tree for 4 - queen problem for a solution (2, 4, 1, 3) is as follows:

39
Fig shows the complete state space for 4 - queens problem. But we can use backtracking
method to generate the necessary node and stop if the next node violates the rule, i.e., if two
queens are attacking.

4 - Queens solution space with nodes numbered in DFS

It can be seen that all the solutions to the 4 queens problem can be represented as 4 - tuples
(x1, x2, x3, x4) where xi represents the column on which queen "qi" is placed.

One possible solution for 8 queens problem is shown in fig:

40
1. Thus, the solution for 8 -queen problem for (4, 6, 8, 2, 7, 1, 3, 5).
2. If two queens are placed at position (i, j) and (k, l).
3. Then they are on same diagonal only if (i - j) = k - l or i + j = k + l.
4. The first equation implies that j - l = i - k.
5. The second equation implies that j - l = k - i.
6. Therefore, two queens lie on the duplicate diagonal if and only if |j-l|=|i-k|

Place (k, i) returns a Boolean value that is true if the kth queen can be placed in column i. It
tests both whether i is distinct from all previous costs x 1, x2,....xk-1 and whether there is no
other queen on the same diagonal.

Using place, we give a precise solution to then n- queens problem.

1. Place (k, i)
2. {
3. For j ← 1 to k - 1
4. do if (x [j] = i)
5. or (Abs x [j]) - i) = (Abs (j - k))
6. then return false;
7. return true;
8. }

Place (k, i) return true if a queen can be placed in the kth row and ith column otherwise return
is false.

x [] is a global array whose final k - 1 values have been set. Abs (r) returns the absolute value
of r.

1. N - Queens (k, n)
2. {
3. For i ← 1 to n
4. do if Place (k, i) then
5. {
6. x [k] ← i;
7. if (k ==n) then
8. write (x [1....n));
9. else
10. N - Queens (k + 1, n);
11. } }

41
Game Playing in Artificial Intelligence
Game Playing is an important domain of artificial intelligence. Games don’t require much
knowledge; the only knowledge we need to provide is the rules, legal moves and the
conditions of winning or losing the game. Both players try to win the game. So, both of them
try to make the best move possible at each turn. Searching techniques like BFS(Breadth First
Search) are not accurate for this as the branching factor is very high, so searching will take a
lot of time. So, we need another search procedures that improve –
 Generate procedure so that only good moves are generated.
 Test procedure so that the best move can be explored first.
Game playing is a popular application of artificial intelligence that involves the development
of computer programs to play games, such as chess, checkers, or Go. The goal of game
playing in artificial intelligence is to develop algorithms that can learn how to play games and
make decisions that will lead to winning outcomes.
1. One of the earliest examples of successful game playing AI is the chess program Deep
Blue, developed by IBM, which defeated the world champion Garry Kasparov in 1997.
Since then, AI has been applied to a wide range of games, including two-player games,
multiplayer games, and video games.
2. There are two main approaches to game playing in AI: rule-based systems and machine
learning-based systems. Rule-based systems use a set of fixed rules to play the game,
whereas machine learning-based systems use algorithms to learn from experience and
make decisions based on that experience.
3. In recent years, machine learning-based systems have become increasingly popular, as
they are able to learn from experience and improve over time, making them well-suited
for complex games such as Go. For example, AlphaGo, developed by DeepMind, was the
first machine learning-based system to defeat a world champion in the game of Go.
Game playing in AI is an active area of research and has many practical applications,
including game development, education, and military training. By simulating game playing
scenarios, AI algorithms can be used to develop more effective decision-making systems for
real-world applications.
The most common search technique in game playing is Minimax search procedure. It is
depth-first depth-limited search procedure. It is used for games like chess and tic-tac-
toe. Minimax algorithm uses two functions – MOVEGEN : It generates all the possible
moves that can be generated from the current position. STATICEVALUATION : It returns
a value depending upon the goodness from the viewpoint of two-player This algorithm is a
two player game, so we call the first player as PLAYER1 and second player as PLAYER2.
The value of each node is backed-up from its children. For PLAYER1 the backed-up value is
the maximum value of its children and for PLAYER2 the backed-up value is the minimum
value of its children. It provides most promising move to PLAYER1, assuming that the
PLAYER2 has make the best move. It is a recursive algorithm, as same procedure occurs at

42
each level. Figure 1: Before
backing-up of values

Figure 2: After backing-up of values We assume that PLAYER1 will start the game. 4 levels
are generated. The value to nodes H, I, J, K, L, M, N, O is provided by
STATICEVALUATION function. Level 3 is maximizing level, so all nodes of level 3 will
take maximum values of their children. Level 2 is minimizing level, so all its nodes will take
minimum values of their children. This process continues. The value of A is 23. That means
A should choose C move to win. Reference : Artificial Intelligence by Rich and Knight

Advantages of Game Playing in Artificial Intelligence:

1. Advancement of AI: Game playing has been a driving force behind the development of
artificial intelligence and has led to the creation of new algorithms and techniques that
can be applied to other areas of AI.
2. Education and training: Game playing can be used to teach AI techniques and algorithms
to students and professionals, as well as to provide training for military and emergency
response personnel.

43
 3. Research: Game playing is an active area of research in AI and provides an opportunity to
study and develop new techniques for decision-making and problem-solving.
4. Real-world applications: The techniques and algorithms developed for game playing can
be applied to real-world applications, such as robotics, autonomous systems, and decision
support systems.

Disadvantages of Game Playing in Artificial Intelligence:

1. Limited scope: The techniques and algorithms developed for game playing may not be
well-suited for other types of applications and may need to be adapted or modified for
different domains.
2. Computational cost: Game playing can be computationally expensive, especially for
complex games such as chess or Go, and may require powerful computers to achieve real-
time performance.
Mini-Max Algorithm in Artificial Intelligence

o Mini-max algorithm is a recursive or backtracking algorithm which is used in

decision-making and game theory. It provides an optimal move for the player
assuming that opponent is also playing optimally.
o Mini-Max algorithm uses recursion to search through the game-tree.
o Min-Max algorithm is mostly used for game playing in AI. Such as Chess, Checkers,
tic-tac-toe, go, and various tow-players game. This Algorithm computes the minimax
decision for the current state.
o In this algorithm two players play the game, one is called MAX and other is called
MIN.
o Both the players fight it as the opponent player gets the minimum benefit while they
get the maximum benefit.
o Both Players of the game are opponent of each other, where MAX will select the
maximized value and MIN will select the minimized value.
o The minimax algorithm performs a depth-first search algorithm for the exploration of
the complete game tree.
o The minimax algorithm proceeds all the way down to the terminal node of the tree,
then backtrack the tree as the recursion.

Pseudo-code for MinMax Algorithm:

1. function minimax(node, depth, maximizingPlayer) is

2. if depth ==0 or node is a terminal node then
3. return static evaluation of node
4.

44
5. if MaximizingPlayer then // for Maximizer Player
6. maxEva= -infinity
7. for each child of node do
8. eva= minimax(child, depth-1, false)
9. maxEva= max(maxEva,eva) //gives Maximum of the values
10. return maxEva
11.
12. else // for Minimizer player
13. minEva= +infinity
14. for each child of node do
15. eva= minimax(child, depth-1, true)
16. minEva= min(minEva, eva) //gives minimum of the values
17. return minEva

Initial call:

Minimax(node, 3, true)

Working of Min-Max Algorithm:

o The working of the minimax algorithm can be easily described using an example.
Below we have taken an example of game-tree which is representing the two-player
game.
o In this example, there are two players one is called Maximizer and other is called
Minimizer.
o Maximizer will try to get the Maximum possible score, and Minimizer will try to get
the minimum possible score.
o This algorithm applies DFS, so in this game-tree, we have to go all the way through
the leaves to reach the terminal nodes.
o At the terminal node, the terminal values are given so we will compare those value
and backtrack the tree until the initial state occurs. Following are the main steps
involved in solving the two-player game tree:

Step-1: In the first step, the algorithm generates the entire game-tree and apply the utility
function to get the utility values for the terminal states. In the below tree diagram, let's take A
is the initial state of the tree. Suppose maximizer takes first turn which has worst-case initial
value =- infinity, and minimizer will take next turn which has worst-case initial value =
+infinity.

45
Step 2: Now, first we find the utilities value for the Maximizer, its initial value is -∞, so we
will compare each value in terminal state with initial value of Maximizer and determines the
higher nodes values. It will find the maximum among the all.

o For node D max(-1,- -∞) => max(-1,4)= 4

o For Node E max(2, -∞) => max(2, 6)= 6
o For Node F max(-3, -∞) => max(-3,-5) = -3
o For node G max(0, -∞) = max(0, 7) = 7

46
Step 3: In the next step, it's a turn for minimizer, so it will compare all nodes value with +∞,
and will find the 3rd layer node values.

o For node B= min(4,6) = 4

o For node C= min (-3, 7) = -3

Step 4: Now it's a turn for Maximizer, and it will again choose the maximum of all nodes
value and find the maximum value for the root node. In this game tree, there are only 4

47
layers, hence we reach immediately to the root node, but in real games, there will be more
than 4 layers.

o For node A max(4, -3)= 4

That was the complete workflow of the minimax two player game.

Properties of Mini-Max algorithm:

o Complete- Min-Max algorithm is Complete. It will definitely find a solution (if

exist), in the finite search tree.
o Optimal- Min-Max algorithm is optimal if both opponents are playing optimally.
o Time complexity- As it performs DFS for the game-tree, so the time complexity of
Min-Max algorithm is O(bm), where b is branching factor of the game-tree, and m is
the maximum depth of the tree.
o Space Complexity- Space complexity of Mini-max algorithm is also similar to DFS
which is O(bm).

48
 Limitation of the minimax Algorithm:

The main drawback of the minimax algorithm is that it gets really slow for complex games
such as Chess, go, etc. This type of games has a huge branching factor, and the player has lots
of choices to decide. This limitation of the minimax algorithm can be improved from alpha-
beta pruning which we have discussed in the next topic.

Alpha-Beta Pruning

o Alpha-beta pruning is a modified version of the minimax algorithm. It is an

optimization technique for the minimax algorithm.
o As we have seen in the minimax search algorithm that the number of game states it
has to examine are exponential in depth of the tree. Since we cannot eliminate the
exponent, but we can cut it to half. Hence there is a technique by which without
checking each node of the game tree we can compute the correct minimax decision,
and this technique is called pruning. This involves two threshold parameter Alpha
and beta for future expansion, so it is called alpha-beta pruning. It is also called
as Alpha-Beta Algorithm.
o Alpha-beta pruning can be applied at any depth of a tree, and sometimes it not only
prune the tree leaves but also entire sub-tree.
o The two-parameter can be defined as:

a. Alpha: The best (highest-value) choice we have found so far at any point
along the path of Maximizer. The initial value of alpha is -∞.
b. Beta: The best (lowest-value) choice we have found so far at any point along
the path of Minimizer. The initial value of beta is +∞.
o The Alpha-beta pruning to a standard minimax algorithm returns the same move as
the standard algorithm does, but it removes all the nodes which are not really
affecting the final decision but making algorithm slow. Hence by pruning these nodes,
it makes the algorithm fast.

Condition for Alpha-beta pruning:

The main condition which required for alpha-beta pruning is:

1. α>=β

Key points about alpha-beta pruning:

o The Max player will only update the value of alpha.

49
 o The Min player will only update the value of beta.
o While backtracking the tree, the node values will be passed to upper nodes instead of
values of alpha and beta.
o We will only pass the alpha, beta values to the child nodes.

Pseudo-code for Alpha-beta Pruning:

1. function minimax(node, depth, alpha, beta, maximizingPlayer) is

2. if depth ==0 or node is a terminal node then
3. return static evaluation of node
4.
5. if MaximizingPlayer then // for Maximizer Player
6. maxEva= -infinity
7. for each child of node do
8. eva= minimax(child, depth-1, alpha, beta, False)
9. maxEva= max(maxEva, eva)
10. alpha= max(alpha, maxEva)
11. if beta<=alpha
12. break
13. return maxEva
14.
15. else // for Minimizer player
16. minEva= +infinity
17. for each child of node do
18. eva= minimax(child, depth-1, alpha, beta, true)
19. minEva= min(minEva, eva)
20. beta= min(beta, eva)
21. if beta<=alpha
22. break
23. return minEva

Working of Alpha-Beta Pruning:

Let's take an example of two-player search tree to understand the working of Alpha-beta
pruning

Step 1: At the first step the, Max player will start first move from node A where α= -∞ and
β= +∞, these value of alpha and beta passed down to node B where again α= -∞ and β= +∞,
and Node B passes the same value to its child D.

50
Step 2: At Node D, the value of α will be calculated as its turn for Max. The value of α is
compared with firstly 2 and then 3, and the max (2, 3) = 3 will be the value of α at node D
and node value will also 3.

Step 3: Now algorithm backtrack to node B, where the value of β will change as this is a turn
of Min, Now β= +∞, will compare with the available subsequent nodes value, i.e. min (∞, 3)
= 3, hence at node B now α= -∞, and β= 3.

51
In the next step, algorithm traverse the next successor of Node B which is node E, and the
values of α= -∞, and β= 3 will also be passed.

Step 4: At node E, Max will take its turn, and the value of alpha will change. The current
value of alpha will be compared with 5, so max (-∞, 5) = 5, hence at node E α= 5 and β= 3,
where α>=β, so the right successor of E will be pruned, and algorithm will not traverse it, and
the value at node E will be 5.

52
Step 5: At next step, algorithm again backtrack the tree, from node B to node A. At node A,
the value of alpha will be changed the maximum available value is 3 as max (-∞, 3)= 3, and
β= +∞, these two values now passes to right successor of A which is Node C.

At node C, α=3 and β= +∞, and the same values will be passed on to node F.

Step 6: At node F, again the value of α will be compared with left child which is 0, and
max(3,0)= 3, and then compared with right child which is 1, and max(3,1)= 3 still α remains
3, but the node value of F will become 1.

53
Step 7: Node F returns the node value 1 to node C, at C α= 3 and β= +∞, here the value of
beta will be changed, it will compare with 1 so min (∞, 1) = 1. Now at C, α=3 and β= 1, and
again it satisfies the condition α>=β, so the next child of C which is G will be pruned, and the
algorithm will not compute the entire sub-tree G.

54
Step 8: C now returns the value of 1 to A here the best value for A is max (3, 1) = 3.
Following is the final game tree which is the showing the nodes which are computed and
nodes which has never computed. Hence the optimal value for the maximizer is 3 for this
example.

Move Ordering in Alpha-Beta pruning:

The effectiveness of alpha-beta pruning is highly dependent on the order in which each node
is examined. Move order is an important aspect of alpha-beta pruning.

It can be of two types:

o Worst ordering: In some cases, alpha-beta pruning algorithm does not prune any of
the leaves of the tree, and works exactly as minimax algorithm. In this case, it also
consumes more time because of alpha-beta factors, such a move of pruning is called
worst ordering. In this case, the best move occurs on the right side of the tree. The
time complexity for such an order is O(bm).
o Ideal ordering: The ideal ordering for alpha-beta pruning occurs when lots of
pruning happens in the tree, and best moves occur at the left side of the tree. We apply
DFS hence it first search left of the tree and go deep twice as minimax algorithm in
the same amount of time. Complexity in ideal ordering is O(bm/2).

55
Rules to find good ordering:

Following are some rules to find good ordering in alpha-beta pruning:

o Occur the best move from the shallowest node.

o Order the nodes in the tree such that the best nodes are checked first.
o Use domain knowledge while finding the best move. Ex: for Chess, try order:
captures first, then threats, then forward moves, backward moves.
o We can bookkeep the states, as there is a possibility that states may repeat.

Stochastic Games in Artificial Intelligence

Many unforeseeable external occurrences can place us in unforeseen circumstances in real
life. Many games, such as dice tossing, have a random element to reflect this unpredictability.
These are known as stochastic games. Backgammon is a classic game that mixes skill and
luck. The legal moves are determined by rolling dice at the start of each player’s turn white,
for example, has rolled a 6–5 and has four alternative moves in the backgammon scenario
shown in the figure below.

This is a standard backgammon position. The object of the game is to get all of one’s pieces
off the board as quickly as possible. White moves in a clockwise direction toward 25, while
Black moves in a counterclockwise direction toward 0. Unless there are many opponent
pieces, a piece can advance to any position; if there is only one opponent, it is caught and
must start over. White has rolled a 6–5 and must pick between four valid moves: (5–10,5–
11), (5–11,19–24), (5–10,10–16), and (5–11,11–16), where the notation (5–11,11–16)
denotes moving one piece from position 5 to 11 and then another from 11 to 16.
Stochastic game tree for a backgammon position
White knows his or her own legal moves, but he or she has no idea how Black will roll, and
thus has no idea what Black’s legal moves will be. That means White won’t be able to build a

56
normal game tree-like in chess or tic-tac-toe. In backgammon, in addition to M A X and M I
N nodes, a game tree must include chance nodes. The figure below depicts chance nodes as
circles. The possible dice rolls are indicated by the branches leading from each chance node;
each branch is labelled with the roll and its probability. There are 36 different ways to roll
two dice, each equally likely, yet there are only 21 distinct rolls because a 6–5 is the same as
a 5–6. P (1–1) = 1/36 because each of the six doubles (1–1 through 6–6) has a probability of
1/36. Each of the other 15 rolls has a 1/18 chance of happening.

The following phase is to learn how to make good decisions. Obviously, we want to choose
the move that will put us in the best position. Positions, on the other hand, do not have
specific minimum and maximum values. Instead, we can only compute a position’s
anticipated value, which is the average of all potential outcomes of the chance nodes.
As a result, we can generalize the deterministic minimax value to an expected-minimax value
for games with chance nodes. Terminal nodes, MAX and MIN nodes (for which the dice roll
is known), and MAX and MIN nodes (for which the dice roll is unknown) all function as
before. We compute the expected value for chance nodes, which is the sum of all outcomes,
weighted by the probability of each chance action.

where r is a possible dice roll (or other random events) and RESULT(s,r) denotes the same
state as s, but with the addition that the dice roll’s result is r.
Many games, such as dice tossing, have a random element to reflect this unpredictability.
These are known as stochastic games. Backgammon is a classic game that mixes skill and
luck.

57