Artificial Intelligence (AI)

LABORATORY WORKBOOK

Name ____________________
Roll No ___________________
Date ______________________
Marks Obtained ____________
Signature___________________
 AI Lab Topics

S
Object of Experiments Remarks Date Signature
No.

1 Introduction to Python

2 Advance Programming in Python

3 Python classes and objects

4 Programming agents

5 Uninformed search

6 Informed Search

7 Introduction to Constraint Satisfaction programming

8 Problem Solving using Constraint Satisfaction Programming

9 Adversarial Search

10 understand the basic concept of problem solving

understand the basic concept of Markov Decision

11 Process

12 Reinforcement Learning

13 Bayesian Networks

14 Markov Chain
 LAB # 01
INTRODUCTION
The purpose of this lab is to get you familiar with Python and its IDE

Name
Date
Registration No
Department
Quiz
Assignment

___________________
Lab Instructor Signature
 Experiment
INTRODUCTION

01
OBJECTIVE

Introduction to python Basic,

Python Installation

To get started working with Python 3, you’ll need to have access to the Python interpreter. There
are several common ways to accomplish this:

• Python can be obtained from the Python Software Foundation website at python.org.
Typically, that involves downloading the appropriate installer for your operating system
and running it on your machine.
• Some operating systems, notably Linux, provide a package manager that can be run to
install Python.
• On macOS, the best way to install Python 3 involves installing a package manager called
Homebrew. You’ll see how to do this in the relevant section in the tutorial.
• On mobile operating systems like Android and iOS, you can install apps that provide a
Python programming environment. This can be a great way to practice your coding skills
on the go.

Alternatively, there are several websites that allow you to access a Python interpreter online
without installing anything on your computer at all.
It is highly unlikely that your Windows system shipped with Python already installed. Windows
systems typically do not. Fortunately, installing does not involve much more than downloading
the Python installer from the python.org website and running it. Let’s take a look at how to install
Python 3 on Windows:
Step 1: Download the Python 3 Installer

1. Open a browser window and navigate to the Download page for Windows at python.org.
2. Underneath the heading at the top that says Python Releases for Windows, click on the
link for the Latest Python 3 Release - Python 3.x.x. (As of this writing, the latest is
Python 3.6.5.)
3. Scroll to the bottom and select either Windows x86-64 executable installer for 64-bit or
Windows x86 executable installer for 32-bit. (See below.)

Sidebar: 32‐bit or 64‐bit Python?

For Windows, you can choose either the 32-bit or 64-bit installer. Here’s what the difference
between the two comes down to:

• If your system has a 32-bit processor, then you should choose the 32-bit installer.
• On a 64-bit system, either installer will actually work for most purposes. The 32-bit
version will generally use less memory, but the 64-bit version performs better for
applications with intensive computation.
• If you’re unsure which version to pick, go with the 64-bit version.

Note: Remember that if you get this choice “wrong” and would like to switch to another version
of Python, you can just uninstall Python and then re-install it by downloading another installer
from python.org.

Step 2: Run the Installer

Once you have chosen and downloaded an installer, simply run it by double-clicking on the
downloaded file. A dialog should appear that looks something like this:
Important: You want to be sure to check the box that says Add Python 3.x to PATH as shown
to ensure that the interpreter will be placed in your execution path.
Then just click Install Now. That should be all there is to it. A few minutes later you should have
a working Python 3 installation on your system.
PyCharm is available in three editions: Professional, Community, and Educational (Edu). The
Community and Edu editions are open-source projects and they are free, but they has less
features. PyCharm Edu provides courses and helps you learn programming with Python. The
Professional edition is commercial, and provides an outstanding set of tools and features. For
details, see the editions comparison matrix.

To install PyCharm

1. Download PyCharm for your operating system.

2. Do the following depending on your operating system: o
Windows installation:
1. Run the PyCharm-*.exe file you've downloaded.
2. Follow the instructions in the installation wizard.

THEORY
Operators

The Python interpreter can be used to evaluate expressions, for example simple arithmetic
expressions. If you enter such expressions at the prompt ( >>>) they will be evaluated and the
result will be returned on the next line.

>>> 1 + 1 2
>>> 2 * 3
6

Boolean operators also exist in Python to manipulate the primitive True and False values.
>>> 1==0
False
>>> not (1==0)
True
>>> (2==2) and (2==3)
False >>> (2==2)
or (2==3)
True

Strings

Like Java, Python has a built in string type. The + operator is overloaded to do string
concatenation on string values.

>>> 'artificial' + "intelligence" 'artificialintelligence'

There are many built-in methods which allow you to manipulate strings.

>>> 'artificial'.upper()
'ARTIFICIAL'
>>> 'HELP'.lower()
'help' >>>
len('Help')
4

Notice that we can use either single quotes ' ' or double quotes " " to surround string. This
allows for easy nesting of strings.

We can also store expressions into variables.

>>> s = 'hello
world' >>> print(s)
hello world >>>
s.upper() 'HELLO
WORLD'
>>> len(s.upper())
11
>>> num = 8.0
>>> num += 2.5
>>> print(num)
10.5

In Python, you do not have declare variables before you assign to them.

Built-in Data Structures

Python comes equipped with some useful built-in data structures, broadly similar to Java's
collections package.

Lists

Lists store a sequence of mutable items:

>>> fruits = ['apple','orange','pear','banana'] >>>

fruits[0] 'apple'

We can use the + operator to do list concatenation:

>>> otherFruits = ['kiwi','strawberry']
>>> fruits + otherFruits
>>> ['apple', 'orange', 'pear', 'banana', 'kiwi', 'strawberry']

Python also allows negative-indexing from the back of the list. For instance, fruits[-1] will
access the last element 'banana':
>>> fruits[-2] 'pear'
>>> fruits.pop()
'banana'
>>> fruits
['apple', 'orange', 'pear']
>>> fruits.append('grapefruit')
>>> fruits
['apple', 'orange', 'pear', 'grapefruit']
>>> fruits[-1] = 'pineapple'
>>> fruits
['apple', 'orange', 'pear', 'pineapple']

We can also index multiple adjacent elements using the slice operator. For instance,
fruits[1:3], returns a list containing the elements at position 1 and 2. In general
fruits[start:stop] will get the elements in start, start+1, ..., stop-1. We can also do
fruits[start:] which returns all elements starting from the start index. Also fruits[:end]
will return all elements before the element at position end:
>>> fruits[0:2]
['apple', 'orange']
>>> fruits[:3]
['apple', 'orange', 'pear']
>>> fruits[2:]
['pear', 'pineapple'] >>>
len(fruits)
4

The items stored in lists can be any Python data type. So for instance we can have lists of lists:

>>> lstOfLsts = [['a','b','c'],[1,2,3],['one','two','three']]

>>> lstOfLsts[1][2] 3
>>> lstOfLsts[0].pop()
'c'
>>> lstOfLsts
[['a', 'b'],[1, 2, 3],['one', 'two', 'three']]

Tuples

A data structure similar to the list is the tuple, which is like a list except that it is immutable once
it is created (i.e. you cannot change its content once created). Note that tuples are surrounded
with parentheses while lists have square brackets.

>>> pair = (3,5)

>>> pair[0] 3
>>> x,y = pair
>>> x 3
>>> y
5
>>> pair[1] = 6
TypeError: object does not support item assignment

The attempt to modify an immutable structure raised an exception. Exceptions indicate errors:
index out of bounds errors, type errors, and so on will all report exceptions in this way.

Sets

A set is another data structure that serves as an unordered list with no duplicate items. Below, we
show how to create a set:
>>> shapes = ['circle','square','triangle','circle'] >>> setOfShapes
= set(shapes)

Another way of creating a set is shown below:

>>> setOfShapes = {‘circle’, ‘square’, ‘triangle’, ‘circle’}

Next, we show how to add things to the set, test if an item is in the set, and perform common set
operations (difference, intersection, union):

>>> setOfShapes set(['circle','square','triangle'])

>>> setOfShapes.add('polygon')
>>> setOfShapes
set(['circle','square','triangle','polygon'])
>>> 'circle' in setOfShapes
True
>>> 'rhombus' in setOfShapes
False
>>> favoriteShapes = ['circle','triangle','hexagon'] >>>
setOfFavoriteShapes = set(favoriteShapes) >>>
setOfShapes - setOfFavoriteShapes
set(['square','polyon'])
>>> setOfShapes & setOfFavoriteShapes set(['circle','triangle'])
>>> setOfShapes | setOfFavoriteShapes
set(['circle','square','triangle','polygon','hexagon'])

Note that the objects in the set are unordered; you cannot assume that their traversal or
print order will be the same across machines!

Dictionaries

The last built-in data structure is the dictionary which stores a map from one type of object (the
key) to another (the value). The key must be an immutable type (string, number, or tuple). The
value can be any Python data type.

Note: In the example below, the printed order of the keys returned by Python could be different
than shown below. The reason is that unlike lists which have a fixed ordering, a dictionary is
simply a hash table for which there is no fixed ordering of the keys (like HashMaps in Java). The
order of the keys depends on how exactly the hashing algorithm maps keys to buckets, and will
usually seem arbitrary. Your code should not rely on key ordering, and you should not be
surprised if even a small modification to how your code uses a dictionary results in a new key
ordering.
>>> studentIds = {'knuth': 42.0, 'turing': 56.0, 'nash': 92.0 } >>>
studentIds['turing']
56.0
>>> studentIds['nash'] = 'ninety-two'
>>> studentIds
{'knuth': 42.0, 'turing': 56.0, 'nash': 'ninety-two'}
>>> del studentIds['knuth']
>>> studentIds
{'turing': 56.0, 'nash': 'ninety-two'}
>>> studentIds['knuth'] = [42.0,'forty-two']
>>> studentIds
{'knuth': [42.0, 'forty-two'], 'turing': 56.0, 'nash': 'ninety-two'}
>>> studentIds.keys()
['knuth', 'turing', 'nash']
>>> studentIds.values()
[[42.0, 'forty-two'], 56.0, 'ninety-two']
>>> studentIds.items()
[('knuth',[42.0, 'forty-two']), ('turing',56.0), ('nash','ninety-two')] >>>
len(studentIds) 3

As with nested lists, you can also create dictionaries of dictionaries.

Writing Scripts

Now that you've got a handle on using Python interactively, let's write a simple Python script that
demonstrates Python's for loop. Open the file called foreach.py, which should contain the
following code:

# This is what a comment looks like

fruits = ['apples', 'oranges', 'pears', 'bananas']
for fruit in fruits: print(fruit + ' for sale')

fruitPrices = {'apples': 2.00, 'oranges': 1.50, 'pears': 1.75} for

fruit, price in fruitPrices.items(): if price < 2.00:
print('%s cost %f a pound' % (fruit, price)) else:
print(fruit + ' are too expensive!')

At the command line, use the following command in the directory containing foreach.py:
[cs188-ta@nova ~/tutorial]$ python foreach.py
apples for sale oranges for sale pears for sale
bananas for sale apples are too expensive!
oranges cost 1.500000 a pound pears cost
1.750000 a pound

Remember that the print statements listing the costs may be in a different order on your screen
than in this tutorial; that's due to the fact that we're looping over dictionary keys, which are
unordered. To learn more about control structures (e.g., if and else) in Python, check out the
official Python tutorial section on this topic.

If you like functional programming you might also like map and filter:
>>> list(map(lambda x: x * x, [1,2,3]))
[1, 4, 9]
>>> list(filter(lambda x: x > 3, [1,2,3,4,5,4,3,2,1])) [4, 5,
4]

The next snippet of code demonstrates Python's list comprehension construction:

nums = [1,2,3,4,5,6] plusOneNums = [x+1
for x in nums] oddNums = [x for x in nums if
x % 2 == 1] print(oddNums)
oddNumsPlusOne = [x+1 for x in nums if x % 2 ==1] print(oddNumsPlusOne)

This code is in a file called listcomp.py, which you can run:

[cs188-ta@nova ~]$ python listcomp.py
[1,3,5]
[2,4,6]
CLASS ASSIGNMENT

Exercise: Dir and Help

Learn about the methods Python provides for strings. To see what methods Python provides for a
datatype, use the dir and help commands:
>>> s = 'abc'

>>> dir(s)
['__add__', '__class__', '__contains__', '__delattr__', '__doc__', '__eq__',
'__ge__', '__getattribute__', '__getitem__', '__getnewargs__',
'__getslice__', '__gt__', '__hash__', '__init__','__le__', '__len__',
'__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__',
'__reduce_ex__','__repr__', '__rmod__', '__rmul__', '__setattr__', '__str__',
'capitalize', 'center', 'count', 'decode', 'encode', 'endswith',
'expandtabs', 'find', 'index', 'isalnum', 'isalpha', 'isdigit', 'islower',
'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip',
'replace', 'rfind','rindex', 'rjust', 'rsplit', 'rstrip', 'split',
'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper',
'zfill']

>>> help(s.find)
Help on built-in function find:

find(...) method of builtins.str instance

S.find(sub[, start[, end]]) -> int

Return the lowest index in S where substring sub is found, such

that sub is contained within S[start:end]. Optional arguments
start and end are interpreted as in slice notation.

Return -1 on failure.

>> s.find('b')
1

Try out some of the string functions listed in dir (ignore those with underscores '_' around the
method name).
Exercise: Lists

Play with some of the list functions. You can find the methods you can call on an object via the
dir and get information about them via the help command:

>>> dir(list)
['__add__', '__class__', '__contains__', '__delattr__', '__delitem__',
'__delslice__', '__doc__', '__eq__', '__ge__', '__getattribute__',
'__getitem__', '__getslice__', '__gt__', '__hash__', '__iadd__', '__imul__',
'__init__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__',
'__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__',
'__rmul__', '__setattr__', '__setitem__', '__setslice__', '__str__', 'append',
'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort']

>>> help(list.reverse)
Help on built-in function reverse:

reverse(...)
L.reverse() -- reverse *IN PLACE*

>>> lst = ['a','b','c']

>>> lst.reverse()
>>> ['c','b','a']

Note: Ignore functions with underscores "_" around the names; these are private helper methods.
Press 'q' to back out of a help screen.

Exercise: Dictionaries

Use dir and help to learn about the functions you can call on dictionaries.

Exercise: List Comprehensions

Write a list comprehension which, from a list, generates a lowercased version of each
string that has length greater than five.

Exercise : Functions

1. Calculate sum of 1st ten natural no.

 EXPERIMENT # 02
INTRODUCTION
The purpose of this lab is to get familiar with some advanced programming in Python

Name
Date
Registration No
Department
Total Marks
Marks Obtained
Remarks

___________________
Lab Instructor Signature
 Experiment
INTRODUCTION

02
OBJECTIVE

You will learn syntax and semantics of advanced concepts of Python, and get introduced to structured
data objects.

THEORY

Beware of Indendation!

Unlike many other languages, Python uses the indentation in the source code for interpretation.
So for instance, for the following script:
if 0 == 1: print('We are in a world of
arithmetic pain') print('Thank you for playing')

will output
Thank you for playing

But if we had written the script as

if 0 == 1: print('We are in a world of
arithmetic pain') print('Thank you for
playing')

there would be no output. The moral of the story: be careful how you indent! It's best to use four
spaces for indentation -- that's what the course code uses.

Tabs vs Spaces

Because Python uses indentation for code evaluation, it needs to keep track of the level of
indentation across code blocks. This means that if your Python file switches from using tabs as
indentation to spaces as indentation, the Python interpreter will not be able to resolve the
ambiguity of the indentation level and throw an exception. Even though the code can be lined up
visually in your text editor, Python "sees" a change in indentation and most likely will throw an
exception (or rarely, produce unexpected behavior).

This most commonly happens when opening up a Python file that uses an indentation scheme
that is opposite from what your text editor uses (aka, your text editor uses spaces and the file uses
tabs). When you write new lines in a code block, there will be a mix of tabs and spaces, even
though the whitespace is aligned. For a longer discussion on tabs vs spaces, see this discussion
on StackOverflow.

Writing Functions

As in Java, in Python you can define your own functions:

fruitPrices = {'apples':2.00, 'oranges': 1.50, 'pears': 1.75}
def buyFruit(fruit, numPounds): if fruit not
in fruitPrices: print("Sorry we don't
have %s" % (fruit)) else: cost =
fruitPrices[fruit] * numPounds
print("That'll be %f please" % (cost))

# Main Function if
__name__ == '__main__':
buyFruit('apples',2.4)
buyFruit('coconuts',2)

Rather than having a main function as in Java, the __name__ == '__main__' check is used to
delimit expressions which are executed when the file is called as a script from the command line.
The code after the main check is thus the same sort of code you would put in a main function in
Java.

Save this script as fruit.py and run it:

$ python fruit.py
That'll be 4.800000 please
Sorry we don't have coconuts

Object Basics

Although this isn't a class in object-oriented programming, you'll have to use some objects in the
programming projects, and so it's worth covering the basics of objects in Python. An object
encapsulates data and provides functions for interacting with that data.
Defining Classes

Here's an example of defining a class named FruitShop: class

FruitShop:

def __init__(self, name,

fruitPrices):
"""
name: Name of the fruit shop

fruitPrices: Dictionary with keys as fruit

strings and prices for values e.g.
{'apples':2.00, 'oranges': 1.50, 'pears': 1.75}
"""
self.fruitPrices = fruitPrices
self.name = name
print('Welcome to %s fruit shop' % (name))
def getCostPerPound(self,
fruit):
"""
fruit: Fruit string
Returns cost of 'fruit', assuming 'fruit'
is in our inventory or None otherwise
""" if fruit not in
self.fruitPrices: return None
return self.fruitPrices[fruit]
def getPriceOfOrder(self,
orderList):
"""
orderList: List of (fruit, numPounds) tuples

Returns cost of orderList, only including the values of

fruits that this fruit shop has.
""" totalCost = 0.0 for fruit,
numPounds in orderList: costPerPound =
self.getCostPerPound(fruit) if
costPerPound != None: totalCost +=
numPounds * costPerPound return totalCost
def getName(self):
return self.name

The FruitShop class has some data, the name of the shop and the prices per pound of some fruit,
and it provides functions, or methods, on this data. What advantage is there to wrapping this data
in a class?

1. Encapsulating the data prevents it from being altered or used inappropriately, 2.

The abstraction that objects provide make it easier to write general-purpose code.
Using Objects

So how do we make an object and use it? Make sure you have the FruitShop implementation in
shop.py. We then import the code from this file (making it accessible to other scripts) using
import shop, since shop.py is the name of the file. Then, we can create FruitShop objects as
follows:
import shop

shopName = 'the Berkeley Bowl'

fruitPrices = {'apples': 1.00, 'oranges': 1.50, 'pears': 1.75}
berkeleyShop = shop.FruitShop(shopName, fruitPrices) applePrice
= berkeleyShop.getCostPerPound('apples') print(applePrice)
print('Apples cost $%.2f at %s.' % (applePrice, shopName))

otherName = 'the Stanford Mall'

otherFruitPrices = {'kiwis':6.00, 'apples': 4.50, 'peaches': 8.75}
otherFruitShop = shop.FruitShop(otherName, otherFruitPrices) otherPrice
= otherFruitShop.getCostPerPound('apples') print(otherPrice)
print('Apples cost $%.2f at %s.' % (otherPrice, otherName))
print("My, that's expensive!")

This code is in shopTest.py; you can run it like this:

[cs188-ta@nova ~]$ python shopTest.py
Welcome to the Berkeley Bowl fruit shop
1.0
Apples cost $1.00 at the Berkeley Bowl.
Welcome to the Stanford Mall fruit shop
4.5
Apples cost $4.50 at the Stanford Mall. My,
that's expensive!

So what just happended? The import shop statement told Python to load all of the functions and
classes in shop.py. The line berkeleyShop = shop.FruitShop(shopName, fruitPrices)
constructs an instance of the FruitShop class defined in shop.py, by calling the __init__
function in that class. Note that we only passed two arguments in, while __init__ seems to take
three arguments: (self, name, fruitPrices). The reason for this is that all methods in a class
have self as the first argument. The self variable's value is automatically set to the object itself;
when calling a method, you only supply the remaining arguments. The self variable contains all
the data (name and fruitPrices) for the current specific instance (similar to this in Java). The
print statements use the substitution operator (described in the Python docs if you're curious).

Static vs Instance Variables

The following example illustrates how to use static and instance variables in Python.
Create the person_class.py containing the following code:
class Person:
population = 0 def
__init__(self, myAge):
self.age = myAge
Person.population += 1 def
get_population(self):
return Person.population
def get_age(self):
return self.age

We first compile the script:

[cs188-ta@nova ~]$ python person_class.py

Now use the class as follows:

>>> import person_class
>>> p1 = person_class.Person(12)
>>> p1.get_population()
1
>>> p2 = person_class.Person(63)
>>> p1.get_population()
2
>>> p2.get_population()
2
>>> p1.get_age()
12
>>> p2.get_age()
63

In the code above, age is an instance variable and population is a static variable. population is
shared by all instances of the Person class whereas each instance has its own age variable.

More Python Tips and Tricks

This tutorial has briefly touched on some major aspects of Python that will be relevant to the
course. Here are some more useful tidbits:

• Use range to generate a sequence of integers, useful for generating

traditional indexed for loops:
• for index in range(3):
• print(lst[index])
• After importing a file, if you edit a source file, the changes will not be
immediately propagated in the interpreter. For this, use the reload
command:

>>> reload(shop)
Python Object Inheritance
Inheritance is the process by which one class takes on the attributes and methods of another.
Newly formed classes are called child classes, and the classes that child classes are derived from
are called parent classes.

It’s important to note that child classes override or extend the functionality (e.g., attributes and
behaviors) of parent classes. In other words, child classes inherit all of the parent’s attributes and
behaviors but can also specify different behavior to follow. The most basic type of class is an
object, which generally all other classes inherit as their parent.

When you define a new class, Python 3 it implicitly uses object as the parent class. So the
following two definitions are equivalent: class Dog(object): pass

# In Python 3, this is the same as:

class
Dog:
pass

Note: In Python 2.x there’s a distinction between new-style and old-style classes. I won’t go into
detail here, but you’ll generally want to specify object as the parent class to ensure you’re
definint a new-style class if you’re writing Python 2 OOP code.

Dog Park Example

Let’s pretend that we’re at a dog park. There are multiple Dog objects engaging in Dog
behaviors, each with different attributes. In regular-speak that means some dogs are running,
while some are stretching and some are just watching other dogs. Furthermore, each dog has
been named by its owner and, since each dog is living and breathing, each ages.
What’s another way to differentiate one dog from another? How about the dog’s breed:
>>> class Dog:
... def __init__(self, breed): ...
self.breed = breed
...
>>> spencer = Dog("German Shepard")
>>> spencer.breed
'German Shepard'
>>> sara = Dog("Boston Terrier")
>>> sara.breed
'Boston Terrier'

Each breed of dog has slightly different behaviors. To take these into account, let’s create
separate classes for each breed. These are child classes of the parent Dog class.

Extending the Functionality of a Parent Class Create

a new file called dog_inheritance.py:
# Parent class class
Dog:

# Class attribute
species = 'mammal'

# Initializer / Instance attributes

def __init__(self, name, age):
self.name = name
self.age = age

# instance method def description(self): return

"{} is {} years old".format(self.name, self.age)

# instance method def speak(self, sound):

return "{} says {}".format(self.name, sound)

# Child class (inherits from Dog class) class

RussellTerrier(Dog): def run(self, speed):
return "{} runs {}".format(self.name, speed)

# Child class (inherits from Dog class) class

Bulldog(Dog): def run(self, speed):
return "{} runs {}".format(self.name, speed)

# Child classes inherit attributes and

# behaviors from the parent class jim
= Bulldog("Jim", 12)
print(jim.description())

# Child classes have specific attributes

# and behaviors as well print(jim.run("slowly"))

Read the comments aloud as you work through this program to help you understand what’s
happening, then before you run the program, see if you can predict the expected output.
You should see:
Jim is 12 years old
Jim runs slowly

We haven’t added any special attributes or methods to differentiate a RussellTerrier from a

Bulldog, but since they’re now two different classes, we could for instance give them different
class attributes defining their respective speeds.

Parent vs. Child Classes

The isinstance() function is used to determine if an instance is also an instance of a certain
parent class.
Save this as dog_isinstance.py:
# Parent class class
Dog:

# Class attribute
species = 'mammal'

# Initializer / Instance attributes

def __init__(self, name, age):
self.name = name self.age = age

# instance method def description(self): return

"{} is {} years old".format(self.name, self.age)

# instance method def speak(self, sound):

return "{} says {}".format(self.name, sound)

# Child class (inherits from Dog() class) class

RussellTerrier(Dog): def run(self, speed):
return "{} runs {}".format(self.name, speed)

# Child class (inherits from Dog() class) class

Bulldog(Dog): def run(self, speed):
return "{} runs {}".format(self.name, speed)

# Child classes inherit attributes and

# behaviors from the parent class jim
= Bulldog("Jim", 12)
print(jim.description())

# Child classes have specific attributes

# and behaviors as well print(jim.run("slowly"))

# Is jim an instance of Dog()? print(isinstance(jim,

Dog))

# Is julie an instance of Dog()? julie

= Dog("Julie", 100)
print(isinstance(julie, Dog))

# Is johnny walker an instance of Bulldog() johnnywalker

= RussellTerrier("Johnny Walker", 4)
print(isinstance(johnnywalker, Bulldog))

# Is julie and instance of jim? print(isinstance(julie,

jim))
Output:
('Jim', 12)
Jim runs slowly
True
True
False
Traceback (most recent call last):
File "dog_isinstance.py", line 50, in <module>
print(isinstance(julie, jim))
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes and
types

Make sense? Both jim and julie are instances of the Dog() class, while johnnywalker is not an
instance of the Bulldog() class. Then as a sanity check, we tested if julie is an instance of jim,
which is impossible since jim is an instance of a class rather than a class itself—hence the
reason for the TypeError.

Overriding the Functionality of a Parent Class

Remember that child classes can also override attributes and behaviors from the parent class. For
examples:
>>> class Dog:
... species = 'mammal'
...
>>> class SomeBreed(Dog): ...
pass
...
>>> class SomeOtherBreed(Dog): ...
species = 'reptile'
...
>>> frank = SomeBreed()
>>> frank.species
'mammal'
>>> beans = SomeOtherBreed()
>>> beans.species
'reptile'

The SomeBreed() class inherits the species from the parent class, while the SomeOtherBreed()
class overrides the species, setting it to reptile.

Exercises

1. Write a insertionSort function in Python using list comprehensions.

 2. Write a mergeSort function in Python using list comprehensions.

3. Write a quickSort function in Python using list comprehensions. Use the first element as
the pivot.

4. For the following Dog class, do the following exercise

class Dog:

# Class Attribute
species = 'mammal'

# Initializer / Instance
Attributes def __init__(self,
name, age): self.name = name
self.age = age

# Instantiate the Dog object

philo = Dog("Philo", 5)
mikey = Dog("Mikey", 6)

# Access the instance attributes

print("{} is {} and {} is {}.".format(
philo.name, philo.age, mikey.name, mikey.age))

# Is Philo a mammal? if philo.species == "mammal":

print("{0} is a {1}!".format(philo.name, philo.species))

Using the same Dog class, instantiate three new dogs, each with a different age. Then
write a function called, get_biggest_number(), that takes any number of ages (*args)
and returns the oldest one. Then output the age of the oldest dog like so:

Create a Pets class that holds instances of dogs; this class is completely separate from the Dog
class. In other words, the Dog class does not inherit from the Pets class. Then assign three dog
instances to an instance of the Pets class. Start with the following code below. Save the file as
pets_class.py. Your output should look like this:

I have 3 dogs. Tom

is 6.
Fletcher is 7.
Larry is 9.
And they're all mammals, of course.
 EXPERIMENT #3
INTRODUCTION
The purpose of this lab is to get familiar with Object Oriented concept of python

Name
Date
Registration No
Department
Total Marks
Marks Obtained
Remarks

___________________
Lab Instructor Signature
 Experiment 3
INTRODUCTION

OBJECTIVE

The goal of this tutorial is to understand oop concept of python

THEORY

Introduction

Object-oriented programming (OOP) is a style of programming that allows you to

think of code in terms of "objects." Here's an example of a Car class:

class Car(object):
num_wheels = 4
def __init__(self,
color):
self.wheels = Car.num_wheels
self.color = color
def drive(self): if
self.wheels <= Car.num_wheels:
return self.color + ' car cannot drive!'
return self.color + ' car goes vroom!'
def pop_tire(self):
if self.wheels > 0:
self.wheels -= 1

Here's some terminology:

● class: a blueprint for how to build a certain type of object. The Car class
(shown above) describes the behavior and data that all Car objects have.
 ● instance: a particular occurrence of a class. In Python, we create
instances of a class like this:
>>> my_car = Car('red')
my_car is an instance of the Car class.

attribute or field: a variable that belongs to the class. Think of an attribute as a quality
of the object: cars have wheels and color, so we have given our Car class self.wheels
and self.color attributes. We can access attributes using dot notation:
>>> my_car.color
'red'
>>> my_car.wheels

● 4

method: Methods are just like normal functions, except that they are tied to an
instance or a class. Think of a method as a "verb" of the class: cars can drive and also
pop their tires, so we have given our Car class the methods drive and pop_tire. We
call methods using dot notation:
>>> my_car = Car('red')
>>> my_car.drive()

● 'red car goes vroom!'

constructor: As with data abstraction, constructors describe how to build an

instance of the class. Most classes have a constructor. In Python, the constructor of the
class defined as __init__. For example, here is the Car class's constructor: def
__init__(self, color):
self.wheels = Car.num_wheels

● self.color = color
The constructor takes in one argument, color. As you can see, the constructor
also creates the self.wheels and self.color attributes.

self: in Python, self is the first parameter for many methods (in this class, we will
only use methods whose first parameter is self). When a method is called, self is
bound to an instance of the class. For example:
>>> my_car = Car('red')

● >>> car.drive()
Notice that the drive method takes in self as an argument, but it looks like we
 didn't pass one in! This is because the dot notation implicitly passes in car as
self for us.

Types of variables

When dealing with OOP, there are three types of variables you should be aware of:

● local variable: These are just like the variables you see in normal
functions — once the function or method is done being called, this variable
is no longer able to be accessed. For example, the color variable in the
__init__ method is a local variable (not the self.color variable).

instance attribute: Unlike local variables, instance attributes will still be accessible
after method calls have finished. Each instance of a class keeps its own version of the
instance attribute — for example, we might have two Car objects, where one's
self.color is red, and the other's self.color is blue.
>>> car1 = Car('red')
>>> car2 = Car('blue')
>>> car1.color 'red'
>>> car2.color
'blue'
>>> car1.color = 'yellow'
>>> car1.color 'yellow'
>>> car2.color

● 'blue'

class attribute: As with instance attributes, class attributes also persist across method
calls. However, unlike instance attributes, all instances of a class will share the same
class attributes. For example, num_wheels is a class attribute of the Car class.
>>> car1 = Car('red')
>>> car2 = Car('blue')
>>> car1.num_wheels 4
>>> car2.num_wheels
4
>>> Car.num_wheels = 2
>>> car1.num_wheels 2
>>> car2.num_wheels
●2
 Notice that we can access class attributes by saying <class
name>.<attribute>, such as Car.num_wheels, or by saying
<instance>.<attribute>, such as car1.num_wheels.

Question 1

Predict the result of evaluating the following calls in the interpreter. Then try them out
yourself!

>>> class Account(object):

... interest = 0.02
... def __init__(self, account_holder):
... self.balance = 0
... self.holder = account_holder
... def deposit(self, amount):
... self.balance = self.balance + amount ...
print("Yes!")
...
>>> a = Account("Billy")
>>> a.account_holder

______
>>> a.holder
_____
>>> Account.holder
______
>>> Account.interest
______
>>> a.interest
______
>>> Account.interest = 0.03
>>> a.interest
______
>>> a.deposit(1000)
______
>>> a.balance
______
>>> a.interest = 9001
>>> Account.interest
Question 2

Modify the following Person class to add a repeat method, which repeats the last
thing said. See the doctests for an example of its use.

Hint: you will have to modify other methods as well, not just the repeat
method.

class Person(object):
"""Person class.

>>> steven = Person("Steven")

>>> steven.repeat() # starts at whatever value you'd
like
'I squirreled it away before it could catch on fire.'
>>> steven.say("Hello")
'Hello'
>>> steven.repeat()
'Hello'
>>> steven.greet()
'Hello, my name is Steven'
>>> steven.repeat()
'Hello, my name is Steven'
>>> steven.ask("preserve abstraction barriers")
'Would you please preserve abstraction barriers' >>>
steven.repeat()
'Would you please preserve abstraction barriers'
""" def
__init__(self, name):
self.name = name
def say(self,
stuff):
return stuff
def ask(self,
stuff):
return self.say("Would you please " + stuff)
def
greet(self):
return self.say("Hello, my name is " + self.name)
def
repeat(self):
"*** YOUR CODE HERE
***"

Inheritance
Question 3

Predict the result of evaluating the following calls in the interpreter. Then try them out
yourself!

>>> class Account(object):

... interest = 0.02
... def __init__(self, account_holder):
... self.balance = 0
... self.holder = account_holder
... def deposit(self, amount):
... self.balance = self.balance + amount ...
print("Yes!")
...
>>> class CheckingAccount(Account):
... def __init__(self, account_holder):
... Account.__init__(self, account_holder)
... def deposit(self, amount):
... Account.deposit(self, amount) ...
print("Have a nice day!")
...
>>> a = Account("Billy")
>>> a.balance

______

>>> c = CheckingAccount("Eric")
>>> c.balance

______

>>> a.deposit(30)
______

>>> c.deposit(30)

______

>>> c.interest

______

Question 4

Suppose now that we wanted to define a class called DoubleTalker to represent people
who always say things twice:

>>> steven = DoubleTalker("Steven")

>>> steven.say("hello")
"hello hello"
>>> steven.say("the sky is falling")
"the sky is falling the sky is falling"

Consider the following three definitions for DoubleTalker that inherit from the
Person class:

class DoubleTalker(Person): def

__init__(self, name):
Person.__init__(self, name) def
say(self, stuff):
return Person.say(self, stuff) + " " + self.repeat()
class DoubleTalker(Person):
def __init__(self, name):
Person.__init__(self, name) def
say(self, stuff):
return stuff + " " + stuff
class DoubleTalker(Person):
def __init__(self, name):
Person.__init__(self, name) def
say(self, stuff):
return Person.say(self, stuff + " " + stuff)
Determine which of these definitions work as intended. Also determine for which of the
methods the three versions would respond differently. (Don't forget about the repeat
method!)

Question 5

Modify the Account class from lecture so that it has a new attribute, transactions,
that is a list keeping track of any transactions performed. See the doctest for an example.

class Account(object):
"""A bank account that allows deposits and withdrawals.

>>> eric_account = Account('Eric')

>>> eric_account.deposit(1000000) # depositing my paycheck
for the week 1000000
>>> eric_account.transactions
[('deposit', 1000000)]
>>> eric_account.withdraw(100) # buying dinner
999900
>>> eric_account.transactions
[('deposit', 1000000), ('withdraw', 100)]
"""
interest = 0.02
def __init__(self,
account_holder): self.balance =
0 self.holder = account_holder
def deposit(self,
amount):
"""Increase the account balance by amount and return the
new balance. """ self.balance = self.balance +
amount return self.balance
def withdraw(self,
amount):
"""Decrease the account balance by amount and return the
new balance. """ if amount > self.balance:
return 'Insufficient funds'
self.balance = self.balance - amount return
self.balance
Extra Questions
The following questions are for extra practice — they can be found in the the
lab06_extra.py file. It is recommended that you complete these problems as
well, but you do not need to turn them in for credit.

Question 6

We'd like to be able to cash checks, so let's add a deposit_check method to our
CheckingAccount class. It will take a Check object as an argument, and check to see
if the payable_to attribute matches the CheckingAccount's holder. If so, it marks
the Check as deposited, and adds the amount specified to the CheckingAccount's
total.

Write an appropriate Check class, and add the deposit_check method to the
CheckingAccount class. Make sure not to copy and paste code! Use inheritance
whenever possible.

See the doctests for examples of how this code should work.

class CheckingAccount(Account):
"""A bank account that charges for withdrawals.

>>> check = Check("Steven", 42) # 42 dollars, payable to

Steven
>>> steven_account = CheckingAccount("Steven")
>>> eric_account = CheckingAccount("Eric")
>>> eric_account.deposit_check(check) # trying to steal
steven’s money
The police have been notified.
>>> eric_account.balance
0
>>> check.deposited
False
>>> steven_account.balance
0
>>> steven_account.deposit_check(check)
42
>>> check.deposited
 True
>>> steven_account.deposit_check(check) # can't cash check
twice
The police have been notified.
"""
withdraw_fee = 1
interest = 0.01
def withdraw(self,
amount):
return Account.withdraw(self, amount +
self.withdraw_fee)
class
Check(object):

"*** YOUR CODE HERE ***"

Question 7

We'd like to create a Keyboard class that takes in an arbitrary number of Buttons and
stores these Buttons in a dictionary. The keys in the dictionary will be ints that
represent the postition on the Keyboard, and the values will be the respective Button.
Fill out the methods in the Keyboard class according to each description, using the
doctests as a reference for the behavior of a Keyboard.

class Keyboard:
"""A Keyboard takes in an arbitrary amount of buttons, and
has a dictionary of positions as keys, and values as
Buttons.

>>> b1 = Button(0, "H")

>>> b2 = Button(1, "I")
>>> k = Keyboard(b1, b2)
>>> k.buttons[0].key
'H'
>>> k.press(1)
'I'
>>> k.typing([0, 1])
'HI'
>>> k.typing([1, 0])
'IH'
 >>> b1.pressed
2
>>> b2.pressed
3
""" def
__init__(self, *args):

"*** YOUR CODE HERE ***"

def press(self,
info):
"""Takes in a position of the button pressed, and
returns that button's output"""

"*** YOUR CODE HERE ***"

def typing(self,
typing_input):
"""Takes in a list of positions of buttons pressed, and
returns the total output"""

"*** YOUR CODE HERE ***"

class Button: def

__init__(self, pos, key):
self.pos = pos
self.key = key
self.pressed = 0
 EXPERIMENT # 04
INTRODUCTION
The purpose of this lab is to understand how agents and environments interact

Name
Date
Registration No
Department
Total Marks
Marks Obtained
Remarks

___________________
Lab Instructor Signature
 Experiment
INTRODUCTION

04
OBJECTIVE

The objective of this lab is to show the loop of interaction between the agent and the
environment

THEORY

Agents and environments

Python code for the abstract Environment

@author: dr.aarij
'''
from abc import abstractmethod
class
Environment(object):
'''
classdocs
'''

@abstractmethod
 def __init__(self, n):
self.n = n
def executeStep(self,n=1): raise
NotImplementedError('action must be defined!')
def executeAll(self): raise
NotImplementedError('action must be defined!')
def delay(self,
n=100): self.delay
= n

For the Two Room Vacuum Cleaner Environment

from com.environment import

Environment from com.environment
import Room from com.agent import
VaccumAgent
class
TwoRoomVaccumCleanerEnvironment(Environment.Environment):
''' classdocs '''
def __init__(self, agent):
'''
Constructor
'''
self.r1 = Room.Room('A','dirty')
self.r2 = Room.Room('B','dirty')
self.agent = agent
self.currentRoom = self.r1
self.delay = 1000 self.step = 1
self.action = ""
def
executeStep(self,n=1):
for _ in range(0,n):
self.displayPerception()
self.agent.sense(self)
res = self.agent.act()
self.action = res if res
== 'clean':
self.currentRoom.status = 'clean'
elif res == 'right':
self.currentRoom = self.r2
else: self.currentRoom =
self.r1 self.displayAction()
self.step += 1
def executeAll(self): raise
NotImplementedError('action must be defined!') def
displayPerception(self): print("Perception at step
%d is [%s,%s]"
%(self.step,self.currentRoom.status,self.currentRoom.location))
def displayAction(self): print("------- Action taken at step
%d is [%s]" %(self.step,self.action))
def delay(self,
n=100):
 self.delay = n

Room class

@author: dr.aarij
''' class
Room:
def __init__(self,location,status="dirty"):
self.location = location
self.status = status

Abstract agent

@author: dr.aarij
'''
from abc import abstractmethod
class
Agent(object):
'''
classdocs
'''

@abstractmethod
def __init__(self):
pass

@abstractmethod def
sense(self,environment):
pass

@abstractmethod
def act(self):
pass

Vaccum cleaner Agent

from com.agent import Agent

class
VaccumAgent(Agent.Agent):
'''
classdocs
'''

def
__init__(self):
'''
 Constructor
''' pass
def
sense(self,env):
self.environment = env
def act(self): if
self.environment.currentRoom.status == 'dirty':
return 'clean' if
self.environment.currentRoom.location == 'A':
return 'right' return 'left'

Test program
if __name__ == '__main__': vcagent =
VaccumAgent.VaccumAgent() env =
TwoRoomVaccumCleanerEnvironment(vcagent)
env.executeStep(50)

Exercises

1. Run the two room vacuum cleaner agent program and understand it. Convert the program
to a Three room environment
2. Convert the environment to a ‘n’ room environment where n >= 2
3. Does the agent ever stop? If no, can you make it stop? Is your program rational?
4. Score your agent, -1 points for moving from a room, +25 points to clean a room that is
dirty, and -10 points if a room is dirty. The scoring will take place after every 1 second
5. Convert the agent to a reflex based agent with a model. Afterwards take the sensors away
from the agents i.e. now the agent cannot perceive anything. Does your agent still work?
if so then why?
 EXPERIMENT # 05
INTRODUCTION
The purpose of this lab is to show you how to solve problems using uninformed search

Name
Date
Registration No
Department
Total Marks
Marks Obtained
Remarks

___________________
Lab Instructor Signature
 Experiment
INTRODUCTION

05
OBJECTIVE

The goal of this tutorial is to understand how to (i) model problems to solve and (ii) implement
the search algorithms as described in the book.

THEORY

Often we encounter problems that do not have a trivial solution i.e. more than one step is required
to solve the problem. To solve these kinds of problems we can search for solutions. We start with
the given state of the problem and then apply the relevant actions that are allowed at that
particular state. We repeat this process until either we get to the solution or we have no other
actions to apply, in which case we declare that there is no solution exists.

To solve such problems, the first step is to model such problems.

The modeling requires the following elements:

• A way to represent the ‘state’ of the problem

• The complete state space of the problem
• The successor function. This function tells that at any given state which actions can be
taken, what are the cost of those actions, and which new state our problem will be in, if
we choose and particular action.
• Initial state, the initial configuration of the problem
• Goal test, a test to verify whether our current state is the goal state or not.

In this lab we will look at some reusable code to model the search problem. In addition, we will
also look at some algorithms to solve these problems
//CLASS NODE: class
Node(object):
'''
classdocs
'''

def __init__(self, state, parentNode=None,depth=0,cost=0,action=''):

'''
Constructor
'''
self.state = state
self.parentNode = parentNode
self.depth =depth
self.cost = cost
self.action = action

def __str__(self) : return self.state + " --

"+self.action+" -- "+self.cost

def __gt__(self, other):

return self.cost > other.cost

//SEARCH PROBLEM CLASS

The SearchProblem class defines the general structure that each problem needs to define. All
problems should define how to model the state, how to generate successors and how to verify
whether the goal has been achieved or not.

from abc import abstractmethod

class
SearchProblem(object):

@abstractmethod
def __init__(self, params):pass
@abstractmethod def
initialState(self): pass
@abstractmethod
def succesorFunction(self,currentState): pass
@abstractmethod
def isGoal(self,currentState): pass
@abstractmethod
def __str__(self) : pass
//SEARCH STATE
The SearchState class is an abstraction of what every state should have. Every state should have a
representation of itself, the action that got the search to this particular state, the cost of getting to
this state and the string representation of the state. This string representation comes handy when
maintaining a duplicate state set.
from abc import
abstractmethod
class
SearchState(object):
'''
classdocs
'''

@abstractmethod
def __init__(self, params): pass
@abstractmethod
def getCurrentState(self): pass
@abstractmethod def
getAction(self): pass
@abstractmethod
def getCost(self):pass
@abstractmethod def
stringRep(self) : pass

Modelling 8 Puzzle problem as a searchproblem

Now we will show how the searchProblem abstract class can be used to model any search
problem. For an example we will model the 8 Puzzle problem. Actually the way we have
modelled this problem, any valid N-Puzzle problem can be solved.

Below is the EightPuzzleProblem class

from com.search.searchProblem import SearchProblem import

copy from com.search.eightPuzzleState import
EightPuzzleState class
EightPuzzleProblem(SearchProblem):
'''
classdocs
'''

def __init__(self, initialState,goalState):

'''
 Constructor
'''
self._initialState = EightPuzzleState(initialState, '', 0)
self._goalState = goalState self._numberOfRows =
len(initialState) self._numberOfColumns =
len(initialState[0])
def
initialState(self):
return self._initialState
def
succesorFunction(self,cs):
nextMoves = []
emptyRow,emptyColumn = 0,0
currentState = cs.currentState

emptyFound = False
for i in
range(len(currentState)):
for j in
range(len(currentState[i])): if
currentState[i][j] == 0:
emptyRow,emptyColumn = i,j
emptyFound = True break
if emptyFound:
break
#check up move
if emptyRow != 0:
newState = copy.deepcopy(currentState)
tempS = newState[emptyRow‐1][emptyColumn]
newState[emptyRow‐1][emptyColumn] = 0
newState[emptyRow][emptyColumn] = tempS ep =
EightPuzzleState(newState, 'Move Up', 1.0)
nextMoves.append(ep)

#check down move if emptyRow + 1 !=

self._numberOfRows: newState =
copy.deepcopy(currentState) tempS =
newState[emptyRow + 1][emptyColumn]
newState[emptyRow + 1][emptyColumn] = 0
newState[emptyRow][emptyColumn] = tempS ep =
EightPuzzleState(newState, 'Move Down', 1.0)
nextMoves.append(ep)
#check left move
if emptyColumn != 0:
newState = copy.deepcopy(currentState)
tempS = newState[emptyRow][emptyColumn‐1]
newState[emptyRow][emptyColumn‐1] = 0
newState[emptyRow][emptyColumn] = tempS ep =
EightPuzzleState(newState, 'Move Left', 1.0)
nextMoves.append(ep)
 #check right move if emptyColumn +1 !=
self._numberOfColumns: newState =
copy.deepcopy(currentState) tempS =
newState[emptyRow][emptyColumn+1]
newState[emptyRow][emptyColumn+1] = 0
newState[emptyRow][emptyColumn] = tempS ep =
EightPuzzleState(newState, 'Move Right', 1.0)
nextMoves.append(ep)

return nextMoves
def
isGoal(self,currentState):
cs = currentState.getCurrentState()
for i in range(len(cs)): for j in
range(len(cs[i])): if cs[i][j] !=
self._goalState[i][j]:
return False return
True

The state of the Puzzle problem is represented as a two dimensional array.

The successor functions whether there is a move available in the up, down, left and right direction.
If there is, then the next state, along with its action, cost is encapsulated in the EightPuzzle state
class shown below.

from com.search.searchState import SearchState

class
EightPuzzleState(SearchState):
'''
classdocs
'''

def __init__(self, currentState,action,cost):

'''
Constructor
'''
self.currentState =
currentState self.action =
action self.cost = cost
self.string = None
def
getCurrentState(self):
return self.currentState
def
getAction(self):
return self.action
def
getCost(self):
return self.cost
 def stringRep(self) :
if self.string is None:
e = '' for i in
range(len(self.currentState)): for j in
range(len(self.currentState[i])):
e += str(self.currentState[i][j])
self.string = e
return self.string

We also have a node class that represents the node of a search tree. The class is as
follows: class Node(object):
'''
classdocs
'''

def __init__(self, state, parentNode=None,depth=0,cost=0,action=''):

'''
Constructor
'''
self.state = state
self.parentNode = parentNode
self.depth =depth self.cost
= cost self.action = action
def __str__(self) : return self.state + " ‐‐
"+self.action+" ‐‐ "+self.cost

Search Strategy

As we have discussed in class, the different search strategies like BFS, DFS and UCS only differ
in the way they maintain the fringe list. Given below is the generic search strategy class. Every
search strategy requires only three operations. 1) to check whether the fringe list is empty or nor,
2) to add a node, 3) to remove a node from the list.

from abc import abstractmethod

class
SearchStrategy(object):
'''
classdocs
'''

@abstractmethod
def __init__(self, params):pass
@abstractmethod
def isEmpty(self):pass

@abstractmethod def
addNode(self,node):pass
 @abstractmethod
def removeNode(self):pass

For this experiment, I will show you how to implement the breadthfirstsearch
strategy. from com.search.strategy.searchStrategy import SearchStrategy
from queue import Queue
class
BreadthFirstSearchStrategy(SearchStrategy):
'''
classdocs
'''

def __init__(self):
self.queue = Queue()
def
isEmpty(self):
return self.queue.empty()

def addNode(self,node):
return self.queue.put(node)

def removeNode(self):
return self.queue.get()

Given these classes we can generically define the search class as follows:
from com.search.node import Node from
com.search.eightPuzzleProblem import EightPuzzleProblem
from com.search.strategy.breadthFirstSearchStrategy import BreadthFirstSearchStrategy
class
Search(object):
''' classdocs
'''

def __init__(self, searchProblem, searchStrategy):

'''
Constructor
'''
self.searchProblem = searchProblem
self.searchStrategy = searchStrategy
def solveProblem(self):
node = Node(self.searchProblem.initialState(), None, 0, 0, '')
self.searchStrategy.addNode(node)
duplicateMap = {}
duplicateMap[node.state.stringRep()] = node.state.stringRep()

result = None
 while not
self.searchStrategy.isEmpty():
.searchStrategy.isEmpty():
currentNode = self
self.searchStrategy.removeNode()
.searchStrategy.removeNode()
if self.searchProblem.isGoal(currentNode.state):
.searchProblem.isGoal(currentNode.state):
result = currentNode break
nextMoves = self.searchProblem.succesorFunction(currentNode.state)
.searchProblem.succesorFunction(currentNode.state)
for nextState in nextMoves:
if nextState.stringRep() not in duplicateMap:
newNode = Node(
Node(nextState,
nextState, currentNode, currentNode.depth + 1,
currentNode.cost + nextState.cost, nextState.action)
self.searchStrategy.addNode(newNode)
.searchStrategy.addNode(newNode)
duplicateMap[newNode.state.stringRep()] = newNode.state.stringRep()
return result
def
printResult(self,result): if
result.parentNode is None:
print("Game
"Game Starts"
Starts")
print("Initial
"Initial State : %s" % result.state.getCurrentState())
return
self.printResult(result.parentNode)
.printResult(result.parentNode) print("Perform
"Perform the
following action %s, New State is %s, cost is
%d"%(result.action,result.state.getCurrentState(),result.cost))
%(result.action,result.state.getCurrentState(),result.cost))

To initialize the search, we need to pass it the search problem and a searching strategy.
The search process if as follows:
ASSIGNMENT

1. Modify the code in the search class to count the number of nodes expanded by the search
.
2. Model the Sudoku solving problem .
3. Understand the code and implement the following search strategies,using problem solving
agent steps
I.Breadth first search
II.Depth first search
III.Uniform cost sea
 EXPERIMENT # 06
INTRODUCTION
The purpose of this lab is to show the working of an informed search agent

Name
Date
Registration No
Department
Total Marks
Marks Obtained
Remarks

___________________
Lab Instructor Signature
 Experiment
INTRODUCTION

06
OBJECTIVE

The goal of this tutorial is to understand how to (i) model problems to solve and (ii) implement
the informed search algorithms as described in the book.

THEORY

Much of the theory for informed search remains the same as it was for uninformed search. As
discussed in class, the main difference between informed search and uninformed search is the use
of heuristics.

Heuristics is a function that informs about the estimate of the shortest distance between the
“current state” and the “Goal”. Heuristics needs to be admissible i.e. the value given by the
heuristics should be less than the actual cost.

Here we will use the same classes as we did in the previous lab.

I will show the construction for “Greedy Search”. Remember, greedy search organizes the fringe
list in the increasing order of the nodes’ heuristic vales i.e. the node with the least heuristic value
is the node with the highest priority.

The code for greedy search is as follows:

from com.search.strategy.searchStrategy import SearchStrategy from

queue import PriorityQueue

class GreedySearch(SearchStrategy):
'''
classdocs
'''

def __init__(self, heuristic):

'''
Constructor
 '''

self.heuristic = heuristic
self.queue = PriorityQueue()

def isEmpty(self):
return self.queue == []

def addNode(self,node):
self.queue.put((self.heuristic.evaluateNode(node.state.currentState),node))
def removeNode(self):
return self.queue.get()[1]

As we have discussed, informed searches need a heuristic to work. Every search

problem has different ways describing their heuristics. I will show you the generic
description of a heuristic and then the implementation of the “Misplaced tiles”
heuristic for the eight puzzle problem

from abc import abstractclassmethod

class
Heuristic(object):
'''
classdocs
'''

@abstractclassmethod
def __init__(self): pass

@abstractclassmethod
def evaluateNode(self,state,goal): pass

from com.search.heuristic.heuristic import Heuristic

class MisPlacedTilesEightPuzzleHeuristic(Heuristic):
'''
classdocs
'''

def __init__(self,goal):
self.goal = goal

def evaluateNode(self,state):
totalCost = 0
for i in range(len(state)):
for j in range(len(state[i])):
 if state[i][j] != self.goal[i][j]:
totalCost += 1

return totalCost

the driver program

if __name__ == "__main__":
# 0,8,7,6,5,4,3,2,1
goal = [[0,1,2],[3,4,5],[6,7,8]]
heuristic = MisPlacedTilesEightPuzzleHeuristic(goal)
searchProblem = EightPuzzleProblem([[0,8,7],[6,5,4],[3,2,1]], goal)

searchStrategy = GreedySearch(heuristic)

search = Search(searchProblem, searchStrategy)

result = search.solveProblem()
if result is not None:
search.printResult(result)
ASSIGNMENT

1. Implement A-Star search

2. Implement the Manhattan heuristics for the Eight Puzzle problem
3. Compare the performance of both A-Star and Greedy search, in terms of number of nodes
expanded, using the Manhattan heuristic and the number of misplaced tiles heuristic
 EXPERIMENT # 07
INTRODUCTION
The purpose of this lab is to introduce you to solve problems by satisfying constraints

Name
Date
Registration No
Total Marks

___________________
Lab Instructor Signature
 Experiment
INTRODUCTION

07
OBJECTIVE

Thus far, we have seen how to search for solutions in a state space. In this lab, we will slightly
modify the search problem. We will see that we will search for values that will satisfy different
constraints. Our solution will be found once we have satisfied all the constraints and assigned
values to all the variables.

THEORY

Previously we have seen how to model the search problems and how to solve them using
informed and uninformed search.

The formulation we used to model our search problem, made the goal test, the successor function
and the structure of the state as a black box i.e. we didn’t know what composed a state, or how to
successors are generated, or when a goal is found.

In this lab, we will see a more specialized form of search formulation. Using this formulation our
problem will become as to assign some values to some variables while making sure that no
constraint, defined on the variables, is violated. Our formulation will require us to model our
problem in terms of Variables, the variables’ Domains and the Constraints defined on those
variables. More formally, this kind of problem solving technique is called constraint satisfaction
problems (CSP). CSP is formally defined as:

Variables (Xi)
Domains (Di)
Constraints

The solution of a CSP is found when we have assigned each variable (Xi) has been assigned a
value from its domain (Di) while all the constraints involving Xi are satisfied.

The constraints can be defined both implicitly or explicitly.

The objective of this lab is to demonstrate to you how to solve a CSP using different strategies.
The formulation used in our code has been inspired by the AIMA book:

A CSP is a collection of variables, their domains and the constraints between the different
variables:

class CSP(object):
'''
classdocs
'''

def __init__(self, variables = [], domains = [], constraints = []):

self._variables = variables
self._domain = domains
self._constraints = constraints
self._domainOfVariable = {}
self._contraintsOfVariable = {}
self.setUpVariableDomains()
self.setUpConstraints()

def setUpVariableDomains(self):
for var in self._variables:
self.addVariableDomain(var, self._domain)

def setUpConstraints(self):
for constraint in self._constraints:
self.addConstraint(constraint)

def addVariableDomain(self,var,domain):
self._domainOfVariable[var] = copy.deepcopy(domain)

def addConstraint(self,constraint):
for var in constraint.getScope():
if var not in self._contraintsOfVariable:
self._contraintsOfVariable[var] = []
self._contraintsOfVariable[var].append(constraint)

def addSingleConstraint(self,constraint):
self._constraints.append(constraint)
for var in constraint.getScope():
if var not in self._contraintsOfVariable:
self._contraintsOfVariable[var] = []
self._contraintsOfVariable[var].append(constraint)

def addVariable(self,variable):
self._variables.append(variable)
self.addVariableDomain(variable,self._domain)

def getVariables(self):
 return self._variables

def getDomainValues(self,var):
return self._domainOfVariable[var]
def getConstraints(self,var):
if var not in self._contraintsOfVariable:
return []
return self._contraintsOfVariable[var]

def getVariableDomains(self):
return self._domainOfVariable

def setVariableDomains(self,domainOfVariable):
self._domainOfVariable = domainOfVariable

def copy(self):
variables = copy.deepcopy(self._variables)
domains = copy.deepcopy(self._variables)
constraints = copy.deepcopy(self._variables)
csp = CSP(variables, domains, constraints)
return csp

def getNeighbour(self,variable,constraint):
neigh = []
for va in constraint.getScope():
if va != variable and (va not in neigh):
neigh.append(va)
return neigh

def removeValueFromDomain(self,variable,value):
values = []
for val in self.getDomainValues(variable):
if val != value:
values.append(val)
self._domainOfVariable[variable] = values

The different functions present in this class are necessary when solving a CSP problem. Their
role will become more evident as we present the other classes.

As mentioned earlier, a csp has three basic elements, variables, domains and constraints. The
classes for these elements are given below

from abc import abstractmethod

class
Constraint(object):
'''
classdocs
 '''

def __init__(self): pass

@abstractmethod
def isConsistentWith(self,assignment): pass

@abstractmethod
def getScope(self): pass

import copy
class
Domain(object):
'''
classdocs
'''

def __init__(self, values):

'''
Constructor
'''

self._values = copy.deepcopy(values)

def getValues(self):
return self._values
class
Variable(object):
'''
classdocs
'''

def __init__(self, name):

'''
Constructor
'''

self._name = name

def getName(self):
return self._name

def __eq__(self, other):

return self.getName() == other.getName()

def __hash__(self):
 return hash(self._name)

def __str__(self):
return self._name

While solving a csp using different strategies, we need to keep track at what point which
variables have been assigned values. This assignment is stored in the following class:

class
Assignment(object):
'''
classdocs
'''

def __init__(self):
self._variables = []
self._valueOfVariable = {}

def addVariableToAssignment(self,var,value):
if var not in self._valueOfVariable:
self._variables.append(var)
self._valueOfVariable[var] = value

def removeVariableFromAssignment(self,var):
if var in self._valueOfVariable:
self._variables.remove(var)
del self._valueOfVariable[var]

def getAssignmentOfVariable(self,var):
if var not in self._valueOfVariable:
return None
return self._valueOfVariable[var]

def isConsistent(self,constraints):
for con in constraints:
if not con.isConsistentWith(self):
return False

return True
 def hasAssignmentFor(self,var):
return var in self._valueOfVariable

def isComplete(self,variables):
for var in variables:
if not self.hasAssignmentFor(var):
return False
return True

def isSolution(self,csp):
return self.isComplete(csp.getVariables()) and
self.isConsistent(csp.getConstraints())

def __str__(self):
result = []

for var in self._variables:

result.append(("%s = %s")%(var,self._valueOfVariable[var]))

return str(result)

The functions in this class are necessary for problem solving.

Now that we have defined the basic data structures of a csp. Let us discuss how can we solve a
csp. A CSP can be solved using a strategy called backtracking search.

Backtracking search is like DFS with two important difference. 1)

We keep track which variables to choose next
2) We backtrack as soon as any constraint is violated.
The code for backtracking search is given below:

from com.ai.csp.strategy.searchStrategy import SearchStrategy

from com.ai.csp.assignment.assignment import Assignment from
com.ai.csp.inference.inferenceInfo import InferenceInfo
import math

class BactrackingSearch(SearchStrategy):
'''
classdocs
'''
 def __init__(self, inferenceProcdeure,listeners =
[],variableOrdering=False,valueOrdering=False):
'''
Constructor
'''
SearchStrategy.__init__(self, listeners)
self._inferenceProcedure = inferenceProcdeure
self._variableOrdering = variableOrdering
self._valueOrdering = valueOrdering

def solve(self,csp):
return self.recursiveBacktrackingSearch(csp, Assignment())

def recursiveBacktrackingSearch(self,csp,assignment):
if assignment.isComplete(csp.getVariables()):
return assignment
var = self.selectUnAssignedVariable(csp, assignment)

for value in self.orderDomainValues(csp, var):

assignment.addVariableToAssignment(var,value)
self.fireListeners(csp,assignment)
if assignment.isConsistent(csp.getConstraints(var)):
inference = InferenceInfo(csp,var,value,self._inferenceProcedure)
inference.doInference(csp, var, value)
if not inference.isFailure(csp, var, value):
inference.setInferencesToAssignments(assignment,csp)
result = self.recursiveBacktrackingSearch(csp,assignment)
if result is not None:
return result
inference.restoreDomains(csp)
assignment.removeVariableFromAssignment(var)
return None

def selectUnAssignedVariable(self,csp,assignment):

for var in csp.getVariables():

if not assignment.hasAssignmentFor(var):
return var

def orderDomainValues(self,csp,var):
return csp.getDomainValues(var)

def fireListeners(self,csp,assignment):
for listener in self._listeners:
listener.fireChange(csp,assignment)

The algorithm is inspired from the AIMA book.

Sample Formulation
Different CSP problems would require different types of constraints. We will show an example
of the map coloring problem (as discussed in class).

Map coloring problem requires a NotEqual constraint. The implementation of this constraint is
given below:

from com.ai.csp.elements.constraint import Constraint

class
NotEqualConstraint(Constraint):
'''
classdocs
'''

def __init__(self, var1, var2):

'''
Constructor
'''
self._scope = [var1,var2]

def getScope(self):
return self._scope

def isConsistentWith(self,assignment):
val1 = assignment.getAssignmentOfVariable(self._scope[0])
val2 = assignment.getAssignmentOfVariable(self._scope[1])
return val1 == None or val2 == None or val1 != val2

This is how we can formulate the map coloring problem using the classes given above
wa = Variable("WA")
sa = Variable("SA")
nt = Variable("NT")
q = Variable("Q")
nsw = Variable("NSW")
v = Variable("V")
t = Variable("T")
variables = [wa,sa,nt,q,nsw,v,t]
domains = ["RED","GREEN","BLUE"]

constraints = [NotEqualConstraint(wa,sa),
NotEqualConstraint(wa,nt),
NotEqualConstraint(nt,sa),
NotEqualConstraint(q,nt),
NotEqualConstraint(sa,q),
NotEqualConstraint(sa,nsw),
NotEqualConstraint(q,nsw),
NotEqualConstraint(sa,v),
NotEqualConstraint(nsw,v)]
 Csp = CSP(variables,domains,constraints)
inPro = SimpleInference() bts =
BactrackingSearch(inPro,[ConsoleListener()],variableOrdering = True)
True

start = time.time()
result = bts.solve(csp)
end = time.time()
print(end ‐ start)

where the simple inference class, is the vanilla inference. This can be
overridden for more advanced methods as Forward checking and Arc
Consistency.

ASSIGNMENT

Question 1) Model and solve the following m

map coloring problem
Question 2) Implement variable ordering heuristic in the backtracking class

Question 3) Implement the N N-Queen

Queen problem. You have to implement the
notattacking constraint as well
 EXPERIMENT # 08
INTRODUCTION
To introduce local search to solve the CSP problems

Name
Date
Registration No
Total Marks

___________________
Lab Instructor Signature
 Experiment
INTRODUCTION

08
OBJECTIVE

In this experiment we will learn how to solve CSPs using local search methods

THEORY
In the previous experiment, we learned about the basics of CSP and how to solve them using
searching algorithms.

As we have seen in class, that we can improve the performance of backtracking if we can detect
failure early. One way of detecting failure early is to propagate the impact of assigning a value to
a variable to that variable’s neighbors. Whenever, we assign a value to a variable, we can impose
the effect of this assignment on the neighboring variables. For example, if we consider the
problem of map coloring, then whenever we assign a color to a variable then we subtract that
color from the neighboring variables, and if by doing so the domain of any variable becomes
empty then we backtrack. In this way we can detect a failure a bit early. Thus, resulting in
improvement in the backtracking algorithm. This is called Forward Checking.

Given below is the code to implement forward checking.

'''

@author: dr.aarij
'''
from com.ai.csp.assignment.assignment import Assignment

class ForwardCheckingInference(object):
'''
classdocs
'''

def __init__(self):pass

def doInference(self,inferenceInfo,csp,variable,value):
 assignment = Assignment()
assignment.addVariableToAssignment(variable, value)
for con in csp.getConstraints(variable):
otherVariables = csp.getNeighbour(variable,con)
for ov in otherVariables:
someValues = []
changed = False
domVals = inferenceInfo.getDomainsOfAffectedVariables(ov)
if domVals is None:
domVals = csp.getDomainValues(ov)

for domVal in domVals:

assignment.addVariableToAssignment(ov, domVal)
if not con.isConsistentWith(assignment):
changed = True
else:
someValues.append(domVal)

if changed:
inferenceInfo.addToAffectedVariables(ov,someValues)

assignment.removeVariableFromAssignment(ov)
return []

This algorithm works as follows, as soon as we assign a value to a variable. We immediately do

some inference. This inference will get all the constraints of the variable, and then for every
constraint it will remove the values from the neighboring variables that are inconsistent with the
values of the original variable.

And if doing so, we find that the domain of any variable has emptied then we can claim that a
failure has occurred and now we have to backtrack.

Forward checking, still rums inside backtracking search and no need is required to make changes
in the backtracking algorithm.

To make backtracking search use forward checking following changes can be done in the main
csp.py file.

if __name__ == "__main__":
csp = createMapColoringCSP()
inPro = ForwardCheckingInference()

bts = BactrackingSearch(inPro,[ConsoleListener()],variableOrdering =
True)
 start = time.time()
result = bts.solve(csp)
end = time.time()
print(end ‐ start)

Arc Consistency

Forward checking is quite useful in detecting failure early, however, it cannot detect all kinds of
failures. For example, consider the following scenario of the map coloring problem, as discussed
in class.

It is obvious that the above situation will return in a failure because the variable NT and SA are
neighbors and both have the same value remaining. If we assign NT the color blue then the
domain of SA will become empty, and vice versa. Forward Checking cannot detect this failure
because Forward Checking can only detect where situations where the domain of a variable has
been emptied.

In order to detect this kind of failure we need a stronger form of filtering called arc-consistency.
We try to enforce that the values at every arc (constraint) are consistent. An arc X  Y is
consistent iff for every x in the tail there is some y in the head which could be assigned without
violating a constraint. A simple form of propagation makes sure all arcs are consistent.

Important: If X loses a value, neighbors of X need to be rechecked! Arc consistency detects

failure earlier than forward checking. Can be run as a preprocessor or after each assignment.

Below is the pseudo code for arc-consistency as in aima

As an exercise you need to implement arc-consistency.

Local Searches

Up till now we were finding a solution by incrementally building it up i.e. we select a variable at
every iteration and then assign it a value. We repeat this process until we have given a value to
every variable and all the constraints are satisfied.

One other way of solving CSP is to use a complete state formulation i.e. start by assigning every
variable some random values. Then at every iteration we select a variable that is violating any
constraint and assign it a value that is the Least Constraining on the other variables. We repeat
this process until all the constraints are satisfied.

This process is called local search, because we are just keeping the information about the current
state and how can we find a neighboring state. In general, local searches work with complete
state formulation, where they compare the current state with a neighboring state and if the
neighboring state is better than the current state then the neighboring state becomes the current
state and the process is repeated until all the constraints are satisfied.

In the AIMA book, we saw a local search algorithm to solve CSP called the min-conflicts search.
The pseudo code of the algorithm is given below:
The implementation of the pseudo code is given below:

from com.ai.csp.strategy.searchStrategy import SearchStrategy

from com.ai.csp.assignment.assignment import Assignment from
com.ai.csp.inference.inferenceInfo import InferenceInfo
import math import random
class
LocalSearch(SearchStrategy):
'''
classdocs
'''

def __init__(self, inferenceProcdeure,listeners =

[],variableOrdering=False,valueOrdering=False,maxSteps = 100000):
'''
Constructor
'''
SearchStrategy.__init__(self, listeners)
self._inferenceProcedure = inferenceProcdeure
self._variableOrdering = variableOrdering
self._valueOrdering = valueOrdering
self._maxSteps = maxSteps

def intiliazeRandomly(self,csp):
self._assignment = Assignment()
domainLength = len(csp.getListOfDomains())
for va in csp.getVariables():
self._assignment.addVariableToAssignment(va,
csp.getListOfDomains()[random.randint(0,domainLength‐1)])

def solve(self,csp):
self.intiliazeRandomly(csp)
 for _ in range(self._maxSteps):
if self._assignment.isSolution(csp):
return self._assignment
cands = self.getConflictedVariable(csp)
var = cands[random.randint(0,len(cands)‐1)]
val = self.getMinConflictValueFor(var,csp)
# print(str(var)+"_"+str(val)+"__"+str(len(cands)))
self.fireListeners(csp, self._assignment)
self._assignment.addVariableToAssignment(var,val)

return False

def getConflictedVariable(self,csp):
resultVariables = []

for con in csp.getListOfConstraints():

if not self._assignment.isConsistent([con]):
for var in con.getScope():
if var not in resultVariables:
resultVariables.append(var)

return resultVariables

def getMinConflictValueFor(self,var,csp):

constraints = csp.getConstraints(var)
assignment = self._assignment.returnCopy()
minConflict = 100000000000
candidates = []

for val in csp.getDomainValues(var):

assignment.addVariableToAssignment(var,val)
count = 0
for con in constraints:
if assignment.isConsistent([con]):
count+=1
if count <= minConflict:
if count < minConflict:
candidates = []
count = minConflict
candidates.append(val)

return candidates[random.randint(0,len(candidates)‐1)]

def fireListeners(self,csp,assignment):
for listener in self._listeners:
listener.fireChange(csp,assignment)
ASSIGNMENT

Question 1) Implement Arc-Consistency

Question 2) Compare the performance of Forward checking and Forward checking with
arcconsistency

Question 3) Use local search to solve the n-queen problem. Compare its performance with
arcconsistency for 1000 queen problem

Question 4) Implement Simulated annealing for local searching.

 EXPERIMENT # 09
INTRODUCTION
In this lab we will learn how to search in an adversarial environment

Name
Date
Registration No
Total Marks

___________________
Lab Instructor Signature
OBJECTIVE

The objectives are as follows.

• understand the basic concept of adversarial search

• be able to write programs that simulates adversarial environments
• be able to formulate real world problems
• be able to use different algorithms to find optimal decisions in adversarial environments.

Theory

So far, we have seen single agent environments. However, there are environments where there is
more than one agent. These are called multi agent environments. Here we are focusing on the
environments that are adversarial i.e. where your agent is up against an agent that is trying to
minimize your utility. The more the adversarial agent minimize your agent’s utility the more the
adversarial agent will maximize its utility.

Board games like chess, checkers, ludo, tic-tac-toe, etc. are examples of this kind of adversarial
environments. These games are turn taking games. Unlike, search, we cannot plan for victory
because after your agent has made a move, the next move belongs to the adversary.

In these environments, we must devise a strategy that make an optimal decision given a state. We
can model these games as a search problem as follows:
We can have an abstract representation of the above formulation as follows:

from abc import abstractmethod

class Game(object):
''' classdocs
'''

@abstractmethod
def __init__(self, params): pass
@abstractmethod
def getInitialState(self):pass
@abstractmethod
def getPlayer(self,state):pass
@abstractmethod
def getActions(self,state): pass
@abstractmethod
def getResult(self, state, action): pass
@abstractmethod
def terminalTest(self,state): pass
@abstractmethod
def utility(self,state,player): pass
@abstractmethod
def getAgentCount(self): pass

Similarly, an abstract class for a game’s state could be designed as follows :

class
State(object):
'''
 classdocs
'''

def __init__(self, params):pass

def isMax(self): pass

def isNextAgentMax(self): pass

def getAction(self):pass

For now, we will consider that the environment is deterministic i.e. we know the outcomes of an
action taken by any agent at any particular state. To make an optimal decision our agent has to
simulate the actions taken by the adversarial agent if the agent chooses any particular agent. In
order to simulate the agent, we need to simulate what the adversarial agent will be thinking about
our agent, and what our agent will be thinking about the adversarial agent and so on. This is
graphically depicted in the following diagram.

So for all the possible actions, available to our agent, at any given state, our agent will take the
action that returns the maximum value. Whereas, the adversarial agent will choose an action that
returns the minimum value, as the lower the value the better it is for our adversarial agent.

One algorithm that calculates these values is called the minimax algorithm. There pseudo code is
given below:
The value of an action is recursively calculated at the terminal nodes, as shown in the figure
below:

We will simulate this behavior in this experiment.

To simulate a search tree, we will create a tree data structure as follows:

from com.ai.adversarial.elements.state import State

class AdversarialNode(State):
'''
 classdocs
'''

def __init__(self, value, name, isMax, children = []):

'''
Constructor '''
self._name = name
self._utility = value
self._isMax = isMax
self._children = children
self._move = 1
self._action = None

def isLeaf(self):
return len(self._children) == 0

def isMax(self):
return self._isMax

def isNextAgentMax(self):
return not self._isMax

def addChild(self,child):
self._children.append(child);

def getAction(self):
return self._action

def __str__(self):
s = "Name is %s, value is %d" %(self._name,self._utility)
s += "\n children are "
for ch in range(len(self._children)):
s+= str(self._children[ch])
return s

To model a tree such as shown in the figure above we could make a game for it:

from com.ai.adversarial.elements.game import Game

from com.ai.adversarial.sample.treegame.adversarialNode import
AdversarialNode import
sys
from com.ai.adversarial.search.simpleMinimax import SimpleMinimax

class MinimaxTreeGame(Game):
'''
classdocs
 '''

def __init__(self):
''' Constructor

A
B C D
E F G H I J K L M 3
12 8 2 4 6 14 5 2 '''

bottom1 = [AdversarialNode(3,"E",True,[]),
AdversarialNode(12,"F",True,[]),
AdversarialNode(8,"G",True,[])]
bottom2 = [AdversarialNode(2,"H",True,[]),
AdversarialNode(4,"I",True,[]),
AdversarialNode(6,"J",True,[])]

bottom3 = [AdversarialNode(14,"K",True,[]),
AdversarialNode(5,"L",True,[]),
AdversarialNode(2,"M",True,[])]

b = AdversarialNode(‐sys.maxsize ‐ 1,"B",False,bottom1)
c = AdversarialNode(‐sys.maxsize ‐ 1,"C",False,bottom2) d =
AdversarialNode(‐sys.maxsize ‐ 1,"D",False,bottom3)

a = AdversarialNode(‐sys.maxsize ‐ 1,"A",True,[b,c,d])

self._root = a

def getInitialState(self):
return self._root

def getPlayer(self,state):
return state.isMax()

def getActions(self,state):
return [x for x in range(len(state._children))]

def getResult(self, state, action):

return state._children[action]

def terminalTest(self,state):
return state.isLeaf()

def utility(self,state,player):
 return state._value

def getAgentCount(self):
return 2

def printState(self,state):
toPrintNodes = []
toPrintNodes.append(state)
while len(toPrintNodes) > 0:
node = toPrintNodes[0]
del toPrintNodes[0]
print("Name = %s, value = %d"%(node._name,node._utility))
toPrintNodes += node._children

An implementation of vanilla minimax is shown below. The code shown below is a bit different
from the pseudo code shown in the AIMA book. However, the code is more generic and could be
used to simulate environments where there could be more than one adversarial agents.

The pseudo code as discussed in-class :

The code that implements the pseudo code is shown below:

class SimpleMinimax(object):
'''
classdocs
 '''

def __init__(self, game, listeners = []):

'''
Constructor '''
self._game = game
self.listeners = listeners
self._expandedNodes = 0
self._duplicateStates = {}

def minimax_decision(self,state):
self._duplicateStates[str(state)] = state

if self._game.terminalTest(state):
return state._utility

if state.isMax():
return self.maxvalue(state)
else:
return self.minvalue(state)

def self
minvalue(
,state): ss =
str(state)
if ss in self._duplicateStates and self._duplicateStates[ss]._utility
> state._utility:
return state._utility
else:
self._duplicateStates[str(state)] = state

self._expandedNodes += 1
retValue = 1000000000000

# player = self._game.getPlayer(state)
actions = self._game.getActions(state)

for action in actions:

tempValue =
self.minimax_decision(self._game.getResult(state,action))
if tempValue < retValue:
retValue = tempValue
state._utility = retValue
state._action = action

return retValue
 def maxvalue(self,state):

ss = str(state)
if ss in self._duplicateStates and self._duplicateStates[ss]._utility
> state._utility:
return state._utility
else:
self._duplicateStates[str(state)] = state

self._expandedNodes += 1

retValue = ‐1000000000000

# player = self._game.getPlayer(state)
actions = self._game.getActions(state)

for action in actions:

tempValue =
self.minimax_decision(self._game.getResult(state,action))
if tempValue > retValue:
retValue = tempValue
state._utility = retValue
state._action = action

return retValue

The driver program for the simple minimax with the game tree discussed above is as follows:

if __name__ == "__main__":
game = MinimaxTreeGame()
minimax = SimpleMinimax(game)
initialState = game.getInitialState()
minimax.minimax_decision(initialState)
game.printState(initialState)

Modeling Tic Tac Toe:

The modeling of a game like tic tac toe is shown below:

class TictactoePlayer(object):
'''
classdocs
'''

def __init__(self, name, symbol):

self._name = name
self._symbol = symbol
self._moves = []

def __str__(self, *args, **kwargs):

return str(self._name)+"_"+str(self._sybmbol)

''' Created on Mar 25, 2019

@author: dr.aarij
'''
from copy import deepcopy

class TictactoeState(object):
'''
classdocs
'''

def __init__(self, board,move,utility=0):

self._board = board
self._move = move
self._utility = utility
self._action = None

def copy(self):
return TictactoeState(deepcopy(self._board),self._move,self._utility)
def isMax(self):
return self._move == 0

def isNextAgentMax(self):
return (self._move + 1) % 2 == 0

def getAction(self):
return self._action
 def __str__(self):

return str(self._board)+"_"+str(self._move)

''' Created on Mar 25, 2019

@author: dr.aarij
'''
from com.ai.adversarial.sample.tictactoe.tictactoeState import TictactoeState
from com.ai.adversarial.sample.tictactoe.tictactoePlayer import
TictactoePlayer
from com.ai.adversarial.elements.game import Game from
com.ai.adversarial.search.minimax import Minimax

class TicTacToeGame(Game):
'''
classdocs
'''

def __init__(self, move = 0):

self._board = [[0,0,0],[0,0,0],[0,0,0]]
self._move = move
self._agents =[ TictactoePlayer("Player","X"),
TictactoePlayer("Opponent","O")]

self._winningPositions = [[(0,0),(0,1),(0,2)],
[(1,0),(1,1),(1,2)], [(2,0),(2,1),(2,2)],
[(0,0),(1,0),(2,0)],
[(0,1),(1,1),(2,1)],
[(0,2),(1,2),(2,2)],
[(0,0),(1,1),(2,2)],
[(2,0),(1,1),(0,2)]]

def getInitialState(self):
return TictactoeState(self._board,self._move)

def getPlayer(self,state):
return self._agents[state._move]

def getActions(self,state):
actions = []

for i in range(3):
 for j in range(3):
if state._board[i][j] == 0:
actions.append((i,j))

return actions

def getResult(self, state, action):

newState = state.copy()
player = self.getPlayer(state)

newState._board[action[0]][action[1]] = player._symbol
newState._move = (newState._move + 1) % 2

winposfound = True
for pos in self._winningPositions:
winposfound = True for indpos
in pos:
if newState._board[indpos[0]][indpos[1]] != player._symbol:
winposfound = False
break
if winposfound:
break

if winposfound:
newState._move = ‐1
if player._symbol == "X":
newState._utility = 1 else:
newState._utility = ‐1
else:
zeroFound = False
for i in range(3):
for j in range(3):
if newState._board[i][j] == 0:
zeroFound = True
break
if zeroFound:
break
if not zeroFound:
newState._move = ‐1
newState._utility = 0
return newState

def terminalTest(self,state):
return state._move == ‐1
 def utility(self,state,player):
return state._utility

def getAgentCount(self):
return 2
ASSIGNMENT

1) Using the formulation of the adversarial tree, model the following tree, and print the values of
all the nodes:

2) Extend the simple minimax program to implement the alpha beta pruning algorithm

3) Using the formulation of the tic-tac-toe game, and play a game against it
 EXPERIMENT # 10
INTRODUCTION
In this lab we will learn how to solve AI problem:

Name
Date
Registration No
Total Marks

___________________
Lab Instructor Signature
OBJECTIVE

The objectives are as follows.

• understand the basic concept of problem solving

Theory

Missionaries and Cannibals problem:

Question: In this problem, three missionaries and three cannibals must cross a river
using a boat which can carry at most two people, under the constraint that, for both
banks, that the missionaries present on the bank cannot be outnumbered by cannibals.
The boat cannot cross the river by itself with no people on board.
Solution:
First let us consider that both the missionaries (M) and cannibals(C) are on the same
side of the river.
Left Right
Initially the positions are : 0M , 0C and 3M , 3C (B)
Now let’s send 2 Cannibals to left of bank : 0M , 2C (B) and 3M , 1C

Send one cannibal from left to right : 0M , 1C and 3M , 2C (B)

Now send the 2 remaining Cannibals to left : 0M , 3C (B) and 3M , 0C
Send 1 cannibal to the right : 0M , 2C and 3M , 1C (B)
Now send 2 missionaries to the left : 2M , 2C (B) and 1M . 1C
Send 1 missionary and 1 cannibal to right : 1M , 1C and 2M , 2C (B)
Send 2 missionaries to left : 3M , 1C (B) and 0M , 2C
Send 1 cannibal to right : 3M , 0C and 0M , 3C (B)
Send 2 cannibals to left : 3M , 2C (B) and 0M , 1C
Send 1 cannibal to right : 3M , 1C and 0M , 2C (B)’
Send 2 cannibals to left : 3M , 3C (B) and 0M , 0C
• Here (B) shows the position of the boat after the action is performed.
Therefore all the missionaries and cannibals have crossed the river safely.
DEMO CODE:
ASSIGNMENT:

A. Modilfy above Code and solve Missionaries and Cannibals problem.

B. Solve water jug problem
 EXPERIMENT # 11
INTRODUCTION
In this lab we will learn how to Implement Markov Decision model

Name
Date
Registration No
Total Marks

___________________
Lab Instructor Signature
OBJECTIVE

The objectives are as follows.

• understand the basic concept of Markov Decision Process

Theory
Markov Decision Processes (MDPs)
1. MDP formalism
2. Value Iteration
3. Policy Iteration

Markov Decision Process

Reinforcement Learning :

Reinforcement Learning is a type of Machine Learning. It allows machines and software agents
to automatically determine the ideal behavior within a specific context, in order to maximize its
performance. Simple reward feedback is required for the agent to learn its behavior; this is
known as the reinforcement signal.
There are many different algorithms that tackle this issue. As a matter of fact, Reinforcement
Learning is defined by a specific type of problem, and all its solutions are classed as
Reinforcement Learning algorithms. In the problem, an agent is supposed to decide the best
action to select based on his current state. When this step is repeated, the problem is known as
a Markov Decision Process.
A Markov Decision Process (MDP) model contains:

 A set of possible world states S.

 A set of Models.
 A set of possible actions A.
 A real valued reward function R(s,a).
 A policy the solution of Markov Decision Process.

Key Reinforcement Learning Terms for MDPs

The following sections explain the key terms of reinforcement learning, namely:

 Policy: Which actions the agent should execute in which state

  State-value function: The expected value of each state with regard to future rewards
 Action-value function: The expected value of performing a specific action in a specific
state with regard to future rewards
 Transition probability: The probability to transition from one state to another
 Reward function: The reward that the agent obtains when transitioning between states

Let us take the example of a grid world:

An agent lives in the grid. The above example is a 3*4 grid. The grid has a START state(grid no
1,1). The purpose of the agent is to wander around the grid to finally reach the Blue Diamond
(grid no 4,3). Under all circumstances, the agent should avoid the Fire grid (orange color, grid
no 4,2). Also the grid no 2,2 is a blocked grid, it acts like a wall hence the agent cannot enter it.
The agent can take any one of these actions: UP, DOWN, LEFT, RIGHT

Walls block the agent path, i.e., if there is a wall in the direction the agent would have taken,
the agent stays in the same place. So for example, if the agent says LEFT in the START grid he
would stay put in the START grid.
First Aim: To find the shortest sequence getting from START to the Diamond. Two such
sequences can be found:
 RIGHT RIGHT UP UP RIGHT
 UP UP RIGHT RIGHT RIGHT
Let us take the second one (UP UP RIGHT RIGHT RIGHT) for the subsequent discussion.
The move is now noisy. 80% of the time the intended action works correctly. 20% of the time
the action agent takes causes it to move at right angles. For example, if the agent says UP the
probability of going UP is 0.8 whereas the probability of going LEFT is 0.1 and probability of
going RIGHT is 0.1 (since LEFT and RIGHT is right angles to UP).
The agent receives rewards each time step:-
 Small reward each step (can be negative when can also be term as punishment, in the
above example entering the Fire can have a reward of -1).
 Big rewards come at the end (good or bad).
 The goal is to Maximize sum of rewards.

ALGORITHM:
DEMO CODE POLICY ITERATION 1:(ELEMENTS)
importsys
Importrandom
class MDP(object):

def __init__(self,states,actions,transition,reward,discount=0.5):
self._states = states
self._actions = actions
self._transition = transition
self._reward = reward
self._discount = discount
self._initial_v = [ 0 for _ in states]
self._initial_q = [[0 for _ in actions] for _ in states]

def valueIteration(self,iterations = 0,threshold = 0.000000001):

previousMatrix = self._initial_v
returnQMatrix = [[0 for _ in self._actions] for _ in self._states]

delta = 0.0
for _ in range(iterations):
returnMatrix = [ 0 for _ in self._states]
for s in range(len(self._states)):
maxValue = -sys.maxsize - 1
for a in range(len(self._actions)):
actionValue = 0
possibleOutcomes = self._transition(s,a)
if len(possibleOutcomes) == 0:
maxValue = self._reward(s,a,None)
continue
for sp, prob in possibleOutcomes:
actionValue += prob * (self._reward(s,a,sp) +
self._discount * previousMatrix[sp])
returnQMatrix[s][a] = actionValue
maxValue = max(maxValue,actionValue)
returnMatrix[s] = maxValue
delta = max(delta, abs(previousMatrix[s] - returnMatrix[s]))
previousMatrix = returnMatrix
if delta < threshold:
break

return previousMatrix, returnQMatrix

 def policyEvaluation(self,policy,start_v,threshold):
while True:
delta = 0.0
for s in range(len(self._states)):
v = start_v[s]
actionValue = 0
possibleOutcomes = self._transition(s,policy[s])
if len(possibleOutcomes) == 0:
actionValue = self._reward(s,policy[s],None)
for sp, prob in possibleOutcomes:
actionValue += prob * (self._reward(s,policy[s],sp) +
self._discount * start_v[sp])
start_v[s] = actionValue
delta = max(delta, abs(v - start_v[s]))
if delta < threshold:
break

def policyIteration(self,threshold = 0.000000001):

'''intialize the policy and values'''
start_v = [random.random() * 100 for _ in self._states]
start_policy = [random.randint(0,len(self._actions)-1) for _ in
self._states]

while True:
policy_stable = True
self.policyEvaluation(start_policy, start_v, threshold)

for s in range(len(self._states)):
old_action = start_policy[s]
maxValue = -sys.maxsize - 1
for a in range(len(self._actions)):
actionValue = 0
possibleOutcomes = self._transition(s,a)
for sp, prob in possibleOutcomes:
actionValue += prob * (self._reward(s,a,sp) +
self._discount * start_v[sp])
if maxValue < actionValue:
maxValue = actionValue
start_policy[s] = a
if old_action != start_policy[s]:
policy_stable = False
 if policy_stable:
break

return start_policy, start_

DEMO CODE POLICY ITERATION 2(GRID):SAMPLE

from com.ai.mdp.element.mdp import MDP

class GridMDP(object):
'''
classdocs
'''

def __init__(self, file, noise=0.2,livingReward = 0.0):

self._livingReward = livingReward
self._noise = noise
self._states = []
self._actions = [(-1,0),(1,0),(0,-1),(0,1)]
self.readFile(file)

def readFile(self,file):
f = open(file,"+r")
lines = f.readlines()

self._rows = int(lines[0])
self._columns = int(lines[1])

for i in range(2, len(lines)):

self._states += [int(x) for x in lines[i].split(" ")]

# print(self._states)

def transition(self,state,action):
returnStates = []

if self._states[state] == 2:
 return returnStates

if self._states[state] == 3 or self._states[state] == 4:
return returnStates

stateRow = int (state / self._columns)

stateColumn = state % self._columns

possibleActions = [ (self._actions[action][0],self._actions[action][1],1-
self._noise),
((self._actions[action][0]**2 +
1)%2,(self._actions[action][1]**2 + 1)%2,self._noise/2.0),
( ((self._actions[action][0]**2 + 1)%2)*-1,
((self._actions[action][1]**2 + 1)%2)*-1,self._noise/2.0)]

for pa in possibleActions:
if stateRow + pa[0] >= 0 and\
stateRow + pa[0] < self._rows and\
stateColumn + pa[1] >= 0 and\
stateColumn + pa[1] < self._columns and\
self._states[int((stateRow + pa[0]) * self._columns + (stateColumn +
pa[1]))] != 2:

returnStates.append(((stateRow + pa[0]) * self._columns +

(stateColumn + pa[1]),pa[2]))
else:
returnStates.append((state,pa[2]))
return returnStates
def reward(self,s,a,sp):
if self._states[s] == 3:
return 1.0
if self._states[s] == 4:
return -1.0
return self._livingReward

if __name__ == "__main__":
grid = GridMDP("grid.txt",livingReward=-2.0)
mdp = MDP(grid._states, grid._actions, grid.transition, grid.reward, .9)
v = mdp.policyIteration()
print(v[0])
print(v[1])
Assignment

Understand above source code and apply policy iteration (MDP)on given grid
 LAB # 12
Reinforcement Learning
The purpose of this lab is to understand the problem solving using Reinforcement Learning

Name
Date
Registration No
Department
Quiz
Assignment

___________________
Lab Instructor Signature
 Experiment

OBJECTIVE
12
To solve the Taxi Parking Problem using Reinforcement Learning (Q Learning)

supervision: Breast Cancer and CIFAR-10.

Introduction

The Q-learning algorithm is used mainly as a simple algorithm for reinforcement. This uses the
environmental incentives to learn the best action to take in a given state over time. In the above
implementation, we have our "P" incentive table, where the agent learns from. Using the reward
table it chooses the next action if it’s beneficial or not and then updates a new value called the Q-
value. The newly generated table is called a combination of the Q-table and the maps named
(state, action). If the Q-values are higher we've got more incentives tailored.

Taxi Parking Problem

Imagine we teach a taxi how to move people to four different locations in a car park
(R, G, Y, B).

Prerequisite : Using OpenAi's Gym to set up the taxi-problem system, one of the most commonly
used libraries for solving reinforcement problems.
To install the library, use the Python package installer (pip): pip install gym

Now let's see how this will make our climate. All models and interfaces are already built in Gym
and called in: Taxi-V2

Use the following code to render this environment:

>>> import gym
>>> env = gym.make("Taxi-v2").env
>>> env.P[328]
{0: [(1.0, 433, -1, False)],
1: [(1.0, 233, -1, False)],
2: [(1.0, 353, -1, False)],
3: [(1.0, 333, -1, False)],
4: [(1.0, 333, -10, False)],
5: [(1.0, 333, -10, False)]
}
 Taxi – V2 Environment

The only car in this parking lot is the taxi. We will split the car park into a 5x5 grid, which gives
us 25 taxi locations. These 25 sites are a part of our room for the administration. Note that our
taxi's current position state is coordinate (3, 1).
There are four potential positions in the area where the passengers can be dropped off: R, G, Y,
B or [(0,0), (0,4), (4,0), (4,3)]in (row, col) coordinates if you can view the above-rendered area
as a coordinate axis.
If we also account for one (1) additional passenger status inside the taxi, we can take all
combinations of passenger locations and destination locations for our taxi setting to a total
number of states — there are four (4) destinations and five (4 + 1) passenger locations. And our
taxi system has 5 x 5 x5 x4=500 possible total states.
In other words, we have six possible actions: pickup, drop, north, east, south, west(These four
directions are the moves by which the taxi is moved.)

This is the action space: the set of all the actions that our agent can take in a given state.

To solve the problem without any reinforcement learning, we can set the target status, pick some
sample spaces and then if it achieves the target status with a number of iterations we conclude
that is the maximum reward, then the reward will be increased if it is close to the goal status and
the penalty will be raised if the reward for the move is -10 which is low.

Now let's code this problem without thinking about reinforcement. Since in each state we have
our P table for default incentives, we can try to make our taxi navigate using just that. We're
going to create an infinite loop that runs until one passenger reaches a destination (one episode),
or in other words when the reward received is 20.The env.action_space.sample() method
automatically selects one random action from set of all possible actions.
import gym
from time import sleep

# Creating thr env

env = gym.make("Taxi-v2").env
env.s = 328

# Setting the number of iterations, penalties and reward to zero,

epochs = 0
penalties, reward = 0, 0
frames = []
done = False
while not done:
action = env.action_space.sample()
state, reward, done, info = env.step(action)
if reward == -10:
penalties += 1
# Put each rendered frame into the dictionary for animation
frames.append({
'frame': env.render(mode='ansi'),
'state': state,
'action': action,
'reward': reward
}
)

epochs += 1

print("Timesteps taken: {}".format(epochs))

print("Penalties incurred: {}".format(penalties))

# Printing all the possible actions, states, rewards.

def frames(frames):
for i, frame in enumerate(frames):
clear_output(wait=True)
print(frame['frame'].getvalue())
print(f"Timestep: {i + 1}")
print(f"State: {frame['state']}")
print(f"Action: {frame['action']}")
print(f"Reward: {frame['reward']}")
sleep(.1)

frames(frames)
Solve Using Q-Learning
Learning Algorithm

When a taxi faces a state that involves a passenger at its current venue, the pick-up
pick Q-value is
highly likely to be higher relative to other behavior, ssuch as drop-off
off or north. Q-values are
initialized to an arbitrary value, and the Q
Q-values
values are modified using the equation as the agent
exposes itself to the world and receives various rewards for performing different actions:

How do we initialize and calculate

alculate the Q
Q-values? Despite that, the Q-values
values are initialized with
arbitrary constants. When the agent is exposed to the world it receives specific rewards by
performing various acts. When the acts are done the equation must execute the Q-values.
Q Here
the parameters for Q-learning
learning algorithm are alpha and gamma. Alpha is defined as learning rate
and the discount factor is gamma. All values vary from 0 to 1 and are rarely equal to one.
Gamma should be zero while alpha can not, so with any learning rate, tthe
he loss will be modified.
Here alpha reflects the same as in supervised learning. Gamma dictates exactly how much we
want to offer potential incentives.

import gym
import numpy as np
import random
from IPython.display import clear_output

# Init Taxi-V2 Env

env = gym.make("Taxi-v2").env

# Init arbitary values

q_table = np.zeros([env.observation_space.n, env.action_space.n])

# Hyperparameters
alpha = 0.1
gamma = 0.6
epsilon = 0.1

all_epochs = []
all_penalties = []

for i in range(1, 100001):

state = env.reset()

# Init Vars
epochs, penalties, reward, = 0, 0, 0
done = False
 while not done:
if random.uniform(0, 1) < epsilon:
# Check the action space
action = env.action_space.sample()
else:
# Check the learned values
action = np.argmax(q_table[state])

next_state, reward, done, info = env.step(action)

old_value = q_table[state, action]

next_max = np.max(q_table[next_state])

# Update the new value

new_value = (1 - alpha) * old_value + alpha * \
(reward + gamma * next_max)
q_table[state, action] = new_value

if reward == -10:
penalties += 1

state = next_state
epochs += 1

if i % 100 == 0:
clear_output(wait=True)
print("Episode: {i}")

print("Training finished.")

Reference : Vihar Kurama, Reinforcement learning with python: a guide to designing and solving problems
https://builtin.com/data-science/reinforcement-learning-python
Lab Assignment:

There are few obstacles present in between the locations (represented with smooth lines). L6 is
the highest priority location for the preparation of guitar bodies, including the polished wood.
Now the task is to allow the robots to find the shortest route on their own from any given
location to another location.

Reference : An introduction to Q-Learning: Reinforcement Learning,

https://blog.floydhub.com/an-introduction-to-q-learning-reinforcement-learning/
 LAB # 13
Bayesian Networks
The purpose of this lab is to understand the problem solving using Beyesian Networks

Name
Date
Registration No
Department
Quiz
Assignment

___________________
Lab Instructor Signature
 Experiment

OBJECTIVE
13
To Solve Monty Hall Problem Using Bayesian Networks In Python

supervision: Breast Cancer and CIFAR-10.

Introduction

Bayesian Networks have given shape to complex problems that provide limited resources and
knowledge. This is being applied in the era's most innovative technologies like Artificial
Intelligence and Machine Learning.
A Bayesian Network comes under the umbrella of Probabilistic Graphical Modeling (PGM)
methodology used to quantify uncertainties through the use of probability concepts. Bayesian
Networks, popularly known as the Belief Networks, use Directed Acyclic Graphs (DAG) to
model uncertainties. Bayesian models are based on the basic probability principle. Joint
Probability is a statistical indicator of two or more simultaneous occurrences, i.e., P(A, B, C), the
likelihood of occurrence of events A, B and C. It can be defined as the probability of two or
more things occurring at the intersection. Conditional likelihood of an event X is the likelihood
of the event occurring provided that an event Y has already occurred.

P(X| Y) is the probability of event X occurring, given that event, Y occurs.

 If X and Y are dependent events then the expression for conditional probability is given
by:
P (X| Y) = P (X and Y) / P (Y)
 If A and B are independent events then the expression for conditional probability is given
by:
P(X| Y) = P (X)

Therefore, we can formulate Bayesian Networks as:

Where, 𝑋 denotes a random variable, whose probability depends on the probability of the parent
nodes, 𝑃𝑎𝑟𝑒𝑛(𝑋 ).
Monty Hall Problem

The Monty Hall problem named after the host of the TV series, ‘Let’s Make A Deal’, is a
paradoxical probability puzzle that has been confusing people for over a decade.
The game includes three doors, provided that there is a car behind one of those doors and the
other two have goats behind them. Thus you begin by picking a random door, say # 2.
The host, on the other hand, knows where the car is hidden and he opens another door, say # 1 (a
goat behind it). Here's the catch, the host will ask you if you want to choose door # 3 instead of
your first choice i.e. # 2.

Is it better if you switch your choice or should you stick to your first choice? That is just what we
will be modeling for. We will create a Bayesian Network to understand the probability of
winning if the participant chooses to switch his option.

The first step is to build a Directed Acyclic Graph.

The graph has three nodes, each representing the door chosen by:

1. The door selected by the Guest

2. The door containing the prize (car)
3. The door Monty chooses to open

In the code blow 'A,' 'B' and 'C' reflect the guest picked doors, the reward door and the Monty
picked door. The conditional likelihood for each of the nodes is drawn out here. There is not
much to say as the reward door and the guest door are chosen randomly. However, the door
picked by Monty depends on the other two doors, so I've figured out the conditional likelihood in
the above code, taking all possible scenarios into consideration.

#Import required packages

import math
from pomegranate import *

# Initially the door selected by the guest is completely random

guest =DiscreteDistribution( { 'A': 1./3, 'B': 1./3, 'C': 1./3 } )

# The door containing the prize is also a random process

prize =DiscreteDistribution( { 'A': 1./3, 'B': 1./3, 'C': 1./3 } )

# The door Monty picks, depends on the choice of the guest and the prize door
monty =ConditionalProbabilityTable(
[[ 'A', 'A', 'A', 0.0 ],
[ 'A', 'A', 'B', 0.5 ],
[ 'A', 'A', 'C', 0.5 ],
[ 'A', 'B', 'A', 0.0 ],
[ 'A', 'B', 'B', 0.0 ],
[ 'A', 'B', 'C', 1.0 ],
[ 'A', 'C', 'A', 0.0 ],
[ 'A', 'C', 'B', 1.0 ],
[ 'A', 'C', 'C', 0.0 ],
[ 'B', 'A', 'A', 0.0 ],
[ 'B', 'A', 'B', 0.0 ],
[ 'B', 'A', 'C', 1.0 ],
[ 'B', 'B', 'A', 0.5 ],
[ 'B', 'B', 'B', 0.0 ],
[ 'B', 'B', 'C', 0.5 ],
[ 'B', 'C', 'A', 1.0 ],
[ 'B', 'C', 'B', 0.0 ],
[ 'B', 'C', 'C', 0.0 ],
[ 'C', 'A', 'A', 0.0 ],
[ 'C', 'A', 'B', 1.0 ],
[ 'C', 'A', 'C', 0.0 ],
[ 'C', 'B', 'A', 1.0 ],
[ 'C', 'B', 'B', 0.0 ],
[ 'C', 'B', 'C', 0.0 ],
[ 'C', 'C', 'A', 0.5 ],
[ 'C', 'C', 'B', 0.5 ],
[ 'C', 'C', 'C', 0.0 ]], [guest, prize] )

d1 = State( guest, name="guest" )

d2 = State( prize, name="prize" )
d3 = State( monty, name="monty" )

#Building the Bayesian Network

network = BayesianNetwork( "Solving the Monty Hall Problem With Bayesian
Networks" )
network.add_states(d1, d2, d3)
network.add_edge(d1, d3)
network.add_edge(d2, d3)
network.bake()
The next move is to use the model to make predictions. One of Bayesian networks 'strengths is
their ability to infer arbitrary' hidden variables 'values provided the values from' observed
variables. There is no need to define these hidden and observable variables in advance, because
the more variables that are observable the easier the inference would be on the hidden variables.

Now that we’ve built the model, it’s time to make predictions.
beliefs = network.predict_proba({ 'guest' : 'A' })
beliefs = map(str, beliefs)
print("n".join( "{}t{}".format( state.name, belief ) for state, belief in
zip( network.states, beliefs ) ))

guest A
prize {
"class" :"Distribution",
"dtype" :"str",
"name" :"DiscreteDistribution",
"parameters" :[
{
"A" :0.3333333333333333,
"B" :0.3333333333333333,
"C" :0.3333333333333333
}
],
}

monty {
"class" :"Distribution",
"dtype" :"str",
"name" :"DiscreteDistribution",
"parameters" :[
{
"C" :0.49999999999999983,
"A" :0.0,
"B" :0.49999999999999983
}
],
}

In the above code snippet, we’ve assumed that the guest picks door ‘A’. Given this information,
the probability of the prize door being ‘A’, ‘B’, ‘C’ is equal (1/3) since it is a random process.
However, the probability of Monty picking ‘A’ is obviously zero since the guest picked door
‘A’.
And the other two doors have a 50 percent chance of being selected by Monty because we don't
know what is the key to the draw.
beliefs = network.predict_proba({'guest' : 'A', 'monty' : 'B'})
print("n".join( "{}t{}".format( state.name, str(belief) ) for state, belief
in zip( network.states, beliefs )))

guest A
prize {
"class" :"Distribution",
"dtype" :"str",
"name" :"DiscreteDistribution",
"parameters" :[
{
"A" :0.3333333333333334,
"B" :0.0,
"C" :0.6666666666666664
}
],
}
monty B

In the above code snippet, we’ve provided two inputs to our Bayesian Network, this is where
things get interesting. We’ve mentioned the following:

1. The guest picks door ‘A’

2. Monty picks door ‘B’

Notice the output, the probability of the car being behind door ‘C’ is approx. 66%. This proves
that if the guest switches his choice, he has a higher probability of winning. Though this might
seem confusing to some of you, it’s a known fact that:

 Guests who decided to switch doors won about 2/3 of the time
 Guests who refused to switch won about 1/3 of the time

For these instances, Bayesian Networks are used, involving predicting unpredictable tasks and
performance. In the section below you'll understand how to use Bayesian Networks to solve
some of these issues.

Reference : Zulaikha Lateef, “How To Implement Bayesian Networks In Python? – Bayesian

Networks Explained With Examples” , https://www.edureka.co/blog/bayesian-networks/
 Exercise

Consider the following Directed Acyclic Graph with probability distribution table.

Find P(R,T,Y) using BN.

Reference : Muhammad Nazim Razali, https://towardsdatascience.com/basics-of-bayesian-

network-79435e11ae7b
 LAB # 14
Markov Chain
Markov Chains Implementation in Python

Name
Date
Registration No
Department
Quiz
Assignment

___________________
Lab Instructor Signature
 Experiment

OBJECTIVE
14
To Understand Markov Chain by Implementation in Python

supervision: Breast Cancer and CIFAR-10.

Introduction

A Markov chain is a mathematical system usually defined as a set of random variables that,
according to certain probabilistic rules, transition from one state to another. This transition set
satisfies the Markov Property, which states that the likelihood of transition to any given state
depends solely on the current state and time elapsed, and not on the sequence of state preceding
it. The peculiar property of Markov processes renders them memoryless.
A Markov chain with Markov property is a random operation. A random process is a
mathematical phenomenon known as a set of random variables, or also called stochastic
properties. A Markov chain has either a discrete state space (list of potential random variables
values) or a discrete index list (often time representation)-provided that there are several variants
for a Markov chain. The term "Markov chain" is typically reserved for a cycle that has a discrete
set of times, which is a Discrete Time Markov chain (DTMC).

Discrete Time Markov chain

Markov chain is a sequence of random variables X1, X2, X3, ... with the Markov property, such
that the probability of moving to the next state depends only on the present state and not on the
previous states. Putting this is mathematical probabilistic formula:

Pr( Xn+1 = x | X1 = x1, X2 = x2, …, Xn = xn) = Pr( Xn+1 = x | Xn = xn)

Implementation of Markov Chain in Pyhon

State Diagram

The Markov Chain shown in the state diagram has three possible statements: sleep, ride,
icecream. So, the matrix for the transformation will be 3 x 3. Remember that the arrows leaving a
state always sum up to exactly one, likewise the entries in each row in the transition matrix will
add up to exactly one-representing the distribution of probability. The cells do the same job in
the transition matrix as the arrows do in the State diagram.

Transition Matrix

With the example you saw, you can now address questions like: "Beginning from the state: sleep,
what is the probability of Cj running (state: run) at the end of a sad 2 day duration?"
Let's figure this one out: to switch from state: sleep to state: run, Cj will either remain in state:
sleep the first switch (or day), then move to state: run the next (second) move (0.2 . 0.6); or move
to state: run the first day and then remain the second day (0.6 . 0.6) or move to state: icecream on
the first move and then state: run in the second movement (0.2 . 0.7). Thus the likelihood: ((0.2 .
0.6) + (0.6 . 0.6) + (0.2 . 0.7)) = 0.62. So, we can now conclude that there's a 62 percent chance
Cj will move to state: run after two days of being sad, if she began to sleep in the state.
Therefore the probability: ((0.2.0.6) + (0.6.0.6) + (0.2.0.7))) = 0.62. So, we can now infer that Cj
is going to switch to state a 62 percent chance: run after two days of being sad if she started
sleeping in the house.

Markov Chains in Python

Let's try coding the above example in Python. And while you'd probably be using a library in
real life that encodes Markov Chains in a really effective way, the code will help you get started.

Let's first import some of the libraries you will use.

import numpy as np
import random as rm

Let's now define the states and their probability: the transition matrix. Remember, the matrix is
going to be a 3 X 3 matrix since you have three states. Also, you will have to define the
transition paths, you can do this using matrices as well.
# The statespace
states = ["Sleep","Icecream","Run"]

# Possible sequences of events

transitionName = [["SS","SR","SI"],["RS","RR","RI"],["IS","IR","II"]]

# Probabilities matrix (transition matrix)

transitionMatrix = [[0.2,0.6,0.2],[0.1,0.6,0.3],[0.2,0.7,0.1]]

Oh, always make sure the probabilities sum up to 1. And it doesn't hurt to leave error messages,
at least when coding
if sum(transitionMatrix[0])+sum(transitionMatrix[1])+sum(transitionMatrix[1])
!= 3:
print("Somewhere, something went wrong. Transition matrix, perhaps?")
else: print("All is gonna be okay, you should move on!! ;)")

Now let's code the real thing. You will use the numpy.random.choice to generate a random
sample from the set of transitions possible. While most of its arguments are self-explanatory, the
p might not be. It is an optional argument that lets you enter the probability distribution for the
sampling set, which is the transition matrix in this case.
# A function that implements the Markov model to forecast the state/mood.
def activity_forecast(days):
# Choose the starting state
activityToday = "Sleep"
print("Start state: " + activityToday)
# Shall store the sequence of states taken. So, this only has the
starting state for now.
activityList = [activityToday]
i = 0
# To calculate the probability of the activityList
prob = 1
while i != days:
if activityToday == "Sleep":
change =
np.random.choice(transitionName[0],replace=True,p=transitionMatrix[0])
if change == "SS":
prob = prob * 0.2
activityList.append("Sleep")
pass
elif change == "SR":
prob = prob * 0.6
activityToday = "Run"
activityList.append("Run")
else:
prob = prob * 0.2
activityToday = "Icecream"
activityList.append("Icecream")
elif activityToday == "Run":
change =
np.random.choice(transitionName[1],replace=True,p=transitionMatrix[1])
if change == "RR":
prob = prob * 0.5
activityList.append("Run")
pass
elif change == "RS":
prob = prob * 0.2
activityToday = "Sleep"
activityList.append("Sleep")
else:
prob = prob * 0.3
 activityToday = "Icecream"
activityList.append("Icecream")
elif activityToday == "Icecream":
change =
np.random.choice(transitionName[2],replace=True,p=transitionMatrix[2])
if change == "II":
prob = prob * 0.1
activityList.append("Icecream")
pass
elif change == "IS":
prob = prob * 0.2
activityToday = "Sleep"
activityList.append("Sleep")
else:
prob = prob * 0.7
activityToday = "Run"
activityList.append("Run")
i += 1
print("Possible states: " + str(activityList))
print("End state after "+ str(days) + " days: " + activityToday)
print("Probability of the possible sequence of states: " + str(prob))

# Function that forecasts the possible state for the next 2 days
activity_forecast(2)

You get a random set of transitions possible along with the probability of it happening, starting
from state: Sleep. Extend the program further to maybe iterate it for a couple of hundred times
with the same starting state, you can then see the expected probability of ending at any particular
state along with its probability. Let's rewrite the function activity_forecast and add a fresh
set of loops to do this...
def activity_forecast(days):
# Choose the starting state
activityToday = "Sleep"
activityList = [activityToday]
i = 0
prob = 1
while i != days:
if activityToday == "Sleep":
change =
np.random.choice(transitionName[0],replace=True,p=transitionMatrix[0])
 if change == "SS":
prob = prob * 0.2
activityList.append("Sleep")
pass
elif change == "SR":
prob = prob * 0.6
activityToday = "Run"
activityList.append("Run")
else:
prob = prob * 0.2
activityToday = "Icecream"
activityList.append("Icecream")
elif activityToday == "Run":
change =
np.random.choice(transitionName[1],replace=True,p=transitionMatrix[1])
if change == "RR":
prob = prob * 0.5
activityList.append("Run")
pass
elif change == "RS":
prob = prob * 0.2
activityToday = "Sleep"
activityList.append("Sleep")
else:
prob = prob * 0.3
activityToday = "Icecream"
activityList.append("Icecream")
elif activityToday == "Icecream":
change =
np.random.choice(transitionName[2],replace=True,p=transitionMatrix[2])
if change == "II":
prob = prob * 0.1
activityList.append("Icecream")
pass
elif change == "IS":
prob = prob * 0.2
activityToday = "Sleep"
activityList.append("Sleep")
else:
prob = prob * 0.7
 activityToday = "Run"
activityList.append("Run")
i += 1
return activityList

# To save every activityList

list_activity = []
count = 0

# `Range` starts from the first count up until but excluding the last count
for iterations in range(1,10000):
list_activity.append(activity_forecast(2))

# Check out all the `activityList` we collected

#print(list_activity)

# Iterate through the list to get a count of all activities ending in

state:'Run'
for smaller_list in list_activity:
if(smaller_list[2] == "Run"):
count += 1

# Calculate the probability of starting from state:'Sleep' and ending at

state:'Run'
percentage = (count/10000) * 100
print("The probability of starting at state:'Sleep' and ending at
state:'Run'= " + str(percentage) + "%")

Reference: Sejal Jaiswal, “Markov Chains in Python: Beginner Tutorial”

https://www.datacamp.com/community/tutorials/markov-chains-python-tutorial
 Exercise :

Implement the following Marcov Chain in Python

Reference: Markov Chain Monte Carlo, Population Health Method, Columbia University Mailman School of
Public Health https://www.mailman.columbia.edu/research/population-health-methods/markov-chain-monte-carlo