L. D.

College of Engineering
Opp Gujarat University, Navrangpura, Ahmedabad - 380015

LAB MANUAL
Honors/Minor Degree: Artificial Intelligence and Machine Learning

Branch: Computer Engineering

Introduction to AI and Machine

Learning (114AG01)
Semester: I V

Faculty Details:
1. Prof. M. K. Shah

2. Dr. N. H. Domadiya

3. Prof. H. D. Rajput
 Introduction to AI and Machine Learning (114AG01)

CERTIFICATE

This is to certify that Mr. Vatsal Maheshbhai Kathiriya ,

Enrollment Number 220280116044 has satisfactorily completed the

practical work in Introduction to AI and Machine Learning
(114AG01) subject at L D College of Engineering, Ahmedabad-380015.

Date of Submission: _______________

Sign of Faculty: _______________

Head of Department: _______________

Computer Science Department L. D. College of Engineering, Ahmedabad-15

 Introduction to AI and Machine Learning (114AG01)

L. D. College of Engineering, Ahmedabad

Department of Computer Engineering

Practical List
Subject Name: Introduction to AI and Machine Learning (114AG01)
Term: 2023-2024

Sr. Title Date Page Marks Sign

No. No. (10)

1 Implement following programs in python: Basics of 1

Python Programs

2 Study about numpy, pandas, Scikit-learn and matplotlib 8

libraries.

3 Getting Started with Python Logic Programming using 9

Kanren and SymPy packages

4 Write the code in Kanren to demonstrate the followings: 24

a) The use of logical variables.
b) The use of membero goal constructor.

5 Write the code in Kanren to create parent and 25

grandparent relationships and use it to state facts and
query based on the facts

6 Write the code in Kanren to demonstrate the constraint 26

system.

7 Write the code in Kanren to match the mathematical 27

expressions.

8 Write the code in python to implement linear regression 28

for one variable.

9 Write the code in python to implement linear regression 33

using gradient descent for one variable.

10 Write the code in python to implement logistic 38

regression for single class classification.

11 Write the code in python to implement logistic 46

regression for multi class classification.

Computer Science Department L. D. College of Engineering, Ahmedabad-15

 Introduction to AI and Machine Learning (114AG01)

L. D. College of Engineering, Ahmedabad

Department of Computer Engineering

Practical Rubrics

Subject Name: Introduction to AI and Machine Subject Code: (114AG01)

Learning

Term: 2022-2023

Rubrics ID Criteria Marks Good (2) Satisfactory (1) Need Improvement

(0)
Moderate (40-
RB1 Regularity 02 High (>70%) Poor (0-40%)
70%)
Limited
Appropriate & Full Very Less
Problem Identification of
Identification of Identification of the
Analysis and the Problem /
RB2 03 the Problem & Problem / Very Less
Development of Incomplete
Complete Solution Solution for the
Solution Solution for the
for the Problem Problem
Problem
Concept Clarity Concept is very Concept is
RB3 and 03 clear with proper clear up to Concept is not clear
Understanding understanding some extent.
Documentation Proper format not
RB4 Documentation 02 Not up to
completed followed,
standard.
neatly. incomplete.

SIGN OF FACULTY

Computer Science Department L. D. College of Engineering, Ahmedabad-15

 Introduction to AI and Machine Learning (114AG01)

L. D. College of Engineering, Ahmedabad-15

Computer Science Department
LABORATORY PRACTICALS ASSESSMENT

Subject Name: Introduction to AI and Machine Learning (114AG01)

Term: 2023-2024
Enrollment No.: 220280116044
Name: Vatsal Kathiriya

Pract. Faculty
RB1 RB2 RB3 RB4 Total Date
No. Sign

10

11

Computer Science Department L. D. College of Engineering, Ahmedabad-15

 Introduction to AI and Machine Learning (114AG01)

Table of Contents

Practical-1: Implement following programs in python: Basics of Python Program....................................... 1

1. Write a program to Convert Celsius to Fahrenheit and vice –a-versa based on the choice of user. ...........1
2. Write a program in python to implement menu driven simple calculator. .................................................2
3. Write a program which will allow user to enter 10 numbers and display largest odd and even number
from them. Also display the count of odd and even numbers. ........................................................................3
4. Write a Python program to check if the number provided by the user is a Prime number. Perform it with
and without user defined function...................................................................................................................4
5. Write a program to create a list of ten numbers entered by user and find smallest and largest number
from the list and display it. ...............................................................................................................................5
6. Write a program to create a tuple to students CPI and display it. ...............................................................5
7. Write a program to create a set of students’ enrolment number and sort them and perform addition
and deletion operation on it. ...........................................................................................................................6
8. Write a program to create a dictionary of student details- EnrollmentNo, Name, Branch and perform
addition and deletion of entry of key-value pair from it. .................................................................................8
9. Write a program to read and display content of the file. Also display number of lines, words in the given
file. ....................................................................................................................................................................9
Practical-2: Study about numpy, pandas, Scikit-learn and matplotlib libraries. ......................................... 10
Practical-3: Getting Started with Python Logic Programming using Kanren and SymPy packages. ............. 12
Kanren: .......................................................................................................................................................... 12
Introduction to sympy ................................................................................................................................... 14
Practical-4: Write the code in Kanren to demonstrate the followings: .......................................................... 30
a) The use of logical variables. ................................................................................................................... 30
b) The use of membero goal constructor. ................................................................................................. 31
Practical-5: Write the code in Kanren to create parent and grandparent relationships and use it to state
facts and query based on the facts. ..................................................................................................... 32

Practical-6: Write the code in Kanren to demonstrate the constraint system. .......................................... 33

Practical-7: Write the code in Kanren to match the mathematical expressions......................................... 34

Practical-8: Write the code in python to implement linear regression for one variable. ............................ 35
Practical-9: Write the code in python to implement linear regression using gradient descent for one
variable. ............................................................................................................................................. 41

Practical-10: Write the code in python to implement logistic regression for single class classification. ...... 47
Practical-11: Write the code in python to implement logistic regression for multi class classification........ 56

Computer Science Department L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-1
Implement following programs in python: Basics of Python
Program

1. Write a program to Convert Celsius to Fahrenheit and vice –a-versa

based on the choice of user.
Answer :
# Write a program to Convert Celsius to Fahrenheit and vice versa based on the choice of user.

mode = int(input("Enter mode:\n(1) Convert from Celsius to Fahrenheit\n(2) Convert

from Fahrenheit to Celsius\n> "))

if mode == 1:
celsius = float(input("Enter temperature in celsius: "))
fahrenheit = (celsius * 9 / 5) + 32
print(f'{celsius:.2f} Celsius is: {fahrenheit:0.2f} Fahrenheit')
elif mode == 2:
fahrenheit = float(input("Enter temperature in fahrenheit: "))
celsius = (fahrenheit - 32) * 5 / 9
print(f'{fahrenheit:.2f} Fahrenheit is: {celsius:0.2f} Celsius')
else:
print("Invalid mode selected")

Output:

Enter mode:
(1) Convert from Celsius to Fahrenheit
(2) Convert from Fahrenheit to Celsius
> 1
Enter temperature in celsius: 43
43.00 Celsius is: 109.40 Fahrenheit

Computer Science Department 1 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

2. Write a program in python to implement menu driven simple

calculator.
Answer:
operation = int(input("Select Mode:\n1. Addition\n2. Subtraction\n3.
Multiplication\n4. Division\n> "))

n1 = float(input("Enter the First Number: "))

n2 = float(input("Enter the Second Number: "))

if operation == 1:
print(n1, "+", n2, "=", (n1 + n2))
elif operation == 2:
print(n1, "-", n2, "=", (n1 - n2))
elif operation == 3:
print(n1, "*", n2, "=", (n1 * n2))
elif operation == 4:
print(n1, "/", n2, "=", (n1 / n2))
else:
print("Invalid Input")

Output:

Select Mode:
1. Addition
2. Subtraction
3. Multiplication
4. Division
> 3
Enter the First Number: 5
Enter the Second Number: 9
5.0 * 9.0 = 45.0

Computer Science Department 2 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

3. Write a program which will allow user to enter 10 numbers and

display largest odd and even number from them. Also display the count
of odd and even numbers.
Answer:
n = 10
print("Enter", n, "numbers in a single line seperated by spaces:")
lst = list(map(float, input().split(" ")))

even = [x for x in lst if x % 2 == 0]

odd = [x for x in lst if x % 2 == 1]

print("Even numbers in the list: ", len(even))

print("Min even: ", max(even))
print("Max even: ", min(even))
print("Odd numbers in the list: ", len(odd))
print("Min odd: ", max(odd))
print("Max odd: ", min(odd))

Output:

Enter 10 numbers in a single line seperated by spaces:

1 2 3 4 5 6 7 8 9 10
Even numbers in the list: 5
Min even: 10.0
Max even: 2.0
Odd numbers in the list: 5
Min odd: 9.0
Max odd: 1.0

Computer Science Department 3 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

4. Write a Python program to check if the number provided by the user is

a Prime number. Perform it with and without user defined function.
Answer:
# Write a Python program to check if the number provided by the user is a Prime
number. Perform it with and without user defined function.
import math

def is_prime(n):
if n % 2 == 0 and n > 2:
return False
return all(n % i for i in range(3, int(math.sqrt(n)) + 1, 2))

number = int(input("Enter a number: "))

print("Output via a user-defined function: ")
if is_prime(number):
print(f"{number} is prime")
else:
print(f"{number} is not prime")

print("Output without a user-defined function: ")

if number == 2 or (number % 2 == 1 and all(number % i for i in range(3,
int(math.sqrt(number)) + 1, 2))):
print(f"{number} is prime")
else:
print(f"{number} is not prime")

Output:

Enter a number: 173

Output via a user-defined function:
173 is prime
Output without a user-defined function:
173 is prime

Computer Science Department 4 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

5. Write a program to create a list of ten numbers entered by user and

find smallest and largest number from the list and display it.
Answer:
n = 10
print("Enter", n, "numbers in a single line seperated by spaces:")
lst = list(map(float, input().split(" ")))

print("Min: ", max(lst))

print("Max: ", min(lst))

Output:

Enter 10 numbers in a single line seperated by spaces:

1 2 3 4 5 6 7 8 9 10
Min: 10.0
Max: 1.0

6. Write a program to create a tuple to students CPI and display it.

Answer:
# Write a program to create a tuple to students CPI and display it.

def input_to_tuple(i):
name = input(f"Enter name for student {i + 1}: ")
cpi = input(f"Enter CPI for student {i + 1}: ")
return name, float(cpi)

n = int(input("Enter number of students: "))

students = tuple(input_to_tuple(i) for i in range(n))
print(students)

Output:

Enter number of students: 3

Enter name for student 1: Aryan Chudasama
Enter CPI for student 1: 9.48
Enter name for student 2: Alpha 1
Enter CPI for student 2: 9.11
Enter name for student 3: Beta 2
Enter CPI for student 3: 8.22
(('Aryan Chudasama', 9.48), ('Alpha 1', 9.11), ('Beta 2', 8.22))

Computer Science Department 5 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

7. Write a program to create a set of students’ enrolment number and

sort them and perform addition and deletion operation on it.
Answer:
students = set()

while True:
mode = int(input("""Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> """))
if mode == 1:
s = input("Enter roll number to add: ")
students.add(s)
elif mode == 2:
s = input("Enter roll number to remove: ")
students.remove(s)
elif mode == 3:
print(students)
else:
break

Output:

Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> 1

Computer Science Department 6 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
Enter roll number to add: 123
Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> 1
Enter roll number to add: 456
Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit

> 1
Enter roll number to add: 782
Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> 3
{'123', '456', '782'}
Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> 2
Enter roll number to remove: 123
Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> 2
Enter roll number to remove: 782
Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> 3
{'456'}
Enter mode:
(1) Add student roll number
(2) Delete student roll number
(3) Display students' roll numbers
(4) Exit
> 4

Computer Science Department 7 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

8. Write a program to create a dictionary of student details-

EnrollmentNo, Name, Branch and perform addition and deletion of entry
of key-value pair from it.
Answer:
from pprint import pprint

student = {"EnrollmentNo": "230283111005", "Name": "Aryan Chudasama",

"Branch": "EC"}
print("Original dictionary: ")
pprint(student)

del student["Name"]
print("After removal: ")
pprint(student)

student["FullName"] = "Chudasama Aryan Hiteshbhai"

print("After insertion: ")
pprint(student)

Output:

Original dictionary:
{'Branch': 'EC', 'EnrollmentNo': '230283111005', 'Name': 'Aryan Chudasama'}
After removal:
{'Branch': 'EC', 'EnrollmentNo': '230283111005'}
After insertion:
{'Branch': 'EC',
'EnrollmentNo': '230283111005',
'FullName': 'Chudasama Aryan Hiteshbhai'}

Computer Science Department 8 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

9. Write a program to read and display content of the file. Also display
number of lines, words in the given file.
Answer:
./test_file.txt

This is a test file.

Alpha beta gamma.
Beep boop.

./Prac1-9.py

with open('./test_file.txt') as f:
data = str(f.read())
num_lines = len(data.splitlines())
num_words = len(data.split())
print(f"""File contents:
{data}
END
Number of lines: {num_lines}
Number of words: {num_words}
""")

Output:

File contents:
This is a test file.
Alpha beta gamma.
Beep boop.
END
Number of lines: 3
Number of words: 10

Computer Science Department 9 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-2
Study about numpy, pandas, Scikit-learn and matplotlib libraries.

Pandas:

Pandas is a powerful Python library for data manipulation and analysis. It provides
easy-to-use data structures such as DataFrames, which are tabular structures that
allow you to store and manipulate data efficiently. Pandas offers a wide range of
functions and methods for data cleaning, preprocessing, merging, grouping, and
more. It is widely used for data wrangling and exploration tasks in data science
projects.

NumPy:

NumPy (Numerical Python) is a fundamental library for scientific computing in

Python. It provides support for large, multi-dimensional arrays and matrices, along
with a collection of mathematical functions to operate on these arrays efficiently.
NumPy's array operations are significantly faster than traditional Python lists, making
it a popular choice for numerical computations. It is the foundation for many other
libraries in the scientific Python ecosystem.

Matplotlib:

Matplotlib is a widely-used plotting library in Python. It provides a comprehensive set

of functions for creating static, animated, and interactive visualizations. Matplotlib
allows you to generate various types of plots, including line plots, scatter plots, bar
plots, histograms, and more. It provides extensive customization options to control
the appearance of plots, such as axes, labels, colors, and styles. Matplotlib is often
used in combination with other libraries like pandas and NumPy to visualize data.

Scikit-learn:

Scikit-learn is a popular machine learning library in Python. It provides a wide range

of algorithms and tools for tasks such as classification, regression, clustering,
dimensionality reduction, and model selection. Scikit-learn is designed with a user-
friendly and consistent API, making it easy to experiment with different machine

Computer Science Department 10 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

learning models and evaluate their performance. It also includes utilities for data
preprocessing, feature extraction, and model evaluation.

Seaborn:

Seaborn is a data visualization library built on top of Matplotlib. It provides a high-

level interface for creating attractive and informative statistical graphics. Seaborn
simplifies the creation of complex visualizations such as heatmaps, pair plots,
distribution plots, and regression plots. It offers a wide range of built-in themes and
color palettes, making it easy to create visually appealing plots. Seaborn is often used
in conjunction with pandas for visualizing patterns and relationships in data.
These libraries, together with their extensive functionality, have become essential
tools in the Python data science ecosystem, enabling tasks such as data
manipulation, numerical computations, plotting, and machine learning.

Computer Science Department 11 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-3
Getting Started with Python Logic Programming using Kanren and
SymPy packages.
Kanren:
kanren enables one to express sophisticated relations—in the form of goals—and
generate values that satisfy the relations. The following code is the "Hello, world!" of
logic programming; it asks for values of the logic variable x such that x == 5:

>>> from kanren import run, eq, membero, var, lall

>>> x = var()
>>> run(1, x, eq(x, 5))
(5,)

Multiple logic variables and goals can be used simultaneously. The following code
asks for one list containing the values of x and z such that x == z and z == 3:

>>> z = var()
>>> run(1, [x, z], eq(x, z),
eq(z, 3))
([3, 3],)

kanren uses unification to match forms within expression trees. The following code
asks for values of x such that (1, 2) == (1, x):

>>> run(1, x, eq((1, 2), (1, x)))

(2,)
The above examples use eq: a goal constructor that creates a goal for unification
between two objects. Other goal constructors, such as membero(item, coll),
express more sophisticated relations and are often constructed from simpler ones
like eq. More specifically, membero states that item is a member of the collection coll.
The following example uses membero to ask for all values of x, such that x is a member
of (1, 2, 3) and x is a member of (2, 3, 4).

>>> run(0, x, membero(x, (1, 2, 3)), # x is a member of (1, 2, 3)

membero(x, (2, 3, 4))) # x is a member of (2, 3, 4)
(2, 3)

The examples above made implicit use of the goal constructors lall and lany, which
represent goal conjunction and disjunction, respectively. Many useful relations can be

Computer Science Department 12 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
expressed with lall, lany, and eq alone, but in kanren it's also easy to leverage the
host language and explicitly create any relation expressible in Python.

Representing Knowledge
kanren stores data as facts that state relationships between terms. The following
code creates a parent relationship and uses it to state facts about who is a parent of
whom within the Simpsons family:

>>> from kanren import Relation, facts

>>> parent = Relation()
>>> facts(parent, ("Homer", "Bart"),
... ("Homer", "Lisa"),
... ("Abe", "Homer"))
>>> run(1, x, parent(x, "Bart"))
('Homer',)
>>> run(2, x, parent("Homer", x))
('Lisa', 'Bart')

We can use intermediate variables for more complex queries. For instance, who is
Bart's grandfather?

>>> grandparent_lv, parent_lv = var(), var()

>>> run(1, grandparent_lv, parent(grandparent_lv, parent_lv),
parent(parent_lv, 'Bart'))
('Abe',)

We can express the grandfather relationship as a distinct relation by creating a goal

constructor:

>>> def grandparent(x, z):

... y = var()
... return lall(parent(x, y), parent(y, z))

>>> run(1, x, grandparent(x, 'Bart'))

('Abe,')

Computer Science Department 13 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Introduction to sympy
What is Symbolic Computation?

import sympy
from sympy import *
import math
from IPython.display import display
init_printing(use_latex="svg", use_unicode=True)

Symbolic computation deals with the computation of mathematical objects

symbolically. This means that the mathematical objects are represented exactly, not
approximately, and mathematical expressions with unevaluated variables are left in
symbolic form.
Let’s take an example. Say we wanted to use the built-in Python functions to
compute square roots. We might do something like this

math.sqrt(9)

9 is a perfect square, so we got the exact answer, 3. But suppose we computed the
square root of a number that isn’t a perfect square

math.sqrt(8)

Here we got an approximate result. 2.82842712475 is not the exact square root of 8
(indeed, the actual square root of 8 cannot be represented by a finite decimal, since
it is an irrational number). If all we cared about was the decimal form of the square
root of 8, we would be done.
But suppose we want to go further. Recall that sqrt(8) = sqrt(4 * 2) = 2 * sqrt(2). We
would have a hard time deducing this from the above result. This is where symbolic
computation comes in. With a symbolic computation system like SymPy, square roots
of numbers that are not perfect squares are left unevaluated by default

sympy.sqrt(3)

Computer Science Department 14 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
Furthermore—and this is where we start to see the real power of symbolic
computation—symbolic results can be symbolically simplified.

sympy.sqrt(8)

A More Interesting Example

The above example starts to show how we can manipulate irrational numbers exactly
using SymPy. But it is much more powerful than that. Symbolic computation systems
(which by the way, are also often called computer algebra systems, or just CASs like
Matlab) such as SymPy are capable of computing symbolic expressions with variables.
As we will see later, in SymPy, variables are defined using symbols. Unlike many
symbolic manipulation systems, variables in SymPy must be defined before they are
used.
Let us define a symbolic expression, representing the mathematical expression x + 2y.

x, y = symbols('x y')
expr = x + 2*y
expr

Note that we wrote x + 2*y just as we would if x and y were ordinary Python
variables. But in this case, instead of evaluating to something, the expression remains
as just x + 2*y. Now let us play around with it:

expr + 1

expr - x

Notice something in the above example. When we typed expr - x, we did not get x
+ 2*y - x, but rather just 2*y. The x and the -x automatically canceled one
another. This is similar to how sqrt(8) automatically turned into 2*sqrt(2) above. This
isn’t always the case in SymPy, however:

Computer Science Department 15 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

x*expr

Here, we might have expected x(x + 2y) to transform into x^2 + 2xy, but
instead we see that the expression was left alone. This is a common theme in SymPy.
Aside from obvious simplifications like x - x = 0 and simplification of radicals,
most simplifications are not performed automatically. This is because we might
prefer the factored form, or we might prefer the expanded form. Both forms are
useful in different circumstances. In SymPy, there are functions to go from one form
to the other

expanded_expr = expand(x*expr)
expanded_expr

factor(expanded_expr)

The Power of Symbolic Computation

The real power of a symbolic computation system such as SymPy is the ability to do
all sorts of computations symbolically. SymPy can simplify expressions, compute
derivatives, integrals, and limits, solve equations, work with matrices, and much,
much more, and do it all symbolically. It includes modules for plotting, printing (like
2D pretty printed output of math formulas, or LATEX), code generation, physics,
statistics, combinatorics, number theory, geometry, logic, and more.

x, t, z, nu = symbols('x t z nu')

Take the derivative of an expression:

exp1 = sin(x)*exp(x)
exp1

diff(exp1, x)

Computer Science Department 16 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Compute an indefinite integral:

exp2 = exp(x)*sin(x) + exp(x)*cos(x)
exp2

integrate(exp2, x)

Find a definite integral:

exp3 = sin(x**2)
exp3

Integral(exp3, (x, -oo, oo))

integrate(exp3, (x, -oo, oo))

Find a limit:

Limit(sin(x)/x, x, 0)

limit(sin(x)/x, x, 0)

Solve an equation

Equality(x ** 2 - 2, x)

Computer Science Department 17 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

solve(x**2 - 2, x)

Solve a differential equation:

y = Function('y')
diff_eqn = Eq(y(t).diff(t, t) - y(t), exp(t))
diff_eqn

dsolve(diff_eqn, y(t))

Find eigen values of a matrix

mat = Matrix([[1, 2], [2, 2]])
mat

mat.eigenvals()

Rewrite the Bessel function in terms of the spherical Bessel function.

besselj(nu, z).rewrite(jn)

Computer Science Department 18 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

A concrete example: Find the exponential Fourier series coefficients c(k) for
a triangular pulse.
import numpy as np
import sympy
from sympy import *
import math
from IPython.display import display
import matplotlib.pyplot as plt
from sympy.utilities.lambdify import lambdify
import builtins

init_printing(use_latex="svg", use_unicode=True)
t = sympy.symbols('t')
n, k = sympy.symbols('n k', integer=True)
x = sympy.Function('x')

# Specify period and amplitude

period_of_x = 2 * sympy.pi
amplitude_of_x = 1

# Determine fundamental_frequency = w0
fundamental_frequency = 2 * pi / period_of_x
half_period = period_of_x / 2
# Define a single period of the function
x_single_period = sympy.Piecewise(
(amplitude_of_x / half_period * (t + half_period), (-half_period <= t) & (t <= 0)),
(-amplitude_of_x / half_period * (t - half_period), (0 < t) & (t <= half_period)),
)
x_single_period

Repeat x_single_period with the period specified

x = x_single_period.subs(
t, (t - floor(t / period_of_x) * period_of_x -
half_period))
x = simplify(x.subs(t, (t + half_period)))

Computer Science Department 19 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
x_np = lambdify(t, x, "numpy")
x

Plot the function for some actual concrete values of t

actual_t = np.arange((-3*half_period).evalf(),
(3*half_period).evalf(), 0.01)
plt.plot(actual_t, x_np(actual_t))
plt.title("Plot of triangle wave x(t)")
plt.ylabel("Amplitude")
plt.xlabel("Time")
plt.grid()
plt.xlim(float(actual_t[0]), float(actual_t[-1]))
plt.show()

Computer Science Department 20 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Compute the Fourier series coefficients by integrating:

t_0 = -half_period
# Find C0 by integrating x(t) over its period
computed_c_0 = 1 / period_of_x * integrate(
x_single_period, # Function
(
t, # Variable of integration
t_0, # Lower limit
t_0 + period_of_x # Upper limit
)
)
computed_c_0

# Find Cn by integrating x(t)*e^-jnw0t over its period

computed_c_n_not_zero = simplify(1 / period_of_x * integrate(
x_single_period * exp(sympy.I * n * fundamental_frequency * t), #
Function
(
t, # Variable of integration
t_0, # Lower limit
t_0 + period_of_x # Upper limit
)
))
computed_c_n_not_zero

Here we see that computed_c_n_not_zero has 3 cases, one for even coefficients, one
for odd coefficients, and one for zero, to find the odd and even coefficients
separately...

# ... First take n_even as even like so

n_even = symbols("n_e", integer=True, even=True, nonzero=True)
# % And evaluate computed_c_n_not_zero at the newly assumed even n, and
store it somewhere
computed_c_n_not_zero_even = computed_c_n_not_zero.subs(
n, n_even).refine().simplify()
computed_c_n_not_zero_even

Computer Science Department 21 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

# Then take n_odd as odd like so

n_odd = symbols("n_o", integer=True, odd=True, nonzero=True)
# % And evaluate computed_c_n_not_zero at the newly assumed even n, and
store it somewhere
computed_c_n_not_zero_odd = computed_c_n_not_zero.subs(
n, n_odd).refine().simplify()
computed_c_n_not_zero_odd

# Generate a complete expression for Cn using piecewise

computed_c_n = Piecewise(
(computed_c_0, Eq(n, 0)),
(computed_c_n_not_zero_even.subs(n_even, n), Eq(n % 2, 0)),
(computed_c_n_not_zero_odd.subs(n_odd, n), Eq(n % 2, 1)),
).refine().simplify()
computed_c_n

computed_c_n_np = lambdify(n, computed_c_n, "numpy")

Thus, we have evaluated the generic Fourier series coefficients in sympy.

Find the Fourier series coefficients by evaluating the integrals by hand, and then
express them here:

ideal_c_0 = amplitude_of_x / 2
ideal_c_n = Piecewise(
(ideal_c_0, Eq(n, 0)),
(0, Eq(n % 2, 0)),
(2 * amplitude_of_x / pow(n * pi, 2), Eq(n % 2, 1))
)
ideal_c_n

Computer Science Department 22 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

ideal_c_n_np = lambdify(n, ideal_c_n, "numpy")

Plot the frequency spectrum for Cn, also compare the values for the 2 expressions for
the Fourier series coefficients:

n_max = 10
n_array = np.arange(-n_max, n_max)
# Compare the Fourier series coefficients, one evaluated by hand, and the
other in Matlab
ideal_c_n_values = ideal_c_n_np(n_array)
computed_c_n_values = computed_c_n_np(n_array)
# Check that they are (at least approximately) the same.
assert (np.all(np.abs(ideal_c_n_values - computed_c_n_values) <
0.00001))
ideal_c_n_values

Output:

array([0. , 0.00250176, 0. , 0.00413556, 0. ,

0.00810569, 0. , 0.02251582, 0. , 0.20264237,
0.5 , 0.20264237, 0. , 0.02251582, 0. ,
0.00810569, 0. , 0.00413556, 0. , 0.00250176])

plt.stem(n_array, ideal_c_n_values)
plt.grid()
plt.title("Plot of frequency spectrum of X(w)")
plt.ylabel("Mangnitude")
plt.xlabel("n")

Computer Science Department 23 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Write x(t) as a generic summation of Fourier series coefficients:

ideal_c_n

def get_reconstructed_x(n_max):
individual_term_of_fourier_series = computed_c_n * \
exp(sympy.I * n * fundamental_frequency * t)
x_as_fourier_series = simplify(
summation(individual_term_of_fourier_series.subs(n, k), (k, -n, n)))
reconstructed_x = simplify(x_as_fourier_series.subs(n, n_max))
reconstructed_x_np = lambdify(t, reconstructed_x, "numpy")
return reconstructed_x, reconstructed_x_np

Reconstruct x(t) from its Fourier series expansion, plot it and compare it with its original expression:

Computer Science Department 24 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

reconstructed_x, reconstructed_x_np = get_reconstructed_x(n_max)

reconstructed_x

actual_t = np.arange((-3*half_period).evalf(),
(3*half_period).evalf(), 0.01, dtype=float)
original_x_vals = x_np(actual_t)
reconstructed_x_vals = np.real(reconstructed_x_np(actual_t))
diff = np.abs(original_x_vals - reconstructed_x_vals)
max_diff = np.max(diff)
rms_diff = np.sqrt(np.mean(diff ** 2))
display(max_diff)
display(rms_diff)

Thus we see that difference is very small.

Plot x(t), and x(t) from its Fourier series:

plt.plot(actual_t, original_x_vals, actual_t, reconstructed_x_vals)

plt.grid()
plt.title("Plot of triangle wave x(t) and its reconstructed variant")
plt.ylabel("Amplitude")
plt.xlabel("Time")
plt.legend(["Original x(t)", "Reconstructed from Fourier series"],
loc='best')
plt.xlim([actual_t[0], actual_t[-1]])

Computer Science Department 25 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Here, we see that their plots line up almost perfectly, with the sharp edge at times t =
0, -pi, +pi, -2pi, +2pi, etc., having been smoothed out in the reconstructed version.
If we plot the reconstructed x(t) alone, like so:

plt.plot(actual_t, reconstructed_x_vals)
plt.grid()
plt.title("Plot of triangle wave x(t)'s reconstructed variant")
plt.ylabel("Amplitude")
plt.xlabel("Time")
plt.xlim([actual_t[0], actual_t[-1]])

Computer Science Department 26 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

We see that the function is a bit wobbly, and the edges have been smoothed out
greately, as compared to the original triangle wave. This is due to the errors in
reconstruction, which can be reduced by increasing n_max.
For eg, for n_max = 30:

n_max = 30
accurate_rec_x, accurate_rec_x_np = get_reconstructed_x(n_max)
accurate_rec_x

accurate_rec_x_vals = np.real(accurate_rec_x_np(actual_t))
diff = np.abs(original_x_vals - accurate_rec_x_vals)
max_diff = np.max(diff)
rms_diff = np.sqrt(np.mean(diff ** 2))

Computer Science Department 27 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
display(max_diff)
display(rms_diff)

Here the difference, is even smaller.

Plot it:

plt.plot(actual_t, accurate_rec_x_vals)
plt.grid()
plt.title(f"Plot of x(t) reconstructed with n_max = {n_max}")
plt.ylabel("Amplitude")
plt.xlabel("Time")
plt.xlim([actual_t[0], actual_t[-1]])

Computer Science Department 28 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
Here, we see that the function is fairly straight, but not as much as the original true
x(t), but still better than the previous approximation. Also, the sharp edges are only
slightly smoothed out, they are still fairly sharp.

Computer Science Department 29 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-4
Write the code in Kanren to demonstrate the followings:
a) The use of logical variables.
Answer:
from pprint import pprint

from kanren import run, eq, var

x = var()

# Find x, such that x equals 5

print("x, such that x equals 5: ")

pprint(run(1, x, eq(x, 5)))
z = var()

# Find x & z, such that x equals z, and z equals 3

print("x & z, such that x equals z, and z equals 3: ")

pprint(run(1, [x, z], eq(x, z), eq(z, 3)))

# Find x, such that (1, x) equals (1, 2)

print("x, such that (1, x) equals (1, 2): ")

pprint(run(1, x, eq((1, 2), (1, x))))

Output:

x, such that x equals 5:

(5,)
x & z, such that x equals z, and z equals 3:
([3, 3],)
x, such that (1, x) equals (1, 2):
(2,)

Computer Science Department 30 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

b) The use of membero goal constructor.

Answer:

from pprint import pprint

from kanren import run, var, membero

x = var()
# Find x, such that x is a member of (1, 2, 3), and x is a member of (2, 3, 4)
print("x, such that x is a member of (1, 2, 3), and x is a member of (2, 3, 4): ")
pprint(run(0, x, membero(x, (1, 2, 3)), membero(x, (2, 3, 4))))

Output:

x, such that x is a member of (1, 2, 3), and x is a member of (2, 3, 4):

(2, 3)

Computer Science Department 31 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-5
Write the code in Kanren to create parent and grandparent
relationships and use it to state facts and query based on the facts.
Answer:
from pprint import pprint

from kanren import Relation, facts, var, run, lall

x = var()
parent = Relation()
facts(parent, ("Homer", "Bart"), ("Homer", "Lisa"), ("Abe", "Homer"))
print("Find x such that x is a parent of 'Bart': ")
pprint(run(1, x, parent(x, "Bart")))

print("Find x such that 'Homer' is a parent of x: ")

pprint(run(2, x, parent("Homer", x)))

grandparent_lv, parent_lv = var(), var()

print("Find grandparent_lv such that grandparent_lv is a parent of parent_lv, and
parent_lv is a parent of 'Bart': ")
pprint(run(1, grandparent_lv, parent(grandparent_lv, parent_lv), parent(parent_lv,
'Bart')))

def grandparent(x, z):

y = var()
return lall(parent(x, y), parent(y, z))

print("Find x such that x is a 'grandparent' of 'Bart': ")

pprint(run(1, x, grandparent(x, 'Bart')))

Output:

Find x such that x is a parent of 'Bart':

('Homer',)
Find x such that 'Homer' is a parent of x:
('Lisa', 'Bart')
Find grandparent_lv such that grandparent_lv is a parent of parent_lv, and parent_lv
is a parent of 'Bart':
('Abe',)
Find x such that x is a 'grandparent' of 'Bart':
('Abe',)

Computer Science Department 32 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-6
Write the code in Kanren to demonstrate the constraint system.

Answer:

from numbers import Integral

from pprint import pprint

from kanren import var, run, membero

from kanren.constraints import neq, isinstanceo

x = var()
print("Find x such that x != 1, and x != 3, and x is a member of {1, 2, 3}: ")
pprint(run(0, x, neq(x, 1), neq(x, 3), membero(x, (1, 2, 3))))

print("Find x such that x is a integer, and x is a member of {1.1, 2, 3.2, 4}: ")
pprint(run(0, x, isinstanceo(x, Integral), membero(x, (1.1, 2, 3.2, 4))))

Output:

Find x such that x != 1, and x != 3, and x is a member of {1, 2, 3}:

(2,)
Find x such that x is a integer, and x is a member of {1.1, 2, 3.2, 4}:
(2, 4)

Computer Science Department 33 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-7
Write the code in Kanren to match the mathematical expressions.

Answer:

from kanren import fact, run, var

from kanren.assoccomm import associative, commutative
from kanren.assoccomm import eq_assoccomm as eq

# Define some dummy Operations

add = "add"
mul = "mul"

# Declare that these ops are commutative & associative using the facts system
fact(commutative, mul)
fact(commutative, add)
fact(associative, mul)
fact(associative, add)

# Define some logic variables

x, y = var(), var()

# Two expressions to match

pattern = (mul, (add, 1, x), y) # (1 + x) * y
expr = (mul, 2, (add, 3, 1)) # 2 * (3 + 1)

res = run(0, (x, y), eq(pattern, expr))

print(res)
# prints ((3, 2),) meaning
# x matches to 3
# y matches to 2

Output:

((3, 2),)

Computer Science Department 34 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-8
Write the code in python to implement linear regression for one
variable.
Answer:
Simple Linear Regression
Import libraries
# Import libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from pandas.core.common import random_state
from sklearn.linear_model import LinearRegression

Import data
# Get dataset
df_sal = pd.read_csv('./Salary_Data.csv').replace([np.inf, -np.inf], np.nan)
df_sal.head()

Output:

YearsExperience Salary
0 1.1 39343.0
1 1.3 46205.0
2 1.5 37731.0
3 2.0 43525.0
4 2.2 39891.0

Analyze data
Describe
# Describe data
df_sal.describe()

Computer Science Department 35 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
Output:

YearsExperience Salary
count 30.000000 30.000000
mean 5.313333 76003.000000
std 2.837888 27414.429785
min 1.100000 37731.000000
25% 3.200000 56720.750000
50% 4.700000 65237.000000
75% 7.700000 100544.750000
max 10.500000 122391.000000

Distribution
# Data distribution
sns.displot(df_sal['Salary'])
plt.title('Salary Distribution Plot')
plt.show()

Computer Science Department 36 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Relationship between Salary and Experience

# Relationship between Salary and Experience
plt.scatter(df_sal['YearsExperience'], df_sal['Salary'], color = 'lightcoral')
plt.title('Salary vs Experience')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.box(False)
plt.show()

Split data
Split into Independent/Dependent variables
# Splitting variables
X = df_sal.iloc[:, :1] # independent
y = df_sal.iloc[:, 1:] # dependent

Split into Train/Test sets

# Splitting dataset into test/train
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state =
0)

Computer Science Department 37 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Train model

# Regressor model
regressor = LinearRegression()
regressor.fit(X_train, y_train)

Output:
LinearRegression()

Predict results
# Prediction result
y_pred_test = regressor.predict(X_test) # predicted value of y_test
y_pred_train = regressor.predict(X_train) # predicted value of y_train

Visualize predictions

Prediction on training set

# Prediction on training set

plt.scatter(X_train, y_train, color = 'lightcoral')

plt.plot(X_train, y_pred_train, color = 'firebrick')
plt.title('Salary vs Experience (Training Set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.legend(['X_train/Pred(y_test)', 'X_train/y_train'], title = 'Sal/Exp', loc='best',
facecolor='white')
plt.box(False)
plt.show()

Computer Science Department 38 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Prediction on test set

# Prediction on test set

plt.scatter(X_test, y_test, color = 'lightcoral')

plt.plot(X_train, y_pred_train, color = 'firebrick')
plt.title('Salary vs Experience (Test Set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.legend(['X_train/Pred(y_test)', 'X_train/y_train'], title = 'Sal/Exp', loc='best',
facecolor='white')
plt.box(False)
plt.show()

Computer Science Department 39 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Coefficient and Intercept

# Regressor coefficients and intercept

print(f'Coefficient: {regressor.coef_}')
print(f'Intercept: {regressor.intercept_}')

Output:

Coefficient: [[9312.57512673]]
Intercept: [26780.09915063]

Computer Science Department 40 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-9
Write the code in python to implement linear regression using
gradient descent for one variable.
Answer:

Import libraries

# Import libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.model_selection import train_test_split
from pandas.core.common import random_state
from sklearn.linear_model import LinearRegression

Import data

# Get dataset
df_sal = pd.read_csv('./Salary_Data.csv').replace([np.inf, -np.inf], np.nan)
df_sal.head()

Output:

YearsExperience Salary
0 1.1 39343.0
1 1.3 46205.0
2 1.5 37731.0
3 2.0 43525.0
4 2.2 39891.0

Computer Science Department 41 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Split data
x = df_sal["YearsExperience"].to_numpy()
y = df_sal["Salary"].to_numpy()

# Splitting dataset into test/train

x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.2, random_state = 0)

x_train_len = len(x_train)

Perform gradient decent

w0 = 0
w1 = 0
eta = 0.01
max_epochs = 10000
epoch_num = 0
past_mse = None
mse_epsilon = 0.01
max_no_improvement_cycles = 25
num_no_improvement_cycles = 0

w0_history = []
w1_history = []
mse_history = []

while epoch_num < max_epochs:

epoch_num = epoch_num + 1
y_pred = w1 * x_train + w0
curr_mse = np.sum(np.power((y_train - y_pred), 2)) / n

# Store history to plot later

w0_history.append(w0)
w1_history.append(w1)
mse_history.append(curr_mse)

if epoch_num % 100 == 0:
print(f"Epoch {epoch_num}: MSE: {curr_mse:.5f}");

Computer Science Department 42 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

if past_mse is not None:

# Check if there is no improvement, or the MSE is worseneing
if past_mse - curr_mse < mse_epsilon:
num_no_improvement_cycles = num_no_improvement_cycles + 1;
else:
num_no_improvement_cycles = 0;

if num_no_improvement_cycles >= max_no_improvement_cycles:

print(f"No improvement in MSE for {num_no_improvement_cycles} epochs,
exitting...");
break

# Perform gradient decent

del_mse_by_del_w0 = 2 / x_train_len * np.sum((y_pred - y_train))
del_mse_by_del_w1 = 2 / x_train_len * np.sum((y_pred - y_train)*x_train)

w0 = w0 - eta * del_mse_by_del_w0
w1 = w1 - eta * del_mse_by_del_w1

# Logging past_mse for comparision with the next iteration

past_mse = curr_mse;

print(f"Done at Epoch {epoch_num}: With MSE: {curr_mse:.5f}\nBest fit: y =

{w1:.5f} * x + {w0:.5f}\n");

Output:

Epoch 100: MSE: 74228998.48715

Epoch 200: MSE: 49097510.66772
Epoch 300: MSE: 37905593.84694
Epoch 400: MSE: 32921447.94108
Epoch 500: MSE: 30701836.16155
Epoch 600: MSE: 29713366.61333
Epoch 700: MSE: 29273167.08981
Epoch 800: MSE: 29077131.08560
Epoch 900: MSE: 28989829.50328
Epoch 1000: MSE: 28950951.10281
Epoch 1100: MSE: 28933637.21247
Epoch 1200: MSE: 28925926.74097
Epoch 1300: MSE: 28922493.00254
Epoch 1400: MSE: 28920963.84059
Epoch 1500: MSE: 28920282.85212
Epoch 1600: MSE: 28919979.58451
Epoch 1700: MSE: 28919844.52900

Computer Science Department 43 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Epoch 1800: MSE: 28919784.38413

Epoch 1900: MSE: 28919757.59955
Epoch 2000: MSE: 28919745.67146
Epoch 2100: MSE: 28919740.35946
Epoch 2200: MSE: 28919737.99385
No improvement in MSE for 25 epochs, exitting...
Done at Epoch 2278: With MSE: 28919737.10512
Best fit: y = 9312.96507 * x + 26777.62791

Predict results

# Prediction result
y_pred_test = w1 * x_test + w0 # predicted value of y_test
y_pred_train = w1 * x_train + w0 # predicted value of y_train

Visualize predictions
Prediction on training set
# Prediction on training set
plt.scatter(x_train, y_train, color = 'lightcoral')
plt.plot(x_train, y_pred_train, color = 'firebrick')
plt.title('Salary vs Experience (Training Set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.legend(['X_train/Pred(y_test)', 'X_train/y_train'], title = 'Sal/Exp', loc='best',
facecolor='white')
plt.box(False)
plt.show()

Computer Science Department 44 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Prediction on test set

# Prediction on test set
plt.scatter(x_test, y_test, color = 'lightcoral')
plt.plot(x_train, y_pred_train, color = 'firebrick')
plt.title('Salary vs Experience (Test Set)')
plt.xlabel('Years of Experience')
plt.ylabel('Salary')
plt.legend(['X_train/Pred(y_test)', 'X_train/y_train'], title = 'Sal/Exp', loc='best',
facecolor='white')
plt.box(False)
plt.show()

Computer Science Department 45 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Coefficient and Intercept

# Regressor coefficients and intercept
print(f'Coefficient: {w1}')
print(f'Intercept: {w0}')

Output:

Coefficient: 9312.965074356625
Intercept: 26777.62790981993

Computer Science Department 46 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-10
Write the code in python to implement logistic regression for
single class classification.
Answer:

Run logistic regression Titanic dataset

i. e. dataset about the sinking of the Titanic and which people survived

# fmt: off
from typing import *

import numpy as np
import pandas as pd
import seaborn as sns

import matplotlib.pyplot as plt

import matplotlib
import mlxtend.plotting as mlx_plot

import sklearn.linear_model as sk_linear_model

import sklearn.metrics as sk_metrics
import sklearn.preprocessing as sk_pre
import sklearn.compose as sk_compose
from sklearn.model_selection import *

from typing import *

import itertools
# fmt: on

sns.set()
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
ROOT_INPUT_DIR = "./"

Computer Science Department 47 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Commonly used utility functions

def evaluate_categorical_predictions(y_true, y_pred, target_names=None):
"""
Returns the accuracy, balanced accuracy,
classification report per class and overall metrics
as a DataFrame, and confision matrix for the predictions
"""
crep = pd.DataFrame(sk_metrics.classification_report(y_true, y_pred,
target_names=target_names, digits=4, output_dict=True))
crep = crep.transpose()
extra_classes = ["accuracy", "macro avg", "weighted avg"]
scores_by_class = crep.drop(extra_classes)
overall_values = crep.loc[extra_classes]
scores_by_class["class-names"] = scores_by_class.index
return sk_metrics.accuracy_score(y_true, y_pred),
sk_metrics.balanced_accuracy_score(y_true, y_pred), scores_by_class, overall_values
def plot_confusion_matrix(cmat: np.ndarray, class_names: Optional[List[str]] =
None, **kwargs):
kwa = dict(
norm_colormap=matplotlib.colors.LogNorm(),
show_absolute=True,
show_normed=True,
colorbar=False,
)
kwa.update(kwargs)
if "figure" not in kwa:
kwa["figure"] = plt.figure(figsize=(8,8))
mlx_plot.plot_confusion_matrix(
cmat,
class_names=class_names,
**kwa
)

def make_confusion_matrix(
y_true,
y_pred,
classes: list = None,
figsize: Tuple[int, int] = (10, 10),
text_size: int = 15,
norm: bool = False,
savefig: Union[str, bool] = False,
remove_diagonal: bool = False):
"""Makes a labelled confusion matrix comparing predictions and ground truth
labels.
If classes is passed, confusion matrix will be labelled, if not, integer class values
will be used.
Computer Science Department 48 L. D. College of Engineering, Ahmedabad-15
220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Args:
y_true: Array of truth labels (must be same shape as y_pred).
y_pred: Array of predicted labels (must be same shape as y_true).
classes: Array of class labels (e.g. string form). If `None`, integer labels are used.
figsize: Size of output figure (default=(10, 10)).
text_size: Size of output figure text (default=15).
norm: normalize values or not (default=False).
savefig: save confusion matrix to file (default=False).

Returns:
A labelled confusion matrix plot comparing y_true and y_pred.

Example usage:
make_confusion_matrix(y_true=test_labels, # ground truth test labels
y_pred=y_preds, # predicted labels
classes=class_names, # array of class label names
figsize=(15, 15),
text_size=10)
"""
# Create the confustion matrix
cm: np.ndarray = sk_metrics.confusion_matrix(y_true, y_pred)
if remove_diagonal:
np.fill_diagonal(cm, 0)
cm_norm = cm.astype("float") / cm.sum(axis=1)[:, np.newaxis] # normalize it
n_classes = cm.shape[0] # find the number of classes we're dealing with
# Plot the figure and make it pretty
fig, ax = plt.subplots(figsize=figsize)
cax = ax.matshow(cm, cmap=plt.cm.Blues) # colors will represent how 'correct' a
class is, darker == better
plt.grid(False)
fig.colorbar(cax)
# Are there a list of classes?
if classes is not None:
labels = classes
else:
labels = np.arange(cm.shape[0])

# Label the axes

ax.set(title="Confusion Matrix",
xlabel="Predicted label",
ylabel="True label",
xticks=np.arange(n_classes), # create enough axis slots for each class
yticks=np.arange(n_classes),
xticklabels=labels, # axes will labeled with class names (if they exist) or ints
yticklabels=labels)
ax.tick_params(axis="x", rotation=50)
# Make x-axis labels appear on bottom
ax.xaxis.set_label_position("bottom")
Computer Science Department 49 L. D. College of Engineering, Ahmedabad-15
220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
ax.xaxis.tick_bottom()
# Set the threshold for different colors
threshold = (cm.max() + cm.min()) / 2.
# Plot the text on each cell
for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
if remove_diagonal and i == j:
continue
if norm:
plt.text(j, i, f"{cm[i, j]} ({cm_norm[i, j]*100:.1f}%)",
horizontalalignment="center",
color="white" if cm[i, j] > threshold else "black",
size=text_size)
else:
plt.text(j, i, f"{cm[i, j]}",
horizontalalignment="center",
color="white" if cm[i, j] > threshold else "black",
size=text_size)
# Save the figure to the current working directory
if savefig != False:
if savefig == True:
savefig = "confusion_matrix.png"
fig.savefig(savefig)

Data loading & preprocessing

raw_data = pd.read_csv(ROOT_INPUT_DIR + "train.csv")

View shape
raw_data.shape

Output:

(891, 12)

Computer Science Department 50 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

View NaN values

raw_data.isna().sum()

Output:

PassengerId 0
Survived 0
Pclass 0
Name 0
Sex 0
Age 177
SibSp 0
Parch 0
Ticket 0
Fare 0
Cabin 687
Embarked 2
dtype: int64

Remove NaNs

numeric_nan_columns = raw_data.columns[pd.DataFrame([raw_data.isna().any(), raw_data.dtypes

!= "object"]).all()]
data_no_nan = raw_data.drop(columns=[
#*numeric_nan_columns,
"Name", "Ticket", "Cabin", "Embarked"
])
categorical_nan_columns = data_no_nan.columns[pd.DataFrame([data_no_nan.isna().any(),
data_no_nan.dtypes == "object"]).all()]
data_no_nan[categorical_nan_columns] = data_no_nan[categorical_nan_columns].fillna("NA")
for i in numeric_nan_columns:
data_no_nan.loc[data_no_nan[i].isna(), i] = data_no_nan[i].median(axis=0)

## Input/Target split
ids = data_no_nan["PassengerId"]
targets = data_no_nan["Survived"]
inputs = data_no_nan.drop(columns=[ids.name, targets.name])

## Categorical columns
categorical_columns = [
*data_no_nan.columns[data_no_nan.dtypes == "object"],
"Pclass"
]

numeric_columns = inputs.columns[~inputs.columns.isin(set(categorical_columns))]

(categorical_columns, numeric_columns)

Computer Science Department 51 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
Output:

(['Sex', 'Pclass'], Index(['Age', 'SibSp', 'Parch', 'Fare'], dtype='object'))

Categorical columns
w=5
h=5
cols = 3
rows = int(np.ceil(len(categorical_columns) / cols))
plt.figure(figsize=(cols * w, rows * h))
for i, col in enumerate(categorical_columns[:]):
plt.subplot(rows, cols, i+1)
sns.countplot(x=inputs[col], hue=targets.astype(str))
#plt.hist(inputs[col])
plt.xticks(rotation=75)
plt.title(col)

w=5
h=5
cols = 3
rows = int(np.ceil(len(numeric_columns) / cols))
plt.figure(figsize=(cols * w, rows * h))

Computer Science Department 52 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

for i, col in enumerate(numeric_columns):

plt.subplot(rows, cols, i+1)
sns.categorical.violinplot(y=inputs[col], x=targets)
#plt.hist(inputs[col])
plt.xticks(rotation=75)
plt.title(col)

Train validation split

X_train: pd.DataFrame
X_val: pd.DataFrame
y_train: pd.Series
y_val: pd.Series
X_train, X_val, y_train, y_val = train_test_split(inputs, targets, test_size=0.1)
X_train.reset_index(drop=True, inplace=True)
X_val.reset_index(drop=True, inplace=True)
y_train.reset_index(drop=True, inplace=True)
y_val.reset_index(drop=True, inplace=True)
X_train.shape, X_val.shape

Output:

((801, 6), (90, 6))

Computer Science Department 53 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Transform data

col_transformer = sk_compose.ColumnTransformer([
("MinMaxScaler", sk_pre.MinMaxScaler(), numeric_columns),
("one_hot_encoder", sk_pre.OneHotEncoder(drop="first",
handle_unknown="ignore"), categorical_columns),
], remainder="passthrough", verbose_feature_names_out=False)
X_train_scaled = pd.DataFrame(col_transformer.fit_transform(X_train),
columns=col_transformer.get_feature_names_out())
X_val_scaled = pd.DataFrame(col_transformer.transform(X_val),
columns=col_transformer.get_feature_names_out())
col_transformer.get_feature_names_out()

Output:

array(['Age', 'SibSp', 'Parch', 'Fare', 'Sex_male', 'Pclass_2',

'Pclass_3'], dtype=object)

Setup model

model = sk_linear_model.LogisticRegression()
model.fit(X_train_scaled, y_train)
m_preds = model.predict(X_val_scaled) > 0.5

View accuracy & balanced accuracy

m_acc, m_bacc, m_class_metrics, m_metrics = evaluate_categorical_predictions(y_val,

m_preds)
m_acc, m_bacc

Output:

(0.7, 0.6809954751131222)

View overall metrics

m_metrics

Computer Science Department 54 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Output:

precision recall f1-score support

accuracy 0.7 0.700000 0.700000 0.7
macro avg 0.7 0.680995 0.682726 90.0
weighted avg 0.7 0.700000 0.692597 90.0

View metrics by class

m_class_metrics

Output:

precision recall f1-score support class-names

0 0.7 0.823529 0.756757 51.0 0
1 0.7 0.538462 0.608696 39.0 1

View confusion matrix

make_confusion_matrix(y_val, m_preds)

Computer Science Department 55 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Practical-11
Write the code in python to implement logistic regression for multi
class classification.
Answer:
Multi class logistic regression in numpy
Softmax regression, also called multinomial logistic regression extends logistic
regression to multiple classes.
Given:

• dataset {(𝐱 (1) , 𝑦 (1) ), . . . , (𝐱 (𝑚) , 𝑦 (𝑚) )}

(𝑖) (𝑖)
• with 𝐱 (𝑖) being a 𝑑 −dimensional vector 𝐱 (𝑖) = (𝑥1 , . . . , 𝑥𝑑 )
• 𝑦 (𝑖) being the target variable for 𝐱 (𝑖) , for example with 𝐾 = 3 classes we might
have 𝑦 (𝑖) ∈ {0,1,2}
A softmax regression model has the following features:
• a separate real-valued weight vector 𝐰 = (𝑤 (1) , . . . , 𝑤 (𝑑) ) for each class. The
weight vectors are typically stored as rows in a weight matrix.
• a separate real-valued bias 𝑏 for each class
• the softmax function as an activation function
• the cross-entropy loss function
The training procedure of a softmax regression model has different steps. In the
beginning (step 0) the model parameters are initialized. The other steps (see below)
are repeated for a specified number of training iterations or until the parameters
have converged.
An illustration of the whole procedure is given below.

Step 0: Initialize the weight matrix and bias values with zeros (or small random
values).

Step 1: For each class 𝑘 compute a linear combination of the input features and the
weight vector of class 𝑘, that is, for each training example compute a score for each
class. For class 𝑘 and input vector 𝐱 (𝑖) we have:
𝑠𝑐𝑜𝑟𝑒𝑘 (𝐱 (𝑖) ) = 𝐰𝑘𝑇 ⋅ 𝐱 (𝑖) + 𝑏𝑘

Computer Science Department 56 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

where is the dot product and 𝐰(𝑘) the weight vector of class 𝑘. We can compute the scores for all
classes and training examples in parallel, using vectorization and broadcasting:

𝒔𝒄𝒐𝒓𝒆𝒔 = 𝑿 ⋅ 𝑾T + 𝒃

where 𝐗 is a matrix of shape (𝑛𝑠𝑎𝑚𝑝𝑙𝑒𝑠 , 𝑛𝑓𝑒𝑎𝑡𝑢𝑟𝑒𝑠 ) that holds all training examples, and 𝐖 is a
matrix of shape (𝑛𝑐𝑙𝑎𝑠𝑠𝑒𝑠 , 𝑛𝑓𝑒𝑎𝑡𝑢𝑟𝑒𝑠 ) that holds the weight vector for each class.

Step 2: Apply the softmax activation function to transform the scores into
probabilities. The probability that an input vector 𝐱 (𝑖) belongs to class 𝑘 is given by
exp (𝑠𝑐𝑜𝑟𝑒𝑘 (𝐱 (𝑖) ))
(𝑖)
𝑝̂ 𝑘 (𝐱 ) =
∑𝐾 (𝑖)
𝑗=1 exp (𝑠𝑐𝑜𝑟𝑒𝑗 (𝐱 ))

Again we can perform this step for all classes and training examples at once using vectorization. The
class predicted by the model for 𝐱 (𝑖) is then simply the class with the highest probability.

Step 3: Compute the cost over the whole training set. We want our model to predict
a high probability for the target class and a low probability for the other classes. This
can be achieved using the cross-entropy loss function:
𝑚 𝐾
1 (𝑖) (𝑖)
𝐽(𝐖, 𝑏) = − ∑ ∑ [𝑦𝑘 log(𝑝̂𝑘 )]
𝑚
𝑖=1 𝑘=1
(𝑖)
In this formula, the target labels are one-hot encoded. So 𝑦𝑘 is 1 is the target class for 𝐱 (𝑖) is k,
(𝑖)
otherwise 𝑦𝑘 is 0.

Note: when there are only two classes, this cost function is equivalent to the cost function of
logistic regression.

Step 4: Compute the gradient of the cost function with respect to each weight vector
and bias.
The general formula for class 𝑘 is given by:
𝑚
1 (𝑖) (𝑖)
∇ ∗ 𝒘𝑘 𝐽(𝑾, 𝑏) = ∑ 𝒙(𝑖) [𝑝̂𝑘 − 𝑦𝑘 ]]
𝑚
𝑖=1

For the biases, the inputs 𝐱 (𝑖) will be given 1.

Computer Science Department 57 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
Step 5: Update the weights and biases for each class 𝑘:
𝐰𝑘 = 𝐰𝑘 − 𝜂 ∇𝐰𝑘 𝐽

𝑏𝑘 = 𝑏𝑘 − 𝜂 ∇𝑏𝑘 𝐽

where 𝜂 is the learning rate.

# fmt: off
from typing import *

import numpy as np
import pandas as pd
import seaborn as sns

import matplotlib.pyplot as plt

import matplotlib
import mlxtend.plotting as mlx_plot

import sklearn.linear_model as sk_linear_model

import sklearn.metrics as sk_metrics
import sklearn.preprocessing as sk_pre
import sklearn.compose as sk_compose
import sklearn.datasets as sk_datasets
from sklearn.model_selection import *

from typing import *

import itertools
# fmt: on

sns.set()
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
ROOT_INPUT_DIR = "./"

Computer Science Department 58 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Commonly used utility functions

def evaluate_categorical_predictions(y_true, y_pred, target_names=None):

"""
Returns the accuracy, balanced accuracy,
classification report per class and overall metrics
as a DataFrame, and confision matrix for the predictions
"""
crep = pd.DataFrame(sk_metrics.classification_report(
y_true, y_pred, target_names=target_names, digits=4, output_dict=True))
crep = crep.transpose()
extra_classes = ["accuracy", "macro avg", "weighted avg"]
scores_by_class = crep.drop(extra_classes)
overall_values = crep.loc[extra_classes]
scores_by_class["class-names"] = scores_by_class.index
return sk_metrics.accuracy_score(y_true, y_pred),
sk_metrics.balanced_accuracy_score(y_true, y_pred), scores_by_class, overall_values

def plot_confusion_matrix(cmat: np.ndarray, class_names: Optional[List[str]] =

None, **kwargs):
kwa = dict(
norm_colormap=matplotlib.colors.LogNorm(),
show_absolute=True,
show_normed=True,
colorbar=False,
)
kwa.update(kwargs)
if "figure" not in kwa:
kwa["figure"] = plt.figure(figsize=(8, 8))
mlx_plot.plot_confusion_matrix(
cmat,
class_names=class_names,
**kwa
)

def make_confusion_matrix(
y_true,
y_pred,
classes: list = None,
figsize: Tuple[int, int] = (10, 10),
text_size: int = 15,
Computer Science Department 59 L. D. College of Engineering, Ahmedabad-15
220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
norm: bool = False,
savefig: Union[str, bool] = False,
remove_diagonal: bool = False):
"""Makes a labelled confusion matrix comparing predictions and ground truth
labels.

If classes is passed, confusion matrix will be labelled, if not, integer class values
will be used.

Args:
y_true: Array of truth labels (must be same shape as y_pred).
y_pred: Array of predicted labels (must be same shape as y_true).
classes: Array of class labels (e.g. string form). If `None`, integer labels are used.
figsize: Size of output figure (default=(10, 10)).
text_size: Size of output figure text (default=15).
norm: normalize values or not (default=False).
savefig: save confusion matrix to file (default=False).

Returns:
A labelled confusion matrix plot comparing y_true and y_pred.

Example usage:
make_confusion_matrix(y_true=test_labels, # ground truth test labels
y_pred=y_preds, # predicted labels
classes=class_names, # array of class label names
figsize=(15, 15),
text_size=10)
"""
# Create the confustion matrix
cm: np.ndarray = sk_metrics.confusion_matrix(y_true, y_pred)
if remove_diagonal:
np.fill_diagonal(cm, 0)
cm_norm = cm.astype("float") / \
cm.sum(axis=1)[:, np.newaxis] # normalize it
n_classes = cm.shape[0] # find the number of classes we're dealing with

# Plot the figure and make it pretty

fig, ax = plt.subplots(figsize=figsize)
# colors will represent how 'correct' a class is, darker == better
cax = ax.matshow(cm, cmap=plt.cm.Blues)
plt.grid(False)
fig.colorbar(cax)

# Are there a list of classes?

if classes is not None:
Computer Science Department 60 L. D. College of Engineering, Ahmedabad-15
220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
labels = classes
else:
labels = np.arange(cm.shape[0])

# Label the axes

ax.set(title="Confusion Matrix",
xlabel="Predicted label",
ylabel="True label",
# create enough axis slots for each class
xticks=np.arange(n_classes),
yticks=np.arange(n_classes),
# axes will labeled with class names (if they exist) or ints
xticklabels=labels,
yticklabels=labels)
ax.tick_params(axis="x", rotation=50)

# Make x-axis labels appear on bottom

ax.xaxis.set_label_position("bottom")
ax.xaxis.tick_bottom()

# Set the threshold for different colors

threshold = (cm.max() + cm.min()) / 2.

# Plot the text on each cell

for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
if remove_diagonal and i == j:
continue
if norm:
plt.text(j, i, f"{cm[i, j]} ({cm_norm[i, j]*100:.1f}%)",
horizontalalignment="center",
color="white" if cm[i, j] > threshold else "black",
size=text_size)
else:
plt.text(j, i, f"{cm[i, j]}",
horizontalalignment="center",
color="white" if cm[i, j] > threshold else "black",
size=text_size)
# Save the figure to the current working directory
if savefig != False:
if savefig == True:
savefig = "confusion_matrix.png"
fig.savefig(savefig)

Computer Science Department 61 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Implementation of previously specified Multi-Class logistic regression algorithm:

class MultiClassLogisticRegression:
def __init__(self, n_iter=10000, thres=1e-3):
self.n_iter = n_iter
self.thres = thres

def fit(self, X, y, batch_size=64, lr=0.001, rand_seed=4, verbose=False):

np.random.seed(rand_seed)
self.classes = np.unique(y)
self.class_labels = {c: i for i, c in enumerate(self.classes)}
X = self.add_bias(X)
y = self.one_hot(y)
self.loss = []
self.weights = np.zeros(shape=(len(self.classes), X.shape[1]))
self.fit_data(X, y, batch_size, lr, verbose)
return self

def fit_data(self, X, y, batch_size, lr, verbose):

i=0
while (not self.n_iter or i < self.n_iter):
self.loss.append(self.cross_entropy(y, self.predict_(X)))
idx = np.random.choice(X.shape[0], batch_size)
X_batch, y_batch = X[idx], y[idx]
error = y_batch - self.predict_(X_batch)
update = (lr * np.dot(error.T, X_batch))
self.weights += update
if np.abs(update).max() < self.thres:
break
if i % 1000 == 0 and verbose:
print(' Training Accuray at {} iterations is {}'.format(
i, self.evaluate_(X, y)))
i += 1

def predict(self, X):

return self.predict_(self.add_bias(X))

def predict_(self, X):

pre_vals = np.dot(X, self.weights.T).reshape(-1, len(self.classes))
return self.softmax(pre_vals)

Computer Science Department 62 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

def softmax(self, z):

return np.exp(z) / np.sum(np.exp(z), axis=1).reshape(-1, 1)

def predict_classes(self, X):

self.probs_ = self.predict(X)
return np.vectorize(lambda c: self.classes[c])(np.argmax(self.probs_, axis=1))

def add_bias(self, X):

return np.insert(X, 0, 1, axis=1)

def get_randon_weights(self, row, col):

return np.zeros(shape=(row, col))

def one_hot(self, y):

return np.eye(len(self.classes))[np.vectorize(lambda c:
self.class_labels[c])(y).reshape(-1)]

def evaluate_(self, X, y):

return np.mean(np.argmax(self.predict_(X), axis=1) == np.argmax(y, axis=1))

def cross_entropy(self, y, probs):

return -1 * np.mean(y * np.log(probs))

Import Iris classification test dataset

df: pd.DataFrame = sk_datasets.load_iris(as_frame=True).frame

df.head()

Output:

sepal length (cm) sepal width (cm) petal length (cm) petal width (cm) target
0 5.1 3.5 1.4 0.2 0
1 4.9 3.0 1.4 0.2 0
2 4.7 3.2 1.3 0.2 0
3 4.6 3.1 1.5 0.2 0
4 5.0 3.6 1.4 0.2 0

Computer Science Department 63 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
Split into input & output datasets

X: pd.DataFrame = df.drop(columns=["target"])
y: pd.Series = df["target"]
X.shape
Output:

(150, 4)

Train-test split

X_train: pd.DataFrame
X_val: pd.DataFrame
y_train: pd.Series
y_val: pd.Series
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.4)
X_train.reset_index(drop=True, inplace=True)
X_val.reset_index(drop=True, inplace=True)
y_train.reset_index(drop=True, inplace=True)
y_val.reset_index(drop=True, inplace=True)
X_train.shape

Output:

(90, 4)

Plot numerical columns

# All columns are numerical

numeric_columns = X_train.columns
w=5
h=5
cols = 2
rows = int(np.ceil(len(numeric_columns) / cols))
plt.figure(figsize=(cols * w, rows * h))
for i, col in enumerate(numeric_columns):
plt.subplot(rows, cols, i+1)
sns.categorical.violinplot(y=X_train[col], x=y_train)
# plt.hist(inputs[col])
plt.xticks(rotation=75)
plt.title(col)

Computer Science Department 64 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Perform Multi-Class Logistic Regression

model = MultiClassLogisticRegression(thres=1e-5)
model.fit(X_train, y_train, lr=0.0001)
m_preds = model.predict(X_val).argmax(axis=1)

View accuracy & balanced accuracy

m_acc, m_bacc, m_class_metrics, m_metrics =

evaluate_categorical_predictions(y_val, m_preds)
m_acc, m_bacc
Output:

(0.95, 0.9444444444444445)

Computer Science Department 65 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

View overall metrics

m_metrics

Output:

precision recall f1-score support

accuracy 0.950000 0.950000 0.950000 0.95

macro avg 0.956522 0.944444 0.946441 60.00

weighted avg 0.956522 0.950000 0.949471 60.00

View metrics by class

m_class_metrics

Output:

precision recall f1-score support class-names

0 1.000000 1.000000 1.000000 22.0 0

1 1.000000 0.833333 0.909091 18.0 1

2 0.869565 1.000000 0.930233 20.0 2

View confusion matrix

make_confusion_matrix(y_val, m_preds)

Computer Science Department 66 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

View development of loss during training

fig = plt.figure(figsize=(8, 6))

plt.plot(np.arange(len(model.loss)), model.loss)

plt.title("Development of loss during training")

plt.xlabel("Number of iterations")

plt.ylabel("Loss")

plt.show()

Computer Science Department 67 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

Visualize the decision boundaries for 2 columns

X_train.columns
Index(['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'],
dtype='object')
col0 = "sepal length (cm)"
col1 = "sepal width (cm)"
X_train_trimmed = X_train[[col0, col1]] # we only take the first two features.

logreg = MultiClassLogisticRegression()

logreg.fit(X_train_trimmed, y_train)

# Plot the decision boundary. For that, we will assign a color to each
# point in the mesh [x_min, x_max]x[y_min, y_max].
x_min, x_max = X[col0].min() - .5, X[col0].max() + .5
y_min, y_max = X[col1].min() - .5, X[col1].max() + .5
h = .02 # step size in the mesh

Computer Science Department 68 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya
xx, yy = np.meshgrid(np.arange(x_min, x_max, h), np.arange(y_min, y_max, h))
Z = logreg.predict_classes(np.c_[xx.ravel(), yy.ravel()])

# Put the result into a color plot

Z = Z.reshape(xx.shape)
plt.figure(1, figsize=(10, 10))
plt.pcolormesh(xx, yy, Z, cmap=plt.cm.rainbow)

# Plot also the training points

plt.scatter(X_train_trimmed[col0], X_train_trimmed[col1], c=y_train, edgecolors='k',
cmap=plt.cm.rainbow)
plt.xlabel(col0)
plt.ylabel(col1)

plt.xlim(xx.min(), xx.max())
plt.ylim(yy.min(), yy.max())
plt.xticks(())
plt.yticks(())

plt.show()

Computer Science Department 69 L. D. College of Engineering, Ahmedabad-15

220280116044 Introduction to AI and Machine Learning (114AG01) Vatsal Kathiriya

fig = plt.figure(figsize=(8, 6))

plt.plot(np.arange(len(logreg.loss)), logreg.loss)
plt.title("Development of loss during training")
plt.xlabel("Number of iterations")
plt.ylabel("Loss")
plt.show()

Computer Science Department 70 L. D. College of Engineering, Ahmedabad-15