CAUVERY INSTITUTE OF TECHNOLOGY

ELECTRONICS AND COMMUNICTAION ENGINEERING DEPARTMENT

Python Programming for Machine Learning Applications

[BEC657D]

6th SEMESTER,
EVEN 2025

Prepared by
TEJASWINI H S

Assistant Professor
Dept. of ECE
CIT, Mandya

ELECTRONICS AND COMMUNICTAION ENGINEERING

DEPARTMENT
 CAUVERY INSTITUTE OF TECHNOLOGY
ELECTRONICS AND COMMUNICTAION ENGINEERING DEPARTMENT

INDEX

Expt. No. Title of Experiments Page No.

What is Python?

Python is an interpreted, object-oriented, high-level programming language with

dynamic semantics. Its high-level built in data structures, combined with dynamic typing
and dynamic binding, make it very attractive for Rapid Application Development, as well
as for use as a scripting or glue language to connect existing components together.

Python's simple, easy to learn syntax emphasizes readability and therefore reduces the
cost of program maintenance.

What is Machine Learning?

Machine learning is a branch of artificial intelligence that enables algorithms to uncover

hidden patterns within datasets. It allows them to predict new, similar data without
explicit programming for each task.

Machine learning finds applications in diverse fields such as image and speech
recognition, natural language processing, recommendation systems, fraud detection,
portfolio optimization, and automating tasks.

Inter-relationship between Python and Machine Learning

Python is considered one of the most popular programming languages used for Machine
Learning due to its readily available libraries like NumPy, Pandas, Scikit-learn,
TensorFlow, and Keras, which significantly simplify the process of building and
implementing machine learning models, making it a widely preferred choice for data
analysis and predictive modeling across various industries.

Software Used:

Python – 3.13.2 Version

PIP

IDLE — Python editor and shell

PyCharm

PIP
Installation links:

https://www.python.org/downloads/

https://www.jetbrains.com/pycharm/download/?section=windows

How to check Python Version in Command prompt:

python –-version

pip install packages

List of Libraries:

1. TensorFlow: This library was developed by Google in collaboration with the Brain
Team. It is an open-source library used for high-level computations. It is also used
in machine learning and deep learning algorithms. It contains a large number of
tensor operations. Researchers also use this Python library to solve complex
computations in Mathematics and Physics.

2. Matplotlib: This library is responsible for plotting numerical data. And that’s why
it is used in data analysis. It is also an open-source library and plots high-defined
figures like pie charts, histograms, scatterplots, graphs, etc.

3. Pandas: Pandas are an important library for data scientists. It is an open-source

machine learning library that provides flexible high-level data structures and a
variety of analysis tools. It eases data analysis, data manipulation, and cleaning
of data. Pandas support operations like Sorting, Re-indexing, Iteration,
Concatenation, Conversion of data, Visualizations, Aggregations, etc.

4. Numpy: The name “Numpy” stands for “Numerical Python”. It is the commonly
used library. It is a popular machine learning library that supports large matrices
and multi-dimensional data. It consists of in-built mathematical functions for
easy computations. Even libraries like TensorFlow use Numpy internally to
perform several operations on tensors. Array Interface is one of the key features of
this library.
5. SciPy: The name “SciPy” stands for “Scientific Python”. It is an open-source
library used for high-level scientific computations. This library is built over an
extension of Numpy. It works with Numpy to handle complex computations. While
Numpy allows sorting and indexing of array data, the numerical data code is
stored in SciPy. It is also widely used by application developers and engineers.

6. Scrapy: It is an open-source library that is used for extracting data from websites.
It provides very fast web crawling and high-level screen scraping. It can also be
used for data mining and automated testing of data.

7. Scikit-learn: It is a famous Python library to work with complex data. Scikit-learn

is an open-source library that supports machine learning. It supports variously
supervised and unsupervised algorithms like linear regression, classification,
clustering, etc. This library works in association with Numpy and SciPy.

8. PyGame: This library provides an easy interface to the Standard Directmedia

Library (SDL) platform-independent graphics, audio, and input libraries. It is used
for developing video games using computer graphics and audio libraries along
with Python programming language.

9. PyTorch: PyTorch is the largest machine learning library that optimizes tensor
computations. It has rich APIs to perform tensor computations with strong GPU
acceleration. It also helps to solve application issues related to neural networks.

10. PyBrain: The name “PyBrain” stands for Python Based Reinforcement
Learning, Artificial Intelligence, and Neural Networks library. It is an open-
source library built for beginners in the field of Machine Learning. It provides fast
and easy-to-use algorithms for machine learning tasks. It is so flexible and easily
understandable and that’s why is really helpful for developers that are new in
research fields

.
1|Page Python Programming for Machine Learning Applications [BEC657D]

Program 1:

Solve the Tic-Tac-Toe problem using the Depth First Search technique.

grid = []
line = []
for i in range(3):
for j in range(3):
line.append(" ")
grid.append(line)
line = []

# grid printing
def print_grid():
for i in range(3):
print("|", end="")
for j in range(3):
print(grid[i][j], "|", end="")
print("")

# player turn
def player_turn(turn_player1):
if turn_player1 == True:
turn_player1 = False
print(f"It's {player2}'s turn")
else:
turn_player1 = True
print(f"It's {player1}'s turn")
return turn_player1

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
2|Page Python Programming for Machine Learning Applications [BEC657D]

# choosing cell
def write_cell(cell):
cell -= 1
i = int(cell / 3)
j = cell % 3
if turn_player1 == True:
grid[i][j] = player1_symbol
else:
grid[i][j] = player2_symbol
return grid

# checking cell
def free_cell(cell):
cell -= 1
i = int(cell / 3)
j = cell % 3
if grid[i][j] == player1_symbol or grid[i][j] == player2_symbol:
print("This cell is not free")
return False
return True

# game opening
print("Welcome to the Tic-Tac-Toe !")
print("")
print_grid()
print("")
player1 = input("Please enter name of player 1 : ")
player1_symbol = input("Please enter the symbol of player 1 : ")
player2 = input("Please enter name of player 2 : ")
player2_symbol = input("Please enter the symbol of player 2 : ")
Department of ECE CIT Mandya Prof.Tejaswini H S,
Asst.Professor
3|Page Python Programming for Machine Learning Applications [BEC657D]

game = True
full_grid = False
turn_player1 = False
winner = ""

# win check
def win_check(grid, player1_symbol, player2_symbol):
full_grid = True
player1_symbol_count = 0
player2_symbol_count = 0
# checking rows
for i in range(3):
for j in range(3):
if grid[i][j] == player1_symbol:
player1_symbol_count += 1
player2_symbol_count = 0
if player1_symbol_count == 3:
game = False
winner = player1
return game, winner
if grid[i][j] == player2_symbol:
player2_symbol_count += 1
player1_symbol_count = 0
if player2_symbol_count == 3:
game = False
winner = player2
return game, winner
if grid[i][j] == " ":
full_grid = False

player1_symbol_count = 0
Department of ECE CIT Mandya Prof.Tejaswini H S,
Asst.Professor
4|Page Python Programming for Machine Learning Applications [BEC657D]

player2_symbol_count = 0
# checking columns
player1_symbol_count = 0
player2_symbol_count = 0
for i in range(3):
for j in range(3):
for k in range(3):
if i + k <= 2:
if grid[i + k][j] == player1_symbol:
player1_symbol_count += 1
player2_symbol_count = 0
if player1_symbol_count == 3:
game = False
winner = player1
return game, winner
if grid[i + k][j] == player2_symbol:
player2_symbol_count += 1
player1_symbol_count = 0
if player2_symbol_count == 3:
game = False
winner = player2
return game, winner
if grid[i][j] == " ":
full_grid = False

player1_symbol_count = 0
player2_symbol_count = 0
# checking diagonals
player1_symbol_count = 0
player2_symbol_count = 0
for i in range(3):
for j in range(3):
Department of ECE CIT Mandya Prof.Tejaswini H S,
Asst.Professor
5|Page Python Programming for Machine Learning Applications [BEC657D]

for k in range(3):
if j + k <= 2 and i + k <= 2:
if grid[i + k][j + k] == player1_symbol:
player1_symbol_count += 1
player2_symbol_count = 0
if player1_symbol_count == 3:
game = False
winner = player1
return game, winner
if grid[i + k][j + k] == player2_symbol:
player2_symbol_count += 1
player1_symbol_count = 0
if player2_symbol_count == 3:
game = False
winner = player2
return game, winner
if grid[i][j] == " ":
full_grid = False

player1_symbol_count = 0
player2_symbol_count = 0

player1_symbol_count = 0
player2_symbol_count = 0
for i in range(3):
for j in range(3):
for k in range(3):
if j - k >= 0 and i + k <= 2:
if grid[i + k][j - k] == player1_symbol:
player1_symbol_count += 1
player2_symbol_count = 0
if player1_symbol_count == 3:
Department of ECE CIT Mandya Prof.Tejaswini H S,
Asst.Professor
6|Page Python Programming for Machine Learning Applications [BEC657D]

game = False
winner = player1
return game, winner
if grid[i + k][j - k] == player2_symbol:
player2_symbol_count += 1
player1_symbol_count = 0
if player2_symbol_count == 3:
game = False
winner = player2
return game, winner
if grid[i][j] == " ":
full_grid = False

player1_symbol_count = 0
player2_symbol_count = 0

# full grid or not

if full_grid == True:
game = False
winner = ""
return game, winner
else:
game = True
winner = ""
return game, winner

# game
while game == True:
turn_player1 = player_turn(turn_player1)
free_box = False
while free_box == False:
Department of ECE CIT Mandya Prof.Tejaswini H S,
Asst.Professor
7|Page Python Programming for Machine Learning Applications [BEC657D]

cell = int(input("Please enter a number for your case (1 to 9 from left to right and from
top to bottom) : "))
free_box = free_cell(cell)
grid = write_cell(cell)
print_grid()
game, winner = win_check(grid, player1_symbol, player2_symbol)

# end of game
if winner == player1:
print(f"Winner is {player1} !")
elif winner == player2:
print(f"Winner is {player2} !")
else:
print(f"Grid is full : equality for {player1} and {player2} !")

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
8|Page Python Programming for Machine Learning Applications [BEC657D]

Program 2:

Show that the 8-puzzle states are divided into two disjoint sets, such that any
state is reachable from any other state in the same set, while no state is reachable
from any state in the other set.

This code is designed to classify a list of states (representing configurations of a sliding

puzzle, such as the 8-puzzle) based on the number of inversions in each state. An
inversion is defined as a pair of tiles that are in the wrong order relative to each other.
The code separates the states into two sets: those with an even number of inversions
and those with an odd number.

Breakdown of the Code

1. Function count_inversions(state):
 Purpose: Counts the number of inversions in a given state.
 Parameters:
 state: A list representing the configuration of the puzzle,
where 0 represents the empty space.
 Logic:
 It initializes a counter inversions to zero.
 It creates a flat_state list that excludes the empty space (0) from the
original state.
 It uses two nested loops to compare each pair of tiles in flat_state. If a
tile at a lower index is greater than a tile at a higher index, it counts as
an inversion.
 Return Value: The total number of inversions.
2. Function classify_states(states):
 Purpose: Classifies a list of states into two sets based on the parity (even or
odd) of their inversion counts.
 Parameters:
 states: A list of states (each state is a list representing a puzzle
configuration).
 Logic:
 It initializes two empty lists: set_even and set_odd.

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
9|Page Python Programming for Machine Learning Applications [BEC657D]

 For each state in the input list, it calls count_inversions to get the
number of inversions.
 Depending on whether the number of inversions is even or odd, it
appends the state to the corresponding list.
 Return Value: A tuple containing the two lists: set_even and set_odd.
3. Example States:
 A list of example states is provided, where each state is a configuration of the
puzzle. The number of inversions for each state is commented next to it.
4. Classification and Output:
 The classify_states function is called with the example states, and the results
are stored in set_even and set_odd.
 Finally, the code prints the states classified as having even inversions and those
with odd inversions.

def count_inversions(state):

"""Count the number of inversions in the given state."""

inversions = 0
flat_state = [tile for tile in state if tile != 0] # Exclude the empty space (0)

for i in range(len(flat_state)):
for j in range(i + 1, len(flat_state)):
if flat_state[i] > flat_state[j]:
inversions += 1

return inversions

def classify_states(states):
"""Classify states into two sets based on the parity of inversions."""
set_even = []
set_odd = []

for state in states:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
10 | P a g e Python Programming for Machine Learning Applications [BEC657D]

inversions = count_inversions(state)
if inversions % 2 == 0:
set_even.append(state)
else:
set_odd.append(state)

return set_even, set_odd

# Example states (0 represents the empty space)

states = [
[1, 2, 3, 4, 5, 6, 7, 8, 0], # Solved state (0 inversions)
[1, 2, 3, 4, 5, 6, 0, 7, 8], # 1 inversion
[1, 2, 3, 4, 0, 5, 6, 7, 8], # 1 inversion
[1, 2, 3, 0, 4, 5, 6, 7, 8], # 2 inversions
[1, 2, 0, 3, 4, 5, 6, 7, 8], # 2 inversions
[0, 1, 2, 3, 4, 5, 6, 7, 8], # 0 inversions
[1, 0, 2, 3, 4, 5, 6, 7, 8], # 1 inversion
[1, 2, 3, 5, 4, 6, 7, 8, 0], # 3 inversions
]

set_even, set_odd = classify_states(states)

print("States with even inversions:")

for state in set_even:
print(state)

print("\nStates with odd inversions:")

for state in set_odd:
print(state)

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
11 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
12 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 3:
To represent and evaluate different scenarios using predicate logic and
knowledge rules.

class Predicate:
def __init__(self, name, *args):
self.name = name
self.args = args

def __repr__(self):
return f"{self.name}({', '.join(map(str, self.args))})"

class KnowledgeBase:
def __init__(self):
self.facts = set()
self.rules = []

def add_fact(self, fact):

self.facts.add(fact)

def add_rule(self, rule):

self.rules.append(rule)

def evaluate(self, query):

# Check if the query is a fact
if query in self.facts:
return True

# Evaluate rules
for rule in self.rules:
if rule['condition'](self):
self.facts.add(rule['conclusion'])
if query == rule['conclusion']:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
13 | P a g e Python Programming for Machine Learning Applications [BEC657D]

return True

return False

# Example predicates
is_mammal = Predicate('is_mammal', 'dog')
is_mammal2 = Predicate('is_mammal', 'cat')
is_bird = Predicate('is_bird', 'sparrow')

# Create a knowledge base

kb = KnowledgeBase()

# Add facts
kb.add_fact(is_mammal)
kb.add_fact(is_mammal2)

# Define rules
def rule1(kb):
return Predicate('is_mammal', 'dog') in kb.facts

def rule2(kb):
return Predicate('is_bird', 'sparrow') in kb.facts

# Add rules with conditions and conclusions

kb.add_rule({'condition': rule1, 'conclusion': Predicate('has_fur', 'dog')})
kb.add_rule({'condition': rule2, 'conclusion': Predicate('can_fly', 'sparrow')})

# Evaluate scenarios
query1 = Predicate('has_fur', 'dog')
query2 = Predicate('can_fly', 'sparrow')

print(f"Does the dog have fur? {kb.evaluate(query1)}") # Should return True

print(f"Can the sparrow fly? {kb.evaluate(query2)}") # Should return True

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
14 | P a g e Python Programming for Machine Learning Applications [BEC657D]

or
class KnowledgeBase:
def __init__(self):
self.facts = {}
self.rules = []

def add_fact(self, predicate, value):

if predicate not in self.facts:
self.facts[predicate] = set()
self.facts[predicate].add(value)

def add_rule(self, rule):

self.rules.append(rule)

def evaluate(self, predicate, value):

if predicate in self.facts and value in self.facts[predicate]:
return True
for rule in self.rules:
if rule['condition'](self, value):
return rule['conclusion'](self, value)
return False

# Define some predicates

def is_mammal(kb, animal):
return animal in kb.facts.get('mammals', set())

def is_bird(kb, animal):

return animal in kb.facts.get('birds', set())

# Define a rule
def can_fly(kb, animal):
if is_bird(kb, animal):

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
15 | P a g e Python Programming for Machine Learning Applications [BEC657D]

return True
return False

# Create a knowledge base

kb = KnowledgeBase()

# Add facts
kb.add_fact('mammals', 'dog')
kb.add_fact('mammals', 'cat')
kb.add_fact('birds', 'sparrow')
kb.add_fact('birds', 'eagle')

# Add rules
kb.add_rule({'condition': is_bird, 'conclusion': can_fly})

# Evaluate scenarios
animals_to_check = ['dog', 'sparrow', 'cat', 'eagle']

for animal in animals_to_check:

if kb.evaluate('mammals', animal):
print(f"{animal} is a mammal.")
elif kb.evaluate('birds', animal):
print(f"{animal} is a bird and can fly: {can_fly(kb, animal)}.")
else:
print(f"{animal} is neither a mammal nor a bird.")

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
16 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 4:

To apply the Find-S and Candidate Elimination algorithms to a concept learning

task and compare their inductive biases and outputs.

Candidate Elimination Algorithm

The Candidate Elimination algorithm maintains a set of hypotheses that are
consistent with the training examples. It keeps track of the most specific and most
general hypotheses and eliminates those that are inconsistent with the training data.

Candidate Elimination Algorithm:

import copy
G = [['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'],
['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?'], ['?', '?', '?', '?', '?', '?']]
S = ['Null', 'Null', 'Null', 'Null', 'Null', 'Null']
def is_consistent(hypothesis, example):
for i in range(len(hypothesis)):
if hypothesis[i] != '?' and hypothesis[i] != example[i]:
return False
return True
def update_hypotheses_positive(G, S, example):
for i in range(len(G)):
if is_consistent(G[i], example):
continue
else:
for j in range(len(G[i])):
if G[i][j] == '?':
G[i][j] = example[j]
else:
G[i][j] = '?'
for j in range(len(S)):
if S[j] == 'Null':
S[j] = example[j]
elif S[j] != example[j]:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
17 | P a g e Python Programming for Machine Learning Applications [BEC657D]

S[j] = '?'
def update_hypotheses_negative(G, S, example):
new_G = copy.deepcopy(G)
for i in range(len(G)):
if not is_consistent(G[i], example):
new_G.remove(G[i])
return new_G
instance1 = ['sunny', 'warm', 'normal', 'strong', 'warm', 'same']
instance2 = ['sunny', 'warm', 'high', 'strong', 'warm', 'same']
instance3 = ['rainy', 'cold', 'high', 'strong', 'warm', 'change']
instance4 = ['sunny', 'warm', 'high', 'strong', 'cool', 'change']
update_hypotheses_positive(G, S, instance1)
print("After instance 1:")
print("G =", G)
print("S =", S)
update_hypotheses_positive(G, S, instance2)
print("\nAfter instance 2:")
print("G =", G)
print("S =", S)
G = update_hypotheses_negative(G, S, instance3)
print("\nAfter instance 3:")
print("G =", G)
print("S =", S)
update_hypotheses_positive(G, S, instance4)
print("\nAfter instance 4:")
print("G =", G)
print("S =", S)

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
18 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Find-S Algorithm

The Find-S algorithm is a simple version of the concept learning algorithm that finds the
most specific hypothesis that is consistent with the training examples. It starts with the most
specific hypothesis (the most specific instance) and generalizes it only when it encounters
negative examples.

Find S – Algorithm

import numpy as np

class FindS:
def __init__(self):
self.hypothesis = None

def fit(self, X, y):

# Initialize the hypothesis to the most specific hypothesis
self.hypothesis = ['0'] * X.shape[1]

for i in range(len(X)):
if y[i] == 'yes': # Only consider positive examples
for j in range(len(X[i])):
# If the attribute is not a wildcard, update the hypothesis
if self.hypothesis[j] == '0':
self.hypothesis[j] = X[i][j]
elif self.hypothesis[j] != X[i][j]:
self.hypothesis[j] = '?'

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
19 | P a g e Python Programming for Machine Learning Applications [BEC657D]

def predict(self, X):

predictions = []
for x in X:
match = True
for h, a in zip(self.hypothesis, x):
if h != '?' and h != a:
match = False
break
predictions.append('yes' if match else 'no')
return predictions

# Example dataset
X = np.array([
['sunny', 'hot', 'high', 'false'],
['sunny', 'hot', 'high', 'true'],
['overcast', 'hot', 'high', 'false'],
['rainy', 'mild', 'high', 'false'],
['rainy', 'cool', 'normal', 'false'],
['rainy', 'cool', 'normal', 'true'],
['overcast', 'cool', 'normal', 'true'],
['sunny', 'mild', 'high', 'false'],
['sunny', 'cool', 'normal', 'false'],
['rainy', 'mild', 'normal', 'false'],
['sunny', 'mild', 'normal', 'true'],
['overcast', 'mild', 'high', 'true'],
['overcast', 'hot', 'normal', 'false'],
['rainy', 'mild', 'high', 'true']
])

y = np.array(['no', 'no', 'yes', 'no', 'yes', 'yes', 'yes', 'no', 'yes', 'no', 'yes', 'yes', 'no', 'yes'])

# Applying Find-S

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
20 | P a g e Python Programming for Machine Learning Applications [BEC657D]

find_s = FindS()
find_s.fit(X, y)
print("Find-S Hypothesis:", find_s.hypothesis)

# Predictions
predictions = find_s.predict(X)
print("Predictions:", predictions)

class VersionSpace:
def __init__(self):
self.hypotheses = []

def fit(self, X, y):

# Start with the most general hypothesis
self.hypotheses = [['?'] * X.shape[1]]

for i in range(len(X)):
if y[i] == 'yes':
# Update hypotheses for positive examples
self.hypotheses = [h for h in self.hypotheses if self.is_consistent(h, X[i])]
self.hypotheses.append(X[i].tolist())
else:
# Remove inconsistent hypotheses for negative examples
self.hypotheses = [h for h in self.hypotheses if not self.is_consistent(h,
X[i])]

def is_consistent(self, hypothesis, example):

for h, e in zip(hypothesis, example):
if h != '?' and h != e:
return False
return True

def predict(self, X):

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
21 | P a g e Python Programming for Machine Learning Applications [BEC657D]

predictions = []
for x in X:
match = any(self.is_consistent(h, x) for h in self.hypotheses)
predictions.append('yes' if match else 'no')
return predictions

# Applying Version Space

version_space = VersionSpace()
version_space.fit(X, y)
print("Version Space Hypotheses:", version_space.hypotheses)

# Predictions
predictions_vs = version_space.predict(X)
print("Predictions (Version Space):", predictions_vs)

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
22 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 5:

To construct a decision tree using the ID3 algorithm on a simple classification dataset

import pandas as pd

The ID3 (Iterative Dichotomiser 3) algorithm is a popular method for constructing decision
trees, which are used for classification tasks in machine learning. The decision tree is a
flowchart-like structure where each internal node represents a test on an attribute, each branch
represents the outcome of the test, and each leaf node represents a class label. The goal of the
ID3 algorithm is to create a decision tree that accurately classifies instances based on their
attributes.

import pandas as pd

# Create a simple dataset

data = {

'Outlook': ['Sunny', 'Sunny', 'Overcast', 'Rain', 'Rain', 'Rain', 'Overcast', 'Sunny',

'Sunny', 'Rain', 'Sunny', 'Overcast', 'Overcast', 'Rain'],

'Temperature': ['Hot', 'Hot', 'Hot', 'Mild', 'Cool', 'Cool', 'Mild', 'Mild', 'Hot', 'Mild',
'Cool', 'Mild', 'Cool', 'Cool'],

'Humidity': ['High', 'High', 'High', 'High', 'Normal', 'Normal', 'Normal', 'High',

'Normal', 'Normal', 'Normal', 'High', 'Normal', 'High'],

'Windy': [False, True, False, False, False, True, True, False, False, False, True, True,
False, True],

'Play': ['No', 'No', 'Yes', 'Yes', 'Yes', 'No', 'Yes', 'No', 'Yes', 'Yes', 'Yes', 'Yes', 'Yes',
'No']

df = pd.DataFrame(data)

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
23 | P a g e Python Programming for Machine Learning Applications [BEC657D]

from sklearn.preprocessing import LabelEncoder

# Initialize label encoders

le = LabelEncoder()

# Encode categorical features

df['Outlook'] = le.fit_transform(df['Outlook'])

df['Temperature'] = le.fit_transform(df['Temperature'])

df['Humidity'] = le.fit_transform(df['Humidity'])

df['Windy'] = le.fit_transform(df['Windy'])

df['Play'] = le.fit_transform(df['Play'])

# Separate features and target variable

X = df.drop('Play', axis=1)

y = df['Play']

from sklearn.tree import DecisionTreeClassifier

from sklearn.model_selection import train_test_split

from sklearn import tree

import matplotlib.pyplot as plt

# Split the dataset into training and testing sets

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
24 | P a g e Python Programming for Machine Learning Applications [BEC657D]

# Create the decision tree classifier

clf = DecisionTreeClassifier(criterion='entropy', random_state=42)

# Fit the model

clf.fit(X_train, y_train)

# Evaluate the model

accuracy = clf.score(X_test, y_test)

print(f'Accuracy: {accuracy:.2f}')

# Visualize the decision tree

plt.figure(figsize=(12,8))

tree.plot_tree(clf, filled=True, feature_names=X.columns, class_names=['No', 'Yes'])

plt.show()

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
25 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 6:

To assess how the ID3 algorithm performs on datasets with varying characteristics and
complexity, examining overfitting, underfitting, and decision tree depth.

import numpy as np

import pandas as pd

import matplotlib.pyplot as plt

from sklearn.datasets import make_classification, make_moons, make_circles

from sklearn.tree import DecisionTreeClassifier

from sklearn.model_selection import train_test_split

from sklearn.metrics import accuracy_score

# Create datasets

datasets = {

"Linear": make_classification(n_samples=100, n_features=2, n_informative=2,

n_redundant=0, random_state=42),

"Moons": make_moons(n_samples=100, noise=0.1, random_state=42),

"Circles": make_circles(n_samples=100, noise=0.1, factor=0.5, random_state=42)

# Function to evaluate decision tree performance

def evaluate_decision_tree(X, y, max_depths):

train_accuracies = []

test_accuracies = []

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
26 | P a g e Python Programming for Machine Learning Applications [BEC657D]

for depth in max_depths:

clf = DecisionTreeClassifier(max_depth=depth, criterion='entropy', random_state=42)

clf.fit(X_train, y_train)

train_pred = clf.predict(X_train)

test_pred = clf.predict(X_test)

train_accuracies.append(accuracy_score(y_train, train_pred))

test_accuracies.append(accuracy_score(y_test, test_pred))

return train_accuracies, test_accuracies

# Define maximum depths to evaluate

max_depths = range(1, 11)

# Store results

results = {}

for name, (X, y) in datasets.items():

train_acc, test_acc = evaluate_decision_tree(X, y, max_depths)

results[name] = (train_acc, test_acc)

# Plotting the results

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
27 | P a g e Python Programming for Machine Learning Applications [BEC657D]

plt.figure(figsize=(15, 10))

for name, (train_acc, test_acc) in results.items():

plt.plot(max_depths, train_acc, label=f'{name} - Train', marker='o')

plt.plot(max_depths, test_acc, label=f'{name} - Test', marker='x')

plt.title('Decision Tree Performance with Varying Depths')

plt.xlabel('Max Depth')

plt.ylabel('Accuracy')

plt.xticks(max_depths)

plt.ylim(0, 1)

plt.legend()

plt.grid()

plt.show()

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
28 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 7:

To examine different types of machine learning approaches (Supervised, Unsupervised,

Semi-supervised, and Reinforcement Learning) by setting up a basic classification
problem and exploring how each type applies differently.

Supervised Learning

 Definition: In supervised learning, algorithms are trained on labeled datasets, meaning

each training example is paired with an output label.

 Application: For a classification problem, you might use a dataset of emails labeled as
"spam" or "not spam." The algorithm learns to classify new emails based on the
features it identifies in the training data.

import numpy as np

from sklearn.model_selection import train_test_split

from sklearn.ensemble import RandomForestClassifier

from sklearn.metrics import accuracy_score

# Sample data (replace this with your actual data)

# Let's say we have 4 features and 10 samples

X = np.array([[1.0, 2.0, 3.0, 4.0],

[2.0, 3.0, 4.0, 5.0],

[3.0, 4.0, 5.0, 6.0],

[4.0, 5.0, 6.0, 7.0],

[5.0, 6.0, 7.0, 8.0],

[6.0, 7.0, 8.0, 9.0],

[7.0, 8.0, 9.0, 10.0],

[8.0, 9.0, 10.0, 11.0],

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
29 | P a g e Python Programming for Machine Learning Applications [BEC657D]

[9.0, 10.0, 11.0, 12.0],

[10.0, 11.0, 12.0, 13.0]])

# Labels (binary classification)

y = np.array(['spam', 'not spam', 'spam', 'not spam', 'spam',

'not spam', 'spam', 'not spam', 'spam', 'not spam'])

# Split the data

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Train the model

model = RandomForestClassifier()

model.fit(X_train, y_train)

# Make predictions

predictions = model.predict(X_test)

print("Accuracy:", accuracy_score(y_test, predictions))

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
30 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Unsupervised:

 Definition: Unsupervised learning involves training algorithms on datasets without

labeled outputs. The goal is to identify patterns or groupings within the data.

 Application: In a classification context, you might use clustering algorithms to group

similar items, such as customer segmentation based on purchasing behavior.

import numpy as np

from sklearn.cluster import KMeans

# Sample data (replace this with your actual data)

# Let's say we have 2 features and 10 samples

X = np.array([[1.0, 2.0],

[1.5, 1.8],

[5.0, 8.0],

[8.0, 8.0],

[1.0, 0.6],

[9.0, 11.0],

[8.0, 2.0],

[10.0, 2.0],

[9.0, 3.0],

[5.0, 4.0]])

# Train the KMeans model

kmeans = KMeans(n_clusters=3)

kmeans.fit(X)

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
31 | P a g e Python Programming for Machine Learning Applications [BEC657D]

# Predict clusters

clusters = kmeans.predict(X)

print("Cluster assignments:", clusters)

Observation and Result:

Semi Supervised:

 Definition: Semi-supervised learning combines both labeled and unlabeled data for
training. It typically starts with a small amount of labeled data and a larger amount of
unlabeled data.

 Application: This approach is useful when acquiring labeled data is expensive or time-
consuming, such as in image classification where only a few images are labeled.

import numpy as np

from sklearn.semi_supervised import SelfTrainingClassifier

from sklearn.ensemble import RandomForestClassifier

from sklearn.model_selection import train_test_split

# Sample data (replace this with your actual data)

# Let's say we have 4 features and 10 samples

X = np.array([[1.0, 2.0, 3.0, 4.0],

[2.0, 3.0, 4.0, 5.0],

[3.0, 4.0, 5.0, 6.0],

[4.0, 5.0, 6.0, 7.0],

[5.0, 6.0, 7.0, 8.0],

[6.0, 7.0, 8.0, 9.0],

[7.0, 8.0, 9.0, 10.0],

Department of ECE CIT Mandya Prof.Tejaswini H S,
Asst.Professor
32 | P a g e Python Programming for Machine Learning Applications [BEC657D]

[8.0, 9.0, 10.0, 11.0],

[9.0, 10.0, 11.0, 12.0],

[10.0, 11.0, 12.0, 13.0]])

# Labels (binary classification), -1 indicates unlabeled

y = np.array(['spam', -1, 'spam', 'not spam', 'spam',

-1, 'spam', 'not spam', 'spam', 'not spam'], dtype=object)

# Train the model

base_model = RandomForestClassifier()

self_training_model = SelfTrainingClassifier(base_model)

# Fit the model

self_training_model.fit(X, y)

# Make predictions

predictions = self_training_model.predict(X)

print("Predictions:", predictions)

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
33 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 8:

To understand how Find-S and Candidate Elimination algorithms search through the
hypothesis space in concept learning tasks, and to observe the role of inductive bias in
shaping the learned concept.

Find-S Algorithm

The Find-S algorithm is a simple version of a concept learning algorithm that finds the most
specific hypothesis that is consistent with the positive training examples. It starts with the most
specific hypothesis and generalizes it based on the positive examples.

Candidate Elimination Algorithm

The Candidate Elimination algorithm maintains a set of hypotheses that are consistent with the
training examples. It uses two sets: the general boundary (G) and the specific boundary (S).
The algorithm refines these sets based on the training examples.

import numpy as np

# Sample dataset

# Each row represents an example: [Sky, Air Temp, Humidity, Windy, Water, Forecast,
Enjoy Sport]

# 1 = Yes, 0 = No

data = np.array([

[1, 1, 0, 1, 1, 1, 1], # Enjoy Sport

[1, 1, 0, 1, 1, 0, 1], # Enjoy Sport

[1, 0, 1, 1, 1, 1, 0], # Do not enjoy Sport

[0, 1, 0, 1, 0, 1, 1], # Enjoy Sport

[0, 1, 1, 0, 0, 1, 0], # Do not enjoy Sport

[1, 1, 1, 0, 1, 1, 0], # Do not enjoy Sport

])

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
34 | P a g e Python Programming for Machine Learning Applications [BEC657D]

# Find-S Algorithm

def find_s(data):

# Initialize the most specific hypothesis

specific_hypothesis = ['0'] * (data.shape[1] - 1)

for example in data:

if example[-1] == 1: # Positive example

for i in range(len(specific_hypothesis)):

if specific_hypothesis[i] == '0':

specific_hypothesis[i] = example[i]

elif specific_hypothesis[i] != example[i]:

specific_hypothesis[i] = '?'

return specific_hypothesis

# Candidate Elimination Algorithm

def candidate_elimination(data):

specific_hypothesis = ['0'] * (data.shape[1] - 1)

general_hypothesis = [['?'] * (data.shape[1] - 1)]

for example in data:

if example[-1] == 1: # Positive example

# Update specific hypothesis

for i in range(len(specific_hypothesis)):

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
35 | P a g e Python Programming for Machine Learning Applications [BEC657D]

if specific_hypothesis[i] == '0':

specific_hypothesis[i] = example[i]

elif specific_hypothesis[i] != example[i]:

specific_hypothesis[i] = '?'

# Remove general hypotheses that are inconsistent with the positive example

general_hypothesis = [g for g in general_hypothesis if is_consistent(g, example)]

else: # Negative example

# Remove specific hypotheses that are inconsistent with the negative example

specific_hypothesis = [s for s in specific_hypothesis if not is_consistent([s],

example)]

# Add new general hypotheses

new_general_hypotheses = generate_general_hypotheses(example,
specific_hypothesis)

general_hypothesis.extend(new_general_hypotheses)

return specific_hypothesis, general_hypothesis

def is_consistent(hypothesis, example):

for h, e in zip(hypothesis, example[:-1]):

if h != '?' and h != e:

return False

return True

def generate_general_hypotheses(negative_example, specific_hypothesis):

new_general_hypotheses = []

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
36 | P a g e Python Programming for Machine Learning Applications [BEC657D]

for i in range(len(specific_hypothesis)):

if specific_hypothesis[i] != '?' and specific_hypothesis[i] != negative_example[i]:

new_hypothesis = specific_hypothesis.copy()

new_hypothesis[i] = '?'

new_general_hypotheses.append(new_hypothesis)

return new_general_hypotheses

# Running the algorithms

find_s_hypothesis = find_s(data)

print("Find-S Hypothesis:", find_s_hypothesis)

specific_hypothesis, general_hypothesis = candidate_elimination(data)

print("Candidate Elimination Specific Hypothesis:", specific_hypothesis)

print("Candidate Elimination General Hypothesis:", general_hypothesis)

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
37 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 9:

To go through all stages of a real-life machine learning project, from data collection to
model fine-tuning, using a regression dataset like the "California Housing Prices."

1. Data Collection

 Acquire the California Housing Prices dataset, which contains metrics such as
population, median income, and median housing prices for various districts in
California.

 Load the dataset using libraries like Pandas.

import pandas as pd

housing = pd.read_csv("F:\\python\\housing.csv")

print(housing.head())

from sklearn.impute import SimpleImputer

imputer = SimpleImputer(strategy="median")

housing_num = housing.drop('ocean_proximity', axis=1)

imputer.fit(housing_num)

X = imputer.transform(housing_num)

housing_tr = pd.DataFrame(X, columns=housing_num.columns)

housing['rooms_per_household'] = housing['total_rooms'] / housing['households']

import matplotlib.pyplot as plt

housing.plot(kind='scatter', x='median_income', y='median_house_value', alpha=0.1)

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
38 | P a g e Python Programming for Machine Learning Applications [BEC657D]

plt.show()

corr_matrix = housing.corr()

print(corr_matrix['median_house_value'].sort_values(ascending=False))

from sklearn.linear_model import LinearRegression

from sklearn.tree import DecisionTreeRegressor

from sklearn.ensemble import RandomForestRegressor

from sklearn.model_selection import train_test_split

train_set, test_set = train_test_split(housing, test_size=0.2, random_state=42)

lin_reg = LinearRegression()

lin_reg.fit(housing_prepared, housing_labels)

from sklearn.metrics import mean_squared_error

housing_predictions = lin_reg.predict(housing_prepared)

lin_mse = mean_squared_error(housing_labels, housing_predictions)

lin_rmse = np.sqrt(lin_mse)

print(lin_rmse)

from sklearn.model_selection import GridSearchCV

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
39 | P a g e Python Programming for Machine Learning Applications [BEC657D]

param_grid = [

{'n_estimators': [3, 10, 30], 'max_features': [2, 4, 6, 8]},

{'bootstrap': [False], 'n_estimators': [3, 10], 'max_features': [2, 3, 4]},

grid_search = GridSearchCV(RandomForestRegressor(), param_grid, cv=5,

scoring='neg_mean_squared_error')

grid_search.fit(housing_prepared, housing_labels)

final_model = grid_search.best_estimator_

X_test_prepared = full_pipeline.transform(X_test)

final_predictions = final_model.predict(X_test_prepared)

final_mse = mean_squared_error(y_test, final_predictions)

final_rmse = np.sqrt(final_mse)

print(final_rmse)

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
40 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Program 10:

To perform binary and multiclass classification on the MNIST dataset, analyze

performance metrics, and perform error analysis.

More Explanation:

https://www.geeksforgeeks.org/evaluation-metrics-for-classification-model-in-python/

from keras.datasets import mnist

from keras.utils import to_categorical

from keras.models import Sequential

from keras.layers import Conv2D, MaxPooling2D, Flatten, Dense

from sklearn.metrics import accuracy_score, confusion_matrix, precision_score,

recall_score, f1_score, roc_auc_score, roc_curve

import matplotlib.pyplot as plt

import seaborn as sns

(train_X, train_y), (test_X, test_y) = mnist.load_data()

train_X = train_X.reshape((train_X.shape[0], 28, 28, 1)).astype('float32') / 255

test_X = test_X.reshape((test_X.shape[0], 28, 28, 1)).astype('float32') / 255

train_y = to_categorical(train_y)

test_y = to_categorical(test_y)

model = Sequential([

Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)),

MaxPooling2D((2, 2)),

Flatten(),

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor
41 | P a g e Python Programming for Machine Learning Applications [BEC657D]

Dense(100, activation='relu'),

Dense(10, activation='softmax')

])

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

model.fit(train_X, train_y, epochs=3, batch_size=200, validation_split=0.2, verbose=2)

y_pred = model.predict(test_X)

y_pred_classes = y_pred.argmax(axis=1)

y_test_classes = test_y.argmax(axis=1)

Observation and Result:

Department of ECE CIT Mandya Prof.Tejaswini H S,

Asst.Professor