Aml File Sam
Aml File Sam
VISION OF INSTITUTE
MISSION OF INSTITUTE
INDEX
Updated
Faculty
Marks Remark Marks
Signature
(If any)
S.No Experiment Date
Class
Laboratory
Participation Viva (5
Assessment
(5 Marks) Marks)
(15 Marks)
Implement a deep neural network from
1 scratch using TensorFlow or PyTorch,
gaining Handson experience in building
complex neural architectures
Utilize pre-trained models and perform
2 transfer learning to solve real-world
problems efficiently.
Experiment No. 1
Problem Statement: Implement a deep neural network from scratch using TensorFlow or
PyTorch, gaining handson experience in building complex neural architectures.
Theory:
Implementing a deep neural network (DNN) from scratch involves constructing and training a
model composed of multiple layers, where each layer is responsible for learning hierarchical
patterns from the input data. A DNN typically includes an input layer, several hidden layers
(each containing neurons or units), and an output layer for predictions. In frameworks like
TensorFlow or PyTorch, you manually define each layer, activation functions (such as ReLU,
Sigmoid), and optimizers (e.g., Adam, SGD) for training. Backpropagation is used to update
weights through gradient descent based on the error from predictions. During training, the
network learns to minimize the loss function, thereby improving accuracy. This hands-on
approach gives deeper insight into how DNNs operate and allows for fine-tuning of architecture
and hyperparameters for better performance.
Source Code:
import numpy as np
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
df = pd.read_csv('/content/winequality-red.csv')
df.head()
input_shape = [X_train.shape[1]]
print("Input shape:", input_shape)
model = tf.keras.Sequential([
tf.keras.layers.Dense(units=128, activation='relu',
input_shape=input_shape),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Dense(units=64, activation='relu'),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Dense(units=32, activation='relu'),
tf.keras.layers.Dense(units=1)
])
model.summary()
early_stopping = tf.keras.callbacks.EarlyStopping(patience=5,
restore_best_weights=True)
history = model.fit(
X_train, y_train,
validation_data=(X_val, y_val),
batch_size=256,
epochs=50,
callbacks=[early_stopping]
)
loss_df = pd.DataFrame(history.history)
loss_df[['loss', 'val_loss']].plot()
plt.title('Training and Validation Loss')
plt.xlabel('Epochs')
plt.ylabel('Loss')
plt.show()
Output:
Learning Outcomes:
Experiment No. 2
Problem Statement: Utilize pre-trained models and perform transfer learning to solve real-
world problems efficiently.
Theory:
Pre-trained models are neural networks trained on large datasets like ImageNet that capture
generic patterns applicable to many tasks. Transfer learning leverages these models by fine-
tuning them on smaller, task-specific datasets. This approach dramatically reduces training time
and computational costs while improving performance, especially when data is limited. By
freezing early layers (which capture basic features like edges or textures) and retraining the later
layers on new data, the model adapts to the target problem. Transfer learning is widely used in
applications such as image classification, object detection, and natural language processing.
Source Code:
import matplotlib.pyplot as plt
import numpy as np
import os
import tensorflow as tf
import tensorflow.keras.layers as tfl
from tensorflow.keras.preprocessing import image_dataset_from_directory
seed = 7
BATCH_SIZE = 32
IMG_SIZE = (160, 160)
directory = "/content/my_extracted_folder/rural_and_urban_photos/train"
train_dataset = image_dataset_from_directory(directory,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMG_SIZE,
validation_split=0.4,
subset='training',
seed=seed)
validation_dataset = image_dataset_from_directory(directory,
shuffle=True,
batch_size=BATCH_SIZE,
image_size=IMG_SIZE,
validation_split=0.4,
subset='validation',
seed=seed)
class_names = train_dataset.class_names
plt.figure(figsize=(10, 10))
for images, labels in train_dataset.take(1):
for i in range(9):
ax = plt.subplot(3, 3, i + 1)
plt.imshow(images[i].numpy().astype("uint8"))
plt.title(class_names[labels[i]])
plt.axis("off")
AUTOTUNE = tf.data.experimental.AUTOTUNE
train_dataset = train_dataset.prefetch(buffer_size=AUTOTUNE)
def data_augmenter():
data_augmentation = tf.keras.Sequential()
data_augmentation.add(tfl.RandomZoom(.5, .2))
data_augmentation.add(tfl.RandomFlip('horizontal'))
data_augmentation.add(tfl.RandomRotation(0.2))
return data_augmentation
data_augmentation = data_augmenter()
for image, _ in train_dataset.take(1):
plt.figure(figsize=(10, 10))
first_image = image[0]
for i in range(9):
ax = plt.subplot(3, 3, i + 1)
augmented_image = data_augmentation(tf.expand_dims(first_image, 0))
plt.imshow(augmented_image[0] / 255)
plt.axis('off')
preprocess_input = tf.keras.applications.mobilenet_v2.preprocess_input
def classification_model(image_shape=IMG_SIZE,
data_augmentation=data_augmenter()):
input_shape = IMG_SIZE + (3,)
base_model = tf.keras.applications.MobileNetV2(input_shape=input_shape,
include_top=False,weights='imagenet')
base_model.trainable = False
inputs = tf.keras.Input(shape=input_shape)
x = data_augmentation(inputs)
x = preprocess_input(x)
x = base_model(x, training=False)
x = tfl.GlobalAveragePooling2D()(x)
x = tfl.Dropout(0.2)(x)
outputs = tfl.Dense(1)(x)
model = tf.keras.Model(inputs, outputs)
return model
model2 = classification_model(IMG_SIZE, data_augmentation)
base_learning_rate = 0.01
model2.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=base_learn
ing_rate),
loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
initial_epochs = 10
history = model2.fit(train_dataset, validation_data=validation_dataset,
epochs=initial_epochs)
#fine-tune
base_model = model2.layers[2]
base_model.trainable = True
fine_tune_at = 120
metrics=['accuracy']
model2.compile(loss=loss_function,
optimizer = optimizer,
metrics=metrics)
fine_tune_epochs = 5
total_epochs = initial_epochs + fine_tune_epochs
history_fine = model2.fit(train_dataset,
epochs=total_epochs,
initial_epoch=history.epoch[-1],
validation_data=validation_dataset)
#evaluate
directory = "/content/my_extracted_folder/rural_and_urban_photos/val"
test_dataset = image_dataset_from_directory(directory, shuffle=True,
batch_size=BATCH_SIZE, image_size=IMG_SIZE, validation_split=0.4,
subset='training', seed=seed)
model2.evaluate(test_dataset)
Output:
Learning Outcomes:
Experiment No. 3
Problem Statement: Implement a GAN to generate synthetic data and explore its applications
in image generation and data augmentation.
Theory:
Generative Adversarial Networks (GANs) consist of two neural networks: a generator that
creates synthetic data and a discriminator that distinguishes between real and synthetic data. Both
networks are trained simultaneously in a competitive setting, with the generator improving to
"fool" the discriminator. GANs are popular for generating high-quality images, video frames,
and data augmentation. Applications include image synthesis (such as faces or artwork), super-
resolution, and filling gaps in missing data. GANs have proven useful in creating synthetic
datasets for training machine learning models when real data is scarce or expensive to collect.
Source Code:
Image Generation
import tensorflow as tf
import tensorflow_datasets as tfds
import numpy as np
import matplotlib.pyplot as plt
def visualize_data(dataset):
plt.figure(figsize=(8, 8))
for i, (image, label) in enumerate(dataset.take(16)):
plt.subplot(4, 4, i + 1)
plt.imshow(image.numpy().squeeze(), cmap='gray')
plt.axis('off')
plt.show()
visualize_data(train_data)
train_data = train_data.map(preprocess).shuffle(60000).batch(256)
LATENT_DIM = 100
def build_generator():
model = tf.keras.Sequential([
tf.keras.layers.Dense(7 * 7 * 256, use_bias=False,
input_shape=(LATENT_DIM,)),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(),
tf.keras.layers.Reshape((7, 7, 256)),
tf.keras.layers.Conv2DTranspose(128, (5, 5), strides=(1, 1),
padding='same', use_bias=False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(),
tf.keras.layers.Conv2DTranspose(64, (5, 5), strides=(2, 2),
padding='same', use_bias=False),
tf.keras.layers.BatchNormalization(),
tf.keras.layers.LeakyReLU(),
tf.keras.layers.Conv2DTranspose(1, (5, 5), strides=(2, 2),
padding='same', use_bias=False, activation='tanh')
])
return model
def build_discriminator():
model = tf.keras.Sequential([
tf.keras.layers.Conv2D(64, (5, 5), strides=(2, 2), padding='same',
input_shape=[28, 28, 1]),
tf.keras.layers.LeakyReLU(),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Conv2D(128, (5, 5), strides=(2, 2),
padding='same'),
tf.keras.layers.LeakyReLU(),
tf.keras.layers.Dropout(0.3),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(1)
])
return model
generator = build_generator()
discriminator = build_discriminator()
cross_entropy = tf.keras.losses.BinaryCrossentropy(from_logits=True)
def generator_loss(fake_output):
return cross_entropy(tf.ones_like(fake_output), fake_output)
generator_optimizer = tf.keras.optimizers.Adam(1e-4)
discriminator_optimizer = tf.keras.optimizers.Adam(1e-4)
EPOCHS = 50
NOISE_DIM = 100
NUM_EXAMPLES_TO_GENERATE = 16
@tf.function
def train_step(images):
noise = tf.random.normal([BATCH_SIZE, NOISE_DIM])
gen_loss = generator_loss(fake_output)
gradients_of_generator = gen_tape.gradient(gen_loss,
generator.trainable_variables)
gradients_of_discriminator = disc_tape.gradient(disc_loss,
discriminator.trainable_variables)
generator_optimizer.apply_gradients(zip(gradients_of_generator,
generator.trainable_variables))
discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator
, discriminator.trainable_variables))
for i in range(predictions.shape[0]):
plt.subplot(4, 4, i+1)
plt.imshow(predictions[i, :, :, 0] * 127.5 + 127.5, cmap='gray')
plt.axis('off')
plt.savefig(f'image_at_epoch_{epoch:04d}.png')
plt.show()
if (epoch + 1) % 10 == 0:
print(f'Epoch {epoch + 1}/{epochs} completed')
generate_and_save_images(generator, epoch, seed)
train(train_data, EPOCHS)
generator.save('fashion_mnist_generator.h5')
discriminator.save('fashion_mnist_discriminator.h5')
loaded_generator =
tf.keras.models.load_model('fashion_mnist_generator.h5')
loaded_discriminator =
tf.keras.models.load_model('fashion_mnist_discriminator.h5')
plt.figure(figsize=(4, 4))
for i in range(generated_images.shape[0]):
plt.subplot(4, 4, i + 1)
plt.imshow(generated_images[i, :, :, 0] * 127.5 + 127.5, cmap='gray')
plt.axis('off')
plt.show()
Data Augmentation
import numpy as np
import pandas as pd
import torch
import torch.nn as nn
import torch.optim as optim
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler, OneHotEncoder
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report, accuracy_score
iris = datasets.load_iris()
X = iris.data
y = iris.target
scaler = MinMaxScaler()
X = scaler.fit_transform(X)
real_labels = y
one_hot_encoder = OneHotEncoder(sparse_output=False)
one_hot_labels =
one_hot_encoder.fit_transform(np.array(real_labels).reshape(-1, 1))
NOISE_DIM = 100
NUM_CLASSES = 3
NUM_FEATURES = 4
BATCH_SIZE = 64
TRAINING_STEPS = 20000
class Generator(nn.Module):
def __init__(self):
super(Generator, self).__init__()
self.model = nn.Sequential(
nn.Linear(NOISE_DIM + NUM_CLASSES, 256),
nn.ReLU(),
nn.Linear(256, 128),
nn.ReLU(),
nn.Linear(128, NUM_FEATURES),
nn.Tanh() # Use Tanh for output layer
)
class Discriminator(nn.Module):
def __init__(self):
super(Discriminator, self).__init__()
self.model = nn.Sequential(
nn.Linear(NUM_FEATURES + NUM_CLASSES, 256),
nn.ReLU(),
nn.Linear(256, 128),
nn.ReLU(),
nn.Linear(128, 1),
nn.Sigmoid()
)
generator = Generator()
discriminator = Discriminator()
criterion = nn.BCELoss()
optimizer_G = optim.Adam(generator.parameters(), lr=0.0002)
optimizer_D = optim.Adam(discriminator.parameters(), lr=0.0002)
optimizer_D.zero_grad()
discriminator_loss.backward()
optimizer_D.step()
noise = torch.randn(BATCH_SIZE, NOISE_DIM)
generator_loss = criterion(discriminator(generator(noise,
labels_batch), labels_batch), torch.ones((BATCH_SIZE, 1)))
optimizer_G.zero_grad()
generator_loss.backward()
optimizer_G.step()
if step % 500 == 0:
print(f"Step: {step}, Discriminator Loss:
{discriminator_loss.item()}, Generator Loss: {generator_loss.item()}")
synthetic_data = pd.DataFrame()
for class_label in range(NUM_CLASSES):
class_data = generate_data(generator, class_label, 50)
synthetic_data = pd.concat([synthetic_data, class_data])
rf_real = RandomForestClassifier(n_estimators=100)
rf_real.fit(real_data.values, real_labels)
y_pred_real = rf_real.predict(real_data.values)
accuracy_real = accuracy_score(real_labels, y_pred_real)
print("Classification Report for Real Data:")
print(classification_report(real_labels, y_pred_real))
print(f"Accuracy for Real Data: {accuracy_real:.4f}")
Output:
Image Generation
Training Data
Generated Images
Data Augmentation
Learning Outcomes:
Problem Statement: Apply NLP techniques to process and analyze textual data, including
sentiment analysis and named entity recognition.
Theory:
Natural Language Processing (NLP) involves using computational techniques to analyze and
understand human language. Common tasks include sentiment analysis (determining the
emotional tone of text), named entity recognition (extracting entities like names, locations, or
organizations), text summarization, and translation. Techniques like tokenization, word
embeddings (Word2Vec, GloVe), and attention mechanisms (such as in Transformer models)
allow machines to process textual data more effectively. NLP is applied in chatbots, search
engines, recommendation systems, and sentiment analysis for customer reviews, providing
valuable insights from unstructured text data.
Source Code:
import numpy as np
import pandas as pd
import tensorflow as tf
from tensorflow.keras.datasets import imdb
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense
import spacy
Output:
Learning Outcomes:
Experiment No. 5
Problem Statement: Build RL agents and train them using OpenAI Gym or Stable Baselines
to solve challenging tasks.
Theory:
To build and train reinforcement learning (RL) agents in OpenAI Gym or Stable Baselines, we
can use various RL algorithms like Q-Learning, Policy Gradient Methods (such as
REINFORCE or A2C), or Deep Q-Networks (DQN), which allow agents to learn through
interactions with the environment. Here’s a brief theoretical outline and code example for
training an RL agent with Stable Baselines in the OpenAI Gym environment.
1. Agent-Environment Interaction:
o An agent interacts with an environment in discrete time steps, choosing actions
based on the current state.
o The environment responds by providing the next state and a reward.
o The agent's goal is to maximize cumulative rewards over time (expected return).
Source Code:
import gymnasium as gym
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv
from stable_baselines3.common.evaluation import evaluate_policy
from stable_baselines3.common.callbacks import EvalCallback
import numpy as np
import matplotlib.pyplot as plt
class RLTrainer:
def __init__(self, env_name="CartPole-v1", total_timesteps=50000):
"""
Initialize the RL trainer with specified environment and training
parameters.
Args:
env_name (str): Name of the Gymnasium environment
total_timesteps (int): Total timesteps for training
"""
self.env_name = env_name
self.total_timesteps = total_timesteps
# Create environment
self.env = gym.make(env_name)
self.vec_env = DummyVecEnv([lambda: gym.make(env_name)])
# Initialize model
self.model = PPO(
"MlpPolicy",
self.vec_env,
learning_rate=0.0003,
n_steps=2048,
batch_size=64,
n_epochs=10,
gamma=0.99,
gae_lambda=0.95,
clip_range=0.2,
verbose=1
def train(self):
"""Train the RL agent with evaluation callback"""
# Set up evaluation callback
eval_callback = EvalCallback(
self.eval_env,
best_model_save_path="./best_model",
log_path="./logs/",
eval_freq=1000,
deterministic=True,
render=False
)
Args:
n_eval_episodes (int): Number of evaluation episodes
Returns:
tuple: Mean reward and standard deviation
"""
mean_reward, std_reward = evaluate_policy(
self.model,
self.eval_env,
n_eval_episodes=n_eval_episodes,
deterministic=True
)
return mean_reward, std_reward
Args:
max_steps (int): Maximum steps per episode
Returns:
list: Episode rewards
"""
obs, _ = self.eval_env.reset()
done = False
rewards = []
step = 0
return rewards
Args:
rewards (list): List of rewards from an episode
"""
plt.figure(figsize=(10, 5))
plt.plot(rewards, label='Reward per step')
plt.title(f'Episode Rewards - {self.env_name}')
plt.xlabel('Step')
plt.ylabel('Reward')
plt.legend()
plt.grid(True)
plt.show()
# Example usage
def main():
# Initialize trainer
trainer = RLTrainer(env_name="CartPole-v1", total_timesteps=50000)
if __name__ == "__main__":
main()
Output:
Learning Outcomes:
Experiment No. 6
Theory:
Interpretable machine learning (IML) is essential for understanding how models make predictions,
especially in high-stakes domains like healthcare and finance. Techniques like LIME (Local
Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations) provide
insights into model behavior.
LIME focuses on local interpretability, generating interpretable models (typically linear) around
individual predictions to highlight feature contributions. It approximates the complex model's
behavior by perturbing input data and observing output changes.
Conversely, SHAP is grounded in cooperative game theory, offering a unified measure of feature
importance across individual predictions. It calculates the contribution of each feature to the
overall prediction using Shapley values, ensuring consistent and theoretically sound explanations.
Both methods empower stakeholders to trust and validate machine learning models by elucidating
how input features influence outputs, thus enhancing decision-making and accountability in model
deployment.
Source Code:
LIME:
import numpy as np
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from lime import lime_tabular
iris = load_iris()
X = pd.DataFrame(data=iris.data, columns=iris.feature_names)
y = iris.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
model = RandomForestClassifier()
model.fit(X_train, y_train)
explainer = lime_tabular.LimeTabularExplainer(X_train.values,
feature_names=X_train.columns, class_names=iris.target_names,
mode='classification')
idx = 0
exp = explainer.explain_instance(X_test.values[idx], model.predict_proba,
num_features=4)
exp.show_in_notebook(show_table=True)
SHAP:
import shap
import xgboost as xgb
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
from sklearn.metrics import accuracy_score
data = load_iris()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = data.target
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3,
random_state=42)
model = xgb.XGBClassifier(use_label_encoder=False, eval_metric='mlogloss')
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
print("Accuracy:", accuracy_score(y_test, y_pred))
explainer = shap.TreeExplainer(model)
shap_values = explainer.shap_values(X_test)
Output:
(LIME)
(SHAP)
Learning Outcomes:
Experiment No. 7
Problem Statement: Use AutoML and Hyperparameter tuning tools to automate the model
selection and optimization process.
Theory:
Using AutoML and hyperparameter tuning tools for reinforcement learning (RL) involves
automating the process of model selection and optimization to improve agent performance. Here’s
a concise theory:
1. Automated Algorithm Selection: AutoML can assist in selecting the most suitable RL
algorithm (e.g., PPO, DQN, SAC) based on the environment and task complexity. Each
algorithm has different strengths, and automating this choice can help identify the best-
suited one without manual trials.
2. Hyperparameter Optimization: Hyperparameters in RL, like learning rate, batch size,
and discount factor, can significantly impact performance. Libraries such as Optuna or
Ray Tune can be used to perform automated tuning by running trials that adjust these
parameters to maximize a predefined metric (e.g., average reward).
3. Search Space Definition: Define ranges or distributions for each hyperparameter,
allowing the tuning tool to explore various configurations. For example, learning rate may
be searched in a logarithmic scale, while network architecture parameters might vary by
layer size or count.
4. Optimization Methods: Tools like Optuna use techniques such as Bayesian
Optimization or Tree-structured Parzen Estimators (TPE) to efficiently explore the
search space and identify optimal configurations with fewer trials than brute-force grid or
random search.
5. Experiment Tracking and Early Stopping: Automated tools often include features to
track experiments and implement early stopping, ending unpromising trials early to save
computation time.
Source Code:
from tpot import TPOTClassifier
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
print(tpot.score(X_test, y_test))
tpot.export('best_model.py')
X, y = load_iris(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2,
random_state=42)
y_pred = automl.predict(X_test)
Learning Outcomes:
Experiment No. 8
Problem Statement: Analyze time series data, perform forecasting, and evaluate model
performance using Prophet or statsmodels .
Theory:
Analyzing and forecasting time series data using Prophet or statsmodels involves a systematic
approach to identify trends, seasonality, and make predictions. Here’s a concise theoretical
framework:
1. Data Preprocessing:
o Convert the data into a time series format, ensuring consistent intervals (e.g.,
daily, monthly).
o Handle any missing values, and create time-based features if needed.
2. Model Selection:
o Prophet: Developed by Facebook, Prophet is highly effective for time series data
with strong seasonal effects and robust to missing data or outliers. It decomposes
time series into trend, seasonality, and holiday effects, making it well-suited for
forecasting with complex seasonality.
o statsmodels (e.g., ARIMA): The ARIMA (Auto-Regressive Integrated Moving
Average) model in statsmodels is commonly used for time series with patterns of
auto-correlation and no strong seasonal component. SARIMA (Seasonal ARIMA)
extends ARIMA for seasonality.
3. Model Training and Forecasting:
o Prophet: Fits trend, seasonality, and holiday components using additive or
multiplicative models. Once trained, it forecasts future values and can visualize
each component separately.
o ARIMA/SARIMA: Uses past values and error terms to model the time series and
create future predictions. ARIMA orders (p, d, q) are chosen based on auto-
correlation and differencing to ensure stationarity.
4. Model Evaluation:
o Common metrics for evaluating forecast accuracy include Mean Absolute Error
(MAE), Mean Squared Error (MSE), or Mean Absolute Percentage Error
(MAPE).
o Cross-validation or splitting the data into training and test sets is recommended to
assess model accuracy and avoid overfitting.
5. Interpretation:
o Plot predictions against actual values and visualize model components (trend,
seasonality) to interpret the results and assess model fit.
Source Code:
!pip install prophet
import pandas as pd
from prophet import Prophet
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt
df = pd.DataFrame({
'date': pd.date_range(start='2020-01-01', periods=100),
'value': range(100)
})
train = df.iloc[:-20]
test = df.iloc[-20:]
model = Prophet()
model.fit(train)
future = model.make_future_dataframe(periods=20)
forecast = model.predict(future)
fig = model.plot(forecast)
plt.title('Prophet Forecast')
plt.show()
y_true = test['y'].values
y_pred = forecast['yhat'][-20:].values
mse = mean_squared_error(y_true, y_pred)
print(f'Mean Squared Error: {mse}')
Output:
Learning Outcomes:
Experiment No. 9
Problem Statement: Compress and quantize large ML models to make them suitable for
deployment on resource constrained devices.
Theory:
Deploying large machine learning (ML) models on resource-constrained devices, such as mobile
or edge devices, can be challenging due to limited memory, storage, and processing power.
Model compression and quantization are two widely used techniques to optimize models for
such environments.
1. Model Compression:
o Compression reduces the number of parameters in a model, which decreases
memory usage and inference latency.
o Pruning is a common compression technique that removes unnecessary weights
or neurons from the model, reducing the model size while minimally impacting
accuracy.
2. Quantization:
o Quantization reduces the precision of the model parameters, often from 32-bit
floating-point (FP32) to lower bit widths (e.g., 16-bit or 8-bit integers).
o By representing weights and activations with lower precision, the model requires
less memory and can execute faster on hardware that supports integer arithmetic.
Source Code:
import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.utils.prune as prune
import torch.quantization as quantization
import matplotlib.pyplot as plt
class SimpleModel(nn.Module):
def __init__(self):
super(SimpleModel, self).__init__()
self.fc1 = nn.Linear(784, 256)
self.fc2 = nn.Linear(256, 128)
self.fc3 = nn.Linear(128, 10)
model = SimpleModel()
model.eval()
model.qconfig = quantization.get_default_qconfig("fbgemm")
quantization.prepare(model, inplace=True)
model(dummy_input)
quantization.convert(model, inplace=True)
plot_weight_distribution(model.fc1.weight().dequantize(), "Quantized
Weights of fc1 Layer")
Output:
Learning Outcomes:
Experiment No. 10
Theory:
Federated Learning (FL) is a distributed approach that enables training machine learning models
directly on decentralized data sources (e.g., mobile devices or local servers) without sharing raw data.
This approach is privacy-preserving because it sends only model updates (e.g., gradients or model
parameters) rather than the data itself to a central server. The server aggregates these updates to
improve the global model iteratively.
Key Concepts in Federated Learning
1. Client-Server Architecture:
o Clients: Devices that hold local data. Each client trains a local model on its data and
sends updates to the server.
o Server: Receives model updates from clients, aggregates them, and sends the
improved global model back to clients.
2. Federated Averaging (FedAvg):
o The server aggregates model updates using a weighted average, which considers the
number of samples each client has. This method is efficient and commonly used in
FL.
3. Privacy and Security:
o Techniques like differential privacy and secure aggregation are often used to
ensure privacy by adding noise to the updates or encrypting them during transmission.
Source Code:
import collections
import tensorflow as tf
import tensorflow_federated as tff
def client_data(n):
return source.create_tf_dataset_for_client(source.client_ids[n]).map(
state = trainer.initialize()
Output:
Learning Outcomes: