CCS355 NNDL MCE/CSE

CCS355 NEURAL NETWORKS AND DEEP LEARNING LTPC

2 02 3

LIST OF LAB EXPERIMENTS:

1. Implement simple vector addition in TensorFlow.
2. Implement a regression model in Keras.
3. Implement a perceptron in TensorFlow/Keras Environment.
4. Implement a Feed-Forward Network in TensorFlow/Keras.
5. Implement an Image Classifier using CNN in TensorFlow/Keras.
6. Improve the Deep learning model by fine tuning hyper parameters.
7. Implement a Transfer Learning concept in Image Classification.
8. Using a pre trained model on Keras for Transfer Learning
9. Perform Sentiment Analysis using RNN
10. Implement an LSTM based Autoencoder in TensorFlow/Keras.
11. Image generation using GAN
Additional Experiments:
12. Train a Deep learning model to classify a given image using pre trained model
13. Recommendation system from sales data using Deep Learning
14. Implement Object Detection using CNN
15. Implement any simple Reinforcement Algorithm for an NLP problem
CCS355 NNDL MCE/CSE

1. IMPLEMENT SIMPLE VECTOR ADDITION IN TENSORFLOW.

AIM:
To implement simple vector addition using TensorFlow.
ALGORITHM:
Step 1: Get values from the user for vector 1
Step 2: Get values from the user for vector 2.
Step 3: Convert the user input into lists of floating-point numbers using float().
Step 4: Create TensorFlow constants for the input vectors using tf.constant.
Step 5: Use the tf.add operation to perform vector addition of the two input
vectors.
Step 6: Print the original vectors i.e., vector 1 and vector 2.
Step 7: Print the resultant vector after the addition.

CODE:
# Import necessary packages
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import tensorflow as tf
# Get user input for vector 1
in1 = input("Enter space-separated values for vector 1: ")
vector1_values = [float(x) for x in in1.split()]
# Get user input for vector 2
in2 = input("Enter space-separated values for vector 2: ")
vector2_values = [float(x) for x in in2.split()]
# Define TensorFlow constants for the input vectors
vector1 = tf.constant(vector1_values, dtype=tf.float32)
vector2 = tf.constant(vector2_values, dtype=tf.float32)
# Perform vector addition
result_vector = tf.add(vector1, vector2)

# Print the original vectors and the result

print("Vector 1:", vector1.numpy())
print("Vector 2:", vector2.numpy())
CCS355 NNDL MCE/CSE

print("Resultant Vector:", result_vector.numpy())

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

2. IMPLEMENT A REGRESSION MODEL IN KERAS.

AIM:
To implement a regression model in keras.
ALGORITHM:
Step 1: Import all necessary packages.
Step 2: Load the dataset from either packages or generate user data.
Step 3: Split the dataset into features(x) and target(y).
Step 4: Again split the dataset into training data(train_x,train_y) and testing
data(test_x,test_y) using traintestsplit().
Step 5: Define the model by creating sequential model with Dense layer and
linear activation function for regression.
Step 6: Compile the model with optimizer, loss function, and metrices.
Step 7: Train the model with training dataset using fit().
Step 8: Evaluate the model with test dataset using evaluate method.
Step 9: By using the trained model make predictions on new data or compare
prediction with ground truth.
CODE:
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import numpy as np
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

# Generate some sample data for regression

X = np.random.rand(100, 1) # Input feature
y = 2 * X + 1 + 0.1 * np.random.rand(100, 1)
# Simulated linear relationship with noise

# Define the model architecture

model = keras.Sequential([
layers.Input(shape=(1,)), # Input layer
layers.Dense(1) # Output layer with 1 unit for regression
])
# Compile the model
model.compile(optimizer='adam', loss='mean_squared_error')
# Train the model
model.fit(X, y, epochs=300, verbose=1) # You can adjust the number of epochs
# Evaluate the model (optional)
loss = model.evaluate(X, y)
print("Mean Squared Error:", loss)
CCS355 NNDL MCE/CSE

# Make predictions
new_data = np.array([[0.5], [0.8], [1.0], [2.1]]) # New data for prediction
predictions = model.predict(new_data)
print("Predictions:", predictions)

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

3. IMPLEMENT A PERCEPTRON IN TENSORFLOW/KERAS

ENVIRONMENT.

AIM:
To implement a perceptron in tensorflow/keras environment.
ALGORITHM:
Step 1: Import all necessary packages.
Step 2: Load the dataset from either packages or generate user data.
Step 3: Split the dataset into features(x) and target(y).
Step 4: Again split the dataset into training data(train_x,train_y) and testing
data(test_x,test_y) using traintestsplit().
Step 5: Define the model by creating sequential model with Dense layer and
activation function for perceptron.
Step 6: Compile the model with optimizer, loss function, and metrices.
Step 7: Train the model with training dataset using fit().
Step 8: Evaluate the model with test dataset using evaluate method.
Step 9: By using the trained model make predictions on new data or compare
prediction with ground truth.

CODE:
import os
from keras.activations import sigmoid
from keras.losses import mean_squared_error
from keras.saving.saved_model.load import metrics

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'

import keras
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Dense
from tensorflow.keras.optimizers import SGD
import numpy as np

# Generate some sample data for a logical OR operation

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]) # Input features
y = np.array([0, 1, 1, 1]) # Output labels (OR gate)

# Define a simple perceptron model

model = keras.Sequential([
Dense(units=1, input_dim=2, activation= sigmoid)
])
CCS355 NNDL MCE/CSE

# Compile the model

model.compile(optimizer=SGD(learning_rate=0.1), loss= mean_squared_error,metrics=
[metrics.categorical_accuracy])
# Train the model
model.fit(X, y, epochs=1000, verbose=0) # You can adjust the number of epochs
# Evaluate the model
loss, accuracy = model.evaluate(X, y)
print("Loss: ", loss)
print("Accuracy:", accuracy)
# Make predictions
predictions = model.predict(X)
print( "Predictions:")
print(predictions)

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

4. IMPLEMENT A FEED-FORWARD NETWORK IN TENSORFLOW/KERAS.

AIM:
To implement a feed-forward network in tensorflow/ keras.
ALGORITHM:
Step 1: Import all necessary packages.
Step 2: Load the dataset from either packages or generate user data.
Step 3: Split the dataset into features(x) and target(y).
Step 4: Again split the dataset into training data(train_x,train_y) and testing
data(test_x,test_y) using traintestsplit().
Step 5: Define the model by creating sequential model with Dense layer and
activation function for feed forward network.
- Input Layer: Set the values of the input layer to the values of the input data.
- Hidden Layers: For each hidden layer, compute the weighted sum of inputs
and apply the activation function to produce the output of the layer.
- Output Layer: Compute the weighted sum of inputs for the output layer and
apply the activation function.
Step 6: Compile the model with optimizer, loss function, and metrices.
Step 7: Train the model with training dataset using fit().
Step 8: Evaluate the model with test dataset using evaluate method.
Step 9: By using the trained model make predictions on new data or compare
prediction with ground truth.

CODE:
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Dense
import numpy as np

# Generate some sample data

X = np.array([[0, 0], [0, 1], [1, 0], [1, 1]]) # Input features
y = np.array([0, 1, 1, 0]) # Output labels (XOR gate)

# Define a feedforward neural network model

model = keras.Sequential([
Dense(units=4, input_dim=2, activation='relu'), # 2 input features, 4 hidden units with
ReLU activation
Dense(units=1, activation='sigmoid') # 1 output unit with sigmoid activation
])
model.history
CCS355 NNDL MCE/CSE

# Compile the model

model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
# Train the model
model.fit(X, y, epochs=100, verbose=1) # You can adjust the number of epochs
# Evaluate the model
loss, accuracy = model.evaluate(X, y)
print("Loss:", loss)
print("Accuracy:", accuracy)
# Make predictions
predictions = model.predict(X)
print("Predictions:")
print(predictions)

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

5. IMPLEMENT AN IMAGE CLASSIFIER USING CNN IN

TENSORFLOW/KERAS.
AIM:
To implement an image classifier using cnn in tensorflow/keras.
ALGORITHM:
Step 1: Import all necessary packages.
Step 2: Load the dataset from either packages or generate user data.
Step 3: Split the dataset into features(x) and target(y).
Step 4: Again split the dataset into training data(train_x,train_y) and testing
data(test_x,test_y) using traintestsplit().
Step 5: Define the model by creating sequential model with convolution layer
Conv2D , Pooling layer and flatten layer and activation function.
Step 6: Compile the model with optimizer, loss function, and metrices.
Step 7: Train the model with training dataset using fit().
Step 8: Evaluate the model with test dataset using evaluate method.
Step 9: By using the trained model make predictions on new data or compare
prediction with ground truth.
CODE:
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE"

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.datasets import mnist
from tensorflow.keras.utils import to_categorical
import matplotlib.pyplot as plt
# Load and preprocess the MNIST dataset
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = train_images.reshape((60000, 28, 28, 1)).astype('float32') / 255
test_images = test_images.reshape((10000, 28, 28, 1)).astype('float32') / 255
train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels)
# Build the CNN model
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.Flatten())
model.add(layers.Dense(64, activation='relu'))
model.add(layers.Dense(10, activation='softmax'))
CCS355 NNDL MCE/CSE

# Compile the model

model.compile(optimizer='adam',
loss='categorical_crossentropy'
, metrics=['accuracy'])
# Train the CNN model
history = model.fit(train_images, train_labels, epochs=5, batch_size=64,
validation_data=(test_images, test_labels))
# Evaluate the model on the test set
test_loss, test_acc = model.evaluate(test_images, test_labels)
print(f'Test accuracy: {test_acc}')
# Plot training history
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()
plt.show()

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

6. IMPROVE THE DEEP LEARNING MODEL BY FINE TUNING HYPER

PARAMETERS.
AIM:
To improve the deep learning model by fine tuning hyper parameters.
ALGORITHM:
Step 1: Import all necessary packages.
Step 2: Load the dataset from either packages or generate user data.

Step 3: Define Hyperparameters: Identify the hyperparameters that you want

to tune. These may include learning rate, batch size, number of layers, number
of neurons per layer, activation functions, dropout rate, etc.
Step 4: Choose Evaluation Metric: Select an evaluation metric that reflects
the performance of your model. This could be accuracy, precision, recall, F1-
score, or any other relevant metric depending on your specific task.
Step 5: Choose Search Space: Define the range or set of values for each
hyperparameter that you want to explore. This could be a discrete set of values
or a continuous range.
Step 6: Choose Search Method: Decide on a search method to explore the
hyperparameter space. Common methods include:
i. Grid Search: Exhaustively search all combinations of
hyperparameters within the specified ranges.
ii. Random Search: Randomly sample combinations of hyperparameters
from the search space.
iii. Bayesian Optimization: Use probabilistic models to guide the search
process based on past evaluations.
iv. Evolutionary Algorithms: Apply evolutionary algorithms such as
genetic algorithms to optimize hyperparameters.
Step 7: Split Data: Split your dataset into training, validation, and test sets.
The validation set is used to evaluate the performance of different
hyperparameter configurations during the search process.
Step 8: Define Cross-Validation Strategy (Optional): If your dataset is
small, consider using cross-validation to evaluate the performance of different
hyperparameter configurations more robustly.
Step 9: Hyperparameter Optimization Loop:
i. Iterate through different hyperparameter configurations based on the
chosen search method.
ii. For each configuration:
i. Train the model on the training set using the specified
hyperparameters.
ii. Evaluate the model on the validation set using the chosen
evaluation metric.
iii. Keep track of the performance metric for each configuration.
Step 10: Select Best Configuration: Choose the hyperparameter configuration
that yields the best performance on the validation set.
Step 11: Compile the model with optimizer, loss function, and metrices.
Step 12: Train the model with training dataset using fit().
CCS355 NNDL MCE/CSE

Step 13: Evaluate the model with test dataset using evaluate method.By using
the trained model make predictions on new data or compare prediction with
ground truth
CODE:
import os
os.environ["KMP_DUPLICATE_LIB_OK"]="TRUE"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.datasets import mnist
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.callbacks import EarlyStopping, ModelCheckpoint
import matplotlib.pyplot as plt
# Load and preprocess the MNIST dataset
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = train_images.reshape((60000, 28, 28, 1)).astype('float32') / 255
test_images = test_images.reshape((10000, 28, 28, 1)).astype('float32') / 255
train_labels = to_categorical(train_labels)
test_labels = to_categorical(test_labels)
# Build the CNN model with hyperparameter tuning
model = models.Sequential()
model.add(layers.Conv2D(32, (3, 3), activation='relu', input_shape=(28, 28, 1)))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(64, (3, 3), activation='relu'))
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Conv2D(128, (3, 3), activation='relu')) # Increased filters
model.add(layers.MaxPooling2D((2, 2)))
model.add(layers.Flatten())
model.add(layers.Dense(128, activation='relu')) # Increased units
model.add(layers.Dense(10, activation='softmax'))
# Compile the model with fine-tuned hyperparameters

optimizer = Adam(learning_rate=0.001) # Adjusted learning rate

model.compile(optimizer=optimizer,
loss='categorical_crossentropy'
, metrics=['accuracy'])
# Define callbacks for early stopping and model checkpoint
early_stopping = EarlyStopping(monitor='val_loss', patience=3,
restore_best_weights=True)
model_checkpoint = ModelCheckpoint('best_model.h5', save_best_only=True)
# Train the CNN model with fine-tuned hyperparameters
history = model.fit(train_images, train_labels, epochs=20, batch_size=64,
validation_data=(test_images, test_labels),
callbacks=[early_stopping, model_checkpoint])
CCS355 NNDL MCE/CSE

# Load the best model from the checkpoint

best_model = models.load_model('best_model.h5')
# Evaluate the best model on the test set
test_loss, test_acc = best_model.evaluate(test_images, test_labels)
print(f'Test accuracy of the best model: {test_acc}')
# Plot training history
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.legend()
plt.show()

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

7. IMPLEMENT A TRANSFER LEARNING CONCEPT IN IMAGE

CLASSIFICATION.
AIM:
To implement a transfer learning concept in image classification.
ALGORITHM:
Step 1: Import all necessary packages.
Step 2: Load the dataset from either packages or generate user data.

Step 3: Select a Pre-trained Model: Choose a pre-trained convolutional

neural network (CNN) model that has been trained on a large dataset like
ImageNet. Common choices include VGG, ResNet, Inception, or MobileNet.
Step 4: Load Pre-trained Model: Load the pre-trained model weights and
architecture using a deep learning framework like TensorFlow or PyTorch.
Step 5: Modify the Model: Modify the pre-trained model to suit the new
classification task. Typically, this involves removing the original output
layer(s) and adding new layers to adapt the model to the number of classes in
your dataset.
Step 6: Freeze Pre-trained Layers (Optional): Optionally, freeze some or all
of the pre-trained layers to prevent their weights from being updated during
training. This is particularly useful when you have a small dataset to prevent
overfitting.
Step 7: Data Preprocessing: Preprocess your image data. This may involve
resizing images to match the input size expected by the pre-trained model,
normalizing pixel values, and performing data augmentation techniques like
rotation, flipping, and scaling to increase the diversity of your training data.
Step 8: Prepare Training and Validation Data: Split your dataset into
training and validation sets. Ensure that they are balanced and representative
of the overall distribution of classes in your dataset.
Step 9: Train the model with training dataset using fit().
Step 10: Evaluate the model with test dataset using evaluate method.By using
the trained model make predictions on new data or compare prediction with
ground truth

CODE:
import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.applications import VGG16
from tensorflow.keras.utils import to_categorical
# Load CIFAR-10 dataset and normalize pixel values
CCS355 NNDL MCE/CSE

(x_train, y_train), (x_test, y_test) = cifar10.load_data()

x_train, x_test = x_train.astype('float32') / 255.0, x_test.astype('float32') /255.0
y_train, y_test = to_categorical(y_train, 10), to_categorical(y_test, 10)
# Load pre-trained VGG16 model without the top classification layer and freezeconvolutional
base
base_model = VGG16(weights='imagenet', include_top=False,
input_shape=(32, 32,
3))
base_model.trainable = False
# Define model architecture
model = models.Sequential([
base_model,
layers.Flatten(),
layers.Dense(256, activation='relu'),
layers.Dropout(0.5),
layers.Dense(10, activation='softmax')
])
# Compile, train, and evaluate the model
model.compile(optimizer='adam', loss='categorical_crossentropy',
metrics=['accuracy'])
model.summary()
history = model.fit(x_train, y_train, epochs=10, batch_size=64,
validation_data=(x_test, y_test))
test_loss, test_acc = model.evaluate(x_test, y_test)
print('Test accuracy:', test_acc)

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE
CCS355 NNDL MCE/CSE

8. USING A PRE-TRAINED MODEL ON KERAS FOR TRANSFER LEARNING.

AIM:
To implement a Transfer learning model on keras using pre trained model.
ALGORITHM:

Step 1: Select a Pre-trained Model: Choose a pre-trained model from Keras

applications such as VGG16, ResNet50, InceptionV3, MobileNet, etc.
Step 2: Load the Pre-trained Model: Load the pre-trained model without
including the fully connected (top) layers. Set include_top=False when
loading the model.
Step 3: Add Custom Top Layers: Add custom top layers to the pre-trained
model. This typically involves flattening or pooling the output of the pre-
trained model and then adding one or more dense layers for classification. The
number of dense layers and units should be chosen based on your specific
classification task.
Step 4: Freeze Pre-trained Layers (Optional): Optionally, freeze some or all
of the layers in the pre-trained model to prevent their weights from being
updated during training. This can help prevent overfitting, especially when
dealing with a small dataset.
Step 5: Compile the Model: Compile the model using an appropriate loss
function, optimizer, and evaluation metric. For classification tasks, categorical
cross-entropy is commonly used as the loss function, and Adam or SGD is
commonly used as the optimizer.
Step 6: Data Preprocessing: Preprocess your image data. This may involve
resizing images to match the input size expected by the pre-trained model,
normalizing pixel values, and performing data augmentation techniques like
rotation, flipping, and scaling to increase the diversity of your training data.
Step 7: Prepare Training and Validation Data: Split your dataset into
training and validation sets. Ensure that they are balanced and representative
of the overall distribution of classes in your dataset.
Step 8: Train the Model: Train the model using the fit() function. Specify the
training data, validation data, batch size, number of epochs, and any other
relevant parameters.
Step 9: Fine-tuning : Optionally, unfreeze some or all of the pre-trained layers
and continue training the model using a lower learning rate to fine-tune the
model's parameters further. This can help improve performance, especially if
your dataset is similar to the original dataset the pre-trained model was trained
on.
Step 10: Evaluate the Model: Evaluate the model's performance on the
validation set using the evaluate() function. Compute metrics such as
accuracy, precision, recall, and F1-score to assess the model's performance.
Step 11: Adjust Hyperparameters: Fine-tune hyperparameters such as
learning rate, batch size, and architecture if necessary to improve model
performance.
Step 12: Testing: Optionally, evaluate the trained model on a separate test set to
assess its generalization ability.
CCS355 NNDL MCE/CSE

CODE:
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.applications import VGG16
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras import layers, models, optimizers
# Load the CIFAR-10 dataset
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
# Normalize pixel values to be between 0 and 1
x_train, x_test = x_train / 255.0, x_test / 255.0
# Load the pretrained VGG16 model without the top (fully connected) layers
base_model = VGG16(weights='imagenet', include_top=False,input_shape=(32, 32,3))
# Freeze the weights of the pretrained layers
for layer in base_model.layers:
layer.trainable = False
# Create your own model by adding custom top layers
model = models.Sequential()
model.add(base_model)
model.add(layers.Flatten())
model.add(layers.Dense(256, activation='relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(10, activation='softmax')) # CIFAR-10 has 10 classes
# Compile the model
model.compile(optimizer=optimizers.Adam(lr=1e-4),
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
# Display the model architecture
model.summary()
# Set up data generator with validation split
datagen = ImageDataGenerator(validation_split=0.2) # 20% of data for validation
batch_size = 32
# Use 'flow' method for image data augmentation
train_generator = datagen.flow(x_train, y_train, batch_size=batch_size,
subset='training')
validation_generator = datagen.flow(x_train, y_train, batch_size=batch_size,
subset='validation')
# Train the model
epochs = 10 # Adjust the number of epochs based on your needs
history = model.fit(train_generator,
steps_per_epoch=len(train_generator),
epochs=epochs,
validation_data=validation_generator,
validation_steps=len(validation_generator))
# Evaluate the model on the test set
test_loss, test_acc = model.evaluate(x_test, y_test)
print(f'Test accuracy: {test_acc * 100:.2f}%')
CCS355 NNDL MCE/CSE

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

9. PERFORM SENTIMENT ANALYSIS USING RNN.

AIM:
To perform sentiment analysis using RNN.
ALGORITHM:

Step 1: Data Preparation:

a. Obtain a dataset containing text samples labeled with sentiment classes
(e.g., positive, negative, neutral).
b. Preprocess the text data by tokenizing it into words or subwords,
removing punctuation, and converting words to lowercase.
c. Map each word to an integer index and pad/truncate sequences to
ensure uniform length.
Step 2: Define the RNN Model:
a. Choose an RNN architecture suitable for the task, such as LSTM
(Long Short-Term Memory) or GRU (Gated Recurrent Unit).
b. Initialize the RNN model using a framework like TensorFlow or
PyTorch.
c. Define the input layer, embedding layer (to convert integer indices to
dense vectors), RNN layer(s), and output layer (e.g., a dense layer with
softmax activation for multi-class sentiment classification).
Step 3: Compile the Model:
a. Specify the loss function (e.g., categorical cross-entropy for multi-class
classification), optimizer (e.g., Adam or RMSprop), and evaluation
metric (e.g., accuracy).
Step 4: Training:
a. Split the dataset into training, validation, and optionally, test sets.
b. Train the RNN model using the training data.
c. Monitor training/validation loss and metrics to detect overfitting or
convergence issues.
d. Optionally, use techniques like dropout or early stopping to improve
generalization and prevent overfitting.
Step 5: Evaluation:
a. Evaluate the trained model on the validation set to assess its
performance using metrics such as accuracy, precision, recall, and F1-
score.
b. Tune hyperparameters or adjust the model architecture based on
validation performance if necessary.
Step 6: Testing:
a. Once satisfied with the model's performance on the validation set,
evaluate it on the test set to assess its generalization ability.

CODE:
import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
CCS355 NNDL MCE/CSE

import numpy as np
import tensorflow as tf
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences

# Sample dataset for demonstration

sentences = [
'I love this movie!',
'This movie is fantastic.',
'Great film, highly recommended.',
'Awful movie, I hated it.',
'The acting was terrible.'
]
labels = np.array([1, 1, 1, 0, 0]) # 1: Positive, 0: Negative

# Hyperparameters
vocab_size = 1000
embedding_dim = 16
max_length = 10
trunc_type = 'post'
padding_type = 'post'
oov_token = '<OOV>'
epochs = 10

# Tokenize the sentences

tokenizer = Tokenizer(num_words=vocab_size, oov_token=oov_token)
tokenizer.fit_on_texts(sentences)
word_index = tokenizer.word_index

# Convert text to sequences

sequences = tokenizer.texts_to_sequences(sentences)
padded_sequences = pad_sequences(sequences, maxlen=max_length, padding=padding_type,
truncating=trunc_type)

# Define the LSTM model

model = tf.keras.Sequential([
tf.keras.layers.Embedding(vocab_size, embedding_dim, input_length=max_length),
tf.keras.layers.LSTM(64), # LSTM layer with 64 units
tf.keras.layers.Dense(1, activation='sigmoid')
])

# Compile the model

CCS355 NNDL MCE/CSE

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# Train the model

model.fit(padded_sequences, labels, epochs=epochs)

# Example usage: Predict sentiment of new sentences

test_sentences = [
'I enjoyed watching this film.',
'It was a waste of time.'
]
test_sequences = tokenizer.texts_to_sequences(test_sentences)
padded_test_sequences = pad_sequences(test_sequences, maxlen=max_length,
padding=padding_type, truncating=trunc_type)
predictions = model.predict(padded_test_sequences)

# Convert probabilities to sentiments

for i, pred in enumerate(predictions):
sentiment = "Positive" if pred > 0.5 else "Negative"
print(f"Sentence: {test_sentences[i]} - Sentiment: {sentiment} - Probability: {pred[0]}")

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

10. IMPLEMENT AN LSTM BASED AUTOENCODER IN

TENSORFLOW/KERAS.
AIM:
To implement an lstm based autoencoder in tensorflow/keras
ALGORITHM:

Step 1: Data Preparation:

a. Obtain a dataset containing sequences of data (e.g., time series, text
data, etc.) for training the autoencoder.
b. Preprocess the data as needed (e.g., normalization, scaling).
Step 2: Define the Encoder:
a. Create an LSTM layer or stack of LSTM layers to encode the input
sequence into a compressed representation (latent space).
Step 3: Define the Decoder:
a. Create an LSTM layer or stack of LSTM layers to decode the
compressed representation back into the original input sequence.
Step 4: Build the Autoencoder:
a. Combine the encoder and decoder layers into a single model.
b. The input and output of the model should be the same sequence length.
Step 5: Compile the Autoencoder:
a. Specify an appropriate loss function (e.g., mean squared error) to
measure the reconstruction error between the input and output
sequences.
b. Choose an optimizer (e.g., Adam) to minimize the loss function during
training.
Step 6: Train the Autoencoder:
a. Train the autoencoder model using the input sequences as both the
input and target output.
b. Split the dataset into training and validation sets.
c. Monitor the training process and validate the reconstruction
performance on the validation set.
Step 7: Evaluate the Autoencoder:
a. Evaluate the performance of the trained autoencoder on unseen data.
b. Measure the reconstruction error and visualize the reconstructed
sequences to assess the quality of the reconstruction.

CODE:
import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Input, LSTM, RepeatVector, TimeDistributed
from tensorflow.keras.models import Model

# Sample dataset for demonstration (sequence data)

# Replace this with your own dataset
sequences = np.random.randn(1000, 10, 1) # 1000 sequences of length 10 with 1 feature
CCS355 NNDL MCE/CSE

# Hyperparameters
latent_dim = 2 # Dimensionality of the latent space

# Define the autoencoder architecture

input_sequence = Input(shape=(10, 1)) # Input shape: (sequence_length, num_features)

# Encoder
encoder = LSTM(latent_dim, activation='relu', return_sequences=False)(input_sequence)

# Repeat the encoded representation for each time step in the input sequence
encoded_sequence = RepeatVector(10)(encoder)

# Decoder
decoder = LSTM(1, activation='relu', return_sequences=True)(encoded_sequence)

# Define the autoencoder model

autoencoder = Model(input_sequence, decoder)

# Compile the model

autoencoder.compile(optimizer='adam', loss='mse')

# Print model summary

autoencoder.summary()

# Train the autoencoder

autoencoder.fit(sequences, sequences, epochs=10, batch_size=32, validation_split=0.2)

# Evaluate the autoencoder (optional)

# reconstruction_error = autoencoder.evaluate(sequences, sequences)

# Use the trained autoencoder for inference

# For example, encode and decode a sample sequence
sample_sequence = sequences[0:1] # Take the first sequence as an example
reconstructed_sequence = autoencoder.predict(sample_sequence)

# Print the original and reconstructed sequences

print("Original Sequence:")
print(sample_sequence.squeeze())
print("\nReconstructed Sequence:")
print(reconstructed_sequence.squeeze())
CCS355 NNDL MCE/CSE

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

11. IMAGE GENERATION USING GAN.

AIM:
To perform image generation using GAN
ALGORITHM:

Step 1: Dataset Preparation:

a. Collect a dataset of images that you want your GAN to learn from.
This dataset should contain a wide variety of images that represent the
distribution of the images you want to generate.
Step 2: GAN Architecture Selection:
a. Choose the architecture for your GAN. Common architectures include
Deep Convolutional GANs (DCGAN), Wasserstein GAN (WGAN), or
Progressive GAN (PGAN).
Step 3: Generator Network:
a. Define the generator network. This network takes random noise as
input and generates images. It typically consists of convolutional layers
followed by upsampling layers to generate high-resolution images
from low-resolution noise.
Step 4: Discriminator Network:
a. Define the discriminator network. This network takes an image as
input and predicts whether it is real (from the dataset) or fake
(generated by the generator). It typically consists of convolutional
layers followed by downsampling layers to make classification
decisions.
Step 5: Training Process:
a. Initialize the parameters of both the generator and discriminator
networks.
b. Alternately train the generator and discriminator networks:
i. Discriminator Training:
1. Sample a batch of real images from the dataset.
2. Generate a batch of fake images using the current
generator.
3. Train the discriminator to correctly classify real and
fake images by optimizing its parameters using a binary
cross-entropy loss function.
ii. Generator Training:
1. Generate a batch of fake images using the current
generator.
2. Train the generator to fool the discriminator by
optimizing its parameters to minimize the
discriminator's ability to distinguish between real and
fake images.
c. Repeat the training process for a predefined number of iterations or
until convergence criteria are met.
Step 6: Generating Images:
CCS355 NNDL MCE/CSE

a. After training, you can generate new images by sampling random noise
vectors from a normal distribution and passing them through the
trained generator network.
Step 7: Evaluation:
a. Evaluate the generated images using metrics like Inception Score,
Frechet Inception Distance, or visual inspection by humans to assess
the quality and diversity of the generated images.
Step 8:
CODE:
import tensorflow as tf
from tensorflow.keras import layers, models, optimizers
import numpy as np
import matplotlib.pyplot as plt

# Define the Generator model

def build_generator(input_shape):
model = models.Sequential([
layers.Dense(128, input_shape=input_shape),
layers.LeakyReLU(alpha=0.01),
layers.Dense(784, activation='tanh'),
layers.Reshape((28, 28, 1))
])
return model

# Define the Discriminator model

def build_discriminator(input_shape):
model = models.Sequential([
layers.Flatten(input_shape=input_shape),
layers.Dense(128),
layers.LeakyReLU(alpha=0.01),
layers.Dense(1, activation='sigmoid')
])
return model

# Define the GAN model

def build_gan(generator, discriminator):
discriminator.trainable = False
model = models.Sequential([
generator,
discriminator
])
return model

# Load MNIST dataset

(train_images, _), (_, _) = tf.keras.datasets.mnist.load_data()
train_images = train_images.reshape(train_images.shape[0], 28, 28, 1).astype('float32')
CCS355 NNDL MCE/CSE

train_images = (train_images - 127.5) / 127.5 # Normalize the images to [-1, 1]

# Set up models
input_shape = (100,)
generator = build_generator(input_shape)
discriminator = build_discriminator((28, 28, 1))
gan = build_gan(generator, discriminator)

# Compile models
discriminator.compile(loss='binary_crossentropy',
optimizer=optimizers.Adam(learning_rate=0.0002), metrics=['accuracy'])
gan.compile(loss='binary_crossentropy', optimizer=optimizers.Adam(learning_rate=0.0002))

# Training loop
epochs = 100
batch_size = 64
num_batches = train_images.shape[0] // batch_size

for epoch in range(epochs):

for batch in range(num_batches):
# Train Discriminator
noise = np.random.normal(0, 1, (batch_size, 100))
fake_images = generator.predict(noise)
real_images = train_images[np.random.randint(0, train_images.shape[0], batch_size)]
X = np.concatenate((real_images, fake_images))
y = np.concatenate((np.ones((batch_size, 1)), np.zeros((batch_size, 1))))
d_loss = discriminator.train_on_batch(X, y)

# Train Generator
noise = np.random.normal(0, 1, (batch_size, 100))
y = np.ones((batch_size, 1))
g_loss = gan.train_on_batch(noise, y)

print(f'Epoch: {epoch+1}, Discriminator Loss: {d_loss[0]}, Generator Loss: {g_loss}')

# Generate images
noise = np.random.normal(0, 1, (10, 100))
generated_images = generator.predict(noise)

# Display generated images

plt.figure(figsize=(10, 1))
for i in range(10):
plt.subplot(1, 10, i+1)
plt.imshow(generated_images[i, :, :, 0], cmap='gray')
plt.axis('off')
plt.show()
CCS355 NNDL MCE/CSE

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

12. TRAIN A DEEP LEARNING MODEL TO CLASSIFY A GIVEN IMAGE

USING PRE-TRAINED MODEL
AIM:
To train a deep learning model to classify a given image using pre-trained model
ALGORITHM:

Step 1: Data Preparation:

a. Collect and preprocess your dataset. This involves:
i. Organizing your data into training, validation, and possibly
testing sets.
ii. Preprocessing images (e.g., resizing, normalization) to fit the
input requirements of the pre-trained model.
Step 2: Select a Pre-trained Model:
a. Choose a pre-trained model that is suitable for your task. Common
choices include models like VGG, ResNet, Inception, or MobileNet,
which are trained on large-scale datasets like ImageNet.
Step 3: Model Adaptation:
a. Remove the top classification layer (or layers) of the pre-trained
model, as these are specific to the original task it was trained on (e.g.,
ImageNet classification).
b. Add a new classification layer appropriate for your specific task. This
new classification layer should have the number of output neurons
equal to the number of classes in your dataset.
Step 4: Freeze Pre-trained Layers (Optional):
a. Optionally, you may choose to freeze some or all of the pre-trained
layers to prevent them from being updated during training. This can be
beneficial if you have a small dataset or want to prevent overfitting.
Step 5: Training:
a. Compile the model with an appropriate loss function, optimizer, and
evaluation metrics.
b. Train the model on your dataset using the fit() method, typically with
data augmentation techniques to increase the effective size of your
training dataset and improve generalization.
c. Monitor the training process using metrics such as accuracy and loss
on the training and validation sets.
Step 6: Fine-tuning (Optional):
a. Optionally, you may unfreeze some of the pre-trained layers and
continue training the model with a lower learning rate. Fine-tuning
allows the model to adapt to the specific characteristics of your dataset.
Step 7: Evaluation:
a. Evaluate the trained model on a separate test dataset to assess its
performance on unseen data.
b. Calculate metrics such as accuracy, precision, recall, and F1-score to
measure the model's performance.
CCS355 NNDL MCE/CSE

CODE:
import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras.models import Model
from tensorflow.keras.layers import GlobalAveragePooling2D, Dense, Input
from tensorflow.keras.datasets import cifar10
from tensorflow.keras.utils import to_categorical
# Load and preprocess the CIFAR-10 dataset
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0 # Normalize pixel values to the range [0, 1]
y_train = to_categorical(y_train, 10)
y_test = to_categorical(y_test, 10)
# Load a pre-trained model (MobileNetV2) excluding the top classification layers

base_model = MobileNetV2(weights='imagenet', include_top=False, input_shape=(32, 32,

3))
# Add custom classification layers on top
x = base_model.output
x = GlobalAveragePooling2D()(x)
x = Dense(256, activation='relu')(x)
predictions = Dense(10, activation='softmax')(x) # 10 classes in CIFAR-10
model = Model(inputs=base_model.input, outputs=predictions)
# Freeze the layers of the pre-trained model
for layer in base_model.layers:
layer.trainable = False
# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# Train the model
model.fit(x_train, y_train, epochs=10, validation_data=(x_test, y_test))
# Evaluate the model
loss, accuracy = model.evaluate(x_test, y_test)
print("Test Loss:", loss)
print("Test Accuracy:", accuracy)
CCS355 NNDL MCE/CSE

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

13. RECOMMENDATION SYSTEM FROM SALES DATA USING DEEP

LEARNING
AIM:
To implement recommendation system from sales data using deep learning
ALGORITHM:

Step 1: Data Collection and Preprocessing:

a. Collect sales data containing information about customers, products,
and their interactions (e.g., purchases, views, ratings).
b. Preprocess the data to ensure it's clean and structured. This may
involve handling missing values, encoding categorical variables, and
normalizing numerical features.
Step 2: Feature Engineering:
a. Extract relevant features from the sales data that can be used to build
the recommendation model. This may include customer demographics,
product attributes, purchase history, and user behavior.
Step 3: Data Splitting:
a. Split the data into training, validation, and test sets. The training set
will be used to train the recommendation model, the validation set to
tune hyperparameters, and the test set to evaluate the model's
performance.
Step 4: Model Selection:
a. Choose a deep learning architecture suitable for recommendation tasks.
Common choices include:
i. Matrix Factorization Models: Models like Collaborative
Filtering or Factorization Machines that decompose the user-
item interaction matrix into latent factors.
ii. Neural Collaborative Filtering (NCF): Models that combine
traditional matrix factorization with neural networks to capture
nonlinearities in user-item interactions.
iii. Sequence Models: Models like Recurrent Neural Networks
(RNNs) or Transformer-based models that can model
sequential user behavior or session-based interactions.
Step 5: Model Training:
a. Train the selected deep learning model using the training data. This
involves optimizing the model parameters to minimize a loss function
that quantifies the model's performance in predicting user-item
interactions.
b. Monitor the model's performance on the validation set and adjust
hyperparameters as needed to prevent overfitting.
Step 6: Model Evaluation:
a. Evaluate the trained model's performance using the test set. Common
evaluation metrics for recommendation systems include precision,
recall, F1-score, and ranking metrics like Mean Average Precision
(MAP) or Normalized Discounted Cumulative Gain (NDCG).
b. Analyze the model's predictions and inspect how well it recommends
relevant items to users based on their historical interactions.
CCS355 NNDL MCE/CSE

CODE:
import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from keras.models import Model
from keras.layers import Input, Embedding, Flatten, Dot, Dense
from keras.callbacks import EarlyStopping

# Load the dataset (Movielens dataset as an example)

# Replace 'ratings.csv' with your dataset file path
ratings_df = pd.read_csv('C://Users//Ananth//Documents//rating.csv')

# Preprocess the data

user_ids = ratings_df['userId'].unique()
item_ids = ratings_df['movieId'].unique()
num_users = len(user_ids)
num_items = len(item_ids)
ratings_df['userId'] = ratings_df['userId'].astype('category').cat.codes
ratings_df['movieId'] = ratings_df['movieId'].astype('category').cat.codes

# Split the data into training and test sets

train_data, test_data = train_test_split(ratings_df, test_size=0.2, random_state=42)

# Define the embedding dimensionality

embedding_dim = 50

# Define the model architecture

user_input = Input(shape=(1,))
item_input = Input(shape=(1,))
user_embedding = Embedding(input_dim=num_users,
output_dim=embedding_dim)(user_input)
item_embedding = Embedding(input_dim=num_items,
output_dim=embedding_dim)(item_input)
user_flat = Flatten()(user_embedding)
item_flat = Flatten()(item_embedding)
dot_product = Dot(axes=1)([user_flat, item_flat])
output = Dense(1, activation='relu')(dot_product)

model = Model(inputs=[user_input, item_input], outputs=output)

model.compile(optimizer='adam', loss='mean_squared_error')

# Train the model

early_stopping = EarlyStopping(patience=3, restore_best_weights=True)
model.fit([train_data['userId'], train_data['movieId']], train_data['rating'],
CCS355 NNDL MCE/CSE

validation_data=([test_data['userId'], test_data['movieId']], test_data['rating']),

epochs=5, batch_size=7200, callbacks=[early_stopping])

# Evaluate the model

test_loss = model.evaluate([test_data['userId'], test_data['movieId']], test_data['rating'])
print("Test Loss:", test_loss)

# Make recommendations for a user

def recommend_for_user(user_id, num_recommendations=5):
user_items = np.array([[user_id]] * num_items)
item_ids = np.arange(num_items).reshape(-1, 1)
ratings = model.predict([user_items, item_ids]).flatten()
top_indices = np.argsort(ratings)[::-1][:num_recommendations]
top_items = item_ids[top_indices].flatten()
return top_items

# Example of making recommendations for a user

user_id = 0 # Example user ID
recommendations = recommend_for_user(user_id)
print("Recommendations for User", user_id, ":", recommendations)

SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

14. IMPLEMENT OBJECT DETECTION USING CNN

AIM:
To implement object detection using CNN
ALGORITHM:

Step 1: Data Collection and Annotation:

a. Collect a dataset of images containing the objects you want to detect.
b. Annotate the dataset by labeling the objects of interest in each image
with bounding boxes. Each bounding box should specify the
coordinates and class label of the object.
Step 2: Data Preprocessing:
a. Preprocess the images and annotations to ensure they are in a suitable
format for training the CNN.
b. Resize the images to a fixed size and normalize the pixel values.
c. Encode the class labels and bounding box coordinates appropriately.
Step 3: Model Selection:
a. Choose a CNN architecture suitable for object detection. Common
choices include:
i. Single Shot Multibox Detector (SSD)
ii. You Only Look Once (YOLO)
iii. Region-based Convolutional Neural Networks (R-CNN),
including Faster R-CNN and Mask R-CNN.
iv. RetinaNet
Step 4: Model Construction:
a. Implement the selected CNN architecture using a deep learning
framework like TensorFlow or PyTorch.
b. Modify the architecture to include layers for both classification and
bounding box regression.
c. Ensure that the network outputs both class predictions and bounding
box coordinates for each detected object.
Step 5: Loss Function:
a. Define a suitable loss function that combines classification loss and
localization loss.
b. Common choices include the combination of cross-entropy loss for
classification and smooth L1 loss for bounding box regression.
Step 6: Training:
a. Train the CNN using the annotated dataset.
b. Use techniques like data augmentation (e.g., random cropping,
rotation, flipping) to increase the diversity of training examples.
c. Monitor the training process and adjust hyperparameters as needed to
improve performance and prevent overfitting.
Step 7: Evaluation:
a. Evaluate the trained model on a separate validation or test set to assess
its performance.
b. Use metrics like precision, recall, and mean Average Precision (mAP)
to measure the accuracy and effectiveness of object detection.
CCS355 NNDL MCE/CSE

CODE:
import os
os.environ["KMP_DUPLICATE_LIB_OK"] = "TRUE"
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1'
import numpy as np
import tensorflow as tf
from tensorflow.keras import layers, models

# Define the CNN model for object detection

def create_model(input_shape, num_classes):
model = models.Sequential([
layers.Conv2D(32, (3, 3), activation='relu', input_shape=input_shape),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.MaxPooling2D((2, 2)),
layers.Conv2D(64, (3, 3), activation='relu'),
layers.Flatten(),
layers.Dense(64, activation='relu'),
layers.Dense(num_classes, activation='softmax')
])
return model

# Dummy data (replace this with your dataset)

# Assuming input images are 100x100 pixels with 3 channels (RGB)
# and there are 10 classes for object detection
input_shape = (100, 100, 3)
num_classes = 10
num_samples = 1000

# Generate random dummy data

X_train = np.random.rand(num_samples, *input_shape)
y_train = np.random.randint(0, num_classes, size=num_samples)

# Normalize input data

X_train = X_train / 255.0

# Create the model

model = create_model(input_shape, num_classes)

# Compile the model

model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])

# Train the model

model.fit(X_train, y_train, epochs=10, batch_size=32)
CCS355 NNDL MCE/CSE

# Save the trained model

model.save('object_detection_model.h5')

# Load the saved model

loaded_model = tf.keras.models.load_model('object_detection_model.h5')

# Example of making predictions

# Replace X_test with your test data
X_test = np.random.rand(10, *input_shape)
predictions = loaded_model.predict(X_test)
print(predictions)
SAMPLE OUTPUT:
CCS355 NNDL MCE/CSE

15. IMPLEMENT ANY SIMPLE REINFORCEMENT ALGORITHM FOR AN

NLP PROBLEM
AIM:
To implement any simple reinforcement algorithm for an NLP problem

ALGORITHM:

Step 1: Problem Formulation:

a. Define the NLP problem you want to solve using RL. For example, it
could be text generation, machine translation, sentiment analysis, or
dialogue generation.
Step 2: Environment Design:
a. Design an environment that represents the NLP task. This environment
should have states, actions, and a reward mechanism.
b. States: Represent the current state of the environment, such as the input
text or the context of a conversation.
c. Actions: Define the possible actions the agent can take, such as
generating a word or making a prediction.
d. Reward: Design a reward mechanism that provides feedback to the
agent based on its actions. The reward can be based on the quality of
generated text, accuracy of predictions, or any other relevant metric for
the NLP task.
Step 3: Agent Definition:
a. Define an agent that interacts with the environment to solve the NLP
problem. The agent should take actions based on the current state of
the environment and the reward received.
b. Choose a suitable RL algorithm for training the agent, such as Q-
learning, Deep Q-Networks (DQN), or Policy Gradient methods.
c. Implement the agent's policy, which specifies how it selects actions
based on the current state and possibly learned parameters.
Step 4: Training Process:
a. Initialize the agent's parameters and the environment.
b. Interact with the environment by selecting actions based on the agent's
policy and observing the resulting state transitions and rewards.
c. Update the agent's parameters using the RL algorithm to maximize
cumulative rewards over time.
d. Repeat the process for multiple episodes until the agent learns an
effective policy for the NLP task.
Step 5: Evaluation:
a. Evaluate the trained agent's performance on a separate validation or
test set of NLP tasks.
b. Measure relevant metrics such as text quality, prediction accuracy, or
task completion rate to assess the agent's effectiveness.
CCS355 NNDL MCE/CSE

CODE:
import numpy as np

# Define the environment

class TextGenerationEnvironment:
def __init__(self, text, window_size=5):
self.text = text
self.window_size = window_size
self.vocab = sorted(set(text))
self.vocab_size = len(self.vocab)
self.state = ''
self.reset()

def reset(self):
self.state = np.random.choice(self.vocab, size=self.window_size).tolist()
return self.state

def step(self, action):

self.state.append(action)
self.state = self.state[1:]
next_char_index = self.vocab.index(action)
reward = self.calculate_reward()
return self.state, reward

def calculate_reward(self):
context = ''.join(self.state)
next_char_index = self.text.find(context)
if next_char_index != -1 and next_char_index + len(context) < len(self.text) - 1:
next_char = self.text[next_char_index + len(context)]
return 1 if next_char in self.state else -1
else:
return -1

# Define the Q-learning agent

class QLearningAgent:
def __init__(self, env, learning_rate=0.1, discount_factor=0.95, exploration_rate=1.0,
exploration_decay=0.995):
self.env = env
self.learning_rate = learning_rate
self.discount_factor = discount_factor
self.exploration_rate = exploration_rate
self.exploration_decay = exploration_decay
self.q_table = {}

def choose_action(self, state):

if np.random.rand() < self.exploration_rate:
CCS355 NNDL MCE/CSE

return np.random.choice(self.env.vocab)
else:
return max(self.env.vocab, key=lambda action: self.q_table.get((tuple(state), action),
0))

def update_q_table(self, state, action, next_state, reward):

current_q = self.q_table.get((tuple(state), action), 0)
max_next_q = max([self.q_table.get((tuple(next_state), a), 0) for a in self.env.vocab])
new_q = current_q + self.learning_rate * (reward + self.discount_factor * max_next_q -
current_q)
self.q_table[(tuple(state), action)] = new_q

def train(self, num_episodes):

for episode in range(num_episodes):
state = self.env.reset()
done = False
total_reward = 0
while not done:
action = self.choose_action(state)
next_state, reward = self.env.step(action)
self.update_q_table(state, action, next_state, reward)
total_reward += reward
state = next_state
if total_reward < 0:
done = True
self.exploration_rate *= self.exploration_decay
print(f"Episode {episode + 1}/{num_episodes}, Total Reward: {total_reward}")

def generate_text(self, length):

state = self.env.reset()
text = ''.join(state)
for _ in range(length):
action = self.choose_action(state)
state, _ = self.env.step(action)
text += action
return text

# Example text data

text_data = "hello world!"

# Initialize environment and agent

env = TextGenerationEnvironment(text_data)
agent = QLearningAgent(env)

# Train the agent

agent.train(num_episodes=1000)
CCS355 NNDL MCE/CSE

# Generate text using the trained agent

generated_text = agent.generate_text(length=20)
print("Generated Text:", generated_text)

SAMPLE OUTPUT: