SUN COLLEGE OF ENGINEERING AND

TECHNOLOGY
UDAYA NAGAR, VELLAMODI

KANYAKUMARI DISTRICT

RECORD NOTE BOOK

Reg. No:

Name :
Semester :
Department :
Subject Name :
Subject Code :
 SUN COLLEGE OF ENGINEERING AND
TECHNOLOGY
UDAYA NAGAR, VELLAMODI
Kanyakumari District

Bonafide Certificate

This is to certify that this bonafide record of work was done by

Mr./Ms register no in the
AD3511- DEEP LEARNING LABORATORY during Semester.

Staff - in - charge Head of the Department

University Reg. No …………………………………….

University Examination held in ………………………

Internal Examiner External Examiner

 INDEX

Sl.
Date Program Name Mark Signature
No.

AD3511-Deep Learning Lab

3
 Exp.No:1 Solving XOR problem using DNN

Aim:

To write a python program for solving XOR problems using DNN.

XOR logical function truth table for 2-bit binary variables, i.e, the input vector and the corresponding
output is,

X1 X2 Y

0 0 0

0 1 1

1 0 1

1 1 0

Procedure:

1. Import the required Python libraries

2.Define Activation Function : Sigmoid Function
3.Initialize neural network parameters (weights, bias)
4. define model hyperparameters (number of iterations, learning rate)
5.Forward Propagation
6. Backward Propagation
7. Update weight and bias parameters
8.Train the learning model
9.Plot Loss value vs Epoch
10.Test the model performance

AD3511-Deep Learning Lab

4
Program:

#import Python Libraries

import numpy as np
from matplotlib import pyplot as plt

# Sigmoid Function
def sigmoid(z):
return 1 / (1 + np.exp(-z))

# Initialization of the neural network parameters

# Initialized all the weights in the range of between 0 and 1
# Bias values are initialized to 0
def initializeParameters(inputFeatures, neuronsInHiddenLayers, outputFeatures):
W1 = np.random.randn(neuronsInHiddenLayers, inputFeatures)
W2 = np.random.randn(outputFeatures, neuronsInHiddenLayers)
b1 = np.zeros((neuronsInHiddenLayers, 1))
b2 = np.zeros((outputFeatures, 1))

parameters = {"W1" : W1, "b1": b1,

"W2" : W2, "b2": b2}
return parameters

# Forward Propagation
def forwardPropagation(X, Y, parameters):
m = X.shape[1]
W1 = parameters["W1"]
W2 = parameters["W2"]
b1 = parameters["b1"]
b2 = parameters["b2"]

Z1 = np.dot(W1, X) + b1
A1 = sigmoid(Z1)
Z2 = np.dot(W2, A1) + b2
A2 = sigmoid(Z2)

cache = (Z1, A1, W1, b1, Z2, A2, W2, b2)

logprobs = np.multiply(np.log(A2), Y) + np.multiply(np.log(1 - A2), (1 - Y))
cost = -np.sum(logprobs) / m
return cost, cache, A2

# Backward Propagation
def backwardPropagation(X, Y, cache):
m = X.shape[1]

AD3511-Deep Learning Lab

5
 (Z1, A1, W1, b1, Z2, A2, W2, b2) = cache

dZ2 = A2 - Y
dW2 = np.dot(dZ2, A1.T) / m
db2 = np.sum(dZ2, axis = 1, keepdims = True)

dA1 = np.dot(W2.T, dZ2)

dZ1 = np.multiply(dA1, A1 * (1- A1))
dW1 = np.dot(dZ1, X.T) / m
db1 = np.sum(dZ1, axis = 1, keepdims = True) / m

gradients = {"dZ2": dZ2, "dW2": dW2, "db2": db2,

"dZ1": dZ1, "dW1": dW1, "db1": db1}
return gradients

# Updating the weights based on the negative gradients

def updateParameters(parameters, gradients, learningRate):
parameters["W1"] = parameters["W1"] - learningRate * gradients["dW1"]
parameters["W2"] = parameters["W2"] - learningRate * gradients["dW2"]
parameters["b1"] = parameters["b1"] - learningRate * gradients["db1"]
parameters["b2"] = parameters["b2"] - learningRate * gradients["db2"]
return parameters

# Model to learn the XOR truth table

X = np.array([[0, 0, 1, 1], [0, 1, 0, 1]]) # XOR input
Y = np.array([[0, 1, 1, 0]]) # XOR output

# Define model parameters

neuronsInHiddenLayers = 2 # number of hidden layer neurons (2)
inputFeatures = X.shape[0] # number of input features (2)
outputFeatures = Y.shape[0] # number of output features (1)
parameters = initializeParameters(inputFeatures, neuronsInHiddenLayers, outputFeatures)
epoch = 100000
learningRate = 0.01
losses = np.zeros((epoch, 1))

for i in range(epoch):
losses[i, 0], cache, A2 = forwardPropagation(X, Y, parameters)
gradients = backwardPropagation(X, Y, cache)
parameters = updateParameters(parameters, gradients, learningRate)

# Evaluating the performance

plt.figure()

AD3511-Deep Learning Lab

6
plt.plot(losses)
plt.xlabel("EPOCHS")
plt.ylabel("Loss value")
plt.show()

# Testing
X = np.array([[1, 1, 0, 0], [0, 1, 0, 1]]) # XOR input
cost, _, A2 = forwardPropagation(X, Y, parameters)
prediction = (A2 > 0.5) * 1.0
# print(A2)
print(prediction)

AD3511-Deep Learning Lab

7
Output:

[[ 1. 0. 0. 1.]]

Result:
Thus the program for solving the XOR problem using DNN was implemented and executed successfully.

AD3511-Deep Learning Lab

8
 Exp.No:2 Character recognition using CNN

Aim:
To write a python program to implement Character recognition using CNN.
Procedure:
1.Data Collection and Preprocessing
2.Model Architecture
3.Compile the Model
4.Model Training
5.Evaluate the Model
6. Fine-Tuning and Optimization

7.Character Recognition
8.Deployment

program:
import numpy as np
import tensorflow as tf
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Conv2D, MaxPooling2D, Flatten, Dense, Dropout
from tensorflow.keras.utils import to_categorical
import matplotlib.pyplot as plt
# Load and preprocess the MNIST dataset
(X_train, y_train), (X_test, y_test) = mnist.load_data()
X_train = X_train.reshape(-1, 28, 28, 1).astype('float32') / 255.0
X_test = X_test.reshape(-1, 28, 28, 1).astype('float32') / 255.0
y_train = to_categorical(y_train, num_classes=10)

AD3511-Deep Learning Lab

9
y_test = to_categorical(y_test, num_classes=10)
# Build the CNN model
model = Sequential([
Conv2D(32, kernel_size=(3, 3), activation='relu', input_shape=(28, 28, 1)),
MaxPooling2D(pool_size=(2, 2)),
Conv2D(64, kernel_size=(3, 3), activation='relu'),
MaxPooling2D(pool_size=(2, 2)),
Flatten(),
Dense(128, activation='relu'),
Dropout(0.5),
Dense(10, activation='softmax')
])
# Compile the model
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Train the model

history = model.fit(X_train, y_train, batch_size=128, epochs=10, validation_data=(X_test, y_test))

# Evaluate the model

test_loss, test_accuracy = model.evaluate(X_test, y_test, verbose=0)
print("Test Accuracy:", test_accuracy)

# Plot training history

plt.figure(figsize=(10, 4))
plt.subplot(1, 2, 1)
plt.plot(history.history['accuracy'], label='Training Accuracy')
plt.plot(history.history['val_accuracy'], label='Validation Accuracy')
plt.xlabel('Epoch')

AD3511-Deep Learning Lab

10
plt.ylabel('Accuracy')
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(history.history['loss'], label='Training Loss')
plt.plot(history.history['val_loss'], label='Validation Loss')
plt.xlabel('Epoch')
plt.ylabel('Loss')
plt.legend()

plt.tight_layout()
plt.show()

# Example prediction
example_index = 0
example_image = X_test[example_index]
example_label = np.argmax(y_test[example_index])
predicted_label = np.argmax(model.predict(np.expand_dims(example_image, axis=0)))

plt.imshow(example_image.squeeze(), cmap='gray')
plt.title(f"True Label: {example_label}, Predicted Label: {predicted_label}")
plt.axis('off')
plt.show()
Epoch 1/10
469/469 [==============================] - 69s 142ms/step - loss: 0.2773 - accuracy: 0.9156 - val_loss: 0.0606 -
val_accuracy: 0.9801
Epoch 2/10
469/469 [==============================] - 71s 151ms/step - loss: 0.0928 - accuracy: 0.9729 - val_loss: 0.0420 -
val_accuracy: 0.9853

AD3511-Deep Learning Lab

11
Epoch 3/10
469/469 [==============================] - 78s 166ms/step - loss: 0.0666 - accuracy: 0.9804 - val_loss: 0.0384 -
val_accuracy: 0.9878
Epoch 4/10
469/469 [==============================] - 58s 123ms/step - loss: 0.0550 - accuracy: 0.9832 - val_loss: 0.0295 -
val_accuracy: 0.9902
Epoch 5/10
469/469 [==============================] - 79s 167ms/step - loss: 0.0472 - accuracy: 0.9857 - val_loss: 0.0299 -
val_accuracy: 0.9900
Epoch 6/10
469/469 [==============================] - 29s 61ms/step - loss: 0.0409 - accuracy: 0.9878 - val_loss: 0.0268 - val_accuracy:
0.9904
Epoch 7/10
469/469 [==============================] - 24s 52ms/step - loss: 0.0365 - accuracy: 0.9886 - val_loss: 0.0280 - val_accuracy:
0.9911
Epoch 8/10
469/469 [==============================] - 25s 53ms/step - loss: 0.0315 - accuracy: 0.9904 - val_loss: 0.0257 - val_accuracy:
0.9915
Epoch 9/10
469/469 [==============================] - 22s 47ms/step - loss: 0.0287 - accuracy: 0.9908 - val_loss: 0.0284 - val_accuracy:
0.9913
Epoch 10/10
469/469 [==============================] - 22s 48ms/step - loss: 0.0259 - accuracy: 0.9917 - val_loss: 0.0252 - val_accuracy:
0.9914
Test Accuracy: 0.9914000034332275

AD3511-Deep Learning Lab

12
1/1 [==============================] - 0s 149ms/step

Result:
Thus a python program to implement character recognition using CNN was implemented and executed
successfully.

AD3511-Deep Learning Lab

13
 Exp.No:3 Face recognition using CNN

Aim:
To write a python program to implement face recognition using CNN.

Definition:
Convolutional Neural Networks are a special kind of Neural Networks which helps the machine to learn
and classify the images. A good example is Face Recognition.

Procedure:

● Import TensorFlow
● Download and prepare the dataset
● Verify the data
● Create the convolutional base
● Add Dense layers on top
● Compile and train the model
● Evaluate the model
● Print the test accuracy

Program:
import tensorflow as tf
from tensorflow.keras import datasets, layers, models
import matplotlib.pyplot as plt

(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data()

# Normalize pixel values to be between 0 and 1
train_images, test_images = train_images / 255.0, test_images / 255.0

AD3511-Deep Learning Lab

14
(train_images, train_labels), (test_images, test_labels) = datasets.cifar10.load_data() #
Normalize pixel values to be between 0 and 1 train_images, test_images = train_images /
255.0, test_images / 255.0
In [ ]:
class_names = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']
plt.figure(figsize=(10,10)) for i in range(25): plt.subplot(5,5,i+1) plt.xticks([]) plt.yticks([]) plt.grid(False)
plt.imshow(train_images[i]) # The CIFAR labels happen to be arrays, # which is why you need the extra index
plt.xlabel(class_names[train_labels[i][0]]) plt.show()
model = models.Sequential() model.add(layers.Conv2D(32, (3, 3), activation

='relu',
input_shape=(32, 32, 3))) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3),
activation='relu')) model.add(layers.MaxPooling2D((2, 2))) model.add(layers.Conv2D(64, (3, 3),
activation='relu'))
In [ ]:

AD3511-Deep Learning Lab

15
model.summary()
Model: "sequential"

Layer (type) Output Shape Param #

=================================================================
conv2d (Conv2D) (None, 30, 30, 32) 896

max_pooling2d (MaxPooling2 (None, 15, 15, 32) 0 D)

conv2d_1 (Conv2D) (None, 13, 13, 64) 18496

max_pooling2d_1 (MaxPoolin (None, 6, 6, 64) 0 g2D)

conv2d_2 (Conv2D) (None, 4, 4, 64) 36928

=================================================================
Total params: 56320 (220.00 KB)
Trainable params: 56320 (220.00 KB)
Non-trainable params: 0 (0.00 Byte)

In [ ]:
model.add(layers.Flatten()) model.add(layers.Dense(64, activation='relu')) model.add(layers.Dense(10))
In [ ]:
model.summary()
conv2d_2 (Conv2D) (None, 4, 4, 64) 36928

flatten (Flatten) (None, 1024) 0

AD3511-Deep Learning Lab

16
dense (Dense) (None, 64) 65600

dense_1 (Dense) (None, 10) 650

=================================================================
Total params: 122570 (478.79 KB)
Trainable params: 122570 (478.79 KB)
Non-trainable params: 0 (0.00 Byte)

In [ ]:
model.compile(optimizer='adam', loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy']) history = model.fit(train_images, train_labels, epochs=10, validation_data=(test_images,
test_labels))
Epoch 1/10
1563/1563 [==============================] - 10s 4ms/step - loss: 1.5733 - accuracy: 0.4257 - val_loss:
1.2938 - val_accuracy: 0.5405
Epoch 2/10
1563/1563 [==============================] - 6s 4ms/step - loss: 1.1916 - accuracy: 0.5761 - val_loss:
1.1120 - val_accuracy: 0.6029
Epoch 3/10
1563/1563 [==============================] - 6s 4ms/step - loss: 1.0424 - accuracy: 0.6315 - val_loss:
1.0490 - val_accuracy: 0.6332
Epoch 4/10
1563/1563 [==============================] - 6s 4ms/step - loss: 0.9586 - accuracy: 0.6631 - val_loss:
0.9473 - val_accuracy: 0.6711
Epoch 5/10
1563/1563 [==============================] - 6s 4ms/step - loss: 0.8903 - accuracy: 0.6875 - val_loss:
0.9499 - val_accuracy: 0.6693

AD3511-Deep Learning Lab

17
Epoch 6/10
1563/1563 [==============================] - 6s 4ms/step - loss: 0.8352 - accuracy: 0.7077 - val_loss:
0.9962 - val_accuracy: 0.6548
Epoch 7/10
1563/1563 [==============================] - 6s 4ms/step - loss: 0.7903 - accuracy: 0.7219 - val_loss:
0.9115 - val_accuracy: 0.6910
Epoch 8/10
1563/1563 [==============================] - 6s 4ms/step - loss: 0.7500 - accuracy: 0.7349 - val_loss:
0.8694 - val_accuracy: 0.6984
Epoch 9/10
1563/1563 [==============================] - 6s 4ms/step - loss: 0.7134 - accuracy: 0.7494 - val_loss:
0.8856 - val_accuracy: 0.6930
Epoch 10/10
1563/1563 [==============================] - 6s 4ms/step - loss: 0.6781 - accuracy: 0.7627 - val_loss:
0.8593 - val_accuracy: 0.7105

In [ ]:
plt.plot(history.history['accuracy'], label='accuracy')
plt.plot(history.history['val_accuracy'], label = 'val_accuracy')
plt.xlabel('Epoch') plt.ylabel('Accuracy')
plt.ylim([0.5, 1])
plt.legend(loc='lower right')
test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)

313/313 - 1s - loss: 0.8593 - accuracy: 0.7105 - 558ms/epoch - 2ms/step

AD3511-Deep Learning Lab

18
 In [ ]:
print(test_acc)

0.7105000019073486

Result:
Thus a python program to implement face recognition using CNN was implemented and executed
successfully.

AD3511-Deep Learning Lab

19
Exp.No:4 Language modeling using RNN

Aim:
To write a python program to implement language modelling using RNN.
Procedure:

1. Convert abstracts from list of strings into list of lists of integers (sequences)

2. Create feature and labels from sequences

3. Build model with Embedding and Dense layers

4. Load in pre-trained embeddings

5. Train model to predict next work in sequence

6. Make predictions by passing in starting sequence

Program:

import numpy as np
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Dropout
from keras.layers import LSTM
from keras.utils import np_utils

#Read the data, turn it into lower case

data = open("Othello.txt").read().lower()
#This get the set of characters used in the data and sorts them
chars = sorted(list(set(data)))

AD3511-Deep Learning Lab

20
#Total number of characters used in the data
totalChars = len(data)
#Number of unique chars
numberOfUniqueChars = len(chars)

#This allows for characters to be represented by numbers

CharsForids = {char:Id for Id, char in enumerate(chars)}

#This is the opposite to the above

idsForChars = {Id:char for Id, char in enumerate(chars)}

#How many timesteps e.g how many characters we want to process in one go
numberOfCharsToLearn = 100

#Since our time step sequence represents a process for every 100 chars we omit
#the first 100 chars so the loop runs a 100 less or there will be index out of
#range
counter = totalChars - numberOfCharsToLearn

#Inpput data
charX = []
#output data
y = []
#This loops through all the characters in the data skipping the first 100
for i in range(0, counter, 1):
#This one goes from 0-100 so it gets 100 values starting from 0 and stops
#just before the 100th value
theInputChars = data[i:i+numberOfCharsToLearn]

AD3511-Deep Learning Lab

21
 #With no : you start with 0, and so you get the actual 100th value
#Essentially, the output Chars is the next char in line for those 100 chars
#in X
theOutputChars = data[i + numberOfCharsToLearn]
#Appends every 100 chars ids as a list into X
charX.append([CharsForids[char] for char in theInputChars])
#For every 100 values there is one y value which is the output
y.append(CharsForids[theOutputChars])
#Len charX represents how many of those time steps we have
#Our features are set to 1 because in the output we are only predicting 1 char
#Finally numberOfCharsToLearn is how many character we process
X = np.reshape(charX, (len(charX), numberOfCharsToLearn, 1))

#This is done for normalization

X = X/float(numberOfUniqueChars)

#This sets it up for us so we can have a categorical(#feature) output format

y = np_utils.to_categorical(y)
print(y)

model = Sequential()
#Since we know the shape of our Data we can input the timestep and feature data
#The number of timestep sequence are dealt with in the fit function
model.add(LSTM(256, input_shape=(X.shape[1], X.shape[2])))
model.add(Dropout(0.2))
#number of features on the output
model.add(Dense(y.shape[1], activation='softmax'))

AD3511-Deep Learning Lab

22
model.compile(loss='categorical_crossentropy', optimizer='adam')
model.fit(X, y, epochs=5, batch_size=128)
model.save_weights("Othello.hdf5")
#model.load_weights("Othello.hdf5")

randomVal = np.random.randint(0, len(charX)-1)

randomStart = charX[randomVal]
for i in range(500):
x = np.reshape(randomStart, (1, len(randomStart), 1))
x = x/float(numberOfUniqueChars)
pred = model.predict(x)
index = np.argmax(pred)
randomStart.append(index)
randomStart = randomStart[1: len(randomStart)]
print("".join([idsForChars[value] for value in randomStart]))

AD3511-Deep Learning Lab

23
 Output:
Epoch 1/5
128/1760 [=> ........................... ] - ETA: 43s - loss: 3.3984

256/1760 [===>..........................] - ETA: 27s - loss: 3.3905

384/1760 [=====>........................] - ETA: 21s - loss: 3.3835

512/1760 [=======>......................] - ETA: 18s - loss: 3.3749

640/1760 [=========>....................] - ETA: 15s - loss: 3.3615

768/1760 [============>.................] - ETA: 13s - loss: 3.3425

896/1760 [==============>...............] - ETA: 11s - loss: 3.3174

1024/1760 [================>.............] - ETA: 9s - loss: 3.3563

Result:
Thus a python program to implement language modelling using RNN was implemented and executed
successfully.

AD3511-Deep Learning Lab

24
 Exp.No:5 Sentiment Analysis using LSTM

Aim:

To write a python program to implement sentiment analysis using LSTM.

Procedure:
1. Loading data

2. Preprocessing the data and Tokenizing

3. Building and fitting the model on data

4. Evaluate the model

5. Predicting

Program:
import tensorflow as tf
from tensorflow.keras.datasets import imdb
from tensorflow.keras.layers import Embedding, Dense, LSTM
from tensorflow.keras.losses import BinaryCrossentropy
from tensorflow.keras.models import Sequential
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.preprocessing.sequence import pad_sequences

additional_metrics = ['accuracy']
batch_size = 128
embedding_output_dims = 15
loss_function = BinaryCrossentropy()
max_sequence_length = 300
num_distinct_words = 5000

AD3511-Deep Learning Lab

25
number_of_epochs = 5
optimizer = Adam()
validation_split = 0.20
verbosity_mode = 1
(x_train, y_train), (x_test, y_test) = imdb.load_data(num_words=num_distinct_words)
print(x_train.shape)
print(x_test.shape)
padded_inputs = pad_sequences(x_train, maxlen=max_sequence_length, value = 0.0)
padded_inputs_test = pad_sequences(x_test, maxlen=max_sequence_length, value = 0.0)
len(x_train[890])
len(padded_inputs[890])
model = Sequential()
model.add(Embedding(num_distinct_words, embedding_output_dims, input_length=max_sequence_length))
model.add(LSTM(10))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer=optimizer, loss=loss_function, metrics=additional_metrics)
model.summary()
Output:

Model: "sequential_4"

Layer (type) Output Shape Param #

=================================================================
embedding_4 (Embedding) (None, 300, 15) 75000

lstm_4 (LSTM) (None, 10) 1040

dense_4 (Dense) (None, 1) 11

=================================================================
Total params: 76,051

AD3511-Deep Learning Lab

26
Trainable params: 76,051
Non-trainable params: 0

test_results = model.evaluate(padded_inputs_test, y_test, verbose=False)

print(f'Test results - Loss: {test_results[0]} - Accuracy: {100*test_results[1]}%')

Output:
Test results - Loss: 0.6933784971427918 - Accuracy: 48.05600047111511%

Result:
Thus a python program to implement sentiment analysis using LSTM was implemented and executed
successfully.

AD3511-Deep Learning Lab

27
Exp.No:6 Parts Of Speech tagging using Sequence to Sequence architecture

Aim:
To write a python program to implement Parts Of Speech tagging using Sequence to Sequence
architecture .
Procedure:

1.Data Preparation:
● Prepare your POS-tagged corpus where each sentence is paired with its corresponding POS tags.
2.Vocabulary Building:
● Create vocabularies for words and POS tags present in your corpus.
● Assign unique indices to each word and POS tag.
3.Encoding:
● Tokenize sentences into word indices and POS tag indices.
● Pad or truncate sequences to a fixed length to maintain uniform input shapes.
● Create an encoder model using Embedding layers to embed word indices and POS tag indices.
● Process the input through the encoder, which will generate a context vector.
4.Decoding:
● Design a decoder model to take the context vector and generate POS tag sequences.
● You might need to use teacher forcing (feeding ground-truth POS tags as inputs) during training.
● During inference, use the decoder to generate POS tag sequences by feeding the context vector and
previously generated tags as inputs.
5.Model Compilation and Training:
● Compile the Seq2Seq model with appropriate loss functions (e.g., categorical cross-entropy) and
optimizers (e.g., Adam).
● Train the model using your POS-tagged data. The decoder will predict POS tags based on the input
words and the context vector.
6.Evaluation:
● Evaluate the model on a separate test dataset to measure its POS tagging accuracy.
7.Inference:
● For inference, input a sentence to the encoder and then generate the corresponding POS tags using
the decoder.
8.Fine-Tuning and Optimization:
● Depending on the model's performance, experiment with hyperparameters, model architecture, and
training strategies.
9.Deployment (Optional):
● If the model performs well, you can deploy it for POS tagging tasks. However, traditional models
like CRF are usually more suitable for POS tagging due to their fixed input/output sequence
lengths.

AD3511-Deep Learning Lab

28
Program:

import numpy as np
import tensorflow as tf
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.layers import Input, Embedding, LSTM, Dense
from tensorflow.keras.models import Model
from tensorflow.keras.utils import to_categorical

# Sample data for demonstration

sentences = [
"The cat is sleeping",
"A dog is barking",
"She is reading a book"
]

pos_tags = [
"DT NN VBZ VBG",
"DT NN VBZ VBG",
"PRP VBZ VBG DT NN"
]
# Tokenize sentences and POS tags
tokenizer = Tokenizer()
tokenizer.fit_on_texts(sentences + pos_tags)

input_sequences = tokenizer.texts_to_sequences(sentences)
output_sequences = tokenizer.texts_to_sequences(pos_tags)

vocab_size = len(tokenizer.word_index) + 1

# Pad sequences
max_sequence_length = max(max(len(seq) for seq in input_sequences), max(len(seq) for seq in
output_sequences))
input_sequences_padded = pad_sequences(input_sequences, maxlen=max_sequence_length, padding='post')
output_sequences_padded = pad_sequences(output_sequences, maxlen=max_sequence_length, padding='post')

# Prepare decoder input and output

decoder_input_sequences = output_sequences_padded[:, :-1]
decoder_output_sequences = output_sequences_padded[:, 1:]

decoder_output_sequences = to_categorical(decoder_output_sequences, num_classes=vocab_size)

AD3511-Deep Learning Lab

29
# Define the model architecture
latent_dim = 256

# Encoder
encoder_input = Input(shape=(max_sequence_length,))
encoder_embedding = Embedding(vocab_size, latent_dim)(encoder_input)
encoder_lstm = LSTM(latent_dim, return_state=True)
encoder_outputs, state_h, state_c = encoder_lstm(encoder_embedding)
encoder_states = [state_h, state_c]

# Decoder
decoder_input = Input(shape=(max_sequence_length - 1,))
decoder_embedding = Embedding(vocab_size, latent_dim)(decoder_input)
decoder_lstm = LSTM(latent_dim, return_sequences=True, return_state=True)
decoder_outputs, _, _ = decoder_lstm(decoder_embedding, initial_state=encoder_states)
decoder_dense = Dense(vocab_size, activation='softmax')
decoder_outputs = decoder_dense(decoder_outputs)

# Build the model

model = Model([encoder_input, decoder_input], decoder_outputs)
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# Train the model

model.fit([input_sequences_padded, decoder_input_sequences], decoder_output_sequences, epochs=50,
verbose=1)

# Generate predictions
encoder_model = Model(encoder_input, encoder_states)

decoder_state_input_h = Input(shape=(latent_dim,))
decoder_state_input_c = Input(shape=(latent_dim,))
decoder_states_inputs = [decoder_state_input_h, decoder_state_input_c]

decoder_outputs, state_h, state_c = decoder_lstm(decoder_embedding, initial_state=decoder_states_inputs)

decoder_states = [state_h, state_c]
decoder_outputs = decoder_dense(decoder_outputs)

decoder_model = Model([decoder_input] + decoder_states_inputs, [decoder_outputs] + decoder_states)

# Function to predict POS tags

def predict_pos(input_text):
input_seq = tokenizer.texts_to_sequences([input_text])[0]
input_seq_padded = pad_sequences([input_seq], maxlen=max_sequence_length, padding='post')

AD3511-Deep Learning Lab

30
 states_value = encoder_model.predict(input_seq_padded)

target_seq = np.zeros((1, 1))

target_seq[0, 0] = tokenizer.word_index['<start>'] # Start token

decoded_sentence = ''
stop_condition = False
while not stop_condition:
output_tokens, h, c = decoder_model.predict([target_seq] + states_value)
sampled_token_index = np.argmax(output_tokens[0, -1, :])
sampled_word = tokenizer.index_word[sampled_token_index]
if sampled_word != '<end>':
decoded_sentence += sampled_word + ' '

if sampled_word == '<end>' or len(decoded_sentence.split()) > max_sequence_length:

stop_condition = True

target_seq = np.zeros((1, 1))

target_seq[0, 0] = sampled_token_index

states_value = [h, c]

return decoded_sentence.strip()

# Test the model

test_sentence = "A cat is sleeping"
predicted_pos = predict_pos(test_sentence)
print(f"Input Sentence: {test_sentence}")
print(f"Predicted POS Tags: {predicted_pos}")

Output:
Epoch 1/50
1/1 [==============================] - 22s 22s/step - loss: 2.7771 - accuracy: 0.0000e+00
Epoch 2/50
1/1 [==============================] - 0s 132ms/step - loss: 2.7341 - accuracy: 0.8333
Epoch 3/50
1/1 [==============================] - 0s 127ms/step - loss: 2.6894 - accuracy: 0.9167
Epoch 4/50
1/1 [==============================] - 0s 124ms/step - loss: 2.6386 - accuracy: 0.9167
Epoch 5/50
1/1 [==============================] - 0s 129ms/step - loss: 2.5771 - accuracy: 0.8333

AD3511-Deep Learning Lab

31
Epoch 6/50
1/1 [==============================] - 0s 143ms/step - loss: 2.4998 - accuracy: 0.8333
Epoch 7/50
1/1 [==============================] - 0s 114ms/step - loss: 2.4000 - accuracy: 0.7500
Epoch 8/50
1/1 [==============================] - 0s 113ms/step - loss: 2.2704 - accuracy: 0.5833
Epoch 9/50
1/1 [==============================] - 0s 189ms/step - loss: 2.1041 - accuracy: 0.5000
Epoch 10/50
1/1 [==============================] - 0s 76ms/step - loss: 1.9022 - accuracy: 0.5000
Epoch 11/50
1/1 [==============================] - 0s 55ms/step - loss: 1.6912 - accuracy: 0.4167
Epoch 12/50
1/1 [==============================] - 0s 52ms/step - loss: 1.5327 - accuracy: 0.3333
Epoch 13/50
1/1 [==============================] - 0s 71ms/step - loss: 1.4486 - accuracy: 0.5000
Epoch 14/50
1/1 [==============================] - 0s 59ms/step - loss: 1.3796 - accuracy: 0.5000
Epoch 15/50
1/1 [==============================] - 0s 56ms/step - loss: 1.2971 - accuracy: 0.7500
Epoch 16/50
1/1 [==============================] - 0s 60ms/step - loss: 1.2208 - accuracy: 0.7500
Epoch 17/50
1/1 [==============================] - 0s 58ms/step - loss: 1.1531 - accuracy: 0.6667
Epoch 18/50
1/1 [==============================] - 0s 62ms/step - loss: 1.0652 - accuracy: 0.8333
Epoch 19/50
1/1 [==============================] - 0s 55ms/step - loss: 0.9677 - accuracy: 0.6667
Epoch 20/50
1/1 [==============================] - 0s 59ms/step - loss: 0.8924 - accuracy: 0.6667
Epoch 21/50
1/1 [==============================] - 0s 64ms/step - loss: 0.8438 - accuracy: 0.7500
Epoch 22/50
1/1 [==============================] - 0s 65ms/step - loss: 0.7938 - accuracy: 0.8333
Epoch 23/50
1/1 [==============================] - 0s 52ms/step - loss: 0.7245 - accuracy: 0.9167
Epoch 24/50
1/1 [==============================] - 0s 47ms/step - loss: 0.6433 - accuracy: 0.9167
Epoch 25/50

AD3511-Deep Learning Lab

32
1/1 [==============================] - 0s 64ms/step - loss: 0.5674 - accuracy: 0.9167
Epoch 26/50
1/1 [==============================] - 0s 53ms/step - loss: 0.5071 - accuracy: 0.9167
Epoch 27/50
1/1 [==============================] - 0s 52ms/step - loss: 0.4552 - accuracy: 1.0000
Epoch 28/50
1/1 [==============================] - 0s 55ms/step - loss: 0.4024 - accuracy: 1.0000
Epoch 29/50
1/1 [==============================] - 0s 57ms/step - loss: 0.3485 - accuracy: 1.0000
Epoch 30/50
1/1 [==============================] - 0s 42ms/step - loss: 0.2988 - accuracy: 1.0000
Epoch 31/50
1/1 [==============================] - 0s 55ms/step - loss: 0.2579 - accuracy: 1.0000
Epoch 32/50
1/1 [==============================] - 0s 55ms/step - loss: 0.2250 - accuracy: 1.0000
Epoch 33/50
1/1 [==============================] - 0s 52ms/step - loss: 0.1960 - accuracy: 1.0000
Epoch 34/50
1/1 [==============================] - 0s 55ms/step - loss: 0.1683 - accuracy: 1.0000
Epoch 35/50
1/1 [==============================] - 0s 62ms/step - loss: 0.1421 - accuracy: 1.0000
Epoch 36/50
1/1 [==============================] - 0s 57ms/step - loss: 0.1191 - accuracy: 1.0000
Epoch 37/50
1/1 [==============================] - 0s 50ms/step - loss: 0.0997 - accuracy: 1.0000
Epoch 38/50
1/1 [==============================] - 0s 55ms/step - loss: 0.0834 - accuracy: 1.0000
Epoch 39/50
1/1 [==============================] - 0s 61ms/step - loss: 0.0698 - accuracy: 1.0000
Epoch 40/50
1/1 [==============================] - 0s 57ms/step - loss: 0.0589 - accuracy: 1.0000
Epoch 41/50
1/1 [==============================] - 0s 62ms/step - loss: 0.0500 - accuracy: 1.0000
Epoch 42/50
1/1 [==============================] - 0s 67ms/step - loss: 0.0427 - accuracy: 1.0000
Epoch 43/50
1/1 [==============================] - 0s 57ms/step - loss: 0.0366 - accuracy: 1.0000
Epoch 44/50
1/1 [==============================] - 0s 56ms/step - loss: 0.0315 - accuracy: 1.0000

AD3511-Deep Learning Lab

33
Epoch 45/50
1/1 [==============================] - 0s 57ms/step - loss: 0.0271 - accuracy: 1.0000
Epoch 46/50
1/1 [==============================] - 0s 69ms/step - loss: 0.0234 - accuracy: 1.0000
Epoch 47/50
1/1 [==============================] - 0s 59ms/step - loss: 0.0203 - accuracy: 1.0000
Epoch 48/50
1/1 [==============================] - 0s 48ms/step - loss: 0.0179 - accuracy: 1.0000
Epoch 49/50
1/1 [==============================] - 0s 60ms/step - loss: 0.0159 - accuracy: 1.0000
Epoch 50/50
1/1 [==============================] - 0s 70ms/step - loss: 0.0143 - accuracy: 1.0000
1/1 [==============================] - 2s 2s/step

Result:
Thus a python program to implement Parts of speech tagging using Sequence to Sequence architecture was
implemented and executed successfully.

AD3511-Deep Learning Lab

34
Ex.No.7 Program For Denoising Image Using Encoder Decoder Model

Aim:

To write a python program for denoising image using autoencoders

Procedure:

1. Implement a deep convolutional autoencoder for image denoising, mapping noisy digits images
from the MNIST dataset to clean digits images.
2. This implementation is based on an original blog post titled Building Autoencoders in Keras
3. Setup the necessary library files.
4. Build the autoencoder
5. We can train our autoencoder using train_data as both our input data and target
6. Predict on our test dataset and display the original image together with the prediction from
our autoencoder.
7. Using the noisy data as our input and the clean data as our target, we want our autoencoder to
learn how to denoise the images.
8. Now predict on the noisy data and display the results of our autoencoder.
9. The autoencoder finally removes the noise from the input images.

PROGRAM:

import numpy as np import tensorflow

as tf

import matplotlib.pyplot as plt

from tensorflow.keras import layers

from tensorflow.keras.datasets import mnist from

tensorflow.keras.models import Model

def preprocess(array):
array = array.astype("float32") / 255.0

array = np.reshape(array, (len(array), 28, 28, 1)) return array

AD3511-Deep Learning Lab

35
 def noise(array):

noise_factor = 0.4

noisy_array = array + noise_factor * np.random.normal( loc=0.0,

scale=1.0, size=array.shape

return np.clip(noisy_array, 0.0, 1.0)

def display(array1, array2):

n = 10
indices = np.random.randint(len(array1), size=n) images1
=array1[indices, :]
images2 = array2[indices, :]
plt.figure(figsize=(20, 4))

for i, (image1, image2) in enumerate(zip(images1, images2)): ax =

plt.subplot(2, n, i + 1)

plt.imshow(image1.reshape(28, 28)) plt.gray()

ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)

ax = plt.subplot(2, n, i + 1 + n)
plt.imshow(image2.reshape(28, 28)) plt.gray()
ax.get_xaxis().set_visible(False)
ax.get_yaxis().set_visible(False)

plt.show()

(train_data, _), (test_data, _) = mnist.load_data()

# Normalize and reshape the data train_data =

preprocess(train_data) test_data =

preprocess(test_data)

AD3511-Deep Learning Lab

36
 # Create a copy of the data with added noise noisy_train_data

= noise(train_data) noisy_test_data = noise(test_data)

# Display the train data and a version of it with added noise display(train_data,

noisy_train_data)

input = layers.Input(shape=(28, 28, 1))

# Encoder

x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(input) x =

layers.MaxPooling2D((2, 2), padding="same")(x)

x = layers.Conv2D(32, (3, 3), activation="relu", padding="same")(x) x =

layers.MaxPooling2D((2, 2), padding="same")(x)

# Decoder

x = layers.Conv2DTranspose(32, (3, 3), strides=2, activation="relu", padding="same")(x) x =

layers.Conv2DTranspose(32, (3, 3), strides=2, activation="relu", padding="same")(x) x =

layers.Conv2D(1, (3, 3), activation="sigmoid", padding="same")(x)

# Autoencoder

autoencoder = Model(input, x) autoencoder.compile(optimizer="adam",

AD3511-Deep Learning Lab

37
 loss="binary_crossentropy") autoencoder.summary()

Result:
Thus a python program to implement Machine Translation using the Encoder-Decoder model was
implemented and executed successfully.

AD3511-Deep Learning Lab

38
 Exp.No:8 Image Augmentation using GAN

Aim:

To write a python program for Image Augmentation using GAN.

Procedure:

1.Dataset Preparation:
● Prepare the original dataset of images that you want to augment. This could be any dataset relevant
to your task, such as images of objects, animals, or scenes.
2.Build a GAN:
● Design and build a GAN architecture. A GAN consists of a generator and a discriminator network.
● The generator network generates new images from random noise.
● The discriminator network tries to distinguish between real images from the original dataset and
fake images generated by the generator.
3.Train the GAN:
● Train the GAN on the original dataset. The generator learns to create images that are increasingly
similar to the real dataset.
● The discriminator's goal is to get better at distinguishing real from generated images.
● Train the generator and discriminator in alternating steps.
4.Generate Augmented Images:
● After training, use the trained generator to generate new images. These images will be similar to
the original dataset but might have slight variations.
5.Augment Your Dataset:
● Combine the generated images with your original dataset to create an augmented dataset.
● The augmented dataset now contains both the original images and the generated images.
6.Train a Model:
● Use the augmented dataset to train your machine learning model. This model can be a neural
network, such as a convolutional neural network (CNN), for various tasks like image classification,
object detection, etc.
7.Evaluate and Compare:
● Evaluate your model's performance using the augmented dataset and compare it with the
performance using the original dataset alone.
● Augmented data can improve the model's generalization ability, especially if the original dataset is
limited.

AD3511-Deep Learning Lab

39
 8.Fine-Tuning (Optional):
● Depending on the performance, you can fine-tune the GAN or the machine learning model to
achieve better results.
9.Inference:
● Use your trained model for inference on real-world data.

Program:
import numpy as np

import matplotlib.pyplot as plt

import tensorflow as tf

from tensorflow.keras.layers import Input, Dense, Reshape, Flatten

from tensorflow.keras.models import Sequential, Model

from tensorflow.keras.optimizers import Adam

from tensorflow.keras.datasets import mnist

# Define the generator

def build_generator(latent_dim, output_shape):

model = Sequential()

model.add(Dense(128, input_dim=latent_dim, activation='relu'))

model.add(Dense(256, activation='relu'))

model.add(Dense(np.prod(output_shape), activation='sigmoid'))

model.add(Reshape(output_shape))

return model

# Define the discriminator

def build_discriminator(input_shape):

model = Sequential()

AD3511-Deep Learning Lab

40
 model.add(Flatten(input_shape=input_shape))

model.add(Dense(256, activation='relu'))

model.add(Dense(128, activation='relu'))

model.add(Dense(1, activation='sigmoid'))

return model

# Build and compile the discriminator

input_shape = (28, 28, 1)

discriminator = build_discriminator(input_shape)

discriminator.compile(loss='binary_crossentropy', optimizer=Adam(learning_rate=0.0002, beta_1=0.5),

metrics=['accuracy'])

# Build and compile the generator

latent_dim = 100

generator = build_generator(latent_dim, input_shape)

generator.compile(loss='binary_crossentropy', optimizer=Adam(learning_rate=0.0002, beta_1=0.5))

# Build GAN by chaining generator and discriminator

discriminator.trainable = False

gan_input = Input(shape=(latent_dim,))

generated_image = generator(gan_input)

gan_output = discriminator(generated_image)

gan = Model(gan_input, gan_output)

gan.compile(loss='binary_crossentropy', optimizer=Adam(learning_rate=0.0002, beta_1=0.5))

AD3511-Deep Learning Lab

41
# Load and preprocess MNIST dataset

(X_train, _), (_, _) = mnist.load_data()

X_train = X_train / 127.5 - 1.0

X_train = np.expand_dims(X_train, axis=-1)

# Training GAN

epochs = 10000

batch_size = 64

sample_interval = 1000

for epoch in range(epochs):

# Train discriminator

idx = np.random.randint(0, X_train.shape[0], batch_size)

real_images = X_train[idx]

fake_images = generator.predict(np.random.randn(batch_size, latent_dim))

d_loss_real = discriminator.train_on_batch(real_images, np.ones((batch_size, 1)))

d_loss_fake = discriminator.train_on_batch(fake_images, np.zeros((batch_size, 1)))

d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)

# Train generator

noise = np.random.randn(batch_size, latent_dim)

g_loss = gan.train_on_batch(noise, np.ones((batch_size, 1)))

AD3511-Deep Learning Lab

42
# Print progress and save generated images

if epoch % sample_interval == 0:

print(f"Epoch {epoch}, D Loss: {d_loss[0]}, G Loss: {g_loss}")

generated_images = generator.predict(noise)

generated_images = 0.5 * generated_images + 0.5

fig, axs = plt.subplots(5, 5)

cnt = 0

for i in range(5):

for j in range(5):

axs[i,j].imshow(generated_images[cnt, :, :, 0], cmap='gray')

axs[i,j].axis('off')

cnt += 1

plt.show()

AD3511-Deep Learning Lab

43
Output:

2/2 [==============================] - 0s 0s/step

Epoch 0, D Loss: 0.811555951833725, G Loss: 0.8482639193534851

2/2 [==============================] - 0s 8ms/step

Result:

Thus a python program to implement Image augmentation using GANs was implemented and executed
successfully.

AD3511-Deep Learning Lab

44