Video Feedback

Sentiment Analysis

A project report submitted in partial fulfillmentof the requirements for

the award of the degree of

MASTER OF COMPUTER APPLICATIONS

in

COMPUTER APPLICATIONS

by

Harsh Goyal (205122031)

OFT
STITUTE

9010N
NATIONA
1964

TIRUCHIAAPPALLLI

DEPARTMENT OF COMPUTER APPLICATIONS

NATIONAL INSTITUTE OF TECHNOLOGY
TIRUCHIRAPPALLI -620015

MAY 2024
 Video Feedback
Sentiment Analysis

in partial fulfillment of the requirements for

A project report submitted

the award of the degree of

MASTER OF COMPUTER APPLICATIONS

in

COMPUTER APPLICATIONS
by

Harsh Goyal (205122031)

6TITUTE oF
INS TE

KTONAL

1964

TIRUCHIRAPPALLI

DEPARTMENT OF COMPUTER APPLICATIONS

NATIONAL INSTITUTEOF TECHNOLO
TIRUCHIRAPPALLI -620015

MAY 2024
OPTIMIZATION OF NEURAL NETWORKS IN THE FINANCIAL
DOMAIN: TAILORING THE BEST OPTIMIZER
in partial fulfillmentof the
A project report submitted

the degree of
requirements for the award of

MASTER OF SCIENCE
IN

COMPUTER SCIENCE
BY
SUNDARRAMAN MADHUSUDANAN
(205322022)

TITUTE OF
INS7 TE
9010NW
ATIONA
RUC

DEPARTMENT OF COMPUTER APPLICATIONS

NATIONAL INSTITUTEOF TECHNOLOGY
TIRUCHIRAPALLI– 62015
Under the guidance of

Dr. GANGADHARAN G. R.
Assistant Professor

Department of Computer Applications

DECEMBER 2023
 BONAFIDECERTIFICATE
"Optimization of Neural Networks in the
This is to certify that the project

Tailoring the Best Optimizer" is a project work

Financial Domain:
(205322022) in partial
successfully done by Sundarraman Madhusudanan

of the requirements for the award of the degree of Master of

fulfilment

Institute of Technology, Tiruchirappalli, during

Science (CS) fromn National

Work).
the academic year 2023-2024 (3rd semester -CAS755
Mini Project

Dr. Michael Arock

Dr. G. R.

(Head of theDepartment)
(Project Guide) TRICHY
of the Department
529915 Dept.of Computer Applications
National Instituse of Technology

Tiruchirappalli - 620 015.

Tamilnadu, India.

Project viva-voce held on .08:12:.202.3

 ABSTRACT
study to optimize neural networks
This project embarks on a comprehensive
the choice and configuration of
within the financial domain by tailoring
and scale of financial data,
optimizers. In responseto the escalating complexity
optimization enhance the
strategies to
the project aims to develop customized
and resilience of neural network models. The
project
precision, efficiency,

encompasses an exhaustive of prevailing optimization algorithms and

review
a meticulous analysis of
their applicability in financial contexts. Through
(SGD), Adam, RMSprop,
popularoptimizers such as stochastic gradient descent
in addressing the unique
and others, the project seeks to discern their efficacy
aspect of the study
challenges posed by financial datasets. The empirical
financial datasets,
involves a series of experiments conducted on authentic
against standard
assessing performance of diverse optimizer configurations
the
contribute to the formulation
of
approaches. Findings from these experiments
practical guidelines for the selection
and fine-tuning of optimizers based on the
Ultimately, this project aims to furnish
specific needs of financial applications.
researchers with actionable insights
financial practitioners and machine learning
between neural network optimization and the
into the nuanced interplay
By tailoring optimizers to suit the distinctive
idiosyncrasies of financial data.

of the financial domain, the project aspires to push the boundaries

characteristics
accuracy
and
processes, fostering heightened
of modeling financial
amid the ever-evolving landscape of finance.
dependability in predictions
 ACKNOWLEDGEMENTS
We express our deep sense of gratitudeto Dr.G. Aghila, DirectorNational

Institute of Technology, Tiruchirappalli for giving us an opportunity to

work on this project.

Iwish to extend my sincere gratitude to Dr. Michael Arock, Professor and

of Computer Applications, National Institute of

Head, Department

Technology, Tiruchirappalli.

I sincerely thank my guide, Dr. Gangadharan G. R., Assistant Professor,

National Institute of Technology,
Department of Computer Applications,
encouraging me to carry out this
Tiruchirappalli, for supporting and

project work and duly evaluating my progress.

I also thank thereview panel for helping me identifyingthe shortages of the

guiding me.
me to overcome those shortages by
project and helping

my regards for all the faculty members of

would like to express

Finally, I

who have helped me

Applications and others
the Department of Computer
or indirectly.
develop this project directly

SUNDARRAMAN MADHUSUDANAN
Table of Contents
1. INTRODUCTION. 6
1.1 Financial Datasets in
Machine Learning 6
1.2 Convolutional Neural
Network ...... 6
1.3 Optimizers
2. LITERATURE REVIEW... ....8
2.1Literature Survey
3. METHODOLOGY. 10
3.1 Description of Dataset. 10
3.2 Convolutional Neural Network and Optimizers Used.. .10
3.3 Implementation of CNN using Python. 11

4. EXPERIMENTAL RESULTS.. 14
4.1 Performance with Loss Function MSE 14
4.2 Performance with Loss Function MAE 16

5. CONCLUSION AND FUTURE WORK.. ...... 18

5.1 Conclusion. 18
5.2 Future Work.. 18
6. BIBLIOGRAPHY 19
CHAPTER 1

INTRODUCTION
1.1
FINANCIAL DATASETS IN
MACHINE LEARNING
In the realm of machine
learning, the use of
financial datasets is
making data-driven decisions in crucial for
the complex and dynamic
These datasets, comprised of field of finance.
diverse financial information
such as stock prices,
economic indicators, and trading volumes,
serve as valuable inputs for
models. Financial data presents training
challenges like non-linearity and
high-frequency
fluctuations, requiring sophisticated modeling approaches. Applications of
machine learning in finance include predictive
modeling, risk management,
portfolio optimization,fraud
detection, and algorithmic trading. This project
tries to explores the unique characteristics of
financial datasets and sets the stage
for further discussionon optimizing neural networks for
enhanced performance
in financial applications.

1.2 CONVOLUTIONAL NEURAL NETWORK

ConvolutionalNeural Networks (CNNS) are a formidablecategory of artificial

neuralnetworks specifically crafted for tasks involving visual perception, such
as image recognition, object detection, and video analysis. They have
significantly advanced the field of computer vision by introducing specialized
architectures adept at capturing spatial hierarchies and intricate patterns within
data. CNNs leverageconvolutional layers to systematicalyscan input data using
filters or kernels, extracting features through convolution operations. These
slide across the input,enabling the network to recognize local paterns.
filters

Following convolutionallayers, pooling layers typically max pooling reduce

dimensions, preserving essential information while enhancing
spatial
translational invarianceand reducing computational complexity.

Non-linear activation functions, such as Rectified Linear Unit (ReLU), introduce

non-linearity to the network, facilitating the learning of complex relationships

within the data. Fully connected layers follow the convolutional and pooling

layers to make predictions based on the extracted features, connecting every

neuron to every neuron in thepreceding and succeeding layers.

Page 6 of 19
CNNs employ weight sharing,
using the same set of parameters across different

regions of promotes the detection of similar patterns throughout

the input. This
the input space. Through the combinationof convolutional and pooling layers,
CNNS automatically learn hierarchical representations of features, progressing
from simple patterns like edges and textures to complex structures.

1.3 OPTIMIZERS

Optimizers are essential components in the training of machine learning models,

playing a crucial role in adjusting model parameters to minimize the error or loss
function. The optimization process is pivotal for enhancing the performanceand

convergence speed of machine learning algorithms. Various optimization

algorithms have been developed,each with unique characteristics and strengths
tailored to different scenarios.

In the context of machine learning, optimization refers to the iterative process of

model parameters to reduce the difference between predicted and

updating
is to find the optimal set of parameters that result in
actual outputs. The objective

a model capable of making accurate predictions on new, unseen data.

components in the training of various machine learning

Optimizers are crucial

models, including neural networks, and their selection can significantly impact
in optimization algorithmsaims to
the model's performance. Ongoing research
and adaptable to different types of
develop techniques that are robust, efficient,
data and model architectures.

Page 7 of 19
CHAPTER 2
LITERATURE REVIEW

In [1], they have taken 3 unrelated data sets (cats and dogs, Fashion MNIST,
Natural Images) and tested it on a ConvNet with 3x3 filters across each layer
and with the following 7 optimizers: SGD (vanilla, with momentum, and
nesterov). RMSProp, Adam, Adamax, Adagrad, Adadelta, and Nadam. The
optimizers were judged on validation accuracy and loss. Nadam was the better
optimizer infirst aand third data set whereas adam performed better than Nadam
in the second data set.

In [2], they have taken four hyperspectral remote sensing data sets: Kennedy
Space Center (KSC),Indian Pines, University of Pavia and Salinas and tested it
on a CNN with 2x2 filters across each layer and with the following 7 optimizers:
SGD, Adam, Adadelta, Adagrade, RMSprop, AdaMax, and Nadam. The
optimizers were judged on accuracy, precision, recall, Fl-score, convergence
speed and stability during training. Adam was the best optimizer on the data set.

In (3], they have tested the data on a (LeNet) CNN with the following 7
set

optimizers: Stochastic Gradient Descent (SGD), Adaptive Gradient (Adagrad),

Adaptive Delta Mean Square Propagation (RMSProp),
(Adadelta), Root
Adaptive Momentum (Adam),Adaptive Max Pooling (Adamax), and Nesterov
AdaptiveMomentum (Nadam). Adam is the best optimizer to improve handling
for
a digital image-pigments content relationship. However, when the resources
is a wise choice to
experimentation is limited, using Adadelta and Adamax

minimize risk.

on a skin cancer using the ISIC standard data set for skin cancer
In 4], tests

detection were applied using three common optimizers (Adaptive Moment,

SGD, and Root Mean Square Propagation)to explore the effect of the algorithms
on the skin images. The second data set is COVIDx CT images. The second data
set was tested with the following 10 optimizers: Stochastic Gradient Descent

(SGD), Stochastic Optimization Descent with Momentum, Rung Kutta,

Adaptive Learning Rate, Root Mean Square Propagation, Adaptive Moment

Estimation, DeepEnsembles, Feedback Alignment, Direct Feedback Alignment,
Adfactor, AMSGrad, and Gravity. The optimizers were judged on accuracy.
Adam wasthe better optimizer in both the data sets.

Page 8 of 19
Rascd on the previ0Us rescarch, it's clear that most studies have primarily
focused on optimizing Convolutional Neural Networks (CNNs), mainly in the
healthcare scctor, with no more than two data sets. We will start with an
domain like finance and use 2 distinct We will
unexplored data sets for analysis.
apply the previously mentioncd optimizer options to determine the best choice
fjor the finance domain,

Page 9 of 19
CHAPTER 3
PROPOSED METHODOLOGY
3.1 DESCRIPTION OF DATASETSs
Dataset 1: Sales Prediction Datasct

This dataset provides acomprehensive historical record spanning from January

2013 to October 2015, encapsulating daily sales data. Within this dataset, there
are a total of 1,048,575 entries, cach capturing specific details related to

Each entry comprises key information, including

individual sales transactions.

the date of sale, a unique shop identifier ranging from 0 to 59,an item identifier,

the correspondingitem's price, and the quantity of items sold.

Dataset 2: US Funds Dataset

comprehensive compilation
offers a of financial information
This dataset
of
from Yahoo Finance, encompassing a comprehensive list
U.S.
sourced
details about the mutual funds
Mutual Funds. The dataset not only includes
for a multifaceted
themselves but also provides historical price data, allowing
The historical prices associated with each
exploration of the financial landscape.
offer a temporal dimension to the analysis. This
mutual fund in the dataset
allows for the study of how the value of each
historical pricing information
the identification of trends,
mutual fund has evolved over time, enabling

and potential correlations with broader

market movements.
patterns,

3.2 CONVOLUTIONAL NEURAL NETWORK ANDOPTIMIZERS USED

The 7 Optimizers used for analysis are SGD, RMSprop, Adadelta, Adagrad,
Adam, AdaMax and Nadam.
a
In the time series analysis, the goal is to predict future sales using
first

Convolutional Neural Network (CNN). Time series analysis involvesexamining

equally spaced intervals, such
data points collected or recorded over successive,

as daily, monthly, or yearly. The use of a CNN suggests an exploration of

and dependencieswithin the sales data.
temporal patterns

Page 10 of 19
lo the second analysis, the focus is on conducting a time series analysis using a
Neural Network (CNN) applied
Convolutional to the second dataset, which
financial information from
comprises Yahoo Finance, specifically focusing on
U.S. Mutual Funds. The objective is to predict the value of Mutual
opening
Funds, a task well-suited for tinme series analysis given the temporal nature of

data.
financial

Thechosen model architecture involves a CNN with specific characteristics:

3ConvolutionalLayers: These layers are responsible for extracting hierarchical

features from the input time series data. Each convolutional layer employs filters

to detect patterns at different levels of abstraction.

Layer: The pooling layer follows the convolutional layers and

1 Pooling
reduces the spatial dimensions of the data, enhancing computational efficiency

while preserving essential temporalpatterns.

20 Epochs: The model is trained over 20 epochs, where each epoch represents a

during training. The number of epochs

complete pass through the entire dataset
dataset and adjusts its
determines how many tines the model sees the entire

parameters accordingly.

3.3 IMPLEMENTATION OF CNN USING PYTHON:

The implementation of the Convolutional Neural Network

(CNN) for time
series analysis involves the utilization
of several popular Python libraries. Each
the overall workflow,contributing to
of these libraries plays a distinctive role in

from data manipulation and visualization to model construction

tasks ranging

and evaluation. The key libraries used in this implementation include NumPy,

Pandas, Matplotlib, Seaborn, Scikit-learn, and Keras.

NumPy: NumPy is a foundationallibrary for numerical operations in Python. In

the context of the CNN implementation, NumPy is likely used for handling
aays and matrices, performing mathematical operations, and managing
numerical data structures efficiently.

Page 11 of 19
 Pandas
is a powerful data manipulation and analysis library. It

Pandas: data structures like DataFrames, which are instrumental for organizing

provides time series data. Pandas facilitates tasks such as loading

manipulating
and
handling missing values, and performing exploratory data analysis.
datasets.

static,
Matplotlib is a versatile plotting library used for creating
of the CNN
Matplotlib:
and interactive visualizations in Python. Inthe context
series data,
animated,
Matplotlib is likely employed to visualize time
implementation,
performance metrics, and generate plots that aid in understanding
model
diplay data
in the sales
the patterns

data visualization library based

on Matplotlib.
Caahorn: Seabornis a statistical

interface for drawing attractive and informative

B nrovides a high-level
the visual appeal of plots, making it
ntistical graphics. Seaborn can enhance
time series data and model
complex information about
the
Psier to convey

performance.

learning library that offers

a wide range
Scikit-learn is a machine
Scikit-learn:
analysis. In the context of the CNN
of tools for data mining and data

may be used for tasks such as data preprocessing,

implementation, Scikit-learn for
provides a consistent interface
It
feature scaling, and model evaluation.

various machine leaming algorithms.

and capable
Keras
networks APIwritten in Python
is ahigh-level neural
Keras:
Cognitive Toolkit. In
ot running on topof
TensorFlow, Theano, or Microsoft
for
as the deep learning framework
he CNN implementation,Keras serves
It provides an abstraction
COnstructing andtraining the neural network model.
complex neural network
layer of building
that
simplifies the process
architectures, including CNNs.

Page 12of 19
 libraries allows for a streamlined and efficient
The combined use of these
series analysis. NumPy and Pandas
implementation of the CNN-based time
data
Matplotlib and Seaborn aid in
handle data manipulationand preprocessing.
visualization, Scikit-learn facilitates
machine learning tasks, and Keras provides
constructing and training the CNN model. This
a convenicnt interface for
collaborative use of libraries ensures
a cohesive and effective workflow for

and building predictive models.

analyzing time series data

Page 13 of 19
 4
CHAPTER
RESULTS
EXPERIMENTAL
with loss function MSE.
Performance
4.1P

dataset,
the following mean squarederror values were obtained when
first
Inthe
the mean squared error as the loss function.

employing

MSE for Different Optimizers

le7

1.2
-
102585569.00

10
(MSE)
Error
0.6
Squared
Mean
2620793.00
2086934.10
1956458.50
0.2 1465390.50
1241828.00
1011876.80
0.
Adam SGD AdaGrad RMSprop AdaDelta
Nadam Adamax
Optimizer Name

Fig: 4.1.1 Daily sales prediction using MSE

Page 14 of 19
 dataset, the following mean squared error values were obtained
In the second
en employing the mean squared error as the loss function.

MSE for Different Optimizers

24.2327
25

20

15.2506
(MSE)
15
Error
10.4709
10.0522
Squared
10
Mean
3.6564 3.7418
5 3.5946

Adamax Nadam Adan RMSprop SGD AdaGrad AdaDelta

Optimizer Name

Fig:4.1.2 Mutual Funds Predictionusing MSE

Page 15 of 19
 A2Performance with loss functionMAE.

In the first dataset, the following mean absolute error values were obtained when
employing the mean absolute error as the loss function.

MAE for Different Optimizers 2994.3Z82

3000

2500

2000
(MAE)
Error
1500
Absolute
897.6266

786.6513
8115195
1000 763.6675 768.4765
Mean
739.5482

500

S6D FMSprop Nadam AdaGrad AdaDelta

Adam Adamax
Optimzer Name

Fig:4.2.1 Daily sales prediction using MSE

Page 16 of 19
 dataset, the follovwing mean absolute error values were obtained
Inthe second
when employng the mean absolute error as the loss function.

MAE for Different Optimizers l8564

1.75

1.50

11948
L25 1.1043
(MAE)
Error
0.8737
1.00 0.8480

0.7195
0.6890
Absolute
0.75 -

Mean

0.50

0.25

0.00
Adam Adamax RMSprop Nadam AdaGrad SGD AdaDelta

Optimizer Name

Fig:4.1.2Mutual Funds Prediction using MAE

Page 17of 19
CHAPTER S
cONCLUSION AND FUTURE WORK
5.1 Conclusion

encounters the issue of a diminishing learning rate,nearly halting the

AdaDelta
the three optimizers, namely
leaming process after initialiterations. Conversely,
momentum concept from
Adamax, Adam, and Nadam, integrate the
the adaptive learning rate of AdaDelta, effectively
SGD/RMSprop with
of a diminishing learning rate.
addressing the challenge

adjustments to
time-varying processes, where continual
scenarios involving and
are desirable, Adamax, Adam,
In

learning rates based on data characteristics learning

choices. In such contexts,
a constant
Nadam emerge as the most fitting
over the diminishing
preferred
by RMSprop and SGD,
is
rate, as offered

rate characteristic
of AdaDelta.
learning

the loss function in both

When Mean Squared Error (MSE)as
utilizing the
demonstrated the most favorable
datasets, Nadam and Adamax consistently
suit.
Adam closely following
outcomes, with

(MAE) as the loss function

the Mean Absolute Error
In the case of employing performance, with
exhibited superior
across both datasets, Adam consistently

trailing closely behin.

Adamax
dataset considered,
loss function employed or the
of the specific
Regardless results.
the least favorable
AdaDelta consistently yielded

5.2 Future Work

financial datasets to
Neural Networks on additional
Executing Convolutional
Nadam as optimal optimizers and
to assess
Confirm the efficacy ofAdamax and
significantly.
Whether one outperforms the other

to datasets from diverse domains,

APplying Convolutional Neural Networks
the objective of identifying the most effective
ueh asweather forecasting,with
domain.
optimizers tailored to each specific

Page 18 of 19
BIBLIOGRAPHY
[]"A Comparative Analysis of Gradient Descent-Based Optimization

on Convolutional
Algorithms Neural Networks" E. M. Dogo, O.J.Afolabi.
NLNwulu, B.Twala, C.0.Aigbavboa Department of Electrical and Electronics
Engineering Science,Department of Construction Management and Quantity
Survey,University of Johannesburg,Johannesburg, South Africa.

121Somenath Bera & Vimal K. Shrivastava (2020) Analysis of various

of
optimizers on deep convolutional neural network model in the application
hyperspectral remote sensing image classification, International Journal of

Remote Sensing, 41:7, 2664-2683, DOI: 10.1080/01431161.2019.1694725.

H. P. Brotosudarmo, S. Anam, A. Suryanto; Performance

13]K. R. Prilianti, T.
neural network optimizer for photosynthetic
comparison of the convolutional
image. AIP Conf. Proc. 22 March 2019;
pigments prediction on plant digital
2084 (1): 020020. https://doi.org/10.1063/1.5094284
N.A. et al. The effect of choosing
[4]Hassan, E., Shams, M.Y., Hikal,
study.
vision tasks: a comparative
optimizeralgorithms to improve computer
(2023).https://doi.org/10.1007/s11042
Multimed Tools Appl 82, 16591-16633
022-13820-0

Page 19 of
19