Recommendation System Using Deep Learning

RAGHAVENDRA C K , GAGANA DHANANJAYA , LAVANYA D S ,

2022 IEEE 7th International Conference on Recent Advances and Innovations in Engineering (ICRAIE) | 978-1-6654-8910-2/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICRAIE56454.2022.10054275

PRIYANKA LAKUR KRISHNAMURTHY , ANIKA SHREYA PAWAR

Computer Science & Engineering, BNM Institute of Technology,
Bangalore, Affiliated to Visvesvaraya Technological University,
Belagavi , Karnataka 590018, India
0000-0002-0321-0674, [email protected], [email protected],
[email protected] , [email protected]

Abstract—Recommendation systems are a commom feature hidden and output layers, aswell as a matrix of weights
of many services nowadays. Using a significant quantity of user between the layers, are kept across all instances of the network.
history that has been stored, this system forecasts what a user A single user's rating for each item is fed into the input layer
will use next. Recommendation systems are frequently used in a of the network during training. In case of DeepNeural
variety of industries, including e-commerce, social services and Network(DNN) input layer, embedded layer and flatten layers
movies. We are attempting to compare three distinct deep are defined to generate vectors to movies and users and
learning models—RBM, Autoencoder, and DNN and comparing predictions are made by fitting the vectors to the Keras model.
their efficiency in order to determine which model will perform
best given the supplied dataset. We conducted experiments on II. RELATED WORK
real world data like MovieLens, which has four csv files to
compare the efficacy of different Deep Learning models. These [1] Byeongjin Choe et al have described that the significance
experimental results show that the query features and item of item usage order and time intervals in user time series data,
features which Deep Neural Network can readily incorporate however, was frequently ignored in earlier studies on
may help identify a user’ s interests and increase the relevancy recommendation systems. Incorporating these temporal
of recommendations. characteristics of the user history, the paper offers a novel
recommendation algorithm. In order to take into account the
Keywords—Deep Neural Network, Restricted Boltzmann order and time intervals of the user's item usage history, they
Machine, Autoencoder, Collaborative filtering, Root Mean Square have created a recurrent neural network (RNN) model with a
Error. hierarchical structure.
I. INTRODUCTION [2] Robin Devooght et al demonstrate that the challenge of
Recommendation algorithms have recently proliferated collaborative filtering can be interpreted as a sequence
across a wide range of websites, including e-commerce. Based prediction problem, and that recurrent neural networks
on the user information, a recommendation system makes provide a very competitive solution. We focus on the
suggestions for things that the users might want to purchase. collaborative filtering capabilities of the long short-term
The secret to a recommendation system's effectiveness is in memory (LSTM) and how it performs against more traditional
particular tailoring it to the consumers. In order to forecast methods for movie selection like nearest neighbours and
user behaviour and provide more effective recommendations, matrix factorization. We demonstrate that the LSTM is very
many different recommendation systems have been put forth. competitive across the board and significantly beats
Typically, recommendation systems analyse historical usage competing approaches in terms of item coverage and short-
data to forecast users' personalised preferences and identify term predictions.
things with the highest desire. Deep learning is used in apps Machine learning algorithms have been used in a variety of
geared towards consumers, like the Siri and Cortana assistants, recommendation system from a long time. But as and when
Google Photos, the tagging tool, Grammarly's proof reader, the scale of the data increases, deep learning algorithms play
and Spotify's song suggestions. In order to better segment a better role in providing the recommendations to the users.
customers, anticipate attrition, and uncover fraud, it is also The reason for going with the deep learning model is because
employed in CRM systems, social media, other online of the feature engineering, which is done implicitly, self-
platforms, and other inline data. learning capabilities which would provide an added advantage
Each unique user in our training data is used as a set of for building a model with higher accuracy without any manual
inputs into the RBM model, which is the general notion behind intervention, higher scalability and highlevel non-linear
it. Inorder to process each user individually as part of a batch characteristics required for precise categorization can be
during training, it is necessary to look at their ratings for each learned.
movie they have seen. As a result, the visible nodes represent III. PROBLEM STATEMENT
the ratings for a specific user for each movie, and we are
working to acquire weights and biases that will enable us to To develop a recommendation system using deep learning
rebuild ratings for user-movie pairs that we are still learning models namely Deep Neural Network (DNN), Restricted
about. In the case of autoencoders, a bias node for both the Boltzmann Machine (RBM) and Autoencoder and comparing

978-1-6654-8910-2/22/$31.00 ©2022 IEEE

99
Authorized licensed use limited to: AMC Engineering College ¿ Karnataka. Downloaded on April 21,2025 at 05:50:43 UTC from IEEE Xplore. Restrictions apply.
the efficiency of these models for movieLens ml-latest-small
dataset. Figure 2: RBM Model

Since the buried layer neurons relate to input features

IV. MODEL ARCHITECTURE while the visible layer represents input and reconstructions,
the visible layer is often larger than the hidden layer. This
indicates that information flows in both directions during
network usage and network training, and those weights are
constant. Predictions are made differently in RBMs than they
are in conventional feed forward neural networks.

RBM simply converts the inputs into a set of integers that

represent them. The inputs can then be recreated by
translating these numbers back. The RBM will be educated
through numerous forward and backward passes, and a
trained RBM can identify which elements are most crucial for
pattern detection. Once the network has been trained, we may
use it to classify newly discovered data (this is known as
unsupervised learning). Three steps are repeated during the
training process: the forward pass, the backward pass, and the
quality assessment.

The system loads the movieLens dataset, and popularity

rankings are produced. The sum of all the ratings for a movie
Figure 1 System Model is added to get the rankings, which are then compared to other
movie sums. The dataset is separated into training and test set
The Figure 1 depicts the model architecture of the system. using a leave-one-out split, where in all data other than the
• Dataset: MovieLens ml-latest-small dataset is test sample is regarded as a training sample. The cosine
used. similarity matrix is employed to assess how diverse the data
is when compared to similarity of the system's genre. For
• Pre- processing : One-hot encoding is used for Contrastive Divergence, Gibbs Sampling is used; in
digitising the data. Unwanted columns and csv accordance with the paper, we assume k=1 rather than
files are removed. repeatedly iterating over the forward pass because it appears
• Exploratory Data Analysis : Overall analysis of to function perfectly. Examples of hidden layers are revealed.
data is carried out in this part like range of movie For hidden probabilities, tensors are produced, and a 2D
years, ranges of rating and such like. matrix connects every sample distribution produced. As
shown in Figure 2 the hidden layer sample is used to recreate
• Model Construction : Three Models have been the visible layer. Rating vectors are created, and then they are
considered for deep learning recommendation flattened once more using the softmax activation function.
system. The models are RBM, Autoencoder and Running forward and backward passes updates both hidden
DNN.
and obvious biases. Following the generation of the graphs
The motive is to predict the best recommending model out for each of the tuned, untuned, and random RBM
of the three models. Different evaluation metrics like RMSE, recommendations by forecasting the movielist while
MAE, HR, CHR have been evaluated. adjusting the model to the dataset, recommendations are
produced. The evaluation metrics calculated are RMSE,
MSE, HR, CHR, coverage, diversity and novelty.
A. Restricted Boltzmann Machine
B. Autoencoder
Restricted Boltzmann Machine is a stochastic neural
network (that is a network of neurons where each neuron has A particular kind of feed forward neural network called
some random behaviour when activated). It is a neural an Autoencoder uses input and output to be identical. The
network with two layers. There are no internal connections output is then rebuilt from this representation once the input
between the layers; they are completely connected to one has been reduced in size through compression. The code,
another. The output layer is concealed, while the visible layer often referred to as the latent-space representation, is a
is another name for the input layer. compressed "summary" of the input. Three parts make up an
AutoEncoder - an encoder, a code, and a decoder. The input
is compressed by the encoder, which also creates a code. The
decoder uses this code to reconstruct the input. Three layers
are present: a hidden layer, an output layer with our
predictions, and a bottom input layer with individual ratings.
Every instance of this network maintains a bias node for the
hidden and output layers as well as a matrix of weights
between the layers. Each item's evaluations from each user

100
Authorized licensed use limited to: AMC Engineering College ¿ Karnataka. Downloaded on April 21,2025 at 05:50:43 UTC from IEEE Xplore. Restrictions apply.
are fed into the input layer of the network once it has been overcome these drawbacks of matrix factorization. Due to the
trained. On the output, a sigmoid activation function was network's input layer's versatility, DNNs may readily add
applied. By taking into account just observed ratings, the data query features and item features, which can help identify a
sparsity is simply handled. As a result, they took care user's individual interests and increase the relevancy of
processing each stage of this neural network independently, suggestions. In a thick layer, a typical fully connected neural
propagating only information from ratings that were actually network layer, each input node is connected to each output
available in the training data, and ignoring the contribution of node. Like a dropout layer, but with some random nodes'
input nodes that correspond to missing data from user-item activations being set to zero when the layer is used. This helps
to avoid over fitting. The embedding layer allows us to convert
pairings that weren't evaluated at all. In TensorFlow, it's still
each word into a fixed-length vector with a predefined size.
challenging to accomplish this.
The resulting dense vector contains not just 0s and 1s, but also
real values. Word vectors' fixed length and decreased
The movieLens dataset is loaded into the system and dimensions enable us to express words more effectively. The
popularity rankings are generated. The rankings are generated multidimensional input tensors are flattened into a single
by adding all the ratings to a movie and comparing it with dimension by the flatten layer, in order to successfully
other movie ratings sum. The dataset is divided into training transmit the data into each and every neuron of the neural
and test set which is done by using leave-one-out split network model after modelling your input layer and creating
according to which apart from the test sample rest all other your neural network model.
data is considered as training sample. The cosine similarity
matrix is used to measure the diversity in data i.e. to check The DNN model used for the dataset initially involves creation
of movies vector and ratings vector. The vectors are created
the similarity of genres between the systems. The hidden
by initializing the tensors by passing the dimension of the
layer and epochs are defined before evaluating the model.
input content and other layers are defined for the initialized
When the model is fit to the dataset the ratings are taken tensor in order to vectorize the movies and users.
between the range of zero and one instead of 0-5 in order to
get effective predictions. Sigmoid function is used to decide
the range of ratings between 0-1.Inorder to give out the
recommendations the biases and weights has to be defined to
each of the layer and optimizations are done to these layers to
reduce the gradient. . Recommendations are given by
predicting the movielist fitting the model to the dataset. The
evaluation metrics calculated are RMSE, MSE, HR, CHR,
coverage, diversity and novelty.

Figure 4 Epochs v/s Train Error

As shown in Figure 4 as epoch layers increases the training

error decreases. The input data is converted to single
dimension using flatten layer. The vectors generated are used
to fit the Keras model and predictions are made to test the
accuracy.
Figure 3: Autoencoders V. IMPLEMENTATION
There is no straightforward way to limit the series of A. Restricted Boltzmann Machine
matrix multiplications and additions required to create a
neural network to just the input nodes with actual data in them.
RBM (v, h, l, size) :
Tensor Flow does provide sparse tensors, though. Any Tensor
/* Input :
Flow or Keras implementation you find simply ignores that v : visible dimension,
issue and models missing ratings as zeroes. With enough work, h : hidden dimension,
we can still achieve respectable results, but the application of L : learning rate,
deep learning to recommender systems has some very serious size: batch size */
flaws. An Autoencoder is the name given to this architecture. //Output : n Recommended Movies
Increasing the weights and biases between the input and Model=RBM(v,h,l,size,epochs)
hidden layer is the process of encoding the input. In that /* The training matrix is generated by creating
hidden layer, as shown in Figure 3 we are encoding the input matrix which involves user and item relationship */
patterns as a set of weights. The output is then decoded as we /*The ratings are taken in range of 0-1
reconstruct it using the weights between the hidden and output considering cosine similarity */
layers. The encoding step is represented by the first set of
weights, and the decoding stage by the second set. maxWeight = -4*
C. Deep Neural Network
weight, hiddenBias, visibleBias← define
The difficulty of utilising side characteristics, or features using v, h
other than those included in a query or item ID. As a result, //Use Sigmoid and Relu Functions for
only users or items from the training set can be used to query Forward ← input user attribute and
the model. Models based on deep neural networks (DNNs) can Backward ← transpose(forward)
//recommendations
for(userId,ratingsin items():
return top n recommendations
101
Authorized licensed use limited to: AMC Engineering College ¿ Karnataka. Downloaded on April 21,2025 at 05:50:43 UTC from IEEE Xplore. Restrictions apply.
 Where D is the dataset and ej,k is the actual rating e′j,k is
predicted rating and |D| is test data.
Mean Absolute Error (MAE) is the averageof absolute errors.
B. Autoencoder Absolute error is calculated as differences between predicted
value and actual value.

∑ | −
Autoencoder (v, h, l, size) :
/*input : v : visible dimension, MAE = | (2)
h : hidden dimension,
l : learning rate Where is predicted rating and is actual rating and n is
total number of observations.
size : batch size */
//output : n recommended movies
Model=Autoencoder(v,h,l,size)
weights ← (visible → hidden) for
ALGORITHM RMSE MEAN
ABSOLUTE
embedding and (hidden → output) for
ERROR
decoding stages randomly initialized
biases ← biases for h and Restricted 1.2808 1.0867
vinput_layer ← input_user Boltzmann
hidden ← activate sigmoid function Machine
output_layer ← predictions
//recommendations Autoencoder 2.0127 1.6638
For user,ratings in items():
Deep Neural 0.879 0.6740
Return top n recommendations Network

Table 1 Results

C. Deep Neural Network

Results
DNN () : 2.2

input_layer ← Input(shape[1]) 1.65

1.1
embedding_layer ← Embedded(n_mo vies + 1,
n_latent_factors_movie) (movie_input) 0.55

rec_vector←Flatten()(embdding_la 0
yer) RBM Autoencoders DNN
RMSE MAE
input_user_layer ←Input(shape=[1]) Column1
embedding_user ← Embedded(n_user s + 1,
n_latent_factors_user) (input_user_layer) Figure 5 Comparison Graph
user_rec_vector←Flatten()(embedding_u
ser)
Table 1 and Figure 5 give the comparison among three models
result ← keras.layers.Dense(activatio using RMSE and MAE. By comparing the RMSE values it is
n='relu',name='Activation')(dense_4) seen that DNN model is giving better accuracy for
recommendations made on dataset.
dnnModel ← keras.Model([movie_in put,
user_input], dnnResult)
model.predict([test.movieId,test.userId],recommenda VII. CONCLUSION AND FUTURE WORK
tions_for_id)
In this paper, we present a novel recommendation system
that compares various Deep Learning Models namely Deep
Neural Network, Restricted Boltzmann Machine and Auto
VI. PERFORMANCE EVALUATION encoder. We demonstrate that the Deep Neural Network
Root Mean Square Error (RMSE): Estimatesthe rate of model, one of the models under consideration in this study,
error while predicting a not rated item for an active user. The outperforms many baseline models derived from trials. In our
equation is asfollows. upcoming work, we'll examine how convolutional neural
networks and other deep learning techniques are used in
′
∑ , ∈D| , , |
recommendation systems in an effort to enhance their
2 performance.
| |
RMSE = (1)

102
Authorized licensed use limited to: AMC Engineering College ¿ Karnataka. Downloaded on April 21,2025 at 05:50:43 UTC from IEEE Xplore. Restrictions apply.
 [6] Raghavendra C K, Srikantaiah KCandVenugopal K R. Personalized
recommendationsystem(PRES): A Comprehensive StudyandResearch Issues,
VIII. REFERENCES International Journal of Modern Education and Computer Science(IJMECS),
Vol.10, No.10, pp. 11-21, 2018.
[7] Raghavendra C K and SrikantaiahKC. Switching Hybrid Model For
[1] Byeongjin Choe, Taegwan Kang and Kyomin Jung; “Recommendation PersonalizedRecommendations by Combining UsersDemographic
System With Hierarchical Recurrent Neural Networks for Long term time Information, Journal of Theoretical and Applied InformationTechnology, Vol.
series”, IEEE, 2021 100, Issue 3, 2022.
[2] Robin Devooght, Hugues Bersini; “Collaborative Filtering with Recurrent [8] Raghavendra C K and SrikantaiahKC. Weighted Hybrid Model for
Neural Network”, ACM 2017. ImprovingPredictive Performance of RecommendationSystems Using
Ensemble Learning, IndianJournal of Computer Science and Engineering, Vol.
[3] Libo Zhang, Tie-in Luo, Fei Zhang and Yanjun WU; “A Recommendation
13, No.2, pp. 513-524, Mar-Apr 2022.
Model Based on Deep Neural Network”, IEEE, 2017
[9] Raghavendra C K and SrikantaiahKC. Similarity Based Collaborative
[4] Lin Wang, Haishu Tan, Fuqiang Zhou, Wangxia Zuo and Pengfei Sun;
FilteringModel for Movie Recommendation Systems, 5th IEEE International
“Unsupervised Anomaly Video Detection via a Double-Flow ConvLSTM
Conference onIntelligent Computing and Control Systems(ICICCS), Vaigai
Variational Autoencoder”, IEEE, 2022
College of Engineering, Madurai, India, May 6-8, 2021.
[5] Balaz Hidasi, Alexandros Karatzoglou, Linas Baltrunas, Domonkos Tikk;
“Session Based Recommendations with Recurrent Neural Networks”, ICLR,
March 2016.

103
Authorized licensed use limited to: AMC Engineering College ¿ Karnataka. Downloaded on April 21,2025 at 05:50:43 UTC from IEEE Xplore. Restrictions apply.