International Journal of Advanced Network, Monitoring and Controls Volume 08, No.

01, 2023

Product Recommendation System Based on Deep

Learning

Ping Lu Pingping Liu

School of Computer Science and Engineering School of Computer Science and Engineering
Xi’an Technological University Xi’an Technological University
Xi’an, 710021, China Xi’an, 710021, China
E-mail: [email protected] E-mail: [email protected]

Abstract—With the development of Internet big data enterprises are also involved in the wave of the
and e-commerce, the widespread popularity of Internet and build relevant platforms with the help
information, information acquisition and personalized of the Internet, such as shopping, video, news,
recommendation technologies have attracted extensive
attention. The core value of personalized
music, marriage, social networking and other types.
recommendation is to provide more accurate content However, the highly developed number of goods
and services around users. The recommended scenarios also brings a huge problem, that is, how to flexibly
are not uniform, and different dimensions need to be help users to accurately meet their needs. At the
considered. For example, we are facing enterprises or same time, under the influence of education and
individuals, different age groups, different levels of social environment, everyone has put forward new
education, social life and other aspects. In this paper, the requirements for self-expression. Self personality
classic DNN (Deep Neural Networks) double tower
recommendation algorithm in the recommendation
has become an important indicator of consumer
algorithm is used as the ranking algorithm of the demand. In the network environment, the variety
recommendation system. It is divided into user and item of complex goods and services and the difference
for embedding respectively. The network model is built of people's personalized needs together constitute
using tensorflow. The data processed by the initial data a dazzling choice. Under the combined effect of
through feature engineering is sent into the model for these multiple factors, the recommendation system
training, and the trained DNN double tower model is also came into being. If you want to overcome the
obtained. Recall adopts collaborative filtering algorithm,
above needs, personalized recommendation
and applies tfidf, w2v, etc. to process feature engineering,
so as to better improve the accuracy of the system and system is essential [1].
balance the EE problem of the recommendation system. The product recommendation system based on
The recommendation module of this system is divided deep learning has the following advantages and
into data cleaning as a whole. Feature engineering
includes the establishment of user portraits, the analysis
characteristics: efficiency, improve the efficiency
of multiple recall and sorting algorithms, the adoption of of users to obtain the required goods;
multiple recall mode, and the implementation of a classic Transparency, the user's imperceptibility in the
recommendation system with in-depth learning. This recommendation process will not affect the user's
makes the recommendation system better balance the use; Accuracy: the recommended results are
interests of both the platform and users, and achieve a consistent with the products required by users;
win-win situation. Novelty, recommend products in many ways, and
Keywords-Deep Learning; Recommendation System; break the inherent search deadlock.
Deep Neural Network At present, domestic and foreign
INTRODUCTION
I. recommendations mainly include three traditional
recommendation methods: content based
At present, the Internet is developing rapidly in recommendation, collaborative filtering
China. Information plays an increasingly recommendation, and hybrid recommendation, and
important role in people's daily life. Many four common deep learning recommendation

DOI: 10.2478/ijanmc-2023-0041 1
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

models: convolutional neural network (CNN), output, and the result is returned through Python
deep neural network (DNN), recurrent neural construction[3].
network (RNN), and graph neural network (GNN).
II. INTRODUCTION TO MACHINE LEARNING AND
At present, due to the continuous development DEEP NEURAL NETWORK
of machine learning technology and the deepening
of people's understanding of this technology, the A. Fundamentals of Machine Learning Theory
recommendation system has made new progress. The development of artificial intelligence has
Take Alibaba and Toutiao today, two companies produced a wide range of branches, and machine
with significant influence in China, for example. learning is one of the important branches that can
The two key technologies included in their not be ignored. If we want to realize artificial
recommendation systems are machine learning intelligence, we must not neglect the technology of
and deep learning. These two technologies machine learning. Under machine learning, the
together constitute their recommendation systems. algorithm that allows the machine to modify and
However, the traditional collaborative filtering and develop itself is adopted. At the same time, the
physical recommendation methods, which were computer obtains the corresponding rules in
more popular in the past, have faded out of continuous data analysis, and the obtained rules
people's vision. Such strategies are often difficult can be used to speculate on new sample data. If we
to solve the drawbacks of the long tail content's use formal language to describe its content. Its
refined personalized recommendation and definition should be: for a certain type of task T
popularity penetration. At the same time, the and performance measurement P, if a computer
benefits of the new recommendation methods are program's performance measured by P on T is self
dwarfed by the traditional ones. In 2017, the improving with experience E, it can be called
National University of Singapore proposed learning from experience E.
NeutralCF, the famous double tower structure, In machine learning, LR is one of the core
which uses neural networks to optimize the algorithms [4]. Logistic Region is also recognized
original collaborative filtering algorithm. Youtobe by all types of enterprises because of its simplicity,
and Google have successively modified the parallelization, and strong interpretability. The
proposed model. The traditional collaborative essence of logistic regression is to assume that the
filtering and LR mode recommendation algorithm data obey this distribution, and then use maximum
are gradually moving towards the double tower likelihood estimation to estimate the parameters.
model [2]. Logistic distribution is a continuous probability
distribution. Its distribution function and density
This paper designs a product recommendation
function are shown in (1) and (2):
system based on deep learning. The system
conducts model training through the product data
1
and user data in the data set. The main contents F x  P X  x  x μ

include: first, feature engineering is carried out, 
1e γ
and basic product data, user data, and user
behavior data are processed to build user portraits x μ

and item portraits; Then, the model is constructed e γ

and trained. The DNN double tower model is f  x   F  X  x   2

constructed using tensorflow, and the data set is  

x μ

γ 1  e γ

imported for model training; The next stage is the 
 
commodity recall stage, which specifies the
quantity of commodities for use in the subsequent Logistic regression essentially uses the
sorting stage; The next step is to sort the goods. predictive value of the linear regression model to
The recalled goods are sent to the model for approximate the logarithmic probability of the real
sorting to get the final result; Finally, the result is mark of the classification task. Its benefits lie in
that: it directly models the probability of

2
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

classification without realizing the hypothetical neural network layers is mainly determined by the
data distribution, which prevents the disadvantages number of hidden layers. As shown in Figure 2.3,
caused by inaccurate distribution (different from it is a single-layer neural network. This neural
the generative model); Not only the category can network has 4 neurons in the input layer, 5
be predicted, but also the probability of the neurons in the hidden layer, and 2 neurons in the
prediction can be obtained[5]. When encountering output layer [8].
some kind of tasks that use probability to assist Input layer Hidden layer Output layer
decision-making, it can often play a huge role; The
logarithmic probability function is a convex
function with arbitrary order derivability, and
many kinds of numerical optimization algorithms
can calculate the optimal solution. Both gradient
descent and Newton iteration can be used to solve
the logic regression, and the gradient descent
method is briefly introduced:

1  n  yilnp  xi  
 
J w     
  
n  i1 1  yi  ln 1  p  xi   


J  w 
gi 
wi
 
 p  xi   yi xi 
Figure 1. Neural network structure diagram.

The corresponding neural network needs to be

wik 1  wik  α gi  designed according to the specific problem to be
solved. Generally speaking, the number of neurons
Gradient descent is to find the descent direction in the input layer and output layer is fixed, while
through the first derivative of J (w) to w and the number of layers in the middle hidden layer
update the parameters in an iterative manner, can be adjusted freely. If compared with machine
where k is the number of iterations. learning, deep learning is more convenient in these
B. Theoretical Basis of Artificial Intelligence aspects:
In 1943, the neuron model was successfully 1) Stronger fitting ability of deep learning
drawn by psychologist Warren McCulloch and model
mathematician Walter Pits. It can be seen that Among feature crossing methods, dot product
there are many inputs on the left side of the figure and other methods are relatively simple, so that
below, similar to the dendrites of neurons [6]. when the sample data is more obscure and
After a nuclear processing, that is, the weighted complex, the phenomenon of under fitting often
sum part in the figure below, and then through the occurs. Deep learning can effectively solve this
activation function, an output is finally obtained. problem by improving the fitting ability of the
When several single neurons are combined, the model. Take NeuralCF (Neural Network
output of some neurons will be used as the input of Collaborative Filtering) model as an example, in
others, thus forming a neural network. If these this model, the point accumulation layer is
neurons are regarded as a whole, the structure has exchanged with the multi-layer neural network.
input and output. The layer receiving data input is Theoretically, multi-layer neural network can fit
called the input layer, and the layer outputting the any function, so as long as the number of neural
results is called the output layer[7]. The middle network layers is increased, the problem of
layer composed of middle neurons is called the insufficient fitting will be solved [9].
hidden layer or hidden layer. The number of

3
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

2) The structure of deep learning model is significance in the process of gradual weight
more flexible optimization of the network (random initialization).
The structures of deep learning models are The weight value can be optimized to help predict
different. Most of them can stack network layers the correct result [10].
with different functions. The simplest is a serial
structure, some like a mesh structure, some like a III. RESEARCH ON RECOMMENDATION
pyramid structure, and so on. The typical cases are ALGORITHM AND MODEL
Alibaba's DIN and DIE, which adopt a sequence A. Recall Algorithm
model with attention mechanism and interest
evolution simulation in the model structure to In the recommendation system, recall is the
achieve a better effect of simulating user behavior. first step. This algorithm mainly requires to select
a small amount of data from the huge item
C. Fundamentals of Deep Neural Networks information database quickly and accurately,
The neural network layer inside DNN can be depending on the user's relevant information and
divided into three different types: input layer, the characteristics of the products they are
hidden layer and output layer. In the absence of interested in. After the recall, the next step is the
special circumstances, the first layer is called the sorting process. In the recall process, huge data
input layer, the last layer is called the output layer, volume and high speed requirements are
and the number of layers in the middle is called significant features. Therefore, the strategies,
the hidden layer. models and features adopted by the algorithm
should not be too complicated. Data acquisition at
Each layer is closely connected. It can be said the recall stage needs to be carried out quickly.
that a neuron randomly selected from layer i must Multi way recall is generally a classic form, such
be connected to any neuron in layer i+1. The small as content-based collaborative filtering recall, user
local model is the same as the perceptron, that is, a based collaborative filtering recall and popular
linear relation input data is used by the neurons in recall, as shown in Figure 2.
the first layer (not hidden), then the output is
provided to the neurons in the next layer, and the Multichannel recall Sort
output is repeated until the final successful output.
The output has the possibility of yes or no Hot recall
(expressed in probability) prediction. Each layer
often has one or more neurons, and each neuron Interest tag recall
calculates a small function, that is, the activation LR
/LM
function. The activation function simulates signal Recall for people of /wide&deep
transmission to the next connected neuron. The age and region /DeepFM
/DIN
value of the afferent neuron will be compared with Collaborative
the threshold value. When the neuron value is filtering recall
large, the output can pass. The weights formed by .
.
the connection between two neurons of the .
continuous layer are related. The influence of Figure 2. Recall ranking diagram.
input on the next neuron and the final output is
defined by weight. In the neural network, the To be specific, multi-channel recall means that
initial and weight values are randomly generated. users can select the products they are interested in
Of course, in order to successfully predict the from the commodity library in different forms at
accurate output value, certain weights will be the same time, and the products selected from
updated in time. Decompose the network mainly multiple ways are summarized and sorted by the
by defining some logical building blocks, such as sorting model, and finally provided to users to
neurons, layers, weights, inputs, outputs, achieve personalized recommendations. The name
activation functions, and so on. At the end is the of multi-channel recall proves that the core of
learning mechanism (optimizer), which is of great multi-channel recall is "multi-channel". Such

4
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

multi-channel form ensures the maximum degree the corresponding sequence. In this case, it
of diversification of product content, and also is considered that this order is relevant, and
meets the needs of users to the maximum items with similar distances from each other
extent[11]. are also highly similar. The modeling of
this method depends on the similarity
This system will also adopt the collaborative
between the order of goods. Among them,
filtering algorithm based on items, and apply tfidf,
the method of modeling the item sequence
w2v, etc. to process feature engineering. The
according to word2vec is often adopted.
collaborative filtering based on items mainly
According to the difference of weight
depends on the weight of the items, selects the
vectors of different items, the Euclidean
items that users like, and then calculates the
distance between item weight vectors is
similarity between other goods in the item library
used to divide the similarity of items.
and the products that users like, and finally pushes
them to the similar products that users once liked.  Item based collaborative filtering combined
In addition, their methods for calculating the with TF-IDF method. This method refers to
similarity weight of items have different focuses, taking the user's browsing and purchase
mainly including the following categories: records as a word, while specific items
appear as letters in the word. The tf idf
 Click, purchase or download co cash as value of each item is not missed, and the
reference data to realize weight calculation. value reflects the user's preference for items.
It can be said that for a user who has Finally, you can rely on the item
purchased Class A, Class B and Class C collaborative filtering method to find
products at the same time. For article A, products that are highly similar to those
there are two co occurrence pairs (A, B) users like.
and (A, C). Another user has purchased
Class A and Class B products, so there is B. Sorting Algorithm
another co occurrence pair (A, B). The When some high-quality goods are selected
meaning of co occurrence is similarity to from the recall stage, the sorting stage is entered.
some extent. In this way, for article A, the The requirements of this stage are high, and users
similarity weight value of article B is are required to accurately recommend the products
1+1=2, and the similarity weight value of they like according to various materials. For
article C is 1. There may be many different sorting algorithms, fast tracking is an important
calculation methods for co occurrence. But requirement that cannot be ignored. Users need
the guiding significance of this thought is quick feedback, but they also need to implement
significant. accurate recommendations. In the final sorting
 The article weight is calculated based on stage, ctr estimation is performed on all goods, and
Word2Vec method. The sequence of items then topK data is delivered upwards [12].
is the key point of this method. When the The design of this paper in the sorting stage
user purchases, clicks or downloads, the adopts the double tower model, and the model
products that the user browses will generate architecture is shown in the figure below.

5
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

In-batch softmax

used for
L2 normalization L2 normalization
indexing

ReLU ReLU

ReLU ReLU

Candidate
seed features user features
features
video id channel id ... views likes past watches ... video id channel id ... views likes

... ...

vide id embeddings channel id embeddings

Figure 3. DNN double tower model architecture diagram.

The double tower model in this paper obtain the negative sample for training, which is
constructs the embedding on the request side and also the negative sample in this batch. However,
the embedding on the commodity side. The output this method may cause errors, making it possible
of the two towers, and the final model is the output for some popular samples to become negative
of their respective embedding vectors, which is the samples. Therefore, it is necessary to perform
result of the inner product of the two embeddings. logQ correction on the inner product calculated by
The inner product formula of the vector is as the two embedding vectors above, that is, to
follows: perform the embedding inner product correction.

s  x,y   u  x,θ  ,v  y,θ   sc  xi ,yj   s  xi ,yj   log  pj  

Based on the output of the softmax function Where, represents the sampling probability
and user preference weight, the loss function is in of commodity j randomly selected into batch.
the form of a weighted logarithmic likelihood Then use SGD to update the parameters. The
function, as follows: specific model training algorithm is as follows:
 Obtain a batch sample from the real-time
LT θ  : 
1
 r  log
T iT i
 
 yi ∣ xi;θ   data stream;
 The sampling probability pi is obtained
When there are a large number of commodities based on the sampling probability
in the commodity library, that is, (M is very large), estimation algorithm to be mentioned below;
it is very difficult to accurately calculate the  Calculate the modified loss function
softmax function above. In order to overcome this described above.
problem, it is obviously an appropriate method to IV. PRODUCT RECOMMENDATION SYSTEM
sample a full set of commodities. The traditional DESIGN
form is to sample the negative samples required
for training from a fixed set, however, the more The functional requirements of the system are
efficient method for the traditional form is to the tasks that must be established before the
sample a batch of data in the real-time stream and system development, which specify the functions

6
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

that the system must achieve and ultimately need used as the development tools for deep learning
to meet the requirements. Through the previous algorithms.
research and analysis, the modules of this system
A. Functional Requirement
are determined as user interface module, search
function module, recommendation function Define abbreviations and acronyms the first
module, and data processing module [13]. time they are used in the text, even after they have
been defined in the abstract. Abbreviations such as
During the development of this program, IEEE, SI, MKS, CGS, sc, dc, and rms do not have
pycharm is used as the IDE to write Python code. to be defined. Do not use abbreviations in the title
At the same time, Tensorflow gpu and Keras are or heads unless they are unavoidable.
Product recommendation system based on deep learning

User interface module Search function module Recommended function module Data processing module

User log
Basic
Search data
Recommended Search keyword Collect
function by Recall Sort processing
interface interface search user logs
id and
function
cleaning

Figure 4. Function module diagram.

This topic is to design and complete a product The recall part is a multi-channel recall.
recommendation system with complete functions. Currently, collaborative filtering is used, including
The program can recommend products to users in item based, user based, and model based recalls.
real time, and provide simple product search Matrix decomposition and failure are used.
function and simple display function. The 2) Sorting part (deep learning)
functional module is shown in Figure. First of all, the recall part will filter all
commodity IDs, and the IDs arriving at the sorting
 User interface module: log in and register, module are only topN. The sorting module is
search, recommend, browse favorites and responsible for sorting the topN commodities.
purchase; (The value of N is controlled by the user). It is
 Search function module: basic search recommended to use tensorflow to build some
function, keyword search; parts and use the python environment of conda.
 Recommendation function module: recall,
sorting, popular recommendation, C. System Architecture Design
personalized recommendation; The recommended architecture adopts big data
 Data processing module: provide log lambda architecture, and the front and rear end
service, transmit the user's information to adopt golang web framework gin for connection,
the background in real time, and clean the and each service uses grpc microservice
data. framework for connection. This topic uses MVC
mode, and the system architecture is shown in
B. Technical Scheme for Recall Sequencing
Figure 5.
1) Recall part (machine learning)

7
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

Front
Recommended User commodity
end Commodity display Login Registration Search display
display management
display

Background
Back end Recommended User information
Search Service commodity User behavior log
service services management
management

grpc microservice framework

flum
Recommended result TopK
Based on
Sort&Reorder kafuka
TensorFlow
Multi strategy recall
Data storage processing
redis
Offline Training of Models Spark/link
ES
Spark/TensorFlow data cleaning
mysql
hbase

hdoop
Figure 5. System architecture diagram.

D. Implementation of Commodity library DistBelief. Its gpu version can improve the
Recommendation System training speed.
2) Keras
First, it introduces the program development Keras is an open source artificial neural
environment, and then analyzes the technical network library written by Python. It can be used
feasibility of program development. This program as the high-level application program interface of
is developed on the PC side using the MAC Tensorflow to design, debug, evaluate, apply and
environment and Python language. At the same visualize in-depth learning models.
time, it involves in-depth learning and network 3) CUDA
training, so it needs to use high-performance GPU NVIDIA has introduced a general GPU parallel
for computing acceleration [14]. computing architecture. Because the deep neural
The specific development environment is as network will involve a lot of matrix operations
follows: the development platform is PC; The when training and using, the use of CUDA to
software environment operating system is MAC; operate on the GPU can greatly improve the
The development language is Python; The IDE is efficiency of the model.
pycharm vscod. In recent years, the development of deep
Next, we will introduce various tools and learning has driven the development of the field of
frameworks used in the development of this artificial intelligence. Research on
program. During the development of this program, recommendation systems through deep learning
pycham is used as the IDE to write Python code. has emerged in an endless stream, and there are
At the same time, Tensorflow gpu and Keras are many open source resources. Therefore, it is
used as the development tools because of the deep technically feasible to develop a commodity
learning algorithm [15]. recommendation system [16].
The program development mainly uses the
following development tools and frameworks: CONCLUSIONS
V.
1) Tensorflow gpu This paper first introduces the development
Tensorflow is a symbolic mathematical system background and research significance of this topic.
based on data flow programming, which is widely By analyzing the status quo of recommendation
used in the programming implementation of technology and deep neural network, and referring
various machine learning algorithms. Its to the development status at home and abroad, this
predecessor is Google's neural network algorithm paper explains the significance of the development
of this system. Then it explains the technology

8
 International Journal of Advanced Network, Monitoring and Controls Volume 08, No.01, 2023

used in the system development. This system uses [6] Rendle S. Factorization machines//Proceedings of the
2010 IEEE 10th International Conference on Data
Python language and Tensorflow architecture. In Mining. Sydney, Australia, 2010:995-1000.
the requirement analysis part, the feasibility is [7] LYU L, MEDO M, YEUNG C, et al. Recommender
analyzed in many aspects, and then the functional systems［J］. Physics Reports, 2012, 519(1):1－50．
[8] Shan Y, Hoens T R, Jiao J, et al. Deep crossing: Web-
requirements and non functional requirements of scale modeling without manually crafted combinatorial
the system are introduced, and the three functions features. SIGKDD 2016: 255-262.
of the system are analyzed. It describes how [9] Song W, Shi C, Xiao Z, et al. Autoint: Automatic
feature interaction learning via self-attentive neural
different functions are progressing, clearly and networks. CIKM 2019: 1161-1170.
concisely shows the system construction method, [10] Chen Q, Zhao H, Li W, et al. Behavior sequence
how different modules collaborate, and shows the transformer for e-commerce recommendation in
Alibaba. Proceedings of the 1st International Workshop
completion of the system and the core code. on Deep Learning Practice for High-Dimensional
In a word, the design of this system can meet Sparse Data. 2019: 1-4.
the needs of the development of the times, so that [11] Hao-Jun Michael Shi;Dheevatsa Mudigere;Maxim
Naumov; Jiyan Yang Compositional Embeddings
people can enjoy the application of the Using Complementary Partitions for Memory-Efficient
recommendation system in the commodity Recommendation Systems [C], 2020.
industry. It is an efficient and novel system. [12] Antonio Ginart; Maxim Naumov; Dheevatsa Mudigere;
Jiyan Yang; James Zou Mixed Dimension Embeddings
with Application to Memory-Efficient
Recommendation Systems. [J] IEICE Transactions on
REFERENCES Fundamentals of Electronics, Communications and
Computer Sciences, 2019.
[13] Hao-Jun Michael Shi; Dheevatsa Mudigere; Maxim
[1] Elkahky A M, He X. A multi-view deep learning Naumov; Jiyan Yang Compositional Embeddings
approach for cross domain user modeling in Using Complementary Partitions for Memory-Efficient
recommendation systems//Proceedings of the 24th Recommendation Systems. [J] IEICE Transactions on
International Conference on World Wide Web. Fundamentals of Electronics, Communications and
Florence, Italy, 2015:278-288. Computer Sciences, 2019.
[2] Huang P S, He X, Gao J, et al. Learning deep structured [14] Yongxin Yang and Timothy Hospedales. 2016. Deep
semantic odels for web search using clickthrough multi-task representation learning: A tensor
data//Proceedings of the 1st Workshop on Deep factorisation approach. arXiv preprint
Learning for Recommender Systems. Boston, USA, arXiv:1605.06391 (2016).
2016:29-34. [15] RAMEZANI M, MORADI P, AKHLAGHIAN F. A
[3] Cheng H T, Koc L, Harmsen J, et al. Wide & deep pattern mining approach to enhance the accuracy of
learning for recommender systems//Proceedings of the collaborative filtering in sparse data domains [J].
1st Workshop on Deep Learning for Recommender Physica A: Statistical Mechanics and its Applications,
Systems.Boston, USA, 2016:7-10. 2014, 408:72–84.
[4] Guo H,Tang R, Ye Y, et al. DeepFM:A factorization- [16] ZHANG F, YUAN N J, LIAN D, et al. Collaborative
machine based neural network for CTR Knowledge Base Embedding for Recommender
prediction//Proceedings of the 26th International Joint Systems[C]//ACM SIGKDD International Conference
Conference on Artificial on Knowledge Discovery and Data Mining. 2016: 353–
Intelligence.Melbourne,Australia, 2017:1725-1731. 362.
[5] Chen C, Meng X, Xu Z, et al. Location-aware
personalized news recommendation with deep semantic
analysis. IEEE Access, 2017:173-182.