11922 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO.

13, 1 JULY 2023

User-Preference-Learning-Based Proactive Edge

Caching for D2D-Assisted Wireless Networks
Dongyang Li , Haixia Zhang , Senior Member, IEEE, Hui Ding , Tiantian Li, Daojun Liang,
and Dongfeng Yuan , Senior Member, IEEE

Abstract—This work investigates proactive edge caching for I. I NTRODUCTION

device-to-device (D2D)-assisted wireless networks, where user
ITH the rapid proliferation of smart mobile phones
equipment (UE) can be selected as caching nodes to assist con-
tent delivery to reduce the content transmission latency. In doing
so, there are two challenges: 1) how to precisely get the user’s
W and the rapid development of Internet of Things (IoT),
wireless networks have been suffering from an unprece-
preference to cache the proper contents at UEs and 2) how to dented data traffic surge in recent years [1]. According to
replace the contents cached at UEs when there are new pop- the Ericsson report [2], mobile data traffic will reach 237
ular contents emerging. To address these, we develop a user
preference learning-based proactive edge caching (UPL-PEC) exabytes per month in 2026, i.e., a fivefold increase over
strategy. In the strategy, we first propose a novel context and 2020. This dramatic increase will impose a heavy traffic stress
social-aware user preference learning method to precisely predict on the already overburdened cellular network and backbones,
user’s dynamic preferences by jointly exploiting the context which in turn will result in even high transmission latency
correlation among different contents, the influence of social
relationships and the time-sequential patterns of user’s content
to mobile users. In particular, with the rapid development
requests. Specifically, the bidirectional long short-term memory of smart devices and artificial intelligence (AI) techniques,
networks are adopted to capture the time-sequential patterns the latency-sensitive applications (such as real-time video
of the user’s content requests. And, the graph convolutional streaming, augmented reality, and virtual reality) emerge and
networks are developed to capture the high-order similarity impose stringent requirements on low network latency [3].
representation among different contents from the constructed
content graph. To learn the social influence representation, an To support these latency-sensitive applications, it is nec-
attention mechanism is designed to generate the social influence essary to introduce advanced networking architectures and
weights to users with different social relationship. Based on the efficient data transmission techniques toward the next gener-
learned user preference, a proactive edge caching architecture is ation mobile networks (i.e., B5G/6G). Against this backdrop,
proposed to integrate the offline caching content placement and
edge caching has been emerged as a promising technique to
the online caching content replacement policy to continuously
cache the popular contents at UEs. Simulation results show that relieve such network transmission pressures and reduce the
the proposed UPL-PEC strategy outperforms the existing similar resulted network latency [4]. By precaching the popular con-
caching strategies at about 3.13%–4.62% in terms of the average tents in proximity to the edges of the networks, the redundant
content transmission latency. transmission of the same contents can be avoided and, thus,
Index Terms—Device-to-device (D2D) communication, graph the backhaul pressure can be effectively relieved [5]. Besides,
convolutional networks (GCNs), proactive edge caching, social with such edge caching, users can access their required con-
network, user preference learning.
tents directly from much closer BSs or adjacent mobile users.
This can remarkably decrease the content delivery latency [6]
Manuscript received 28 July 2022; revised 28 December 2022 and and greatly improve the user’s experience [7]. There have been
29 January 2023; accepted 9 February 2023. Date of publication
14 February 2023; date of current version 23 June 2023. This work was some real-world systems/applications adopting edge caching.
supported in part by the Project of International Cooperation and Exchanges For instance, Cisco has developed a Web caching communica-
NSFC under Grant 61860206005; in part by the Joint Funds of the NSFC tion system, in which routers are linked with caches and share
under Grant U22A2003; in part by NSFC under Grant 62271288; and in
part by the Guangdong Basic and Applied Basic Research Foundation under the popular contents to reduce the Internet traffic [8]. Besides,
Grant 2021B1515120066. (Corresponding author: Haixia Zhang.) some online video applications (such as YouTube and TikTok)
Dongyang Li is with the Shandong Key Laboratory of Wireless are also adopted the edge caching strategies.They recommend
Communication Technologies, Shandong University, Jinan 250061, China,
and also with the School of Information Science and Engineering, Shandong the popular contents to users based on the preferences of users
University (Qingdao Campus), Qingdao 266237, China (e-mail: lidongyang@ and precache the contents to reduce the content downloading
mail.sdu.edu.cn).
Haixia Zhang is with the Shandong Key Laboratory of Wireless
latency and improve users’ experience [9].
Communication Technologies and the School of Control Science According to the location of the content caching, existing
and Engineering, Shandong University, Jinan 250061, China (e-mail: edge caching techniques can be roughly divided into two cat-
[email protected]).
Hui Ding, Tiantian Li, Daojun Liang, and Dongfeng Yuan are with
egories: 1) base station (BS) caching and 2) user equipment
the Shandong Key Laboratory of Wireless Communication Technologies, (UE) caching [10]. Compared with BS caching, UE caching
Shandong University, Jinan 250061, China (e-mail: [email protected]; can further relieve the network pressure since it can directly
[email protected]; [email protected]; dfyuan@
sdu.edu.cn). offload the traffic with the aid of device-to-device (D2D)
Digital Object Identifier 10.1109/JIOT.2023.3244621 communications [11]. Besides, through caching contents as
2327-4662 
c 2023 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11923

close as to users, UE caching can also help reduce the con- UEs. To maximize the caching hit ratio, LSTM and ensemble
tent transmission latency [12]. To this end, UE caching (i.e., learnings was applied in [29] to develop an effective common
caching at UEs with D2D communications) has attracted caching framework that satisfies diverse user preferences. It
intensive attention recently and lots of works have been car- is true that the works in [24], [25], [26], [27], [28], and [29]
ried out on it. For instance, in [13], a mobility-aware UE have captured the time-sequential patterns of content request,
caching placement strategy was proposed to maximize the traf- but they did not consider the context correlation among dif-
fic offloading ratio. In order to reduce the average content ferent contents. In practice, the contents with similar context
transmission latency, Li et al. [14] studied the delay-oriented are usually requested by the same person. Motivated by this,
UE caching placement problem. In fact, compared with BS in [30], a clustering-based long short-term memory approach
caching the caching resources of UEs are more limited. It is was proposed to predict the number of content requests by
impractical to cache all the popular contents on the storage exploiting the correlation of the request patterns among differ-
of UEs. Therefore, which content should be cached at UEs ent contents through clustering. To further prompt the caching
becomes a main issue in designing UE caching schemes. At performance, Li et al. [31] proposed a stacked autoencoder-
present, the primary criterion for deciding what to cache heav- based long short term memory network to capture both the
ily relies on the popularity of the contents, which is typically correlation of the request patterns among different content
unknown in advance due to the high dynamics of users’ con- and the periodicity in time domain to improve the prediction
tent requests. To solve this, machine learning (ML) has been accuracy of the content popularity. In [32], a model based on
adopted as a powerful tool to predict the content popularity convolutional neural networks (CNNs) and attention mech-
or users’ requests based on historical observations, and then anism was proposed to extract the sequential behavioral of
make the cache decision to optimize the objective functions users to further improve the user preference. What is more,
in wireless networks. In [15] and [16], a transfer learning user preference will also be affected by the social relationships
approach was proposed for improving the content popular- among users, since they may share their interests with others,
ity estimation by leveraging prior information obtained from including their friends, family members, classmates and so on.
a surrogate domain. In [17], support vector machine (SVM) Thus, the works [33], [34] utilized the influence of social rela-
was used to classify all contents into different triple classes, tionships on user interests to design the proactive edge caching
and then Bayesian optimizer (BO) was applied to partition strategy. Specifically, in [33], a prediction model by virtue
caching resources according to content classes. To further of users’ social network information was proposed to evalu-
enhance the caching decision ability, reinforcement-learning ate the users’ encounter probability, where the users’ context
(RL) approaches for proactive caching was proposed to exploit information was utilized to predict the future content request
the knowledge of content popularity [18], [19]. Nevertheless, probability for each UE. Based on the predicted results, a con-
a common assumption of the aforementioned works is that tent placement algorithm was established to optimize system
the content popularity does not change over time. This is not performance. In [34], a user-interest prediction model based on
applicable since the content popularity is highly dynamic in social proximity, user preference, and freshness was proposed
practical systems. To address this, the works in [20], [21], to estimate the probability of each user’s request for a con-
[22], and [23] adopted time-series analysis methods to improve tent more accurately. However, all the above works mainly
the accuracy of the time-variant content popularity prediction. focused on exploiting either the correlation among different
As a classical time-series analysis method, the auto-regressive contents or the influence of social relationships to predict the
moving-average (ARMA) model was utilized in [20] to predict user preferences. How to make use of both the correlation
the time-variant content popularity. In [21] and [22], the con- among different contents and the influence of social rela-
tent popularity prediction model based on bidirectional long tionships to learn the users’ preference and cache the proper
short term memory (Bi-LSTM) networks was proposed to contents in the right place remains to be an open problem.
predict the popularity of the contents and then update the Besides, most of the above-mentioned works assumed that the
edge caching strategy accordingly. To further improve the content library is stationary. In practice, the contents on the
caching performance, two potential recurrent neural networks server evolves with time, i.e., there are new contents coming
were adopted to predict the users’ mobility and the content to the server from time to time. If we cannot immediately
popularity to determine which contents to cache [23]. replace the caching list at UEs when new popular contents
However, the content popularity distribution in [20], [21], appear, it will cause a waste of caching resources, which in
[22], and [23] estimated based on historical request data of turn results in higher content transmission latency. To the best
all the UEs in the networks, which cannot reflect the pref- of our knowledge, there are no related works on solving how
erence of individual user, and, hence, may not perform well to replace the caching contents on UEs when the new popu-
when being used for determining which content an individual lar contents emerge, which is also an open issue need to be
UE should cache. To further improve the performance of UE solved.
caching, in [24] and [25], the user preference learning aided To deal with the above-mentioned issues, we develop a user
collaborative edge caching strategy was proposed, which dis- preference learning-based proactive edge caching (UPL-PEC)
covers the user’s preference through the historically request strategy to minimize the average content transmission latency.
information. In [26], [27], and [28], the deep reinforcement First, a novel context and social-aware user preference learn-
learning was adopted to learn the user’s content access pat- ing method is proposed to predict the user preferences.
tern over time and then determined which content to cache on Then, a learning-based proactive edge caching architecture

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
11924 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO. 13, 1 JULY 2023

is developed to integrate the offline user preference learn-

ing of existing contents and the online user preference
learning of newly added popular contents to continuously
cache the popular contents at UEs. The main contributions
are summarized as follows.
1) We propose a UPL-PEC strategy by considering the
dynamic user preference, the user’s mobility as well
as their social properties to effectively reduce the
average content transmission latency in D2D-assisted
wireless networks. In the strategy, a deep learning
(DL)-based user preference learning method and a
learning-based proactive edge caching architecture are
proposed to minimize the average content transmission
latency.
2) A novel user preference learning method based on DL is Fig. 1. Illustration of the network architecture of D2D-assisted wireless
networks.
proposed, namely, content- and social-aware user pref-
erence learning (CS-UPL), to precisely learn the user’s TABLE I
dynamic preferences by jointly capturing the context K EY PARAMETERS
correlation among different contents, the influence of
the social relationships and the time-sequential patterns
of the user’s content requests. In detail, the graph con-
volutional networks (GCNs) are adopted to capture the
high-order similarity among different contents and an
attention mechanism is proposed to adaptively generate
the social influence weights of users with different social
relationship.
3) Based on the user preference learning, a learning-based
proactive edge caching architecture is proposed, which
integrates the offline user preference learning-based edge
caching and the online user preference learning of new
popular contents to continuously caching the popular
contents on UEs.
4) Simulation results based on real-world data set show
that the proposed CS-UPL method achieves higher A. Network Architecture
prediction accuracy of user preference than the base- As shown in Fig. 1, a cache-enabled D2D-assisted cellu-
line methods. Moreover, the proposed UPL-PEC strategy lar network consisting of one BS and U randomly distributed
outperform the existing similar caching strategies at UEs is studied. In the system, each UE in set U =
about 3.13%–4.62% in terms of the average content {1, 2, . . . , u, . . . , U} can request any contents from the con-
transmission latency. tent library F = {1, 2, . . . , f , . . . , F}, in which the length of
The remainder of this article is organized as follows. content f is denoted as lf . To avoid possible traffic conges-
The system model and the optimization problem are intro- tion caused by repetitive content downloading requests, UEs
duced in Section II. Section III describes the proposed can perform as caching nodes to provide caching services and
CS-UPL method. Section IV details the proposed learning- share the cached contents with other UEs via D2D commu-
based proactive edge caching architecture. The performance nications. The caching capacity of UE u is denoted by Su .
evaluations are provided in Section V. Section VI concludes The caching state of UEs is denoted by X = (xu,f t )
U×F , where
this work. xu,f = 1 if content f is cached in UE u at time slot t; otherwise,
t
t
xu,f = 0. We assume that the cache capacity of BS is large
enough to store the entire content library F. The assumption
II. S YSTEM M ODEL AND P ROBLEM F ORMULATION is justified by the fact that the BS can easily retrieve any con-
In this section, the considered system model is described. tent item from the core networks if the content item is not
Specifically, the detailed network architecture of the D2D- cached [26].
assisted cellular network is introduced in Section II-A. In specific, we consider a time-slotted system, as shown
The D2D sharing model is presented in Section II-B. in Fig. 2. Each time slot is divided into two phases: 1) off-
The content transmission latency analysis is detailed in peak hours and 2) peak hours. At the beginning of the each
Section II-C. In Section II-D, an optimization problem is time slot, the DL model is trained offline using the historical
formulated to minimize the average content transmission content request data. And then, the user preference of differ-
latency. For better readability, the key parameters are listed ent contents is predicted by the trained model. Based on the
in Table I. user preference prediction, we get the caching decision and

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11925

Fig. 2. Overall execution sequence of the proposed method in each time slot.

update the cached contents on all UEs accordingly. During

the peak hours, the contents are delivered according to the Fig. 3. Illustration of the D2D sharing model based on physical and social
f ,t
users’ requests. Let rtu = [ru1,t , ru2,t , . . . , ru , . . . , ruF,t ] denote domains.
f ,t
the content request vector of user u, where ru ≥ 0 repre-
sents the number of requests for content f . Considering that
1) Physical Domain: The physical distance between two
the continuous emergence of new contents, if content f is not
f ,t UEs is always changing due to their mobility nature. To char-
available yet at time slot t, ru = 0 ∀u ∈ U ∀f ∈ F. There is a acterize the user’s mobility pattern, the intercontact model [35]
resource scheduler that allocates subchannels to UEs dynam- is adopted. In the model, the timeline of a UE contact process
ically to avoid interferences and the system suffers from both alternatively consists of the contact time and the intercontact
large scale and small-scale fading, the transmission rates of time. The former is the time that two UEs are in the commu-
the content f from UE v and BS to UE u can be expressed as nication range and may share the required contents via D2D
  links. The latter is the time between two consecutive contacts.
PUE |hv,u |2 dv,u −α
Rv,u = BD2D log2 1 + (1) By analyzing the actual user traces, the connection time and
σ2 interval time between UE u and UE v follows the exponential
 
PBS |hs,u |2 ds,u −α distribution with parameters λu,v and μu,v , respectively. We
Rs,u = BBS log2 1 + (2) define the binary random variable mtu,v as connection indica-
σ2
tor for UE u and UE v at time slot t. During the time slot t,
where BD2D and BBS denote the bandwidth allocated to the if UE u and UE v can establish a D2D communication link to
channels for D2D transmission and cellular links, respectively, transmit the requested contents, mtu,v = 1, otherwise mtu,v = 0.
PUE and PBS denote the transmit powers of UEs and BS, According to [36], the connection between UE u and UE v
respectively, hv,u and hs,u represent the Rayleigh fast fading follows the continuous-time Markov chain model, which can
coefficient of the communication link for UE v to UE u and be expressed as
  
BS to UE u, respectively, dv,u and ds,u denote the physical P mtu,v = 1mtu,v
0

distance from UE u to UE v and BS, respectively, α is the ⎧

⎨ μu,v −μu,v e−(λu,v +μu,v )(t−t0 ) , if mtu,v 0
=0
path-loss exponent, σ 2 represents the noise power. Then, the λu,v +μu,v
= −(λ +μ )( ) (5)
latency of UE u downloading the content f from BS and UE v ⎩ μu,v +λu,v e u,v u,v t−t 0 t0
, if m = 1
λu,v +μu,v u,v
can be written as
where t0 is the initial time slot.
τs,u
f
= lf /Rs,u (3) 2) Social Domain: Due to the selfish nature of human
τv,u
f
= lf /Rv,u . (4) beings, UEs are more willing to share their own content
to those they have stronger social relationship with. Based
Considering that the distance between two UEs communicat- on this, the social graph Gs = {U , ES } is built for D2D
ing via D2D is typically shorter than that between the BS communications, where U represents the set of UEs and
and UEs, we assume that the content downloading latency ES = {eu,v ∀u, v ∈ U } is the set of the social relationship
f f
τv,u < τs,u . among users. When two users have a close social relation-
ship, eu,v = 1, otherwise, eu,v = 0. As described in [37],
people tend to associate and bond with others that have sim-
B. D2D Sharing Model ilar preferences, i.e., users with common preferences have a
Due to the physical transmit power constraints and social higher probability to establish a close social relationship. So,
trust considerations, D2D communication link is considered in this work, we utilize the similarity of content preference to
to be successfully established only if the distance between the measure the social relationships between two users. Let stu,v
two UEs is within the maximal D2D communication range, denote the similarity of the content preference between user u
and the involved pair of UEs have a direct social relationship. and user v, which can be calculate by the cosine similarity as
To this end, we model the D2D sharing pattern between two F f ,t f ,t
UEs based on both the physical and social domains, as shown f =1 pu pv
stu,v = (6)
in Fig. 3. The details of these two domains are discussed as F f ,t 2 F f ,t 2
follows. f =1 pu · f =1 pv

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
11926 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO. 13, 1 JULY 2023

As a result, the latency for UEs in the networks to fetch the

desired contents at time slot t can be calculated by
⎛ ⎛ ⎞ ⎞
  
τu,f
t
= ruf ,t 1 − xu,f
t ⎝ min f ,t
τv,u + ⎝1 − t ⎠ f ,t ⎠
xv,f τs,u
v∈Nut ,xv,f =1
t
v∈Nut
(7)
 t = xt ⊕ xt ⊕ · · · ⊕ xt · · · ⊕ xt
where v∈Nut xv,f 1,f 2,f v,f |Nut |,f is a
0-1 variable that indicates whether
 the content f is cached in
one of the neighbors of UE u. v∈Nut xv,ft = 1, if and only if
t = 1, v ∈ N t .
there is any xv,f
Fig. 4. Illustration of the content request path through different caching u
nodes.
D. Problem Formulation
f ,t f ,t f ,t f ,t f ,t f ,t f ,t Based on the analysis of the content transmission latency
where pu = (ru / Ff=1 ru ), pv = (rv / Ff=1 rv ), ru and in Section II-C, we can get the average content downloading
f ,t
rv represent the request numbers of content f by user u and latency of all UEs in the system as
user v in time slot t, respectively. If the similarity of content
T U F
f =1 τu,f
t
preference between user u and user v is larger than a certain t=1 u=1
τc = . (8)
threshold θ , UE u and UE v have a close social relationship, T U F f ,t
t=1 u=1 f =1 ru
i.e., when stu,v ≥ θ , eu,v = 1; otherwise, eu,v = 0.
3) D2D Sharing Link: Based on the physical connection Furthermore, the average content transmission latency
mtu,v and social relationship eu,v , whether a D2D sharing link reduced by UE caching strategy can be expressed as
can be established between UE u and UE v at time slot t is
judged by atu,v = mtu,v eu,v . Particularly, atu,v = 1 holds only τb = τnc − τc (9)
if mtu,v = 1 and eu,v = 1, i.e., the D2D sharing link can
where τnc represents the average content transmission latency
be successfully established only if the corresponding physical
of all UEs for fetching their desired contents when no UEs
distance between the involved pair of UEs is within the D2D
are selected as caching nodes to assist content delivery in the
communication range, and the two users have a close social
system. It can be written as
relationship. When atu,v = 1, UE v is called the neighbor of
UE u. The potential neighbors set of UE u at time slot t is T U F f ,t
f =1 ru,f τs,u
t
t=1 u=1
denoted as Nut = {v|v ∈ U , atu,v = 1}. Additionally, upon the τnc = . (10)
T U F f ,t
f ,t
untrusted connection, we define the τv,u = ∞, when v ∈ / Nut . t=1 u=1 f =1 ru

From (10), it is seen that the value of τnc is irrelevant to the

caching strategy. Hence, for simplicity, we convert the problem
C. Content Transmission Latency Analysis maximizing the average content transmission latency reduced
As shown in Fig. 4, there are three ways for UEs to by UE caching strategy τb into the problem minimizing the
fetch the desired contents in this network. The corresponding average content downloading latency τc .
latency of each way to fetch contents at time slot t is shown With the objective of minimizing the average content down-
as follows. loading latency, the optimization problem is formulated as
Case 1: If the content f requested by UE u is cached in its
local caching content list, then the content f can be satisfied P1 : min {τc }
X
locally. As a result, the corresponding latency for fetching the
f ,t 
F
content f , denoted by τu , can be regarded as zero. s.t. C1 : t
xu,f lf ≤ Su ∀u ∈ U ∀t ∈ T
Case 2: If the request cannot be satisfied through the local f =1
caching content list, UE u then seeks help from the sur- t
C2 : xu,f ∈ {0, 1} ∀u ∈ U ∀f ∈ F ∀t ∈ T (11)
rounding UEs that can establish direct D2D sharing links. In
this case, the corresponding latency fetching the content f is where C1 constrains the maximum caching capacity of each
f ,t
denoted as τv,u . Note that when there are multiple surrounding t is a binary variable. It is obvious
UE, C2 guarantees that xu,f
UEs who can establish D2D sharing links with UE u and have that the problem P1 is mainly decided by the user prefer-
cached the requested content f , UE u will obtain the desired ences of different contents. To solve problem P1 , a UPL-PEC
content f from UE v with the minimum transmission latency strategy is proposed in the following. First, a context and
f ,t
minv∈Nut ,xt =1 {τv,u }. social-aware user preference learning method is proposed to
v,f
Case 3: If the content request is not available locally or precisely predict user’s dynamic preference in Section III.
through surrounding UEs, UE u will send its request to the Based on the user preference prediction, a learning-based
BS for the desired content f . In this case, the latency of UE u proactive edge caching architecture is proposed to minimize
f ,t
fetching the desired content f is denoted as τs,u . the average content transmission latency in Section IV.

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11927

Fig. 5. Framework of the proposed CS-UPL method.

III. C ONTEXT AND S OCIAL -AWARE U SER

P REFERENCE L EARNING M ETHOD
In this section, a CS-UPL method is proposed to predict
user’s future content requests by jointly considering the cor-
relation among different contents, the influence of social
relationships and the time-sequential patterns of the user’s
content requests. As shown in Fig. 5, we first utilize the
Bi-LSTM networks to capture the time-sequential patterns of
user’s content request. Then, GCNs are adopted to learn the
high-order similarity representation among different contents Fig. 6. Detailed framework of the BiLSTM block.
from the constructed content graph. Afterward, an attention
mechanism is adopted to learn the social influence represen-
f ,t
tation through the user social networks. Finally, the learned where rtu = [ru1,t , ru2,t , . . . , ru , . . . , ruF,t ] represents the differ-
content similarity representation, social influence representa- ent content request numbers of user u in time slot t. After
tion and time-sequential representation are combined by fully inputting Rtu , the memory of the historical information for
connected networks (FCNs) to improve the performance of long and short time periods will be controlled through three
user preference learning. self-parameterized controlling gates, i.e., input gate gti , forget
gate gtf and output gate gto . The operations can be expressed as
A. Learning Time-Sequential Representation by BiLSTM    
As described in [32], most user requests in daily life are time gti = δ Wiu ht−1 , rtu + biu (12)
 
sequential. For example, on browsers, at e-commerce sites, gtf = δ Wfu ht−1 , rtu + bfu (13)
and in several other scenarios, requests usually have a certain  o  
timing. Since BiLSTM is one of the powerful tools process- go = δ Wu ht−1 , ru + bu
t t o
(14)
 c  
ing sequential data as has been confirmed in [21], we utilize ct = gf ct−1 + gi tanh Wu ht−1 , ru + bu
t t t c
(15)
BiLSTM as the first block of our proposed user preference ht = gto tanh(ct ) (16)
learning model to capture this time-sequential representation,
f f
as shown in Fig. 6. where Wiu , biu , Wu , bu , Wou , bou , Wcu , and bcu are the
In this block, the historical numbers of content requests in weights and biases for BiLSTM to be learned, δ denotes the
an observation window with length of ρ time periods is utilized activation function, tanh represents the tanh activation func-
tion (ex − e−x )/(ex + e−x ). Besides, to enhance the memory
t−ρ t−ρ+1
as input, which is denoted by Rtu = [ru , ru , . . . , rtu ],

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
11928 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO. 13, 1 JULY 2023

where EF = EF + I is the sum of the adjacent matrix and

self-connections of the content graph Gc , I is a matrix whose
diagonals are all ones (i.e., the identity matrix). This allows
the characteristic information of the vertex itself to be retained
when the information is propagated.  R is the degree matrix
of EF denoted as  R = diag(  ei,j ). Hlc is the output of
the l layer, Wlc is the training weight of the layer and σ
is the activation function of ReLU, which is applied in all
layers. Different from the CNN, the GCNs perform the mul-
Fig. 7. Example showing the connectivity of contents with similar context. R−(1/2) EF 
tiplication  R−(1/2) to extract the features from the
neighborhood. Subsequently, we choose the LSTM to capture
the temporal correlation of the GCNs’ outputs of different time
capability, BiLSTM includes additional backward layers to slots. When the outputs of GCNs are input into the LSTM, the
exploit the contexts from both directional transition patterns. operations done are the same as that in (12)–(16). Afterward,
The process can be written as the content similarity representation Hc is obtained.
 
qt = ht ⊕ ht (17)
C. Social Representation Learning With Attention
where ht and ht are the outputs of BiLSTM. Finally, FCN Mechanism
is adopted to fuse the outputs of BiLSTM to obtain the Based on the idea of collaborative filtering in [39], the user
time-sequential representation Hu . The operations can be preference can be obtained by aggregating the behavior of sim-
expressed as ilar users. This motivates by the fact that the users with strong
social relationship may have common interests. To understand
Hu = σ (qt Wu + bu ). (18)
the relationship among users, we take the user social network
Gs and the social similarity as inputs to choose the top-k simi-
B. Learning Content Similarity Representation via GCNs lar users of user u, where the value of k is set according to the
As described in [30], there is a strong correlation among experiments in Section V. Since the top-k users have different
different contents since the contents with similar context are social relationship with user u, the influence of the features
probably requested by the same UE. To incorporate this captured from different top-k users to user u are different.
correlation as the auxiliary information into user preference Inspired by this, an attention mechanism is adopted to gen-
learning, a GCNs block is utilized to learn the content similar- erate different weights to the top-k users to learn the social
ity representation from a constructed content graph. In doing influence representation. First, the FCN is utilized to capture
so, we first construct a content to the content interaction graph the preference representation of different top-k users. It can
Gc = {F, EF }, as shown in Fig. 7, where F represents the set be written as
of contents, EF = {ei,j ∀i, j ∈ F} is a sparse adjacency matrix
and ei,j denotes the content similarity for a pair of contents H0s,top−k = Rttop−k (22)
i and j. In this work, we choose a category-based method l+1
Hs,top−k = σ Hls,top−k Wls + bls , l = 0, 1 (23)
to measure the content similarity ei,j ∈ {0, 1}, which can be
expressed as 1,t 2,t f ,t F,t
where Rttop−k = [rtop−k , rtop−k , . . . , rtop−k , . . . , rtop−k ] is the

1, if content i and j are in the same category different content request number of the top-k similar user in
ei,j = (19) time slot t. After the preference representation of different top-
0, otherwise.
k users have been captured, an attention mechanism is adopted
In addition, each content node f in Gc contains a fea- to the outputs of the FCNs. The operation is expressed as
f ,ρ f ,t−ρ+1 f ,t−ρ f ,t
ture vector rc = [rc , rc , . . . , rc ], which is the  
requested numbers of content f in historical ρ time periods. α = softmax H2s,top−1 , H2s,top−2 , . . . , H2s,top−k (24)
Concomitantly, the feature matrix for all content nodes can be
F,ρ 1,ρ 2,ρ f ,ρ F,ρ where α is the attention weight vector. Finally, the social
denoted by Rc = [rc , rc , . . . , rc , . . . , rc ].
After that the content graph Gc is constructed, the GCNs influence representation Hs can be computed as
block is adopted to capture the high-order similarity represen-  T
tation among different contents. For GCNs with l layers, we Hs = α H2s,top−1 , H2s,top−2 , . . . , H2s,top−k . (25)
F,ρ
take the feature matrix Rc as an input to the first layer
D. Incorporating Multiple Representations for User
H0c = RF,ρ
c . (20) Preference Learning
As described in [38], multilayer GCNs follow the layer-wise As shown in Fig. 5, after the time-sequential representation
propagation rule. The propagation rules of each GCN layer Hu , content similarity representation Hc and social influence
can be defined as the following expression: representation Hs have been learned. The Hu , Hc , and Hs are
corporated by FCN to improve the performance of the user
R− 2 EF 
Hcl+1 = σ 
1 1
R− 2 Hlc Wlc , l = 0, 1 (21) preference learning.

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11929

From the perspective of contents, we first leverage the Algorithm 1 Proposed CS-UPL Method for User’s Future
content similarity representation Hc , combined with the time- Content Requests Prediction
sequential representation Hu to get the context influence on Input: The historical content requests of users R =
user preference Hc,u . The detailed relationship is [R1 , ..., Ru , ..., RU ], the observation window ρ, social
network Gs , the numbers of the selected similar users k,
Hc,u = [Hc , Hu ]Wc,u + bc,u (26) the maximum number of epoch E.
1: for all u ∈ U do
where Wc,u , bc,u are the weights and biases to be learned.
2: Construct content graph Gc = {F, EF } from the
Then, from the perspective of users, the social influence
historical content request Ru of user u;
representation Hs is combined with the time-sequential rep-
3: Select the top-k users who have close social relation-
resentation Hu to get the social influence on user preference
ship with user u and get the historical content preference
Hs,u by
of each top-k user Rttop−k ;
Hs,u = [Hs , Hu ]Ws,u + bs,u (27) 4: for epoch =1 to E do
5: Input Ru to the BiLSTM block to learn the time-
where Ws,u , bs,u are the weights and biases to be learned. 6: sequential representation Hu by (18);
Finally, we combine Hu , Hc,u , Hs,u to predict the content 7: Input Gc to the GCN block to learn the content
f ,t+1
requests Ŷt+1
u = [r̂u1,t+1 , r̂u2,t+1 , . . . , r̂u , . . . , r̂uF,t+1 ] by 8: similarity representation Hc by (21);
  9: Input Rttop−k to the attention block to learn the
Ŷt+1
u = Hu Hc,u , Hs,u Wo + bo (28) 10: social influence representation Hs by (25);
where Wo , bo are the weights and biases to be learned. Note 11: Combine Hc with Hu to get the context influence
that, all weights and biases of the proposed CS-UPL model 12: on user preference Hc,u by (26);
are learned and updated by minimizing the loss between the 13: Combine Hs with Hu to get the social influence
predicted and the actual number of content requests, which is 14: on user preference Hs,u by (27);
defined as 15: Combine Hs , Hc,u and Hs,u to output the content
  16: requests Ŷ of user u in time slot t + 1 by (28);
 t+1 
Lloss = arg minŶt+1
u − Yu . (29) 17: Update the weights W and biases b CS-UPL by
W,b
18: minimizing the loss function (29);
Overall, the whole process of the proposed CS-UPL method 19: Return the optimal weights W∗ and biases b∗ .
is shown in Algorithm 1. 20: Return the optimal weights W∗ and biases b∗ of all users.
Output: The optimal weights W∗ and biases b∗ of all users.
IV. L EARNING -BASED P ROACTIVE E DGE
C ACHING A RCHITECTURE
In this section, a learning-based proactive edge caching
architecture is proposed, which integrates offline caching con-
tent placement and online caching content replacement to
continuously cache the popular contents. In our proposed
architecture, by considering that the BS is equipped with
AI chipsets, both offline user preference learning and future
content request prediction are implemented inside the BS to
cache the proper contents at each UE during off-peak time
period. Moreover, considering the continuous emergence of
the new contents, an online caching content replacement pol-
icy is proposed to replace the existing cached contents with
those popular newly added contents. The detailed architecture
are described in the following.
Fig. 8. Illustration of the learning-based proactive edge caching architecture.

A. Offline User Preference Learning and Caching considered user u from the Request Database and the histori-
Content Placement cal data of the top-k similar users from the Feature Database
As shown in Fig. 8, the proposed architecture at the BS to generate a new training sample set, and then trains the
consists of three fundamental modules: 1) User Preference proposed CS-UPL model to update the parameters, as shown
Learning Model Update; 2) User Future Content Request in Fig. 8, step 1 . After the user preference learning model is
Prediction; and 3) Caching Content Placement. To ensure the updated, step 2 predicts the content request numbers of dif-
accuracy of the future content request prediction, the parame- ferent contents for the user u. The probability that the content
ters of the proposed CS-UPL model need to be continuously f is requested at the next time slot t + 1 can be expressed as
updated. By doing so, the offline learning module initiated f ,t+1
r̂u
at off-peak time. Then, the User Preference Learning Model puf ,t+1 = F f ,t+1
(30)
Update module retrieves the historical requested data of the f =1 r̂u

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
11930 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO. 13, 1 JULY 2023

where r̂u1,t+1 is the predicted content request numbers of con- larger, the local cached content j will be replaced by the new
tent f by user u at the time slot t + 1. Afterward, according to content f ; otherwise, the local caching content list of user u
f ,t+1
pt+1
u = [p1,t+1
u , p2,t+1
u , . . . , pu , . . . , puF,t+1 ], the most popu- remain unchanged. Repeat the above process until the next
lar Su contents are determined. And, the offline caching content off-peak time coming.
placement strategy xt+1u is derived by setting the elements cor- We remark that the proposed learning-based proactive edge
responding to the most popular Su contents to 1 and the others caching architecture will show more advantages in future wire-
to 0. Finally, user u downloads the most popular Su contents less communications for the rapid development of AI and
from the BS to update the local caching list of UE u, as shown smart UEs.
in Fig. 8, step 3 .

V. P ERFORMANCE E VALUATION
B. Online Caching Content Replacement
To show the performance of the proposed UPL-PEC strat-
During the peak hours, the contents are delivered accord-
egy, extensive experiments are conducted based on a real-
ing to the state of the content caching at each UE at off-peak
world data set. In this section, we first describe the utilized
time. When the user u sends a request to access the content
data set. And then, we list the parameter settings in detail.
f , the User Interface module will deliver the user request-
After that, we introduce the baseline schemes and performance
ing information to the Request Processor module, as shown
metrics for performance comparisons and evaluate. The learn-
in Fig. 8, step 4 . The Request Processor module initiates a
ing rate for those experiments and the selected number of
data updating procedure. At step 5 , the requested content
social similar users for the proposed CS-UPL method are set
information of user u will be recorded at both the BS’ and local
by analyzing the effect of different hyperparameters on the
user’s Request Database. If the requested content f is stored
experimental results. At last, we verify the performances of
locally, the Cache Management module delivers it directly to
the proposed CS-UPL and UPL-PEC.
the User Interface module and serves the request of user u, as
step 6 . If the requested content f is not cached locally, the
Request Processor module will seek help from the surround- A. Data Set Description
ing UEs that can establish D2D sharing links. If the requested
The data set utilized in this work is YouTube videos from
content f is cached at the surrounding UEs, the content f will
Kaggle [40]. It records every hour real-time count observa-
be delivered to the User Interface module of user u by D2D
tions (views, comments, likes, and dislikes) of 1600 videos
communication, as step 7 .
in May 2018. The video list is retrieved on 7th May 2018
At step 8 , if the content request is not available locally
and the total number of time slots is 643 h. Before utilizing
or at the surrounding UEs, the Request Processor module will
it, we analyze the data set from different perspectives. The
send the request to the BS for the desired content f and get the
obtained results are shown in Fig. 9. From Fig. 9(a), it is con-
content by cellular link. Moreover, considering the continuous
firmed that the request patterns of the videos are periodic in
emergence of new contents, the content f fetched from the BS
time domain. And, the user’s preferences for different contents
may be more popular than the contents cached at user u. To
at different time slot are shown in Fig. 9(b), from which it is
minimize the average content transmission latency, we need to
intuitively observed that there are differences in viewed counts
replace the local caching content by the popular newly added
for different videos. To demonstrate the reasonableness of the
content, as shown in Fig. 8, step 9 . Doing so, how to properly
proposed category-based method in Section III-B and the pop-
replace the local caching contents becomes the key issue. To
ularity prediction method for new contents in Section IV-B,
solve this, we need to predict the popularity of the newly
we analyze the correlation of videos in different categories
added content. In general, when one hot content is heavily
by Pearson correlation coefficient. The results are shown in
requested by users, the request frequency of the other similar
Fig. 9(c). From Fig. 9(c), it is seen that the videos 0, 3, 5,
contents will also increase. Inspired by this, a context-aware
6 have a strong correlation. This is because they all belong to
content popularity prediction method is proposed to evaluate
the music category, which is consistent with our assumptions.
the popularity of the new contents by the request frequency
Since the number of content requests in real-world data
of other similar contents in the same category. According to
set is recorded on a per-file basis without user information,
the records in the Request Database of user u, we can get the
to obtain the request data information of users, we allocate
popularity of the new content f by
these requests to users in different proportions. In detail, a
j,t+1 random distribution is adopted to generate the proportion of
j∈J ru
r̂uf ,t+1 = (31) the content request numbers of each user to the total content
|J |
request numbers, which can represent the activity level of each
where J ∈ F denotes the set of contents that belong to the user. Then, to ensure that the cumulative sum of the propor-
same category with content f , |J | represents the number of the tions of different users is 1, we conducted a normalization
j,t+1
contents in J , ru is the requested numbers of the content process, i.e., the proportion of each user divided by the cumu-
j that have been recorded in the Request Database currently. lative sum of the proportions of all users. Finally, according to
After we get the popularity of the new content f , we com- the normalized proportion of each user, content requests from
pared it with the local cached content j that with the least the real-world data set are assigned to each user to obtain the
requested numbers. If the popularity of the new content f is content request data of each user. To better show the obtained

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11931

Fig. 9. Request patterns of videos and the user’s preferences for different video categories. (a) Dynamic request of videos. (b) Popularity distribution of
different videos. (c) Correlation analysis of different videos.

correlation between the content requests of the same user, which

is consistent with the real-world scenario. So, we can conclude
that doing the request data partition can mitigate the limitation
of the existing data set of no user-related information.

B. System Setups
A single macrocell with radius 300 m and 100 UEs is built.
The maximum transmission distance of D2D communication
is assumed to be 30 m [26]. The transmit powers of the BS
Fig. 10. Content requests of partial users. and UEs are set to be 40 and 0.25 W, respectively. The band-
width allocated to the BS and D2D are 5 MHz and 10 MHz,
respectively. It is assumed that the system suffers from both
large scale and small scale fading. The path loss exponent
of the large scale fading α is set to be 4, the small scale
fading is Rayleigh fading and the noise power σ 2 is set to
be −174 dBm/Hz. The number of the contents in the content
library is 400 and the length of each content is 1 GB. The
(a) (b) (c) parameters of the exponential distribution of the connection
time and the interval time between two UEs are set to be e−3
and e−4 , respectively, [41]. The total time slot is set to be 50.
The parameters of the proposed CS-UPL model are set as
follows. In the BiLSTM block, the historical request data from
t − ρ to t time slots is chosen as input Rtu when predicting
(d) (e) (f) user preferences in t + 1 time slot. The feature dimension of
the input ρ is 24. The number of neural nodes in the two-layer
Fig. 11. Content requests of user 1 and 2 at different time slots. (a)–(c) BiLSTM network are 32 and 64, respectively. The number of
Show the request numbers of content 1, 2, 3 by user 1 at different time slots,
respectively. (d)–(f) Show the request numbers of content 4, 5, 6 by user 2 the neural nodes in the FC network is 400. In the GCNs block,
at different time slots, respectively. the content interaction graph GF constructed in Section III-B as
input. The number of neural nodes in the two-layer GCN is 32
content request data of each user, we plot the content requests and 64, respectively. For the LSTM network, the neural nodes
of partial users. The results are shown in Fig. 10. are set to be 32 and 64, respectively. For the attention block,
Besides, in the process obtaining content request data for the top-k similar users of user u is chosen according to the
each user, only the proportion of the content request numbers of social similarity stu,v , which is calculated by (6) in Section II-B.
each user is generated randomly to represent the activity level And then, the historical content requests of the top-k similar
of users, and content requests from the real-world data set are users are obtained as input Rttop−k . The numbers of the neural
assigned to each user according to the proportion of each user, nodes in the two-layer FC network are set to be 512 and 100 in
the request patterns of the contents in time domain are retained, the attention block. In the final incorporating block, the size of
as shown in Fig. 9(a). We also show the content request data the context and social impact on user preference are both set
of different users at different time slots in Fig. 11. It can be to be 100. It should be mentioned that the scalability of the
found that the request patterns of different contents for each proposed framework is not restricted by the aforementioned
user are periodic in time domain. There is obviously a strong parameters.

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
11932 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO. 13, 1 JULY 2023

Fig. 12. Performance of the proposed CS-UPL versus different hyperparameters. (a) LR versus training loss (single user). (b) LR versus training loss (multiple
user). (c) MAE versus the value of k.

C. Baseline Schemes and Performance Metrics that is predictable from independent variables. For RMSE and
To evaluate the performance of the proposed CS-UPL MAE, a smaller value means a better performance. On the con-
method, five baseline schemes are adopted and compared. trary, for R2, a larger value implies a better fitting to the data.
1) C-BiLSTM: A clustering-based LSTM method was In addition, to evaluate the performance of different caching
proposed in [30], which predicts the number of con- strategies, the following metrics are adopted: 1) the Average
tent requests by exploiting the correlation of the request Downloading Latency and 2) the Caching Hit Ratio.
patterns among different contents.
2) CNN-LSTM: A model proposed in [22], which combines D. Effects of Different Hyperparameters
CNN with BiLSTM to predict time-varying requests of The effects of different hyperparameters (such as learning
users. rate and the selected number of social similar users) on the
3) UPL: A user preference learning method. Compared performance of the proposed CS-UPL method are studied.
with the proposed CS-UPL, it does not consider the Fig. 12(a) and (b) shows the impact of the learning rate on
context correlation among different contents and the the training loss in single user and multiple users scenarios,
influence of the social relationship. respectively. It is seen that the loss curves of all the learn-
4) C-UPL: A user preference learning method. Compared ing rate gradually converge as the epoch increases. And, the
with the proposed CS-UPL, it does not consider the smallest loss is achieved when the learning rate is 0.001, not
influence of the social relationship. 0.01 or 0.00005. The reason behind is that when the learning
5) S-UPL: A user preference learning method without con- rate is relatively large, the weights of the model are updated
sidering the context correlation among different contents too rapidly and it is not easy for it to reach the optimum.
compared with the proposed CS-UPL. Instead, it may be trapped in a local optimum rather than a
Besides, to evaluate the performance of the proposed global optimum. So, the learning rate is set to 0.001 in all
UPL-PEC strategy, six baseline schemes are adopted and the rest experiments. Fig. 12(c) shows the effect of differ-
compared. ent selected numbers of social similar users on the training
1) Random: A caching strategy that randomly caches the loss in terms of MAE. It is seen that when the selected
contents until the maximum capacity is reached. number of social similar users is 3, the best performance is
2) LRU: A caching strategy, with which the least recently reached, not 6. This is because when the selected number of
used contents are replaced by the popular contents. the similar users is too larger, more redundant features will
3) LFU: A caching strategy, where the least frequently used be introduced, thus, reducing the prediction performance. To
content is replaced first. achieve the best performance of the CS-UPL, the selected
4) Proposed_ND: A caching strategy compared that does number of social similar users is set to be 3 in the rest
not consider D2D communication compared with the experiments.
proposed UPL-PEC strategy.
5) Proposed_NE: A caching strategy that does not consider
the emergence of new popular contents compared with E. Convergence Performance of the Proposed CS-UPL
the proposed UPL-PEC strategy. The convergence of the proposed CS-UPL method is inves-
6) Optimal: A caching strategy designed based on the prior tigated, and comparisons are made with UPL, C-UPL, and
knowledge of the future content requests. S-UPL in single user and multiple users scenarios. The results
The following three performance metrics are adopted to are shown in Fig. 13, from which it is seen that the train-
evaluate the accuracy of the content popularity prediction [42]: ing loss of four methods all gradually converges as the epoch
1) MAE, measuring the absolute errors between the content increases. But, the proposed CS-UPL method outperforms
popularity prediction and the ground truth; 2) RMSE, measur- UPL, C-UPL, and S-UPL methods on the convergence speed
ing the difference between the content popularity prediction and prediction accuracy for all the single user and multiple
and the ground truth; and 3) R2, a statistical metric repre- users scenarios. This proves that the user preference learn-
sents the proportion of the variance in the dependent variable ing method jointly considering the context correlation among
Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11933

TABLE II
P REDICTION P ERFORMANCE C OMPARISON BY T HREE
E VALUATION M ETRICS

Fig. 13. Convergence of the proposed CS-UPL compared with different

methods.
(h) and (i), it is seen that the locations of the bright colored
points predicted by CS-UPL are consistent with the ground
different contents and the influence of social relationships true.
can have a better prediction performance. More interestingly,
it is observed that S-UPL can achieve better performance
G. Caching Performance of the Proposed UPL-PEC Strategy
than C-UPL. This illustrates that the social influence on
user preference is greater than the context influence on user To reveal the superiority of the proposed UPL-PEC strat-
preference. egy, Random, LFU, LRU, Proposed_ND, and Proposed_NE
are adopted as baselines. The obtained results for different
caching sizes, numbers of contents and numbers of UEs when
F. Prediction Performance of the Proposed CS-UPL adopting all these strategies are shown in Fig 17.
Based on the real-world data set, experiments are conducted Fig. 17(a) and (d) shows the performance comparisons of
to verify the performance on content popularity prediction of different caching strategies under different caching sizes. In
CS-UPL. The obtained results are shown in Figs. 14 and 15. doing these experiments, the maximum UE caching size ranges
It is observed from the results that CS-UPL achieves the best from 5 GB to 30 GB. As expected, the average downloading
performance in terms of MAE, RMSE, and R2 among all the latency of all strategies decreases as the caching size increases,
conducted learning methods in both single user and multiple since more contents can be downloaded and cached at UEs.
users scenarios. In detail, the proposed CS-UPL can, respec- The results also show that the proposed UPL-PEC achieves the
tively, outperform up to 23.5%, 18.2%, 12.8%, 6.7%, and best performance in terms of the average downloading latency
3.6% in terms of MAE compared with C-LSTM, CNN-LSTM, and the caching hit ratio. In detail, the average downloading
UPL, C-UPL, and S-UPL in single user scenario, as shown in latency gains brought by the proposed UPL-PEC strategy are
Table II. It is shown that the proposed CS-UPL performs better up to 4.62% and 3.13% compared with those of LFU and LRU
than both C-LSTM and CNN-LSTM. This demonstrates that when the UE caching size reaches 20 GB. From Fig. 17(d),
the proposed user preference learning method performs better it is observed that the proposed UPL-PEC offloads 64.94%
than the traditional content popularity prediction method when traffic of the system when the UE caching size reaches 20 GB
being used for determining UE caching. For the prediction and outperforms LFU and LRU with the improvements of
methods UPL, C-UPL, and S-UPL, the performance is not so up to 5.49% and 3.89%, respectively. The reason behind is
good as CS-UPL. This illustrates incorporating the correlation that the proposed UPL-PEC has a better prior knowledge of
among different contents and the influence of social relation- the future content requests than the traditional caching strate-
ships as the auxiliary information into user preference learning gies. Compared with Proposed_ND, the average downloading
can improve the prediction accuracy. latency and caching hit ratio gains brought by UPL-PEC are
To further demonstrate the prediction performance of the up to 24.01% and 44.87% when the UE caching size reaches
proposed CS-UPL method, the user preference predicted by 20 GB. The reason is that the D2D communications are uti-
CS-UPL is compared with that of the ground truth for dif- lized in the proposed UPL-PEC to assist content delivery
ferent users at different time slots, and the results are shown which can lead to lower content transmission latency. More
in Fig. 16. From Fig. 16(a), (d), and (g), it is observed that interestingly, the Proposed_ND has a better performance than
the content popularity is highly dynamic in different users, but LFU and LRU before the UE caching size reaches 10 GB,
the user preference predicted by CS-UPL fits the ground truth then as the caching size increases, its performance is inferior
well. To more intuitively show that the proposed CS-UPL can to them. This is because UEs will be more likely to cache
accurately locate the contents with higher user preference, we the contents requested by other users when the UE caching
plot the heat maps to show the predicted results, where the size increases. So, the LFU and LRU who uses D2D commu-
bright colored points indicate the contents with higher user nications will have a better performance than Proposed_ND
preference. Comparing Fig. 16(b) and (c), (e), and (f), and when the UE caching size is large. Besides, compared with the

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
11934 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO. 13, 1 JULY 2023

(a) (b) (c)

Fig. 14. Comparisons of different prediction methods in single user scenario. (a) Prediction methods versus MAE. (b) Prediction methods versus RMSE.
(c) Prediction methods versus R2.

(a) (b) (c)

Fig. 15. Comparisons of different prediction methods in multiple user scenario. (a) Prediction methods versus MAE. (b) Prediction methods versus RMSE.
(c) Prediction methods versus R2.

Proposed_NE, the average downloading latency and caching

hit ratio gains brought by UPL-PEC are up to 1.11% and
1.08% when the UE caching size reaches 20 GB. This con-
firms that the proposed learning-based proactive edge caching
architecture, considering the continuous emergence of the new
contents, can improve the caching performance. (a) (b) (c)
Fig. 17(b) and (e) shows the performance comparisons of
different caching strategies for different number of UEs. In
doing the experiments, the numbers of UEs are chosen accord-
ing to different scenarios like dense urban and sparse rural
networks. As shown in Fig. 17(b), the proposed UPL-PEC
offers the lowest downloading latency among all the compared
(d) (e) (f)
caching strategies. Specifically, when the number of UEs is
100, the average downloading latency of UPL-PEC outper-
forms LFU, LRU, Proposed_ND, and Proposed_NE at about
4.58%, 3.29%, 12.79%, and 1.56%, respectively. Moreover,
as shown in Fig. 17(e), the caching hit ratio gains brought by
UPL-PEC over LFU, LRU, Proposed_ND, and Proposed_NE
are around 10.92%, 8.21%, 42.22%, and 3.12%, respec- (g) (h) (i)
tively. More interestingly, from Fig. 17(b), it is seen that the
Fig. 16. User preference predicted by the CS-UPL versus ground true.
performance of all strategies remains essentially constant when (a), (d), and (g) Prediction versus true. (b), (e), and (h) Prediction. (c), (f),
the number of users increases from 80 to 110. This is because and (i) Ground true.
the caching space and the number of users’ content requests
are both increasing with the increase of the number of users
in the system. Only after the increased caching space is able From the results, it is seen that the average content down-
to satisfy the increased users’ content requests, the average loading latency of all the strategies increases as the number
content downloading latency of all strategies decreases. of contents increases since the caching size of UEs limits.
Fig. 17(c) and (f) shows the performance comparisons of Moreover, it is also observed that the proposed UPL-PEC
different caching strategies for different number of contents. achieves the lowest downloading latency compared with LFU,

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11935

(a) (b) (c)

(d) (e) (f)

Fig. 17. Performance of different caching strategies versus different caching sizes, number of UEs, and number of contents.

TABLE III TABLE IV

C OMPUTATIONAL T IME OF D IFFERENT C ACHING S TRATEGIES C OMPUTATIONAL T IME OF D IFFERENT C ACHING S TRATEGIES
U NDER D IFFERENT U SER N UMBERS , W HERE C ACHING U NDER D IFFERENT C ONTENT N UMBERS , W HERE C ACHING
S IZE I S 10 AND C ONTENT N UMBER I S 400 S IZE I S 10 AND U SER N UMBER I S 100

LRU, Proposed_ND, and Proposed_NE. In detail, the aver- increase. The proposed UPL-PEC takes more computational
age downloading latency of UPL-PEC outperforms LFU, time than LRU and LFU since it as a deep-learning-based
LRU, Proposed_ND, and Proposed_NE at 4.26%, 2.64%, proactive caching strategy requires computational time for
4.28%, and 1.25%, respectively, when the number of con- model training and the user preference prediction, but it is less
tents is 600. Moreover, as shown in Fig. 17(f), the caching than 1 h under different user numbers and content numbers.
hit ratio of the proposed UPL-PEC outperforms LFU, LRU, Since the caching strategy is developed at off-peak hours, there
and Proposed_ND at 12.99%, 8.13%, 18.66%, and 3.50%, is sufficient time to train the proposed user preference learn-
respectively. ing model and make the corresponding caching strategy. Thus,
the proposed UPL-PEC strategy is applicable to the actual
H. Analysis of Computation Overhead systems.
To illustrate the computation overhead introduced by the
proactive caching strategy, we compare the computational time VI. C ONCLUSION
of the proposed UPL-PEC with the reactive caching strategies This work investigated the proactive edge caching strategy
LRU and LFU. The obtained results for different user numbers in D2D-assisted wireless networks to minimize the average
and content numbers are shown in Tables III and IV, respec- content transmission latency by caching contents at UEs. In
tively. It is seen that the computational time of all the strategies doing so, two challenging problems are solved: 1) How to
increases as the number of contents and the number of users precisely learn user’s preference to cache the proper contents

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
11936 IEEE INTERNET OF THINGS JOURNAL, VOL. 10, NO. 13, 1 JULY 2023

on each UE and 2) How to replace the contents cached [15] B. Bharath, K. G. Nagananda, and H. V. Poor, “A learning-based
on UEs when there are new popular contents continuously approach to caching in heterogenous small cell networks,” IEEE Trans.
Commun., vol. 64, no. 4, pp. 1674–1686, Apr. 2016.
emerging. In solving these problems, a UPL-PEC strategy [16] T. Hou, G. Feng, S. Qin, and W. Jiang, “Proactive content caching by
was proposed. Specifically, a novel context and social-aware exploiting transfer learning for mobile edge computing,” in Proc. IEEE
user preference learning method was proposed to precisely Global Commun. Conf. (GLOBECOM), Dec. 2017, pp. 1–6.
[17] J. Qiu, Z. Hua, L. Liu, M. Cao, and D. Chen, “Machine-learning-based
predict user’s dynamic preferences by jointly exploiting the cache partition method in cloud environment,” Peer-to-Peer Netw. Appl.,
context correlation among different contents, the influence vol. 15, no. 1, pp. 149–162, Jan. 2022.
of social relationships and the time-sequential patterns of [18] J. Tang, H. Tang, N. Zhao, K. Cumanan, S. Zhang, and Y. Zhou, “A rein-
forcement learning approach for D2D-assisted cache-enabled HetNets,”
user’s content requests. Based on the user preference learn- in Proc. IEEE Global Commun. Conf. (GLOBECOM), Dec. 2019,
ing, a learning-based proactive edge caching architecture was pp. 1–6.
proposed, which integrates the offline caching content place- [19] A. Sadeghi, G. Wang, and G. B. Giannakis, “Deep reinforcement learn-
ing for adaptive caching in hierarchical content delivery networks,” IEEE
ment and the online caching content replacement policy to Trans. Cogn. Commun. Netw., vol. 5, no. 4, pp. 1024–1033, Dec. 2019.
continuously caching the popular contents on UEs. Intensive [20] N. B. Hassine, R. Milocco, and P. Minet, “ARMA based popularity
experiments were carried out, and the results verified that the prediction for caching in content delivery networks,” in Proc. Wireless
Days, Mar. 2017, pp. 113–120.
proposed CS-UPL method achieves the best user preference [21] L. Ale, N. Zhang, H. Wu, D. Chen, and T. Han, “Online proactive
learning performance compared with those state-of-the-art caching in mobile edge computing using bidirectional deep recurrent
methods and the proposed UPL-PEC strategy outperforms the neural network,” IEEE Internet Things J., vol. 6, no. 3, pp. 5520–5530,
Jun. 2019.
compared existing caching strategies. It should be mentioned [22] Y. Jiang, H. Feng, F.-C. Zheng, D. Niyato, and X. You, “Deep learning-
that on the basis of this work, joint allocation of communica- based edge caching in fog radio access networks,” IEEE Trans. Wireless
tion and caching resources can be done to further reduce the Commun., vol. 19, no. 12, pp. 8442–8454, Dec. 2020.
[23] L. Li et al., “Deep reinforcement learning approaches for content caching
content downloading latency. The source code of this work is in cache-enabled D2D networks,” IEEE Internet Things J., vol. 7, no. 1,
available at https://github.com/lidongyang1/CS-UPL. pp. 544–557, Jan. 2020.
[24] Y. Jiang, M. Ma, M. Bennis, F.-C. Zheng, and X. You, “User preference
learning-based edge caching for fog radio access network,” IEEE Trans.
Commun., vol. 67, no. 2, pp. 1268–1283, Feb. 2019.
R EFERENCES [25] M. F. Pervej, L. T. Tan, and R. Q. Hu, “User preference learning-
[1] C. L. Stergiou, K. E. Psannis, and B. B. Gupta, “IoT-based big data aided collaborative edge caching for small cell networks,” in Proc. IEEE
secure management in the fog over a 6G wireless network,” IEEE Global Commun. Conf. (GLOBECOM), Dec. 2020, pp. 1–6.
Internet Things J., vol. 8, no. 7, pp. 5164–5171, Apr. 2020. [26] W. Jiang, G. Feng, S. Qin, T. S. P. Yum, and G. Cao, “Multi-
[2] P. Jonsson, S. Carson, S. Davis, and P. Linder, Ericsson Mobility Report, agent reinforcement learning for efficient content caching in mobile
Ericsson, Stockholm, Sweden, Jun. 2021. D2D networks,” IEEE Trans. Wireless Commun., vol. 18, no. 3,
[3] R. Sun et al., “Delay-oriented caching strategies in D2D mobile pp. 1610–1622, Mar. 2019.
networks,” IEEE Trans. Veh. Technol., vol. 69, no. 8, pp. 8529–8541, [27] J. Tang et al., “Energy minimization in D2D-assisted cache-enabled
Aug. 2020. Internet of Things: A deep reinforcement learning approach,” IEEE
Trans. Ind. Informat., vol. 16, no. 8, pp. 5412–5423, Aug. 2020.
[4] J. Yao, T. Han, and N. Ansari, “On mobile edge caching,” IEEE
[28] X. Wang, R. Li, C. Wang, X. Li, T. Taleb, and V. C. M. Leung,
Commun. Surveys Tuts., vol. 21, no. 3, pp. 2525–2553, 3rd Quart., 2019.
“Attention-weighted federated deep reinforcement learning for device-
[5] Y. Qian, R. Wang, J. Wu, B. Tan, and H. Ren, “Reinforcement learning-
to-device assisted heterogeneous collaborative edge caching,” IEEE J.
based optimal computing and caching in mobile edge network,” IEEE
Sel. Areas Commun., vol. 39, no. 1, pp. 154–169, Jan. 2021.
J. Sel. Areas Commun., vol. 38, no. 10, pp. 2343–2355, Oct. 2020.
[29] T.-V. Nguyen, N.-N. Dao, V. Dat Tuong, W. Noh, and S. Cho,
[6] Z. Yang, Y. Liu, Y. Chen, and L. Jiao, “Learning automata based “User-aware and flexible proactive caching using LSTM and ensem-
Q-learning for content placement in cooperative caching,” IEEE Trans. ble learning in IoT-MEC networks,” IEEE Internet Things J., vol. 9,
Commun., vol. 68, no. 6, pp. 3667–3680, Jun. 2020. no. 5, pp. 3251–3269, Mar. 2022.
[7] J. Tang, T. Q. S. Quek, T.-H. Chang, and B. Shim, “Systematic resource [30] Z. Zhang and M. Tao, “Deep learning for wireless coded caching with
allocation in cloud RAN with caching as a service under two timescales,” unknown and time-variant content popularity,” IEEE Trans. Wireless
IEEE Trans. Commun., vol. 67, no. 11, pp. 7755–7770, Nov. 2019. Commun., vol. 20, no. 2, pp. 1152–1163, Feb. 2021.
[8] Y. Cui, J. Song, M. Li, Q. Ren, Y. Zhang, and X. Cai, “SDN-based big [31] D. Li, H. Zhang, D. Yuan, and M. Zhang, “Learning-based hier-
data caching in ISP networks,” IEEE Trans. Big Data, vol. 4, no. 3, archical edge caching for cloud-aided heterogeneous networks,”
pp. 356–367, Sep. 2018. IEEE Trans. Wireless Commun., early access, Sep. 20, 2022,
[9] D. Liu and C. Yang, “A deep reinforcement learning approach to proac- doi: 10.1109/TWC.2022.3206236.
tive content pushing and recommendation for mobile users,” IEEE [32] Y. Zhang, Y. Li, R. Wang, J. Lu, X. Ma, and M. Qiu, “PSAC: Proactive
Access, vol. 7, pp. 83120–83136, 2019. sequence-aware content caching via deep learning at the network
[10] N. Golrezaei, A. F. Molisch, A. G. Dimakis, and G. Caire, edge,” IEEE Trans. Netw. Sci. Eng., vol. 7, no. 4, pp. 2145–2154,
“Femtocaching and device-to-device collaboration: A new architecture Oct.–Dec. 2020.
for wireless video distribution,” IEEE Commun. Mag., vol. 51, no. 4, [33] J. Ren et al., “Incentivized social-aware proactive device caching with
pp. 142–149, Apr. 2013. user preference prediction,” IEEE Access, vol. 7, pp. 136148–136160,
[11] X. Li, X. Wang, P.-J. Wan, Z. Han, and V. C. M. Leung, “Hierarchical 2019.
edge caching in device-to-device aided mobile networks: Modeling, [34] X. Zhao and Q. Zhu, “Mobility-aware and interest-predicted caching
optimization, and design,” IEEE J. Sel. Areas Commun., vol. 36, no. 8, strategy based on IoT data freshness in D2D networks,” IEEE Internet
pp. 1768–1785, Aug. 2018. Things J., vol. 8, no. 7, pp. 6024–6038, Apr. 2021.
[12] R. Karasik, O. Simeone, and S. S. Shitz, “How much can D2D [35] A. Passarella and M. Conti, “Analysis of individual pair and aggre-
communication reduce content delivery latency in fog networks with gate intercontact times in heterogeneous opportunistic networks,” IEEE
edge caching?” IEEE Trans. Commun., vol. 68, no. 4, pp. 2308–2323, Trans. Mobile Comput., vol. 12, no. 12, pp. 2483–2495, Dec. 2013.
Apr. 2020. [36] Z. Wang, H. Shah-Mansouri, and V. W. S. Wong, “How to download
[13] R. Wang, J. Zhang, S. H. Song, and K. B. Letaief, “Mobility-aware more data from neighbors? A metric for D2D data offloading oppor-
caching in D2D networks,” IEEE Trans. Wireless Commun., vol. 16, tunity,” IEEE Trans. Mobile Comput., vol. 16, no. 6, pp. 1658–1675,
no. 8, pp. 5001–5015, Aug. 2017. Jun. 2017.
[14] Y. Li, M. C. Gursoy, and S. Velipasalar, “A delay-aware caching algo- [37] L. Wu et al., “Modeling the evolution of users’ preferences and social
rithm for wireless D2D caching networks,” in Proc. IEEE INFOCOM links in social networking services,” IEEE Trans. Knowl. Data Eng.,
WKSHPS, May 2017, pp. 456–461. vol. 29, no. 6, pp. 1240–1253, Jun. 2017.

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.
LI et al.: USER-PREFERENCE-LEARNING-BASED PROACTIVE EDGE CACHING 11937

[38] G. Chen, J. Wu, W. Yang, A. K. Bashir, G. Li, and M. Hammoudeh, Hui Ding received the Ph.D. degree from the School
“Leveraging graph convolutional-LSTM for energy-efficient caching in of Information Science and Engineering, Shandong
blockchain-based green IoT,” IEEE Trans. Green Commun. Netw., vol. 5, University, Jinan, China, in 2020.
no. 3, pp. 1154–1164, Sep. 2021. He is currently with the Shandong Key Laboratory
[39] Y. Wang, M. Ding, Z. Chen, and L. Luo, “Caching placement with rec- of Wireless Communication Technologies, Jinan,
ommendation systems for cache-enabled mobile social networks,” IEEE and also with the Yunnan Development and Reform
Commun. Lett., vol. 21, no. 10, pp. 2266–2269, Oct. 2017. Commission, Kunming, China. His research interests
[40] “YouTube videos viewCount every hour.” Kmmd. 2018. [Online]. include heterogeneous network in 5G, resource allo-
Available: https://www.kaggle.com/nnqkfdjq/statistics-observation-of- cation, user association and power control, non-
random-youtube-video convex optimization, reinforcement learning, and
[41] W. Zhang, D. Wu, W. Yang, and Y. Cai, “Caching on the move: A user vehicular networks.
interest-driven caching strategy for D2D content sharing,” IEEE Trans.
Veh. Technol., vol. 68, no. 3, pp. 2958–2971, Mar. 2019.
[42] C. Zhang, H. Zhang, J. Qiao, D. Yuan, and M. Zhang, “Deep transfer
learning for intelligent cellular traffic prediction based on cross-domain
big data,” IEEE J. Sel. Areas Commun., vol. 37, no. 6, pp. 1389–1401, Tiantian Li received the B.E. and M.E. degrees
Jun. 2019. from the School of Physics and Electronics,
Shandong Normal University, Jinan, China, in
2009 and 2012, respectively, and the Ph.D. degree
in information and communication engineering
from the School of Information Science and
Engineering, Shandong University, Jinan, in
2021.
She is currently with the School of Information
Science and Engineering, Shandong Normal
University, and also with the Shandong Key
Laboratory of Wireless Communication Technologies, Jinan. Her research
Dongyang Li received the B.E. and M.Eng. degrees interests include rate splitting multiple access, ultrareliable low-latency
in communication engineering from the School communications, and 6G wireless communications.
of Information Technology, Wuhan University of
Technology, Wuhan, China, in 2015 and 2017,
respectively. He is currently pursuing the Ph.D.
degree with the School of Information Science and
Engineering, Shandong University, Jinan, China. Daojun Liang received the B.S. degree in com-
His research interests include mobile edge com- puter science from Taishan University, Tai’an, China,
puting, wireless edge caching, federated learning, in 2016, and the M.S. degree from the School
and smart communication technologies. of Information Science and Engineering, Shandong
Normal University, Jinan, China, in 2019. He is cur-
rently pursuing the Ph.D. degree with the School
of Information Science and Engineering, Shandong
University, Jinan.
His research interests include deep learning,
machine learning, computer vision, and natural lan-
guage processing.

Haixia Zhang (Senior Member, IEEE) received the

B.E. degree from the Department of Communication Dongfeng Yuan (Senior Member, IEEE) received
and Information Engineering, Guilin University of the M.S. degree from the Department of Electrical
Electronic Technology, Guilin, China, in 2001, and Engineering, Shandong University, Jinan, China, in
the M.Eng. and Ph.D. degrees in communication and 1988, and the Ph.D. degree from the Department of
information systems from the School of Information Electrical Engineering, Tsinghua University, Beijing,
Science and Engineering, Shandong University, China, in January 2000.
Jinan, China, in 2004 and 2008, respectively. He is currently a Full Professor with the School
From 2006 to 2008, she was with the Institute of Civil Engineering, Shandong University. From
for Circuit and Signal Processing, Munich University 1993 to 1994, he was with the Electrical and
of Technology, Munich, Germany, as an Academic Computer Department, The University of Calgary,
Assistant. From 2016 to 2017, she was a Visiting Professor with the University Calgary, AB, Canada. He was with the Department
of Florida, Gainesville, FL, USA. She is currently a Distinguished Professor of Electrical Engineering, The University of Erlangen, Erlangen, Germany,
with Shandong University. Her current research interests include Industrial from 1998 to 1999; the Department of Electrical Engineering and Computer
Internet of Things, resource management, mobile edge computing, and smart Science, University of Michigan at Ann Arbor, Ann Arbor, MI, USA,
communication technologies. from 2001 to 2002; the Department of Electrical Engineering, Munich
Dr. Zhang serves on editorial boards of the IEEE T RANSACTIONS University of Technology, Munich, Germany, in 2005; and the Department of
ON W IRELESS C OMMUNICATIONS , IEEE W IRELESS C OMMUNICATION Electrical Engineering, Heriot–Watt University, Edinburgh, U.K., in 2006. His
L ETTERS, and China Communications. She has been serving as a TPC current research interests include intelligent communication systems, mobile-
member, the session chair, an invited speaker, and a keynote speaker for edge computing and cloud computing, and AI and big data processing for
conferences. communications.

Authorized licensed use limited to: Chang'an University. Downloaded on May 12,2024 at 04:52:59 UTC from IEEE Xplore. Restrictions apply.