INTELLIGENT RADIO: WHEN ARTIFICIAL INTELLIGENCE MEETS THE RADIO NETWORK

Intelligent Edge:
Leveraging Deep Imitation Learning for
Mobile Edge Computation Offloading
Shuai Yu, Xu Chen, Lei Yang, Di Wu, Mehdi Bennis, and Junshan Zhang

Abstract and uncertain MEC environments. On one hand,

the wireless and backhaul links between the mobile
In this work, we propose a new deep imitation devices and edge-cloud servers are time-varying
learning (DIL)-driven edge-cloud computation off- and uncertain. On the other hand, the MEC server
loading framework for MEC networks. A key objec- offers only limited radio, storage, and computation-
tive for the framework is to minimize the offloading al resources, especially in hotspot areas.
cost in time-varying network environments through To this end, a new research area, called intelli-
optimal behavioral cloning. Specifically, we first gent edge learning, is emerging [2, 3], which refers
introduce our computation offloading model for to the deployment of machine learning algorithms
MEC in detail. Then we make fine-grained offload- at the network edge. One of the key motivations
ing decisions for a mobile device, and the problem of pushing machine learning toward the edge is to
is formulated as a multi-label classification prob- allow rapid access to the enormous real-time data
lem, with local execution cost and remote network generated by mobile users for fast training and fast
resource usage consideration. To minimize the off- response to real-time offloading requirements.
loading cost, we train our decision making engine Recently, deep imitation learning (DIL) [4],
by leveraging the deep imitation learning method, which is the problem of training robotic skills from
and further evaluate its performance through an human demonstration, has attracted the atten-
extensive numerical study. Simulation results show tion of researchers in the field of robotics (e.g.,
that our proposal outperforms other benchmark autonomous driving, gesturing, and manipulation).
policies in offloading accuracy and offloading cost Compared to traditional machine-learning-based
reduction. At last, we discuss the directions and offloading methods, DIL carries four advantages:
advantages of applying deep learning methods to • Better performance with large data scale
multiple MEC research areas, including edge data • Noteworthy accuracy in decision making
analytics, dynamic resource allocation, security, • Fast inference speed
and privacy, respectively. • Easy and quick to deploy
Thus, it makes sense to deploy a novel DIL-based
Introduction offloading schedule in MEC-empowered 5G net-
With the development of emerging mobile appli- works.
cations (e.g., augmented reality, 3D gaming, and In this article, we study the issue of making rapid
various Internet of things [IoT] applications), more offloading decisions for a single mobile device in
and more mobile applications become resource- MEC network environments. Our objective is to
thirsty and delay-sensitive. To this end, the Europe- minimize the offloading cost in a time-varying net-
an Telecommunications Standards Institute (ETSI) work environment, subject to network resource
provided a concept of multi-access edge comput- constraints. To this end, we propose an intelligent
ing (MEC) in their 5G standard [1]. In the MEC edge computation offloading framework to make
architecture, distributed MEC servers are located fine-grained offloading decisions for the mobile
at the network edge to provide cloud-computing device in the MEC network. The offloading deci-
capabilities and IT services with low latency, high sions made by the mobile device comprehensively
bandwidth, and real-time processing. The edge consider both the execution cost on the mobile
servers can be connected to the remote cloud device side and time-varying network conditions
through backhaul links to leverage the resource- (including available communication and computa-
ful computation capacities and IT services of tion resources, wired and wireless channel condi-
the remote cloud. By the use of the collabora- tions) on the MEC side.
tive edge-cloud computation offloading between In summary, the contributions of this article are
mobile users and servers, mobile users’ commu- summarized as follows. Based on behavioral clon-
nication overhead and execution delay can be ing [4], which performs supervised learning from
significantly reduced. the observation of demonstrations (i.e., the opti-
Nevertheless, mobile devices usually fail to mal offloading decisions in this article), we design
make the most appropriate fine-grained offloading a DIL-based offloading model for the intelligent
decisions in real time, especially in the time-varying framework. Our model is first trained from learn-

Digital Object Identifier: Shuai Yu, Xu Chen (corresponding author), and Di Wu are with Sun Yat-sen University; Lei Yang is with the University of Nevada;
10.1109/MWC.001.1900232 Mehdi Bennis is with the University of Oulu; Junshan Zhang is with Arizona State University.

92 1536-1284/20/$25.00 © 2020 IEEE IEEE Wireless Communications • February 2020

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.
ing demonstrations in an offline manner. After a Related
quick and easy deployment, our model can make Methods Advantages Disadvantages
works
near-optimal online offloading decisions at a very
fast inference speed. We discuss potential direc- Traditional [5, 6] Performance guarantee High complexity
tions and advantages for applying deep learning
into multiple MEC research areas. Reinforcement
[7, 8] Model-free Curse of dimensionality
The rest of this article is organized as follows. learning
We first introduce the related works in the follow- Deep reinforcement Suitable for dynamic
ing. Then we present our computation offloading [9] Long online training time
learning environments
model. Next, we formulate the optimization prob-
lem and describe the DIL-based offloading model. Deep imitation
Our work
Quick and easy to deploy, Requires a large number of
Simulation results are then shown. We further dis- learning fast online inference speed offline demonstrations
cuss directions and advantages of deep learning TABLE 1. Machine-learning-based computation offloading methods.
for MEC. Finally, we conclude the article in the
final section.
ing decisions to minimize their long-term cost.
Related Work The authors of [8] proposed a Markov decision
In this section, we first survey the traditional com- process (MDP)-based dynamic offloading frame-
putation offloading strategies. Then we review the work in a single-user intermittently connected
state-of-the-art machine-learning-based computa- cloudlet network. Through a value iteration algo-
tion offloading strategies. Last but not least, we rithm, their decision engine can obtain an optimal
introduce DIL The related works are summarized policy to minimize the long-term offloading costs
in Table 1. (i.e., computation and communication costs). The
main advantage of RL is that it can learn with-
Traditional Computation Offloading Strategies out a priori knowledge (i.e., the model-free fea-
From the perspective of a mobile user in the MEC ture). However, with the increase of the number
network, it needs to decide whether and where to of system and action states, the computational
offload its computational tasks to enhance its qual- complexity of RL will grow rapidly (i.e., the curse
ity of service (QoS). However, in practical edge of dimensionality problem). Besides, the perfor-
network environments, the decision making prob- mance of such offloading framework heavily relies
lem is sophisticated because the network environ- on hand-crafted features (e.g., the pre-calculated
ments are randomly uncertain and time varying. transition probability of MDP).
Traditional optimization approaches (e.g., game Recently, researchers’ attention has turned to
theory [5], Lyapunov optimization [6]) for making deep reinforcement learning (DRL). DRL, which
computation offloading decisions in edge com- combines traditional reinforcement learning and
puting environments has been widely studied. For deep learning, is an emerging area of machine
example, Chen et al. [5] study the computation learning research. DRL is based on representation
offloading problem in multi-user MEC environ- learning to automatically extract features from mas-
ments. They prove that it is NP-hard to obtain a sive raw data, and can be regarded as an ideal tool
centralized optimal solution, and propose a game to predict computation offloading decisions. For
theoretic approach to achieve optimal offloading example, the authors of [9] jointly optimize net-
decisions in a distributed manner. The authors of working, caching, and computing resources for
[6] investigate the computation offloading issue a vehicular network. Due to the high complexity
for energy harvesting (EH) devices in MEC envi- of the joint optimization problem, they propose
ronments. They exploit Lyapunov optimization a DRL method to solve the problem. The main
to jointly minimize the execution latency and advantage of DRL for computation offloading
task failure for EH devices. The main drawback relates to its online training manner, which is suit-
of traditional computation offloading strategies is able in a dynamic network environment. However,
their high algorithm complexity, especially in the the corresponding training time is very long.
multi-user multi-server edge computing environ-
ments. Thus, it is hard to deploy the strategies to Deep Imitation Learning
practical edge network environments. Deep imitation learning is an efficient approach
to teach intelligent agents skills through learning
Reinforcement-Learning-Based demonstrations. The authors of [4] consider a vir-
Computation Offloading Strategies tual reality (VR) scenario to teach a PR2 robot to
Reinforcement learning (RL) can solve the prob- learn policies from robotic manipulation demon-
lem of how a decision engine chooses the optimal strations. They show that high-quality robotic
action through interacting with outside environ- manipulation demonstrations play a key role in
ments. The main objective of RL is to choose an DIL. The advantages of DIL relate to its offline
action for each state of the system in order to training and online decision making. Thus, a
maximize the long-term (delayed) cost. Thus, RL is trained model can be deployed easily and quick-
suitable for the decision making problem of com- ly. However, the main limitation is that the train-
putation offloading in a stochastic and dynamic ing phase of DIL heavily relies on a large number
edge computing network. For example, Dinh et of demonstrations, and it is hard to collect the
al. [7] studied the computation offloading prob- demonstrations.
lem in time-varying MEC environments. They con- In this work, we propose a DIL-based com-
sider a multi-user multi-MEC-server environment putation offloading strategy for edge computing
and propose a model-free reinforcement learn- networks. We first generate high-quality demon-
ing (RL) offloading scheme. The objective is to strations (i.e., the optimal offloading actions) and
make mobile users learn their long-term offload- train our model in an offline manner. Then, after a

IEEE Wireless Communications • February 2020 93

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.
TL allows us to deal quick and easy deployment, our model can make transfer through cellular transmission is determined
with variational envi- near-optimal online offloading decisions with a very by the data size of data exchange between sub-
ronments by leveraging fast online inference speed. tasks and the cellular data rates. In addition, the
the already existing Note that DIL is a traditional supervised learning edge execution time (i.e., for the SCceNB to exe-
approach, and its training and evaluation operate cute sub-task t) is determined by the total comput-
labeled data of some in the same domain. If we want to apply a trained ing resource of the available CPU cores.
related task or domain. model to a new domain, we can retrain the model, For the remote cloud execution, the end-to-end
In practical edge or take advantage of transfer learning (TL) [10]. (E2E) latency is decided by the RAN and core net-
computing scenarios, Transfer learning is the ability of a system to rec- work as well as the backhaul between them. In this
we can combine DIL ognize and apply knowledge and skills learned in article, we consider that the E2E delay consists of
previous domains/tasks to novel domains/tasks. wireless and wired delays. Let W denote the wired
and TL to deal with TL allows us to deal with variational environments delay between the SCceNB and the remote cloud.
more complex tasks by leveraging the already existing labeled data of Note that the delay consists of:
(e.g., finding optimal some related task or domain. In practical edge • The backhaul delay between SCceNB and the
resource allocation computing scenarios, we can combine DIL and TL core network
schemes) that are based to deal with more complex tasks (e.g., finding opti- • The processing delay of the core network
mal resource allocation schemes) that are based • The communication delay for data transmission
on already trained on already trained models. between the core network and remote cloud/
models. Internet
Computation Offloading Model
We study the computation offloading for a single
Problem Formulation
mobile device in a small cell-based MEC system. Decision Making Procedure
Note that the small cell-based MEC system con- When the mobile device receives the offloading
sists of: requirement of application A, it first sends a mes-
• Mobile devices sage on the data size D of the sub-tasks for the
• MEC server, also called small cell cloud-en- application. The report also includes the current
hanced e-Node B (SCceNB) wireless channel state (e.g., the channel quality
• Remote cloud between the mobile device and the SCceNB).
Thus, the mobile device can: After receiving the message, the SCceNB allo-
• Execute its computational tasks locally cates m subcarriers (m ∈ M) and n CPU cores
• Offload its tasks to the SCceNB through a wire- (n ∈ N) to each sub-task for the mobile device,
less link according to the entire available computation and
• Offload its tasks to the remote cloud through communication resources and the received mes-
wireless and backhaul links sage. Thus, the current system state of computa-
tion offloading can be denoted by S = (T, D, N, M,
Application Model W), which consists of the mobile device’s task pro-
We model a mobile application A as a weighted files, network resource status, as well as the wired
directed graph A = (T, D), where T represents delay status.
the sub-tasks, and D the data dependencies (i.e., According to the observed system state S, the
input and output data) between the sub-tasks. mobile device calculates the immediate costs of
Then we split the application into multiple sub- local-edge-cloud executions for each sub-task, and
tasks by fine-grained partitioning. Note that each makes action decisions of either processing the
sub-task of the application can be offloaded and sub-task locally or offloading to the edge-cloud side
executed independently. for the current mobile application A.
We adopt a parameter tuple 〈t, xt, dt–1,t, dt,t+1〉
to characterize the mobile application A for the Computation Offloading Optimization Problem
mobile device, where t is the current sub-task, xt(t The system state of the MEC network is given as
∈ T) represents the workload of sub-task t. dt–1,t S. Assume that the action space for computation
and dt,t+1 denote the size of input and output data offloading optimization is I = {It ∈ 0, 1, 2}, t ∈ T,
for sub-task t, respectively. Let r t (in CPU cycles indicating that the mobile device can execute a
per byte), denote the complexity of sub-task t. It sub-task t locally (It = 0) or offload the sub-task to
denotes the required CPU cycles a CPU core will SCceNB (It = 1) or to the remote cloud server (It
perform per byte for the input data processed by = 2). Under current system state S, E(S, It) denotes
sub-task t. Thus, xt can be given as xt = rt · dt–1,t. the execution cost of sub-task t, which is:
Note that x t is decided by the nature (e.g., algo- • The immediate local execution cost if sub-task t
rithm complexity) of the sub-task t. is executed locally
• The immediate edge offloading costs if the sub-
Execution Model task is executed at the SCceNB
The mobile device can process the mobile appli- • The immediate cloud offloading costs if the sub-
cation A locally. According to the application task is executed at the remote cloud server
parameter tuple, the task execution time for Apparently, the edge offloading cost consists
the mobile device to execute sub-task t locally of radio and computation resource usage cost,
is decided by the computation capacity of the the SCceNB computation cost (i.e., task execution
mobile device (in million instructions per second). time), and the data transmission cost (i.e., trans-
For the edge execution, the mobile device can mission delay) for offloading. The cloud offloading
establish a cellular link with the SCceNB and off- cost consists of radio and wired resource usage
load its own sub-tasks to the SCceNB via the radio cost, the remote cloud server computation cost
access network (RAN). Based on the assumptions (i.e., task execution time), and the data transmis-
above, the delay for sub-task input and output data sion cost (i.e., transmission delay) for offloading.

94 IEEE Wireless Communications • February 2020

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.
 Application Offloading Decision
0 : Local Execution
1 : Offload to MEC server
φ1 φ2 ... φV
2 : Offload to Remote cloud
Mobile e12
User C1 C2 C6

Input
Wireless Layer Hidden Layer
Output
Unit Mobile MEC Remote
Subcarrier ... Layer
User Server Cloud
1
1 C1

MEC
Subcarrier
2 ... 2 C2
...

Server
...
Subcarrier
N 1 C3

0 C4
Processor
... C5
0
CPU Core

...
1
1 C6
CPU Core Backhaul
2
...

Offloading
CPU Core
M Deep Neural Network Decision
System State

FIGURE 1. Proposed deep-imitation-learning-based offloading model.

Then the objective of the computation offloading decision in the output layer is a T-dimensional vec-
optimization problem is to obtain a near-optimal tor for the application. If a sub-task is offloaded,
offloading policy * that can minimize the offload- its value is 2 (cloud) or 1 (edge); otherwise, it is
ing cost given by t∈TE(S, It). Note that the offload- local. We define the multi-label offloading accu-
ing cost is the sum of costs for the sub-tasks of racy as the proportion of the predicted correct
mobile application A, which is not provided imme- labels to the total number of labels. Through the
diately. We can obtain the long-term cost when all accuracy, we can evaluate the output (i.e., pre-
the sub-tasks have been processed. dicted offloading actions) with respect to the opti-
mal offloading actions.
deep ImItAtIon leArnIng for Figure 2 illustrates the flowchart of our model.
It consists of three phases: offline demonstration
computAtIon offloAdIng generation, offline model training, and online
The optimization problem of minimizing the off- decision making. In the following, we describe
loading cost is a combinatorial optimization prob- these phases.
lem. Thus, it is impossible to achieve the optimal Offline Demonstration Generation: Based on
solution in real time by using standard optimiza- behavioral cloning [4], imitation learning performs
tion methods. Another possible approach is to supervised learning through imitating demonstra-
utilize RL. Nevertheless, since the action space is tions (i.e., optimal offloading action). Thus, the
defined over the combination of the execution objective of this phase is to generate demonstra-
selections for multiple sub-tasks, it suffers from the tions to train our DIL framework. We acquire a
curse of dimensionality and hence converges very large number of decision samples by leveraging
slowly in practical implementation. the offline optimization scheme for solving the opti-
To address these challenges, we explore a mization problem. In general, when the decision
novel scheme of autonomous computation off- space is:
loading decision by leveraging DIL. Intuitively, we • Small, we can use an exhaustive approach
first obtain the demonstrations (i.e., the optimal to obtain the optimal offloading decision by
decision samples) by solving the computation off- searching the whole action space (there are 3T
loading optimization problem in an offline manner. possibilities in the space).
Then, using these demonstrations, we train a DIL • Medium, the problem can be solved by some
model for imitating the optimal decision patterns mixed integer programming solver (e.g.,
and generate efficient online computation offload- CPLEX).
ing decisions in real time. • Huge, we can leverage some approximate
offline algorithms to obtain efficient decision
deep multI-lAbel clAssIfIcAtIon model for samples.
computAtIon offloAdIng Then the network state S as well as its optimal
As shown in Fig. 1, the optimization problem can offloading decision are recorded as raw decision
be formulated as a multi-label classification [11] samples to train our framework in the next phase.
problem. Assume that mobile application A con- Offline Model Training: In this phase, we use
sists of T sub-tasks. The input layer of our training the deep neural network (DNN) to extract and
model consists of the observation of the applica- train the features of training data. We conventional-
tion features and network states. Our offloading ly use the rectified linear unit (ReLU) as the activa-

IEEE Wireless Communications • February 2020 95

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.
 the outputs, we can evaluate the offloading accura-
cy and offloading costs of our DIL model.
begin
complexIty AnAlysIs
end Traditionally, using DIL to train an artificial intelli-
gence (AI) model is computation-intensive, espe-
Phase 1: Offline Demonstration
cially in the offline demonstration generation and
Gerenation (remote cloud side)
offline model training phases. Fortunately, it can
Phase 3: Online Decision be done using historical data in an offline manner.
system state S Thus, we can offload the data to the resourceful
Making (edge server/mobile
device side) remote cloud data center when the associated
computational overhead is high.
offline optimization π*:near-optimal In the offline demonstration generation phase,
scheme offloading policy the complexity for this phase is (|I|T), where
|I| represents the size of the action space I, and
T denotes the number of sub-tasks for the mobile
optimal decision online decision application. The complexity for the offline model
training phase is only (T3Q3), where Q represents
samples making
the number of neurons in each hidden layer. After
the offline training, our model can be deployed on
either the mobile side or the edge server side, in
order to make real-time offloading decisions. In the
online decision making phase, our decision model
offline model decision making
DNN training has constant complexity (1), which is highly scal-
training engine able and real-time.
Phase 2: Offline Model Training (remote cloud side) In order to alleviate the tension between
resource-intensive DNNs and resource-poor edge
servers, DNN compression can reduce the model
FIGURE 2. Flowchart of the proposed deep-imitation-learning-based offloading
complexity and resource requirement. Two typi-
framework. cal DNN compression technologies can be used:
weight pruning, which can remove redundant
weights (i.e., connections between neurons) from
1.1
ROS 1 a trained DNN, and data quantization, which can
1.0
GOS reduce the computation overhead by using a more
0.9 LOS compact format to represent layer inputs, weights,
0.8 EOS or both.
0.7 COS 0.64794
Accuracy

0.6
0.5
DROS
SOS
0.5932
0.6197
proof-of-concept performAnce evAluAtIon
0.4 DIOS sImulAtIon settIng
0.3 OOS In order to evaluate the performance of our DIL-
0.2 based offloading scheme, we consider a MEC
0.1 0.08715 network consisting of a mobile device and a MEC
0.0546 0.05
0.0011 0.01345 server. The number of CPU cores for the SCceNB
0.0
Offloading Schemes is set to be 16 (i.e., M = 16). For the edge network,
we consider the Rayleigh-fading environment, and
FIGURE 3. Comparison of offloading decision the total bandwidth is divided into 256 subcarriers
accuracy. (i.e., N = 256). The wired (backhaul) delay between
the SCceNB and the remote cloud is W ∈ [0.01,
0.02] s. FThe mobile application usually consists of
tion function for the hidden layers. Our offloading a few sub-tasks to dozens of sub-tasks in reality. In
model inputs the system state S and outputs off- this article, the mobile application consists of 6 sub-
loading decisions It(t = 1, 2, …, T). The sigmoid tasks (i.e., T = 6). The data dependencies and the
function is used as the output of our model. Note workload for the sub-tasks follow uniform distribu-
that it can be formulated as a multi-label classifica- tion, similar to [14]. Note that the random variables
tion problem to maximize the multi-label (i.e., pre- for different sub-tasks are independent.
dicted offloading actions) accuracy. We consider In the offline demonstration generation phase,
the cross-entropy loss [12] to measure the perfor- we use MATLAB to generate 100,000 demonstra-
mance of the model, and use the Adam optimizer tions, which means that the mobile application is
[13] to optimize the neural network. The output executed 100,000 times independently under var-
layer consists of T neurons that represent the off- ious network environments. At the same time, the
loading actions of the T sub-tasks. If an output sample of the optimal offloading scheme can be
neuron is less than 0.5, it denotes local execution; obtained in this phase. In the online decision mak-
otherwise, offloading. ing phase, we evaluate the performance of our
Online Decision Making: Once the offline DIL-based offloading scheme (DIOS) by leveraging
model training phase of the DNN is finished, it can the Jupyter notebook. We consider the following
be used to make real-time computation offload- eight benchmark schemes from the literature.
ing decisions in an online manner. At this time, the Optimal Offloading Scheme: We search the
DNN outputs a sequence of offloading decisions whole action space to find the optimal offloading
for all sub-tasks of the mobile application. Based on scheme (OOS).

96 IEEE Wireless Communications • February 2020

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.
 Local Offloading Scheme (LOS): The mobile Because deep learning
1.8
application is executed on the mobile device local- ROS can extract accurate
ly. Thus the offloading decision variables are It = 0, 1.7 GOS information from the
(t = 1, 2,…, T). LOS
huge IoT data in such
1.6
Deep-Reinforcement-Learning-Based Offload- EOS
ing Scheme (DROS): This is a computation off- complex network

System Cost
1.5 1.47696 COS
loading scheme that is based on the DRL method 1.41503 DROS environments. Com-
[9]. 1.4 1.38794
SOS pared to the traditional
1.31364
Greedy Algorithm-Based Offloading Scheme 1.3 DIOS machine learning meth-
(GOS): The mobile device chooses offloading 1.2384
OOS ods, deep learning out-
actions through a greedy algorithm, which means 1.2 1.15241.161971.13479
that the mobile device chooses the sub-action that performs in processing
1.1 1.07561
can maximize the offloading cost in each sub-task huge data, since it can
execution step. 1.0 precisely learn high-lev-
Offloading Schemes
Random Offloading Scheme (ROS): The off- el features (e.g., faces
loading decisions are generated randomly. FIGURE 4. Comparison of offloading cost. and voices), extracts
Shallow Learning-Based Offloading Scheme
(SOS): The number of hidden layers is set to be 1. new features automati-
Edge Offloading Scheme (EOS): With coarse 50 cally for different prob-
offloading strategies, the entire mobile application ROS lems, and takes much
45 GOS
is offloaded to the MEC server side. less time to inference

Task Execution Time (s)

Cloud Offloading Scheme (COS): In coarse 40 LOS
35 EOS information.
offloading strategies, the entire mobile application
is offloaded to the remote cloud side. 30 COS
DROS
25
evAluAtIon results 20
SOS
DIOS
Simulation results of our DIOS method are shown 16.05
15 OOS
in Figs. 3–5. 10.88
9.64 10.15 10.27 9.42 9.84
Figures 3 and 4 report the offloading accuracy 10 8.35 7.94

and corresponding offloading cost of different off- 5

loading schemes with respect to the OOS. Figure 3 0
shows that our DIOS outperforms other offloading Offloading Schemes
schemes in offloading accuracy. At the same time,
DIOS reduces the offloading cost on average by FIGURE 5. Comparison of task execution time.
19.80, 18.24, 23.17, 8.37, 13.61, 1.15, and 2.34
percent compared to the ROS, GOS, LOS, EOS,
COS, DROS, and SOS schemes, respectively. Note edge dAtA AnAlytIcs
that the EOS (offload computation to the edge) Edge data analytics refers to the analysis of data
performs better than COS (offload computation to from the distributed edge servers in a MEC sys-
the remote cloud) and LOS (local execution). This tem, and usually goes along with IoT applications
proves that the MEC server can reduce energy cost and data caching.
on the mobile terminal side, as well as the back- IoT Application Scenario: Recently, MEC
haul usage on the remote cloud side. has received extensive attention in IoT scenarios,
Figure 5 shows the task execution time using where inexpensive simple devices can generate
different offloading schemes with respect to the huge volumes of raw data for big data process-
OOS. Note that our DIOS reduces the execution ing. When considering the limited computation
time by 23.25, 8.77, 47.98, 17.73, 18.70, 11.36, and storage resources of each single edge serv-
and 15.14 percent compared to the ROS, GOS, er, applying traditional machine learning and AI
LOS, EOS, COS DROS, and SOS schemes, respec- algorithms (usually compute-intensive) is inefficient.
tively. Thus, one huge problem in this scenario is how to
As a proof of concept, the numerical perfor- process such big data in real time. We can apply
mance evaluation results above corroborate the deep learning in the MEC in order to improve the
feasibility and promise of the proposed DIL-driven efficiency of data analyzing and processing. Deep
computation offloading scheme. We are working learning can extract accurate information from
on exploring other deep neural network architec- the huge IoT data in such complex network envi-
tures such as deep residual learning [15] for further ronments. Compared to the traditional machine
performance gain and generalizing the approach learning methods, deep learning outperforms in
to the challenging multi-MEC multi-user scenario. processing huge data, since it can precisely learn
high-level features (e.g., faces and voices), extracts
future dIrectIons on new features automatically for different problems,
and takes much less time to infer information.
IntellIgent edge computIng Data Caching Scenario: Data caching is one
In the sections above, we focus on the deep- of the key features of a MEC system [1], and usu-
learning-based computation offloading approach ally consists of content caching and computation
for a MEC system. In this section, we further caching. Content caching refers to caching pop-
introduce several potential directions for apply- ular contents (e.g., segments of popular movies)
ing deep learning into multiple intelligent edge at the edge server in order to avoid retransmitting
computing research areas, including edge data the same contents. This approach can significantly
analytics, dynamic resource allocation, security, reduce the backhaul traffic and transmission delay,
and privacy, respectively. whereas computation caching denotes caching

IEEE Wireless Communications • February 2020 97

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.
Deep learning can pro- parts of popular computation result data (e.g., rec- can be executed on the mobile device locally,
vide the privacy pro- ognized face) that is likely to be reused by others. offloaded to an edge server, or offloaded to the
tection by transferring This approach can reduce not only the retransmis- remote cloud. In particular, we first introduce the
sensitive training data sion delay, but also the re-computation latency. application model and execution model, respective-
We can apply the deep supervised learning (DSL) ly. Then we present our offloading decision making
into intermediate data. method to the edge servers to analyze and extract procedure, and formulate the optimization prob-
Such intermediate data the features of the collected data from mobile lem to minimize the overall offloading cost. After
in DNN usually have devices. It makes more precise caching place- that, we propose a deep-imitation-learning-based
different semantics ment decisions than traditional machine learning algorithm to obtain a near-optimal solution rapidly
compared to the sen- approaches. Moreover, the popularity of different for the optimization problem. Numerical results
data is usually time-varying. Thus, we need to col- confirm that our proposal achieves an offloading
sitive training data. For lect and process large amounts of data to obtain accuracy up to 64.79 percent and reduces at most
example, after extract- statistical inference from the data. Thanks to the 23.17 percent offloading cost at the same time.
ing the features by the model-free feature, we can maximize the long-term At last, we discuss the important directions and
DNN filter, hackers cache hit rate through DSL without knowledge of advantages of applying deep learning methods to
cannot obtain the orig- the data popularity distribution. multiple MEC research areas.
inal information from Dynamic Resource Allocation Acknowledgments
the hidden layer. Dynamic resource allocation (DRA) is a key tech- This work was supported in part by the Nation-
nology to improve network performance in a al Key Research and Development Program of
dynamic environment. Note that the MEC perfor- China (No. 2017YFB1001703); the National Sci-
mance is influenced by a variety of time-varying ence Foundation of China (No. U1711265, No.
factors, including communication and computa- 61972432); the Fundamental Research Funds for
tion resources, workloads of mobile users, data the Central Universities (No. 17lgjc40, 19LGZD37);
caching and power management policies, and the Program for Basic and Applied Basic Research
so on, which is a huge project. Therefore, there Fund of Guangdong (No. 2019A1515010030); the
is a strong demand on intelligent edge resource Program for Guangdong Introducing Innovative
management to maximize long-term resource uti- and Enterpreneurial Teams (No. 2017ZT07X355);
lization. DRL has the potential to handle high-di- the Pearl River Talent Recruitment Program (No.
mension state spaces of complicated control 2017GC010465); the U.S. National Science Foun-
problems, and could be used to solve the DRA dation (IIS-1838024); and the Guangdong Special
problem for MEC. It makes edge servers automat- Support Program (2017TX04X148).
ically and efficiently negotiate the most appropri-
ate configuration directly from the complicated References
state space. Moreover, it can explore deep con- [1] ETSI, “Multi-access Edge Computing (MEC),” 2018; https://
www.etsi.org/technologies-clusters/technologies/ multi-ac-
nections in the data and obtain accurate prediction cess-edge-computing
of resource allocation schemes for MEC network. [2] J. Park et al., “Wireless Network Intelligence at the Edge,”
CoRR, vol. abs/1812.02858, 2018.
Security and Privacy [3] E. Li, L. Zeng, Z. Zhou, and X. Chen “Edge AI: On-Demand
Accelerating Deep Neural Network Inference via Edge
Recently, security and privacy issues pose a tough Computing.” IEEE Trans. Wireless Commun., vol. 19, no. 1,
challenge for the development of MEC. Security is Jan. 2020, pp. 447–57.
becoming an increasingly important issue in MEC- [4] T. Zhang et al., “Deep Imitation Learning for Complex
based applications. Since edge servers are located Manipulation Tasks from Virtual Reality Teleoperation,”
Proc. 2018 IEEE Int’l. Conf. Robotics and Automation, May
at the edge and physically closer to attackers. MEC 2018, pp. 1–8.
systems face multiple security threats such as wire- [5] X. Chen et al., “Efficient Multi-User Computation Offloading
less jamming, distributed denial of service (DoS) for Mobile-Edge Cloud Computing,” IEEE/ACM Trans. Net-
attacks, and smart attacks. Due to the sophistica- working, vol. 24, no. 5, Oct. 2016, pp. 2795–2808.
[6] Y. Mao, J. Zhang, and K. B. Letaief, “Dynamic Computation
tion and self-learning capability, deep learning pro- Offloading for Mobile-Edge Computing with Energy Har-
vides more accurate and faster processing than vesting Devices,” IEEE JSAC, vol. 34, no. 12, Dec. 2016, pp.
shallow learning algorithms. It can play a key role 3590–3605.
in attack detection to deal with attacks. The priva- [7] T. Q. Dinh et al., “Learning for Computation Offloading in
Mobile Edge Computing,” IEEE Trans. Commun., vol. 66, no.
cy issue is another important threat for the cloud- 12, Dec 2018, pp. 6353–67.
based MEC system, where users risk exposing their [8] Y. Zhang, D. Niyato, and P. Wang, “Offloading in Mobile
sensitive data by sharing it and allowing edge data Cloudlet Systems with Intermittent Connectivity,” IEEE Trans.
analytics. Moreover, MEC can provide location Mobile Computing, vol. 14, no. 12, Dec. 2015, pp. 2516–29.
[9] Y. He, N. Zhao, and H. Yin, “Integrated Networking, Cach-
awareness services for cellular-network-based appli- ing, and Computing for Connected Vehicles: A Deep Rein-
cations, which result in location privacy and tra- forcement Learning Approach,” IEEE Trans. Vehic. Tech., vol.
jectory privacy issues. Deep learning can provide 67, no. 1, Jan. 2018, pp. 44–55.
privacy protection by transferring sensitive train- [10] S. J. Pan and Q. Yang, “A Survey on Transfer Learning,”
IEEE Trans. Knowledge and Data Engineering, vol. 22, no. 10,
ing data into intermediate data. Such intermedi- Oct 2010, pp. 1345–59.
ate data in DNN usually have different semantics [11] G. Tsoumakas and I. Katakis, “Multi-Label Classification: An
compared to the sensitive training data. For exam- Overview,” Int’l. J. Data Warehousing and Mining, vol. 2007,
ple, as shown in Fig. 1, after extracting the features 2007, pp. 1–13.
[12] P. T. D. Boer et al., “A Tutorial on the Cross-Entropy Meth-
through the DNN filter, hackers cannot obtain the od,” Annals of Operations Research, vol. 134, 2002.
original information from the hidden layer. [13] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic
Optimization,” CoRR, vol. abs/1412.6980, 2014; http://
Conclusion arxiv.org/abs/1412.6980.
[14] C. You et al., “Asynchronous Mobile-Edge Computation
In this article, we study the fine-grained compu- Offloading: Energy-Efficient Resource Management,” IEEE
tation offloading issues for a single mobile device Trans. Wireless Commun., vol. 17, no. 11, Nov 2018, pp.
within MEC networks, that is, a computation task 7590–7605.

98 IEEE Wireless Communications • February 2020

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.
[15] K. He et al., “Deep Residual Learning for Image Recogni- Sciences, Beijing, China, in 2003, and his Ph.D. degree in com-
tion,” Proc. 2016 IEEE Conf. Computer Vision and Pattern puter science and engineering from the Chinese University of
Recognition, 2016, pp. 770–78. Hong Kong in 2007. He was a postdoctoral researcher with the
Department of Computer Science and Engineering, Polytechnic
Biographies Institute of New York University, Brooklyn, from 2007 to 2009,
advised by Prof. K. W. Ross. He is currently a professor and the
Shuai Yu is currently a postdoctoral research fellow in the School
of Data and Computer Science, Sun Yat-sen University, Guang- Assistant Dean of the School of Data and Computer Science
zhou, China. He received his Ph.D. degree from University Pierre at Sun Yat-sen University. His research interests include cloud/
and Marie Curie (now Sorbonne University), Paris, France, in edge computing, multimedia communication, Internet measure-
2018, his M.S. degree from Beijing University of Post and Tele- ment, and network security. He was a co-recipient of the IEEE
communications (BUPT), China, in 2014, and his B.S. degree from INFOCOM 2009 Best Paper Award and the IEEE Jack Neubauer
Nanjing University of Post and Telecommunications (NJUPT), Memorial Award (2019).
China, in 2009. His research interests include wireless communi-
cations, mobile computing, and machine learning. Mehdi Bennis is currently an associate professor with the Cen-
tre for Wireless Communications, University of Oulu, Finland,
X u C hen received his Ph.D. degree in information engineer- and also an Academy of Finland Research Fellow. His main
ing from the Chinese University of Hong Kong in 2012. He research interests are in radio resource management, hetero-
was a postdoctoral research associate at Arizona State Uni- geneous networks, game theory, and machine learning in 5G
versity, Tempe, from 2012 to 2014, and a Humboldt Scholar networks and beyond. He has coauthored one book and pub-
Fellow with the Institute of Computer Science, University of lished more than 200 research papers in international confer-
Goettingen, Germany, from 2014 to 2016. He is currently a ences, journals, and book chapters. He has been a recipient of
full professor at Sun Yat-sen University, and the Vice Director several awards, including the 2015 Fred W. Ellersick Prize from
of the National and Local Joint Engineering Laboratory of Dig- the IEEE Communications Society, the 2016 Best Tutorial Prize
ital Home Interactive Applications. He received the prestigious from the IEEE Communications Society, the 2017 EURASIP
Humboldt Research Fellowship from the Alexander von Hum- Best Paper Award for the Journal of Wireless Communications
boldt Foundation of Germany, the 2014 Hong Kong Young Sci- and Networks, the all-University of Oulu award for research,
entist Runner-Up Award, the 2016 Thousand Talents Plan Award and the 2019 IEEE ComSoc Radio Communications Commit-
for Young Professionals of China, the 2017 IEEE Communication tee Early Achievement Award. He is an Editor of IEEE Transac-
Society Asia-Pacific Outstanding Young Researcher Award, the tions on Communications.
2017 IEEE ComSoc Young Professional Best Paper Award, the
Honorable Mention Award from the 2010 IEEE International J unshan Z hang received his Ph.D. degree from the School
Conference on Intelligence and Security Informatics, the Best of Electrical and Computer Engineering), Purdue Universi-
Paper Runner-Up Award from the 2014 IEEE INFOCOM, and ty, West Lafayette, Indiana, in 2000.,In 2000, he joined the
the Best Paper Award from the 2017 IEEE ICC. He is currently School of Electrical, Computer and Energy Engineering, Ari-
an Associate Editor of the IEEE Internet of Things Journal and zona State University, Tempe, where he has been the Fulton
the IEEE Journal on Selected Areas in Communications Series on Chair Professor since 2015. His current research interests
Network Softwarization and Enablers. include information networks and data science, including
communication networks, the Internet of Things (IoT), fog
Lei Yang received his B.S. and M.S. degrees in electrical engi- computing, social networks, and smart grid. He was a recipi-
neering from Southeast University, Nanjing, China, in 2005 and ent of the ONR Young Investigator Award in 2005, the NSF
2008, respectively, and his Ph.D. degree from the School of Elec- CAREER Award in 2003, the IEEE Wireless Communications
trical Computer and Energy Engineering, Arizona State University, Technical Committee Recognition Award in 2016, the Ken-
Tempe, in 2012. He was a postdoctoral scholar at Princeton Uni- neth C. Sevcik Outstanding Student Paper Award of ACM
versity, New Jersey, and an assistant research professor with the SIGMETRICS/IFIP Performance in 2016, the Best Paper Run-
School of Electrical Computer and Energy Engineering, Arizona ner-Up Awards of IEEE INFOCOM 2009 and IEEE INFOCOM
State University. He is currently an assistant professor with the 2014, and the Best Paper Award at IEEE ICC 2008 and 2017.
Department of Computer Science and Engineering, University of He was a TPC Co-Chair for a number of major conferences
Nevada, Reno. His research interests include big data analytics, in communication networks, including IEEE INFOCOM 2012
edge computing and its applications in IoT and 5G, stochastic and ACM MOBIHOC 2015. He was the General Chair of
optimization and modeling in smart cities and cyber-physical sys- ACM/IEEE SEC 2017, WiOPT 2016, and IEEE Communication
tems, data privacy and security in crowdsensing, and optimization Theory Workshop 2007. He was an Associate Editor of IEEE
and control in mobile social networks. He was a recipient of the Transactions on Wireless Communications, an Editor of the
Best Paper Award Runner-Up award at IEEE INFOCOM 2014. He Computer Networks Journal, and an Editor of IEEE Wireless
is currently an Associate Editor for IEEE Access. Communications. He currently serves as the Editor-in-Chief
for IEEE Transactions on Wireless Communications, an Editor-
Di Wu received his B.S. degree from the University of Science at-Large for IEEE/ACM Transactions on Networking, and an
and Technology of China, Hefei, in 2000, his M.S. degree from Editor for IEEE Network. He was a Distinguished Lecturer of
the Institute of Computing Technology, Chinese Academy of the IEEE Communications Society.

IEEE Wireless Communications • February 2020 99

Authorized licensed use limited to: NEC Labs. Downloaded on January 05,2021 at 18:12:45 UTC from IEEE Xplore. Restrictions apply.