Received January 10, 2021, accepted January 22, 2021, date of publication February 17, 2021, date of current

version March 3, 2021.

Digital Object Identifier 10.1109/ACCESS.2021.3059652

Intelligent Environment Enabling

Autonomous Driving
MANZOOR AHMED KHAN
Department of Computer and Network Engineering, Collage of IT, UAE University (UAEU), Al Ain, United Arab Emirates
e-mail: [email protected]
The deployment of infrastructure was supported by projects supported by German Ministry of Transport and Digital Infrastructure, Berlin
Senate. The analyses of the publicly available data (available from the testroad) was carried out at UAEU, partially supported by Research
office under grant 31T140.

ABSTRACT Automated driving is expected to enormously evolve the transportation industry and ecosys-
tems. Advancement in communications and sensor technologies have further accelerated the realization
process of the autonomous driving goals. There are a number of autonomous driving initiatives around the
world with varying objectives and scope, e.g. vehicle perception in a controlled environment or highway
settings. Autonomous driving in a more complex environment with mixed traffic poses major challenges.
The solutions for such environments is the focus of this paper. We start with a quick overview of
current autonomous driving development activities worldwide. We then discuss the solution concept for
autonomous driving in urban environments and its enabling components, e.g. road digitization and flexible
communication infrastructure, to realize an urban autonomous driving testbed. We highlight the major
challenges hindering the realization use-cases of Level 5 autonomous driving. Solution sketches to address
these or similar changes are briefly discussed. We also implement some elements of the solution approaches
on the real test-road. We demonstrate an artificial intelligence based approach for the analysis of real traffic
data measured on the testbed. We implement approaches for predicting the network resource demands and
allocation, which are crucial for realizing the use-cases of autonomous driving in complex environments. For
the experiments, real data from the test-road is used. Results show that traffic patterns and resource demands
are predicted accurately. These experiments are expected to instrumental for realizing other use-cases of
autonomous driving.

INDEX TERMS Autonomous systems, intelligent vehicles, network function virtualization.

I. INTRODUCTION in 2007 focused on a 60 mile urban environment [3]. High-

The advancement of sensor and communication technologies Tech companies like Google have also become active in the
has accelerated the pace of realizing autonomous driving domain by initiating their autonomous driving activities. The
(AD). Although autonomous driving has gained more atten- journey of building AVs has made its way through the initial
tion in the recent past, the concept was implemented decades levels of automation and we will soon be witnessing Level 4
back and the farther past has witnessed a number of activities autonomous vehicles on public roads. Automobile makers
in this regard. Autonomous Vehicles (AV) date back to 70s, have already produced vehicles with features of level 3
when the first autonomous car was presented by Tsugawa automation, though some of the automobile makers have
at Japan’s Tsukuba mechanical engineering laboratory [1], recently withdrawn from the race e.g. Audi’s A8 Sedan [4].
which was then followed by various other activities around The Society of Automotive Engineers (SAE [5]) provides a
the world. Ernst Dickmann’s vision guided Mercedes Benz clear understanding of the SAE international J3016 standard
in 1980 could reach 39mph in a controlled environment [2]. describing the role of driver, system, and level of vehicle
The two Daimler-Benz vehicles VaMP and Vita-2 drove for automation. For each level, the Dynamic Driving Task (DDT)
over 620 miles in Paris in 1994. The DARPA urban challenge procedure and fallback policy are defined. DDT is described:
as the real-time loopback procedure required to maintain
The associate editor coordinating the review of this manuscript and the operational and tactical functions intact while driving a
approving it for publication was Maurice J. Khabbaz . vehicle. DDT fallback defines the course of action taken to

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 9, 2021 32997
 M. A. Khan: Intelligent Environment Enabling AD

FIGURE 1. Figure capturing the complex environmental dynamics for autonomous driving.

achieve required DDT performance or achieve minimal risk infrastructure. This is still a great challenge for autonomous
condition (MRC), in case of any vehicle failure, or Automated driving as evident from recent incidents involving AVs on
Driving System (ADS) failure or approaching exit from public roads. Cooperative driving relying purely on sensors
Operational Design Domain (ODD). ODD is the operat- is prone to sensor errors, processing delays, and line-of-sight
ing condition in which the system is intended; to perform restrictions [6].
the automation. At levels 1, 2, and 3 the DDT fallback The idea of autonomous driving rests on the capabilities
is performed by the driver, and at Level 4 and Level 5, of vehicles to understand their environment and to react to
the procedure is performed by the system, conditions applied. the dynamic events of the environment, which we term as the
In Level 4 automation, the automated driving system is vehicle’s perception or situational awareness. With the infor-
responsible for any lateral or longitudinal vehicle motion, the mation and sensory data from on-vehicle sensors, the vehicles
ADS is responsible for sensing, monitoring, and responding are able to create a perception of their environment. Vehicles’
to events. ADS is responsible for achieving the MRC in the perception together with the capability of interacting with
response of any vehicle failure, ADS failure, or approaching other vehicles does allow some level of automation but relies
an ODD exit. However, ADS may request the passengers to on the vehicle’s visibility. It is unclear as to how capable
intervene and perform the DDT but would not reply if the the AV can cope with different situations and environments.
passenger does not reply. In the case of Level 5 automation, Autonomous driving does pledge an increase in road safety.
the scope of ODD is unlimited. Meaning thereby, in any The fact that a vehicle’s sensors are a limiting factor on
condition, the ADS is responsible for DDT performance and the quality and extent of the vehicles perception is however
undertakes the DDT fallout procedure if required. detrimental to that pledge of increased safety.
The race towards achieving the goals of fully autonomous The challenging question is: Will autonomous vehicles be
vehicles (i.e., Level 5) is continuing and the stakeholders, able to cope with unprecedented and complex situations?
including car makers and technology leaders, are consis- Roads with unregulated traffic, temporary or dynamic obsta-
tently evolving their solutions to achieve the required level cles, vulnerable road users, sharp turns, etc., Figure 1 capture
of automation for their vehicles. A further step towards the these dynamics, by defining different road segments e.g., seg-
autonomous driving reality was achieved with the establish- ment 1 is with simplified setting i.e., a straight road with clear
ment of testing facilities and demonstrators around the world. road marking. Autonomous vehicles operate on knowledge
These testing environments are of vital importance for eval- from past experiences, either built-in by engineers or trained
uating the capabilities of the autonomous vehicles under dif- using machine learning. However, not all variables and situa-
ferent environmental dynamics. One of the major objectives tions for decision-making are known in advance. Relying on
of testing facilities is to create a realistic environment that the information from on-vehicle sensors alone or implement-
best represents the real environment autonomous vehicles are ing pre-trained reactions to events may not suffice to achieve
going to operate in. the goals of level 5 autonomous driving [5].
Dynamic traffic situations on public roads require coor- To improve the situational awareness and environment
dination between the AV, conventional vehicles and road understanding of vehicles in complex urban environments,

32998 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

an improved version of Cooperative, Connected and Auto- buildings, sidewalks, obstacles, and footpaths. AV function-
mated Mobility (CCAM) is provisioned, which is expected alities for different use cases are evaluated by simulating
to push the automated driving to the next level of vehicle scenario specific environments. Similarly, there are initiatives
automation (i.e., SAE Level 4 and Level 5). This asks for with closed and controlled testing facilities to evaluate the
the upgrade of existing street furniture and key assets e.g., performance of AV solutions. Shanghai International Auto-
network infrastructure, roadside and cloud infrastructure and mobile City in China, K-City autonomous driving facility
vehicles to realise improved performance of autonomous in Korea [9], or Gunma University’s Aramaki campus in
driving in complex urban environment. Japan [10] are some of the examples for controlled test facil-
These requirements drive a project of Germany (author ities that are developed through public private partnerships.
was the technical lead of the project) [7] to follow the Automobile makers have also been involved in creating such
philosophy of ‘‘Intelligent vehicle is good but intelligent testing facilities, e.g. Toyota Research Institute’s oval track
environment is better’’. The project focused on providing in the USA. Readers are also encouraged to refer to the
an end-to-end solution with a distributed and autonomous following list of autonomous driving testing facilities around
decision-making framework that is envisioned as a promis- the world: e.g., C-ITS test corridor [11], [12], Colorado
ing step towards achieving human-like perception within testroad [13], Columbus connected corridor [14], Forth Road
the vehicle and ensures road safety with fully autonomous Bridge Corridor in Edinburgh [15], [16] featuring full sized
vehicles. autonomous buses operating at AV Level 4 autonomy, DTU
In this work, we discuss the necessary ingredients and Lyngby Campus corridor [17], On-demand self-driving car
solutions that help achieve the objectives of higher automa- service to Frisco, Texas [18], and iMove, Autonomous park-
tion levels. We provide details of road infrastructure and ing service for VW, Audi, and Porsche vehicles at Hamburg
communication infrastructure for cooperative, connected and Airport [19]. There are large number of test-roads focusing
automated mobility (CCAM) use cases and demonstrate on investigating different aspects of autonomous driving.
their implementations. The paper is structured as follows: Readers are encouraged to refer to [20] for an updated list
In Section II, we present efforts around the world to create of the test-roads around the globe and their scope.
testing facilities for autonomous driving. Section III details Most of the facilities are in controlled or closed envi-
how a testing facility for autonomous driving in an urban ronment, which do provide more realistic environmental
environment is realized. Sections IV and V present this testing dynamics but pose the challenge to properly replicate the
facility’s distributed architecture of the compute and storage dynamics of public roads in the controlled testing facility.
infrastructure and the communication infrastructure respec- It goes without saying that no matter how detailed the envi-
tively. Section VI highlights major challenges hindering the ronmental dynamics are modeled for validating AV solutions
realization of fully autonomous driving. Some examples of in simulation or controlled environments, the real environ-
implemented use cases are presented in Section VII. Poten- ment (urban/rural) presents with unprecedented dynamics
tial solution sketches are discussed in Section VIII. This that cannot easily be captured. This strengthens the need for
article also focuses on realizing the flexible communication open and urban test facilities.
infrastructure employing 5G enabling technologies and end- Consequently, recent clarifications of legal frameworks for
to-end autonomous management enabled by artificial intel- autonomous vehicles testing and operation have paved the
ligence (AI) i.e., demand attentive infrastructure adaptation way for testing facilities to be extended or built on public
based on learned traffic patterns. We demonstrate the fun- highways and even in urban areas, especially in the US,
damental step, which is the analysis of real traffic data col- Sweden, Germany, and China. These real environments allow
lected by various traffic sensors in the urban autonomous probing AVs in a wider breadth of autonomous opera-
driving testbed. Knowing future traffic demands provides tions, for example cooperative driving and mixed traffic.
valuable inputs for efficient, low delay management of var- End-to-end intelligent transport services, e.g. parking man-
ious autonomous driving infrastructure operations, e.g. com- agement, route planning, traffic control, infotainment can
munication, MEC, or vertical services, this is discussed in also be developed and tested there. For this purpose, public
section IX-A. Section X concludes the paper. roads are upgraded with sensor and communication infras-
tructures supporting AVs. Germany’s ‘‘Digital Motorway
II. RELATED PROJECTS AND ENABLING TECHNOLOGIES Testbed’’ on A9 autobahn [21] is the first public road that
Realistic testing tracks are crucial to probing the capabil- allows testing AVs, which recognizes the importance of the
ities of autonomous vehicles. Various autonomous driving environment perception. The digitization of the A9 segments
labs aim at providing realistic traffic situations for train- includes, among others, communication infrastructure allow-
ing intelligent vehicles, e.g. obstacles detection, self-braking ing V2X communication and the use of 700 Mhz bands for
and steering, and collision avoidance. Mcity [8] is a testing V2V communication. Additional public testbeds are being
facility at the North Campus of Michigan University in Ann deployed in several German cities, e.g. Karlsruhe, Düssel-
Arbor, which aims at evaluating the potential of automated dorf, Berlin. In the city of Karlsruhe, the test site include
and connected vehicles by simulating the urban environment. different road types, from reduced-speed areas and parking
The simulation includes urban roads, roundabouts, crossings, lots up to interstate and highway roads. Urban test beds

VOLUME 9, 2021 32999

 M. A. Khan: Intelligent Environment Enabling AD

worldwide also aim at providing complex public environ- III. THE INGREDIENTS FOR AUTONOMOUS DRIVING
ments for testing innovations of autonomous driving, e.g. IN URBAN AND OPEN ENVIRONMENT
London (UK), Brainport (NL), Tampere (FI), Vigo (SP), or The challenges of urban and open environments are a lot
Daejeon (KR). In the mentioned public test beds, commu- different than those of controlled, rural, and highway envi-
nication infrastructure is identified as an important enabler ronments. This work is in parts inspired by the Diginetps
for autonomous driving, which explains the participation of and Smart City Berlin projects, an autonomous driving test-
network operators and providers of vertical services in corri- road and digitized roundabout in Berlin. To understand the
dor use-cases i.e., A2-M2 corridor [22] in the UK connecting challenges, let us have a brief overview of the ground truths of
London urban and link roads in Kent or 5G network cover- urban test-road for autonomous driving at the center of Berlin,
age for the Thessaloniki - Sofia - Belgrade corridor, which Germany. There are two roundabouts with 5 ins and 5 outs,
will allow tests to be conducted with AVs over hundreds of the road itself has three lanes in each direction. Co-existence
kilometers of motorways. of an autonomous vehicle with conventional vehicles in these
Achieving the evolved CCAM for higher levels of complex roundabouts require efficient control and extensive
autonomous driving is expected to be heavily comple- knowledge of the roundabouts and their vicinity. We believe
mented by the communication infrastructure that meets that relying on vehicle research alone will not suffice to reach
the service requirements (e.g., bandwidth, delay, sup- the goals of Level 5 autonomous driving, which is why we
ported speed for handover management, connection den- suggest a three-level solution architecture i.e., vehicle level,
sity, etc.) of communication amongst CCAM infrastructure, roadside level, and central data-center level. The proposed
autonomous vehicles, and backends of the automobile big picture is depicted in Figure 3. At each level, the sensory
makers. Recognizing the importance of communication data is collected by the IoT middleware that further makes the
infrastructure for cooperative driving, recent development of data available to a smart decision engine that encamps vari-
future communication network technologies aims at identi- ous AI mechanisms for different decision making instances.
fying the requirements, the architecture and the approaches The middleware APIs are exposed for developer of different
with focus on autonomous driving applications. Standard- applications including traffic, parking, impact of traffic inten-
ization groups like the Next Generation Mobile Networks sity on environment, traffic data flow management, security,
Alliance (NGMN) V2X task force, 5G Automotive Associ- filtering, and others. Let us now discuss the three levels and
ation (5GAA), and TD-LTE, are cooperating with the auto- their operations, which we depict in Figure 3.
motive industry to promote LTE and New Radio (NR) based
V2X solutions known as cellular V2X (C-V2X) technology. A. INTELLIGENCE AT THE VEHICLE LEVEL
Multi-Access edge computing (MEC) extends cloud comput- The Level 5 autonomous vehicles are required to have the
ing capabilities closer to devices and services to the network competencies of a human driver, specifically the ability to
edge and is standardized by ETSI MEC group [23]. It is make safe and rule-conforming decisions based on their
increasingly recognized as an important enabler for future environment perception. An autonomous vehicle builds its
mobile networks to support autonomous driving. MEC infras- perception of the environment based on the sensory data
tructure has been designed to realize vehicle to everything it captures through on-board sensors. It corresponds to the
(V2X) communication in various autonomous driving test capability of the vehicle to know its environment including
beds, forexample in C-V2X (China), CAV (UK), or Concorda obstacles, road markings, traffic lights, other vehicles, pedes-
(NL). An extensive review of MEC architectures is provided trians, cyclists, or objects. The autonomous vehicle is a com-
in [24]. bination of sensors, actuators, sophisticated algorithms, and
Beside the various digitization and communication tech- powerful processors to execute software. There are hundreds
nologies for enabling constant data flows, a cooperative of such sensors and actuators in an AV, which are situated
and distributed decision making framework also plays a in various parts of the vehicle. All these sensors feed into
crucial role in enabling a stable autonomous driving sys- what we know as Local Dynamic Map (LDM), which holds
tem. In [25], the benefit of cloud-based, central vehicles all the vehicles knowledge. The LDM plays a key role when it
control making use of collective sensing data from multi- comes to the decision making at the vehicle level. In Figure 4,
ple vehicles is realized. The authors demonstrate flexible we depict the well known four hierarchical layer structure
coordination of data processing between data center and of the LDM [27], ranging from very static environmental
MEC infrastructure to eliminate the former’s delay con- information to transient static to transient dynamic to very
straint and the latter’s resource constraint in order to real- dynamic information.
ize stability of AV fleets. The authors of [26] propose an The standardized four layered architecture of LDM may
integrated urban traffic management architecture with 5G not cope with the requirements of Level 5 autonomous
and MEC. The work demonstrates the benefit of an effi- driving i.e., capturing the real-time and external informa-
cient communication system in supporting vehicle local- tion (e.g., perception created through on-road deployed sen-
ization, data pre-fetching, traffic light control, and traffic sors, information from backends of automobile makers, city
prediction during an accident rescue operation in urban authorities, prediction of environmental variables from edge,
settings. etc.). Hence, evolution of the classical LDM is imperative.

33000 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

FIGURE 2. Different options/views of the developed AV GUI.

create a predicted environment layer in the evolved LDM

that assists in decision instances like trajectory planning, lane
change, or speed adaption. We skip the details of imple-
mented algorithms, as they are out of the scope of this
paper and will be published in a separate article. However,
readers are encouraged to refer to our earlier publications
using similar approaches [28], [29]. In Figure 2, we show the
screenshots of the GUI while carrying out some experiments
to study the performance of extended perception i.e., through
fusing the data from external sources (e.g., from the on-road
deployed sensors).

B. INTELLIGENCE AT THE ROADSIDE LEVEL

With the view to extending the vision of autonomous vehicles,
the environment is digitized and the distant information is
made available to the vehicles. The purpose of digitizing the
environment is to enable the environment to communicate
with the autonomous vehicles and other relevant stakeholders
(e.g., city authorities, communication service providers, etc.).
Digitization helps to describe the dynamic features of roads,
e.g. i) road condition sensors notify the vehicles if the road
segment is icy or muddy, ii) car parking sensors update the
FIGURE 3. 3-layer architecture of perception creation and decision
making. autonomous vehicles of available parking spaces in the vicin-
ity, iii) traffic analysis sensors keep updating the autonomous
vehicles on the traffic situation in different road segments
that are even miles away, iv) environment sensors assist the
autonomous vehicles to reduce their impact on emissions and
other environmental variables. In Table 1, we provide the
high level technical description and the technologies of the
required sensors. The last column of table lists the parameters
measured by sensors.
Roadside level is the intermediate level in the architec-
ture. It corresponds to an enhanced roadside unit (eRSU),
FIGURE 4. The traditional LDM layers of information.
which hosts computer hardware and communication inter-
faces. It does not only allow Vehicle to Infrastructure/Vehicle
We propose the extension of the LDM by introducing addi- (V2X) communication but also provides a platform to host
tional information layers that are fed with information from local services. It allows the connectivity to the near edge
external sources and other stakeholders. A version of the of the mobile operator. This level creates upstream link to
evoled LDM is developed by the author and his research connect with central data centre and creates downstream
group. The evolved LDM is capable of capturing and trans- link to connect with on-road deployed sensors and vehicles.
lating the external information. We have also designed and The eRSU on the downstream connects with various on-road
developed coalitional learning approaches for improving the deployed sensors over Wi-Fi and to vehicles over DSRC
perception and by training the machine learning models with (802.11p) links. Being capable of collecting the data from
the local data of on-vehicles and on-road deployed sensors environment (via on-road deployed sensors) and from vehi-
(Similar to federated learning approaches). We were able cles, we implement approaches for sensory data fusion and

VOLUME 9, 2021 33001

 M. A. Khan: Intelligent Environment Enabling AD

TABLE 1. Types and description of sensors.

filtering to create patterns for different use cases. For this

purpose, we deploy an IoT middleware, an optimization and
machine learning toolbox, and a decision toolbox in the eRSU
(cf. Figure 3). The eRSU creates a map based on the infor-
mation collected from connected road sensors and vehicles.
We term the knowledge gained based on external environ-
ment information Perception at Edge (PAE). It is shared with
the vehicles for more informed decision making. For ready
reference the components of the proposed architecture at this
layer is pictorially shown in Figure 5.
FIGURE 5. An overview of the edge architecture.
C. INTELLIGENCE AT THE DATA-CENTER LEVEL
This level has the global view, as it communicates with all
the eRSUs and the vehicles. It hosts the backend of all the (e.g., [28], [31]) for modeling the decision mechanisms in
on-road deployed sensors, the communication network core, different scenarios.
the network operation controllers, the service orchestrator, In the next sections, we further provide the detailed spec-
computation infrastructure and the optimization and machine ifications of the infrastructure that support the operation and
learning toolboxes. The real-time and delay sensitive deci- shared perception among the three intelligent layers dis-
sions taken by the vehicle and edge levels. However, at the cussed in this section.
central data-center level, the patterns are created based on the
sensory data collected from vehicles and all on-road sensors. IV. MULTI-ACCESS EDGE AND CLOUD INFRASTRUCTURE
This global knowledge is contained in the Global Information The autonomous driving infrastructure plays an important
Module (GIM), which is shared with the lower level decision role in enabling CCAM operations, e.g., by enabling extended
making entities for a coordinated and consistent operation perception. A detailed architecture of the system components
of the overall infrastructure. In section IX-A, we demon- and connectivity of such infrastructure is depicted in Figure 6.
strate the extraction of traffic demand patterns from sensor In this section, we provide a detailed specification of the
data and prediction of future demand with machine learn- road infrastructure including the application protocols and
ing approach. The predicted results allow different network message, and essential software and hardware components.
and service orchestrators to make informed decisions for the
managed infrastructures. For example, a proactive mobility A. V2X INTERFACES SPECIFICATION
management approach based on user’s mobility patterns was For different use case scenarios, the AVs rely on V2I and
proposed in our previous work [30]. We rely on our previous V2V communication to exchange perception and

33002 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

FIGURE 6. End-to-end infrastructure components and interfaces for CCAM application.

control information. The LDM component in the AV control application protocols and messages are
constantly updates its environment perception and shares implemented, e.g. Decentralised Environmental Noti-
this information with the PAE instances hosted on the fication Messages (DENM), Cooperative Awareness
eRSUs using available Wi-Fi, C-V2X (PC5 sidelink) inter- Messages (CAM) or Basic Safety Message (BSM).
faces. This allows additional information for the PAE and Other roadside infrastructure related messages are
GIM to construct a virtual map of a broader area. The also transmitted over the DSRC interface: Sig-
global map is also shared with all vehicles in the area nal Phase and Timing Message (SPAT), In Vehi-
through PAE applications. Based on the environment per- cle Information Message (IVI), and Service Request
ceptions, the AVs coordinate their driving decision through Message (SRM).
V2V interfaces. The vehicle interfaces are specified as • V2I: Communication for CCAM applications relies on
follows: high bandwidth technologies with less stringent delay
• V2V: Interfaces include both short-range low delay requirements in contrast to the short-range communica-
communication for autonomous driving operations and tion technology. These interfaces are based on C-V2X
high bandwidth communication for CCAM applica- (LTE-V2X, 5G-V2X) and Wi-Fi technologies for the
tion and vehicle perception. The short-range interface transfer of media data, messages of the ITS V2X ref-
is based on DSRC (802.11p) wireless technology. erence architecture protocols [32], or other IP-based
On DSRC interface, cooperative driving and traffic protocols.

VOLUME 9, 2021 33003

 M. A. Khan: Intelligent Environment Enabling AD

B. E-RSU INTERFACES SPECIFICATION applications are required to be placed towards UEs and AVs,
CCAM infrastructure operations involve the real-time update they must be dynamically deployed on MEH available in
of the GIM and PAE maps, their interactions with other ITS different mobile network segments.
and road management applications, and the management of Intuitively, SC segment will produce and consume most
network and edge computing resources for those applications. application data. It goes without saying that applications for
CCAM applications have a global instance deployed in data autonomous driving are mainly data sensitive e.g., appli-
center while location specific instances deployed on-demand cations for generating collision warnings, applications for
on eRSUs. PAE instances on eRSUs are constantly fed with identifying the vulnerable road users, etc. The data locally
data from nearby road sensors through Wi-Fi and wired generated by AV sensors and roadside components are com-
interfaces. They synchronize the analyzed sensor information bined with downstream data from cloud services to provide
with the global instance using application specific protocols AV agents context for autonomous decision-making. With
through cellular and wired interfaces. The RSU interfaces are their inherent computing capacity, AVs can be seen as moving
specified as follows: MEHs. This raises the challenges for the management of
mobile edge platforms (MEP) and the orchestration of edge
• eRSU-2-eRSU: communication is based on high band-
applications. Due to the wider coverage of Radio Access Net-
width and low latency direct Wi-Fi connection. These work (RAN) components (eNB/gNB), mobile edge applica-
links serve as reliable transport and back-haul network tions (MEA) are expected to aggregate situational data from
for road side infrastructure. The interfaces between
AVs and road sensors and provide AVs with a broader con-
eRSUs is used by the Software Defined Network (SDN)
text for more strategical decision-making. Samples of such
data and control plane, which provides a virtualized
applications are traffic information and route planning ser-
network for the distributed MEC platform hosted on
vices. Where eNBs provide wireless backhaul to the eRSUs,
the eRSUs. additional network management functions, e.g. mobility
• eRSU-2-MEC/Cloud: communication is provided by management or resource management, could employ the
4G/5G. The use of Mobile Network Operator’s MEC
associated MEHs to increase network control efficiency by
infrastructure in the core network (CN) for CCAM appli-
timely reconfiguration of network components. Towards CN
cations enables their low delay communication from
and service segment, critical MEAs are increasingly con-
eRSU. Autonomous vehicles with broadband access can
cerned with the management and orchestration of mobile net-
make use of this interface to access services hosted in the work provisioning, and MEC infrastructure in lower network
road infrastructure. segment. Disappearing network and computing constraints of
centralized computing infrastructures allows CN and service
C. E-RSU AND MEC SPECIFICATION applications to be deployed in in data center or on a dedicated
Based on the eRSU interface specification above, the eRSUs server. Nevertheless, they may take advantage of manage-
include network interfaces and computing capability. For ment and orchestration functions for MEP i.e., sharing of
the access network, the roadside unit provides DSRC and user context, dynamic migration, load balancing, or high
Wi-Fi interfaces. The transport network has two inter- availability. The application services must be designed and
faces: Cellular network and a redundant P2P wireless link. implemented with the support for cloud-based provisioning
In some cases, neighbouring eRSUs are connected through paradigms, e.g. XaaS, container, and micro-service-based
another P2P wireless link. To provide edge computing architectures.
resources, a machine may be connected to the router in the In proportion with the high degree of distribution of MEHs,
roadside unit. Depending on the deployment, roadside sen- management and orchestration (MANO) of both MEPs and
sors are connected through either wired or wireless con- application services has to deal with increasingly com-
nections. Given that the transport network is created by plex network operations, service provisioning and compo-
wireless links, only a 230V power supply is required to sitions. While distributed MANO functions themselves can
provide power to the devices (230V, Power-over-Ethernet be deployed on MEHs closer to the network segments to be
(PoE), PoE+). managed, centralized MANO in system layer is an inevitable
Given the high mobility nature of autonomous vehicle part of the architecture. With the global view and aggregated
applications, the mobile broadband network has been the intelligence, it deals with the business objectives of multi-
baseline infrastructure for the design of their MEC enabled ple stakeholders i.e., service providers, operators, and users.
architecture. The ETSI MEC reference architecture [23] com- These objectives are translated to lower level operational
ponents are required to be flexibly placed in cellular infras- objectives and configurations to be carried out by the respec-
tructure. eRSUs coverage extend the mobile network with tive network, VIM and MEP control elements. As a result,
small cell (SC) segments, which plays a significant role in the control interfaces (reference points) between MANO ele-
future autonomous driving scenarios. The Mobile Edge Host ments constitutes a MEC control plane, which utilizes non-
(MEH) in the MEC architecture corresponds to the eRSU negligible mobile network data plane resources. The delay
component that provides computing, storage, and network and reserved bandwidth for the MEC control plane must be
capacity for edge applications. Depending on how close the guaranteed to provide consistence and stable operation of

33004 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

the MEC platform depending on the selected application and application (i.e., GIM). Examples for such services are
virtualization technologies. the control center, data collection and storage, data ana-
lytics, machine learning components, etc.
D. DATA-CENTER INFRASTRUCTURE INTERFACES • IoT middleware, data fusion, analytics, etc.
SPECIFICATION • Autonomous Network and Service Management Frame-
Beside the centralized CCAM service instances, the cloud work: it is based on a meta-machine learning approach
infrastructure also hosts network management and orchestra- that enables the autonomic management of network enti-
tion (MANO) applications. The network slice management ties and dynamically orchestrates the services in differ-
and orchestration service deployed in the central cloud plat- ent segments of the network.
form may coordinate difference network functions deployed • Software Defined Network (SDN) enabled core and
on eRSUs and AVs. Mobility and application aware net- transport network: is designed and developed as cus-
work management protocols are developed to meet e2e QoS tomized virtualized lite-core network (L-EPC) that
requirements of CCAM applications. The cloud interfaces are implements specialized network functions.
specified as follows:
V. FLEXIBLE COMMUNICATION INFRASTRUCTURE
• eRSU (MEC)-2-Cloud: To meet the high availability and
The communication infrastructure enables vehicle to vehicle
QoS requirements of CCAM scenarios, MEC infrastruc-
(V2V), vehicle to sensors, vehicle to data-center, and sensors
ture with central and distributed mobile edge hosts is
to data-center communication. Figure 7 presents an overview
the main platform for microservice based CCAM appli-
of the communication infrastructure of the testbed. In this
cations. Mobile edge CCAM services are developed as
section, the two constituting parts of the communication net-
web services and composed by REST based protocols.
work are described: roadside access and transport network,
• Multi-Cloud: Platforms with large computing and net-
and mobile broadband network.
work capacities host various backend components of
CCAM services, e.g. data analytics, data fusion pro-
cesses, data bases, or service implementations. Depend-
ing on application scenario and service providers,
CCAM services can be deployed on cloud, near edge,
and far edge platforms. This raises the challenges for
service composition and interactions. The solutions for
such a highly flexible deployment is based on network
virtualization technologies, e.g. SD-WAN, tunnelling,
and hybrid cloud management and orchestration.
FIGURE 7. An overview of communication infrastructure architecture.
E. CLOUD INFRASTRUCTURE SPECIFICATION
The cloud infrastructure consists of a data center and a far
edge. The cloud infrastructure provides different QoS for A. SOFTWARE DEFINED ACCESS AND TRANSPORT
CCAM services allowing them to be flexibly deployed to NETWORK
meet the use cases’ requirements. It is connected to the Figure 8 depicts the solution approaches at the roadside
Internet and external network through the data center net- level, which includes the software defined eRSU access and
work. Additionally, a SDN based transport network allows a transport network connecting them. Each eRSU has an SDN
direct high bandwidth and low latency connectivity between switch with multiple radio interfaces, e.g., Wi-Fi, DSRC, etc.
the data centre and other (eRSU) infrastructure in the test- A logically centralized controller is able to control the data
road. The far edge infrastructure provides universal access flows in the access network. The centralized control plane
to deployed CCAM services through mobile broad band net- is implemented as multiple controllers, which coordinate the
work. In contrast to the access to public cloud infrastructure control of multiple access network segments.
through the Internet, the direct connection between the far To enable SDN control of wireless interfaces, multiple
edge infrastructure and cellular CN allows very low delay extensions and customization are made to the OpenFlow
connectivity to the deployed services. protocol implementation and wireless network stack. A vir-
Depending on the QoS requirements, autonomous driv- tual interface managed by Open vSwitch is created for each
ing and network infrastructure management services may be wireless interface. The wireless stack converts wireless net-
deployed on far edge or data centre. These services provide work events and measurements to Open vSwitch data and
the solution approaches at the data-centre level to support update the flow tables for the managed interfaces, as shown
autonomous driving use cases: in the left of Figure 8. To overcome the lack of features
• Autonomous Driving Services at the Global Level: and restrictions current OpenFlow version (v1.5), various
Coordinate the distributed CCAM service instances extension are proposed and implemented as customized SDN
and provide global context for autonomous driving switch for the eRSU. This includes new OpenFlow header,

VOLUME 9, 2021 33005

 M. A. Khan: Intelligent Environment Enabling AD

FIGURE 8. SDNized wireless network.

instructions, and actions. Further extensions are made to

Open vSwitch data path (kernel forwarding path) by apply-
ing protocol oblivious forwarding (POF [33]) paradigm.
Packet processing rules are implemented as eBPF programs FIGURE 9. Flexible communication infrastructure for CCAM application.
and installed to the SDN switches using OpenFlow proto-
col by the central controller. The controller implements an 1) AD SERVICE HANDOVER
autonomous network and service management framework. To ensure the seamless mobility of the vehicles and avail-
It incorporates meta-machine learning approaches on global ability of the AD services throughout the road segments,
network information that enables the autonomic management mechanisms should be in place not only on the intra-operator
network entities and dynamically orchestrates the services (i.e., support for both horizontal and vertical handovers) but
in different segments of the network. As a result, proactive also at the inter-operator levels. This asks for agreement on
flow control rules are calculated to dynamically adapt the the inter-core interactions and accordingly the development
network to application requirements and network states. The of fitting interfaces/protocols. The situation gets more com-
proactive rules allows the low delay processing of packet plex when the AD services are provided by the third parties
in data plane, increase control plane efficiency, and support and where the authorities are actively involved. One potential
novel delegated mobility management. More details of the direction to address this or similar issues is dynamic network
mentioned techniques are provided in [30], [34]–[36]. slicing. This is to say that an operator may compose a service
specific network slice, which the technologies of the target
B. BEYOND 5G NETWORK TECHNOLOGIES FOR CCAM operator may replicate by implementing proactive mobility
While the SDNized eRSU network can efficiently manage management and dynamic Service Level Agreement (SLAs)
the flows of large amount of data generated by the road- approaches. In this connection, core network’s APIs may
side sensor and processed by roadside MEC infrastructure, be exposed to the other operator and stakeholders. Amongst
the mobile broadband access provides a reliable coverage others, this will also allow flexible routing of user data plane
for the communication with the global CCAM services in and accessing user context for 3rd party MEC infrastructure.
central data-center platform. The lite-EPC core is an in- We implement an architecture for the interconnection of the
house developed LTE core with focus on mobility manage- virtualized lite-EPC and 5G core network, inspired by the
ment, session continuity to support high mobility operation architecture depicted in Figure 10, which is based on the local
of AVs. The testbed may also provide emulate the multi- breakout architecture specified by 3GPP [37].
operator and mobility management scenarios, which allow In a typical scenario, CCAM services hosted on the MEC
the investigation of handover delays and service switchover are accessible to AVs and users through 5G core network
delays when the communication bit-pipes are provided by dif- (i.e., P-GW). The functionalities of the 5G network are stored
ferent providers. Additional interfaces may be implemented in the Network Resource Function (NRF), whereas the ser-
to allow the interactions with other LTE/5G CN and the vices produced by the MEC are registered in the service
end-to-end services and network orchestration platforms as registry of the MEC platform. Hence, to use the 5G ser-
shown in Figure 9. The two core networks may implement vices, MEC communicates with the Network Expose Func-
all the autonomous network management services, for this we tion (NEF), which allows 3rd party application to access the
rely on author’s earlier publications. We next provide details APIs of 5G CN functions. NEF will act as a centralized point
of the core network features specific to the AD operations of to expose the services, and it will also check if all the requests
connected infrastructure and vehicles. coming from outside the system are authorized to access the

33006 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

do not share a PDU Session; though different Network Slice

instances may have slice-specific PDU sessions using the
same data network name. During the Handover procedure
the source AMF selects a target AMF by interacting with the
NRF. Further description of the related procedures is provided
in [37], [38].
One important component of the network slicing infras-
tructure deployed in the testbed is the management and
orchestration (MANO) platform with slice management
functions as part of the service chain as shown in Figure 11.
These functions are managed by the global MANO plat-
form and expose a subset of management functions to slice
consumers. The MANO platform realizes a service delivery
paradigm with highly integrated networks, computing infras-
tructures, and application services. The boundary between
FIGURE 10. Multi-core-network functions chaining for network services
and common network functions for roaming support.
software applications and physical infrastructure has been
removed by the advancements of virtualization technologies.
services. Far edge MEC may directly be connected with the The virtual 5G CN and application services are implemented
5G core. From the MEC perspective the User-Plane UPF is a as dynamic compositions of micro services realizing applica-
configurable distributed data plane. The control of that data tion and network functions. However, the complexity brought
plane, i.e. the traffic rules configuration, now follows the about by the resulting increased interactions between the
NEF-PCF-SMF route. application services and network functions, new business
models, new stakeholders, and their relationships requires
2) NETWORK SLICING more efficient and intelligent management and orchestration
A 5G network slice instance is defined by 3GPP in [37], framework. The application of AI in MEC platforms makes
which includes the CN Control Plane, User Plane Network traffic optimization and network resource management more
Functions and the next-generation Radio Access Network efficient. Dynamic network slicing, for instance, includes
described in [38]. real-time selection of optimal data rate or choosing the best
To achieve the network slices for autonomous driving use- 5G slice configuration for CCAM services. Using AI in
cases, the core network is virtualized and multiple instances dynamic network slicing enables differentiated qualities of
are created. The virtualized platform are capable of com- service (QoSs) for autonomous driving infrastructure compo-
puting and network hardware virtualization technologies and nents. More details of the end-to-end virtual network function
managed by a cloud management components. This allows MANO framework and implementation is provided in [39].
the flexible deployment of network slices, which consist
of various virtual network functions and services for the
infrastructure components’ communication demands. The
deployed system only supports a subset of autonomous driv-
ing use case related procedures, information and configura-
tions specified for network slice instance selection support
and network slicing for roaming support, which are described
in [37].
Multiple network slices may be deployed, which deliver
exactly the same features but for different groups of CCAM
infrastructure components. The slicing of 5G CN supports
all standard slicing and service types defined in [37]: eMBB,
URLLC, MIoT. The network may serve a single UE (e.g., AV)
with one or more network slice instances simultaneously
via a 5G-AN. The AMF instance serving the UE logically
FIGURE 11. Network slice management and orchestration architecture
belongs to each of the network slice instances serving the with slice management function as a service.
UE i.e., this AMF instance is common to the network slice
instances serving a single UE. The selection of the set of VI. MAJOR CHALLENGES
network slice instances for a UE is triggered by the first Availability of information from different stakeholders
contacted AMF in a registration procedure normally by inter- (e.g, street furniture, city authorities, service providers,
acting with the NSSF, and can lead to a change of AMF. neighbouring vehicles, etc.) to the autonomous vehicle in any
A PDU session belongs to one and only one specific Network scenario plays very crucial role. This positions the commu-
Slice instance per PLMN. Different Network Slice instances nication bit-pipes as a major solution component to achieve

VOLUME 9, 2021 33007

 M. A. Khan: Intelligent Environment Enabling AD

the objectives of Level 5 autonomous driving. Hence, in this resources (specifically in terms of throughput) should be
section, we list major communication related challenges. ensured.
• Ch 1: Delay in Switching between 5G Standalone • Ch 8: Different Network Functions - Device isolation
Networks - Vehicles relying on external information provisions different network functions so that it can pro-
and C-V2X communication for creation of their environ- vide access to external traffice (i.e., from other vendors,
ment understanding will need near real-time availability sources, etc.)
of information. However, a vehicle moving with the • Ch 9: Cloud Imparted Latency - It is expected that
speed of 50kmph or more will go through coverage of MEC deployment will be a new normal when it comes
many small cells and likely to carryout frequent han- to infrastructure deployment of mobile networks. It is
dovers between standalone networks of 5G, which may expected to reduced the latency. However, when the
incur latency. vehicle moves to a different cell the latency will increase
• Ch 2: Delay in Switching between 4G eNodeB and 5G e.g., due to data exchange with neighbouring MECs via
gNodeB - On the similar lines as in Ch2, if the coverage cloud node, etc.
footprint constitutes of heterogeneous mobile networks. • Ch 10: APIs for 3rd Party Services - Autonomous
Handing over the between these mobile network tech- driving is looking into adapting the eco-system by
nologies incur higher handover delays, specifically if introducing new stakeholders and creating the rela-
one of the technologies if pre-5G. Reducing the han- tionships amongst them. For instance, the road infras-
dover delays in such settings is a major challenge. tructure provider will be a key player in sharing the
• Ch 3: Delay in Switching between 5G Standalone on-road created perception with vehicles, city authorities
and Non-standalone networks - 5G network capacity will be actively involved in policy implementation, net-
planning is expected to following the deployment of work providers will ensure the exchange of information
NSA followed by SA. Hence, the coverage footprint of amongst the relevant stakeholders, vehicles, and infras-
the network will be a mix of these network technologies tructure. Hence, right interfaces, protocols, and APIs
over the stretch of a trajectory. On the similar lines as should be developed, which are then exposed to the
above, in complex scenarios of L-5 autonomous driving, stakeholders to implement the defined operations of the
the handover latency between these technologies should inter-stakeholders relationships.
be as low as possible. • Ch 11: Support for Multi-interface Communication
• Ch 4: Under-covered and Low Capacity Coverage - Owing to the fact that operator has deployment of
Footprint - It goes without saying that not all the road heterogeneous technologies covering different areas and
segments and city areas have enough network capacity. road segments, the need for optimal use of the available
Even if the installed capacity is enough for a region, network resources is imperative. Hence, operators may
it may choke during busy hours of the day. To realize opt for the simultaneous use of the multiple network
all the use-cases of level 5 autonomous driving, the net- interfaces for enabling communication amongst the enti-
work resources with required service quality should be ties of autonomous driving. This however, is challenging
ensured under all circumstances. to achieve specially if most of the operations are to be
• Ch 5: Improper Handover - In a complex urban envi- executed in real-time.
ronment, the coverage footprint of mobile network is
usually irregular and creates improper overlapping/ han-
dover regions. Hence, to facilitate the required commu- VII. USE-CASE SCENARIOS FOR LEVEL-5 AUTONOMOUS
nication of autonomously driven vehicles, the execution DRIVING
of handovers in the irregular coverage regions should be Having discussed the intelligent environment concepts at the
smooth. edge and cloud levels and their realization by different infras-
• Ch 6: Impact on Efficiency of Protocol(s) - As we tructure components to support AD, this section discusses
know that the change of point of attachment results a few use cases of complex scenarios that are representa-
in: change of IP address, flow tables in the network tive of level 5 autonomous driving. The use-case scenarios
devices, and other services over the backhaul, core, are inspired by the ones detailed in 3GPP TS 22.186 [40]
and radio segments of the network. Considering that and the documents mentioned therein. Readers are further
a large number of vehicles will frequently change the encouraged to refer to communication relevant requirements
point of attachments (switching eNodeBs, gNodeBs, for different use-case scenes in this 3GPP document. It should
etc.), it should ensured that the required operations for be noted that use-case scenarios are carefully selected to
the aforementioned switching are executed within the highlight the communication relevant challenges owing to
acceptable time (i.e., seamless handover). the fact that efficient communication is expected to be the
• Ch 7: Unprecedented Network Load - The recent crucial component for realizing level 5 autonomous driving.
transit from Internet-of-People (IoP) to Internet-of- Although a few selected use-case scenarios are detailed in this
Things (IoT) consequences in dynamically varying paper, we discuss the realization of the crucial scenes of the
network resource demands. Availability of required scenarios due to limited paper length.

33008 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

A. AUTONOMOUS DRIVING IN COMPLEX SETTINGS

The standard document [40] refers to such driving as enabling
the semi-automated or fully automated driving. Consider
Figure 1, where a number of dynamics of the complex urban
environments are visualized. Some of the crucial scenes of
this scenario are: i) Lane merge - analysing the traffic flow of
the target lane e.g., system is capable of detecting existing
vehicles with their lane position, acceleration, speed, size,
etc. Such information should be made available for any road
segment of the urban areas. Having received this information,
the automated vehicle determines the best merge manoeuvre.
ii) Overtaking - analyzing the traffic and intention of vehicles
in front in the same and neighbouring lanes. iii) 360 percep-
tion - detects the events and objects on the road segments
around the autonomous vehicles, which are then used to cre-
ate the 360 degrees understanding of the environment. This FIGURE 12. In the smart parking use case, intelligent infrastructure
assists connected vehicles to find free parking spaces and to navigate in
scene covers all the major critical maneuvers of the vehicle parking maneuvers.
of driving in urban environment.
For lane merging scene, automated vehicle may share their
about its surroundings i.e., creating a perception of the full
sensory data directly or over the infrastructure with each other
surrounding of the autonomous vehicle. For creating such a
to determine the best merge manoeuvre. For this the Vehi-
perception, the information from on-board sensors, on-road
cles will rely on capabilities of C-V2X and other enabling
sensors, that received from other vehicles and stakeholders is
technologies like: MEC node, roadside infrastructure, road-
fused. The information may represent: exact position, speed,
side radars and object detection sensors, etc. Presence of
dimensions, trajectories, intention of maneuvers, etc. of other
the other vehicle or object (information about its size, exact
vehicles/objects. Assuming that environmental variables are
position, direction, speed, etc.) in the merging area should
very dynamically changing, the information about these vari-
be notified in real-time to the autonomous vehicle intending
ables have to be made available in real-time for processing
to lane merge. Such information is exchanged by exploiting
and decision making of autonomous vehicles. The challenges
the communication bit-pipes (either direct, PC-5 or indirect
(Ch 1 - Ch 10) are more relevant to this scene.
5G/LTE-V/DSRC). Most of the aforementioned challenges
(i.e., Ch 1 - Ch 10) may be potential challenges for this scene
B. SMART PARKING
of the use-case scenario.
In the overtaking scene, there may be complexities as: This use case showcases how roadside infrastructure and tra-
hampering of vision because of large trucks, faster moving ditional, connected, and autonomous cars interact with each
vehicle coming out a side street, sharp turn in front of the other and with parking areas that are monitored by sensors.
preceding vehicle, vulnerable road user/pedestrian passage On the test-road, different parking areas can be observed.
in front of preceding vehicle, etc. Furthermore, the overtake Parking lots next to the road to on-street parking (both parallel
may take place on 2 lanes, 3 lanes, or more, where the situ- on the side of the road and slanted parking in the middle of
ations of multiple simultaneous overtakes may occur. Hence, the road) allow for the evaluation of infrastructure and driving
relying on the communication and on-board sensors may functions in different settings.
not suffice to meet the above or similar challenges. Relying Three scenarios are evaluated in the test-road:
on the on-road deployed sensors and sensory information • Parking Management: Parking sensors monitor the state
from other vehicles is the way to go. This obviously requires of parking spaces in the testbed and communicate the
reliable connectivity, adequate bandwidth (e.g., for stream- state to roadside and cloud infrastructure. The service
ing the video sensory data of preceding vehicle temporar- allows operators to guide connected vehicles to free
ily), timely handover (e.g., for ensuring seamless availability parking and long term data collection enables predicting
external information and 3rd party perception application, if parking will be available when a vehicle reaches its
etc.), extended vision of autonomous vehicles so that vehicles destination in the area
may create perception of more than a few kilometers rather • Sensor Fusion: Sensor data from vehicles and roadside
than a few hundred meters. Similar to the lane changing infrastructure (e.g. traffic cameras) is combined to detect
maneuver, this scene also highlights most of the challenges potentially hazardous situations, for example when a
(i.e., Ch 1 - Ch 10). vehicle pulls out of a roadside parking spot into oncom-
The 360 perception scene is an important scene and crucial ing traffic
in enabling the use-cases of level 5 autonomous driving. The • Maneuver Sharing: Connected and automated vehicles
idea is that autonomous vehicle is provided with information communicate their planned parking maneuver to other

VOLUME 9, 2021 33009

 M. A. Khan: Intelligent Environment Enabling AD

vehicles and roadside infrastructure, for example when In a later section of this paper, we discuss the details of our
braking to pull into a parking spot. experiments for demand estimation, traffic patterns investi-
gation and resource allocation.
C. GREEN DRIVING
A vehicles’ environmental impact is caused by the use of its B. EVOLVING THE LOCAL DYNAMIC MAP
engine, tires and brakes in the form of emissions (e.g. Carbon The IETF standardized four layered architecture of Local
monoxide, NOx) and fine dust. These emissions and particles Dynamic Map (LDM) may be evolved with additional layers
are caused by any moving vehicle and cannot be entirely of information, which are populated with perception created
eliminated. However, intelligent and anticipatory driving as from on-road deployed sensors and learning mechanism in
well as smart traffic management can reduce the amount of the edge and cloud. These layers may also be populated AI
emissions and fine dust significantly. While the emissions of a enabled approaches for predicting events, objects, variables
combustion engine are directly linked to the fuel consumption of the environment.
and the efficiency of the exhaust filtration system, fine dust
created by abrasion of brakes and tires are linked to the C. INTELLIGENT ROAD INFRASTRUCTURE
driving behavior. For example, a shorter route reduces tire Traditional roadside units should be evolved to intelligent
abrasions while a green wave reduces brake abrasions. edge that do not only support V2X communication but
The Green Driving use case explores the efficacy of the also serve as the perception creator, network manager, and
testbed, its systems and its autonomous vehicles in reducing service orchestrator. In this connection, contributions may
vehicles’ emission of exhaust gases and fine dust. Measuring include: IoT middleware integrating all the on-road deployed
the levels of fine dust and emissions harmful for humans’ sensors by developing the right drivers and protocols for
health, traffic control and intelligent transport systems can the type of sensors; communication solutions for down-
steer traffic in a way to mitigate the vehicle traffic’s impact stream and upstream for implemented network slicing to meet
on these levels in certain areas. the autonomous driving service requirements for the use-
By integrating with ITS services (i.e., traffic analysis), cases. Traffic engineering approaches by developing the SDN
environment reading can be linked with the traffic volumes native applications, smart management and dynamic knitting
at different times of the day. The impact of the traffic can of VNFs, exploiting the service based architectural features
also be accessed relatively with the sensing of visitors in of 5G-C for efficient service crafting and network slicing, etc.
the area by activity analysis sensors (visualizing motion and are some of the potential communication solution areas.
dwelling time, objective measurement of hot spots, statistical
evaluation). Based on the assessment, traffic regulation can D. ACTIVE LEARNING FOR PLANNING, BEHAVIOR, AND
be dynamically adapted to the perceived load of emission. CONTROL LAYERS
Traffic control and management services deployed at the edge Since the objects and environment is to be detected in real-
can regulate traffic lights and average speed of AVs based on time and best maneuver should be executed for realizing
both statistical and real-time data to minimize environmen- level-5 autonomous driving scenarios, it is imperative to
tal impacts. Such dynamic regulation immediately results in achieve real-time learning framework federating different
‘‘green-line’’ driving experience and avoids forming queues learning instances (e.g., on vehicle, intelligent edge, and
and stop-and-go traffic, which greatly increase emission and clouds) to meet the requirements of complex environmen-
fuel consumption. tal dynamics. Multi-layer coalitional learning with speedup
framework is one potential research direction in this regard.
VIII. POTENTIAL SOLUTION DIRECTIONS It should be noted that this approach advances and exploits
In this section, we sketch the solution directions to address the deep learning approaches. The interplay of game-theory
the aforementioned challenges and realize the use-cases. and deep learning for autonomous driving may be investi-
gated. Approaches for speeding up for both deep learning
A. DYNAMIC DEMAND ESTIMATION and gradient-based algorithm in the L-5 autonomous driving
Challenges Ch 1 - Ch 9 may be addressed if the system is pro- use-cases may be investigated, as we expect deep learning
vided with an estimated network resource demand in advance algorithms will have larger depth. Our earlier approaches
so that relevant operations required for resource reservation like Bregman based algorithms for speedup, meta-learning
and allocation are carried out beforehand. For instance, if the for self-x network management, strategic deep learning algo-
traffic intensity at different road segments is known then the rithms for robust games may be evolved. A coalition frame-
operator may do the proper capacity planning proactively. work to federate the learning mechanisms at the vehicle,
Ch 1 - Ch 3 may be addressed by preempting the mobility edge, and cloud levels. Furthermore, coalitional framework
pattern the vehicles on the road segments and proactively may also be implanted with features for inter-vehicles coor-
managing the handover/mobility operations. Ch 4, Ch 5, and dination (specifically in platooning use-case). For instance,
Ch 7 may be addressed by implementing proper resource platoon formation may be based on coalition game-theoretic
reservation and allocation, which can be made very efficient approach. The utilities of vehicles will be modelled and
if the algorithm is provided with estimated resource demands. intra/inter-platoons coordination will be complemented by

33010 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

the information captured from on-road deployed sensors, on-

board sensors, and federated clouds [28], [29].

IX. IMPLEMENTATION OF BASIC SOLUTION

APPROACHES
Although a number of use-cases are crafted to be tested on the
testroad, in this section, we discuss initial set of experiments.
These experiments focus on dynamic demand estimation and
study of traffic pattern on the testroad, which are crucial
when it comes to realizing use-cases of autonomous driving
FIGURE 13. Observed road segments (indicated by letters within white
(refer to Section VII). circles, right) throughout the intersection including aggregated vehicle
counts (left, numbers on arrows) of 24 hours on 2018-01-24 for the
driving direction (indicated by the arrows’ directions).
A. TRAFFIC PATTERN GENERATION
In this section, we demonstrate the analysis of traffic data
using a neural network regressor. The learned patterns enable
affects the accuracy. But the experiments show that such
the prediction of future traffic demands, which are valu-
instances did not occur too often.
able inputs for other network and vertical service orchestra-
The sensors and equipment are configured to capture the
tion functions deployed in our data-center. For this study,
count of vehicles in the roundabout. In this connection the
we rely on the traffic analysis sensors deployed around
following are is carried out: i) the traffic flow analyzers are
the Ernst-Reuter-Platz, a major traffic junction of Berlin,
configured for the inflows and outflows at all the incoming
and image data has been automatically analyzed for a few
and outgoing segments of the roundabout; ii) the segments
years. The data gathered (also available at https://flow.dai-
nearer the incoming and outgoing roads are also covered by
labor.de/datasets/ and https://daten.berlin.de, the outcome of
the activity analysis sensors; iii) people count are configured
Smart City Berlin Project) includes vehicle counts for most
on the pedestrian areas and zebra crossings; iv) communica-
road segments, counts of pedestrians entering or leaving two
tion infrastructure is configured to enable near real-time com-
selected buildings as well as other metrics (e.g., number of
munication with the data-center. This configuration ensures
connected mobile devices) that correlate with activity on and
that the vehicles entered in the roundabout will remain in the
around the Ernst-Reuter-Platz.
roundabout and may be tracked. Counts are aggregated in one
Although various sensors are detailed in previous section,
minute intervals and vehicle are classified by their types.
for traffic pattern analysis, we focus here on a few types
of (visual) traffic, pedestrian and parking space observation
sensor, which are deployed around the Ernst-Reuter-Platz 2) RESULTS OF TRAFFIC ANALYSIS
and analyses of the image data captured since January 2018. The experiments were initiated by carrying out the sanity
The data includes vehicle counts, counts of pedestrians enter- checks i.e., by visually conforming the traffic intensity pat-
ing or leaving two selected buildings, number of connected ters at different times of the day and evaluating against the
mobile devices, parameters that correlate with activity on the expected patterns. We also carried out consistency checks and
road segments. an assessment of the system’s accuracy by:
• Properties Analyses: to analyze the properties and
1) AN OVERVIEW OF DEPLOYMENT SENSORS nature of the recorded data, aggregation type, and time
For the traffic generation around the considered round- resolution.
about, 13 camera based sensors are deployed at the locations • Aggregation: to aggregate the statistics and compare
depicted in Fig. 13. those with reference values (i.e., please refer to [41]).
• Time Relationship Analysis: to analyze time relation-
• Location 1: multiple traffic analysis sensors, parking ships and constraints in the observed data based on the
sensor, and people counter. spatial relation of the point of measurement (i.e., causal
• Location 2: multiple traffic analysis sensors. and spatio-temporal correlations).
• Location 3: traffic analysis sensor, people counter, and
Vehicle counts are extracted from the image streams and
multiple access points.
the numbers are recorded. The vehicles and types are classi-
• Location 4: multiple traffic analysis sensors and multiple
fied with visual object detection. At specific times in the data
access points.
the typical distribution of vehicle types changes drastically.
• Location 5: multiple traffic analysis sensors and APs.
Consequently, we decide to only use the aggregated (total)
Although the deployment was so carried out that the sensor counts of vehicles in this work and include the vehicle type
capture all the dynamics of the round about, there were still classification in future works.
some discrepancies at different times e.g., some dynamics For the traffic analysis, the data is accumulated in minute
were hidden behind the trucks or busses. Obviously, this intervals, our expectations include:

VOLUME 9, 2021 33011

 M. A. Khan: Intelligent Environment Enabling AD

FIGURE 14. Example for raw traffic volume (number of vehicles per minute) data as provided by the system over a period of 1.5 days
around Wednesday 2018-01-10 (top) and 5 minute averages of four selected locations highlighting a traffic interruption on 2018-01-25
(bottom).

• Matching: In this, we match the data with data from well with other sources. The peaks (commonly referred
other sources e.g., distribution over day and peaks, daily to as ‘rush hour’, usually present in the morning and
averages, and other general trends for working days and evening) are expected (see e.g. [41]) and easily spotted
holidays etc. (cf. figures 14 & 15). They are attributed to people going
• Temporal Closure: It is validated that the temporal clo- to their daily business (8h-11h) and back home from work
sures may easily be detectable. (16h-19h). Weekends, holidays and bridge-days differ in this
• Correlation: There should be simple and obvious cor- regard, so that the first peak is reduced or absent and that
relations observable between sensor stations, refer there is overall slightly less total activity. The data collected
to Figure 15. at Ernst-Reuter-Platz differs only slightly in its characteristics
• Detectability: Depending on time resolution, the phase from those published in the 2014 report. One example being
of traffic lights may be detectable, at least as statistical that the 2nd daily peak is not reduced as much.
artifact. The data is valuable for creating valid models for the
We explore the dataset by visualizing it using interactive traffic occurring on the Ernst-Reuter-Platz intersection to
plots with freely selectable time ranges, freely selectable sub- optimize the routing of vehicles, by adapting traffic signal
sets of sensor locations, and smoothing options. For a spatial phases or individual navigation decisions or can be used as
understanding the cumulative counts for a time-interval of all input to predict the state at intersections for the communica-
the stations can be visualized on a map (cf. Fig 13). tion network.
It was observed that there are some common pat- For traffic volume predictions we trained a simple Neu-
terns in daily traffic with simple explanations that match ral Network regressor on the first 20 days of the test set

33012 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

FIGURE 15. Exemplary causal relations between sensor readings (here: inclusion): Because of the spatial arrangement all vehicles passing
sensor location ‘O’ (cmp. fig. 13) (the lower, green plot) should also pass sensor location P (the higher, blue plot) with some delay; the
plots show data of roughly three days around Sat./Sun., 13/14th of Jan. 2018.

FIGURE 16. Response of neural network regressor trained on 2 weeks of data. Predicting the traffic volume given the minute of day. The
two curves are predictions for workday (Mon.-Tue. in cyan) and weekend (Sat./Sun. in red) with correspondingly colored training instances
in the background.

from a total of an equivalent of 38 days of available data. not yet included because of the limited amount of available
We experimented with different input formatting, the number data (less than a year) at this point.
of layers and neurons trying to find a comparatively sim-
ple and small network, that produces acceptable predictions,
B. AI ENABLED NETWORK DEMAND ESTIMATION AND
as shown in Figure 16. Of course, this depends on the accu-
NETWORK RESOURCE ALLOCATION
racy required for the application intended. We finally settled
on a network with 100 neurons in a single hidden layer, This experiment focuses on estimating the network resource
L-BFGS [42] as solver and tanh as activation function, using demands and allocation for autonomous vehicles on the test-
the following features: road. As mentioned earlier that the road segment shown
in Figure 17 is driven by the fact that it is most complex
• Minute of the day (0..1439), segment with 5 ins and 5 outs, where each in and out are
• Day of week (0..6 Mon.-Sun.), 3 lanes. It also has cuts and walkways for pedestrians. Hence,
• Is a holiday (true, false), meeting the requirements for communication services is of
• Vehicle count of that minute (number) - the class utmost importance to enable the exchange of right informa-
label/output. tion for execution of the right critical maneuvers.
Figure 17 depicts our understanding of the traffic flow on
Additional promising features, which we think to have a the considered road segment. There are 5 paths defined (as
high potential to improve prediction accuracy, are: Seasonal can be seen in the figure), which are represented by different
vacations (school) or weather information. However these are colors i.e., blue, grey, black, green, and red. These paths are

VOLUME 9, 2021 33013

 M. A. Khan: Intelligent Environment Enabling AD

pictorially depicted in Figure 18. As can be seen that the data

server contains regularized dataset to perform supervised and
unsupervised learning techniques. We process the real dataset
(captured through on-road deployed sensors) with different
machine learning algorithms to trace mobility and activity of
the road users.

FIGURE 17. Figure highlighting the trajectories on the Ernst-Reuter-Platz.

Different colors correspond to traffic flow with specific entry and exit FIGURE 18. Figure depicting the conceptual view of the implemented
points. approach for demand estimation by explicitly highlighting different
stages.

covered by different network cells. The paths are decomposed We implemented a data pre-processing stage, which fur-
into multiple trajectories to highlight the entry and departure ther implements data regularization and data personalization
areas with respect to network cells, which are summarized stages. In the data presonalization stage, necessary adjust-
in Table 2. ments to the data is made for different scenes of the scenarios.
To separate features of the datasets, we carried out the data
TABLE 2. Trajectories (Represented by T).
representation activity. In this connection, operations such as
k-field validation were performed. When it comes to improv-
ing the results, standardization. These stages are followed by
choosing and applying algorithm to predicts traffic intensity
and resource demands with highest accuracy.

2) GROUND TRUTH AND DATA QUALITY VALIDATION

Having acquired the right dataset, we needed to verify the
ground truth of the dataset. Figure 20 shows the first two
The data is capture through IoT middleware from on-road Monday’s dataset and pattern in this graph correlate with
deployed sensors is forwarded to the central database after each other means data is in the right shape. We have carried
being anonymized at the sensors. So far, 8 months of data out a complete exploratory data analysis to understand traffic
have been collected on the considered round-about through density patterns in Ernst-Reuter-Platz roundabout. One of
the on-road deployed sensors and communication infrastruc- the major objectives of data analyses was to assess the data
ture. The sensors capture the data of different road users for its quality to be used for modeling the demands exam-
including: motorcycles, cars, trucks, pedestrian, etc. Obvi- ination. During the analyses, it was observed that sensory
ously, these road users have varying communication bit-pipe data of some of the sensors is missing for a few hours.
requirements for C-V2X communication and infotainment This consequences in dataset, where not everyday has the
based on their position, on-vehicle deployed sensors, and same number of rows. However, we assume that a day fully
inter-vehicle-infrastructure-pedestrian communication, etc. captured, if 54,000 sensory readings from sensor is available.
Analyses of such datasets enable us to predict the traffic We believe that the assumption is realistic, as it captures
intensity on, and occupancy of the roads. Hence, smartly cho- the environmental dynamics on small enough time quanta.
sen assumptions for communication bit-pipe requirements for In our experimental settings, as soon as road user is detected,
different applications (e.g., ITS services, infotainment, etc.), we define two boxes, source and destination. If the object
we are able to predict the communication requirements and passes through the source and then through the destination
consequently assist in capacity planning on different road box, it counts as +1. So the number in the data indicates
segments. the total number of detected vehicles that passed through the
defined boxes. The former definition implies that it will count
1) SOLUTIONS IMPLEMENTATION STAGES only moving vehicles.
To achieve the basic objective of identifying and track-
ing the road users so that fitting communication C-V2X 3) RESOURCE ALLOCATION AND DEMANDS PREDICTION
bit-pipes are made available at all locations on the road Based on the discussion in previous subsections, it is evident
segment, we worked out a basic design. The design is that we are now capable of estimating the communication,

33014 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

TABLE 3. Population setting results.

FIGURE 19. Figure depicting the processing stages of data analysis and
machine learning implementation.

TABLE 4. Proportional resource allocation.

which include: over-provisioning, proportional priority, uni-

form, and manual resource allocations. However, for ready
reference in what follows next, we provide the details of a
FIGURE 20. Curves showing the traffic count on 2 Mondays. few selected models.

1) PROPORTIONAL RESOURCE ALLOCATION

bit-pipes in dynamic setting. This allows us to implement In this method of resource allocation, we focus on achieving
various algorithms for resource allocation against the pre- the fairness. This is to say if the available bandwidth is
dicted demands. The performance of the implemented algo- more than or equal to demand then the cell is provided with
rithms was measured e.g., through Root Mean Square Error required bandwidth. Otherwise, the available bandwidth is
(RMSE). We capable of predicting the next location and allocated proportional to the bandwidth demands by each cell.
signal strength of a moving vehicle in the road segment. Table summarizes the resource allocation for the available
Hence, accordingly we can allocate the right amount of com- cells. Experimental results are shown in Figure 21, where the
munication resources following the estimated demands. since resource allocation is carried out as expected i.e., the band-
the considered settings are specific to a complex road seg- width is allocated proportional to the bandwidth demands.
ment, time series dataset results proved to be very attractive
together with classification algorithms and k-fold validation.
The results are summarized in Table 3
An important fact to highlight here is that inspired by the
user-centric paradigm, we have taken great care of satisfying
device layer demands and at the same time maximizing the
profit function of operators i.e., by modeling the profit func-
tion of the operator as a function of users’/ vehicles’ satis-
faction. The details are avoided here due to space limitations,
however, the readers are encouraged to refer to a variant of
our approach in [28], [43], and [44]. FIGURE 21. Proportional resource allocation.

C. RESOURCE ALLOCATION MODELS

We implemented different variant of the resource allocation 2) PRIORITY BASED RESOURCE ALLOCATION
approaches based on the estimated network demands. The In this settings, the implemented algorithm allocates the
details of resource allocation models may be found in [44], resources by taking into account the priority assigned to cells

VOLUME 9, 2021 33015

 M. A. Khan: Intelligent Environment Enabling AD

e.g., the cells covering path in red color of Figure 17 is given autonomous driving project of Berlin. We also demonstrated
higher priority. This is to say in situations of higher demands the use of machine leaning approach for traffic demand
than the capacity, the system will allow the reservation of prediction.
resources following the defined priority values, which could
be in terms of percentage. ACKNOWLEDGMENT
This turns out to be an effective way to provide the The author would like thank Xuan Thuy Dang and Martin
bandwidth where traffic congestion becomes a normal phe- Berger, who were in his team and assisted in this work. Major
nomenon. This algorithm gives priority to the max traffic work in this article was carried out when the authors was with
either two-node or one node. TUB, Germany.

3) MANUAL RESOURCE ALLOCATION REFERENCES

In this method, operators can manually select the bandwidth [1] O. M. Group. (2020). The Future of Automotive: Still a Long
for a specific cell for certain periods of time. There are two Ride. [Online]. Available: www.omnicommediagroup.com/news/global-
news/the-future-of-automotive-st
options in this algorithm: one operator can assign the band- [2] J. Delcker. (Nov. 2020). The Man Who Invented the Self-Driving Car (in
width by percentage, the second operator can give the quan- 1986). [Online]. Available: www.politico.eu/article/delf-driving-car-born-
tity of bandwidth each cell manually. Figure 22 expresses 1986-ernst-dickmanns-merc%edes/
[3] C. Reinholtz, T. Alberi, D. Anderson, A. Bacha, C. Bauman, S. Cacciola,
the manual technique that we did, bandwidth value is not P. Currier, A. Dalton, J. Farmer, R. Faruque, and M. Fleming, ‘‘Darpa
changing because this value is to remain the same for certain urban challenge technical paper,’’ Virginia Tech, Blacksburg, VA, USA,
time window. Tech. Rep., 2007.
[4] C. Davies. (Nov. 2020). Audi Abandons Self-Driving Plans for Cur-
rent Flagship. [Online]. Available: www.slashgear.com/audi-a8-traffic-
jam-pilot-level-3-cancelled-tech-self%-driving-legislations-28618493/
[5] S. Taxonomy, ‘‘Definitions for terms related to on-road motor vehicle
automated driving systems,’’ SAE Int., Warrendale, PA, USA, Tech. Rep.
J3016_201806, 2014.
[6] H. Cao, S. Gangakhedkar, A. R. Ali, M. Gharba, and J. Eichinger, ‘‘A 5G
V2X testbed for cooperative automated driving,’’ Proc. IEEE Veh. Netw.
Conf., VNC, Dec. 2017, pp. 1–4.
[7] Diginetps. (2019). The Test-Road in Berlin, Germany. [Online]. Available:
http://diginet-ps.de/
[8] Mcity Autonomous Driving Testing Facility. Accessed: Dec. 1, 2020.
[Online]. Available: www.mcity.umich.edu/contact/
[9] Kcity Autonomous Driving Testing Facility. Accessed: Dec. 1, 2020.
FIGURE 22. Priority based resource allocation.
[Online]. Available: www.qz.com/1121372/south-korea-opens-k-city-the-
latest-fake-town-built-%for-self-driving-cars
[10] Gumna University Autonomous Driving Testing Facility. Accessed:
Dec. 1, 2020. [Online]. Available: www.mainichi.jp/english/articles/
20180611/p2a/00m/0na/018000c
[11] C.-I. Corridor. (Nov. 2020). Cooperative its Coorditor. Funded by German
Ministry of Transportation and Digital Intrastructure. [Online]. Available:
http://c-its-korridor.de/?menuId=26&sp=en
[12] D.-I. T. Trost. (Jan. 2021). Cooperative its Corridor Organisatorische Und
Technische Erprobung Des Baustellenwarners Unter Realen Bedingun-
gen. [Online]. Available: https://c-its-korridor.de/data/download/SVT_11-
2018_Seiten_780-786.p%df
[13] C. Corridor. (Nov. 2020). Cdot and Panasonic Take First Steps to Turn
i-70 Into Connected Roadway. [Online]. Available: www.codot.gov/
news/2018/july/cdot-and-panasonic-take-first-steps-to-tur%n-i-70-into-
connected-roadway
[14] C. Corridor. (Nov. 2020). Smart Columbus Connected Vehicle Envi-
FIGURE 23. Manual resource allocation. ronment Research. [Online]. Available: www.smart.columbus.gov/get-
involved/connected-vehicle-environment
[15] N. Newsdesk. (Nov. 2020). Forth Road Bridge Corridor in Edinburgh.
[Online]. Available: www.thenational.scot/news/18034842.driverless-
X. CONCLUSION buses-carry-10-000-people%-week-edinburgh/
In this paper, we briefly reviewed current activities related [16] F. Industry. (Jan. 2021). Cavforth: Autonomous Vehicle Pilot. [Online].
to autonomous driving, especially the deployment of test Available: https://www.fusionproc.com/automated-vehicle-systems/
projects/cavforth/
beds in controlled environments and on public roads world- [17] D. L. Campus. (Nov. 2020). Dtu lyngby Campus, Before Widening to
wide. We have discussed the concepts, e.g. road digitaliza- a Trial in Albertslund Municipality. [Online]. Available: www.dtu.dk/
tion or flexible communication infrastructure, for reaching english/news/Nyhed?id=ff6c63e7-829f-4fe4-902c-1f99818ba63b
[18] DRIVE.AI. (Nov. 2020). On-Demand Self-Driving Car Service to Frisco,
the ambitious goals of fully autonomous driving vehicles. Texas. [Online]. Available: www.wired.com/story/self-driving-cars-texas-
We have highlighted the challenges hindering the realization frisco-driveai/
of autonomous driving. The potential solution approaches [19] iMove. (Nov. 2020). Autonomous Parking Service for Vw, Audi, and
Porsche Vehicles at Hamburg Airport. [Online]. Available: www.
are discussed directing the readers towards potential research imoveaustralia.com/news-articles/personal-public-mobility/volkwagen%-
areas. We discussed the solutions designed for the large-scale test-autonomous-parking-hamburg-airport/

33016 VOLUME 9, 2021

M. A. Khan: Intelligent Environment Enabling AD

[20] iMove. (Jan. 2021). Smart Mobility Projects and Trials Across the World. [32] Intelligent Transport Systems (ITS); Security; Security Header and Certifi-
[Online]. Available: https://imoveaustralia.com/smart-mobility-projects- cate Formats, Standard ETSI TS 103 097 V1.3.1, v1.2.1. Tech. Specifica-
trials-list/ tion, 2015, [Online]. Available: https://portal.3gpp.org
[21] Bavarian a9 Autobahn Test-Road for Digital Field Highway. Accessed: [33] H. Song, ‘‘Protocol-oblivious forwarding: Unleash the power of SDN
Dec. 1, 2020. [Online]. Available: www.nuancespublicaffairsblogen. through a future-proof forwarding plane,’’ in Proc. 2nd ACM SIGCOMM
wordpress.com/2015/05/22/bavarian-a9-aut%obahn-test-road-for-digital- Workshop Hot Topics Softw. Defined Netw. - HotSDN, 2013, pp. 127–132.
field-highway/ [34] M. A. Khan, S. Peters, X. T. Dang, T. Dorsch, and P. Engelhard, ‘‘On
[22] M. Blackwell and A. Jamal, ‘‘A2/m2 connected corridor connected the approaches for efficient mobility management in SDNized wireless
autonomous vehicle testbed,’’ Contemp. EHF, vol. 5, no. 4, 2020. networks,’’ in Proc. 6th Int. Conf. Commun. Netw. (ComNet), Mar. 2017,
[Online]. Available: https://publications.ergonomics.org.uk/uploads/A2- pp. 1–8.
M2-Connected-Corridor-Connected-Autonomous-Vehicle-Testbed.pdf [35] M. A. Khan, P. Engelhard, and T. Dörsch, ‘‘SDNizing the wireless
[23] ETSI. (2016). Mobile Edge Computing (MEC); Framework and Ref- LAN—A practical approach,’’ in Proc. 9th EUROSIM Congr. Model.
erence Architecture. ETSI GS MEC. [Online]. Available: www.etsi. Simul., EUROSIM , 57th SIMS Conf. Simul. Model. SIMS, Dec. 2018,
org/deliver/etsi_gs/MEC pp. 1116–1121.
[24] T. Taleb, K. Samdanis, B. Mada, H. Flinck, S. Dutta, and D. Sabella, [36] X.-T. Dang, M. A. Khan, S. Peters, and T. Dorsch, ‘‘Realization of
‘‘On multi-access edge computing: A survey of the emerging 5G network handover management in SDNized 3GPP architecture with protocol
edge cloud architecture and orchestration,’’ IEEE Commun. Surveys Tuts., independent forwarding,’’ in Proc. Wireless Days (WD), Apr. 2018,
vol. 19, no. 3, pp. 1657–1681, 3rd Quart., 2017. pp. 60–67.
[25] K. Sasaki, N. Suzuki, S. Makido, and A. Nakao, ‘‘Vehicle control system [37] System Architecture for the 5G System; Stage 2, Standard ETSI TS 123
coordinated between cloud and mobile edge computing,’’ in Proc. 501 V15.3.0, Technical Specification (TS) 23.501, 06 2019. [Online].
55th Annu. Conf. Soc. Instrum. Control Eng. Jpn. (SICE), Sep. 2016, Available: https://portal.3gpp.org
pp. 1122–1127. [38] NR; Overall description; Stage-2, Standard ETSI TS 138 300 V15.8.0,
[26] J. Liu, J. Wan, D. Jia, B. Zeng, D. Li, C.-H. Hsu, and H. Che, ‘‘High- 3rd Generation Partnership Project (3GPP), Tech. Specification (TS),
efficiency urban traffic management in context-aware computing and 5G Jun. 2019. [Online]. Available: https://portal.3gpp.org
communication,’’ IEEE Commun. Mag., vol. 55, no. 1, pp. 34–40, Jan. [39] X.-T. Dang, M. A. Khan, and F. Sivrikaya, ‘‘An autonomous service-
2017, doi: 10.1109/MCOM.2017.1600371CM. oriented orchestration framework for software defined mobile networks,’’
[27] Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set in Proc. 22nd Conf. Innov. Clouds, Internet Netw. Workshops, Feb. 2019,
of Applications; Local Dynamic Map (LDM); Rationale for and Guidance pp. 277–284.
on Standardization, Standard ETSI TR 102 863 V1.1.1, ETSI, Tech. Rep., [40] 5G; Service Requirements for Enhanced V2X Scenarios, Standard ETSI
2011. TS 122 186 V15.3.0. 3GPP, Jul. 2018.
[28] M. A. Khan and H. Tembine, ‘‘Meta-learning for realizing self-x [41] Senatverwaltung Stadtentwiclung und Umwelt, ‘‘Straßenverkehrszählung
management of future networks,’’ IEEE Access, vol. 5, pp. 19072–19083, Berlin 2014,’’ Senatsverwaltung Für Stadtentwicklung Und Umwelt,
2017. Berlin, Germany, Tech. Rep. StranssenverkehrszÃC̆Âdlung-2014, 2014.
[29] H. Tembine, ‘‘Deep learning meets game theory: bregman-based algo- [42] D. C. Liu and J. Nocedal, ‘‘On the limited memory BFGS method for
rithms for interactive deep generative adversarial networks,’’ IEEE Trans. large scale optimization,’’ Math. Program., vol. 45, nos. 1–3, pp. 503–528,
Cybern., vol. 50, no. 3, pp. 1132–1145, Mar. 2020. Aug. 1989, doi: 10.1007/BF01589116.
[30] M. A. Khan, X. T. Dang, and S. Peters, ‘‘Preemptive flow management in [43] M. Khan and U. Toseef, ‘‘User utility function as quality of experience
future SDNized wireless networks,’’ in Proc. IEEE 12th Int. Conf. Wireless (QOE),’’ in Proc. 10th Int. Conf. Netw., Jan. 2011, pp. 99–104.
Mobile Comput., Netw. Commun. (WiMob), Oct. 2016, pp. 1–8. [44] U. R. Mughal, M. Ahmed Khan, A. Beg, and G. Q. Mughal, ‘‘AI enabled
[31] M. A. Khan, H. Tembine, and A. V. Vasilakos, ‘‘Game dynamics and cost resource allocation in future mobile networks,’’ in Proc. NOMS IEEE/IFIP
of learning in heterogeneous 4G networks,’’ IEEE J. Sel. Areas Commun., Netw. Oper. Manage. Symp., Apr. 2020, pp. 1–6.
vol. 30, no. 1, pp. 198–213, Jan. 2012.

VOLUME 9, 2021 33017