2021 7th International Conference on Advanced Computing & Communication Systems (ICACCS)

Latency based Re-Enforcement Learning

over
441725

9
Cognitive Software Defined 5G Networks
2021.

Baburao Kodavati1, Madhu Ramarakula2

51430.
'JNTUK, Department of ECE, URCET,Telaprolu,AP, India - 521109.
CCS
2Department of ECE, JNTUK, Kakinada, Andhra Pradesh, India -533003.
ICA
[email protected], [email protected]
11
up
.109/
0
Abstract - In present days, in software defined networks, Adventitiously, the subjective system innovation can make
:O cognitive network (CN) is the key control to enable Internet of for these lacks [11-15]. Subjective hubs are wise
remote
I
D
changes,
E
|
Things (IoT) services, whenever CNplays a important task in gadgets that can detect the earth, watch the system
E
E
the future network internet applications used in different types of utilize the information gained from the past association with
I2the
1
real time applications such as agriculture, monitoring system, and settle on canny choices to take advantage of
0
2
© healthcare and smart metering with different scenario’s. lucky breaks to transmit. The procedure of ceaselessly
detecting
00
Because of increasing popularity of variety software defined the earth data, trading control data, learning data, choosing
and
$31.
crowed applications; it follow on low transmission rate in executing a technique in the system can give the capacity
of
20/
8/ communication, major challenge behind this is to improve insight and flexibility to the remote sensor systems and
the
efficiency ofpacket transmission in software defined network future IoTs. Consequently, the psychological radio innovation
0521-
is
data transmission. So that in this paper, we propose A Novel a key correspondence approach for asset obliged remote
sensor
6654-
1- Re-Enforcement Learning Approach (NRELA) to improve the systems and future remote system [1-5]. At the point
when
transmission efficiency using cognitive radio software defined psychological clients, i.e., sensors in remote sensor systems,
978-
get
)the networks through multiple channel to increase network to the range, to viably utilize the system assets and fulfill
|
S
C throughput. To increase the system ability in between nodes throughput interest for sight and sound applications,
compelling
C
A
C with respect to re-enforcement learning approach to find instruments are required to organize the activities of
the
I
( optimal data communication in software defined networks. intellectual clients (tx power control, range get to, tx
planning,
sm
in
s Establish the connection between different nodes to accelerate et al.). With the quick increment in number of remote gadgets
e
t
ynhubs. the solutionfor efficient data transmission in software defined the IoTs, more information will be put away in the system
S
oa communication. An experimental result ofproposed approach In this way, the strategy to quickly advance information
with
i
t
cu
an
i gives better and efficient latencyand energy results in data the constrained extra room and transfer speed is
n
m
m transmission in software defined networks. extraordinary test for the present remote system of Internet
of
od Things.
nC
a
Index Terms: Software defined networks, learning approach,
cognitive networks, packet data transmission, multiple
gtApproach We propose A Novel Re-Enforcement Learning
n
i
uusing
m
p
channel communication and network throughput. (NRELA) to improve the transmission efficiency
oC
cognitive radio software defined networks through
multiple
d
c I. Introduction channel to increase network throughput. To increase the
n
system
e
adenforcement
v
ability in between nodes with respect to re-

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:10:04 UTC from IEEE Xplore.
 2021 7th International Conference on Advanced Computing & Communication Systems
(ICACCS)

Markov Decision Process (MDP) is introduced to address with the objective of limiting the time by which all parcels from
the issues present in the system by performing Dynamic the two clients are conveyed to the goal. The below average
Programming technique [4]. But this technique doesn’t involves online methodologies [14] [15] [16]. Creators in [14]
moderate the issue scale which is augmented. Hence, to allocate considered a multi-get to remote framework with EH
this issues, deep reinforcement learning (DRL), which joins transmitters, and the entrance issue was displayed as an in part
Deep Neural Networks (DNNs) and Reinforcement Learning recognizable Markov choice procedure (POMDP). In [15], the
(RL) techniques are introduced. Multi-Measurement arrangements for controlling the power ideally for EH hubs in a
information will plays an elective role in finding the route of its multi-get to framework was studied and while performing
capacity. procedure of EH will collect the elements from dam model.

II. Related Work In [16]at transmitter side few measurement learning

aspects with respect to the dynamic framework is studied. In
numerous
At present, many existing written works (see in [5]-[15]) reasonable applications, the total non-easygoing learning or
have considered the issue of system information transmission even factual information of the framework elements (counting
with obscure condition data in a cross-layer structure way. both the channel and vitality parts) probably won't be
Among these written works, [5] and [6] have analyzed the accessible, particularly when the EH procedures are non-
versatile adjustment calculation in the information transmission stationary or from sources with obscure circulations. For
arrange, while [7] concentrated on the dependable course instance, in a remote system with sunlight based EH hubs
revelation to lessen the time required for the information online. appropriated arbitrarily over a geological region, the qualities of
The adjustment of power and mindfulness schedulerQuality of the collected vitality at every hub rely upon the hub area, and
service (QoS)for medium access control (MAC) layers at uplink change after some time in a non-stationary style [17]. It is very
and downlink to arrange the activity of the lower layers for difficult to get the information which is coming from different
asset proficiency was proposed in [8]. The throughput along elements from various sources.
with decency is concentrated in [9]. Contingent upon the
inclination of the two highlights, calculations and techniques III. Background Approach
are proposed to dole out or plan clients to organize to boost
throughput, augment decency or finding the proper harmony CIoT that exists together with an authorized framework is
between the two. The better performance of system shown in figure 1. In this we studied about the point ot guide
administration is obtained by using topology configuration, transmission channels which are transfeered to the sin and area
steering conventions and cross layer plan of MAC in [30]. channels.
Markov decision process (MDP) [1-3] will overcome the issues The parcels from its K neighbor hubs are accumulated by
that
one are streaming from the remote systems. In any case, it is
hand-off, and at indistinguishable length Lthese packets
hard to take care of the MDP issue in light of the fact that the
are put away in K cushions. Thosepackets that originate from K
MDP has numerous factors. Thusly, the fortification learning neighbor hubs are accepted to have Poisson dissemination
technique can be acquainted with tackle it. The research for
with
indistinguishable entry rate!.
various applications and administrations in circulated remote
systems is carried out in [33-37]. The performance
parameters like Quality of experience (QoE) and Quality of
Service (QoS) are related with booking instruments and the
planning systems are considered in across-layer way.

This related research takes a shot at EH based

frameworks that can be classified into two different classes
dependent on the accessibility about the information of
vitality entries. Basically, the methodologies that are
disconnected with the stochastic frame work should consists
of non-casual learning [11] [12] [13]. Specifically, the ideal
uplink asset assignment was
examined in [12] for the situation where two EH clients initially Fig 1 Channel transmission methodology with respect to
gathered vitality from the remote flag and after that agreeably node communication.

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:10:04 UTC from IEEE Xplore.
 2021 7th International Conference on Advanced Computing & Communication Systems
(ICACCS)

M channels are autonomous and indistinguishably this TS is divided into two sections mainly, present
channel dispersed. The transmission calendar is chosen by the hand-off. state and present UE battery state. The
present channel A channel is used by the hand-off process to transmit the hub to state is represented as Ht = {H1t, •
,HNt} and the present sink. The transfer does not include the transmission of bundle if UE battery state is represented as Bt =
{B1t, • , BNt}. The the channel state is poor. At the point when a specific cradle is combination of these two will give the
St = {Ht, Bt} which full, in the event that the transfer does not transmit parcel for it, is a UE choice decision. The condition
P Kt 6 A, with |Kt| at that point, the bundle is lost if parcels keep on landing in the = K, N i=1 Iit = K will be satisfied
by using these following casing. Along these lines, and for the transmission strategies. The signal Rt is described as
shown in below process of bundles completely depends on the channel state and equation which is nothing but total
rate.
the support expresses that relate to the correspondence sets and
CO
transmission mode.

IV. Re-enforcement Learning with Multi Channel k-t

Access Procedure
^ I W( PH )
;
In this section, we discuss about re-enforcement learning =Z *
procedure with multi channel access control in software defined k=t ie K M V ® J

network communication. Basic problem formation described in

figure 2. Main re-enforcement learning model maximize the
cumulative discounted with access controly from
starting state with optimization formulation for
different channel

max Jl (n)
n

4.2. Re-enforcement Model for Multi-

Access Control

From figure 3 we can observe that the proposed

learning system will overcome the issues that are coming
from
Fig 2 System implementation for k-channel access uplink. This will mainly control the access in effective way. At
every starting point of TS, BS gets the data based on controller.
control
In this both BS and UEs are used for learning procedures.
The main intent of BS is to learn about the current
channel additions and in the same way the main intent of
UEs is to check the states of battery. In this the system
works depend on the channels performance. Generally,
when framework enters into a new state then it changes
the current position from BS to TS by using RL system.
After this process, all the channels in the system will
communicate with the UEs. Now, after some time, UEs
will transfer the information to the BS by controlling the
P individually. At last the information is executed and
safely, transferred and saved into battery for future use.

4.1. Problem Description

Fig 3 NRELA Network simulation model with
Basically, to boost up the framework a control strategy

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:10:04 UTC from IEEE Xplore.
 2021 7th International Conference on Advanced Computing & Communication Systems
(ICACCS)

In this the layer which is available in the information is with likelihood 1 - , where 0 << 1. In the wake of executing the
more smaller than the reinforcement model. Qminibatch(s, a) £ chose activity At, the BS gets the remunerate Rt and
the R batch size^L, is the present state model. Here the size of framework changes to the new state. We use experience replay
activity space is L based on the condition a £ A. to store the BS's encounters at every TS, which is indicated
by
tuple et = (St, At, Rt, St+1) in a dataset D = {e1, ..., et}. The
value of L is set to replay memory size, which signifies that
L
experience tuples could be stored. Here, et is produced by the
Now to alter the position vector for Q-esteem, a control approach n(a|s). In each TS, rather than refreshing 9a
system layer is introduced ot used. The condition for this
dependent on advances from the present state, we arbitrarily test
system layer is given as i.e., Q(s, a) 6 R 1*L. from figure 3 we
a tuple (~s, a, ~ r, ~ s") from D. Refreshing system parameters
can observe that ^A(9a) is nothing but an neural system which
along these lines to dodge issues brought about by
gives the output Q(s, a). In this LSTM layer parameters {w1, •
solid
We connections among changes of a similar scene [22].
,wn}are quiet associated with the parameter wf. The quantity of
parameterize a surmised worth capacity Q(s, a; 9a) utilizing
LSTM units is donated by n. Q(St, an) is evaluated by ^A(9a)
the
proposed learning system with system parameters (loads) of
9a
when the learning schedule is opened.
as shown in figure 3. With the examined advances, yt = ~r +
y
maxa" Q("s, a"; 9 - a ) is the objective Q-esteem with
1:Initializetheexperience memory D, system loads 9 - a acquired from past cycle. Finally
multi channel
2: Initializetheparameters of activitygeneratororganize(pAwithirregularloads 0a, access control with different attribute sequences in
software
3: Initializetheabsolutenumberofscenes Ep, defined network communication.
4: Initializethe earthandget beginningperceptionSl,
V. Experimental Simulations
5:fort= 1, ■,oodo
6: in the eventthat megularO < atthat point 1:Select an arbitraryactivityAt EA; In this section, we describeNovel Re-
Enforcement
8:else Learning Approach (NRELA) to improve the
transmission
9: Compute Q(St, a) for all activitiesa E An utilizing tpA, efficiency using cognitive radio software defined
networks
through multiple channel to increase network throughput
10:
and Select At= argmax aEAQ(St, a).
energy consumption with respect to different nodes.
11: end if
For
efficient simulation setup, use NS3 with Ubuntu
operating
12: ExecuteAt, watch compensateRt andnew state St+1, system with different node communication.
Simulation
13: Storeprogress(St,At,Rt, St+1) inD, parameters used in our implementation shown in table 1.
14: Sample irregularlittle clump of advances ("s. a,‘ r,‘ s") ftomD,
Parameter Value
15: Setyt= ~rif t +1 is the terminal advance ofthe scene (t + 1= Ep); generally, yt= ~r+ y maxa' Qfs, a”; fl-a).
Area of network 1500*1500
16: Perfonn stochastic slope dropventure onthe misfortuneworkLt(0a)= (yt - Q('s, a"; 0a))2 to refresharrange Noodes with 60
parameters 0a as indicatedby (13).
presented area
17:endfor Time of Simulation 30S
Range of 250 m
Algorithm 1 NRELA algorithm procedure with Transmission
respect to access control. Speed of Mobility 0-20m/sec
Number of 10
At that accomplishes the most extreme Q(St, At), and

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:10:04 UTC from IEEE Xplore.
 2021 7th International Conference on Advanced Computing & Communication Systems (ICACCS)

Time comparison results in software defined networks with As the number of nodes increased then the number
of nodes communication with respect to time for packets droping outcomes in real time data transmission of host to
host in middle of data delivery by hop by hop communication. Table communication energy consumption in our NRELA
schema
2 shows analysis results with respect to time in data
gives efficient communication without loss of data delivery
in
communication between nodes.
software defined network communication as shown in
figure Number NRELA CWA-CD 4.5.
of Nodes
VI. Conclusion
10 1.3 1.28 In this paper, we present Novel Re-enforcement
Learning
20 2.0. 2.7 model (NRELA) to provide solution for the user access
control
30 3.2 2.8 and also describe the battery prediction problems with
respect
40 4.2 4.5 to multi user energy sharing based communication system.
The
50 5.6 4.9 main intent of proposed system is to maximize the uplink
sum
60 5.4 5.5 rate which is driven by system instantaneous information
in
data sharing. Energy optimization is also discussed to
minimize Table 2: Node communication with time comparison results. the packet loss. Simulation results of proposed
approach are Time comparison results for different approaches with satisfied different conditions with increase
effectiveness in respect to different nodes with efficient communication. terms of parameters like throughput, packet
loss, latency and
energy optimization.

References

[1]. Jiang Zhu, Yonghui Song, Dingde Jiang, “ A New Deep-Q- Leaming-
Based Transmission Scheduling Mechanism for the Cognitive
Internet of Things”2327-4662 (c) 2017 IEEE. Personal use is
permitted, but republication/redistribution requires IEEE
permission.
[2]. Khalil N, Abid M R, Benhaddou D, et al. Software defined sensors
networks for Internet of Things [C]// IEEE Ninth International
Conference on Intelligent Sensors, Sensor Networks and
Information Processing. IEEE, 2014:1-6.
[3]. S. Jeschke, et al. (Eds.), "Industrial Internet of Things: Cyber-
manufacturing Systems," ISBN 978-3-319-42559-7, Springer,
Switzerland, 2016.
[4]. Yang J, He S, Lin Y, et al. Multimedia cloud transmission and
Fig 4 Comparison of time with respect to node storage system based on Internet of Things[J]. Multimedia
Tools

communication and Applications, 2015: 1-16.

[5]. Yuan B, Fu C, Chen D. Building a Large Scale Software defined
Sensor Network for the Industrial Environment[C]// IEEE,
International Conference on Embedded and Real-Time Computing
Systems and Applications. IEEE Computer Society, 2016:96-96.
[6]. Khan A A, Rehmani M H, Rachedi A. When Cognitive Radio
meets the Internet of Things?[C]// International Software defined
Communications and Mobile Computing Conference. 2016:469-
474.
[7]. Haustein, Thomas, et al. "Cognitive Software defined
Communications-A paradigm shift in dealing with radio
resources as a prerequisite for the software defined network of the
future-An overview on the topic of cognitive software defined
technologies." Frequenz 70.7-8 (2016): 281-288.
[8]. Otermat D T, Kostanic I, Otero C E. Analysis of the FM Radio

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:10:04 UTC from IEEE Xplore.
 2021 7th International Conference on Advanced Computing & Communication Systems (ICACCS)
[11] . Man Chu, Hang Li, “Reinforcement Learning based Multi-Access
Control and Battery Prediction with Energy Harvesting in IoT
Systems”, arXiv:1805.05929v2 [eess.SP] 21 Sep 2018.
[12] . A. Haldorai and A. Ramu, “Security and channel noise
management in cognitive radio networks,” Computers & Electrical
Engineering, vol. 87, p. 106784, Oct.
2020. doi: 10.1016/j.compeleceng.2020.106784
[13] . A. Haldorai and A. Ramu, “Canonical Correlation Analysis Based
Hyper Basis Feedforward Neural Network Classification for
Urban Sustainability,” Neural Processing Letters, Aug. 2020.
doi:10.1007/s11063-020-10327-3
[14] . A. Cammarano, C. Petrioli, and D. Spenza, “Online energy
harvesting prediction in environmentally powered software
defined sensor networks,” IEEE Sensors J., vol. 16, no. 17, pp. 6793-
6804, Sep. 2016.
[15] . H. U. Yildiz, V. C. Gungor, and B. Tavli, “A hybrid energy
harvesting framework for energy efficiency in software defined
sensor networks based smart grid applications,” in 2018 17th
Annual Mediterranean Ad Hoc Networking Workshop (Med-Hoc-
Net). Capri, Italy: IEEE, June 2018, pp. 1-6.
[16] . A. A. Nasir, X. Zhou, S. Durrani, and R. A. Kennedy, “Relaying
protocols for software defined energy harvesting and information
processing,” IEEE Trans. Software definedCommun., vol. 12,
no. 7, pp. 3622-3636, Jul. 2013.
[17] . R. Zhang, J. Wang, Z. Zhong, C. Li, X. Du, and M. Guizani,
“Energyefficient beamforming for 3.5 GHz 5G cellular networks
based on 3D spatial channel characteristics,” Elsevier Comput.
Commun., vol. 121, no. 5, pp. 59-70, Mar. 2018.
[18] . X. Zhou, B. Bai, and W. Chen, “Greedy relay antenna selection
for sum rate maximization in amplify-and-forward mimo two-
way relay channels under a holistic power model,” IEEE Trans.
Software definedCommun., vol. 19, no. 9, pp. 1648-1651, Jun.
2015

1853

Authorized licensed use limited to: California State University Fresno. Downloaded on July 01,2021 at 02:10:04 UTC from IEEE Xplore.