2021 8th Intl.

Conference on Soft Computing & Machine Intelligence

A Survey of Predictive Maintenance and Self-

optimization in Telecom Field Based on Machine
Learning
2021 8th International Conference on Soft Computing & Machine Intelligence (ISCMI) | 978-1-7281-8683-2/21/$31.00 ©2021 IEEE | DOI: 10.1109/ISCMI53840.2021.9654819

Mahmoud Fouad Darwish

Information Systems Department
Faculty of Computers and Artifictial Intelligence
Cairo, Egypt
[email protected]

Abstract—The role of telecommunication and mobile early detection system where sensors collect operation data
networks changes from a medium that connects people through about current machine status and send it to be analyzed by
voice to a complex system that affects almost everything in our machine learning models which based on received data, can
life. It is not just connecting people but also connecting things. It predict equipment health, recommend maintenance and feed it
also acts as a medium for many services in almost all domains to decision support system to take action.
including health, education, security, and other domains. As
telecom network grows in a complex way and our needs for a
stable and high-quality service grow every day, there is a
necessity to improve network and telecommunication services
quality and ensure service availability 24/7. Nowadays we are
also heading towards the industry 4.0 revolution, which
revolutionizes maintenance, quality, and productivity. Network
predictive maintenance is an important research field to take
corrective actions that prevent network failure. In addition to
that having autonomous network and telecommunication systems
that could adapt to traffic changes and customer changing
behavior becomes an important research field. Machine learning
has always contributed to the implementation of predictive
maintenance of telecom networks along with the autonomous
self-adapted network. In this paper, we survey predictive
maintenance in the telecommunication field and mobile networks.
In addition to that, we survey network self-optimization through
self-organizing networks, network slicing, and software-defined
networks. Fig. 1. CPS architecture for maintenance and PdM.

Keywords—machine learning, predictive maintenance, self- Self-organizing Network (SON) [4][5][6][16] is considered
organizing networks, network slicing, 5G, telecom a key factor for enhancing Operations, Administration, and
I. INTRODUCTION Management (OAM) activities. SON aims at reducing the cost
of installation and management of 4G and future 5G networks,
Industry 4.0 introduce several changes to industrial by simplifying operational tasks by adding the ability to
automation. It transforms traditional manufacturing into configure, optimize and heal itself. It reduces the need for
intelligent sensor-based factories that depend on technology human intervention through dynamic and self-configuration. It
and decision support systems. Self-awareness and self- is introduced by 3GPP in release 8. The main objective of a
maintenance are some of the characteristics of Industry 4.0 self-organizing network is to introduce autonomous
Predictive Maintenance (PdM) [1][2][3][15] is maintenance adaptability into mobile and telecommunication networks.
that monitors the condition and performance of equipment Furthermore, enhancing cellular network performance in terms
during normal operation to reduce the probability of failures. of capacity, coverage, and services. It also targets to self-
The goal is to identify when the equipment would fail so that optimize network and adjusts network configuration when
we can take corrective maintenance actions that prevent this needed automatically. Machine learning can make use of SON
failure. Predictive maintenance brings many advantages to learn from network experience and current status then take
including productivity improvement, reduction of system errors, appropriate optimization and configuration action on the
reduction of sudden issues, reduction of unplanned outages and network through SON.
increase efficiency. Using machine learning can help in the
prediction of failures through analysis of the historical and Software-Defined Networking (SDN) and Network
current status of equipment. Fig. 1 presents an overview of the Function Virtualization (NFV) are the key factors that bring the
network architecture from a purely hardware-based paradigm

978-1-7281-8683-2/21/$31.00 ©2021 IEEE 9

Authorized licensed use limited to: ULAKBIM UASL - Galatasaray Universitesi. Downloaded on August 30,2023 at 12:30:19 UTC from IEEE Xplore. Restrictions apply.
to a completely software-based paradigm created virtually over from the first customer trouble ticket type to another customer
the physical hardware layer [5][6][8][9]. SDN and NFV change trouble ticket type. μAB is the probability of transition from
the network function from being hardware-based (e.g., state customer trouble ticket A to customer trouble ticket B. t1
baseband processing, mobility management) to software-based represents the time taken to move from customer trouble ticket
Virtual Network Functions (VNFs) running on a, possibly A to customer trouble ticket B, so from this figure we can
hierarchical, general-purpose telco-cloud. Network slice predict that if trouble ticket of type A occurs then there is a
[6][7][8][9][10][11][12] is composed of a set of VNFs that run probability with value μAB that another trouble ticket type B
on virtual network infrastructure and provide a specific would occur after t1 time. Authors propose another metric that
telecommunication service. Network slicing allows different is helpful to assess the relationship between customer trouble
services to be handled on the same infrastructure while each tickets is the mean time between failures, MTBF, which
service can be configured and implemented separately represents the average time interval between failures. MTBF
according to their specific requirements [13]. Network slicing (t1) is the average time difference between CTT (A) and
is also introducing new business models and allows CCT(B) while MTBF(t2) represents the mean time difference
infrastructure provided to rent resources to tenants and tailor between failure where two types of failure CCT(A) & CCT(B)
different services to specific needs on the same are resolved together and the next subsequent trouble type as in
infrastructure[14]. Network slicing is considered the same Fig. 2.
business and technical model as Infrastructure as a service.
They allow having multiple different services with different
hardware configurations and service levels over the same
physical infrastructure.
The rest of this paper is organized as follows. Section 2
presents researches in predictive maintenance in
telecommunication. Section 3, presents researches in self-
organizing networks. Section 4, presents research in network
slicing optimization. Section 5, presents the conclusion and
future work. Finally, References are given in the last section.
II. PREDICTIVE MAINTENANCE IN TELECOMMUNICATION
DOMAIN
In this section, we present summary of literature review and Fig. 2. Generic MP model and MTBF for CTT types.
related work of predictive maintenance in telecommunication
domain. The proposed approach applied to the Malaysian Telecom
operator, the data used in excremental results represents one
A. Predictive Maintenance of Network Elements using
year of customer trouble tickets in the Malaysian telecom
Markov Model to Reduce Customer Trouble Tickets [1][2]
operator. It contains CTT Type, the status of trouble ticket
Md. Kamal Hossain et.al proposed an approach to predict whether open or closed, creation date, priority, platform, and
maintenance of telecom network components through applying location. As datasets contain many trouble ticket types which
Markov Process on customer trouble tickets for telecom exceed 150 types which would make it hard to have a 150*150
service issues or service degradation. The main idea behind matrix to represent Markov decision probabilities so filtering is
this approach is to predict whether the occurrence of a specific performed to keep only CTT types that contribute to eighty
trouble ticket could lead to the occurrence of another trouble percent of the majority of overall customer trouble tickets. The
ticket type after some time. Markov Process, MP, is used to filter process generated 11 CTT types. The eleven CTT types
provide a model that can approximate the probability of the are mapped in a transition matrix that shows probabilities for a
next occurrence of a service problem considering the current transition from CTT type to another CTT along with the MTBF
occurrence of another service problem. The prediction process value. These probabilities are estimated using the Markov
requires monitoring for the existing sequence of events and process model. These probabilities help to predict the
analyze the sequence of trouble tickets creation. MP model probability of a next trouble type occurrence considering
prediction allows technicians to quickly take proactive actions currently raised trouble tickets. Experimental results show that
towards preventing predicted issues. This helps to stabilize the Markov model could help to predict future network
network issues, increase the quality of service and decrease problems which help to prevent issues through taking proactive
telecommunication network downtime. actions.
The proposed Markov Process model is represented by a B. Enabling Predictive and Preventive Maintenance using
series of failed states. Each failure state represents a customer IoT and Big Data in the Telecom Sector [3]
trouble ticket, CTT, type raised by the customer. Figure 2
shows a simple representation of a Markov Process model with Improving network quality through reducing telecom
two customer trouble ticket types represented as circles. The network errors is an important objective in the telecom sector
arrows between circles represent the direction of transition to attain customer satisfaction. researchers present a predictive
between different trouble ticket types. Each transition has a and preventive maintenance framework for radio access
probability value that represents the probability of moving network (RAN) based on IoT and Big Data platforms.
Handling maintenance operation for RANs is very challenging

10
Authorized licensed use limited to: ULAKBIM UASL - Galatasaray Universitesi. Downloaded on August 30,2023 at 12:30:19 UTC from IEEE Xplore. Restrictions apply.
as they usually belong to different geographical locations and A. Optimal-Capacity, Shortest Path Routing in Self-
maintenance and increase operational costs. The proposed organizing 5G Networks Using Machine Learning [4]
framework by researchers [3] for predictive maintenance Murudkar et.al [4] proposes a machine learning approach
depends on data from telecom equipment along with historical based on the reinforcement learning technique, Q-Learning, to
and current data collected from sensors and IoT devices. This define the shortest and optimal path between source and
process decreases the cost needed for maintenance and destination for the end-user. The proposed method is called
increases network stability. Proposed approach targets user-specific optimal capacity and shortest path (US-OCSP).
maintenance of telecom base stations equipment and Self-Organizing network, SON, represents an automation
infrastructure. ‎proposed approach covers predictive technology that is designed to make the planning, management,
maintenance for all Base station sites components which configuration, optimization, and maintenance of mobile radio
include receiver and transmitter device, AC Power converter, access autonomous, simpler and quicker. The proposed
air conditioner, battery, RF cables along with oil storage, approach avoids congested network nodes to achieve optimal
electricity generators, and others. throughput and reach optimal resource allocation in a 5g SON
The proposed framework as shown is composed of four network. SON is considered a core part of mobile network
core activities in order to predict the need to perform capacity optimization. Optimization of capacity in a Self-
maintenance. The first activity requires accessing data, the organizing network includes mobility load balancing (MLB).
second activity is to process received data, the third activity MLB is a process in which cells suffering from congestion
develops a predictive model, and the last activity is to feed could transfer the load to other cells that have available
model prediction into the analytics framework. The outcome of resources.
the framework is to define which areas needed preventive US-OCSP customizes the capacity needs of the end-user to
maintenance. The proposed framework depends on different identify the shortest path with the best capacity for input source
types of data sources including hardware, alarms, IoT sensors and destination. Capacity is measured by the available physical
whether temperature sensor or other sensors and key resource blocks at all nodes that exist as part of the path from
performance indicators related data that can help to predict source to destination. US-OCSP depends on machine learning
whether the base station is working fine or need maintenance. algorithm q-learning to identify the shorted path and avoid
The proposed Framework depends on Hadoop and hive as big congested network nodes. The algorithm secures the shorted
data platforms to save different received data. It is also path where most of the resources are available and avoids any
integrated with the business intelligence platform to visualize busy nodes' resources. Given the source and destination of a
maintenance recommendation and framework prediction user inquiry, US-OCSP calculates the available capacity of all
results in order to allow field engineers to start the maintenance nodes gNodeBs in terms of 5g and eNodeBs in terms of 4g.
process for suggested base stations and radio access network Capacity is calculated in terms of physical resource blocks
towers. (PRB). Physical resource block utilization is calculated for
The architecture of telecom network base station sites each to be able to measure its capacity. In case of utilization is
contains multiple components. Each component has its own above 70% then the node is marked as busy otherwise it would
prediction factors which means some information contributes be marked as available.
greatly to the prediction of specific components while the same
attributes might not be helpful in maintenance prediction for
other base station components. The proposed approach
depends on running diagnostic tests on radio hardware in order
to predict whether there is an issue in the receiver or
transmitter or not. The result of these tests is sent to our
predictive model. These test results are either pass or fail. In
order to identify the health of RF cables and connectors, IoT
devices information are gathered including RF measurements,
voltage standing wave ratio, cable loss, return loss, and others.
In order to predict health status for civil infrastructure for base
stations, the proposed approach depends on different sensors Fig. 3. Representation of a reinforcement learning system.
are installed in the base stations and send live updates to our
prediction model. in order to predict over network failure status, Q-Learning, which is a reinforcement learning method is
the proposed framework depends on key performance used to define the shortest part that can be taken with optimal
indicators which include calls attempted, calls dropped, calls resources utilization. Reinforcement learning address the
successful, calls failure, total throughput, throughput on question of how an autonomous agent can take actions in his
downlink and uplink, packet loss, packet dropped and other environment to achieve his goals and maximize rewards. Fig. 3
important factors. ‎ is a representation of reinforcement learning. each time agent
perform an action a from list of possible actions A in a certain
III. SELF-ORGANIZING NETWORKS state S, the agent receives as a result of either a reward or
In this section, we present the summary of the literature penalty r as a result of his action a. repetitive actions generates
review and related work of the self-organizing network in the a sequence of si, actions ai, and reward ri as each action ai in a
telecommunication domain. certain state si would generate a reward ri as a result. The agent

11
Authorized licensed use limited to: ULAKBIM UASL - Galatasaray Universitesi. Downloaded on August 30,2023 at 12:30:19 UTC from IEEE Xplore. Restrictions apply.
aims to learn the actions that maximize the total sum of metrics, the correlation coefficient (R) is also used to show
rewards. as delayed rewards might have low weight in how the prediction model fits the date. The proposed approach
comparison to quick rewards, a discounted factor y is is compared against the normal approach which performs
introduced to the reward summation equation to control giving prediction for whole network cells. ‎ Experiments show that the
higher weight for quick rewards and low weight for delayed cluster-based prediction approach outperforms the original
rewards to teach an agent to take actions that achieve quick whole network cells prediction approach. In all experiments
rewards as quickly as possible. using different machine learning prediction models, the new
proposed approach achieves higher correlation, lower mean
B. Applying Big Data, Machine Learning, and SDN/NFV to square error, and smaller mean absolute error.
5G Traffic Clustering, Forecasting, and Management
[5][6] Predicting network traffic is important to plan for network
Big Data, Machine Learning, Software Defined Network design and capacity. Researchers [6] propose a framework to
(SDN), Network Function Virtualization (NFV) are control and avoid network congestion. The proposed approach
empowering the self-organizing network (SON) of 5G to depends on cluster-based prediction model results to set a
handle optimization problems. Luong-Vy le and other safety margin for congestion avoidance. The safety margin is
researchers proposed a practical framework based on SDN, calculated using (1), considering that average error and
NFV, and Machine learning for predicting and managing standard deviation errors are calculated based on the previous
traffic behavior in a huge amount of network consists of seven values for the same hour. safety margin could lead to
different types of cells. Firstly, the proposed approach performs decrease congestion probability however sometimes abnormal
clustering for traffic in all network cells to identify complicated problems occur and traffic patterns change so researchers
traffic behavior and extract cells that share similar traffic proposed two alarms for detecting changes and take action.
patterns. The proposed framework depends on k-means to Those alarms are minor alarm and major alarm, minor alarm
apply traffic clustering with an initial size of 40 clusters. ‎ means actual traffic exceeds predicted traffic while major
TABLE I. shows an example from input features shared as an alarm represents repeated minor alarm for past five values.
input to clustering algorithm including average and the
percentage value of traffic on different time intervals on a 3g Safety Margin=Forecasted Traffic Average Error+ 3*
cell. ‎ TABLE II‎ ‎ presents traffic clustering results. Grouping Standard Deviation (Error)  
cells into clusters makes it easy to understand traffic behavior
as cells inside a cluster share a similar traffic profile. Software-defined networking (SDN) is used to dynamically
change network traffic rules and topology based on traffic
TABLE I. INPUT TRAFFIC SAMPLE PER HOUR AND PER DAY patterns and sudden shutdown or failure of a specific network
cell. The proposed method for dynamically assign a new route
Hour 0 1 02 03 …
for traffic is proposed based on SDN [5][6]. The proposed
Avg usage (MB) 10 20 30 10 … method consists of three steps. The first step is building a new
network topology and calculate shorted paths between different
Usage 1% 2% 3% 1% …
network elements using an SDN controller based on predicted
Percentage
(Hour Mon Tue Wed Thu … traffic and network components. The second step, switch on
new network elements and install new routing rules using the
Avg usage (MB) 1000 200 3000 1000 …
0 0
SDN controller. Path between nodes dynamically change based
Usage 10 % 20% 30% 10% … on traffic and network changes dynamically. The third step
Percentage would switch off all unnecessary network elements and assign
all traffic to new paths. Redesigning network elements and
TABLE II. TRAFFIC CLUSTER SAMPLE paths between network cells based on traffic dynamically helps
to avoid congestion through self-organizing networks and
Cluster # 1 2 3 4 …
software define networking.
Cluster 140 190 52 20 …
Size
IV. NETWORK SLICING
In this section, we present a summary of the literature
The proposed approach performs traffic forecasting based review and related work of network slicing in the
on clustering results as traffic forecasting on cell level would telecommunication domain.
be computationally expensive and traffic forecasting on all
network-level might not be accurate due to differences in A. Big Data for 5G Intelligent Network Slicing Management
traffic behavior on cells. The proposed approach uses different [7] [8] [9]
machine learning algorithms to perform traffic prediction on Network slicing is considered an emerging application in
the cluster level. The proposed approach utilizes linear telecommunication and 5g cellular networks. It creates
regression (LR), polynomial regression (PR), Neural Network different and independent logical layers over the same physical
(NN), and Gaussian process (GP) to predict traffic based on infrastructure. as different slices share the same physical
historical traffic patterns in previous time series. The proposed resources then dynamic resources allocations need to be
approach considers meaning absolute error (Ema) and root managed in a way that achieves a better quality of service for
mean square error (Erms) as performance metrics to evaluate the physical network resources. Machine learning and deep
the performance of the prediction model. In addition to these learning techniques are used to facilitate having dynamic

12
Authorized licensed use limited to: ULAKBIM UASL - Galatasaray Universitesi. Downloaded on August 30,2023 at 12:30:19 UTC from IEEE Xplore. Restrictions apply.
network slicing that fits traffic prediction and achieves network learning that applies deep learning techniques in order to reach
service level constraints. They help in utilizing physical optimal policy. The reward in 5g could be an improvement in
network resources and at the same time satisfy service network, reduction in system outages, optimization in power
requirements for applications using different network slices. consumption, and other factors that related to quality or
improvement of 5g network. DQN is equivalent to Q-function
Network Slicing requires big data storage and distributed in RL, it takes the current state as an input and output set of Q-
processing framework that could handle storage processing in a values for all possible actions that could be taken in the input
distributed way. In addition to that, it requires machine state.
learning to predict traffic and control resource allocations.
Network Function Virtualization (NFV) [8] in 5G networks is Mobile 5G network complexity and network resources
expected to fill the void of programmable control and management make it necessary to implement network slicing.
management of network resources. It makes managing virtual Slicing distributes mobile networks into different separated
network resources dynamically possible. NFV allows also parts. Each slice is serving different service requirements.
reconfiguration and reassigning of resources to virtual network Several researchers proposed optimal dynamic resource
functions in real-time. Researchers [7] proposed core building management for network slicing to improve service quality
blocks that enable smart network slicing management in a [13]. However, the existing approaches are not efficient due to
cellular network. The first key building block is the ability to 5g network complexity and diverse service requests from users
predict the traffic on the network slice level. The second factor in different slices which occurs on a random basis. Therefore,
is implementing a dynamic resource provisioning solution researchers [10] propose DRL based approach for network
based on deep neural networks and machine learning slicing optimization in terms of resource allocation for different
techniques. To generate prediction rules faster from the deep users’ requests for network slices. The system resources in the
neural network, researchers propose to capitalize on distributed proposed approach [10] are virtualized into different network
and parallel processing frameworks like Hadoop and/or Spark. slices. Each network slice handles a set of services and network
resources to serve incoming user requests as appears in the
B. Deep Reinforcement Learning for Mobile 5G [10] [11] "service view" part. Through using slice mode, users can
[12] [13] request different services provided by the network without
Reinforcement learning is an important machine learning being aware of the physical layer. The scheduling process is
technique. It allows an agent to improve decision-making presented under "Optimization view", in this part, received user
without any need to know a previous knowledge of the requests would be processed then assigned to different slice
surrounding environment. The agent performs some decisions requests to be handled in different assigned slices. The slice-
and actions in a specific system state then notices the allocation controller handles different slice requests from the
environment responsible for the action taken. The agent either queue based on the first in first out (FIFO) mechanism and
receives a penalty or a reward for his action and the state allocates corresponding network slices to the required network
changes to a new state. The reward represents that action taken resources. Once the request is processed and served, the slice
is a good action while penalty means that action taken is a bad controller deletes the request from the queue.
action. The agent learns through try and error. The goal is to
identify the best action in a certain state or identify state-action The proposed DRL-Based approach for slice allocation
pairs that lead to the optimal solution. The state-action pairs are controller aims to optimize slice request allocation to the
called policy. The agent is encouraged to perform actions that corresponding network slice. The proposed approach monitors
would lead to maximum benefits and rewards based on trials system states including current queue length in addition to
experience. Q-learning is a famous reinforcement learning (RL) currently available resources. When a slice request is handled
algorithm. It computes optimal policy that achieves maximum successfully, a reward is being given to the slice allocation
long-term reward. Q function is introduced in Q-learning RL controller. The controller targets to maximize rewards through
algorithm in order to map a state-action pair to the expected handling all requests successfully. When a queue is delayed,
reward result. It allows agent to identify optimal actions in the allocation controller is given a penalty so the total utility
different states. Through recording list of all actions that gained would be the output of total rewards received subtracted
archives maximum Q-Value, agent is able to generate optimal by the total delays penalties. To achieve the maximum utility
state-action pairs which reflects optimal policy. RL is used to for the slice allocation process, DQN is being implemented by
solve network resource management and optimize resource the allocation controller. DQN implemented in the slice-
allocation in mobile communication systems. In order to define allocation controller doesn't need any previous information of
the optimal service and resources, agent of a network entity the network as it is model-free. It would learn from trial and
calculates the Q-function for best action in currents state. error of the service allocation process to different user requests.
Based on historical information of service requests along with
Q-learning requires the agent to calculate Q-values for all their reward or penalty result, the slice allocation policies can
state-action pairs. This operation could be computationally be adjusted dynamically by the controller.
complex in future generation wireless networks due to the size
and scale of wireless communication networks. As the number C. A Machine Learning Approach to 5G Infrastructure
of possible states and actions becomes very large therefore it is Market Optimization [14]
really difficult to maintain Q-values for all states and actions. Network slicing is creating a new business model by
In order to solve this problem, Deep Reinforcement Learning allowing infrastructure providers to share parts with their own
(DRL) is introduced. Deep Q-learning is reinforcement network with tenants to deliver services to their customers. In

13
Authorized licensed use limited to: ULAKBIM UASL - Galatasaray Universitesi. Downloaded on August 30,2023 at 12:30:19 UTC from IEEE Xplore. Restrictions apply.
this situation, maximizing infrastructure profits is a core slicing to handle resource allocations for different network
problem that requires a solution. Begat et al. proposed slices, in addition, to accept or reject the rental request in a way
Network-slicing Neural Network Admission Control (N3AC) that maximizes rewards without affecting the quality of service.
algorithm that is developed based on deep reinforcement As A future work, we plan to merge predictive maintenance
learning in order to act as an admission control algorithm. The along with self-network optimizing and reinforcement learning
admission control algorithm aims to maximize the to predict and take the best corrective action automatically
monetization of the infrastructure by taking the best decisions rather than having to handle manual corrective action which
in terms of accepting to rent a network slice or reject in a way requires human intervention. We also plan to include customer
that maximizes rewards and at the same time ensures network feedback and external structured and unstructured data sources
quality. Due to the 5g network in terms of complexity and size, like social media platforms as an important factor in predictive
it becomes hard for normal approaches to handle network maintenance.
slicing recommendations. Therefore N3AC was proposed to
reduce computational complexity and deal with nowadays REFERENCES
network characteristics. [1] Abiad, Mohammad, Seifedine Kadry, and Sorin Ionescu. "Preventive &
Predictive Maintenance of Telecommunication Equipment-A Review."
N3AC [14] is considered a deep reinforcement learning 2018 4th International Conference on Applied and Theoretical
approach where an agent is represented by an Infrastructure Computing and Communication Technology (iCATccT). IEEE, 2018.
provider (InP) deals with the environment and takes decisions [2] Hossain, M. K., Shahrir, M. S., Yusof, M. I. M., Yusof, Z., & Asraf, N.
at a specific state leads the environment to respond to these M. (2018). Predictive maintenance of network elements using Markov
model to reduce customer trouble tickets. 2017 IEEE Conference on Big
actions with rewards or penalties. The feedback which agents Data and Analytics, ICBDA 2017.
receive from the environment helps the agent to identify the
[3] Mahmood, Tahir, and Kamran Munir. "Enabling Predictive and
optimal action in certain situations. The agent learns through Preventive Maintenance using IoT and Big Data in the Telecom Sector."
historical decisions how to estimate the expected reward for (2020).
each state-action pair.N3AC doesn't need any prior output to [4] Murudkar, Chetana V., and Richard D. Gitlin. "Optimal-capacity,
operate as it is integrated with reinforcement learning shortest path routing in self-organizing 5G networks using machine
framework which enables it to learn through interacting with learning." 2019 IEEE 20th Wireless and Microwave Technology
the environment. N3AC considers the average revenues Conference (WAMICON). IEEE, 2019.
expected at a given state after taking a certain action. N3AC [5] LI, Zhengmao. Telecommunication 4.0. 2018. (SDF/NFV)
uses two neural networks, the first one aims to estimate [6] Le, Luong-Vy, et al. "Applying big data, machine learning, and
SDN/NFV to 5G traffic clustering, forecasting, and management." 2018
revenue for each state when actin is to accept an incoming 4th IEEE Conference on Network Softwarization and Workshops
request, the second one aims to calculate revenue if we reject (NetSoft). IEEE, 2018.
the incoming request. N3AC selects the action with the highest [7] Chergui, Hatim, and Christos Verikoukis. "Big Data for 5G Intelligent
revenue. Network Slicing Management." IEEE Network 34.4 (2020): 56-61.
[8] Barakabitze, Alcardo Alex, et al. "5G network slicing using SDN and
V. CONCLUSION AND FUTURE WORK NFV: A survey of taxonomy, architectures and future challenges."
Machine learning plays an important role in the field of Computer Networks 167 (2020): 106984.
telecommunication and mobile-cellular networks. It is used [9] Barakabitze, Alcardo Alex, et al. "5G network slicing using SDN and
NFV: A survey of taxonomy, architectures and future challenges."
effectively in predictive maintenance to predict failure of Computer Networks 167 (2020): 106984.
different parts of a mobile network including radio access [10] Xiong, Zehui, et al. "Deep reinforcement learning for mobile 5g and
network which helps to plan maintenance and take corrective beyond: Fundamentals, applications, and challenges." IEEE Vehicular
actions that lead to increase service availability and quality of Technology Magazine 14.2 (2019): 44-52.
service. It is used also to predict customer trouble tickets based [11] Morocho-Cayamcela, Manuel Eugenio, Haeyoung Lee, and Wansu Lim.
on current trouble tickets through mapping relationships "Machine learning for 5G/B5G mobile and wireless communications:
between different customer complaint types based on machine Potential, limitations, and future directions." IEEE Access 7 (2019):
137184-137206.
learning techniques. Predicting the next customer complaint is
helping to avoid sudden outages and optimize network [12] Sciancalepore, Vincenzo, Xavier Costa-Perez, and Albert Banchs. "RL-
NSB: Reinforcement learning-based 5G network slice broker."
performance based on analysis results of machine learning IEEE/ACM Transactions on Networking 27.4 (2019): 1543-1557.
techniques to existing information in the ticketing system. [13] P. Caballero, A. Banchs, G. de Veciana, X. Costa-Pérez, and A. Azcorra,
Machine learning techniques are also used to avoid congestion “Network slicing for guaranteed rate services: Admission control and
in telecom networks and selecting the best shortest path resource allocation games,” IEEE Trans. Wireless Commun., vol. 17, no.
between nodes based on current network traffic through 10, pp. 6419–6432, 2018. doi:10.1109/TWC.2018.2859918.
applying reinforcement learning on current network traffic then [14] Bega, Dario, et al. "A machine learning approach to 5G infrastructure
using the self-organizing network to reconfigure the network to market optimization." IEEE Transactions on Mobile Computing 19.3
(2019): 498-512.
avoid congestion. Clustering and regression machine learning
[15] Dalzochio, Jovani, et al. "Machine learning and reasoning for predictive
techniques are also used to categorize network cells that have maintenance in Industry 4.0: Current status and challenges." Computers
similar behavior then predict usage for each identified cluster in Industry 123 (2020): 103298.
using regression methods then self-optimize the network [16] Moysen, Jessica, and Lorenza Giupponi. "From 4G to 5G: Self-
through making use of new network advances in terms of organized network management meets machine learning." Computer
software-defined networks and network virtual functions which Communications 129 (2018): 248-268.
become available in current networks. Reinforcement Machine
learning techniques are also applied successfully on network

14
Authorized licensed use limited to: ULAKBIM UASL - Galatasaray Universitesi. Downloaded on August 30,2023 at 12:30:19 UTC from IEEE Xplore. Restrictions apply.