WELCOME

TO OUR PRESENTATION

Presentation on
“Machine Learning for Network Automation”
 Presented By
Group-15
Group Members:

Khandokar Jidny Islam

Mehnaj Tabassum
ID:19102903
ID:19102912
Session:2018-19
Session:2018-19
EEE,JKKNIU
EEE,JKKNIU

Presented To
Bijoy Kumer Karmaker
Associate Professor
Electrical and Electronic Engineering
Jatiya Kabi Kazi Nazrul Islam University
Contents:-
 Introduction
 Machine Learning Workflow
 Algorithm
 Evaluations
 Data Management
 Network And Model Management
 Application Scenarios
 Future Work And Industrial Perspective
 Conclusion
Introduction
Optical communication networks may very well be regarded as the cornerstone of
modern society. Internet-based applications and classical enterprise facilities both rely
on a complex mesh of optical networking infrastructure to address connectivity
requirements. In order to support such differentiated service offerings, optical networks
have incorporated a series of innovations over recent decades, including the
development of lasers, amplifiers, fibers, coherent detection, and digital signal
processing, to name a few.
Continued…
Artificial intelligence (AI) is the science of creating intelligent machines capable of
autonomously making decisions based on their perceived environment. Machine
learning (ML), a branch of AI, enables this learning paradigm. ML may be used to
achieve network-domain goals ranging from laser characterization to erbiumdoped
fiber amplifier (EDFA) equalization, predictive maintenance, and failure localization.
Several optical networking challenges may be partially or fully addressed using ML
paradigms. The key motivations together with underlying application scenarios are
listed below:
Heterogeneity, Reliability, Capacity, Complexity, Quality Assurance, Data Aspects.
Machine Learning
Workflow
Figure 2 depicts a typical workflow of a ML
framework—sometimes referred to as knowledge
discovery in databases (KDD)—focusing on
algorithm development and test cycle.
Initially, ML models are constructed in a training
phase, where data is retrieved from historical
databases and pre-processed to remove or normalize
outliers, handle missing data, filter and aggregate
parameters, etc. The next step relates to data
transformation, selecting relevant data features,
minimizing redundancies, data format adaptation to
the given task, etc.
Continued…
This is a crucial step in the ML workflow, as not all data at hand is necessarily useful in
terms of algorithm performance and complexity—the curse of dimensionality.
The most important steps of model selection and behavior learning are typically carried
out in tandem. The process involves high-dimensional parameter optimizations and
corresponding evaluations to trade off
performance, complexity, and computational effort.
Algorithm
ML approaches may be categorized based on objectives of the learning task, where these objectives
may target pattern identification for classification and prediction, learning for action, or inductive
learning methods.
The algorithms may be classified into three distinct learning families, i.e., supervised learning,
unsupervised learning, and reinforcement learning. Semi-supervised learning—or hybrid learning—is
sometimes considered a fourth branch, borrowing features from the supervised and unsupervised
categories.
Evaluations
The ML algorithm evaluation approach impacts the
way a model is constructed and eventually selected
among several competing options. A poorly defined
evaluation criterion may result in an unoptimized
model selection procedure, resulting in erroneous
conclusions when comparing classifier
performances.
In Fig. 5. It can be seen that as the model
approaches convergence—underfitting, which refers
to an overly simplistic model—the training and
validation data sets show similar results.
Continued…

However, beyond this phase, training errors continue to improve,

whereas validation errors start deteriorating—overfitting, which refers to
an unnecessarily complex model. The model that enables the best
performance for the validation set is selected as the optimum model. B
Data Management
An optical network generates a large amount of heterogeneous data
streams, which must be fetched, processed, and analyzed in a timely
manner to enable everyday network operations. In order to enable data-
driven ML analysis, it is important to explore several aspects of network
data including its extent, monitoring, query mechanisms, storage, and
representation attributes. In this section, we address these data
management features.
 Continued…

A) Sources: In the following, we discuss the five most common sources of ON data
collection:
• Network probes include both intrusive and non-intrusive data access.
• Sensors are the core data sources of optical networks, whose goal is to measure a
physical quantity.
• Network logs include alarm and event data sets from different network devices.
• Control signaling refers to supervisory channel, header data, etc., typically used
for path initialization, link control, channel setup, etc.
• Network management data is primarily focused on two types of data sources:
first, configuration, topology, and connectivity data at different network layers,
and second, monitored data obtained from the network sensors.
Continued…
B) Data Storage and Representation: Due to the
growing size of the network and complexity of the
information sources, the data sources are often
scattered across multiple storage and management
systems with their own peculiar features.
Most enterprises today have dedicated teams of IT
specialists to develop database queries for the
domain experts. This means that the domain
experts always have to pass their information needs
to IT specialists (see Fig. 9), and this can
drastically affect the efficiency of finding the right
data in the right format that can be used for timely
decision-making.
Network And Model Management
A. Architecture
(i) Closed Loop Operation
(ii) Centralized and Distributed
 Continue……….
B. ML Model Life Cycle
After a machine learning (ML) model is built , it needs to be
maintained throughout its lifetime. This involves deciding when
and how to update the model. It is important to figure out the
right time to update the model, balancing the need for accuracy
and minimizing the computational load. Let's break down the
two key aspects of this process:
1.Data Profile: A common method for updating ML models is
to do so at regular intervals, called constant updates. This
makes sure the model keeps up with changes in the network. .
However, this approach can sometimes lead to unnecessary
updates if the network hasn’t changed much.
An alternative approach is to use adaptive updates, where the
model is only updated when there is a significant change in the
data.
2.Reconstruction:
The entire model is rebuilt at each update cycle, using new
data and discarding old data.
 Application Scenarios
1. Predictive Maintenance
As networks become more complex and diverse, there need for
better asset management to minimize downtime and make better use
of resources. Common maintenance issues include:
•Scheduled Maintenance:
•Reactive Monitoring:
•Manual Analysis:
Examples of Network Faults: Common faults include cooling unit
failures, laser degradation, and subsystem control unit failures.
Detecting these issues early can prevent downtime and allow for
planned maintenance.
Fault Detection Approach: The fault management system works in
two stages:
First Stage: Monitors optical power levels to detect potential issues.
1.
Second Stage: Monitors more specific parameters like amplifier
2.
gain, shelf temperature, current draw, and internal optical power.
This two-stage approach helps reduce the amount of monitoring and
processing needed. When a potential fault is detected, the system
uses ML algorithms to analyze the data and identify patterns. This
helps predict and detect possible issues before they cause problems.
 Continue……….
2.Optical Spectrum Analysis

Optical signals often experience filtering effects, such as

laser drifts and narrow spectral grids, leading to Quality of
Transmission (QoT) degradation.
i) Filter Effects: A well-configured optical signal should
have its central frequency aligned with the center of the
allocated spectrum slot. However, when filter failures occur,
the spectrum can become distorted in two ways:
1.Filter Shift (FS): The spectrum becomes asymmetrical
due to misaligned filters with respect to the central
frequency. Fig. Example of filter failures : (a) FS and (b) FT
2.Filter Tightening (FT): The spectrum's edges appear
overly rounded because the filter bandwidth is narrower
than the allocated frequency slot.
In case of a filter shift, the spectrum might shift by 10 GHz,
while filter tightening might cause a 20 GHz distortion.
Continue……….
ii) ML Algorithms for Failure Detection and Identification:
 Extract Signal Features
 Apply Correction Mask
 Classifier
 Magnitude Estimators (FS & FT)

Fig. Workflow for filter failure detection and identification

Future Work And Industrial Perspective:-

While ML has great potential in optical networking, several challenges must be overcome for it
to be widely adopted in the industry.
Standardization: As network hardware becomes more open and compatible, it's important that
ML solutions follow industry rules, no matter who makes the product.
Data Availability: Not having enough good data is a big problem. The industry needs to focus
on creating and sharing safe, anonymous datasets for ML development.
Algorithm Complexity: ML algorithms should balance how complex they are with how much
processing power they need. The field is still growing, so the best solutions might not be
available yet.
Scalability: New systems and software should be built to grow easily, both in terms of apps and
storage
Security: As networks become more self-managed and rely on data, ML must improve to handle
security risks like data access, authentication, and detecting attacks.
Conclusion
Traditional networks face challenges due to static operations, limiting their
scalability and efficiency. Machine learning (ML) offers adaptive techniques to
address these issues, making it a valuable tool for the optical networking industry.
We discuss ML in optical networks, covering key aspects:
ML Paradigms: Introduction to fundamental ML concepts, workflows, and
algorithms relevant to optical networking.
Data Management: Exploration of network data sources, monitoring
frameworks, and data storage challenges.
Network and Model Management: Discussion on advanced architectures
integrating ML with SDN, and strategies for updating deployed models.
ML can significantly enhance optical network design, operation, and
maintenance. Techniques like proactive failure detection and traffic optimization
demonstrate its potential for improving network reliability and resource
utilization.
Thank You