UNIT-4

Implementation of IoT with Raspberry Pi

Introduction to Software defined Network (SDN), SDN for IoT, Data Handling and
Analytics.
Software defined networking (SDN)

Software defined networking (SDN) is an approach to network management that enables

dynamic, programmatically efficient network configuration to improve network performance and
monitoring. It is a new way of managing computer networks that makes them easier and more
flexible to control.

SDN Architecture
In traditional networks, the hardware (like routers and switches) decides how data moves through
the network, but SDN changes this by moving the decision-making to a central software system.
This is done by separating the control plane (which decides where traffic is sent) from the data
plane (which moves packets to the selected destination).
Data Plane:
All the activities involving and resulting from data packets sent by the end-user belong to this
plane. Data Plane includes:
 Forwarding of packets.
 Segmentation and reassembly of data.
 Replication of packets for multicasting.
Control Plane:
All activities necessary to perform data plane activities but do not involve end-user data packets
belong to this plane. In other words, this is the brain of the network. The activities of the control
plane include:
 Making routing tables.
 Setting packet handling policies.
Components of Software Defining Networking (SDN)
The three main components that make the SDN are:
 SDN APIs: SDN Applications relay requests or networks through SDN Controller using API.
 SDN Controller : SDN Controller collects network information from hardware and sends
this information to applications.
 SDN Networking Devices: SDN Network devices help in forwarding and data processing
tasks.

FIG:Software Defined Network

In a traditional network, each switch has its own control plane and data plane. Switches
exchange topology information to build a forwarding table that decides where to send data
packets. In Software-Defined Networking (SDN), the control plane is removed from switches
and assigned to a centralized SDN controller. This allows network administrators to manage
traffic from a single console instead of configuring each switch individually.
The data plane remains in the switch, forwarding packets based on flow tables set by the
controller. These tables contain match fields (like input port and packet header) and instructions
(forward, drop, or modify packets). If a packet doesn’t match any entry, the switch contacts the
controller, which provides a new flow entry to decide the packet’s path.
A typical SDN architecture consists of three layers.
 FIG:SDN LAYERS

 Application Layer: It contains the typical network applications like intrusion

detection, firewall, and load balancing.
 Control Layer: It consists of the SDN controller which acts as the brain of the network. It
also allows hardware abstraction to the applications written on top of it.
 Infrastructure Layer: This consists of physical switches which form the data plane and
carries out the actual movement of data packets.
The layers communicate via a set of interfaces called the north-bound APIs(between the
application and control layer) and southbound APIs(between the control and infrastructure
layer).

Northbound APIs (NBAs):

These APIs provide a standard interface for applications, services, and orchestration systems to
interact with the SDN controller. They allow applications to program the network, control
network resources, and access network information. Think of them as the link between
applications and the SDN controller. RESTful APIs are commonly used for NBAs.

Southbound APIs (SBAs):

These APIs enable communication between the SDN controller and the network devices
(switches, routers, etc.). They allow the controller to manage the network infrastructure,
configure devices, and enforce policies. OpenFlow and NETCONF are common SBAs used in
many SDN implementations.
Where is Software Defined Networking Used?
 Enterprises use SDN, the most widely used method for application deployment, to deploy
applications faster while lowering overall deployment and operating costs. SDN allows IT
administrators to manage and provision network services from a single location.
 Cloud networking software-defined uses white-box systems. Cloud providers often use
generic hardware so that the Cloud data center can be changed and the cost of CAPEX and
OPEX saved.
Why Software Defined Networking is Important?
 Better Network Connectivity: SDN provides very better network connectivity for sales,
services, and internal communications. SDN also helps in faster data sharing.
 Better Deployment of Applications: Deployment of new applications, services, and many
business models can be speed up using Software Defined Networking.
 Better Security: Software-defined network provides better visibility throughout the network.
Operators can create separate zones for devices that require different levels of security. SDN
networks give more freedom to operators.
 Better Control With High Speed: Software-defined networking provides better speed than
other networking types by applying an open standard software-based controller.
How Does Software-Defined Networking (SDN) Works?
In Software-Defined Networking (SDN), the software that controls the network is separated from
the hardware. SDN moves the part that decides where to send data (control plane) to software,
while the part that actually forwards the data (data plane) stays in the hardware.
This setup allows network administrators to manage and control the entire network using a
single, unified interface. Instead of configuring each device individually, they can program and
adjust the network from one central place. This makes managing the network much easier and
more efficient.
In a network, physical or virtual devices move data from one place to another. Sometimes,
virtual switches, which can be part of either software or hardware, take over the jobs of physical
switches. These virtual switches combine multiple functions into one smart switch. They check
the data packets and their destinations to make sure everything is correct, then move the packets
to where they need to go.
TYPES of SDN MODELS
There are several models, which are used in SDN:
 Open SDN
 SDN via APIs
 SDN via Hypervisor-based Overlay Network
 Hybrid SDN
1) Open SDN
Open SDN is implemented using the OpenFlow switch. It is a straight forward implementation
of SDN. In Open SDN, the controller communicates with the switches using south-bound API
with the help of OpenFlow protocol.

2) SDN via APIs

In SDN via API, the functions in remote devices like switches are invoked using conventional
methods like SNMP or CLI or through newer methods like Rest API. Here, the devices are
provided with control points enabling the controller to manipulate the remote devices using
APIs.
3) SDN via Hypervisor-based Overlay Network
In SDN via the hypervisor, the configuration of physical devices is unchanged. Instead,
Hypervisor based overlay networks are created over the physical network. Only the devices at
the edge of the physical network are connected to the virtualized networks, thereby concealing
the information of other devices in the physical network.
 4) Hybrid SDN
Hybrid Networking is a combination of Traditional Networking with software-defined
networking in one network to support different types of functions on a networ

Conventional Network Architecture

Traditional network refers to the old conventional way of networking which uses fixed and
dedicated hardware devices such as routers and switches to control network traffic. Inability to
scale and network security and Performance are the major concern now a days in the current
growing business situation so that SDN is taking control to traditional network. Traditional
network is static and based on hardware network appliances. Traditional network architecture
was used by many companies till recent years but now a days due to its drawbacks Software
Defined Network has been developed and in coming years it will be used more.
Components of Traditional Network

Network devices: Traditional networks use physical network devices, such as routers, switches,
and firewalls, to manage and direct network traffic.

Cabling: Traditional networks use physical cabling to connect network devices to each other.

Protocols: Traditional networks rely on standard networking protocols, such as TCP/IP and
Ethernet, for communication between network devices.

Advantages of Traditional Network

Well-established: Traditional networks are well-established and widely used in various

organizations.

Predictable performance: Traditional networks offer predictable performance as network devices

are configured based on specific requirements

Familiarity: Traditional networks are familiar to network administrators and require minimal
training.
Disadvantages of Traditional Network

Limited scalability: Traditional networks have limited scalability due to the dependence on
physical hardware devices.

Limited automation: Traditional networks have limited automation capabilities and require
significant manual intervention.

Rigid architecture: Traditional networks have a rigid, hierarchical architecture that is difficult to
modify or adapt to changing business needs.

DIFFERENCES BETWEEN TRADITIONAL NETWORKS AND SDN

Network Functions Virtualization

The term “Network Functions Virtualization” (NFV) refers to the use of virtual machines in
place of physical network appliances. There is a requirement for a hypervisor to operate
networking software and procedures like load balancing and routing by virtual computers. A
network functions virtualization standard was first proposed at the OpenFlow World Congress in
2012 by the European Telecommunications Standards Institute (ETSI), a group of service
providers that includes AT&T, China Mobile, BT Group, Deutsche Telekom, and many more.

Need of NFV:

With the help of NFV, it becomes possible to separate communication services from specialized
hardware like routers and firewalls. This eliminates the need for buying new hardware and
network operations can offer new services on demand. With this, it is possible to deploy network
components in a matter of hours as opposed to months as with conventional networking.
Furthermore, the virtualized services can run on less expensive generic servers.

Advantages:

Lower expenses as it follows Pay as you go which implies companies only pay for what they
require.

Less equipment as it works on virtual machines rather than actual machines which leads to fewer
appliances, which lowers operating expenses as well.

Scalability of network architecture is quite quick and simple using virtual functions in NFV. As a
result, it does not call for the purchase of more hardware.

Working:

Usage of software by virtual machines enables to carry out the same networking tasks as
conventional hardware. The software handles the task of load balancing, routing, and firewall
security. Network engineers can automate the provisioning of the virtual network and program
all of its various components using a hypervisor or software-defined networking controller.

Structure of NFV

Benefits of NFV:
Many service providers believe that advantages outweigh the issues of NFV.

1) Traditional hardware-based networks are time-consuming as these require network

administrators to buy specialized hardware units, manually configure them, then join them to
form a network. For this skilled or well-equipped worker is required.

2)It costs less as it works under the management of a hypervisor, which is significantly less
expensive than buying specialized hardware that serves the same purpose.

3) Easy to configure and administer the network because of a virtualized network. As a result,
network capabilities may be updated or added instantly.

Risks of NFV:

Security hazards do exist, though, and network functions virtualization security issues have
shown to be a barrier to widespread adoption among telecom companies. The following are some
dangers associated with implementing network function virtualization that service providers
should take into account:

Physical security measures do not work: Comparing virtualized network components to locked-
down physical equipment in a data center enhances their susceptibility to new types of assaults.

Malware is difficult to isolate and contain: Malware travels more easily among virtual
components running on the same virtual computer than between hardware components that can
be isolated or physically separated.

Network activity is less visible: Because traditional traffic monitoring tools struggle to detect
potentially malicious anomalies in network traffic going east-west between virtual machines,
NFV necessitates more fine-grained security solutions.

NFV Architecture:

An individual proprietary hardware component, such as a router, switch, gateway, firewall, load
balancer, or intrusion detection system, performs a specific networking function in a typical
network architecture. A virtualized network substitutes software programs that operate on virtual
machines for these pieces of hardware to carry out networking operations.
Three components make up an NFV architecture:

Centralized virtual network infrastructure: The foundation of an NFV infrastructure can be either
a platform for managing containers or a hypervisor that abstracts the resources for computation,
storage, and networking.

Applications: Software delivers many forms of network functionality by substituting for the
hardware elements of a conventional network design (virtualized network functions).

Framework: To manage the infrastructure and provide network functionality, a framework is

required (commonly abbreviated as MANO, meaning Management, Automation, and Network
Orchestration).

Data Handling & analytics

IoT Data and BigData

The rise of future internet technologies, including cloud computing and BigData analytics,
enables the wider deployment and use of sophisticated IoT analytics applications, beyond simple
sensor processing applications. It is therefore no accident that IoT technologies are converging
with cloud computing and BigData analytics technologies towards creating and deploying
advanced applications that process IoT streams. The integration of IoT data streams within cloud
computing infrastructures enables IoT analytics applications to benefit from the capacity,
performance and scalability of cloud computing infrastructures. In several cases, IoT analytics
applications are also integrated with edge computing infrastructures, which decentralize
processing of IoT data streams at the very edge of the network, while transferring only selected
IoT data from the edge devices to the cloud. Therefore, it is very common to deploy IoT
analytics applications within edge and/or cloud computing infrastructures.

• Volume: IoT data sources (such as sensors) produce in most cases very large volumes of data,
which typically exceed the storage and processing capabilities of conventional database systems.

• Velocity: IoT data streams have commonly very high ingestion rates, as they are produced
continually, in very high frequencies and in several times in very short timescales.
• Variety: Due to the large diversity of IoT devices, IoT data sources can be very heterogeneous
both in terms of semantics and data formats.

• Veracity: IoT data are a classical example of noise data, which are characterized by
uncertainty. Therefore, systems, tools and techniques

IoT Analytics Lifecycle and Techniques

The IoT analytics lifecycle comprises the phases of data collection, analysis and reuse. In
particular:

• 1st Phase – IoT Data Collection: As part of this phase IoT data are collected and enriched with
the proper contextual metadata, such as location information and timestamps. Moreover, the data
are validated in terms of their format and source of origin. Also, they are validated in terms of
their integrity, accuracy and consistency. Hence, this phase addresses several IoT analytics
challenges, such as the need to ensure consistency and quality. Note that IoT data collection
presents several peculiarities, when compared to traditional data consolidation of distributed
data sources, such as the need to deal with heterogeneous IoT streams.

• 2nd Phase – IoT Data Analysis:

This phase deals with the structuring, storage and ultimate analysis of IoT data streams. The
latter analysis involves the employment of data mining and machine learning techniques such as
classification, clustering and rules mining. These techniques are typically used to transform IoT
data to actionable knowledge.

• 3rd Phase – IoT Data Deployment, Operationalization and Reuse:

As part of this phase, the IoT analytics techniques identified in the previous steps are actually
deployed, thus becoming operational. This phase ensures also the visualization of the IoT
data/knowledge according to the needs of the application. Moreover, it enables the reuse of IoT
knowledge and datasets across different applications.

Characteristics of IoT Generated Data

The volume and quality of the data generated by IoT devices is very different from the traditional
transaction-oriented business data. Coming from millions of sensors and sensor-enabled devices,
IoT data is more dynamic, heterogeneous, imperfect, unprocessed, unstructured and real-time
than typical business data. It demands more sophisticated, IoT-specific analytics to make it
meaningful.
As illustrated in Figure, the BigData is defined by 4 “Vs”, which are Volume, Velocity, Variety
and Veracity. The first V is for a large volume of data, not gigabytes but rather thousands of
terabytes. The second V is referencing data streams and real-time processing. The third V is
referencing the heterogeneity of the data: structure and unstructured, diverse data models, query
language, and data sources.The fourth V is defining the data uncertainty, which can be due to
data inconsistency, incompleteness, ambiguities, latency and lack of precise model. The IoT
faces all 4 Vs of the BigData challenges. However the velocity is the main challenge: we need to
process in real-time the data coming from IoT devices. For example, medical wearable such as
Electro Cardio Graphic sensors produce up to 1000 events per second, which is a challenge for
real-time processing. The volume of data is another important challenge. For example General
Electric gathers each day 50 million pieces of data from 10 million sensors. A wearable sensor
produces about 55 million data points per day. In addition, IoT also faces verity and veracity
BigData challenges.

Data Analytic Techniques and Technologies

A cloud-based IoT analytics platform provides IoT-specific analytics that reduce the time, cost
and required expertise to develop analytics-rich, vertical IoT applications. Platform’s IoT-
specific analytics uncover insights, create new information, monitor complex environments,
make accurate predictions, and optimize business processes and operations. The applications of
the IoT BigData Platform can be classified into four main categories

i) deep understanding and insight knowledge

ii) Real time actionable insight

iii) Performance optimization and

iv) proactive and predictive applications.

In the following we provide various technologies allowing building IoT analytics platform.

1 )Batch Processing

Batch processing supposes that the data to be treated is present in a database. The most widely
used tool for the case is Hadoop MapReduce. MapReduce is a programming model and Hadoop
an implementation, allowing processing large data sets with a parallel, distributed algorithm on a
cluster. It can run on inexpensive hardware, lowering the cost of a computing cluster. The latest
version of MapReduce is YARN, called also MapReduce 2.0. Pig provides a higher level of
programming, on top of MapReduce. It has its own language, PigLatin, similar to SQL. Pig
Engine parses, optimizes and automatically executes PigLatin scripts as a series of MapReduce
jobs on a Hadoop cluster. Apache Spark is a fast and general-purpose cluster computing system.
It provides high-levelAPIs in Java, Scala, Python and R, and an optimized engine that supports
general execution graphs. It can be up to a hundred times faster than MapReduce with its
capacity to work in-memory, allowing keeping large working datasets in memory between jobs,
reducing considerably the latency. It supports batch and stream processing.

2) Stream Processing

Stream processing is a computer programming paradigm, equivalent to dataflow programming

and reactive programming, which allows some applications to more easily exploit a limited form
of parallel processing. Flink is a streaming dataflow engine that provides data distribution,
communication and fault tolerance. It has almost no latency as the data are streamed in real-time
(row by row). It runs on YARN and works with its own extended version of MapReduce.

3) Machine Learning

Machine learning is the field of study that gives computers the ability to learn without being
explicitly programmed. It is especially useful in the context of IoT when some properties of the
data collected need to be discovered automatically. Apache Spark comes with its own machine
learning library, called MLib. It consists ofcommonlearning algorithms and utilities, including
classification, regression, clustering, collaborative filtering, dimensionality reduction.
Algorithms can be grouped in 3 domains of actions: Classification, association and clustering. To
choose an algorithm, different parameters must be considered: scalability, robustness,
transparency and proportionality. KNIME is an analytic platform that allows the user to process
the data in a userfriendly graphical interface. It allows training of models and evaluation of
different machine learning algorithms rapidly. If the workflow is already deployed on Hadoop,
Mahout, a machine learning library can be used. Spark also has his own machine learning library
called MLib. H20 is a software dedicated to machine-learning, which can be deployed on
Hadoop and Spark. It has an easy to use Web interface, which makes possible to combine
BigData analytics easily with machine learning algorithm to train models.

4) Data Visualisation

Freeboard offers simple dashboards, which are readily useable sets of widgets able to display
data. There is a direct Orion Fiware connector. Freeboard offers a REST API allowing
controlling of the displays. Tableau Public is a free service that lets anyone publish interactive
data to the web. Once on the web, anyone can interact with the data, download it, or create their
own visualizations of it. No programming skills are required. Tableau allows the upload of
analysed data from .csv format, for instance. The visualisation tool is very powerful and allows
a deep exploration the data. Kibana is an open source analytics and visualization platform
designed to work with Elasticsearch. Kibana allows searching, viewing, and interacting with
data stored in Elasticsearch indices. It can perform advanced data analysis and visualize data in
a variety of charts, tables, and maps.
BIG DATA VALUE CHAIN

Hadoop ecosystem

Hadoop ecosystem is a collection of open-source software tools that enable the storage,
processing, and analysis of large datasets, often referred to as big data. It's not a single tool, but
rather a framework built around core components like HDFS (storage), YARN (resource
management), and MapReduce (processing), along with a suite of supporting tools like Hive,
Pig, HBase, and Spark. This modular design allows for flexible and scalable solutions for diverse
big data workloads.

Core Components:
Hadoop Distributed File System (HDFS):

This is the foundation of Hadoop, providing a distributed file system for storing vast amounts of
structured, semi-structured, and unstructured data across multiple nodes. It's designed to handle
large files and provides fault tolerance and high availability.

Yet Another Resource Negotiator (YARN):

YARN manages and allocates resources within a Hadoop cluster, enabling efficient execution of
user applications. It handles scheduling and resource allocation across the Hadoop system.

MapReduce:

This is a programming model for processing large datasets in parallel across multiple nodes. It
divides large tasks into smaller, manageable chunks, processes them on different nodes, and then
combines the results.

Supporting Tools:

Apache Hive: A data warehouse system built on top of Hadoop that provides a SQL-like
interface for querying data stored in HDFS.

Apache Pig: A high-level platform for analyzing large datasets that simplifies data processing
tasks through a scripting language called Pig Latin.

Apache HBase: A NoSQL database that runs on top of HDFS, offering random, real-time
read/write access to large datasets.

Apache Spark: An in-memory data processing engine that is faster than MapReduce for certain
types of workloads, especially those involving iterative processing.

Apache Kafka: A distributed streaming platform for building real-time data pipelines.

Apache Sqoop: A tool for transferring data between Hadoop and relational databases.

Apache Flume: A system for collecting, aggregating, and moving large amounts of streaming
data into HDFS.
Benefits of using the Hadoop Ecosystem:

Scalability: Hadoop can easily scale to handle massive datasets by adding more nodes to the
cluster.

Cost-effectiveness: Hadoop can run on commodity hardware, reducing the cost of infrastructure.

Fault tolerance: HDFS replicates data across multiple nodes, ensuring data availability even if
some nodes fail.

Flexibility: The ecosystem offers a wide range of tools for different data processing and analysis
needs.

Open source: Being open source, Hadoop is free to use and customize.

The Hadoop ecosystem provides a powerful and versatile platform for big data management,
enabling organizations to extract valuable insights from their data and drive business value.

EXAMPLES OF BIG DATA ANALYTICS

Let us consider several examples of companies that are using big data analytics. The examples
illustrate the use of different sources of big data and the different kinds of analytics that can be
performed.

Introducing a New Coffee Product at Starbucks

Starbucks was introducing a new coffee product but was concerned that customers would find
its taste too strong. The morning that the coffee was rolled out, Starbucks monitored blogs,
Twitter, and niche coffee forum discussion groups to assess customers’ reactions. By mid-
morning, Starbucks discovered that although people liked the taste of the coffee, they thought
that it was too expensive. Starbucks lowered the price, and by the end of the day all of the
negative comments had disappeared. Compare this fast response with a more traditional
approach of waiting for the sales reports to come in and noticing that sales are disappointing. A
next step might be to run a focus group to discover why. Perhaps in several weeks Starbucks
would have discovered the reason and responded by lowering the price.

Drilling for Oil at Chevron

Each drilling miss in the Gulf of Mexico costs Chevron upwards of $100 million. To improve its
chances of finding oil, Chevron analyzes 50 terabytes of seismic data. Even with this, the odds of
finding oil have been around 1 in 5. In the summer of 2010, because of BP’s Gulf oil spill, the
federal government suspended all deep water drilling permits. The geologists at Chevron took
this time to seize the opportunity offered by advances in computing power and storage capacity
to refine their already advanced computer models. With these enhancements, Chevron has
improved the odds of drilling a successful well to nearly 1 in 3, resulting in tremendous cost
savings. Monitoring Trucks at U.S. Xpress U.S. Xpress is a transportation company. Its cabs
continuously stream more than 900 pieces of data related to the condition of the trucks and their
locations [Watson and Leonard, 2011]. This data is stored in the cloud and analyzed in various
ways, with information delivered to various users, from drivers to senior executives, on iPads
and other tablet computers. For example, when a sensor shows that a truck is low on fuel, the
driver is directed to a filling station where the price is low. If a truck appears to need
maintenance, drivers are sent to a specific service depot. Routes and destinations are changed to
ensure that orders are delivered on time.

Types of Big Data Analytics

Prescriptive Analytics

The most valuable and most underused big data analytics technique, prescriptive analytics gives
you a laser-like focus to answer a specific question. It helps to determine the best solution among
a variety of choices, given the known parameters and suggests options for how to take advantage
of a future opportunity or mitigate a future risk. It can also illustrate the implications of each
decision to improve decision-making.

Examples of prescriptive analytics for customer retention include next best action and

next best offer analysis.

Forward looking

Focused on optimal decisions for future situations

Simple rules to complex models that are applied on an automated or

programmatic basis

Discrete prediction of individual data set members based on similarities and

differences

Optimization and decision rules for future events

Diagnostic Analytics

Data scientists turn to this technique when trying to determine why something happened. It is
useful when researching leading churn indicators and usage trends amongst your most loyal
customers. Examples of diagnostic analytics include churn reason analysis and customer health
score analysis. Key points:

Backward looking

Focused on causal relationships and sequences

Relative ranking of dimensions/variable based on inferred explanatory power)

Target/dependent variable with independent variables/dimensions

Includes both frequentist and Bayesian causal inferential analyses

Descriptive Analytics

This technique is the most time-intensive and often produces the least value; however,

it is useful for uncovering patterns within a certain segment of customers. Descriptive

analytics provide insight into what has happened historically and will provide you with

trends to dig into in more detail. Examples of descriptive analytics include summary

statistics, clustering and association rules used in market basket analysis. Key points:

Backward looking

Focused on descriptions and comparisons

 Pattern detection and descriptions

MECE (mutually exclusive and collectively exhaustive) categorization

Category development based on similarities and differences (segmentation)

Predictive Analytics

The most commonly used technique; predictive analytics use models to forecast what

might happen in specific scenarios. Examples of predictive analytics include next best

offers, churn risk and renewal risk analysis.

Forward looking

Focused on non-discrete predictions of future states, relationship, and patterns

Description of prediction result set probability distributions and likelihoods

Model application

Non-discrete forecasting (forecasts communicated in probability distributions)

Outcome Analytics

Also referred to as consumption analytics, this technique provides insight into customer

behavior that drives specific outcomes. This analysis is meant to help you know your

customers better and learn how they are interacting with your products and services.

Backward looking, Real-time and Forward looking

Focused on consumption patterns and associated business outcomes

Description of usage thresholds

Model application