Presented at the 13th International Conference on

Informatics in Control, Automation and Robotics (ICINCO),

July 2016, http://www.icinco.org/

Software-defined Networking for Real-time Ethernet

Jia Lei Du and Matthias Herlich

Salzburg Research, Jakob Haringer Straße 5/3, Salzburg, Austria
{jia.du, matthias.herlich}@salzburgresearch.at

Keywords: Software-defined Networking, Real-time Ethernet

Abstract: Real-time Ethernet is used in many industrial and embedded systems, but has so far mostly been statically
configured. However, in the future these network configurations will be required to change dynamically, for
example for highly flexible production lines or even software upgrades in modern cars that add new features
which require changes to the in-vehicle network. Software-defined networking (SDN) is already
increasingly used to dynamically configure non-real-time networks. In this paper we explore the idea of a
software-defined real-time Ethernet. We analyze the features of current real-time Ethernet protocols, the
applicability of SDN and give an overview of potential advantages of software-defined networking for real-
time communication which can enable features not achievable using current solutions. In the future this
development will likely lead to more flexible, efficient and robust real-time networks.

1 INTRODUCTION centralized intelligent controller with “dumb”

forwarding devices in the data plane of the network
Real-time Ethernet (RTE) allows the use of cost (McKeown et al., 2008). By monitoring network-
effective, widespread and high-bandwidth Ethernet wide state, the controller obtains an up-to-date view
technology in industrial environments like of the network and can dynamically adapt flows as
automation, process control and transportation necessary. The concept of SDN allows a wide range
where one key challenge is real-time of traffic engineering, security and other
communication, i.e. communication with guaranteed applications. For example, flows can be dynamically
upper bounds for latency and latency variations rerouted based on load, failure or security scenarios
(jitter). Various solutions like Ethernet Powerlink, to provide certain bandwidth or latency properties,
VARAN, Profinet and TTEthernet have been fast failover mechanisms or security services. From
developed to extend standard Ethernet with real-time an economic point of view, through standardization
capabilities. and centralization, SDN has the potential to simplify
Typical RTE deployments in the past have been and reduce costs for network setup and operation.
configured once to run without re-configuration for In this paper we will describe our idea to apply
years or even decades. However, in the future RTE software-defined networking in real-time Ethernet
networks will need to be more flexible due to a networks to benefit from SDN advantages while
variety of reasons: To produce small lot sizes in a keeping the deterministic properties of RTE. In
production environment efficiently, the underlying detail we propose replacing the switches/hubs of
network must support quick reconfigurations to real-time Ethernet solutions with SDN-capable
fulfill new requirements (Dürkop, Jasperneite & switches. Note that we do not consider replacing the
Fay, 2015). Or in-vehicle networks could be real-time protocols themselves but to extend RTE
reconfigured through software updates for example protocols by providing additional features that the
when a new driver assistance feature needs a higher use of SDN controllers and switches make possible.
sample rate from a proximity sensor. For this purpose we first describe SDN in the next
In non-real-time networks software-defined section. Then we describe features typical RTE
networking (SDN) is a technology that provides a solutions provide. Finally, we discuss the advantages
great range of freedom to flexibly and centrally and disadvantages of using SDN in an RTE network
reconfigure the network on-demand. The basic idea and give an approach how to validate these claims.
of SDN is to control network flows through a
2 RELATED WORK we will use OpenFlow as example when illustrating
our ideas.
Gopalakrishnan (Gopalakrishnan, 2014) and
Kalman (Kalman, 2014) both consider how SDN
can be used in industrial communication networks.
Gopalakrishnan provides a general list of SDN
features and gives some examples how the
advantages could be applied to an IEC 61850-based
network, but only mentions real-time capabilities in
passing. Kalman focuses on hardware abstractions
and the ability to automatically configure networks
using SDN. While both consider the advantages of
SDN, they do not focus on the specific requirements Figure 1. In SDN the network devices (middle) forward
and advantages SDN can bring to real-time network flows programmed by a SDN controller (top)
networks, but on industrial communication networks between end-devices (left/right). The SDN controller
in general. could also be integrated into one of the end-devices.
(Dürkop, Jasperneite & Fay, 2015) provides a
high-level concept for the automatic configuration of One key risk of an SDN is related to the
real-time Ethernet solutions. Our paper focuses on availability of the controller that is required for
the communication aspect in more detail and configuring the network devices. Both the controller
proposes using SDN as an approach for network itself and the connection between network devices
(re-)configurations. Furthermore, automatic and controller represent possible single points of
(re-)configuration is only one of the advantages we failures and bottlenecks. To mitigate risks of
describe in this paper that software-defined controller unavailability usually the use of multiple
networking can bring to RTE networks. controllers in an SDN is suggested such as in
(Yeganeh & Ganjali, 2012; Jain et al., 2013; Yazici,
Sunay & Ercan, 2014).
3 SOFTWARE-DEFINED
NETWORKING BACKGROUND
4 REAL-TIME ETHERNET
In most conventional communication networks, FEATURES
traffic flows are established based on forwarding
rules that are locally determined using distributed We have exemplarily chosen Ethernet Powerlink
algorithms. In contrast to this approach, traffic flows (Ethernet Powerlink Standardization Group, 2016),
in software-defined networks (SDNs) are centrally Profinet (International Electrotechnical Commission,
configured by network applications via so-called 2014), TTEthernet (SAE Aerospace, 2011),
controllers. This effectively decouples the control VARAN (VARAN Bus User Organization, 2016)
plane, which determines where traffic is sent, from and TSN (Time-Sensitive Networking Task Group,
the data plane, which forwards packets to their 2016) (an upcoming but not yet finalized IEEE
destinations. When a packet that matches a rule standard and the successor of AVB) for
arrives at a network device, the associated actions investigation. Industrial communication protocols
are performed. Possible actions include the like Ethernet/IP are implemented on application
modification of packet headers and the dropping or layer based on TCP/UDP over IP communication
forwarding of packets. Figure 1 illustrates the stacks. Protocols in this category are usually highly
interaction between lower layer SDN forwarding compatible and do not require special hardware or
devices, the SDN controller with its applications, modifications. However, due to the use of the entire
and RTE devices. Internet stack cycle times are generally higher than
One standard for the implementation of those achieved by protocols implemented based on
software defined networks is OpenFlow (Open lower communication layers. The studied protocols
Networking Foundation, 2015). The OpenFlow are instead implemented directly on top of Ethernet
standard defines a communication protocol between and achieve significantly lower cycle times. Thus,
network switches and one or more controllers. The the application of OpenFlow and software-defined
ideas in this paper can be applied to all SDNs, but networking to real-time networks of the second
category is technologically more challenging and broadcast (transmission from one-to-all devices) can
findings and improvements are more likely to be be used to implement a multicast (one-to-some) by
transferable to protocols of the first category. filtering out frames at the devices which are not
Additionally, our selection of protocols covers both intended to receive the frame. A more efficient
time triggered and polling-based protocols as well as method which we call real multicast is to transmit
protocols that use the entire Ethernet stack or only the frame only to the intended receivers in the first
parts of the stack. For an analysis of the features of place. Using multipath routing several paths can
the studied real-time Ethernet protocols with respect deliver data from a source to a destination. This can
to SDN we group the features in the following be used for redundancy or to increase the data rate.
categories. (1) Performance: quantifiable We define concurrency as the ability of two pairs of
measurements about the RTE solutions. (2) senders and receivers to simultaneously
Compatibility: RTE solutions usage of standard communicate. This feature is, for example, easily
Ethernet features. (3) Features relevant for SDN: achieved using switches, but not using hubs.
Specifics of RTE protocols that are relevant for Network topology describes the configuration of
SDN. network devices the RTE solution supports. Hot
The performance of an RTE can be described by plugging is the ability to connect and disconnect
cycle times and data rate. The cycle time is the devices during network operation. Note that it is
duration of one transmission cycle, which is usually necessary to prepare the configuration for devices to
repeated as long as the network is operating. The be hot plugged in advance in some RTE protocols.
cycle time is relevant for applications that need to One of SDN’s main capabilities is the fine-
transmit small amounts of data often. The data rate is grained control of data flows in the network.
the maximum achievable rate of data that can be Therefore, RTE features like broadcasting, real
transmitted over a single link under optimal multicasting, concurrency, arbitrary topologies,
circumstances. The data rate is important for redundancy and multipath routing will be as
applications that want to transmit large amounts of realizable using SDN as using more traditional
data. Both the cycles times and data rates given in networking approaches – SDN will potentially even
Table 1 are for optimal conditions and are not allow a more efficient solution. However, one key
necessarily achievable in practice. limitation needs to be pointed out: Standard SDN
RTE protocols are all based on Ethernet, but use devices currently do not support frame forwarding at
different network modes and some change Ethernet precise points in time and, thus, do not naturally
standard formats. Network mode describes whether support time scheduled protocols. However, adding
the RTE currently uses switches or hubs. All RTE the notion of time does not conceptually contradict
protocols we consider can transmit non-real time the use of SDN and is thus rather an implementation
traffic (for example web traffic) in time slots not issue.
reserved for higher priority traffic. VARAN uses its
own kind of frame, while all other protocols we
analyzed use standard Ethernet frames. 5 ADVANTAGES OF SOFTWARE-
RTE solutions have two basic operating
principles: Time scheduled and polling. In polling a DEFINED REAL-TIME
single master server queries all clients according to ETHERNET
its internal schedule. The clients are only allowed to
transmit data in response to a query by the server. In In this section we discuss the advantages of applying
a time scheduled network a pre-defined schedule is software-defined networking to the design and
shared by all devices. The schedule describes which implementation of real-time Ethernet. Some
device is allowed to transmit at which time. While described features may already be supported or
both time scheduled and polling based RTEs could be implemented with sufficient effort in
typically use a schedule, in polling the schedule is existing solutions. However, even in those cases the
known only to the server and can be changed use of SDN would still provide the advantage of
dynamically more easily. To use a distributed being able to use an existing, consistent framework
schedule precise time synchronization is necessary. to implement all described advantages in a simpler
In case of link failures (such as cable breaks) way. Additionally, SDN enables features (e.g. the
some RTEs offer redundancy features, which active use of network loops) that are not possible in
automatically use alternate links to transmit the data existing solutions.
and thereby hide the failure from the application. A
5.1 Advantages not related to path Global network information: OpenFlow-
selection compatible network devices can collect a many
usage statistics such as the number of received/sent
Central configuration: Centralized software-based frames/bytes per flow/port/queue. This information
(re-)configuration of network devices is a key can help with error diagnostics and
feature of SDN. It enables centrally controlled performance/traffic pattern evaluations. This feature
configuration of network nodes both with regard to is more valuable for real-time Ethernet networks in
device settings and communication patterns (this which the RTE controller does not already have a
advantage has also been named “Flow Engineering” comprehensive overview of most or even all
(Gopalakrishnan, 2014) or “Central Resource communication.
Management” (Kalman, 2014)). In difference to
current RTE solutions where device settings and 5.2 Advantages related to
communication patterns are often configured once switching/routing/path selection
during design, using an SDN approach device
settings and communication paths and schedules can Central addition and removal of network nodes:
be adapted on-the-fly with little or no disruption. Based on OpenFlow network nodes can be
From an application point of view a different dynamically added to or removed from the real-time
production objective in a factory or a new feature in network at network level and removed nodes would
an autonomous vehicle could be activated through a no longer receive messages. Using this feature
software update even if the requirements towards the machines, sensors or actuators can, for example, be
underlying RTE network changes, for example dynamically recombined to fulfill different tasks.
because certain sensor data is required at a higher Arbitrary topology: Currently existing protocols
rate or from a different set of connected sensors. usually support only standard Ethernet topologies
Standardization: First, OpenFlow defines a set of and do not permit the existence of loops on network
functionalities that all OF compatible network level and algorithms like spanning trees protocols
devices must fulfill and a standard interface to are used to block redundant paths. Due to the central
access these functions (also mentioned as “open configuration of communication paths the existence
standards-based and vendor-neutral” in (Kalman, of loops does not pose a problem for SDN and
2014)). Second, as the intelligence is mostly located arbitrary network topologies can even be actively
in the centralized controller, the network devices are exploited.
comparatively simple. These two properties lead to Fast reroute and failover: Additional links in the
simple, exchangeable, inexpensive, and future-proof network can be used as backup routes in case of
network devices (except the SDN controller). failures in the network. This feature can be more

Table 1: A comparison of real-time Ethernet protocols and features and their relevance for SDN

TSN Profinet TTEthernet Powerlink VARAN

Performance
Min. cycle times 30 μs + 31.25 μs <100 μs <100 μs <100 μs
Max. data rate 1 Gbit/s + 100 Mbit/s 1 Gbit/s 100 Mbit/s 100 Mbit/s
Compatibility
Network devices Switches Switches Switches Hubs Hubs
Non-RT traffic Yes Yes Yes Yes Yes
Ethernet frames Yes Yes Yes Yes No
SDN relevant
Operating principle Time schedule Time schedule Time schedule Polling Polling
Redundancy Yes Ring/multi- Dual and triple Ring and dual No
controller
Real multicast Yes No ? No No
Broadcast Yes Possible/not used Yes Yes Master to slaves
Multipath routing Yes No Yes No No
Concurrency Yes Yes Yes No No
Topologies Arbitrary Line, tree, star, ring Line, tree, star, Line, tree, star, Line, tree, star
ring ring
Hot plugging Yes Yes ? Yes Yes
easily implemented for polling-based RTE protocols Even a selective isolation of a node is possible:
which often use broadcasts. In case of link failures, correct frames are allowed to pass and only incorrect
frames can be rerouted (Pfeiffenberger et al., 2015) frames that are sent at the wrong time or to wrong
transparently for end nodes as long as the frames destinations are blocked.
arrive in time. For time-scheduled protocols the Dynamic load balancing: Dynamic load-
schedules in the network devices might have to be balancing allows the dynamic change of
adapted after an incident to avoid congestion in the communication paths and/or the simultaneous use of
backup paths. For zero-loss/zero-time failover, flows multiple communication paths between a sender and
can be duplicated on the network layer and delivered a receiver as a function of network load. Within the
via two distinct paths. scope of this paper/project we use the term only in
Multiple simultaneous communication paths: the context of asynchronous traffic which potentially
Additionally available network links cannot only be has more volatile communication patterns that are
used as backups in case of failures but also to not known beforehand but less strict latency
increase available bandwidth during normal requirements compared to isochronous traffic.
operation. Even multipath routing is imaginable, that Efficient multicasting: When delivering multicast
is, splitting up and delivering flows via multiple traffic using OpenFlow, it is comparatively
paths. straightforward to avoid sending frames over a link
Multiple networks over one infrastructure: An if there is no subscriber of that multicast traffic at
OpenFlow-/SDN-based approach to RTE networks the other end of the link. In difference to standard
could enable or simplify the operation of multiple Ethernet implementations where multicast frames
real-time Ethernet networks over a single physical are actually broadcasted in the network, this can
infrastructure, for example, in the most simple case both be a security benefit and to save bandwidth.
by reserving half of the time for network 1 and half More efficient bandwidth usage through efficient
of the time for network 2. The devices in the two multicasting is possible for real-time Ethernet
networks would never receive messages from the protocols which allow multiple parallel
other network and thus this sharing of the physical communication flows. And protocols that only allow
infrastructure could be completely transparent to the one sender in the network at any time would still
participating devices. Such a setup may require benefit from a security point of view as nodes that
some form of time synchronization between devices are not subscribers of the multicast traffic would not
in the two networks which could take place in a third receive any of those frames.
virtual network. This feature could be highly
attractive for many polling-based protocols as such
an operation can currently not be supported (due to 6 DISADVANTAGES AND
the use of broadcasting for communication). For
some time-based protocols like TTEthernet such a POTENTIAL PROBLEMS
behavior could already be supported conceptually
but the use of SDN would still significantly simplify Implementation of RTE using current SDN
the implementation by guaranteeing safety technology: The most important feature a RTE
properties (e.g. nodes in network 1 will never see network has to implement is the deterministic
messages from nodes in network 2) similar to a guarantee of traffic latency. To make these
virtualization layer in computing. guarantees usually polling or predefined
Isolation of faulty nodes: Using OpenFlow faulty communication schedules are used. If the creation of
network nodes can be easily disconnected from the a polling-based software-defined RTE network was
network in the sense that messages of faulty nodes the goal, hubs would have to be replaced with
can be simply dropped at the closest functioning switches. To implement a software-defined RTE
network node. The isolation of faulty network nodes network based on predefined schedules the SDN
consists of two separate problems: The detection of switches would additionally need to have a notion of
faulty behavior through the RTE and/or SDN time and schedules. While we do not know any
controller and the disconnection of the faulty node conceptual reason which would prevent the support
through the SDN controller. Detection of very basic of time schedules in SDN switches, we are not
faults can, for example, be done through simple aware of any standard SDN switches which support
SDN-based frame counting. For the detection of schedules. Additionally, when low cycle times are
complex faults the cooperation between RTE required, the performance guarantees depend on the
controller and SDN controller is likely necessary. achievable forwarding latency and jitter of SDN
switches. It is necessary to measure the performance disadvantages of the application of SDN approaches
of SDN switches and compare it to current Ethernet to RTE networks and described how we plan to
switches and hubs used for RTE. Finally, SDN in demonstrate the advantages in practice. We conclude
general does not dependent on the use of Ethernet- that the development of a software-defined real-time
compatible frames, however current OpenFlow- Ethernet is a highly promising endeavor and are in
compatible switches do pose that requirement. the process of validating our concepts in a test
Disadvantages introduced by SDN: One key network.
disadvantage of SDN is the need for a controller. (This work was partially funded by the Austrian
Such a controller is a single point of failure (if not Federal Ministry for Transport, Innovation and
replicated, see section II) and a controller failure Technology in the project OpenheaRTEd, FFG No.
would disable further central network 849972.)
configurations. However, this shortcoming prevents
only use cases in which it is necessary to reconfigure
the network while deterministic traffic is transferred REFERENCES
over the network. In all other cases, the guaranteed
performance would not be affected even if the SDN Dürkop, L., Jasperneite, J., Fay, A., 2015. An Analysis of
controller failed, only reconfiguration would be Real-Time Ethernets With Regard to Their Automatic
disabled. Configuration. In IEEE World Conference on Factory
Communication Systems (WFCS).
Ethernet Powerlink Standardization Group, 2016. Ethernet
Powerlink Communication Profile Specification.
7 VALIDATION CONCEPT Version 1.3.0.
Gopalakrishnan, A., 2014. Applications of Software
We are currently developing a proof-of-concept Defined Networks in Industrial Automation.
based on openPowerlink and SDN switches. International Electrotechnical Commission, 2014.
openPowerlink is an open source implementation of Additional fieldbus profiles for real-time networks
the Powerlink real-time Ethernet protocol. A real- based on ISO/IEC 8802-3. IEC Standard 61784-
time Ethernet network with a cycle time of 1 ms has 2:2014, section CPF3.
been built based on openPowerlink and OpenFlow- Jain, S., Kumar, A., Mandal, S. et al., 2013. B4:
Experience with a globally-deployed software defined
capable switches in our test lab. We are currently in WAN. ACM SIGCOMM Computer Communication
the process of implementing key use cases to Review, vol. 43, no. 4, pp. 3-14.
demonstrate some of the advantages described in Kalman, G., 2014. Applicability of Software Defined
this paper. Particular emphasis is put on Networking in industrial Ethernet. In IEEE
demonstrating use cases which can be easily Telecommunications Forum (TELFOR).
implemented using SDN but would be complex or McKeown, N., Anderson, T., Balakrishnan, H. et al.,
impossible to implement using current standard RTE 2008. OpenFlow: enabling innovation in campus
technologies. Finally, we focused on network level networks. ACM SIGCOMM Computer Communication
reconfigurations in this paper. However for the Review, vol. 38, no. 2, pp. 69-74.
Open Networking Foundation, 2015. OpenFlow Switch
implementation of some of the described Specification Version 1.5.1.
advantages, a tight integration and interaction with Pfeiffenberger, T., Du, J. L., Bittencourt, P., et al., 2015.
the respective RTE protocol would be necessary Reliable and Flexible Com. for Power Systems: Fault-
(e.g. to distribute new time schedules to the network tolerant Multicast with SDN/OpenFlow. In 7th IFIP
devices). Thus, the long-term goal is to develop a Conf. on New Technologies, Mobility and Security.
complete software-defined real-time Ethernet SAE Aerospace, 2011. Time-Triggered Ethernet. SAE
solution in which the OpenFlow controller is Aerospace Standard AS 6802.
integrated in the RTE devices and seamlessly Time-Sensitive Networking Task Group, 2016.
interacts with the RTE protocols and its features. http://www.ieee802.org/1/ pages/tsn.html
VARAN Bus User Organization, 2016. “VARAN Real-
Time Ethernet”.
Yazici, V., Sunay, M. O., Ercan, A. O., 2014. Controlling
8 CONCLUSION a software-defined network via distributed controllers.
arXiv preprint, arXiv:1401.7651.
Yeganeh, S. H., Ganjali, Y., 2012. Kandoo: a framework
We first described software-defined networking and
for efficient and scalable offloading of control
features of real-time Ethernet solutions from a SDN applications, Workshop on Hot Topics in Software
point of view. Then we analyzed the advantages and Defined Networks.