INT-path: Towards Optimal Path Planning for

In-band Network-Wide Telemetry

Tian Pan† , Enge Song† , Zizheng Bian† , Xingchen Lin†
Xiaoyu Peng† , Jiao Zhang† , Tao Huang† , Bin Liu‡ , Yunjie Liu†
†
State Key Laboratory of Networking and Switching Technology, BUPT, Beijing 100876, China
‡
Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
{pan, songenge, imagine, linxingchen, pengxy, jiaozhang, htao, liuyj}@bupt.edu.cn, [email protected]

Abstract—With the ever-increasing complexity of networks, from the control plane CPU. Typically, INT relies on a probe
fine-grained network monitoring enables better network reliabil- packet with a variable-length label stack reserved in the
ity and timely feedback control. The In-band Network Telemetry packet header. The probe packets are periodically generated
(INT) allows cost-effective network monitoring by encapsulating
device-internal states into probe packets. However, INT only from an INT agent and injected into the network, where the
specifies an underlying device-level primitive while how to achieve probe packets will be queued and forwarded together with the
network-wide traffic monitoring remains undefined. In this work, background traffic. In each router/switch along the forwarding
we propose INT-path, a network-wide telemetry framework, by path, the probe packet will extract device-internal states and
decoupling the system into a routing mechanism and a routing push them into the INT label stack. At the last hop, the probe
path generation policy. Specifically, we embed source routing
into INT probes to allow specifying the route the probe packet packet containing the end-to-end monitoring data will be sent
takes through the network. Above the mechanism, we develop an to the remote centralized controller for further analysis.
Euler trail-based path planning policy to generate non-overlapped INT is essentially an underlying primitive that need the
INT paths that cover the entire network with a minimum path support of special hardware for internal state exposure. With
number. Besides, an exhaustive analysis of algorithm’s run-time such data extraction interface, network operators can obtain
complexity is also provided. INT-path can “encode” the network-
wide traffic status into a series of “bitmap images”, transforming the real-time traffic status of a single device or a device chain
network troubleshooting into pattern recognition problems. INT- along the probing path. However, to improve network man-
path is very suitable for deployment in data center networks agement, INT further requires a high-level mechanism built
thanks to their symmetric network topologies. upon it to efficiently extract the network-wide traffic status.
More specifically, as Software-Defined Networking (SDN) is
I. I NTRODUCTION increasingly deployed, it is usually expected that the controller
Today’s data center networks have become mega-scale with will have a global view (i.e., network-wide visibility) to make
increasing deployment of diverse cloud services. With the the optimal control decisions. Besides, network management
continuous expansion of network size, fine-grained network automation via machine learning also requires timely feedback
monitoring has become one prerequisite for better network from the environment as the fed-in training data [3].
reliability and enhanced user experience. In traditional network In this work, for the first time, we raise the concept of
monitoring, management protocols, such as SNMP [1], are “In-band Network-Wide Telemetry” and propose INT-path, a
widely adopted to periodically poll the router/switch CPU for telemetry framework to achieve network-wide traffic moni-
collecting device-internal states every few seconds or minutes. toring. We tackle the problem using the divide-and-conquer
However, due to the constant interaction between the control design philosophy by decoupling the solution into a routing
plane and the data plane as well as the limited on-board CPU mechanism and a routing path generation policy. Specifically,
capability, such monitoring mechanism is coarse-grained and we embed source routing into INT probes to allow specifying
involves a large query latency, which cannot scale well in the route the probe packet takes through the network. Based on
today’s high-density data centers with drastic traffic dynamics. the routing mechanism, we design two path planning policies
To ameliorate the scalability issue, In-band Network to generate multiple non-overlapped INT paths that cover the
Telemetry (INT) is proposed by the P4 Language Consortium entire network. The first is based on depth-first search (DFS)
(P4.org) to achieve fine-grained real-time data plane monitor- which is straightforward but time-efficient. The second is an
ing [2]. INT allows packets to query device-internal states such Euler trail-based algorithm that can optimally generate non-
as queue depth, queuing latency when they pass through the overlapped INT paths with a minimum path number.
data plane pipeline, without requiring additional intervention Based on INT, our approach can “encode” the network-wide
traffic status into a series of “bitmap images”, which further
This work is supported by National Natural Science Foundation of China allows using advanced techniques, such as pattern recognition,
(NSFC) (61702049, 61872401, 61432009, 61872213), National Science and to automate network monitoring and troubleshooting (although
Technology Major Project of the Ministry of Science and Technology of
China (No:2018ZX03001014-003) and Huawei Innovation Research Program pattern recognition has been widely used in image processing,
(HIRP). Corresponding author: Tao Huang. we rarely find their usage in networking area). Such transfor-

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
mation from traffic status to bitmap images will have profound number of separated INT paths, the more INT agents need be
significance because when the network becomes mega-scale, deployed). (3) Besides, as the INT agent number grows, the
automated approaches can be much more efficient than traffic controller will suffer from a performance penalty for handling
analysis purely by human efforts. Coincidentally, there is a increased telemetry workload sent from those INT agents.
very related piece of work named “Barefoot Deep Insight”, Synchronized multi-path monitoring. In each monitoring
which is also built upon INT and claims to enable end-to- round, the centralized controller cannot evaluate the global
end traffic monitoring [4]. However, Barefoot Deep Insight is network view until the probe packets from all the monitoring
a proprietary solution without disclosing any technical detail, paths are collected. That is to say, if one path is much longer
while our proposal provides a completely open solution to than the others, the controller has to delay the control decision
achieve network-wide traffic monitoring. making until the return of the last probe packet from the
Our major contributions are summarized as follows: longest path, which will degrade the timeliness of centralized
• We raise “In-band Network-Wide Telemetry” and design control. Hence, balanced path generation is preferable.
a mechanism-policy-separation framework. Specifically,
B. Design Space Analysis
we propose a source routing-based telemetry mechanism
by coupling the INT probe with a source routing label As an auxiliary function for better network management,
stack for user-specified path monitoring (§II). a cost-effective telemetry implementation is welcome. To
• We develop DFS-based and Euler trail-based path plan- enable network-wide telemetry, all edges of the network graph
ning algorithms to generate non-overlapped INT paths should be covered by the probing paths1 . To reduce unneces-
covering the entire network. The Euler trail-based algo- sary bandwidth occupation, non-overlapped probing paths are
rithm can generate theoretically minimum path number preferable. To lessen the processing overhead of the telemetry
to maximally reduce the telemetry overhead (§III). workload at the controller, the path number should be kept
• We conduct an exhaustive analysis of algorithm’s run- as small as possible. To retain the timeliness of centralized
time complexity and derive an upper bound of algorithm’s control, the path length should be as balanced as possible. To
complexity as O(k(3E + V − 15 2 k)) (§III-D).
simultaneously satisfy the above design goals, the start/end
• We implement an INT-path prototype with 2995 lines of node of each probing path need be carefully planned. However,
code, available at our git repository [5]. The extensive as mentioned earlier, the original INT lacks the ability to spec-
evaluation shows that INT-path is very suitable for sym- ify the route the probe packet takes through the network, which
metric topologies of today’s data center networks (§IV). is the exact prerequisite for accurate path planning. Here, we
propose INT-path, a framework for network-wide telemetry,
II. N ETWORK -W IDE T ELEMETRY S YSTEM by decoupling the system into a routing mechanism and a
In this section, we first describe the technical challenges of route generation policy. The underlying mechanism allows
building a network-wide telemetry system with INT and then network operators/INT agents to specify a particular path for
provide our own design to conquer these challenges via the monitoring, addressing the uncontrollable path issue (§II-C).
separation of mechanism and policy. The policy built upon the mechanism generates multiple INT
paths to cover the entire network and a good policy is expected
A. Challenging Issues to minimize the telemetry overhead (§II-D and §III).
Uncontrollable probing path. As an underlying primitive,
INT only defines how to extract device-internal states using C. Source Routing-based Path Monitoring (Mechanism)
probe packets. However, the probe packet itself cannot proac- We address the uncontrollable path issue via the technique
tively decide which path to monitor since it does not have any of source routing (SR) [6]. Fig. 1 shows the source routing-
path-related prior knowledge. If the INT header is embedded in based telemetry architecture as well as the probe packet for-
an IP packet, the probing path will be passively decided by its mat. In computer networking, source routing allows a sender
destination IP address together with the routing table in each of a packet to specify the route the packet takes through
network device, leaving probing path totally uncontrollable by the network, usually by labeling the route onto the packet
INT agents. Given the uncontrollable probing path, it is not header. In contrast, in non-source routing protocols, packets
easy to work out purposive strategies to optimally generate are forwarded by routers based on the destination addresses.
multiple probing paths for achieving network-wide telemetry. Although source routing is not a new technique, we innovate
Telemetry overhead. During traffic monitoring, we need to couple it with the INT probe using the P4 language [7] to
periodically perform the INT operation at all devices and no- implement user-specified/on-demand path monitoring.
tify the controller about the underlying traffic status. However, Packet header format. In Fig. 1, we use a UDP packet to
straightforwardly conducting INT at each device or device carry the SR-INT payload. To inform the packet parser that
chain incurs significant performance overhead: (1) INT will it is an SR-INT probe, the destination port (DP) number is
inject probes into the network, which will occupy a fraction
1 Since the INT probe packet can only collect the queue depth or the queuing
of link bandwidth (the finer INT sampling granularity, the
latency from the network interfaces it travels through, constructing monitoring
more bandwidth will be consumed). (2) INT agents must paths into a spanning tree or a Steiner tree is insufficient for obtaining the
be deployed for probe generation and collection (the higher network-wide link status at the controller side.

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
 d, /W hW ^Z /Ed /WсŽŶƚƌŽůůĞƌ͛Ɛ/W ZĞŵŽƚĞĂŶĂůLJƐŝƐ port ID popped from the SR label stack (the DIP is no longer
Wс^Zͺ/EdͺWKZd
used in this case). The SR label is popped once at a router by
WŽƌƚŶ ͙͙ WŽƌƚϮ WŽƌƚϭ /EdŶ ͙͘͘͘ /EdϮ /Edϭ /Edϰ
ϰď ϮϮ
ŽŶƚƌŽůůĞƌ /Edϯ
/EdϮ
right shifting the SR header by 4 bits at each hop. Besides, the
ϱϭϮď
WƌŽďĞƉĂĐŬĞƚĨŽƌŵĂƚ
sĂƌŝĂďůĞůĞŶŐƚŚ /Edϭ
INT forwarder will also push its local INT information into
WƌŽďĞƉĂĐŬĞƚ ϰ͕ϱ
ϰ
/Edϯ
the INT label stack before forwarding the probe.
/Edϰ
/EdϮ /EdϮ
ϰ͕ϱ͕Ϯ
/Edϭ
/Edϭ ϱ /Edϭ
/Edϯ
/EdϮ At the last hop of the monitoring path, since the DIP is
Ϯ /Edϭ
ϯ
/EdĨŽƌǁĂƌĚĞƌ
;ůĂďĞůƐǁŝƚĐŚŝŶŐͿ
ϰ filled with controller’s IP address, the INT collector will finally
ϭ
/EdŐĞŶĞƌĂƚŽƌ ϯ /EdĐŽůůĞĐƚŽƌ forward the probe packet to the controller for further analysis.
;^ZůĂďĞůͿ ;/EdƌĞƉŽƌƚͿ
ŶĚŚŽƐƚ
hƐĞƌͲƐƉĞĐŝĨŝĞĚ/EdƉĂƚŚ
D. INT Path Planning for Graph Coverage (Policy)
Based on the SR-based telemetry mechanism, we can ac-
Fig. 1. Source routing-based path monitoring.
curately control each probing path thus can develop diverse
INT path planning algorithms for non-overlapped INT path
set to “SR_INT_PORT”. Above the UDP header, we reserve
generation that covers the entire network graph. Here, we first
512-bit for the SR label stack. We allocate 4-bit for each SR
propose a simple algorithm based on depth-first search (DFS),
label to denote the router output port ID thus can maximally
followed by a more sophisticated one based on Euler trail
support 16 output ports for each router. Above the fixed-length
which generates the optimal result.
SR label stack, we allocate a variable-length INT label stack.
When traversing a tree or a graph, DFS starts at the root
Each INT label occupies 22B containing the information such
and explores as far as possible along each branch before
as device ID, ingress/egress port, egress queue depth. Since
backtracking. The basic idea of the DFS-based path planning
P4 currently does not well support parsing double variable-
algorithm is to consecutively add the visited vertices into the
length stacks in the packet header, we statically allocate the SR
current path before backtracking; if we have nowhere to go
label stack with a fixed length and use the right shift operation
and have to backtrack, we just create a new path and add the
(“”) to perform the “stack pop” behavior. The destination IP
fork vertex (the first vertex along the backtracking path that
(DIP) address of the probe packet is set using controller’s IP to
has unvisited edges) as the first node of the new path. After all
guarantee that the probe packet will finally be forwarded to the
the edges are visited in the DFS order, we can extract multiple
controller for further analysis. Notice that although we design
non-overlapped paths covering the entire graph.
a customized header format for the probe packets, the network
Fig. 2 shows a recursive version of the DFS-based path
devices can still correctly forward these packets provided that
planning algorithm. For each visited vertex, we will choose
protocol-independent forwarding [7] is supported.
one of its unvisited edges for depth-first traversal (line 6-7 in
Forwarding behaviors. In the SR-based telemetry archi- Fig. 2). If we have chosen an unvisited edge, we will mark
tecture, we propose three types of logic routers with different it as visited (line 8-9). We will explore as far as possible
functionalities: the INT generator, the INT forwarder and the and add the visited vertices into the current INT path before
INT collector. A physical device can simultaneously act as an backtracking occurs (line 17). On backtracking, we will create
INT generator and an INT collector. In other words, a physical a new path from the fork vertex as the current path (line 12). In
device can be at the end of one monitoring path while at the Fig. 2, we use the technique of recursion (line 19) to perform
start of another monitoring path at the same time. backtracking and leverage an additional flag to locate the fork
The INT generator is responsible for spawning the SR-INT vertex during recursion. When a backtracking occurs during
probe packets at the first hop of the monitoring path. Since DFS, the current function call will return with true (line 22) as
packet generation directly from the data plane is currently its return value (which will be assigned to the bool variable flag
undefined in P4, we consider a workaround to periodically in its caller function in line 19). When the flag is true (line 10)
generate “empty” probes from the outside by either the and there is at least one unvisited edge (line 7), the currently
router/switch CPU or a host attached to the network device. visited vertex can be identified as a fork vertex (line 12-13)
When the probe arrives at the data plane, the INT generator according to the previous definition. The recursion will finally
will rewrite its packet header to allocate the SR label stack and halt when all the edges are visited. Besides, the first vertex
add its local INT information using header.setValid() v0 of the depth-first traversal should be specially treated as a
before forwarding the packet. Specifically, the INT generator boundary case (line 1-3) and we have to invoke PathPlan(v0 ,
will push the output port IDs into the SR label stack in the true) to start the algorithm. The run-time complexity of the
packet header. The sequence of the output port IDs (i.e., how algorithm is O(V 2 ), similar with the classic DFS, if the graph
to forward the packet across the network) is predetermined at is implemented with an adjacency matrix.
the controller via centralized route calculation. Fig. 3 shows an algorithm example on a network graph of
The INT forwarder performs packet forwarding of either the five devices. After PathPlan(v0 , true) is invoked, v0 is pushed
SR-INT probes or the background traffic, according to the DP into the call stack, path1 = {v0 , v1 } and the edge between
number of the incoming traffic. If the DP is “SR_INT_PORT”, v0 and v1 is marked as visited. The path1 expands as more
the INT forwarder will perform label switching (similar with and more vertices are visited in the DFS order. When path1
MPLS) and forward the packet only according to the output expands to {v0 , v1 , v2 , v3 , v1 }, we have nowhere to go and

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
 SURFHGXUH3$7+3/$1YHUWH[YERROLVBILUVW^
odd vertices contains at least k distinct trails which, together,
LILVBILUVW^VSHFLDOWUHDWPHQWRIWKHILUVWQRGH traverse all edges of the graph exactly once [8]. The proposed
FUHDWHDQHZSDWKDVWKHFXUUHQWSDWK optimal algorithm heavily relies on the above theorems.
DGGYLQWRWKHFXUUHQWSDWK
`
IODJ IDOVH B. Algorithm Description
IRUHDFKYHUWH[MRIY¶VDGMDFHQWYHUWLFHV^
In fact, the above properties (especially theorem (4)) have
LIDGMDFHQF\BPDWUL[>Y@>M@ō^
ODEHOWKHXQYLVLWHGHGJHDVYLVLWHG already indicated the theoretical value of the minimum non-
DGMDFHQF\BPDWUL[>Y@>M@ DGMDFHQF\BPDWUL[>M@>Y@  overlapped path number for covering a given graph. To achieve
LIIODJ^
LIEDFNWUDFNLQJRFFXUVYEHFRPHVWKHVWDUWQRGHRIDQRWKHUQHZSDWK
the theoretical minimum, each extracted path from a graph
FUHDWHDQHZSDWKDVWKHFXUUHQWSDWK should start from one odd vertex and end at another odd
DGGYLQWRWKHFXUUHQWSDWKYLVWKH³IRUNYHUWH[´ vertex. In other words, removing one such path from a graph
DGGMLQWRWKHFXUUHQWSDWKMLVWKHVHFRQGQRGHRIWKHQHZSDWK
` will eliminate a pair of odd vertices from that graph. According
HOVH^ to the above observation, we devise an Euler trail-based
DGGMLQWRWKHFXUUHQWSDWKIROORZWKHGHSWKILUVWWUDYHUVDO
`
algorithm to iteratively extract a path between a pair of odd
IODJ 3DWK3ODQYIDOVHUHFXUVLYHFDOO vertices until all the vertices/edges are extracted from the
` original graph (i.e., the degree of every vertex becomes zero).
`
UHWXUQWUXH³QRZKHUHWRJR´KDYHWREDFNWUDFN For a given graph with 2k odd vertices, our algorithm will
` generate k non-overlapped monitoring paths (the theoretical
minimum) to cover all edges of the graph.
Fig. 2. DFS-based INT path planning algorithm (a recursive version).
Although the algorithm sounds rather straightforward as an
iterative path extraction process, the devil lies in the detail
have to backtrack. At this time, we pop v1 from the stack and
of dealing with several boundary cases. To be more specific,
return to v3 , which has an unvisited edge to v4 (notice that the
the devil lies in the possibility that an extracted path can split
call stack push/pop operations are implicitly performed during
one connected graph into multiple subgraphs, which definitely
recursion). Then, we identify v3 as the fork vertex because it is
complicates the iterative path extraction process. For example,
the first vertex along the backtracking path that has unvisited
from Fig. 4(a) to Fig. 4(b) after the extraction of path 1-4-3,
edges. Based on v3 , we create a new path as path2 = {v3 , v4 }.
the connected graph is split into two subgraphs. Besides, the
When path2 expands to {v3 , v4 , v2 }, we have again nowhere to
algorithm also need deal with (sub)graphs with no odd vertex
go and have to backtrack. But at this time, although we check
(e.g., subgraph {1,2,3} in Fig. 4(b)). We put the pseudocode
all the vertices popped from the call stack, we still cannot find
of the optimal algorithm in Fig. 5 and explain it in detail.
any fork vertex. The recursion halts when the call stack finally
Actually, without considering the complexity of graph split,
becomes empty. At last, we extract two non-overlapped INT
for a given connected graph, there are mainly three different
probing paths (i.e., path1 and path2).
cases for path extraction. If the graph contains two odd
III. O PTIMAL PATH P LANNING P OLICY vertices, we just find an Euler trail between the two odd
vertices as the path to be extracted. If the graph contains more
A. Properties of Euler Trail than two odd vertices, we should extract an Euler trail between
Although the DFS-based approach is time-efficient, it has any pair of odd vertices. If the graph does not contain any odd
no guarantee to minimize the number of generated paths, vertex, we can extract an Euler circuit from the graph, which
which will increase the telemetry overhead, especially the will traverse every vertex of the graph.
telemetry workload at the centralized controller (as mentioned Specifically, We use G to represent the network graph,
in §II-A). Besides, it exerts no optimization on the balanced which is initialized to be one connected graph and may
path generation. Here, we propose an optimal path planning also become multiple disconnected subgraphs caused by path
algorithm taking advantage of the mathematical properties of extraction during algorithm iteration. We use Q to represent
the Euler trail/circuit. In graph theory, a trail is a walk in a the path set which is initialized as an empty set and will
graph (e.g., v0 , e1 , v1 , e2 , ..., vk ) without repeated edges and finally contain the generated non-overlapped INT paths. We
an Euler trail is a trail which visits every edge exactly once. use G − p to represent extracting a path p from the graph G
Similarly, an Euler circuit is a special Euler trail which starts which will possibly further split the graph(s) in G into more
and ends on the same vertex. To clarify the important theorems subgraphs. First, we consider a simple case that the algorithm
of the Euler trail/circuit, we give the definition of odd vertex input is only one connected graph. If the graph has no odd
first. An odd vertex is a vertex who has an odd degree (e.g., vertex, an Euler circuit can be found with the Hierholzer’s
vertex 1, 3, 5, 6 in Fig. 4(a)). The following four theorems algorithm [9] and inserted into Q (as implemented in line 17-
are well proved in the graph theory: (1) a connected graph 19 in Fig. 5). Here, the Hierholzer’s algorithm is an algorithm
with no odd vertex has an Euler circuit; (2) a connected graph for finding Euler trails/circuits with a run-time complexity of
with only one odd vertex does not exist; (3) a connected graph O(E) [10], which is more efficient than the famous Fleury’s
with two odd vertices has an Euler trail starts from one odd algorithm [11]. If the graph has two odd vertices, an Euler trail
vertex and ends at the other one; (4) a connected graph with 2k can also be found with the Hierholzer’s algorithm and inserted

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
 Ϯ Ϯ

ŽĚĚǀĞƌƚĞdž
ϭ ϯ ϭ ϯ

Y ϰ ϰ ϰ

Y
ϱ ϲ ϱ ϲ ϱ ϲ


Y  
 ϳ ϳ ϳ
 ;ĂͿ ;ďͿ ;ĐͿ
  'ϭс΂ϭ͕Ϯ͕ϯ΃͕'Ϯс΂ϰ͕ϱ͕ϲ͕ϳ΃͖
'ϭс΂ϭ͕Ϯ͕ϯ͕ϰ͕ϱ͕ϲ͕ϳ΃͖ 'ϭс΂ϰ͕ϱ͕ϲ͕ϳ΃͖
SXVK  ^с΂'ϭ͕'Ϯ΃͕dс΂'ϭ΃͖ ^с΂'ϭ΃͕dсѫ͖
Y ^с΂'ϭ΃͕dсѫ͖
dͺƉĂƚŚсϭͲϰͲϯ͕dͺĐŝƌĐƵŝƚсϭͲϮͲϯͲϭ͖ ŽĚĚͺŶƵŵсϮ͖
Y ŽĚĚͺŶƵŵсϰхϮ͖
SRS ƉĂƚŚсϭͲϮͲϯͲϭͲϰͲϯ͖ Yс΂ϭͲϮͲϯͲϭͲϰͲϯ͕ϱͲϰͲϲͲϱͲϳͲϲ΃͖
Yс΂ϭͲϰͲϯ΃͖
 Yс΂ϭͲϮͲϯͲϭͲϰͲϯ΃͖
Fig. 3. Depth-first graph traversal. Fig. 4. Path extraction process of the Euler trail-based algorithm.

SURFHGXUH2373$7+3/$1JUDSK*SDWKBVHW4^ possibility that T _circuit must be an integral part of a final

ZKLOH*ōQXOO^ INT path. In other words, T _circuit should be successfully
6 JUDSK6SOLW*VSOLW*LQWRPXOWLSOHGLVFRQQHFWHGJUDSKVYLD')6
7 JUDSK6HOHFW6VHOHFWJUDSKVQRWFRQWDLQLQJRGGYHUWH[IURP6 connected to one calculated Euler trail to form a longer Euler
LI7ōѫ^ trail. Here, we give a simple proof of the above hypothesis
LI4ōѫ^
IRUHDFKJUDSKJLQ7^ that T _circuit is an integral part of a final INT path.
7BSDWK ILQG5HODWHG4JILQGDSDWKKDYLQJDWOHDVWRQHJ¶VYHUWH[IURP4 Proposition 1. In the Euler trail-based path planning algo-
4 4±7BSDWKUHPRYH7BSDWKIURP4
7BFLUFXLW HXOHU&LUFXLWJUHWXUQWKH(XOHUFLUFXLWRIJDVDQHZSDWK rithm (Fig. 5), if T = ∅ and Q = ∅ (i.e., there is at least one
SDWK SDWK3DVWH7BFLUFXLW7BSDWKFRQQHFWSDWK7BFLUFXLW ZLWKSDWK7BSDWK (sub)graph g that does not contain odd vertex and there has
4 4SDWKDGGSDWKLQWR4
* *7BFLUFXLWGHOHWHWKHHGJHVDORQJSDWK7BFLUFXLWIURP*  been at least one calculated Euler trail in Q), T _circuit (the
`
`
Euler circuit of g) should be successfully connected to one of
HOVH^ the calculated Euler trails in Q to form a longer Euler trail.
IRUHDFKJUDSKJLQ7^
7BFLUFXLW HXOHU&LUFXLWJUHWXUQWKH(XOHUFLUFXLWRIJDVDQHZSDWK Proof. We construct a simple proof by contradiction. Suppose
* *7BFLUFXLWGHOHWHWKHHGJHVDORQJSDWK7BFLUFXLWIURP*
4 47BFLUFXLWDGG7BFLUFXLWLQWR4  T _circuit cannot connect to any of the Euler trails in Q. Then,
`
`
there must be two possibilities: (1) g itself is the initial graph
` G; (2) g is once a part of the initial graph G. For case (1),
IRUHDFKJUDSKJLQ67^IRUHDFKJUDSKFRQWDLQLQJRGGYHUWH[IURP6
RGGBQXP JHW2GG1XPJFDOFXODWHWKHQXPEHURIRGGYHUWLFHVLQJ
if g is the initial graph, then Q must be an empty set, which
LIRGGBQXP ^WKHUHVKRXOGEHRQH(XOHUWUDLOFRYHULQJDOOHGJHVRIJUDSKJ contradicts with the premise that Q = ∅. For case (2), if g is
SDWK HXOHU7UDLOJILQGDQ(XOHUWUDLOEHWZHHQWKHWZRRGGYHUWLFHV
`
once a part of the initial graph G, then g must have at least
HOVHLIRGGBQXP!^ one common vertex v with graph G−T _circuit. If T _circuit
SDWK UDQGRP7UDLOJILQGDWUDLOEHWZHHQWZRUDQGRPO\VHOHFWHGRGGYHUWLFHV
` cannot connect to any of the Euler trails in Q, then v does
* *SDWKGHOHWHWKHHGJHVDORQJSDWKSDWKIURP* not belong to any of the Euler trails in Q. However, according
4 4SDWKDGGSDWKLQWR4 
` to the algorithm in Fig. 5, graph split occurs only after the
`
`
extraction of Euler trails. If v is not on the Euler trails, then
g must be still connected with G − T _circuit on v, which
contradicts with the premise that g is a separate subgraph.
Fig. 5. Euler trail-based path planning algorithm.
If T = ∅ and Q = ∅, we search the current Q to find a
into Q (line 25-26, line 31-32). If the graph has more than two path T _path having at least a same vertex with T _circuit
odd vertices, we just randomly choose two odd vertices and (line 7). Then, we connect T _circuit with T _path to create
find a path (e.g., with the Dijkstra’s algorithm or any other a longer new path via pathPaste() (line 10) and replace
algorithms) to connect the pair of odd vertices, then we insert the original T _path with the new path in Q (line 8, 11).
the path into Q (line 28-29, line 31-32). Notice that each time In summary, the algorithm iteratively extracts paths connect-
we extract a path from G and insert it into Q, the graph(s) in ing two odd vertices or pastes Euler circuits to the previous
G may be broken into multiple disconnected subgraphs. Next, extracted paths to form the final INT paths. The algorithm
we discuss how to handle this boundary case in detail. halts if all edges are extracted from the graph (i.e., G = null).
We use S to store the disconnected subgraphs split from G
via DFS traversal (line 2). Then, we select graphs with no odd C. Case Study
vertex from S and store them into T (line 3). For each graph Fig. 4 shows a path extraction process of the Euler trail-
in set T , we can generate an Euler circuit T _circuit for its based algorithm. At the start, there is only one connected graph
full edge coverage (line 9 and line 17). However, T _circuit G1 {1,2,3,4,5,6,7} with 4 odd vertices. Since the number of the
is inadequate to become a final INT path, since if the initial odd vertices is larger than 2 (line 28), we randomly choose two
graph has 2k odd vertices (k > 0), all the k INT paths should odd vertices (1 and 3), extract a path 1-4-3 from G1 and insert
connect k pairs of odd vertices. By contrast, T _circuit does the path into Q (Fig. 4(a)). The above path extraction behavior
not contain any odd vertex. For this situation, there is only one will split the original G1 into two subgraphs G1 {1,2,3} and

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
 
G2 {4,5,6,7}. Since G1 has no odd vertex and Q = ∅ (line this
 step can be derived as T4 = y1 T (Euler(vy1 , ey1 )) +
 
4-5), we paste the path 1-4-3 in Q with the Euler circuit 1-2- y2 T (Dij(vy2 , ey2 )).
3-1 generated from G1 to create a new path 1-2-3-1-4-3. The 2.6. The iteration will go back to 2.2. until G becomes null.
new path will replace the original path 1-4-3 in Q (Fig. 4(b)). Given the algorithm is iterated for m m−1
times, the total time
After the path paste, there is only one graph G1 {4,5,6,7} with cost can be derived as T = T1 + i (T2i + T3i + T4i ),
2 odd vertices (line 25). We use the Hierholzer’s algorithm to where i stands for the (i + 1)th iteration (0 < i < m).
find its Euler trail 5-4-6-5-7-6 as the second INT path in Q Upper bound derivation. The difficulty of complexity
(Fig. 4(c)). The algorithm halts after all the paths are extracted. analysis lies in that the subgraph number in T or S−T for each
iteration as well as algorithm’s execution path cannot be pre-
D. Run-Time Complexity Analysis determined. Here, we derive an upper bound for algorithm’s
The complexity of the above algorithm is analyzed. Since run-time complexity considering its  worst-case execution
 
path.
the algorithm is quite sophisticated, here, we strive to give an First, we try to solve T3 = x T (Euler(v xx , e )). Con-

upper bound for its run-time complexity as follows. sidering  T (Euler(v, e)) = et 0, given e = x xe , we have
Assumption. The input of the algorithm is an undirected T3 = x T (Euler(vx , ex )) = x ex t0 = e t0 .
connected graph G = (V, E) with 2k odd vertices (k = Then, we try to solve T4 . According to our algorithm, the
0, 1, 2, ...). The graph is implemented via the adjacency list. number of odd vertices of each subgraph in S − T will be
We only consider the most time-consuming subprocedures in decreased by 2 in each iteration. Hence, the total number of
the algorithm, namely, (a) DFS for graph split (line 2), (b) decreased odd vertices in each iteration will be 2(y1 + y2 ).
eulerCircuit() and eulerTrail(), both using the According to the assumption, the total number of decreased
Hierholzer’s algorithm (line 9, 17, 26), (c) randomTrail() odd vertices in m iterations is 2k. In the worst case, the
using the Dijkstra’s algorithm (line 29). The time cost T of the iteration times m should be maximized while the number of
three subprocedures can be represented as T (DF S(V, E)) = decreased odd vertices in each iteration should be minimized.
(E + V )t0 , T (Euler(V, E)) = Et0 , T (Dij(V, E)) = Et0 , In this case, m = k, y1 = 0, y2 = 1 for the first k-1 iterations
all based on their fastest implementation [9, 12, 13], where t0 and y1 = 1, y2 = 0 for the kth iteration. That is to say, the
is the one memory access time of the adjacency list. total number of decreased odd vertices in each iteration will
Complexity analysis case by case. be limited to 2. In this case, we have T4i = T (Dij(vi , ei ))
1. If k = 0, 1, G has one Euler circuit or one Euler (i = 1, 2, ..., k − 2) and T4i = T (Euler(vi , ei )) (i = k − 1).
trail. In this case, DFS will be performed to check the graph According to the above analysis, we have T = T1 +
m−1
connectivity, and eulerCircuit() or eulerTrail() (T 2i + T 3i + T4i ) ≤ T (DF S(V, E)) + T (Dij(V, E)) +
ik−2   
will be performed to extract the path. Hence, the time cost i=1 (T (DF S(v i , ei )) + ei t0 + T (Dij(vi , ei ))) +
can be derived as T = T (DF S(V, E)) + T (Euler(V, E)). T (DF S(vk−1 , ek−1 )) + ek−1 t0 + T (Euler(vk−1 , ek−1 )),
 

2. If k > 1, G must have at least 4 odd vertices. where ei is the edge set of G = (v, e), ei is the edge set of
2.1. For the first iteration of the algorithm, DFS will be the subgraph subset T and ei is the edge set of the subgraph
performed to check the connectivity, then, randomTrail() subset S − T , all in the (i + 1)th iteration. Similarly, vi and vi
will be performed to extract one shortest path between 2 odd are the vertex sets of G and S − T , respectively. Apparently,
vertices. The time cost T1 = T (DF S(V, E))+T (Dij(V, E)). we have ei = ei + ei and vi = vi + vi .
2.2. For the next iteration, after the path extraction, the According to the worst-case assumption, there is at least one
graph G = (V, E) becomes G = (v, e). Then, we use DFS to subgraph in T and there is only one subgraph in S−T for each
check its connectivity. The path extraction may split G into iteration. Therefore, Gi for the (i + 1)th iteration is generated
subgraphs. We use S to represent the subgraph set. For this from Gi−1 by removing all the subgraphs from T and one
step, the time cost T2 = T (DF S(v, e)). shortest path from S −T . Using the above recurrence relation,
2.3. Set S can further be partitioned into two subsets with we can further derive algorithm’s run-time complexity T .
one subset T containing subgraphs without odd vertices and Actually, the minimal subgraph without odd vertices is the
the other subset S −T containing subgraphs with odd vertices. triangle, which has 3 vertices and 3 edges, while the minimal
We define T = {gx |gx = (vx , ex ), x = 1, 2, ...}, S− T = shortest path is the line segment which has 2 (odd) vertices
{gy
|gy = (v 
y , ey ), y = 1, 2, ...}. We have

x vx +

y vy = and 1 edge. Therefore we have ei ≥ 3, vi ≥ 3, ei ≥ 1,
 
v, x ex + y ey = e. vi ≥ 2, ei ≤ ei−1 − 3 − 1, vi ≤ vi−1 − 3. By plugging the
2.4. For each subgraph gx in subset T , we will initial values e0 = E, v0 = V into the above relations, we
perform eulerCircuit() and pathPaste(). Since have ei ≤ E − 4i, vi ≤ V − 3i. Based on above relations, we
eulerCircuit() is much more  time-consuming, we only can further derive
consider its time cost as T3 = x T (Euler(vx , ex )). ei = ei − ei ≤ ei − 1 ≤ E − 4i − 1,
2.5. For each subgraph gy in subset S −T , if gy has exactly vi = vi − vi ≤ vi − 2 ≤ V − 3i − 2,
2 odd vertices, we will perform eulerTrail(); if gy has ei = ei − ei ≤ ei − 3 ≤ E − 4i − 3,
more than 2 odd vertices, we will perform randomTrail(). vi = vi − vi ≤ vi − 3 ≤ V − 3i − 3.
Given we have y1 subgraphs with exactly 2 odd vertices and By plugging the above relations into the expression of T ,
y2 subgraphs with more than 2 odd vertices, the time cost of we derive an upper bound for the run-time complexity as

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
 k−2
T ≤ (E + V )t0 + Et0 + i=1 ((ei + vi )t0 + ei t0 + ei t0 ) +
k−1
(ek−1 +vk−1 )t0 +ek−1 t0 +ek−1 t0 ≤ (2E +V )t0 + i=1 (ei +


WĂĐŬĞƚEƵŵďĞƌŝŶYƵĞƵĞ

WĂĐŬĞƚEƵŵďĞƌŝŶYƵĞƵĞ
k−1
vi + ei + ei )t0 ≤ (2E + V )t0 + i=1 (E − 4i + V − 3i + E −
4i − 1 + E − 4i − 3)t0 ≤ (3kE + kV − 15 2 7

^ǁŝƚĐŚ/
2 k − E + 2 k + 4)t0 .

^ǁŝƚĐŚ/
Therefore, one upper bound for algorithm’s run-time com-
plexity can be represented as O(k(3E + V − 15 2 k)).

E. Heuristic for Balanced Path Generation

^ǁŝƚĐŚ/ ^ǁŝƚĐŚ/
As mentioned in §II-A, balanced path generation is prefer- (a) Network is lightly loaded. (b) Network is heavily loaded.
able to facilitate synchronized multi-path monitoring. How-
Fig. 6. Network-wide traffic visibility as a series of “bitmap images”.
ever, our previous algorithm does not explicitly optimize for
this particular purpose. Here, we propose a simple heuristic for and create the SR-INT probes. The deployment locations of
more balanced path generation based on the original algorithm. the INT generators/collectors as well as the source routing
According to our extensive debugging of the original al- paths embedded in the SR-INT probes are precalculated by
gorithm, we find that the path connecting two odd vertices the Euler trail-based path planning algorithm. The default INT
generated by function eulerTrail() in line 26 is often too packet sending rate is set to 100pps. On receiving the INT
short compared with the average length of the Euler trails. The packet, the controller will parse the packet header and write
reason lies in that in our implementation of eulerTrail(), the telemetry data into a local MySQL database, where the
we return the shortest path between two randomly selected global traffic status can be summarized and visualized.
odd vertices with the Dijkstra’s algorithm. If the path cannot The two path planning algorithms (i.e., DFS and Euler) are
properly paste to an Euler circuit afterwards to form a longer implemented with Python. The efficiency of the algorithms
Euler trail, the final INT paths will be less balanced. are evaluated under randomly generated graphs of different
To address this issue, we propose a heuristic for more sizes as well as two data center topologies (i.e., FatTree and
balanced path generation simply by rewriting the function LeafSpine). We evaluate the metrics such as generated INT
eulerTrail(). Instead of returning the shortest path be- path number, INT path length variance, telemetry overhead,
tween two randomly selected odd vertices, we calculate all algorithm execution time, impact of odd vertex number.
the shortest paths between any given pair of odd vertices and The entire system consists of 2464 lines of Python, 324
return the longest path from all these shortest paths. Such a lines of P4, 191 lines of C and 16 lines of SQL, which is
tweak greatly reduces the length variance of final INT paths. available at our git repository [5].
However, on each extraction of an Euler trail, we have to We show some output results from INT-path to give intuitive
recalculate Cn2 (n is the number of odd vertices) shortest paths impression about our system. Fig. 6 shows the network-wide
due to the graph update, which is time-consuming. Here, we traffic status in one snapshot under varying traffic loads. The
propose a technique to further minimize the computation cost. network-wide traffic status (here, queue depth) is “encoded”
Actually, in our algorithm, we always remove edges from the into a series of “bitmap images” in real time. By analyzing the
graph during path extraction while never add edges into the “bitmap images”, one can dig out insightful knowledge from
graph. Hence, if the calculated shortest path has no intersection the underlying network (e.g., from Fig. 6(b), we can find that
with the extracted Euler trail, the shortest path need not be link(12, 11) is heavily congested in that time slice).
recalculated. By checking the intersection of the extracted path
and the shortest paths, we can update the shortest paths on B. Experimental Results
demand and eliminate unnecessary computation cost. Number of generated INT path. Fig. 7 shows the number
of generated INT paths from DFS and Euler trail-based
IV. E VALUATION
algorithms under randomly generated graphs. The path number
A. Experimental Setting of both DFS and Euler grow roughly linearly with the graph
We build a network prototype to demonstrate the proposed size while the Euler generates less INT paths than DFS under
INT-path. The hardware is with i7-6700K CPU and 16GB any fixed switch number. As we have mentioned, the number
memory, running Ubuntu 16.04 OS. The prototype is imple- of paths generated from Euler is the theoretical minimum
mented using Mininet and consists of 1 controller, 18 switches and only decided by the odd vertex number of the graph.
and 12 end hosts. Specifically, we adopt the P4 software In Fig. 7, since the graph is randomly created, Euler’s path
switch (i.e., bmv2 [14]) to achieve the capability of protocol- number grows together with the odd vertex number as the
independent header rewriting and forwarding of customized graph becomes larger. In Fig. 8, when we fix the number of
INT packets which contain additional source routing metadata. odd vertices in a graph, the INT path number generated from
The controller speaks with bmv2 using Thrift. Each end host is Euler becomes a constant no matter what size the network
responsible for generating 1kpps background UDP traffic with grows into. In Fig 9, when we fix the number of vertices to
randomly changed destination addresses. The packet interval a constant (e.g., 39 in our experiment) and grow the network
follows Poisson distribution. The end hosts also generate size by adding more edges, we can find that the path number
“empty” probes for P4 switches to further rewrite the header generated from both DFS and Euler grow at the first and

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
 60 18 35 20
DFS Balance
17 DFS Unbalance
50 Euler 30

Variance of Path Lengths

Euler
16 15
INT Path Number

INT Path Number

INT Path Number

DFS 25
40 Euler 15
14 20 10
30
13 15
20 12 5
10
10 11
5 0
10
0
0 20 40 60 80 100 9 0
40 60 80 100 120 140 0 100 200 300 400 500 600 700 5 10 15 20 25 30
Switch Number Switch Number
Switch Number Edge Number

Fig. 7. Number of INT paths from Fig. 8. Number of INT paths gener- Fig. 9. Number of INT paths gener- Fig. 10. Variance of path lengths with
DFS and Euler trail-based algorithms. ated under a fixed odd vertex number ated under a fixed vertex number balanced and unbalanced algorithms.
6 0.008 0.25 100
250 14000
0.007 90

Call Times of Hierholzer’s

Controller Overhead (kpps)

DFS-10-Odd

End-to-End Latency (us)

DFS-Euler-Switch 5 12000
Switch Overhead (kpps)

200 DFS-Controller Euler-10-Odd 0.2 80

0.006

Execution Time (s)

Execution Time (s)

Euler-Controller DFS-20-Odd 70 10000

4 Euler-20-Odd
0.005 0.15 60
150 8000
3 0.004 50
6000 Bg:2000pps
100 0.003 0.1 40 Bg:1000pps
2 4000 Bg:833pps
30
0.002 Bg:666pps
50 1 0.05 20 2000 Bg:588pps
Execution Time
0.001 Hierholzer Call Times 10
0 Comfort Zone
0 0 0 0 0
0 20 40 60 80 100 20 40 60 80 100 0 50 100 150 200 250 -50 50 150 250 350 450 550
Switch Number Switch Number Odd Vertex Number INT Probe Speed (pps)

Fig. 11. Telemetry overhead of DFS Fig. 12. Execution time of DFS and Fig. 13. Impact of odd vertex number on Fig. 14. Impact of telemetry granu-
and Euler on controller and data plane. Euler under a growing network size. Euler’s execution time (500 switches.) larity on system performance.

decline in the end. This is because adding more edges at the than DFS’s. Besides, Euler’s execution time is more sensitive
start makes the graph become more complex with more odd to the odd vertex number of the graph. In Fig. 13, we
vertices while in the end, the graph finally becomes a complete find that the execution time of Euler trail-based algorithm
graph without odd vertices (in a complete graph, every pair grows roughly linearly with the odd vertex number. This
of distinct vertices is connected by a unique edge and each of result also validates our run-time complexity analysis that
the 39 vertices has 38 connected edges). Despite this, Euler the subprocedures randomTrail(), EulerTrail() and
still outperforms DFS to generate less INT paths in Fig 9. EulerCircuit() are the most frequently executed in our
Balanced vs unbalanced path generation. Fig 10 shows algorithm which are related with the odd vertex number.
the variance of the path lengths generated from the original Impact of telemetry granularity. To achieve fine-grained
version of Euler and a later improved version considering path network-wide telemetry, i.e., to obtain the precise real-time
balance (§III-E). We can figure out that by adding the heuristic traffic dynamics in the network, the INT probe generation
for balanced path generation by rewriting randomTrail(), frequency is expected to be as high as possible. However, this
the variance of the path lengths becomes lower compared with will not only lay heavy performance burden on the controller,
the original Euler under randomly generated graphs. but also eat up the limited link bandwidth, causing traffic
Telemetry overhead. Fig. 11 shows the INT telemetry congestion or even router buffer overflow. That is to say, there
overhead of DFS and Euler trail-based algorithms on both should be an upper limit set for the telemetry granularity,
the controller and the switches. The telemetry overhead on the constrained by controller’s ability and link’s capacity. Fig. 14
controller is mainly caused by INT probe collection. The traffic shows this extreme situation. In Fig. 14, the x-axis is the INT
volume of the probes forwarded to the controller is decided by probe generation speed at each end host and the y-axis is the
the INT sending rate as well as the number of INT collectors, end-to-end link latency along the probing path. We can figure
which is further decided by the number of INT paths generated out that when the INT probe generation speed grows, there is
by the algorithms. In Fig. 11, when the switch number is 100, a sudden change of the end-to-end latency. This is because the
DFS floods 5kpps probes to the controller while Euler only buffer of the switches along the probing path become overflow
floods 2.5kpps due to the minimum generated path number. at that time and start to drop packets. To make the experimental
For the data plane, the telemetry overhead is measured as the results more significant, we set a shallow buffer of 100 packets
total number of INT probes processed by switches per second. for each P4 switch and the buffer overflow occurs very soon.
Since both DFS and Euler will traverse all edges of the graph, Besides, the background traffic rate (Bg in Fig. 14) also has
they have the same switch telemetry overhead. The switch an impact on the end-to-end latency of the probe packets. For
telemetry overhead is only affected by the network size and INT-path deployment in real-world networks, we recommend
the INT sampling granularity. to properly arrange the telemetry granularity into the “comfort
Execution time of path planning algorithms. Fig. 12 zone” as shown in Fig. 14, considering both the switch buffer
shows the execution time of DFS and Euler under a growing size and the background traffic rate.
network size. According to our exhaustive analysis of Euler’s Network-wide telemetry in DC networks. In the previous
run-time complexity in §III-D, Euler’s execution time is larger sections, we discuss the performance of Euler trail-based algo-

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.
 &ĂƚdƌĞĞ
>ϭ͗ŶŽŽĚĚǀĞƌƚŝĐĞƐ
solution without disclosing any technical detail. Some other
ŶŽŽĚĚǀĞƌƚŝĐĞƐ͕ŬсϮ͕ϰ͕ϲ͕͘͘͘
>Ϯ͗ĚĞƉĞŶĚƐŽŶηƵŶŝƚƐ works such as [15] conduct solid telemetry implementation
ϮŬǀĞƌƚŝĐĞƐ͕Ŭсϭ͕ϯ͕ϱ͕͘͘͘
ŬŝƐηƵŶŝƚƐ but still do not touch the high-level orchestration. In 2015,
>ϯ͗ŶŽŽĚĚǀĞƌƚŝĐĞƐ
hŶŝƚ Pingmesh [16] conducts a similar network-wide telemetry in
>ĞĂĨ^ƉŝŶĞ one of Microsoft’s data centers. However, Pingmesh only relies
>ϭ͗ŶŽŽĚĚǀĞƌƚŝĐĞƐ
ŶŽŽĚĚǀĞƌƚŝĐĞƐ͕ŬсϮ͕ϰ͕ϲ͕͘͘͘
on the end-host feedback without diving into network devices.
>Ϯ͗ĚĞƉĞŶĚƐŽŶηƵŶŝƚƐ
ϮŬǀĞƌƚŝĐĞƐ͕Ŭсϭ͕ϯ͕ϱ͕͘͘͘
As the rise of the AI age, many network AI systems claim
hŶŝƚ ŬŝƐηƵŶŝƚƐ to build closed-loop control systems depending on real-time
Fig. 15. Two classic data center network topologies. traffic status collection [3]. However, they talk less on the
25
DFS
70 telemetry detail and our approach can be a solid complement.
60 DFS
Euler Euler
20
VI. C ONCLUSION
INT Path Number

INT Path Number

50
15 40 On top of the original INT primitive, we propose INT-path,
10 30 a network-wide telemetry framework/system that includes a
20
5
source routing-based path monitoring mechanism and an Euler
10
trail-based path planning policy. The mechanism allows user-
0 0
0 20 40 60 80 100 0 20 40 60 80 100 specified path monitoring while the policy can optimally gen-
Switch Number (FatTree) Switch Number (LeafSpine)
erate non-overlapped INT paths that cover the entire network.
(a) FatTree topology. (b) LeafSpine topology.
Our approach can “encode” the network-wide traffic status
Fig. 16. Euler outperforms DFS with classic data center network topologies.
into a series of “bitmap images”, making traffic analysis as
rithm under randomly generated graphs. Here, we investigate straightforward as image pattern recognition.
its performance in today’s data center networks. Designing R EFERENCES
for better load balancing, classic data center network topolo- [1] J. D. Case, M. Fedor, M. L. Schoffstall, and J. Davin, “Simple network
gies are symmetric and have multi-path topological patterns. management protocol (snmp),” Tech. Rep., 1990.
Fig. 15 shows two classic DC network topologies: FatTree [2] C. Kim, A. Sivaraman, N. Katta, A. Bas, A. Dixit, and L. J. Wobker,
“In-band network telemetry via programmable dataplanes,” in ACM
and LeafSpine. We find that the odd vertex number in both SIGCOMM Symposium on SDN Research (SOSR), 2015.
topologies are very predictable depending on the network [3] A. Mestres, A. Rodriguez-Natal, J. Carner, P. Barlet-Ros, E. Alarcón,
scale. For example, in FatTree, there are three layers of switch M. Solé, V. Muntés-Mulero, D. Meyer, S. Barkai, M. J. Hibbett
et al., “Knowledge-defined networking,” ACM SIGCOMM Computer
nodes, and the layer 1 and layer 3 do not have odd vertices no Communication Review, vol. 47, no. 3, pp. 2–10, 2017.
matter what the scale of the network is. However, in layer 2, [4] “Barefoot deep insight,” https://barefootnetworks.com/products/brief-
the number of the odd vertices regularly changes depending on deep-insight/, 2017.
[5] “Int-path repository,” https://github.com/graytower/INT_PATH, 2018.
the size of the network. LeafSpine shows the similar property. [6] C. A. Sunshine, “Source routing in computer networks,” ACM SIG-
This property is helpful when we design DC networks with COMM Computer Communication Review, vol. 7, no. 1, pp. 29–33,
In-band Network-Wide Telemetry features. If the DC network 1977.
[7] P. Bosshart, D. Daly, G. Gibb, M. Izzard, N. McKeown, J. Rexford,
topology has an even number of basic units (as shown in C. Schlesinger, D. Talayco, A. Vahdat, G. Varghese et al., “P4: Pro-
Fig 15), at best, we can use one INT telemetry path to cover the gramming protocol-independent packet processors,” ACM SIGCOMM
entire network. If it has an odd number of units (i .e., k), we Computer Communication Review, vol. 44, no. 3, pp. 87–95, 2014.
[8] J. A. Bondy, U. S. R. Murty et al., Graph theory with applications.
have to plan k telemetry paths for the full network coverage. Citeseer, 1976, vol. 290.
Fig. 16 shows the comparison between DFS and Euler under [9] C. Hierholzer and C. Wiener, “Über die möglichkeit, einen linienzug
DC network topologies. We can find the symmetric topology ohne wiederholung und ohne unterbrechung zu umfahren,” Mathematis-
che Annalen, vol. 6, no. 1, pp. 30–32, 1873.
of DC networks even makes DFS output predicable! However, [10] H. Fleischner, “X. 1 algorithms for eulerian trails,” Eulerian Graphs and
Euler still outperforms DFS in either DC topology. Related Topics: Part 1 (Annals of Discrete Mathematics), vol. 2, no. 50,
pp. 1–13, 1991.
V. R ELATED W ORK [11] M. Fleury, “Deux problemes de geometrie de situation,” Journal de
The proposal of INT-path builds on the giant shoulders of mathematiques elementaires, vol. 2, no. 2, pp. 257–261, 1883.
past researches. [7] invents P4, a protocol-independent packet [12] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction
to algorithms. MIT press, 2009.
processing architecture as well as the programming language, [13] M. Thorup, “Undirected single-source shortest paths with positive inte-
which enables network operators to arbitrarily modify packet ger weights in linear time,” Journal of the ACM (JACM), vol. 46, no. 3,
headers. INT [2] is a telemetry application using P4, which can pp. 362–394, 1999.
[14] “P4 switch behavioral model,” https://github.com/p4lang/behavioral-
obtain device-internal states without disturbing the controller. model, 2018.
However, INT itself is an underlying telemetry primitive for [15] N. Van Tu, J. Hyun, and J. W.-K. Hong, “Towards onos-based sdn mon-
monitoring one device or a device chain without defining itoring using in-band network telemetry,” in Network Operations and
Management Symposium (APNOMS), 2017 19th Asia-Pacific. IEEE,
how to achieve network-wide telemetry, which further requires 2017, pp. 76–81.
high-level orchestration. There is a very related piece of [16] C. Guo, L. Yuan, D. Xiang, Y. Dang, R. Huang, D. Maltz, Z. Liu,
work named “Barefoot Deep Insight”, which is also built V. Wang, B. Pang, H. Chen et al., “Pingmesh: A large-scale system
for data center network latency measurement and analysis,” in ACM
upon INT and claims to enable end-to-end traffic monitor- SIGCOMM Computer Communication Review, vol. 45, no. 4. ACM,
ing [4]. However, Barefoot Deep Insight is a proprietary 2015, pp. 139–152.

Authorized licensed use limited to: Southeast University. Downloaded on October 19,2021 at 06:20:39 UTC from IEEE Xplore. Restrictions apply.