Scribd
Upload
372 views24 pages

BGP Convergence

This document discusses delayed convergence in internet routing. It finds that routing convergence following failures takes much longer than expected, around 10 minutes on average. Withdrawals propagate more slowly than announcements. Repairs converge faster than failures. Delayed convergence can cause connectivity issues, packet loss, and latency for several minutes after a fault occurs. The underlying cause is that distance vector routing protocols like BGP can explore invalid paths during convergence.

Uploaded by

sushmsn
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
372 views24 pages

BGP Convergence

This document discusses delayed convergence in internet routing. It finds that routing convergence following failures takes much longer than expected, around 10 minutes on average. Withdrawals propagate more slowly than announcements. Repairs converge faster than failures. Delayed convergence can cause connectivity issues, packet loss, and latency for several minutes after a fault occurs. The underlying cause is that distance vector routing protocols like BGP can explore invalid paths during convergence.

Uploaded by

sushmsn
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 24

Delayed Internet Routing

Convergence

Craig Labovitz, Microsoft Research


Abha Ahuja, University of Michigan
Farnam Jahanian, University of Michigan
Abhit Bose, University of Michigan
The Internet: Failure Analysis

Mostly seems to work

Something
happens.
Doesn’t work.
Time

Mostly seems to work


Motivation
 Routing reliability/fault-tolerance on small time
scales (minutes) not previously a priority
 Emerging transaction oriented and interactive

applications (e.g. Internet Telephony) will


require higher levels of end-to-end network
reliability
 How well does the Internet routing

infrastructure tolerate faults?


Conventional Wisdom
 Internet routing is robust under faults
 Supports path re-routing and restoral on the order of
seconds
 BGP has good convergence properties
 Does not exhibit looping/bouncing problems of RIP
 Internet fail-over will improve with faster routers
and faster links
 More redundant connections (multi-homing) to

Internet will always improve site fault-


tolerances
In this talk …
 We will show that most of the conventional
wisdom about routing convergence is not
accurate
 Measurement of BGP convergence in the Internet
 Analysis/Intuition behind delayed BGP routing

convergence
 Modifications to BGP implementations which would

improve convergence times


Open Question
After a fault in a path to multi-homed site, how
long does it take for the majority of Internet
routers to fail-over to the secondary path?
– Routing table
convergence
BGP Primary ISP
(backbone routers
reach steady-state)
Customer BG
P
after a fault
– End-to-end paths
Backup ISP
stable (“normal”
levels of loss and
TRAFFIC latency)
BGP: Bad news
 With unconstrained policies (Griffin99, Varadhan96)
 Divergence
 Possible create mutually unsatisfiable policies
 NP-complete to identify these policies in IRR
 Happening today?
 With constrained policies (e.g. shortest path first)
 Transient oscillations
 BGP usually converges
 It might just take a very long time …
 This talk is about constrained policies
How long until routes return?

What is happening here?


16 Month Study of Convergence
 Instrument the Internet
 Inject BGP faults (announcements/withdrawals) of

varied prefix and ASPath length into topologically


and geographically diverse ISP peering sessions
(Mae-West, Japan, Michigan, London)
 Monitor impact faults through

 Recording of default-free BGP peering

sessions with 20 tier1/tier2 ISPs


 Active ICMP measurements (512

byte/second to 100 random web sites)


 Wait two years (and 250,000 faults)
Figure 1:

Diagram of the fault injection and measurement infrastructure


Fault Scenarios
 Tup – a new route is advertised
 Tdown – A route is withdrawn (i.e. single-horned

failure)
 Tshort – Advertise a shorter/better ASPath (i.e.

primary path repaired)


 Tlong – Advertise a longer/worse ASPath (i.e.

primary path fails)


Major Convergence Results
 Routing convergence requires an order of
magnitude longer than expected (10s of minutes)
 Routes converge more quickly following Tup/Repair

than Tdown/Failure events (“bad news travels more


slowly”)
 Curiously, withdrawals (Tdown) generate several

times the number of announcements than


announcements (Tup)
Example:

 BGP log of updates from AS2117 for route via AS2129


 One BGP withdrawal triggers 6 announcements and one withdrawal
from 2117
 Increasing ASPath length until final withdrawal
Example:
How Many Announcements Does it Take For an AS to Withdraw
a Route?

Answer: up to 19
CDF of BGP Routing Table Convergence Times
100

90

er
80

ov
Cumulative Percentage of Events

70

Sh o o u te
ail-

er
rt F
60 Tup

v
R

-O
Lon New Tshort

ail
50
Tlong

gF
g- >
40 Tdow n

on
>L
30

r t-

re
o
20
Sh

ilu
Fa
10

0
0 20 40 60 80 100 120 140 160
Seconds Until Convergence
• Less than half of Tdown events converge within two minutes
• Tup/Tshort and Tdown/Tlong form equivalence classes
• Long tailed distribution (up to 15 minutes)
Failures, Fail-overs and Repairs
 Bad news does not travel fast…
 Repairs (Tup) exhibit similar convergence

properties as long-short ASPath fail-over


 Failures (Tdown) and short-long fail-overs (e.g.

primary to secondary path) also similar


 Slower than Tup (e.g. a repair)
 60% take longer than two minutes
 Fail-over times degrade the greater the degree of multi-homing
Impact of Delayed Convergence
 Why do we care about routing table
convergence?
 It deleteriously impacts end-to-end Internet paths
 ICMP experiment results
 Loss of connectivity, packet loss, latency, and packet re-
ordering for an average of 3-5 minutes after a fault
 Why? Routers drop packets for which they do not have a
valid next hop. Also problems with cache flushing in some
older routers
Intuition for Delayed BGP Convergence

 ICMP loss to 100 randomly chosen web sites with VIF source
address of our probe
 Tlong/Tshort exhibit similar relationship as before
Delayed Convergence Background
 Well known that distance vector protocols exhibit
poor convergence behaviors
 Counting to infinity, looping, bouncing problem
 RIP redefines infinity and adds split-horizon, poison
reverse, etc.
 Still, slow convergence and not scalable
 BGP advertises ASPaths instead of distance
 Solves counting to infinity and RIP looping problem, but …
 BGP can still explore “invalid” paths during convergence

(i.e. the bouncing problem)


Problems with Distance Vector Protocols
Counting to Infinity

R=5
R=3R=7

A B 1
A 2 B
R
B 2 A 2
R
R 73 R 15 R

Node Distance Node Distance


BGP Convergence Example
R
AS2
AS3

AS0 AS1

*B R via AS3 *B R via AS3 *B R via AS3


*B R via AS1,AS3 * B R via AS0,AS3 *B
*B R via
viaAS0,AS3
013
B R via AS2,AS3 *BB R via 203
AS2,AS3 B R via AS1,AS3
AS0 AS1 AS2
Intuition for Delayed BGP Convergence
 There exists possible ordering of messages such
that BGP will explore ALL possible ASPaths of
ALL possible lengths
 BGP is O(N!), where N number of default-free BGP speakers
in a complete graph with default policy
 Although seemingly very different protocols, BGP
and RIP share very similar convergence
behaviors. Major difference:
 RIP explores metrics (1 … N)
 BGP ASPath provides multiple ways to represent metric (path)
of length N, or (N-1)!
In real life …
 Discussed worst case BGP behavior
 In practice, BGP policy prevents worst case from

happening
 BGP timers also provide synchronization and

limits possible orderings of messages


Conclusion and Future Work
 Internet does not posses effective inter-domain
fail-over (15 minutes is a long time for a phone
call)
 Majority of BGP convergence delay due to vendor

implementation decisions of MinRouteAdver and


loop detection
 In practice, Internet is not a complete graph and

same degree of message re-ordering unlikely.


Our current work:
 What is the impact of ISP policy and topology on BGP
convergence?
 Can we improve BGP convergence times?

You might also like