Network Level Multihoming and BGP Challenges: Li Jia Helsinki University of Technology Jili@cc - Hut.fi
Network Level Multihoming and BGP Challenges: Li Jia Helsinki University of Technology Jili@cc - Hut.fi
Network Level Multihoming and BGP Challenges: Li Jia Helsinki University of Technology Jili@cc - Hut.fi
Li Jia
Helsinki University of Technology
[email protected]
routing information inside the AS[2]. On the other hand, AS901 can not be aggregated by its upstream ISPs - AS101
routers use an exterior gateway protocol to route packets and AS103 because AS901’s prefix is not included by that of
between ASes. BGP is an inter-Autonomous System rout- AS101 and AS103.
ing protocol [13]. It is used to exchange network reach- In Fig. 3, AS101 assigns a PA address block
ability information with other Autonomous Systems(ASes) "198.18.1.0/19" to AS901. For outgoing traffic, AS901
in TCP/IP networks. Based on the AS hop counts and the sends an announcement "198.18.1.0/19 901". AS101 can
preference level, BGP chooses the shortest route. When the combine the route of "198.18.1.0/19 901" with the one
reachability information is learned by an AS from the exte- announced by AS65500 and then sends an aggregated
rior, it will be distributed within the AS so that every router route announcement "198.18.0.0/16 101:901". By contrast,
in the AS could reach the routes advertised by the exterior. AS103 will send "198.18.1.0/19 103:901" because it cannot
When reachability information is exchanged between two aggregate the route announcement sent by AS901. For
routers located in different ASes, the protocol is referred to incoming traffic, routers forward the packets along the most
as external BGP(eBGP). When reachability information is specific route according to BGP protocol. The most specific
exchanged between routers inside the same AS, the protocol route refers to one that has smaller address range. Here the
is referred to as internal BGP(iBGP). most specific route is "198.18.1.0/19 103:901". Therefore,
Next, details about address management, routing process AS901 will receive all packets via AS103 unless links
and failure handling in multihoming networks and are dis- between AS901 and AS103 is not available. In this case,
cussed. links between AS901 and AS101 will be used to forward
In order to obtain multihoming using the BGP protocol, a traffic.
stub network must have:
1. A minimum address space identified by a 24-bit address
prefix or larger to deploy BGP multihoming.
2. An autonomous system number(ASN). Each AS must
have a unique ASN.
Two schemes exist to allocate address space: provider
independent address(PI address) and provider assigned ad-
dress(PA address). A registry such as ARIN has demon-
strated that a requirement for more than a /21 can request
a minimum of a /20 of IP address space directly form ARIN.
This type of IP address space is known as PI address. IP
subnets (also known as routes, prefixes, net blocks) can be
provided from an upstream ISP if the requirement for IP ad- Figure 2: Routing process of BGP using PI address
dresses is not sufficient. These subnets are commonly part
of a larger block of address space that the upstream ISP has
been assigned by ARIN. This type of IP address prefix is
known as PA space. Different issues arise when BGP multi-
homing is deployed adopting different address schemes.
PI addresses imply independence from a stub network’s
upstream providers. Due to the independence, routes with PI
addresses can not be aggregated by the upstream ISPs. This
leads to routing table overhead.
In a scenario of a network employing PA addresses, one
address management mechanism is to use only one address
block assigned by one of its upstream ISPs, which is called
the default address block. Other upstream ISPs maintain
a specific routing table entry for the route associated with
the default address block. This approach does not automati- Figure 3: Routing process of BGP using PA address: adopt
cally maintain the back-up routes. Another mechanism is to one address block
logically separate the whole stub network into several sub-
networks, each of which inherits a separate address prefix
from the upstream ISP closest to it[11]. The problem here is In Fig. 4, AS901 divides itself into two sub-networks and
the update of the routing table entries. gets the address block 198.18.1.0/19 and 65.3.10.0/19 from
Non-aggregated routes can be advertised across multi- the corresponding upstream ISPs respectively. Accordingly,
homing networks with PI address, and aggregated routes the traffic for these two sub-networks is aggregated by the
can be advertised with PA addresses[11]. For instance, in upstream ISPes - AS101 and AS103 respectively. The two
Fig. 2, AS65500 is a non-multihoming network and AS901 sub-networks are treated by the upstream ASes as separate
is a multihoming network. The route announcement of networks. That is to say, the upstream only accepts the
AS65500 "198.18.32.0/24 65500" is aggregated by its up- outgoing traffic with a prefix that is advertised to this AS
stream ISP - AS101 because 198.18.32.0/24 is a sub-block by the stub network. In this example, AS101 only accept
of 198.18.0.0/16. By contrast, the route announcement of traffic coming from 198.18.1.0/19. The problem is that if
TKK T-110.5190 Seminar on Internetworking 2008-04-28/29
•Traffic targeting 65.3.10.0/19 should primarily be deliv- whole network. But it uses NAT to assign multiple ad-
ered through AS103 and AS101 is used as a backup path. dresses.
•Traffic targeting 198.18.1.0/19 should primarily be deliv-
ered through AS101 and AS103 is used as a backup path.
3.2 Lack of Multipath Routing
A BGP router can receive multiple advertisements for the
same route from multiple upstream routers. For instance, in
Fig. 5, the router in AS198 received two advertisements for
the prefix 65.3.0.0/19. Thus, the router needs to run its BGP
decision process to select the best path. BGP protocol selects
only one best path. Accordingly, the BGP router advertises
to its peers the best route to any given destination.
This behavior causes at least two limitations. First, one
best route conflicts with the concept of load balancing. In
respond to this, some venders support multipath extensions
in their BGP implementations. Second, given that a BGP
Figure 5: Growth of BGP routing tables: lack of aggregation
router only advertises the best route, many alternative paths
and load balancing
that could have been potentially used will be unknown[10].
This introduces problems to the current interdomain routing
paradigm from the end-to-end quality of service(QoS) and
AS901 prepends its own AS number in its BGP adver-
traffic engineering(TE) viewpoints[3].
tisements with the aim of identifying the specific prefixes.
As mentioned earlier, the specific prefix implies the best Efforts have been put to this issue so that a BGP router can
route when the upstream ASes select routes. In this figure, advertise multiple routes for the same destination to its peers.
AS101 and AS103 are configured differently. AS101 prop- However, this mechanism will make the existing problem
agates the two BGP advertisements. AS103 sends an aggre- of BGP multihoming more difficult to tackle. For example,
gate advertisement for 198.18.0.0/16, since which includes multipath will increase the size of routing table dramatically,
198.18.1.0/19. As shown in Fig. 5, even though AS901 orig- which in future impacts scalability issue.
inates only two prefixes, AS198 receives four routes for three
different prefixes. Thus, the size of the BGP routing table is 3.3 Slow convergence
increased at AS198, since it receives more than one route for
the same prefix. Two BGP routers have to establish a BGP session to ex-
Despite the prepending operation, all traffic from AS901 change reachability information. This session is supported
toward AS198 will be routed via AS101, because: by a TCP connection through which the routers exchange
•The shortest path for 65.3.10.0/19 is via AS101. There- different types of messages:
fore, the traffic for 65.3.10.0/19 will be sent via AS101. •OPEN: to open a BGP session.
•The BGP router always prefer more specific prefix to for- •UPDATE: to transfer reachability information.
ward traffic. In this figure, 198.18.1.0/19 is more specific •NOTIFICATION: to identify an error detected. The BGP
than 198.18.0.0/16. In this case, AS103 will stop aggregat- session is shut down after this message is sent.
ing AS901’s prefix. This non-aggregation causes AS 103 to •KEEPALIVE: to verify that the peer is reachable.
advertise two prefixes to AS198. OPEN message can help to determine if the BGP session
To reduce the routing table size, different routes with com- corresponds to an iBGP or eBGP. When a session starts, each
mon characteristics can be aggregated into a single route. peer will advertise its entire set of routes. Then only incre-
However, a multihomed network inherits multiple IP pre- mental updates and KEEPALIVE messages are exchanged.
fixes from different upstream ASes and thus its prefixes can- Convergence time - the time required to reroute packets
not be aggregated by all the ASes. In this example, pre- when a failure happens, is an important performance metric
fix 198.18.1.0/19 belongs to AS 103, so this prefix cannot for a routing protocol. A study[8] shows that the conver-
be aggregated by another ISP(AS101). Another reason for gence time of BGP is rather slow. One important reason is
non-aggregation is that an AS may have to announce sev- that a single link failure can force BGP routers to exchange
eral prefixes due to address fragmentation, load balancing a large number of advertisements to explore for alternative
and failure to aggregate[4]. This example illustrates the non- paths toward the affected destinations. This problem is re-
aggregation caused by load balancing. ferred to as path exploration.
Most ISPs filter the advertisements of long prefixes to Routers may exchange several advertisements concerning
cope with the routing table problem. For example, some ISPs the same prefix in the process of BGP convergence. To avoid
do not allow advertising to the global Internet prefixes longer this problem, most BGP routers use a timer called minimum
than /22. However, this strategy does not tackle the root of route advertisement interval. The default value of this timer
the problem but just works around it. is 30 seconds. This method prevents BGP routers from send-
Some efforts are being made to deal with this prob- ing a new advertisement for the same prefix within 30 sec-
lem in IPv6. For IPv4, the problem is largely unsolved. onds. In this way, the number of BGP advertisements is re-
SIMPLER[7] forces address prefix aggregation over the duced. However, it introduces another problem - delay. In
TKK T-110.5190 Seminar on Internetworking 2008-04-28/29
some cases, important BGP advertisements are unnecessar- Internet connectivity solutions, multihoming still has many
ily delayed, which has important influence on the network unsolved challenges.
performance. The purpose of this paper is to review the deployment
Some new proposals have been brought up to solve this solutions available for multihoming and discuss challenges
problem. For example, BGP-RCN[12] reduces the number faced by BGP multihoming.
of BGP messages exchanged in the convergence by adding As an important interdomain routing protocol, BGP has
an identifier to each BGP message. This identifier indicates several limitations. These limitations are becoming more
the root-cause of the BGP message. When a failure hap- and more noticeable in the last few years due to the ex-
pens, distant routers can avoid selecting a path that is af- plosive growth of the network. Current research concen-
fected by the failure. However, this additional information is trates on scalability, route selection, convergence, Qos etc.
not built into BGP advertisements and is against the scalabil- In addition to the technical factors, routing management and
ity of BGP. policies performed by different ISPs also contribute to these
Another solution is the ghost-flushing[6]. This method problems. Usually, ISPs are reluctant to introduce changes if
improves the convergence by making the messages indi- there is no promising source of revenue. This increases the
cating bad news distributed quickly, while good news dis- difficulties to tackle the existing problems associated with
tributed slowly. However, it just tries to speed up the con- BGP.
vergence of BGP instead of tackling the root of the problem,
i.e., path exploration.
References
3.4 Lack of Qos Support [1] T. Bates and Y. Rekhter. Scalable Support for Multi-
Most of the studies of Qos were based on non-multihoming homed Multi-provider Connectivity. Technical Report
networks. BGP doesn’t have built-in Qos capabilities since 2260, 1998.
it was designed to exchange reachability information. Some [2] S. H. cisco Systems. Bgp4 case studies/tutorial. 1995.
applications, such as VoIP, require strong Qos to across
interdomain[5]. [3] B. H. et al. Distance metrics in the internet. 2002.
New proposals have been put forward in recent years, but
none has been appealing to be deployed in practice. One [4] GLBECOMM. On Characterizing BGP Routing Table
reason is that ISPs prefer over-provisioning their networks Growth, January 2002.
to manage Qos. More issues have to be considered be- [5] IEEE. Challenges in Enabling Interprovider Service
fore ISPs determine to use the Qos management mechanism. Quality in the Internet, June 2005.
Such considerations include the monetary cost to deploy and
maintain Qos and the possible new businesses that might be [6] IEEE INFOCOM. Improved BGP Convergence via
developed to tangible profit for ISPs etc. From the technical Ghost Flushing.
side, all the proposals referring to Qos have strong limita-
tions at the interdomain level. [7] IEEE INFOCOM. Practical Routing-Layer Support for
Scalable Multihoming, 2005.
3.5 Optimizing route selection [8] C. Labovitz, A. Ahuja, A. Bose, and F. Jahanian. De-
layed internet routing convergence. 9(3):293–306, June
Route optimizing refers to distribute the traffic among a stub 2001.
network’s multiple connections to the Internet. Two aspects
must be considered in order to select an optimizing route. [9] P. Morrissey. Route optimizers: Map-
First, the most qualified upstream provider must be chosen. ping out the best route. December 2003.
Second, the traffic should be leveraged among multiple con- http://www.networkcomputing.com/
nections, which refers to load balance problem. showitem.jhtml?docid=1425f2.
Selection for inbound traffic is difficult. Mechanisms to
implement load balance for outbound traffic are available, [10] Network,IEEE. Open Issues in Interdomain Routing: a
but no mechanism is available to implement load balance for Survey, November 2005.
inbound traffic without NAT. One limitation caused by NAT [11] Network,IEEE. A Survey of Multihoming Technology
is that it does not support non-client/server applications since in Stub Networks: Current Research and Open Issues,
it initially was designed in the context of client/server envi- May 2007.
ronment.
[12] D. Pei, M. Azuma, N. Nguyen, J. Chen, D. Massey,
and L. Zhang. Bgp-rcn: Improving bgp conver-
4 Conclusion gence through root cause notification. Technical report,
UCLA Computer Science Department, 2003.
Multihoming can help enterprises meet their Internet perfor-
mance, reliability, and redundancy goals. It also helps to [13] Y. Rekhter, T. Li, and S. Hares. A border gateway pro-
reduce dependency on a single provider, giving them dra- tocol 4 (bgp-4). Technical report, The Internet Engi-
matically greater opportunities for bandwidth cost control neering Task Force, January 2006.
and contract flexibility. Despite its promising role in future