IXP Design
IXP Design
ISP/IXP Workshops
Background
Why set up an IXP?
Layer 2 Exchange Point
Layer 3 Exchange Point
Design Considerations
Route Collectors & Servers
What can go wrong?
International bandwidth
Costs significantly more than domestic bandwidth
Congested with local traffic
Wastes money, harms performance
Solution:
Two competing ISPs peer with each other
Result:
Both save money
Local traffic stays local
Better network performance, better QoS,…
More international bandwidth for expensive international traffic
Everyone is happy
Solution:
Internet Exchange Point
Solution
Every ISP participates in the IXP
Cost is minimal – one local circuit covers all domestic traffic
International circuits are used for just international traffic – and
backing up domestic links in case the IXP fails
Result:
Local traffic stays local
QoS considerations for local traffic is not an issue
RTTs are typically sub 10ms
Customers enjoy the Internet experience
Local Internet economy grows rapidly
IXP Services:
TLD DNS, IXP
Management
Routing Registry Network
Looking Glass, Ethernet Switch
news, etc
ISP 1 ISP 3
ISP 2
IXP Services:
TLD DNS, IXP
Management
Routing Registry Network
Looking Glass, Ethernet Switches
news, etc
ISP 1 ISP 3
ISP 2
IXP Services:
TLD DNS, IXP
Management
Routing Registry Network
Looking Glass, IXP Router
news, etc
ISP 1 ISP 3
ISP 2
IXP Services:
TLD DNS, IXP
Management
Routing Registry Network
Looking Glass, IXP Routers
news, etc
ISP 1 ISP 3
ISP 2
ISP
IXP Services:
TLD DNS, IXP
Management
Routing Registry Network
IXP Routers
Looking Glass,
news, etc
ISP 1 ISP 3
ISP 2
Layer 3
IXP team requires good BGP knowledge
Rely on 3rd party for BGP configuration
Less freedom on who peers with whom
Usually competes with IXP membership
Tends to be distributed over wide area
Layer 2
IXP team does not need routing knowledge
Easy to get started
More complicated to distribute over wide area
ISPs free to set up peering agreements with each other as they
wish
Bi-Lateral Peering
Participants set up peering with each other according to their
own requirements and business relationships
This is the most common situation at IXPs today
ccTLD DNS
the country IXP could host the country’s top level DNS
e.g. “SE.” TLD is hosted at Netnod IXes in Sweden
Offer back up of other country ccTLD DNS
Root server
Anycast instances of I.root-servers.net, F.root-servers.net etc
are present at many IXes
Usenet News
Usenet News is high volume
could save bandwidth to all IXP members
Route Collector
Route collector shows the reachability information available at
the exchange
Technical detail covered later on
Looking Glass
One way of making the Route Collector routes available for
global view (e.g. www.traceroute.org)
Public or members only access
Content Redistribution/Caching
For example, Akamised update distribution service
Routing Registry
Used to register the routing policy of the IXP membership (more
later)
R3
R2 R4
R1 SWITCH R5
Route Collector
RS RS
RS
TRAFFIC FLOW
ROUTING INFORMATION FLOW
ISPs peer:
Directly with significant peers
With Route Server for the rest
An IXP:
is NOT a competition
is NOT a profit making business
Interconnected IXPs
IXP in one location believes it should connect directly to the IXP
in another location
Who pays for the interconnect?
How is traffic metered?
Competes with the ISPs who already provide transit between
the two locations (who then refuse to join IX, harming the
viability of the IX)
Metro interconnections work ok (e.g. LINX)
Actually a Disadvantage
IXP Operator has to know BGP
ISP not knowing BGP is big commercial disadvantage
ISPs who would like to have a growing successful business
need to be able to multi-home, peer with other ISPs, etc —
these activities require BGP
Fact:
IXP is only an Ethernet switch — setting up an iBGP mesh with
participants is no quicker than setting up an eBGP mesh
Actually a Disadvantage
ISP participants surrender control of:
Their border router; it is located in IXP’s AS
Their routing and peering policy
IXP operator is single point of failure
If they aren’t available 24x7, then neither is the IXP
BGP configuration errors by IXP operator have real impacts
on ISP operations
AUPs
Acceptable Use Policy
Minimal rules for connection
Fees?
Some IXPs charge no fee
Other IXPs charge cost recovery
A few IXPs are commercial
LINX in London, UK
TorIX in Toronto, Canada
AMS-IX in Amsterdam, Netherlands
SIX in Seattle, Washington, US
PA-IX in Palo Alto, California, US
JPNAP in Tokyo, Japan
DE-CIX in Frankfurt, Germany
…
All use Ethernet Switches
ISP Workshops © 2008 Cisco Systems, Inc. All rights reserved. 76
Features of IXPs (1)
Support
NOC to provide 24x7 support for problems at the exchange
Location
neutral co-location facilities
Address space
Peering LAN
AS Number
If using Route Collector/Server
http://www.ep.net/ep-main.html
Excellent resource for ip address allocation for exchanges,
locations of XPs in the world, AUPs and other policies
http://www.pch.net/documents
Another excellent resource of IXP locations, papers, IXP
statistics, etc
http://www.telegeography.com/ee/ix/index.php
A collection of IXPs and interconnect points for ISPs
ISP/IXP Workshops