Mitigation using BGP flow spec

Matthieu Texier, Senior CE, Southern Europe

Yaroslav Rosomakho, Senior Channel CE, EMEA
 BGP flow spec

DEFINITION AND NETWORK

DESIGN

2
Proprietary and Confidential Information of Arbor Networks, Inc.
BGP flow spec in few words

• BGP flow spec is a MP BGP capability just like IPv4 or IP VPN,

IPv6, …
– Negotiated during BGP session establishment,
– Address Family Identifier (AFI) Subsequent Address Family Identifier
(SAFI)
• AFI 1 (IPv4) / SAFI 133 (flow spec) and 1 (IPv4) /134 (IPv4 VPN)
– Dedicated NLRI : Network Layer Reachability Information
• Opaque key transported by MP BGP and managed by control plan application
layer
– Allows to specify flow information via BGP NLRI
– Allows to define action associated to that flow
• Traffic rate in bytes per seconds (0 means black hole)
• Traffic action : start stop filter, apply sampling
• Redirect : redirect traffic to a IP VPN (Route Target)
• Traffic Marking
• RFC 5575
– Standard track 2009

Proprietary and Confidential Information of Arbor Networks, Inc.

BGP routing design
• This model is, in theory (RFC 5575), not allowed
Juniper JunOS :
type internal;
description "Arbor collector";
local-address x.x.x.y;
import arbor-in;
family inet {
unicast;
flow {
BGP sessions no-validate arbor-mitigation;
}
}
cluster n.n.n.n;
peer-as XYZ;
multipath multiple-as;
neighbor x.x.x.z;

7750 SR/XRS:
*A:PE26>config>router>bgp# flowspec-validate
- flowspec-validate
- no flowspec-validate

• Make sure you disable flow spec route validation

procedure
4
Proprietary and Confidential Information of Arbor Networks, Inc.
 BGP Flow Spec

PEAKFLOW IMPLEMENTATION

5
Proprietary and Confidential Information of Arbor Networks, Inc.
BGP flow spec
Implementation milestones

• Until release 5.7, Peakflow SP supports

BGP flow spec action rate limit and black
hole
– Allows similar behavior RTBH and S/RTBH:
source IP / destination IP (simple L3
blackhole)
– Allows traffic drop a flow using matching on
TCP/UDP ports, ICMP type, ICMP code, TCP
flags, packet lengths, DSCP, Fragment, …

Proprietary and Confidential Information of Arbor Networks, Inc.

Preparation for FlowSpec in Peakflow SP

• Enable Flow Specification capability for

router

Proprietary and Confidential Information of Arbor Networks, Inc.

FlowSpec mitigation in Peakflow SP

• New mitigation from Mitigation->Flow

Specification or from alert

Proprietary and Confidential Information of Arbor Networks, Inc.

FlowSpec filter definition in Peakflow SP

Proprietary and Confidential Information of Arbor Networks, Inc.

FlowSpec action definition in Peakflow SP

Proprietary and Confidential Information of Arbor Networks, Inc.

BGP flow spec
Implementation milestones

• Starting release 5.8, Peakflow SP

supports BGP flow spec action redirect
– Allows to redirect IP packets matching a flow
to an IP VPN for off ramp purposes

Proprietary and Confidential Information of Arbor Networks, Inc.

 BGP off ramp: today’s constraints
TMS as default route TMS as default route
VRF on ramp in dirty VRF
in dirty VRF
GRE on ramp

Data scrubbing Data scrubbing

Center VRF GRT Center

IP MPLS GRT @
GRT
Backbone
VRF
GRT
195.115.0.1
@

 BGP Off ramp: breaks managed object matching on BGP attributes

which makes issues with alerting and auto-mitigation impossible
 Off ramp update doesn’t preserve AS path nor communities
 VRF on ramp:
 Manual leaking and stating routing for each and every protected prefix
 Always challenging to use the same interface/vlan for GRP and on ramp traffic on
the server side
 GRE tunnel
 Scalability issues for GRE termination, service card issue
 GRE proliferation issue when we have several TMS’s
 Static route and GRE manual provisioning
 GRE troubleshooting: no real OAM, keep alive not always easy to use
Proprietary and Confidential Information of Arbor Networks, Inc.
BGP diversion using BGP flow spec
• BGP flow spec can be applied on a predefined set of routers
– Typically peering edge routers

Peakflow collector
TMS peer to dirty VRF TMS as peer to dirty VRF
BGP Flow Spec Peer
CE/PE CE/PE

Data scrubbing Data scrubbing

Center VRF GRT Center

B
IP/MPLS VRF
F
GRT @
Backbone GRT S Counter measure
VRF VRF return traffic

195.115.0.1 B
F
S
Main pro’s against today’s approach
– Can be automatically provisioned without any manual configuration and for whatever IP
being under attack
• no manual configuration like route leaking, static route configuration, …
• We do not impact global routing table for the return path of the clean traffic
– We are really surgical : only diversion of specific flows
• We can select traffic based on source/dest IP, TCP/UDP ports

Proprietary and Confidential Information of Arbor Networks, Inc.

BGP flow spec diversion workflow
DDoS Alert

BGP flow spec

Diversion (action redirect)

VRF

GRT
VRF B
F
GRT S
IP/MPLS VRF B
F
VRF B
Backbone GRT S
F B
GRT VRF
S F
GRT S

BGP Flow spec configured on peering edge routers only

BGP flow spec configured only on untrusted interface

Proprietary and Confidential Information of Arbor Networks, Inc.

Re-injection with BGP flow spec

 You just have nothing to do as you didn’t impact routing table

for diversion

TMS as default route TMS as default route

in dirty VRF in dirty VRF

Data scrubbing Data scrubbing

Center VRF GRT Center

VRF B
IP MPLS F @
GRT
Backbone GRT S
VRF
VRF
GRT B
195.115.0.1 F
S @

 BGP flow spec only applied on ingress untrusted interfaces

 BGP flow spec only on peering edge
 Make sure that protected customer or server is not attached to a peering
router

Proprietary and Confidential Information of Arbor Networks, Inc.

Re-injection with BGP flow spec: deep dive

• Main issue to be solved

– TMS is sending packets to remote host to
challenge their protocol stack
• Countermeasure Example : TCP syn auth, HTTP auth,
DNS auth, …
• With use of Patch Panel : Those packets are sent via
the off ramp interface attached to the dirty/off ramp
VRF
• With use of L3 forwarding in TMS: If a default route is
configured on the clean Interface, all packets including
challenge packets should be routed to on ramp
interface

Proprietary and Confidential Information of Arbor Networks, Inc.

TMS with L3 forwarding
TMS with default
route to the Potential malicious
On ramp Interface traffic
Counter measure
Return traffic
On Ramp Off Ramp Clean traffic

Default NH is
TMS
GRT VRF LBL or IP Lookup

TMS challenge packets will follow default route

Proprietary and Confidential Information of Arbor Networks, Inc.

TMS standard patch Panel
Ex: VRF with GRT route leaking

Route Reflector BGP sessions to

Export protected Peakflow
Prefixes (flow spec)
TMS as default route TMS as default route
in dirty VRF in dirty VRF

Data scrubbing Data scrubbing

Center VRF GRT Center

IP VRF B
F
GRT Backbone GRT S @
VRF VRF
GRT B
195.115.0.1
F
S @
VRF with GRT leaking
Import protected prefixes with TMS as NH

TMS challenge packets end up in the dirty VRF

Proprietary and Confidential Information of Arbor Networks, Inc.

TMS standard patch Panel
Ex: VRF with GRT route leaking

• Originally design to offer Internet access within

business customer VPN
• Behavior
– For packets coming in the VRF, if no route to
destination IP is found in the VRF, the router is doing
a extra lookup in the GRT and route packet according
to GRT
• We propose to enable this feature on the dirty
VRF to manage routing of challenge packets sent
from off ramp interface of the TMS
• This feature is available on both ALU 7750 (route
leaking) and Juniper (RIB-groups)

Proprietary and Confidential Information of Arbor Networks, Inc.

TMS standard patch Panel
Ex: VRF with GRT route leaking

Potential malicious
traffic
Counter measure
Return traffic
On Ramp Off Ramp Clean traffic

VRF off ramp routes

ISP protected prefixes
No route to Host with TMS as NH
GRT leaking (@IP victim server BGP NH = TMS)

D IP = victim
NH is TMS
GRT VRF LBL or IP Lookup

Proprietary and Confidential Information of Arbor Networks, Inc.

TMS standard patch Panel
Same as previous design with default route

Potential malicious
traffic
Counter measure
Return traffic
On Ramp Off Ramp Clean traffic

VRF off ramp routes

Policy Based Routing Default route TMS as next-hop
All ingress traffic
to GRT
D IP = victim
NH is TMS
GRT VRF LBL or IP Lookup

Proprietary and Confidential Information of Arbor Networks, Inc.

Appliance level offramp configuration

• Offramp peering needs to be done from

Peakflow SP

• For TMS-CGSE and TMS-ISA offramping

is configured at cluster level
Proprietary and Confidential Information of Arbor Networks, Inc.
Mitigation configuration

Proprietary and Confidential Information of Arbor Networks, Inc.

Caveats and limitations

• Flow Spec availability

– ALU 7750:
• R9 and above
• Full support and flexibility to enable flow spec on a
per interface basis
• IPv4 and IPv6
– Juniper:
• JunOS R7.2 (may be earlier version) and above
• Full support but flow spec rules applied to all router
interface
– Other vendors:
• Working on it…

24
Proprietary and Confidential Information of Arbor Networks, Inc.
FlowSpec in Wireshark
Matthieu Texier ([email protected]), our Senior EMEA CE,
developed Wireshark dissector for FlowSpec. It should
appear in an upcoming 1.11.0 build.
See
https://bugs.wireshark.org/bugzilla/show_bug.cgi?id=8691
for details.

25
Proprietary and Confidential Information of Arbor Networks, Inc.
FlowSpec in Wireshark

26
Proprietary and Confidential Information of Arbor Networks, Inc.
FlowSpec in Wireshark

27
Proprietary and Confidential Information of Arbor Networks, Inc.
BGP flow spec

FUTURE

Proprietary and Confidential Information of Arbor Networks, Inc.

Future
• Extend flow spec RFC 5575 to avoid IP VPN
provisioning
• Extend existing flow spec community to advertise BGP NH
for redirect traffic (today only Route Target is allowed)
• Originally planned as a new BGP TLV, there was a
consensus to change for an extended community (quicker
adoption)
– Redirect-to-ip (0x0800) (IANA assignment)
– Redirect-to-vrf (0x8008)
• As requested by working group (Robert Raszuk) an option to
mirror traffic has been added (community encoding)
• http://tools.ietf.org/html/draft-simpson-idr-flowspec-redirect-
02
• IPv6 support
– http://tools.ietf.org/html/draft-raszuk-idr-flow-spec-v6-
01

Proprietary and Confidential Information of Arbor Networks, Inc.

Thank you