Fortigate BGP Configuration and Debug Commands-2
Fortigate BGP Configuration and Debug Commands-2
Fortigate BGP Configuration and Debug Commands-2
BGP with two ISPs for multi-homing, each advertising default gateway and full routing table.
Uses route-map, prefix list, weight
Prevent our Fortigate from becoming a transit AS, do not advertise learned via eBGP routes.
Uses route-map, aspath-list
Force FG1 to advertise default route without having one in RIB and without using blackhole
routing. Uses default-originate
Limit announced connected routes to 3.3.3.3 only. Uses route-map with redistribution
Secure BGP session between ISP1 and FG3 with one way hash. Uses MD5 authentication
Make sure we can see received routing advertisements before and after any filtering is applied.
Uses soft reconfiguration
Set up BGP peering between FG3 and FG1 using loopback in FG3
Remotely Triggered Black Hole Routing configuration
Solution.
• Create prefix list to allow ONLY default route (0.0.0.0/0) and deny everything else.
• Now I can configure both BGP peers on FG3, including redistributing the connected
networks (here it is 10.10.10.1/32 of the loopback interface) to BGP:
Verification.
As remote peers are not configured yet, the status will be oscillating between Active and
Connect:
...
FG1, FG6.
BGP settings of both peers are almost identical (except local to each AS number and FG3
peering IP) so I will list just FG1. One note: unlike in FG3, which distributes into BGP the
directly connected loopback 10.10.10.1, I need both Fortigates here to advertise default route
0.0.0.0/0 which they don't have. As I mentioned in the Configuration Flow graph - BGP will
only advertise routes present in the active routing table (RIB) by default. The Fortigate has 2
ways to circumvent this BGP standard requirement: we can announce the default route
with capability-default-originate, and for other routes we can use set network-import-
check disable. But I am not using either of them here.
To satisfy this condition, I add blackhole route to the 0.0.0.0/0 route, in Cisco world it is called
"route to Null0". This adds 0.0.0.0/0 as static route which I can redistribute into BGP.
Note 1: Additionally, to simulate "Internet" IPs, I added 8.8.8.8 as loopback in both FG1 and
FG6 and redistribute them via redistribute connected.
Note 2: Important point I glossed over in FG3 is router-id. Fortigate (as well as Cisco and most
others) will take the highest IP address on the loopback interface available unless explicitly set.
In this specific setup I have 8.8.8.8 address on both FG1 and FG6 set on their loopbacks to
advertise them as "Internet" addresses to FG3. And this may cause a problem - if any BGP peer
detects its own router-id coming from the peer, the BGP session will be torn down with
NOTIFICATION sent. So, here it is a must, but generally is a good idea to set router-
id manually to unique IP address. I will add unique router-id to FG3 and FG6.
When such situation of duplicate router-id happens, Fortigate will show the error:
Verification
Note: to save me typing, I add this alias to show routing table:
# alias rt
Verification
First, let's see if the BGP peering with two ISPs has been established (yes, it has).
On FG3:
111
12.12.12.12 from 12.12.12.12 (1.1.1.1) <--- default route
from ISP1
Origin incomplete metric 0, localpref 100, weight 10, valid,
external, best <--- preferred because its weight is 10
Last update: Wed May 20 12:05:58 2020
<--- the 2nd ISP peer has weight not set, think 0
Now we need to make sure we advertise our network 10.10.10.1 to both peers:
This is what we advertise to ISP1:
The good news is that the route-map prepending AS 1680 to make ISP2 less preferred for our
network works.
Let's have a look at the work the prefix-list filtering is doing on FG3. The BGP debug should
show it:
First is to explicitly allow our own networks in outgoing advertisements and block everything
else. About blocking everything else - both prefix lists and ACLs have implicit deny any, so it is
not necessary to explicitly deny everything else.
0.0.0.0/0 ge 24 Matches any network with subnet mask of 24 bits or longer. Usually used
by uplink providers to block incoming routes which are too specific, for
preserving manageable size of the routing table.
10.0.0.0/8 Matches
13.13.0.0/16 ge Match any network prefix in range of 13.13.0.0/16 i.e. from 13.13.0.0 to
25 13.13.255.255, provided it also has bit mask length of 25 bits or longer.
E.g. 13.13.123.0/25, 13.13.123.128/25 but not 13.13.0.0/24
AS What matches
PATH
regex
^$ Local routes only. In other words - match routes with empty AS path.
.* All and any routes
^111$ Routes originating from a directly attached peer, i.e routes that have just one AS
number in their path. Here it is routes originated by ISP1 (AS 111).
_111$ Routes originated by the specified AS, but not necessarily learned directly from
the source AS. If the given AS 111 advertises its routes to say AS 333, and we
have peering with this AS 333, then we could learn routes from AS 333 that were
originated by AS 111 and their AS path would look 333 111, and they would be
matched. Also, we don't impose length limit on AS path here, so the path 777
999 333 111 would match as well.
_111_ Routes that passed on their way the specified AS, without looking in which order.
This will match routes with AS paths like: 333 111 777 999, 111 777, 111 (see
table above as _ will match as $ and ^ as well)
Now back to our FG3, let's create and apply AS path list filtering to advertise only our own nets
to the BGP peers.
edit "LocalRoutesOnly"
config rule
edit 1
set action permit
set regexp "^$"
next
end
next
end
Step 2. Create if needed (for ISP1) and/or edit existing route-map (for ISP2 there is
already prepend-out for prepending AS) that uses the aspath-list for matching.
edit "LocalRoutesOut"
config rule
edit 1
set match-as-path "LocalRoutesOnly"
next
end
next
end
Step 3. Finally, apply route-map LocalRoutesOut to ISP1 and refreshing BGP session with ISP2
to activate the changes.
Verification
ISP1, routes received from FG3:
So, matching our loopback networks 3.3.3.3/32, 10.10.10.1/32, and directly attached
12.12.12.0/24, and 13.13.13.0/24 with ACls will look like:
edit "allow-3.3.3.3-only"
config rule
edit 1
set prefix 3.3.3.3 255.255.255.255
unset ge
unset le
next
end
Step 2 Update/create route-map to use the prefix-list.
edit "redist-3.3.3.3-only"
config rule
edit 1
set match-ip-address "allow-3.3.3.3-only"
next
end
next
Step 3 Use this route-map.
config neighbor
edit "12.12.12.12"
set remote-as 111
set route-map-out "LocalOutACL"
set weight 10
set password secretsuperpassword
More interesting though is to see what happens when misconfiguration occurs.
Case 1 FG3 has the password set, FG1 has not. As BGP RFC requires, the peer with BGP
authentication enabled should drop and NOT acknowledge or give any other information when it
receives unauthenticated packet. So, basically BGP sessions times out:
FG1-AS111 #
BGP: 12.12.12.3-Outgoing [NETWORK] FD=21, Sock Status: 110-Connection
timed out
BGP: 12.12.12.3-Outgoing [FSM] State: Connect Event: 18
FG3-AS1680 #
BGP: 12.12.12.12-Outgoing [NETWORK] FD=23, Sock Status: 110-Connection
timed out
BGP: 12.12.12.12-Outgoing [FSM] State: Connect Event: 18
Case 2: One of the peers has wrong password set.
Well, here we have no clue from the Fortigate as well, just the same connection time out:
Still, the feature is there and we can enable with soft-reconfiguration enable.
• Loopback is an interface by all means, so you have to add security rules to allow traffic
(TCP port 179 in BGP's case) to/from it for BGP session to be established. The rule from
loopback outbound is enough for Fortigate to be BGP client, always establishing
connection to the peer.
• Loopback adds 1 routing hop so for eBGP sessions you have to enable eBGP multihop
for session to come up. You do it on the remote peer at least.
Configure FG3.
Configure security rule to allow outgoing from Loop2 connections:
Verification
To see that FG3 is indeed client and FG1 (12.12.12.12) is server for this peering:
The workflow for the operator should be: She receives alert on client's
network being attacked. She enters Null Fortigate and set static route to
the network choosing where border-wise she wants to block the
attacked network.
The solution here will adhere to the Remotely Triggered Black Hole
Filtering—destination Based And Source Based except that the final
step - routing "dummy" IP address to Null0 interface, which works in
Cisco, will not work in Fortigate - from trial and error, I had to route
such dummy IP to Loopback and thus drop packets on it. The Fortinet
documentation has no explanation for this, and no one I asked knew
the answer why it is so.
The issue with type blackhole static route is that when dynamically
learned via BGP route is allocated such blackhole-ed route as next hop,
the Fortigate does NOT install the learned network in RIB. I will try
opening ticket to may be find the answer, follow here if interested: BGP
does not install route in RIB if the next hop is a blackhole, RTBH
configuration
Configuration
While I configured all this using steps defined above, I will show the
final configuration with comments to see the whole picture better.
JLM-Edge:
config system interface
edit "port1"
set vdom "root"
set ip 217.132.10.15 255.255.255.0
set allowaccess ping
set type physical
set snmp-index 1
next
edit "Loop1"
set vdom "root"
set ip 10.10.10.15 255.255.255.255
set allowaccess ping
set type loopback
set snmp-index 6
====BGP====
Null:
#### OSPF
config router ospf
set router-id 10.10.10.14
config area
edit 0.0.0.0
next
end
config network
edit 1
set prefix 10.10.10.0 255.255.255.0
next
edit 2
set prefix 217.132.10.0 255.255.255.0
next
end
#### Interfaces
config system interface
edit "port1"
set vdom "root"
set ip 217.132.10.14 255.255.255.0
set allowaccess ping
set type physical
set snmp-index 1
edit "Loop1"
set vdom "root"
set ip 10.10.10.14 255.255.255.255
set type loopback
set snmp-index 6
#### BGP
#### Now I will use the devices set above to match them in route-map
and set communities
NYC-brdr:
config system interface
edit "port1"
set vdom "root"
set ip 11.11.11.2 255.255.255.0
set allowaccess ping
set type physical
set snmp-index 1
next
edit "port2"
set vdom "root"
set ip 217.132.10.12 255.255.255.0
set allowaccess ping
set type physical
set snmp-index 2
next
edit "Loop1"
set vdom "root"
set ip 10.10.10.12 255.255.255.255
set allowaccess ping
set type loopback
set snmp-index 6
next
end
#### OSPF
config router ospf
set router-id 10.10.10.12
config area
edit 0.0.0.0
next
end
config network
edit 1
set prefix 10.10.10.0 255.255.255.0
next
edit 2
set prefix 217.132.10.0 255.255.255.0
next
end
#### Static route to dummy IP
#### BGP
edit "blackhole-777"
config rule
edit 1
set action permit
set match "777" <-- Block on NYC only
next
end
next
edit "blackhole-888"
config rule
edit 1
set action permit
set match "888" <-- Block on all borders
next
end
next
edit "blackhole-999"
config rule
edit 1
set action permit
set match "999" <-- Block on LON border only
next
end
next
edit "972"
config rule
edit 1
set action permit
set match "972" <-- Explicit allow for the rest of the
networks
next
end
next
end
edit "Loop1"
set vdom "root"
set ip 10.10.10.13 255.255.255.255
set type loopback
set snmp-index 6
next
Cisco CSR1000:
CSR1000-LON#show run
version 15.6
hostname CSR1000-LON
! Loopbacks below simulate "Internet" routes, just for verification
later
interface Loopback1
ip address 22.22.1.1 255.255.255.255
!
interface Loopback2
ip address 22.22.2.2 255.255.255.255
!
interface GigabitEthernet1
ip address 22.22.22.1 255.255.255.0
negotiation auto
!
interface GigabitEthernet2
no ip address
shutdown
negotiation auto
!
router bgp 222
bgp log-neighbor-changes
neighbor 22.22.22.2 remote-as 1680
!
address-family ipv4
redistribute connected
neighbor 22.22.22.2 activate
neighbor 22.22.22.2 send-community
exit-address-family
!
# Interface
set interfaces ge-0/0/0 unit 0 family inet address 11.11.11.1/24
set interfaces lo0 unit 0 family inet address 11.11.1.1/32
set interfaces lo0 unit 0 family inet address 11.11.3.1/32
# BGP
set routing-options router-id 11.11.11.1
set routing-options autonomous-system 111
set protocols bgp group eBGP type external
set protocols bgp group eBGP local-address 11.11.11.1
set protocols bgp group eBGP peer-as 1680
set protocols bgp group eBGP neighbor 11.11.11.2 export connected-to-
bgp
set policy-options policy-statement connected-to-bgp from protocol
direct
set policy-options policy-statement connected-to-bgp then accept
# Firewall, defaults plus untrust zone configs to allow BGP and pings
set security policies from-zone trust to-zone trust policy default-
permit match source-address any
set security policies from-zone trust to-zone trust policy default-
permit match destination-address any
set security policies from-zone trust to-zone trust policy default-
permit match application any
set security policies from-zone trust to-zone trust policy default-
permit then permit
set security policies from-zone trust to-zone untrust policy default-
permit match source-address any
set security policies from-zone trust to-zone untrust policy default-
permit match destination-address any
set security policies from-zone trust to-zone untrust policy default-
permit match application any
set security policies from-zone trust to-zone untrust policy default-
permit then permit
set security policies from-zone untrust to-zone trust policy default-
deny match source-address any
set security policies from-zone untrust to-zone trust policy default-
deny match destination-address any
set security policies from-zone untrust to-zone trust policy default-
deny match application any
set security policies from-zone untrust to-zone trust policy default-
deny then deny
set security policies from-zone untrust to-zone untrust policy
default-permit match source-address any
set security policies from-zone untrust to-zone untrust policy
default-permit match destination-address any
set security policies from-zone untrust to-zone untrust policy
default-permit match application any
set security policies from-zone untrust to-zone untrust policy
default-permit then permit
set security zones security-zone trust tcp-rst
set security zones security-zone trust host-inbound-traffic system-
services ping
set security zones security-zone trust host-inbound-traffic protocols
bgp
set security zones security-zone untrust screen untrust-screen
# Allow BGP incoming (not enabled by default), but also ping only for
verification
set security zones security-zone untrust interfaces ge-0/0/0.0 host-
inbound-traffic system-services http
set security zones security-zone untrust interfaces ge-0/0/0.0 host-
inbound-traffic system-services https
set security zones security-zone untrust interfaces ge-0/0/0.0 host-
inbound-traffic system-services ssh
set security zones security-zone untrust interfaces ge-0/0/0.0 host-
inbound-traffic system-services telnet
set security zones security-zone untrust interfaces ge-0/0/0.0 host-
inbound-traffic system-services dhcp
set security zones security-zone untrust interfaces ge-0/0/0.0 host-
inbound-traffic system-services ping
set security zones security-zone untrust interfaces ge-0/0/0.0 host-
inbound-traffic protocols bgp
Verification
Make sure this blocked network is not advertised to our eBGP peers by
mistake:
LON-brdr # get router info bgp network 192.168.15.15/32
BGP routing table entry for 192.168.15.15/32
Paths: (1 available, best #1, table Default-IP-Routing-Table, not
advertised to EBGP peer) <-- Good, no eBGP ads to CSR1000
Not advertised to any peer
192.0.2.1 from 10.10.10.14 (10.10.10.14)
Origin incomplete metric 0, localpref 110, valid, internal, best
Community: no-export
Last update: Sun Aug 9 13:08:06 2020
References
• Config router bgp commands: https://docs.fortinet.com/document/fortigate/6.0.0/cli-
reference/722994/router-bgp
• Get commands for BGP https://docs.fortinet.com/document/fortigate/6.0.0/cli-
reference/67947/router-info-bgp
• Execute router set of commands: (https://docs.fortinet.com/document/fortigate/6.0.0/cli-
reference/381055/router-clear-bgp)