Fortinet Auto Discovery VPN (ADVPN)

Stéphane HAMELIN – Support Engineering Team

© Copyright Fortinet Inc. All rights reserved.

 Change Log
Date Author
2018-06-28 S. Hamelin Added slide and reference for the “net-device” KB article [link]
Grey background color used for slides referring to the historical dialup behavior (equivalent to “net-device enable”)
2018-05-17 S. Hamelin IKEv1 aggressive-mode is supported as of FortiOS 6.0.1 [link]
As of 5.6.3 and 6.0: new “net-device” dialup phase1 setting [multiple slides added & modified]
2018-03-16 S. Hamelin PIM/Multicast is supported as of FortiOS 5.6.1 [link]
IKE debug filter supports multiple IP addresses as of FortiOS 6.0 [link]

Added the configuration snippets for Paris and Madrid Hubs

2017-09-14 S. Hamelin IKEv2 is supported as of FortiOS 5.6.1 [link]
ADVPN Hubs can be DNATed as of FortiOS 5.6.1 [link]

Added KB reference for this document

Added KB reference for scenario mixing ADVPN & non-ADVPN Spokes [link]
2017-02-01 S. Hamelin Added a setting in ADVPN Spoke configuration

2016-07-01 S. Hamelin Initial version for Fortinet Xperts Academy event

Where to find the latest version of this document ?

http://kb.fortinet.com/kb/documentLink.do?externalID=FD39360

2
IPsec VPN Topology
How to organize the collection of point-to-point IPsec virtual links
between all sites ?
Hub and Spoke
Hub nodes concentrate Spoke nodes in a Star topology
Spoke
site

Spoke
site

Hub
Site
Internet

Spoke
site
The simplest topology

Spoke
Spoke to Spoke traffic: site
- must go through the Hub (delay, latency)
- needlessly consume resources on Hub site (CPU, memory, Internet link)

4
Partial Mesh
Typically a Hub-and-Spoke topology with additional direct tunnels between some Spokes
Spoke
site

Spoke
site

Hub
Site
Internet

Spoke
site

Spoke
site

A middle ground between Hub-and-Spoke and Full-Mesh topologies

5
Full Mesh

Direct connectivity between all sites

N sites = N (N-1) / 2 tunnels

Internet

10 sites = 45 tunnels !

Efficient for Spoke-to-Spoke traffic

Complex configuration
Not scalable
6
Auto-Discovery VPN (as of FortiOS 5.4)

Direct connectivity between all sites

Static tunnels
Dynamic tunnels
(shortcuts)
« The simplicity of Hub & Spoke
with the
Internet
efficiency of Full-Mesh »

VPN configuration is as simple as configuring a simple Hub & Spoke setup

7
FortiOS ADVPN
On-demand tunnels between Spokes
Shortcut is triggered by data flowing through the Hub

Static tunnels

Spoke-B

Hub Internet

Spoke-A

9
Shortcut negotiation is orchestrated by the Hub

Static tunnels

Spoke-B

Hub Internet

Spoke-A

10
Shortcut tunnel is established between the Spokes

Static tunnels

Dynamic tunnel (shortcut) Spoke-B

Hub Internet

Spoke-A

11
Spoke-to-Spoke traffic flows through the shortcut

Static tunnels

Dynamic tunnel (shortcut) Spoke-B

Hub Internet

Spoke-A

12
Summary – ADVPN Sequence of Events
Spoke-A Hub Spoke-B

Encrypt IPsec flow

Forward (data plane)
SHORTCUT
Decrypt
OFFER
SHORTCUT IKE flow
QUERY Forward (control plane)

SHORTCUT
Forward REPLY

SHORTCUT NEGOTIATION

Encrypt
Decrypt
13
Fortinet Auto-Discovery VPN

Fortinet ADVPN is a proprietary solution solely based on IKE & IPsec

It is incompatible with Cisco DMVPN which relies on mGRE-over-IPsec and NHRP

IKE:

- IKEv1 main-mode is supported (pre-shared key & certificate authentication)

- IKEv1 aggressive-mode is supported as of FortiOS 6.0.1 (pre-shared key & certificate authentication)
- IKEv2 is supported as of FortiOS 5.6.1

- Both IPv4 & IPv6 are supported

Dynamic Routing:

- BGP and RIPv2/RIPng are supported

- PIM/Multicast is supported as of FortiOS 5.6.1

- OSPF and IS-IS are not supported

14
Fortinet Auto-Discovery VPN - FAQ
- Is it mandatory that the Hub runs FortiOS 5.4 (or newer) ?

Yes.

The Hub is responsible for triggering the shortcut OFFER and for relaying the shortcut
QUERY/REPLY messages between the Spokes.
The Hub must run at least FortiOS 5.4 if shortcuts are desired.

- Is it mandatory that all Spokes be FortiGate running FOS 5.4 (or newer) ?

No.

If a Spoke runs a firmware older than FortiOS 5.4 or if it is an IPsec Gateway from another vendor, it
can still participate to the Hub & Spoke architecture but it will not be able to negotiate shortcuts with
other Spokes.

Connecting ADVPN and non-ADVPN IPsec gateways on the same Hub’s phase1 requires specific
configuration on the Hub and the non-ADVPN gateways:
KB article http://kb.fortinet.com/kb/documentLink.do?externalID=FD40359
15
Fortinet Auto-Discovery VPN - FAQ
- Can some of my Spokes be NATed ?

Yes.
A shortcut can be established between two Spokes as long as one of the two is not NATed.
A shortcut cannot be established between two Spokes that are both NATed.

- Can the Hub be DNATed (VIP) ?

Support for the Hub being DNATed is supported as of FortiOS 5.6.1

- Can shortcuts be torn down when they are idle ?

Yes.
It can be configured per Spoke under the phase1 connected the Hub:

config vpn ipsec phase1-interface

edit <to-the-Hub>
set idle-timeout enable // default= disable
set idle-timeoutinterval <in_minutes> // default=15
// range=[10 ; 43200]
end 16
Fortinet Auto-Discovery VPN - FAQ
- I would like to spread the Spokes between my two ISPs (wan1, wan2).
Will the Spokes bound to the phase1 on wan1 be able to establish shortcuts with the Spokes bound
to the phase1 on wan2 ?

Yes.

No additional configuration is required to cover this scenario.

- I have two independent Hub & Spoke architectures (Hub-A H&S and Hub-B H&S). Is it possible to
establish shortcuts between the Spokes of Hub-A and the Spokes of Hub-B ?

Yes.

It requires that an IPsec tunnel be configured between Hub-A and Hub-B and that a specific ADVPN
setting be enabled on this tunnel (auto-discovery-forwarder).
This scenario is covered in the “Lab Workshop” described in this document.

17
ADVPN configuration
IPsec configuration
Dual Regions Hub & Spoke

Tunnel between Hubs

19
France Hub & Spoke Network

20
France Hub & Spoke Network

21
France Hub & Spoke Overlay subnet

10.10.10.1
Spoke_0 Spoke_1

Paris Paris
10.10.10.2 10.10.10.3

22
 ADVPN Hub configuration
config vpn ipsec phase1-interface
edit "Spoke"
auto-discovery-sender enable set type dynamic
set interface "wan1"
Indicates that when IPsec traffic transits the hub it should set proposal aes128-sha1
set auto-discovery-sender enable
send a SHORTCUT-OFFER to the initiator of the traffic set add-route disable
to indicate that it could perhaps establish a more direct set psksecret fortinet
connection (shortcut) 10.10.10.1/32 next
end
Spoke_0 Spoke_1
config vpn ipsec phase2-interface
add-route disable edit "Spoke"
set phase1name "Spoke"
ensures that IKE does not automatically add a route back set proposal aes128-sha1
next
over the spoke and instead leaves routing end
to a separately configured routing protocol
config system interface
Paris Paris edit "Spoke"
10.10.10.2/32 10.10.10.3/32 set ip 10.10.10.1/32
/32 set remote-ip 10.10.10.254
next
A /32 host IP address is used as the end
overlay IP for each ADVPN participant

The remote-ip is dummy

It can be any unused IP

23
ADVPN Hub configuration
 As of FortiOS 5.6.3 & 6.0 a new behavior is implemented
for route-based IPsec dialup tunnels
 This behavior is controlled by new CLI settings
Hub configuration

config vpn ipsec phase1-interface route-based

edit Spoke (aka, interface-mode)
set type dynamic
set net-device { disable* | enable }
set tunnel-search { selectors* | nexthop } New
( ... )
end

 This new behavior is detailed in KB article FD41498:

http://kb.fortinet.com/kb/documentLink.do?externalID=FD41498
24
 ADVPN Hub configuration
config vpn ipsec phase1-interface
edit "Spoke"
auto-discovery-sender enable set type dynamic
add-route disable set net-device disable
set tunnel-search nexthop
Covered in previous slide set interface "wan1"
set proposal aes128-sha1
net-device disable set auto-discovery-sender enable
10.10.10.1/24 set add-route disable
Default setting for dialup phase1 as of FortiOS 5.6.3 & 6.0 set psksecret fortinet
Spoke_0 Spoke_1 next
A dedicated interface is no longer created for each dialer
end
A shared interface is used instead
config vpn ipsec phase2-interface
tunnel-search nexthop edit "Spoke"
set phase1name "Spoke"
The next-hop IP of the route matched by a packet is used set proposal aes128-sha1
to decide into which tunnel the packet must be sent next
end
Paris Paris
config system interface
10.10.10.2/24 10.10.10.3/24 edit "Spoke"
set ip 10.10.10.1/32
In FortiOS 5.6.3 & 5.6.4, net-device and set remote-ip 10.10.10.254/24
tunnel-search settings cannot be modified next
after the phase1 was created end

This limitation is removed in FortiOS 6.0

and as of FortiOS 5.6.5

25
 ADVPN Hub configuration
config vpn ipsec phase1-interface
edit "Spoke"
/24 set type dynamic
The overlay IPs of all ADVPN participants are in the same set net-device disable
set tunnel-search nexthop
subnet set interface "wan1"
set proposal aes128-sha1
The mask for the local ip can only be /32 10.10.10.1/24
set auto-discovery-sender enable
set add-route disable
So, the mask for the overlay subnet set psksecret fortinet
must be specified in ‘remote-ip’ Spoke_0 Spoke_1 next
end
set ip 10.10.10.1/32
config vpn ipsec phase2-interface
Set remote-ip 10.10.10.254/24 edit "Spoke"
set phase1name "Spoke"
The remote-ip is an unused IP from the set proposal aes128-sha1
overlay subnet next
end
Paris Paris
config system interface
10.10.10.2/24 10.10.10.3/24 edit "Spoke"
set ip 10.10.10.1/32
set remote-ip 10.10.10.254/24
next
end

26
 ADVPN Hub configuration
config firewall policy
edit 1
set name "To Spokes"
set srcintf "internal"
set dstintf "Spoke"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always" 10.10.10.1
set service "ALL"
next Spoke_0 Spoke_1
edit 2
set name "From Spokes"
set srcintf "Spoke"
set dstintf "internal"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set service "ALL" Paris Paris
next
edit 3 10.10.10.2 10.10.10.3
set name "Spokes to Spokes"
set srcintf "Spoke"
set dstintf "Spoke"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set service "ALL"
next
end

27
 ADVPN Spoke configuration
config vpn ipsec phase1-interface
edit "Paris"
auto-discovery-receiver enable set interface "wan1"
set proposal aes128-sha1
set auto-discovery-receiver enable
To indicate that this IPsec tunnel wishes to participate
set add-route disable
in an Auto-Discovery VPN (i.e., receive a SHORTCUT-OFFER) set remote-gw 198.51.100.1
set psksecret fortinet
10.10.10.1 next
end
Spoke_0 Spoke_1
config vpn ipsec phase2-interface
edit "Paris"
set phase1name "Paris"
set proposal aes128-sha1
next
end

config system interface

Paris Paris edit "Paris"
set ip 10.10.10.2/32
10.10.10.2/32 10.10.10.3
set remote-ip 10.10.10.1
next
end

The remote-ip is the IP of the Hub

28
 ADVPN Spoke configuration
config vpn ipsec phase1-interface
edit "Paris"
auto-discovery-receiver enable set interface "wan1"
set proposal aes128-sha1
set auto-discovery-receiver enable
To indicate that this IPsec tunnel wishes to participate
set add-route disable
in an Auto-Discovery VPN (i.e., receive a SHORTCUT-OFFER) set remote-gw 198.51.100.1
set psksecret fortinet
10.10.10.1 next
end
/24 Spoke_0 Spoke_1
config vpn ipsec phase2-interface
The overlay IPs of all ADVPN participants are in the same edit "Paris"
subnet set phase1name "Paris"
set proposal aes128-sha1
next
The mask for the local ip can only be /32 end
So, the mask for the overlay subnet
must be specified in ‘remote-ip’ config system interface
Paris Paris edit "Paris"
set ip 10.10.10.2/32 set ip 10.10.10.2/32
10.10.10.2/24 10.10.10.3
Set remote-ip 10.10.10.1/24 set remote-ip 10.10.10.1/24
next
end
The remote-ip can be any other IP
in the overlay.
For clarity, the IP of the Hub is used

29
 ADVPN Spoke configuration
config firewall policy
edit 1
set name "To Hub/Spokes"
set srcintf "internal"
set dstintf "Paris"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always" 10.10.10.1
set service "ALL"
next Spoke_0 Spoke_1
edit 2
set name "From Hub/Spokes"
set srcintf "Paris"
set dstintf "internal"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set service "ALL" Paris Paris
next
end 10.10.10.2 10.10.10.3

No specific policies are needed

for traffic to/from other Spokes.

Traffic to/from other Spokes is

checked against the policies
to/from the Hub
30
ADVPN with BGP dynamic routing
iBGP with Route-Reflector
 iBGP – Route Reflector (RR) and RR-Clients
Paris # get router info bgp network
BGP table version is 4, local router ID is 10.10.10.1
Status codes: s suppressed, d damped, h history, * valid, > best,
i - internal,
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path BGP Route Reflector
*> 192.168.1.0 0.0.0.0 100 32768 i
*>i192.168.3.0 10.10.10.3 0 100 0 i 10.10.10.1
Total number of prefixes 2 Spoke_0 Spoke_1
BGP Update:
Prefix= 192.168.3.0/24
iBGP AS 65000 iBGP Next-Hop = 10.10.10.3

Paris Paris
10.10.10.2 10.10.10.3

RR-Client RR-Client

32
 iBGP – Route Reflector (RR) and RR-Clients
Paris # get router info bgp network
BGP table version is 4, local router ID is 10.10.10.1
Status codes: s suppressed, d damped, h history, * valid, > best,
i - internal,
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

BGP Route Reflector
*> 192.168.1.0 0.0.0.0 100 32768 i
*>i192.168.3.0 10.10.10.3 0 100 0 i 10.10.10.1
Total number of prefixes 2 Spoke_0 Spoke_1

BGP Update:
Prefix= 192.168.3.0/24 iBGP iBGP
Next-Hop = 10.10.10.3
France02 # get router info bgp network
BGP table version is 4, local router ID is 10.10.10.2 Paris Paris
Status codes: s suppressed, d damped, h history, * valid, > best,
i - internal, 10.10.10.2 10.10.10.3
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

*>i192.168.1.0 10.10.10.1 0 100 0 i
*> 192.168.2.0 0.0.0.0 100 32768 i
*>i192.168.3.0 10.10.10.3 0 100 0 i

Total number of prefixes 3

33
 iBGP Next Hop Reachability
On each Spoke, add a static route covering the ADVPN overlay :
config router static
edit 0
set dst 10.10.10.0 255.255.255.0
set device "Paris"
set comment "ADVPN overlay subnet"
BGP Route Reflector
next
end
10.10.10.1
Spoke_0 Spoke_1

BGP Next-Hop must be accessible

iBGP iBGP
through the tunnel
France02 # get router info bgp network
BGP table version is 4, local router ID is 10.10.10.2 Paris Paris
Status codes: s suppressed, d damped, h history, * valid, > best,
i - internal, 10.10.10.2/32 10.10.10.3
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

*>i192.168.1.0 10.10.10.1 0 100 0 i
*> 192.168.2.0 0.0.0.0 100 32768 i
*>i192.168.3.0 10.10.10.3 0 100 0 i

Total number of prefixes 2

34
 No shortcut – BGP Next-Hop is reached via the Hub
On each Spoke, add a static route covering the ADVPN overlay :
config router static
edit 0
set dst 10.10.10.0 255.255.255.0
set device "Paris"
set comment "ADVPN overlay subnet"
BGP Route Reflector
next
end
10.10.10.1
Spoke_0 Spoke_1
France02 # get router info routing-table details 10.10.10.3
Routing entry for 10.10.10.0/24
Known via "static", distance 10, metric 0, best
* 10.10.10.1, via Paris
iBGP iBGP

BGP Next-Hop of France03 Spoke (10.10.10.3)

is accessible via Paris Hub (10.10.10.1) Paris Paris
10.10.10.2/32 10.10.10.3

35
 iBGP Next Hop Reachability
The ADVPN overlay subnet is defined on the tunnel interface:
config system interface
edit "Paris"
set ip 10.10.10.2 255.255.255.255
set remote-ip 10.10.10.1 255.255.255.0 BGP Route Reflector
next
end 10.10.10.1
Spoke_0 Spoke_1

BGP Next-Hop must be accessible

iBGP iBGP
through the tunnel
France02 # get router info bgp network
BGP table version is 4, local router ID is 10.10.10.2 Paris Paris
Status codes: s suppressed, d damped, h history, * valid, > best,
i - internal, 10.10.10.2/24 10.10.10.3
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

*>i192.168.1.0 10.10.10.1 0 100 0 i
*> 192.168.2.0 0.0.0.0 100 32768 i
*>i192.168.3.0 10.10.10.3 0 100 0 i

Total number of prefixes 2

36
 No shortcut – BGP Next-Hop is reached via the Hub
The ADVPN overlay subnet is defined on the tunnel interface:

France02 # get router info routing-table connected

(…)
C 10.10.10.0/24 is directly connected, Paris BGP Route Reflector
(…)
10.10.10.1
Spoke_0 Spoke_1
France02 # get router info routing-table details 10.10.10.3
Routing entry for 10.10.10.0/24
Known via "connected", distance 0, metric 0, best
* is directly connected, Paris
iBGP iBGP

BGP Next-Hop of France03 Spoke (10.10.10.3)

is accessible via Paris’ connected subnet Paris Paris
10.10.10.2/24 10.10.10.3

37
 No shortcut – Recursive RIB lookup
France02 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default BGP Route Reflector
S* 0.0.0.0/0 [10/0] via 198.51.100.254, wan1 10.10.10.1
S 10.10.10.0/24 [10/0] via 10.10.10.1, Paris Spoke_0 Spoke_1
C 10.10.10.1/32 is directly connected, Paris
C 10.10.10.2/32 is directly connected, Paris

B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 02:01:11

C 192.168.2.0/24 is directly connected, internal iBGP iBGP
B 192.168.3.0/24 [200/0] via 10.10.10.3 (recursive via 10.10.10.1), 00:16:29

C 198.51.100.0/24 is directly connected, wan1

Paris Paris
10.10.10.2/32 10.10.10.3

Spoke-to-Spoke traffic flows through the Hub

38
 No shortcut – RIB lookup
France02 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default BGP Route Reflector
S* 0.0.0.0/0 [10/0] via 198.51.100.254, wan1 10.10.10.1
C 10.10.10.0/24 is directly connected, Paris Spoke_0 Spoke_1
is directly connected, Paris

B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 03:51:14

C 192.168.2.0/24 is directly connected, port1
B 192.168.3.0/24 [200/0] via 10.10.10.3, Paris, 00:16:20 iBGP iBGP
B 192.168.4.0/24 [200/0] via 10.10.10.4, Paris, 00:16:20
B 192.168.5.0/24 [200/0] via 10.10.10.5, Paris, 00:16:20

C 198.51.100.0/24 is directly connected, wan1 Paris Paris

10.10.10.2/24 10.10.10.3

Spoke-to-Spoke traffic flows through the Hub

39
 With shortcut – ADVPN Overlay
France02 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default BGP Route Reflector
S* 0.0.0.0/0 [10/0] via 198.51.100.254, wan1 10.10.10.1
S 10.10.10.0/24 [10/0] via 10.10.10.1, Paris Spoke_0 Spoke_1
C 10.10.10.1/32 is directly connected, Paris
C 10.10.10.2/32 is directly connected, Paris
is directly connected, Paris_0
C 10.10.10.3/32 is directly connected, Paris_0 iBGP iBGP
B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 02:38:15
C 192.168.2.0/24 is directly connected, internal
B 192.168.3.0/24 [200/0] via 10.10.10.3, Paris_0, 00:00:28 shorcut
Paris Paris
C 198.51.100.0/24 is directly connected, wan1 10.10.10.2 10.10.10.3

Paris_0 Paris_0
Shortcut name = <phase1name>_<index>

40
 With shortcut – ADVPN Overlay

France02 # diag ip address list | grep Paris

BGP Route Reflector
IP=10.10.10.2->10.10.10.1/255.255.255.255 index=15 devname=Paris
IP=10.10.10.2->10.10.10.3/255.255.255.255 index=19 devname=Paris_0 10.10.10.1
Spoke_0 Spoke_1

The same overlay IP is assigned to:

iBGP iBGP
- the tunnel toward the Hub
- the shortcut(s) toward other Spoke(s)
shorcut
Paris Paris
10.10.10.2 10.10.10.3

Paris_0 Paris_0

41
 With shortcut – BGP Next-Hop is directly connected
France02 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default BGP Route Reflector
S* 0.0.0.0/0 [10/0] via 198.51.100.254, wan1 10.10.10.1
S 10.10.10.0/24 [10/0] via 10.10.10.1, Paris Spoke_0 Spoke_1
C 10.10.10.1/32 is directly connected, Paris
C 10.10.10.2/32 is directly connected, Paris
is directly connected, Paris_0
C 10.10.10.3/32 is directly connected, Paris_0 iBGP iBGP
B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 02:38:15
C 192.168.2.0/24 is directly connected, internal
B 192.168.3.0/24 [200/0] via 10.10.10.3, Paris_0, 00:00:28 shorcut
Paris Paris
C 198.51.100.0/24 is directly connected, wan1 10.10.10.2 10.10.10.3

Paris_0 Paris_0
The BGP Next-Hop of France03
is directly connected over the shortcut
interface

42
 With shortcut – RIB lookup
France02 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default BGP Route Reflector
S* 0.0.0.0/0 [10/0] via 198.51.100.254, wan1 10.10.10.1
S 10.10.10.0/24 [10/0] via 10.10.10.1, Paris Spoke_0 Spoke_1
C 10.10.10.1/32 is directly connected, Paris
C 10.10.10.2/32 is directly connected, Paris
is directly connected, Paris_0
C 10.10.10.3/32 is directly connected, Paris_0 iBGP iBGP
B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 02:38:15
C 192.168.2.0/24 is directly connected, internal
B 192.168.3.0/24 [200/0] via 10.10.10.3, Paris_0, 00:00:28 shorcut
Paris Paris
C 198.51.100.0/24 is directly connected, wan1 10.10.10.2 10.10.10.3

Paris_0 Paris_0
Spoke-to-Spoke traffic flows through the shortcut

43
ADVPN configuration
BGP configuration
 Hub configuration = iBGP Route Reflector (RR)
Paris # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.1
config neighbor-group BGP Route Reflector
edit "advn_peers" 10.10.10.1
set remote-as 65000
set route-reflector-client enable Spoke_0 Spoke_1
next
end
config neighbor-range
edit 1 iBGP AS 65000 iBGP
set prefix 10.10.10.0 255.255.255.0
set neighbor-group "advn_peers"
next Paris overlay Paris
end 10.10.10.0/24
10.10.10.2 10.10.10.3
config network
edit 1
set prefix 192.168.1.0 255.255.255.0
next
RR-Client RR-Client
end
end

45
 Hub configuration = iBGP Route Reflector (RR)
Paris # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.1
config neighbor-group BGP Route Reflector
edit "advn_peers" 10.10.10.1
set remote-as 65000
set route-reflector-client enable Spoke_0 Spoke_1
next
end
config neighbor-range
edit 1 iBGP AS 65000 iBGP
set prefix 10.10.10.0 255.255.255.0
set neighbor-group "advn_peers"
next Paris overlay Paris
end 10.10.10.0/24
10.10.10.2 10.10.10.3
config network
edit 1
set prefix 192.168.1.0 255.255.255.0
next
RR-Client RR-Client
end
end

46
 Hub configuration = iBGP Route Reflector (RR)
Paris # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.1
config neighbor-group BGP Route Reflector
edit "advn_peers" 10.10.10.1
set remote-as 65000
set route-reflector-client enable Spoke_0 Spoke_1
next
end
config neighbor-range
edit 1 iBGP AS 65000 iBGP
set prefix 10.10.10.0 255.255.255.0
set neighbor-group "advn_peers"
next Paris overlay Paris
end 10.10.10.0/24
10.10.10.2 10.10.10.3
config network
edit 1
set prefix 192.168.1.0 255.255.255.0
next
RR-Client RR-Client
end
end

47
 Hub configuration = iBGP Route Reflector (RR)
Paris # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.1
config neighbor-group BGP Route Reflector
edit "advn_peers" 10.10.10.1
set remote-as 65000
set route-reflector-client enable Spoke_0 Spoke_1
next
end
config neighbor-range
edit 1 iBGP AS 65000 iBGP
set prefix 10.10.10.0 255.255.255.0
set neighbor-group "advn_peers"
next Paris overlay Paris
end 10.10.10.0/24
10.10.10.2 10.10.10.3
config network
edit 1
set prefix 192.168.1.0 255.255.255.0
next
RR-Client RR-Client
end
end

48
 Hub configuration = iBGP Route Reflector (RR)
Paris # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.1
config neighbor-group BGP Route Reflector
edit "advn_peers" 10.10.10.1
set remote-as 65000
set route-reflector-client enable Spoke_0 Spoke_1
next
end
config neighbor-range
edit 1 iBGP AS 65000 iBGP
set prefix 10.10.10.0 255.255.255.0
set neighbor-group "advn_peers"
next Paris overlay Paris
end 10.10.10.0/24
10.10.10.2 10.10.10.3
config network
edit 1
set prefix 192.168.1.0 255.255.255.0
next
RR-Client RR-Client
end
end

49
 Spoke configuration = iBGP RR-Client
France02 # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.2
config neighbor BGP Route Reflector
edit "10.10.10.1" 10.10.10.1
set remote-as 65000
next Spoke_0 Spoke_1
end
config network
edit 1
set prefix 192.168.2.0 255.255.255.0 iBGP AS 65000 iBGP
next
end
end Paris overlay Paris
10.10.10.0/24
10.10.10.2 10.10.10.3

RR-Client RR-Client

50
 Spoke configuration = iBGP RR-Client
France02 # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.2
config neighbor BGP Route Reflector
edit "10.10.10.1" 10.10.10.1
set remote-as 65000
next Spoke_0 Spoke_1
end
config network
edit 1
set prefix 192.168.2.0 255.255.255.0 iBGP AS 65000 iBGP
next
end
end Paris overlay Paris
10.10.10.0/24
10.10.10.2 10.10.10.3

RR-Client RR-Client

51
 Spoke configuration = iBGP RR-Client
France02 # show router bgp
config router bgp
set as 65000
set router-id 10.10.10.2
config neighbor BGP Route Reflector
edit "10.10.10.1" 10.10.10.1
set remote-as 65000
next Spoke_0 Spoke_1
end
config network
edit 1
set prefix 192.168.2.0 255.255.255.0 iBGP AS 65000 iBGP
next
end
end Paris overlay Paris
10.10.10.0/24
10.10.10.2 10.10.10.3

RR-Client RR-Client

52
Useful CLI commands
IPsec
Useful CLI commands – IPsec
France02 # diag ip address list
<...>
IP=10.10.10.2->10.10.10.1/255.255.255.255 index=15 devname=Paris List of IP addresses
<...>

overlay local-ip and remote-ip

France02 # diag vpn ike gateway list

vd: root/0
name: Paris
version: 1
interface: port2 4
addr: 198.51.100.2:500 -> 198.51.100.1:500
virtual-interface-addr: 10.10.10.2 -> 10.10.10.1
created: 231s ago
auto-discovery: 2 receiver List all the IKE SA (“phase1 up”)
IKE SA: created 1/1 established 1/1 time 0/0/0 ms
IPsec SA: created 1/1 established 1/1 time 0/0/0 ms

id/spi: 18 4886536f45c96972/07719003173cab1d
direction: initiator
status: established 231-231s ago = 0ms
proposal: aes128-sha1
key: 9ae438a4e65febe2-ad76a0143b49344b
lifetime/rekey: 86400/85868
DPD sent/recv: 00000000/00000000

54
Useful CLI commands – IPsec
France02 # diag vpn tunnel list
list all ipsec tunnel in vd 0
------------------------------------------------------
name=Paris ver=1 serial=1 198.51.100.2:0->198.51.100.1:0
bound_if=4 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/0
proxyid_num=1 child_num=0 refcnt=21 ilast=1 olast=1 auto-discovery=2
stat: rxp=63 txp=58 rxb=8424 txb=3627
dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=1 List all the IPsec SA (“phase2/tunnel up”)
natt: mode=none draft=0 interval=0 remote_port=0
proxyid=Paris proto=0 sa=1 ref=2 serial=1 adr
src: 0:0.0.0.0/0.0.0.0:0
dst: 0:0.0.0.0/0.0.0.0:0
SA: ref=3 options=2e type=00 soft=0 mtu=1438 expire=42941/0B replaywin=2048 seqno=3b esn=0 replaywin_lastseq=00000040
life: type=01 bytes=0/0 timeout=43172/43200
dec: spi=8135a40e esp=aes key=16 97e4ad16da7bb2a3880b65311da8e50e
ah=sha1 key=20 968a42ff561c7ea5fd4d8a94546c09f4812f0dcf
enc: spi=e77918f5 esp=aes key=16 0143920846bd2087bb56250e855a4352
ah=sha1 key=20 328d6e7713f2beb8d44ec4944ca868258236c812
dec:pkts/bytes=63/4278, enc:pkts/bytes=58/7488

France02 # get vpn ipsec tunnel summary

'Paris' 198.51.100.1:0 selectors(total,up): 1/1 rx(pkt,err): 66/0 tx(pkt,err): 60/0

France02 # diag vpn ike status detailed

vd: root/0
name: Paris
version: 1
connection: 1/12
IKE SA: created 1/19 established 1/19 times 0/2/10 ms
IPsec SA: created 1/27 established 1/27 times 0/1/10 ms

55
Dual Regions Hub & Spoke

Tunnel between Hubs

56
Useful CLI commands – IPsec
[root:~]# ping 192.168.3.1
PING 192.168.3.1 (192.168.3.1): 56 data bytes
64 bytes from 192.168.3.1: icmp_seq=0 ttl=252 time=1.1 ms
64 bytes from 192.168.3.1: icmp_seq=1 ttl=253 time=0.6 ms
64 bytes from 192.168.3.1: icmp_seq=2 ttl=253 time=0.5 ms Ping from France02 LAN to France03 LAN
64 bytes from 192.168.3.1: icmp_seq=3 ttl=253 time=0.3 ms
64 bytes from 192.168.3.1: icmp_seq=4 ttl=253 time=0.4 ms

--- 192.168.3.1 ping statistics ---

5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.3/0.5/1.1 ms

France02 # get vpn ipsec tunnel summary

'Paris_0' 198.51.100.3:0 selectors(total,up): 1/1 rx(pkt,err): 6/0 tx(pkt,err): 6/0 Shortcut to France03
'Paris' 198.51.100.1:0 selectors(total,up): 1/1 rx(pkt,err): 125/0 tx(pkt,err): 113/0

[root:~]# ping 192.168.102.1

PING 192.168.102.1 (192.168.102.1): 56 data bytes
64 bytes from 192.168.102.1: icmp_seq=0 ttl=251 time=1.8 ms
64 bytes from 192.168.102.1: icmp_seq=1 ttl=253 time=0.7 ms
64 bytes from 192.168.102.1: icmp_seq=2 ttl=253 time=0.7 ms Ping from France02 LAN to Spain102 LAN
64 bytes from 192.168.102.1: icmp_seq=3 ttl=253 time=0.8 ms
64 bytes from 192.168.102.1: icmp_seq=4 ttl=253 time=0.7 ms

--- 192.168.102.1 ping statistics ---

5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.7/0.9/1.8 ms

France02 # get vpn ipsec tunnel summary

'Paris_0' 198.51.100.3:0 selectors(total,up): 1/1 rx(pkt,err): 7/0 tx(pkt,err): 7/0
'Paris_1' 203.0.113.102:0 selectors(total,up): 1/1 rx(pkt,err): 5/0 tx(pkt,err): 5/0 Shortcut to Spain102
'Paris' 198.51.100.1:0 selectors(total,up): 1/1 rx(pkt,err): 134/0 tx(pkt,err): 121/0

57
Useful CLI commands – IPsec
[root:~]# ping 192.168.101.1
PING 192.168.101.1 (192.168.101.1): 56 data bytes
64 bytes from 192.168.101.1: icmp_seq=0 ttl=252 time=1.1 ms
64 bytes from 192.168.101.1: icmp_seq=1 ttl=253 time=0.8 ms
64 bytes from 192.168.101.1: icmp_seq=2 ttl=253 time=0.7 ms Ping from France02 LAN to Madrid LAN
64 bytes from 192.168.101.1: icmp_seq=3 ttl=253 time=0.7 ms
64 bytes from 192.168.101.1: icmp_seq=4 ttl=253 time=0.5 ms

--- 192.168.101.1 ping statistics ---

5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.5/0.7/1.1 ms

France02 # get vpn ipsec tunnel summary

'Paris_0' 198.51.100.3:0 selectors(total,up): 1/1 rx(pkt,err): 8/0 tx(pkt,err): 10/0
'Paris_1' 203.0.113.102:0 selectors(total,up): 1/1 rx(pkt,err): 7/0 tx(pkt,err): 8/0
'Paris_2' 203.0.113.1:0 selectors(total,up): 1/1 rx(pkt,err): 5/0 tx(pkt,err): 5/0 Shortcut to Madrid
'Paris' 198.51.100.1:0 selectors(total,up): 1/1 rx(pkt,err): 144/0 tx(pkt,err): 130/0

France02 # diag ip address list

<...>
IP=10.10.10.2->10.10.10.1/255.255.255.255 index=15 devname=Paris
IP=10.10.10.2->10.10.10.3/255.255.255.255 index=39 devname=Paris_0
IP=10.10.10.2->10.20.20.2/255.255.255.255 index=40 devname=Paris_1
IP=10.10.10.2->10.255.255.2/255.255.255.255 index=41 devname=Paris_2
<...>

58
Useful CLI commands – IPsec
France02 # diag vpn ike status detailed
vd: root/0
name: Paris
version: 1
used-indices: 0-2
connection: 4/15
IKE SA: created 4/22 established 4/22 times 0/2/10 ms
IPsec SA: created 4/30 established 4/30 times 0/1/10 ms

France02 # diag netlink interface list

<...>
if=Paris family=00 type=768 index=15 mtu=1438 link=0 master=0
ref=24 state=off start fw_flags=0 flags=up p2p run noarp multicast

if=Paris_0 family=00 type=768 index=39 mtu=1438 link=15 master=0

ref=21 state=off start fw_flags=0 flags=up p2p run noarp multicast
List of IPsec interfaces
if=Paris_1 family=00 type=768 index=40 mtu=1438 link=15 master=0
ref=21 state=off start fw_flags=0 flags=up p2p run noarp multicast

if=Paris_2 family=00 type=768 index=41 mtu=1438 link=15 master=0

ref=25 state=off start fw_flags=0 flags=up p2p run noarp multicast
<...>

France02 # diag vpn ike gateway flush name Paris_2 Bring down a shortcut
France02 # get vpn ipsec tunnel summary Shortcuts cannot be flushed
'Paris_0' 198.51.100.3:0 selectors(total,up): 1/1 rx(pkt,err): 8/0 tx(pkt,err): 10/0
'Paris_1' 203.0.113.102:0 selectors(total,up): 1/1 rx(pkt,err): 7/0 tx(pkt,err): 8/0 via the GUI, use the CLI
'Paris' 198.51.100.1:0 selectors(total,up): 1/1 rx(pkt,err): 144/0 tx(pkt,err): 130/0

59
Useful CLI commands – IPsec
As of 6.0, multiple IP addresses can be specified to filter the IKE debug (mdst-addr4)
It simplifies the debugging of Spoke-to-Spoke shortcut negotiations:
# From Spoke-A, check the shortcut negotiation with Spoke-B (which initially passes through the Hub)
diag debug console timestamp enable
diag vpn ike log filter clear
diag vpn ike log filter mdst-addr4 <ip.of.Hub> <ip.of.Spoke-B>
diag debug application ike -1
diag debug application fnbamd -1 # only if certificate authentication is used
diag debug enable

Up to 5.6, a single IP address can be specified to filter the IKE debug (dst-addr4)
Spoke-to-Spoke shortcut negotiations must therefore be investigated in two phases:
- 1st phase: investigate the Spoke-to-Hub negotiation which takes place at the beginning of the shortcut setup
- 2nd phase: investigate the Spoke-to-Spoke negotiation during another failing shortcut setup
diag debug console timestamp enable
diag vpn ike log filter clear
diag vpn ike log filter dst-addr4 <ip.of.Hub or ip.of.Spoke-B>
diag debug application ike -1
diag debug application fnbamd -1 # only if certificate authentication is used
diag debug enable

60
Useful CLI commands
Routing
Useful CLI commands – Routing
France02 # get router info bgp summary
BGP router identifier 10.10.10.2, local AS number 65000
BGP table version is 2
2 BGP AS-PATH entries
0 BGP community entries

Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd

10.10.10.1 4 65000 2061 2049 2 0 0 00:00:23 4 BGP peers
Total number of neighbors 1

France02 # get router info bgp network

BGP table version is 2, local router ID is 10.10.10.2
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

*>i192.168.1.0 10.10.10.1 0 100 0 i BGP table
*> 192.168.2.0 0.0.0.0 100 32768 i
*>i192.168.3.0 10.10.10.3 0 100 0 i
*>i192.168.101.0 10.255.255.2 0 100 0 65100 i
*>i192.168.102.0 10.20.20.2 0 100 0 65100 i

Total number of prefixes 5

62
Useful CLI commands – Routing
France02 # get router info bgp network 192.168.102.0
BGP routing table entry for 192.168.102.0/24
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Not advertised to any peer BGP details of a specific prefix
65100
10.20.20.2 from 10.10.10.1 (10.10.10.1)
Origin IGP metric 0, localpref 100, valid, internal, best
Last update: Tue Jun 21 17:03:02 2016

France02 # get router info routing-table bgp

B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 00:01:03
B 192.168.3.0/24 [200/0] via 10.10.10.3 (recursive via 10.10.10.1), 00:01:03
B 192.168.101.0/24 [200/0] via 10.255.255.2 (recursive via 10.10.10.1), 00:01:03 BGP routes in the RIB
B 192.168.102.0/24 [200/0] via 10.20.20.2 (recursive via 10.10.10.1), 00:01:03

France02 # get router info routing-table static

S* 0.0.0.0/0 [10/0] via 198.51.100.254, port2
S 10.10.10.0/24 [10/0] via 10.10.10.1, Paris
S 10.20.20.0/24 [10/0] via 10.10.10.1, Paris Static routes in the RIB
S 10.255.255.0/30 [10/0] via 10.10.10.1, Paris

France02 # get router info routing-table connected

C 10.5.48.0/20 is directly connected, port10
C 10.10.10.1/32 is directly connected, Paris
C 10.10.10.2/32 is directly connected, Paris Connected routes in the RIB
C 192.168.2.0/24 is directly connected, port1
C 198.51.100.0/24 is directly connected, port2

63
Useful CLI commands – Routing
France02 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default

S* 0.0.0.0/0 [10/0] via 198.51.100.254, port2

C 10.5.48.0/20 is directly connected, port10
S 10.10.10.0/24 [10/0] via 10.10.10.1, Paris
C 10.10.10.1/32 is directly connected, Paris
C 10.10.10.2/32 is directly connected, Paris
S 10.20.20.0/24 [10/0] via 10.10.10.1, Paris
All active routes in the RIB
R 10.255.255.0/24 [120/2] via 10.10.10.1, Paris, 00:02:14
S 10.255.255.0/30 [10/0] via 10.10.10.1, Paris
B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 00:01:08
C 192.168.2.0/24 is directly connected, port1
B 192.168.3.0/24 [200/0] via 10.10.10.3 (recursive via 10.10.10.1), 00:01:08
B 192.168.101.0/24 [200/0] via 10.255.255.2 (recursive via 10.10.10.1), 00:01:08
B 192.168.102.0/24 [200/0] via 10.20.20.2 (recursive via 10.10.10.1), 00:01:08
C 198.51.100.0/24 is directly connected, port2

France02 # get router info routing-table details 192.168.102.1

Routing entry for 192.168.102.0/24
Known via "bgp", distance 200, metric 0, best
Last update 00:02:48 ago
Details of a specific route in the RIB
* 10.20.20.2 (recursive via 10.10.10.1)

64
Useful CLI commands – Routing
[root:~]# ping 192.168.3.1
[root:~]# ping 192.168.101.1 Bring up some shortcuts
[root:~]# ping 192.168.102.1

France02 (root) # get router info routing-table all

Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default

S* 0.0.0.0/0 [10/0] via 198.51.100.254, port2

C 10.5.48.0/20 is directly connected, port10
S 10.10.10.0/24 [10/0] via 10.10.10.1, Paris
C 10.10.10.1/32 is directly connected, Paris
C 10.10.10.2/32 is directly connected, Paris
is directly connected, Paris_0
is directly connected, Paris_1
Overlay local-ip
is directly connected, Paris_2
C 10.10.10.3/32 is directly connected, Paris_0
S 10.20.20.0/24 [10/0] via 10.10.10.1, Paris
C 10.20.20.2/32 is directly connected, Paris_2 BGP Next-Hop are now directly connected
S 10.255.255.0/24 [10/0] via 10.10.10.1, Paris
C 10.255.255.2/32 is directly connected, Paris_1
B 192.168.1.0/24 [200/0] via 10.10.10.1, Paris, 00:20:39
C 192.168.2.0/24 is directly connected, port1
B 192.168.3.0/24 [200/0] via 10.10.10.3, Paris_0, 00:01:11
B 192.168.101.0/24 [200/0] via 10.255.255.2, Paris_1, 00:00:11 Routes through the shortcuts
B 192.168.102.0/24 [200/0] via 10.20.20.2, Paris_2, 00:00:11
C 198.51.100.0/24 is directly connected, port2

65
Dual Regions Hub & Spoke
Interconnecting two Hub & Spoke Regions
Dual Regions Hub & Spoke

BGP BGP
AS 65000 AS 65100

67
Dual Regions Hub & Spoke
IPsec configuration
 Dual Regions Hub & Spoke
config vpn ipsec phase1-interface
edit "Madrid"
set interface "port2" Requires an IPsec tunnel between the two Hubs
set proposal aes128-sha1
set auto-discovery-forwarder enable
“auto-discovery-forwarder” must be enabled on the Hub-to-Hub tunnel
set remote-gw 203.0.113.1
set psksecret fortinet
next
end

config vpn ipsec phase2-interface

edit "Madrid"
set phase1name "Madrid"
set proposal aes128-sha1
next
end

config system interface

edit "Madrid"
set ip 10.255.255.1 255.255.255.255
set remote-ip 10.255.255.2
next
end

+ Firewall policies:
. “To Madrid H&S”:
set srcintf "LAN" "Spokes"
set dstintf "Madrid"

. “From Madrid H&S”:

set dstintf "LAN" "Spokes"
set srcintf "Madrid"

69
 Dual Regions Hub & Spoke
config vpn ipsec phase1-interface
edit "Paris"
Requires an IPsec tunnel between the two Hubs set interface "port2"
“auto-discovery-forwarder” must be enabled on the Hub-to-Hub tunnel set proposal aes128-sha1
set auto-discovery-forwarder enable
set remote-gw 198.51.100.1
set psksecret fortinet
next
end

config vpn ipsec phase2-interface

edit "Paris"
set phase1name "Paris"
set proposal aes128-sha1
next
end

config system interface

edit "Paris"
set ip 10.255.255.2 255.255.255.255
set remote-ip 10.255.255.1
next
end

+ Firewall policies:
. “To Paris H&S”:
set srcintf "LAN" "Spokes"
set dstintf "Paris"

. “From Paris H&S”:

set dstintf "LAN" "Spokes"
set srcintf "Paris"

70
Dual Regions Hub & Spoke
BGP configuration
 Dual Regions Hub & Spoke
config router bgp eBGP: each Hub & Spoke belongs to a different AS
set as 65000
set router-id 10.10.10.1
config neighbor
edit "10.255.255.2"
set attribute-unchanged next-hop
set ebgp-enforce-multihop enable
set remote-as 65100
next
end
end
eBGP
BGP BGP
attribute-unchanged next-hop AS 65000 AS 65100

Keep the BGP Next-Hop attributes unchanged

when BGP routes exit the AS.
This is mandatory to allow routing convergence
over the ADVPN shortcuts.

ebgp-enforce-multihop
Is required for next-hop-unchanged

72
 Dual Regions Hub & Spoke
eBGP: each Hub & Spoke belongs to a different AS
config router bgp
set as 65100
set router-id 10.20.20.1
config neighbor
edit "10.255.255.1"
set attribute-unchanged next-hop
set ebgp-enforce-multihop enable
set remote-as 65000
eBGP next
end
BGP BGP end
AS 65000 AS 65100

73
ADVPN Hands-On Experience
Dual Regions Hub & Spoke

© Copyright Fortinet Inc. All rights reserved.

Dual Regions Hub & Spoke

Tunnel between Hubs

75
Dual Regions Overlay Networks

BGP BGP
AS 65000 AS 65100

76
 ADVPN Hands-On Experience
Configuration Steps: Validation Steps:

1. Create the tunnel towards the Hub (GUI) 1. Tunnel to the Hub:

2. Configure the overlay IPs on the tunnel interface (GUI) . From the FGT-VM Ping the Hub’s overlay IP
. From the Linux-VM Ping the Hub’s server
3. Enable ADVPN on the tunnel (CLI)
2. Shortcuts:
4. Create firewall policies (GUI)
. Enable IKE debug
5. Configure the static routes for the VPN overlay subnets . From the Linux-VM Ping resources behind
(GUI) other Spokes
. Check the IKE shortcut negotiation
6. Configure BGP peering with the Hub (GUI) . Check the tunnels
. Check the routing

77
Config Step 1: Create the tunnel [1/4]

78
Config Step 1: Create the tunnel [2/4]

Madrid = 203.0.113.101
Paris = 198.51.100.1
“Internet” interface
of the FGT-VM

79
Config Step 1: Create the tunnel [3/4]

80
Config Step 1: Create the tunnel [4/4]

81
Config Step 2: Overlay IP on the tunnel interface

France overlay = 10.10.10.0/24

Spain overlay = 10.20.20.0/24

Spain125 overlay IP
Madrid overlay IP

82
Config Step 3: Enable ADVPN on the tunnel

Spain-125 #
Spain-125 # config vpn ipsec phase1-interface

Spain-125 (phase1-interface) # edit Madrid

Spain-125 (Madrid) #
Spain-125 (Madrid) # set auto-discovery-receiver enable
Spain-125 (Madrid) # set add-route disable

Spain-125 (Madrid) #
Spain-125 (Madrid) # end

Spain-125 #
Spain-125 #

83
 Config Step 4 & 5: Firewall policies and Overlay routes

84
Config Step 6: Configure BGP
France = AS 65000
Spain = AS 65100

85
Config: Spoke “Spain125”
Tunnel: Interfaces:

config system interface

config vpn ipsec phase1-interface
edit "port1"
edit "Madrid"
set ip 192.168.125.254 255.255.255.0
set interface "port2"
set allowaccess ping https ssh
set proposal aes128-sha1
set alias "LAN"
set auto-discovery-receiver enable
next
set add-route disable
edit "port2"
set remote-gw 203.0.113.101
set ip 203.0.113.125 255.255.255.0
set psksecret fortinet
set allowaccess ping https ssh
next
set alias "ISP2"
end
next
config vpn ipsec phase2-interface
edit "Madrid"
edit "Madrid"
set ip 10.20.20.25 255.255.255.255 For FortiOS 5.4
set phase1name "Madrid"
set proposal aes128-sha1
set remote-ip 10.20.20.1
and 5.6.0/5.6.1/5.6.2
next
set remote-ip 10.20.20.1 255.255.255.0
end
set allowaccess ping
next As of FortiOS 5.6.3 and 6.0
end

86
Config: Spoke “Spain125”
Overlay routes:
config router static
edit 1
set gateway 203.0.113.254
set device "port2"
next

edit 2
set dst 10.20.20.0 255.255.255.0 Only mandatory
set device "Madrid" for FortiOS 5.4
set comment "Spain local overlay subnet"
next and 5.6.0/5.6.1/5.6.2
edit 3
set dst 10.10.10.0 255.255.255.0
set device "Madrid"
set comment "France remote overlay subnet"
next
edit 4
set dst 10.255.255.0 255.255.255.252
set device "Madrid"
set comment "Paris-Madrid remote overlay"
next
end

87
Config: Spoke “Spain125”

BGP: Policies:
config router bgp config firewall policy
set as 65100 edit 1
set router-id 10.20.20.25 set name "to ADVPN"
config neighbor set srcintf "port1"
edit "10.20.20.1" set dstintf "Madrid"
set remote-as 65100 set srcaddr "all"
next set dstaddr "all"
end set action accept
config network set schedule "always"
edit 1 set service "ALL"
set prefix 192.168.125.0 255.255.255.0 next
next edit 2
end set name "from ADVPN"
end set srcintf "Madrid"
set dstintf "port1"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set service "ALL"
next
end

88
 Validation Step 1: Tunnel towards the Hub
Spain-125 # exec ping 10.20.20.1
PING 10.20.20.1 (10.20.20.1): 56 data bytes
64 bytes from 10.20.20.1: icmp_seq=0 ttl=255 time=0.6 ms From the FGT-VM Ping the Hub’s overlay IP
64 bytes from 10.20.20.1: icmp_seq=1 ttl=255 time=0.4 ms
64 bytes from 10.20.20.1: icmp_seq=2 ttl=255 time=0.4 ms
64 bytes from 10.20.20.1: icmp_seq=3 ttl=255 time=0.4 ms Paris overlay IP = 10.10.10.1
64 bytes from 10.20.20.1: icmp_seq=4 ttl=255 time=0.4 ms Madrid overlay IP = 10.20.20.1
--- 10.20.20.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.4/0.4/0.6 ms

mint@mint ~ $ ping 192.168.101.1

PING 192.168.101.1 (192.168.101.1): 56 data bytes
64 bytes from 192.168.101.1: icmp_seq=0 ttl=253 time=0.7 ms
From the Linux-VM Ping the Hub’s server 64 bytes from 192.168.101.1: icmp_seq=1 ttl=253 time=0.6 ms
64 bytes from 192.168.101.1: icmp_seq=2 ttl=253 time=0.5 ms
64 bytes from 192.168.101.1: icmp_seq=3 ttl=253 time=0.6 ms
Paris server = 192.168.1.1 64 bytes from 192.168.101.1: icmp_seq=4 ttl=253 time=0.5 ms
Madrid server = 192.168.101.1
--- 192.168.101.1 ping statistics ---
5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.5/0.6/0.7 ms

89
 Validation Step 2: Shortcuts

Spain-125 # diag debug console timestamp enable

Start IKE debug
Spain-125 # diag vpn ike log filter clear (don’t use any filter so that you can witness all IKE
Spain-125 # diag debug application ike -1 negotiations with the Hub and all the other Spokes)
Spain-125 # diag debug enable

mint@mint ~ $ ping 192.168.2.1

PING 192.168.2.1 (192.168.2.1): 56 data bytes
64 bytes from 192.168.2.1: icmp_seq=0 ttl=251 time=1.8 ms
64 bytes from 192.168.2.1: icmp_seq=1 ttl=253 time=0.7 ms
From the Linux-VM (here, Spain125) 64 bytes from 192.168.2.1: icmp_seq=2 ttl=253 time=0.7 ms
Ping the Linux-VM of another Spoke 64 bytes from 192.168.2.1: icmp_seq=3 ttl=253 time=0.8 ms
(here, France02) 64 bytes from 192.168.2.1: icmp_seq=4 ttl=253 time=0.7 ms

--- 192.168.2.1 ping statistics ---

5 packets transmitted, 5 packets received, 0% packet loss
round-trip min/avg/max = 0.7/0.9/1.8 ms

90
 Validation Step 2: Shortcuts
IKE Shortcut negotiation between Spain102 and France02:

ike 0: comes 203.0.113.1:500->203.0.113.125:500,ifindex=4....

ike 0: IKEv1 exchange=Informational id=8b0f747cb261ee48/965a8ff8f3cd8b12:75dcd867 len=172
ike 0:Madrid:39: notify msg received: SHORTCUT-OFFER Shortcut-offer received from the Hub
ike 0:Madrid: shortcut-offer 192.168.125.1->192.168.2.1 psk 64

ike 0:Madrid:39: sent IKE msg (SHORTCUT-QUERY): 203.0.113.125:500->203.0.113.1:500, len=204, id=8b0f747cb261ee48/965a8ff8f3cd8b12:4ac6f02e

Shortcut-query sent to France02
ike 0: comes 203.0.113.1:500->203.0.113.125:500,ifindex=4.... via the Hub
ike 0: IKEv1 exchange=Informational id=8b0f747cb261ee48/965a8ff8f3cd8b12:11a2754f len=188
ike 0:Madrid:39: notify msg received: SHORTCUT-REPLY
ike 0:Madrid: recv shortcut-reply 14169010131944755998 a0862cdd0191a334/7f408cabf9c1a64e 198.51.100.2 to 192.168.125.1 psk 64
ike 0:Madrid: shortcut-reply route to 192.168.125.1 via root 13
Shortcut-reply received from France02
via the Hub
ike 0:Madrid: created connection: 0x6e35e00 4 203.0.113.125->198.51.100.2:500.
ike 0:Madrid: adding new dynamic tunnel for 198.51.100.2:500
ike 0:Madrid_0: added new dynamic tunnel for 198.51.100.2:500
ike 0:Madrid_0:41: initiator: main mode is sending 1st message...
ike 0:Madrid_0:41: cookie a0862cdd0191a334/7f408cabf9c1a64e
ike 0:Madrid_0:41: sent IKE msg (ident_i1send): 203.0.113.125:500->198.51.100.2:500, len=372, id=a0862cdd0191a334/7f408cabf9c1a64e
...
...
...
Shortcut ‘Madrid_0’ is created and IKE negotiation with France02 is starting…

91
Config: Hub “Paris”
Tunnels:
config vpn ipsec phase1-interface config vpn ipsec phase2-interface
edit "Spokes" edit "Spokes"
set type dynamic set phase1name "Spokes"
set interface "port2" set proposal aes128-sha1
set proposal aes128-sha1 next
set auto-discovery-sender enable edit "Madrid"
set add-route disable set phase1name "Madrid"
set psksecret fortinet set proposal aes128-sha1
next
set net-device disable end
set tunnel-search nexthop As of FortiOS 5.6.3 and 6.0
next

edit "Madrid"
set interface "port2"
set proposal aes128-sha1
set auto-discovery-sender enable
set auto-discovery-receiver enable
set auto-discovery-forwarder enable
set remote-gw 203.0.113.1
set psksecret fortinet
next
end

92
Config: Hub “Paris”
Policies:
Interfaces: config firewall policy
edit 1
config system interface set name "To Spokes"
edit "port1" set srcintf "port1"
set ip 192.168.1.254 255.255.255.0 set dstintf "Spokes"
set allowaccess ping https ssh set srcaddr "all"
set alias "LAN" set dstaddr "all"
next set action accept
edit "port2" set schedule "always"
set ip 198.51.100.1 255.255.255.0 set service "ALL"
set allowaccess ping https ssh next
set alias "INTERNET" edit 2
next set name "From Spokes"
edit "Madrid" set srcintf "Spokes"
set ip 10.255.255.1 255.255.255.255 set dstintf "port1"
set remote-ip 10.255.255.2 set srcaddr "all"
set allowaccess ping set dstaddr "all"
next set action accept
set schedule "always"
edit "Spokes" set service "ALL"
set ip 10.10.10.1 255.255.255.255 For FortiOS 5.4 next

set remote-ip 10.10.10.254

and 5.6.0/5.6.1/5.6.2 edit 3
set name "Spokes to Spokes"
set remote-ip 10.10.10.254 255.255.255.0 set srcintf "Spokes"
set dstintf "Spokes"
set allowaccess ping set srcaddr "all"
next As of FortiOS 5.6.3 and 6.0 set dstaddr "all"
end set action accept
set schedule "always"
set service "ALL"
next

93
 Config: Hub “Paris”
Policies: Routes: BGP:
(cont.) config router bgp
config router static
edit 4 set as 65000
edit 1
set name "To Madrid" set router-id 10.10.10.1
set gateway 198.51.100.254
set srcintf "port1" "Spokes" config neighbor
set device "port2"
set dstintf "Madrid" edit "10.255.255.2"
next
set srcaddr "all" set attribute-unchanged next-hop
end
set dstaddr "all" set ebgp-enforce-multihop enable
set action accept set remote-as 65100
set schedule "always" next
set service "ALL" end
next config neighbor-group
edit 5 edit "advn_peers"
set name "From Madrid" set remote-as 65000
set srcintf "Madrid" set route-reflector-client enable
set dstintf "Spokes" "port1" next
set srcaddr "all" end
set dstaddr "all" config neighbor-range
set action accept edit 1
set schedule "always" set prefix 10.10.10.0 255.255.255.0
set service "ALL" set neighbor-group "advn_peers"
next next
end end
config network
edit 1
set prefix 192.168.1.0 255.255.255.0
next
end
end

94
Config: Hub “Madrid”
Tunnels:
config vpn ipsec phase1-interface config vpn ipsec phase2-interface
edit "Spokes" edit "Spokes"
set type dynamic set phase1name "Spokes"
set interface "port2" set proposal aes128-sha1
set proposal aes128-sha1 next
set auto-discovery-sender enable edit "Paris"
set add-route disable set phase1name "Paris"
set psksecret Fortinet set proposal aes128-sha1
next
set net-device disable end
set tunnel-search nexthop As of FortiOS 5.6.3 and 6.0
next

edit "Paris"
set interface "port2"
set proposal aes128-sha1
set auto-discovery-sender enable
set auto-discovery-receiver enable
set auto-discovery-forwarder enable
set remote-gw 198.51.100.1
set psksecret fortinet
next
end

95
Config: Hub “Madrid”
Policies:
Interfaces: config firewall policy
edit 1
config system interface set name "To Spokes"
edit "port1" set srcintf "port1"
set ip 192.168.101.254 255.255.255.0 set dstintf "Spokes"
set allowaccess ping https ssh set srcaddr "all"
set alias "LAN" set dstaddr "all"
next set action accept
edit "port2" set schedule "always"
set ip 203.0.113.1 255.255.255.0 set service "ALL"
set allowaccess ping https ssh next
set alias "INTERNET" edit 2
next set name "From Spokes"
edit "Paris" set srcintf "Spokes"
set ip 10.255.255.2 255.255.255.255 set dstintf "port1"
set allowaccess ping set srcaddr "all"
set remote-ip 10.255.255.1 set dstaddr "all"
next set action accept
set schedule "always"
edit "Spokes" set service "ALL"
set ip 10.20.20.1 255.255.255.255 For FortiOS 5.4 next
set remote-ip 10.20.20.254
and 5.6.0/5.6.1/5.6.2 edit 3
set name "Spokes to Spokes"
set remote-ip 10.20.20.254 255.255.255.0 set srcintf "Spokes"
set dstintf "Spokes"
set allowaccess ping set srcaddr "all"
next As of FortiOS 5.6.3 and 6.0 set dstaddr "all"
end set action accept
set schedule "always"
set service "ALL"
next

96
 Config: Hub “Madrid”
Policies: Routes: BGP:
(cont.) config router bgp
config router static
edit 4 set as 65100
edit 1
set name "To Paris" set router-id 10.20.20.1
set gateway 203.0.113.254
set srcintf "port1" "Spokes" config neighbor
set device "port2"
set dstintf "Paris" edit "10.255.255.1"
next
set srcaddr "all" set attribute-unchanged next-hop
end
set dstaddr "all" set ebgp-enforce-multihop enable
set action accept set remote-as 65000
set schedule "always" next
set service "ALL" end
next config neighbor-group
edit 5 edit "advn_peers"
set name "From Paris" set remote-as 65100
set srcintf "Paris" set route-reflector-client enable
set dstintf "Spokes" "port1" next
set srcaddr "all" end
set dstaddr "all" config neighbor-range
set action accept edit 1
set schedule "always" set prefix 10.20.20.0 255.255.255.0
set service "ALL" set neighbor-group "advn_peers"
next next
end end
config network
edit 1
set prefix 192.168.101.0 255.255.255.0
next
end
end

97