Cisco VXLAN Configuration v1

First Published: 2021-02-16

Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
© 2021 Cisco Systems, Inc. All rights reserved.
 CONTENTS

CHAPTER 1 About 1
About This Demonstration 1
Limitations 1
Customization Options 1
Requirements 2
About This Solution 2
Topology 2
Equipment Details 4
Switch Information 5
Tenant Information 5
Server Information 5
Component Details 6
Before You Present 6
Get Started 6
Accessing Devices 7

CHAPTER 2 Scenarios 9

Build the VXLAN Underlay 9

Build the Underlay IGP Routing Protocol 9
Verification of the Underlay IGP 11
Configure Multicast IP Routing with PIM 13

Configure Anycast RP 13
Configure RP Address 14
Configure Interfaces 14
Configure RP Address 15
Verification 15

Cisco VXLAN Configuration v1

iii
 Contents

Configuring the BGP Underlay 16

Configure BGP Basics on each Switch 16
Verification 19
Configuring Overlay 19
Create VLAN/VNI 20
Create Tenants 21
Build VXLAN Tunnel 21
Verification 22
Configure EVPN 23
Configure BGP for Tenant Routing 23
Configure SVIs 24
Configure Host Connections 25
Configure Server-2 and Server-3 Access 25
Verification 26
Configure Server-1 Access (including vPC Configuration) 26
Verification 27
Configure Server-1 Ports 28
Verification 29
External Routing 30
Set up Interfaces on Leaf-4 to reach the WAN Router 30
Verification 30
Configure BGP Peering To WAN Router 31
Verification on Leaf-4 31
Configure BGP to Filter Host Routes to WAN Router 32
Verification on Server-4 33

CHAPTER 3 Appendix 35

Device Toubleshooting 35

CHAPTER 4 What’s Next? 39

Cisco VXLAN Configuration v1

iv
 CHAPTER 1
About
• About This Demonstration, on page 1
• About This Solution, on page 2
• Topology, on page 2
• Equipment Details, on page 4
• Switch Information, on page 5
• Tenant Information, on page 5
• Server Information, on page 5
• Component Details, on page 6
• Before You Present, on page 6
• Get Started, on page 6

About This Demonstration

VXLAN is essentially an overlay that transports Layer 2 frames across a Layer 3 IP Network. It provides the
ability to scale a Layer 2 broadcast domain at a larger scale while utilizing the full bandwidth of the network
without blocking ports due to spanning-tree.

Limitations
Certain features of the Cisco VXLAN solution are outside the scope of this demonstration, because the
demonstration uses virtual devices rather than a physical fabric:
• Due to the way the Nexus 9000v operates, it does not start with a boot statement and will get stuck in
loader on boot. To prevent this, make sure the show boot command contains a valid image to boot from.
• Some commands are not available on the virtual Nexus 9k that might be required on a CloudScale Nexus
9K. Please consult the documentation.
• Since the hardware is virtual, some things may appear odd when it comes to the interfaces. For example,
if two interfaces are directly connected, shutting one side down should show "down" on the other side.
This does not occur in virtual hardware.

Customization Options
We recommend that you test different scenarios after building the VXLAN Fabric.

Cisco VXLAN Configuration v1

1
 About
Requirements

• Tenant-2 is built but is not really used throughout the demo. We recommend that you move some of the
servers to Tenant-2 to show how multi-tenancy works to isolate traffic.
• The vPC configuration is very generic. It is outside the scope of this lab to set up the vPC configuration
with "advertise-pip". We recommend that you try and play with it. It does work, and it is helpful to know
the differences.

Requirements
The table below outlines the requirements for this preconfigured demonstration.

Required Optional
Laptop Cisco AnyConnect®

Router, registered and configured for Cisco dCloud

About This Solution

VXLAN is essentially an overlay that transports Layer 2 frames across a Layer 3 IP Network. It provides the
ability to scale a Layer 2 broadcast domain at a larger scale while utilizing the full bandwidth of the network
without blocking ports due to spanning-tree. It provides the following advantages
• Uses a Layer 3 network to tranfer Layer 2 frames
• Allows for ECMP through the Spine Switches
• Limits spanning-tree to the host facing Leaf switch ports
• Uses BGP to transfer the Layer 2 reachability information
• Provides Multi-tenancy to have different customer networks reside in the same fabric

Topology
This content includes preconfigured users and components to illustrate the scripted scenarios and features of
the solution.
dCloud Topology

Cisco VXLAN Configuration v1

2
About
Topology

Physical Topology

Cisco VXLAN Configuration v1

3
 About
Equipment Details

Equipment Details
Name Description Host Name (FQDN) IP Address Username Password
CML Cisco Modeling Labs cml.dcloud.cisco.com 198.18.133.3

Spine-1 Nexus 9K admin C1sco12345

Spine-2 Nexus 9K admin C1sco12345

Leaf-1 Nexus 9K admin C1sco12345

Leaf-2 Nexus 9K admin C1sco12345

Leaf-3 Nexus 9K admin C1sco12345

Leaf-4 Nexus 9K admin C1sco12345

WAN IOSv admin C1sco12345

Router

Server-1 Ubuntu Image cisco cisco

Server-2 TinyCore Linux cisco cisco

Server-3 TinyCore Linux cisco cisco

Server-4 TinyCore Linux cisco cisco

Cisco VXLAN Configuration v1

4
 About
Switch Information

Switch Information
Name Loopback 0 IP Loopback 1 IP Loopback 1 Secondary Loopback 15 IP
Spine-1 10.0.0.1 10.0.1.1 10.255.255.255

Spine-2 10.0.0.2 10.0.1.2 10.255.255.255

Leaf-1 10.0.0.11 10.0.1.11 10.0.1.100

Leaf-2 10.0.0.12 10.0.1.12 10.0.1.100

Leaf-3 10.0.0.13 10.0.1.13

Leaf-4 10.0.0.14 10.0.1.14

Tenant Information
Multicast
Name VLAN ID VLAN Name VNI SVI IP
Group
Tenant-1 101 Tenant-1_Network-1 10101 239.0.0.101 192.168.101.1/24

Tenant-1 102 Tenant-1_Network-2 10102 239.0.0.102 192.168.102.1/24

Tenant-1 1001 Tenant-1_L3VNI 101001 N/A N/A

Tenant-2 201 Tenant-2_Network-1 10201 239.0.0.201 192.168.201.1/24

Tenant-2 202 Tenant-2_Network-2 10202 239.0.0.202 192.168.202.2/24

Tenant-2 1002 Tenant-2_L3VNI 101002 N/A N/A

Server Information
Name VLAN IP Address Gateway
Server-1 101 192.168.101.10/24 192.168.101.1

Server-2 101 192.168.101.20/24 192.168.101.1

Server-3 102 192.168.102.30/24 192.168.102.1

Server-4 N/A 172.16.3.40/24 172.16.3.1

Cisco VXLAN Configuration v1

5
 About
Component Details

Component Details
• CML - 2.1.1-b19
• Nexus 9K - 9.3(6)
• IOSv - 15.9(3)M2
• Ubuntu - 20.04.1
• TinyCore Linux - 5.4.3-tinycore

Before You Present

Cisco dCloud strongly recommends that you perform the tasks in this document before presenting it in front
of a live audience. This will allow you to become familiar with the structure of the document and content.
dCloud recommends using the Chrome browser for all demos.

PREPARATION IS KEY TO A SUCCESSFUL PRESENTATION.

Get Started
Follow these steps to schedule a session of the content and configure your presentation environment.

Procedure

Step 1 Initiate your dCloud session. [Show Me How]

Note It may take up to 10 minutes for your session to become active.

Step 2 For best performance, connect to the workstation with Cisco AnyConnect VPN [Show Me How] and the
local RDP client on your laptop [Show Me How]
• Workstation 1: 198.18.133.252, Username: administrator, Password: C1sco12345.
Important After you access the remote desktop, wait 15 minutes for the devices to fully initialize. If you
do not wait accordingly, the devices may not be accessible.
This demonstration/lab is designed to be completed in one sitting without interruption, otherwise
you may see some errors and may have to log back into the application and/or devices.
The Nexus 9000v I/O is demanding of dCloud platform resources. As a result, device crashes
may occur. To recover failed devices, refer to the Device Troubleshooting Appendix in this
document.

Cisco VXLAN Configuration v1

6
 About
Accessing Devices

Accessing Devices

Important After you access the remote desktop, wait 15 minutes for the devices to fully initialize. If you do not
wait accordingly, the devices may not be accessible.

Procedure

Step 1 On the remote windows desktop, double-click the devices folder.

Step 2 Double-click a Leaf, Spine, or server icon to launch the device and access its command line interface (CLI).
Note that it can take several moments for a terminal session to fully load. If you receive a security warning
when launching a device, click Yes to continue.

Cisco VXLAN Configuration v1

7
 About
Accessing Devices

Cisco VXLAN Configuration v1

8
 CHAPTER 2
Scenarios
• Build the VXLAN Underlay, on page 9
• Configuring the BGP Underlay, on page 16
• Configuring Overlay, on page 19
• Configure Host Connections, on page 25
• External Routing, on page 30

Build the VXLAN Underlay

Value Proposition: In this scenario, we will build the routing tables using OSPF. We will utilize IP Unnumbered
on the interfaces in order to minimize the amount of IP Addresses we must use. The feature PIM also has to
be enabled. In order to successfully pass BUM traffic, either multicast or ingress replication must be used to
pass traffic such as ARP or other similar traffic between the Leaf switches. BGP, or in this case Multiprotocol
BGP, is required to pass the Layer 2 reachability information between the Leaf switches for their connected
hosts. This step will actually be covered in Configuring the BGP Underlay.

Build the Underlay IGP Routing Protocol

Important After you access the remote desktop, wait 15 minutes for the devices to fully initialize. If you do not
wait accordingly, the devices may not be accessible.
The Nexus 9000v I/O is demanding of dCloud platform resources. As a result, device crashes may occur.
To recover failed devices, refer to the Device Troubleshooting Appendix in this document.

Procedure

Step 1 On all of the Spine and Leaf switches, enter the following commands to enable the OSPF routing protocol
and set the Router-ID to match the Loopback 0 IP address.
Spine-1:
Spine-1# configure
feature ospf
router ospf UNDERLAY

Cisco VXLAN Configuration v1

9
 Scenarios
Build the Underlay IGP Routing Protocol

router-id 10.0.0.1
end
copy run start

Spine-2:
Spine-2# configure
feature ospf
router ospf UNDERLAY
router-id 10.0.0.2
end
copy run start

Leaf-1:
Leaf-1# configure
feature ospf
router ospf UNDERLAY
router-id 10.0.0.11
end
copy run start

Leaf-2:
Leaf-2# configure
feature ospf
router ospf UNDERLAY
router-id 10.0.0.12
end
copy run start

Leaf-3:
Leaf-3# configure
feature ospf
router ospf UNDERLAY
router-id 10.0.0.13
end
copy run start

Leaf-4:
Leaf-4# configure
feature ospf
router ospf UNDERLAY
router-id 10.0.0.14
end
copy run start

Now we will configure the interfaces for OSPF. In this setup, the goal is to enable OSPF with a point-to-point
network for faster convergence. Each of the loopback interfaces must be reachable throughout the network.
The loopbacks have already been created. In this case, the goal is to save IP Space also inside the fabric by
using “ip unnumbered”.
Loopback addresses described:
• Loopback0 – Used for the “ip unnumbered” and for the BGP Peering source/destination
• Loopback1 – Used for the VXLAN tunnel interface source and destination
• Loopback15 – Used only on spine switches for the Anycast RP address for multicast routing. Multicast
routing is used for BUM traffic discovery.

Spine-1 and Spine-2:

Cisco VXLAN Configuration v1

10
 Scenarios
Verification of the Underlay IGP

Step 2 Throughout the lab, the same config can be used on multiple devices. In this setup, we recommend that you
use a text editor in order to copy and paste the configuration. In this situation, all of the Spine Switches use
the exact same config. Only Spine-1 is shown below. Make sure to put the config on both Spine-1 and Spine-2.
Spine-1# configure
interface ethernet1/1-4
no switchport
medium p2p
ip router ospf UNDERLAY area 0.0.0.0
ip unnumbered loopback0
no shutdown
exit
interface loopback0
ip router ospf UNDERLAY area 0.0.0.0
interface loopback1
ip router ospf UNDERLAY area 0.0.0.0
interface loopback15
ip router ospf UNDERLAY area 0.0.0.0
end
copy run start

Leaf-1, Leaf-2, Leaf-3, and Leaf-4:

Step 3 Here, all of the leaf switches will utilize the same config. Make sure to put the config on all four Leaf switches.
Leaf-1# configure
interface ethernet 1/1-2
no switchport
medium p2p
ip router ospf UNDERLAY area 0.0.0.0
ip unnumbered loopback0
no shutdown
exit
interface loopback0
ip router ospf UNDERLAY area 0.0.0.0
interface loopback1
ip router ospf UNDERLAY area 0.0.0.0
end
copy run start

Verification of the Underlay IGP

It is important to verify that routing is up and working properly on the Spines/Leaf switches. The details of
OSPF are outside the scope of this lab. Please verify it’s functionality. A few of the commands to look for
are shown below.

Procedure

Step 1 Enter the following command on Spine-1 and Spine-2. In this output, we are looking to verify that all of the
Leaf switches did form an OSPF neighbor adjacency.
Spine-1:
Spine-1# show ip ospf neighbors
OSPF Process ID UNDERLAY VRF default
Total number of neighbors: 4
Neighbor ID Pri State Up Time Address Interface

Cisco VXLAN Configuration v1

11
 Scenarios
Verification of the Underlay IGP

10.0.0.11 1 FULL/ - 00:01:05 10.0.0.11 Eth1/1

10.0.0.12 1 FULL/ - 00:01:07 10.0.0.12 Eth1/2
10.0.0.13 1 FULL/ - 00:00:56 10.0.0.13 Eth1/3
10.0.0.14 1 FULL/ - 00:00:54 10.0.0.14 Eth1/4
Spine-1#

Spine-2:
Spine-2# show ip ospf neighbors
OSPF Process ID UNDERLAY VRF default
Total number of neighbors: 4
Neighbor ID Pri State Up Time Address Interface
10.0.0.11 1 FULL/ - 00:01:48 10.0.0.11 Eth1/1
10.0.0.12 1 FULL/ - 00:01:48 10.0.0.12 Eth1/2
10.0.0.13 1 FULL/ - 00:01:37 10.0.0.13 Eth1/3
10.0.0.14 1 FULL/ - 00:01:38 10.0.0.14 Eth1/4
Spine-2#

Spine-1:

Step 2 Enter the following command on Spine-1. In this output, the goal is to verify that all of the loopback IP
Addresses are reachable from each device. In this example, only the view from Spine-1 is shown. It is highly
recommended to check this output on each switch (Spine 1 and 2; Leaf-1, Leaf-2, Leaf-3 and Leaf-4.
Spine-1# show ip route
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.0.0.1/32, ubest/mbest: 2/0, attached
*via 10.0.0.1, Lo0, [0/0], 00:29:47, local
*via 10.0.0.1, Lo0, [0/0], 00:29:47, direct
10.0.0.2/32, ubest/mbest: 4/0
*via 10.0.0.11, Eth1/1, [110/81], 00:03:49, ospf-UNDERLAY, intra
*via 10.0.0.12, Eth1/2, [110/81], 00:03:49, ospf-UNDERLAY, intra
*via 10.0.0.13, Eth1/3, [110/81], 00:03:49, ospf-UNDERLAY, intra
*via 10.0.0.14, Eth1/4, [110/81], 00:03:49, ospf-UNDERLAY, intra
10.0.0.11/32, ubest/mbest: 1/0
*via 10.0.0.11, Eth1/1, [110/41], 00:02:06, ospf-UNDERLAY, intra
10.0.0.12/32, ubest/mbest: 1/0
*via 10.0.0.12, Eth1/2, [110/41], 00:01:55, ospf-UNDERLAY, intra
10.0.0.13/32, ubest/mbest: 1/0
*via 10.0.0.13, Eth1/3, [110/41], 00:01:48, ospf-UNDERLAY, intra
10.0.0.14/32, ubest/mbest: 1/0
*via 10.0.0.14, Eth1/4, [110/41], 00:01:41, ospf-UNDERLAY, intra
10.0.1.1/32, ubest/mbest: 2/0, attached
*via 10.0.1.1, Lo1, [0/0], 00:29:47, local
*via 10.0.1.1, Lo1, [0/0], 00:29:47, direct
10.0.1.2/32, ubest/mbest: 4/0
*via 10.0.0.11, Eth1/1, [110/81], 00:03:44, ospf-UNDERLAY, intra
*via 10.0.0.12, Eth1/2, [110/81], 00:03:44, ospf-UNDERLAY, intra
*via 10.0.0.13, Eth1/3, [110/81], 00:03:44, ospf-UNDERLAY, intra
*via 10.0.0.14, Eth1/4, [110/81], 00:03:44, ospf-UNDERLAY, intra
10.0.1.11/32, ubest/mbest: 1/0
*via 10.0.0.11, Eth1/1, [110/41], 00:02:01, ospf-UNDERLAY, intra
10.0.1.12/32, ubest/mbest: 1/0
*via 10.0.0.12, Eth1/2, [110/41], 00:01:50, ospf-UNDERLAY, intra
10.0.1.13/32, ubest/mbest: 1/0
*via 10.0.0.13, Eth1/3, [110/41], 00:01:43, ospf-UNDERLAY, intra
10.0.1.14/32, ubest/mbest: 1/0
*via 10.0.0.14, Eth1/4, [110/41], 00:01:36, ospf-UNDERLAY, intra
10.0.1.100/32, ubest/mbest: 2/0
*via 10.0.0.11, Eth1/1, [110/41], 00:01:50, ospf-UNDERLAY, intra
*via 10.0.0.12, Eth1/2, [110/41], 00:01:50, ospf-UNDERLAY, intra

Cisco VXLAN Configuration v1

12
 Scenarios
Configure Multicast IP Routing with PIM

10.255.255.255/32, ubest/mbest: 2/0, attached

*via 10.255.255.255, Lo15, [0/0], 00:29:47, local
*via 10.255.255.255, Lo15, [0/0], 00:29:47, direct
Spine-1#

Configure Multicast IP Routing with PIM

One of the bigger hurdles to get a VXLAN fabric working is to make sure that Broadcast, Unknown Unicast,
and Multicast traffic operates as expected. This type of traffic is also called BUM traffic.
There are actually 2 ways to solve the problem.
• Option 1 – ingress-replication using BGP. This option removes the need for Multicast on the Underlay.
It requires each Leaf Switch to be able to replicate a packet to every other leaf switch using the specific
VNI
• Option 2 – Multicast. This option uses Multicast Groups to forward BUM traffic. It requires the Spine
Switch to be able to replicate a packet to all Leaf Switches listening on a multicast group

In this example, we will use Multicast. There are a handful of ways to configure multicast. For simplicity in
the configuration, we will be using Anycast RP. It involves some extra configuration on the spine switches
using Loopback15 (was previously configured) with the same IP Address on both spine switches.
For the Spine switches, each of the loopback interfaces and physical interfaces that are connected to Leaf
switches need to be configured to run “ip pim sparse-mode”. The “anycast-rp” configuration tells the switch
which IP will be the RP address and the Loopback0 for the other switches that are running the Anycast RP
Address. Finally, it needs to be told what the RP address is.

Configure Anycast RP

Procedure

Here, both Spine switches will utilize the same config. Make sure to enter the following commands on both
Spine-1 and Spine-2 switches.
Note Both spine switches will be configured exactly the same. Since we already configured the PIM
Feature and the Loopback 3 Interface, it is rather trivial to enable Anycast RP. The first two lines
shown below are all it takes. The first IP is the IP Address of the RP and the second IP is the
loopback0 interface of all the Spine switches acting as an Anycast RP including this one. The
configuration can be copied and pasted on both spines. The second section is where the RP is
statically assigned to the switch

Spine-1, Spine-2:
Spine-1#configure
feature pim
ip pim anycast-rp 10.255.255.255 10.0.0.1
ip pim anycast-rp 10.255.255.255 10.0.0.2
ip pim rp-address 10.255.255.255
end
copy run start

Cisco VXLAN Configuration v1

13
 Scenarios
Configure RP Address

The Leaf switch configuration is simpler than the Spine configuration. Each of the loopback and physical
interfaces that are connected to the Spine Switches must be configured with “ip pim sparse-mode’. The only
other requirement is to specify the Anycast RP address of the Spines.

Configure RP Address

Procedure

Here, all four Leaf switches will utilize the same config. Make sure to put the config on all Leaf switches. All
of the leaf configurations will use the same IP Address as the RP. A switch will choose either path to the
10.255.255.255 IP Address via uplink. It will not matter which one it picks because they are synchronized
using anycast-rp. The same one line should be applied to each Leaf switch.
Leaf-1, Leaf-2, Leaf-3, Leaf-4:
Leaf-1# configure
feature pim
ip pim rp-address 10.255.255.255
end
copy run start

Configure Interfaces

Procedure

Here, the Spine switches will utilize the same config. Make sure to put the following configuration on both
Spine-1 and Spine-2. Both spine switches will be configured exactly the same. Since we already configured
the PIM Feature and the Loopback 3 Interface, it is rather trivial to enable Anycast RP. The first two lines
shown below are all it takes. The first IP is the IP Address of the RP and the second IP is the loopback0
interface of all the Spine switches acting as an Anycast RP including this one. The configuration can be copied
and pasted on both spines. The second section is where the RP is statically assigned to the switch.
Spine-1, Spine-2:
Spine-1# configure
interface loopback 0
ip pim sparse-mode
interface loopback 1
ip pim sparse-mode
interface loopback 15
ip pim sparse-mode
exit
interface ethernet 1/1-4
ip pim sparse-mode
end
copy run start

Cisco VXLAN Configuration v1

14
 Scenarios
Configure RP Address

Configure RP Address

Procedure

Here, all of the leaf switches will utilize the same config. Make sure to enter the following configuration on
all four Leaf switches. All of the leaf configurations will use the same IP Address as the RP. A switch will
choose either path to the 10.255.255.255 IP Address via uplink. It will not matter which one it picks because
they are synchronized using anycast-rp. The same one line should be applied to each Leaf Switch.
Leaf-1, Leaf-2, Leaf-3, Leaf-4:
Leaf-1# configure
interface loopback0
ip pim sparse-mode
interface loopback1
ip pim sparse-mode
exit
int ethernet1/1-2
ip pim sparse-mode
end
copy run start

Verification

Procedure

Please be sure to run the following verification commands on both Spine-1 and Spine-2. The output should
be very similar for both switches. However, covering the details of Multicast is outside the scope of this lab.

Spine-1# show ip pim neighbor

PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD ECMP
Redirect
Priority Capable State Capable
10.0.0.11 Ethernet1/1 16:51:03 00:01:23 1 yes n/a no
10.0.0.12 Ethernet1/2 16:49:02 00:01:30 1 yes n/a no
10.0.0.13 Ethernet1/3 16:48:57 00:01:26 1 yes n/a no
10.0.0.14 Ethernet1/4 16:48:52 00:01:25 1 yes n/a no
Spine-1# show ip pim interface brief
PIM Interface Status for VRF "default"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Ethernet1/1 10.0.0.1 10.0.0.11 1 no
Ethernet1/2 10.0.0.1 10.0.0.12 1 no
Ethernet1/3 10.0.0.1 10.0.0.13 1 no
Ethernet1/4 10.0.0.1 10.0.0.14 1 no
loopback0 10.0.0.1 10.0.0.1 0 no
loopback1 10.0.1.1 10.0.1.1 0 no
loopback15 10.255.255.255 10.255.255.255 0 no
Spine-1#
Spine-2# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD ECMP
Redirect

Cisco VXLAN Configuration v1

15
 Scenarios
Configuring the BGP Underlay

Priority Capable State Capable

10.0.0.11 Ethernet1/1 16:54:20 00:01:37 1 yes n/a no
10.0.0.12 Ethernet1/2 16:52:19 00:01:42 1 yes n/a no
10.0.0.13 Ethernet1/3 16:52:14 00:01:41 1 yes n/a no
10.0.0.14 Ethernet1/4 16:52:09 00:01:37 1 yes n/a no
Spine-2# show ip pim interface brief
PIM Interface Status for VRF "default"
Interface IP Address PIM DR Address Neighbor Border
Count Interface
Ethernet1/1 10.0.0.2 10.0.0.11 1 no
Ethernet1/2 10.0.0.2 10.0.0.12 1 no
Ethernet1/3 10.0.0.2 10.0.0.13 1 no
Ethernet1/4 10.0.0.2 10.0.0.14 1 no
loopback0 10.0.0.2 10.0.0.2 0 no
loopback1 10.0.1.2 10.0.1.2 0 no
loopback15 10.255.255.255 10.255.255.255 0 no
Spine-2#

This concludes the scenario.

Configuring the BGP Underlay

Value Proposition: For this implementation, the plan is to use iBGP in order to form BGP peering adjacencies.
iBGP is a great way to separate multiple VXLAN Fabrics for future integration. However, there is one rule
of iBGP that must be overcome first. iBGP does not add anything to the AS-PATH so it cannot use the
AS-PATH as a loop prevention option. The loop prevention option in iBGP states that it will not advertise
any route it receives from an iBGP peer to another iBGP peer.
The best way to get around this is to use route-reflectors. Route-reflectors by default do not update the
NEXT_HOP. That is perfectly fine in our scenario as long as the route-reflectors are configured on both spine
switches.
It is also worth noting the additional address-family that might be unfamiliar. It is this new address family
that carries all the control-plane information for EVPN.

Configure BGP Basics on each Switch

On each switch we will enable the necessary two features: BGP and NV Overlay and an enable of Address
family EVPN.
• BGP – The primary protocol for passing reachability information for hosts connected to the VXLAN
Fabric
• NV Overlay – Enable the Network Virtualization capability

The “nv overlay evpn” command enables the l2vpn evpn address-family for BGP. The command provides
the capability of using the control-plan for endpoint learning instead of the dataplane with flood and learn.

Procedure

Step 1 On each Spine and Leaf switch, enter the following commands to enable NV Overlay.

Cisco VXLAN Configuration v1

16
Scenarios
Configure BGP Basics on each Switch

Spine 1, Spine 2, Leaf-1, Leaf-2, Leaf-3 and Leaf-4

Spine-1# configure
feature bgp
feature nv overlay
nv overlay evpn
end
copy run start

Step 2 Enter the following commands to configure the BGP Process. On the Spine switches, the “retain route-target
all” command is required since the Spine switches will be passing the VXLAN traffic, but will not know
about any of the Tenant information. For the most part, each switch will actually be configured the same.
However, it is recommended to specify a router-id that matches the Loopback0 interface.
Spine-1:
Spine-1# configure
router bgp 65001
router-id 10.0.0.1
address-family ipv4 unicast
address-family l2vpn evpn
retain route-target all
end
copy run start

Spine-2:
Spine-2# configure
router bgp 65001
router-id 10.0.0.2
address-family ipv4 unicast
address-family l2vpn evpn
retain route-target all
end
copy run start

Leaf-1:
Leaf-1# configure
router bgp 65001
router-id 10.0.0.11
address-family ipv4 unicast
address-family l2vpn evpn
end
copy run start

Leaf-2:
Leaf-2# configure
router bgp 65001
router-id 10.0.0.12
address-family ipv4 unicast
address-family l2vpn evpn
end
copy run start

Leaf-3:
Leaf-3# configure
router bgp 65001
router-id 10.0.0.13
address-family ipv4 unicast
address-family l2vpn evpn
end
copy run start

Cisco VXLAN Configuration v1

17
 Scenarios
Configure BGP Basics on each Switch

Leaf-4:
Leaf-4# configure
router bgp 65001
router-id 10.0.0.14
address-family ipv4 unicast
address-family l2vpn evpn
end
copy run start

Step 3 Enter the following commands to configure Spine switches to connect to Leaf switches. Templates are used
to make the configuration more scalable and easier to read. While not completely necessary, it makes the
config cleaner if more neighbors are added.
Spine-1 and Spine-2:
Spine-1# configure
router bgp 65001
template peer iBGP-Leafs
remote-as 65001
update-source loopback0
address-family ipv4 unicast
send-community both
route-reflector-client
address-family l2vpn evpn
send-community both
route-reflector-client
exit
exit
neighbor 10.0.0.11
description Leaf-1 Loopback0
inherit peer iBGP-Leafs
neighbor 10.0.0.12
description Leaf-2 Loopback0
inherit peer iBGP-Leafs
neighbor 10.0.0.13
description Leaf-3 Loopback0
inherit peer iBGP-Leafs
neighbor 10.0.0.14
description Leaf-4 Loopback0
inherit peer iBGP-Leafs
end
copy run start

Leaf-1, Leaf-2, Leaf-3, and Leaf-4:

Leaf-1# configure
router bgp 65001
template peer iBGP-Spines
remote-as 65001
update-source loopback0
address-family ipv4 unicast
send-community both
address-family l2vpn evpn
send-community both
neighbor 10.0.0.1
inherit peer iBGP-Spines
description Spine-1 Loopback0
neighbor 10.0.0.2
inherit peer iBGP-Spines
description Spine-2 Loopback0

Cisco VXLAN Configuration v1

18
 Scenarios
Verification

end
copy run start

Verification
Procedure

Enter the following commands on Spine-1 to verify the BGP neighbor relationships formed between the Spine
and Leaf pairs. It doesn’t matter at this time that the tables are empty with 0 routes.
Spine-1# show bgp ipv4 unicast summary
BGP summary information for VRF default, address family IPv4 Unicast
BGP router identifier 10.0.0.1, local AS number 65001
BGP table version is 6, IPv4 Unicast config peers 4, capable peers 4
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.0.0.11 4 65001 9 9 6 0 0 00:00:21 0
10.0.0.12 4 65001 9 9 6 0 0 00:00:19 0
10.0.0.13 4 65001 9 9 6 0 0 00:00:24 0
10.0.0.14 4 65001 9 9 6 0 0 00:00:20 0
Spine-1# show bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 10.0.0.1, local AS number 65001
BGP table version is 6, L2VPN EVPN config peers 4, capable peers 4
0 network entries and 0 paths using 0 bytes of memory
BGP attribute entries [0/0], BGP AS path entries [0/0]
BGP community entries [0/0], BGP clusterlist entries [0/0]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.0.0.11 4 65001 9 9 6 0 0 00:00:28 0
10.0.0.12 4 65001 9 9 6 0 0 00:00:26 0
10.0.0.13 4 65001 9 9 6 0 0 00:00:31 0
10.0.0.14 4 65001 9 9 6 0 0 00:00:27 0

This concludes the scenario.

Configuring Overlay
Value Proposition: In this scenario, the goal is to build a VLAN to VNI reference to match the prior Table
information. In reality, it is only 1 addition command that most people aren’t familiar with. The only addition
command is to associate the VNI to the VLAN. This is solved using the “vn-segment” command under a
VLAN.
Looking at the design, there are not any hosts plugged into Leaf-4. Leaf-4 is called a “Border Leaf”. It is not
common to plug end hosts into a border leaf. Therefore, the Layer 2 VLAN’s and VNI’s are not necessary to
be configured on the Border Leaf.

Cisco VXLAN Configuration v1

19
 Scenarios
Create VLAN/VNI

Create VLAN/VNI
Procedure

Enter the following commands to configure VLAN/VNI. Note that it is not necessary to configure VLAN
102 on Leaf-1 and Leaf-2 since there are no hosts plugged into it. It is included to make the configuration the
same. Once the lab is completed, it is also recommended to move hosts between VLANs in order to further
enhance understanding.
Leaf-1, Leaf-2, Leaf-3:
Leaf-1# configure
feature vn-segment-vlan-based
vlan 101
name Tenant-1_Network-1
vn-segment 10101
exit
vlan 102
name Tenant-1_Network-2
vn-segment 10102
exit
vlan 201
name Tenant-2_Network-1
vn-segment 10201
exit
vlan 202
name Tenant-2_Network-2
vn-segment 10202
exit
vlan 1001
name Tenant-1_L3VNI
vn-segment 101001
exit
vlan 1002
name Tenant-2_L3VNI
vn-segment 101002
end
copy run start

Note The “Warning” you receive in the command output can be ignored. This occurs due the fact that
the devices are actually Virtual.

Leaf-4:
Leaf-4# configure
feature vn-segment-vlan-based
vlan 1001
name Tenant-1_L3VNI
vn-segment 101001
exit
vlan 1002
name Tenant-2_L3VNI
vn-segment 101002
end
copy run start

Cisco VXLAN Configuration v1

20
 Scenarios
Create Tenants

Note Ignore the “Warning” you receive in the command output. This is occurring due the fact that the
devices are actually Virtual.

Create Tenants
Procedure

Enter the following commands on all Leaf switches to create tenants. VRFs are used to separate different
tenants at Layer 3. In reality, it is what makes a fabric multi-tenant. VRFs are not new. However, they do need
to have the proper VNI configured above to match the Tenant and the additional route-target command for
evpn.
Leaf-1, Leaf-2, Leaf-3, Leaf-4
Leaf-1# configure
vrf context Tenant-1
rd auto
vni 101001
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
exit
exit
vrf context Tenant-2
rd auto
vni 101002
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
end
copy run start

Build VXLAN Tunnel

At the core, VXLAN is a tunnel. Just like with other tunnel capabilities, it needs a tunnel interface. The NVE
interface is exactly that. It is essentially the VXLAN Tunnel Interface.
In this config, we are setting the “host-reachability protocol” to BGP so that it looks to the BGP Table for all
of its control plane information. It will use the Loopback 1 source interface for the tunnel. The mcast-groups
should be configured to match Table above.
Leaf-4 is also still a “Border Leaf”. It will require the Layer 3 configurations but none of the Layer 2
configurations.

Procedure

Enter the following commands to build the VXLAN tunnel.

Cisco VXLAN Configuration v1

21
 Scenarios
Verification

Leaf-1, Leaf-2, Leaf-3:

Leaf-1# configure
interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
member vni 10101
mcast-group 239.0.0.101
exit
member vni 10102
mcast-group 239.0.0.102
exit
member vni 10201
mcast-group 239.0.0.201
exit
member vni 10202
mcast-group 239.0.0.202
exit
member vni 101001 associate-vrf
exit
member vni 101002 associate-vrf
end
copy run start

Leaf-4:
Leaf-4# configure
interface nve1
no shutdown
source-interface loopback1
host-reachability protocol bgp
member vni 101001 associate-vrf
exit
member vni 101002 associate-vrf
end
copy run start

Verification
Procedure

Enter the following commands on Leaf-1 to verify the configuration.

Leaf-1# configure
Enter configuration commands, one per line. End with CNTL/Z.
Leaf-1(config)# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 10101 239.0.0.101 Up CP L2 [101]
nve1 10102 239.0.0.102 Up CP L2 [102]
nve1 10201 239.0.0.201 Up CP L2 [201]
nve1 10202 239.0.0.202 Up CP L2 [202]

Cisco VXLAN Configuration v1

22
 Scenarios
Configure EVPN

nve1 101001 n/a Up CP L3 [Tenant-1]

nve1 101002 n/a Up CP L3 [Tenant-2]
Leaf-1(config)# exit

Configure EVPN
The EVPN section is what sets up Layer 2 connectivity across the fabric. It only requires configuration on
Leaf switches with hosts connected.

Procedure

Enter the following commands to configure EVPN.

Leaf-1, Leaf-2, Leaf-3:
Leaf-1# configure
evpn
vni 10101 l2
rd auto
exit
vni 10101 l2
rd auto
route-target both auto
exit
vni 10102 l2
rd auto
route-target both auto
exit
vni 10201 l2
rd auto
route-target both auto
exit
vni 10202 l2
rd auto
route-target both auto
end
copy run start

Configure BGP for Tenant Routing

Procedure

Enter the following commands to add the Layer 3 routing capability across the fabric. Adding the VRF
information to BGP puts all the data into BGP l2vpn evpn table. Utilizing a route-map will bring all the SVI
interface subnets into the BGP process as well.
Leaf-1, Leaf-2, Leaf-3, Leaf-4
Leaf-1# configure
route-map DIRECT permit 10
match tag 12345

Cisco VXLAN Configuration v1

23
 Scenarios
Configure SVIs

route-map DIRECT deny 90

exit
router bgp 65001
vrf Tenant-1
address-family ipv4 unicast
redistribute direct route-map DIRECT
exit
exit
vrf Tenant-2
address-family ipv4 unicast
redistribute direct route-map DIRECT
end
copy run start

Configure SVIs
The SVIs utilize what is called an “Anycast gateway.” This feature puts the same IP/MAC address on each
of the Leaf switches for hosts to connect to. It specifies a “universal” MAC Address for all the switches to
use so that if a host migrates between switches, the gateway MAC address doesn’t change.

Procedure

Step 1 Enter the following command to configure the feature on all Leaf switches.
Leaf-1, Leaf-2, Leaf-3, and Leaf-4
Leaf-1# configure
feature interface-vlan
end

Step 2 The first SVIs to configure are the Layer 2 SVIs. Notice how they are “tagging” the routes. This will be useful
later when the networks are configured to be routed externally.
Layer 2 SVIs on Leaf-1, Leaf-2, Leaf-3
Leaf-1# configure
fabric forwarding anycast-gateway-mac 1234.1234.1234
interface vlan 101
vrf member Tenant-1
ip address 192.168.101.1/24 tag 12345
mtu 9216
no ip redirects
fabric forwarding mode anycast-gateway
no shutdown
interface vlan 102
vrf member Tenant-1
ip address 192.168.102.1/24 tag 12345
mtu 9216
no ip redirects
fabric forwarding mode anycast-gateway
no shutdown
interface vlan 201
vrf member Tenant-2
ip address 192.168.201.1/24 tag 12345
no ip redirects
mtu 9216
fabric forwarding mode anycast-gateway
no shut

Cisco VXLAN Configuration v1

24
 Scenarios
Configure Host Connections

interface vlan 202

vrf member Tenant-2
ip address 192.168.202.1/24 tag 12345
no ip redirects
mtu 9216
fabric forwarding mode anycast-gateway
no shutdown
end
copy run start

Step 3 Layer 3 SVIs are used to route traffic across the fabric. The configuration is similar to Layer 2 SVIs except
they don’t have an IP address. They use “ip forward” to inform the SVI of its role.
Layer 3 SVIs on Leaf-1, Leaf-2, Leaf-3, Leaf-4
Leaf-1# configure
interface vlan 1001
vrf member Tenant-1
ip forward
mtu 9216
no ip redirects
no shut
exit
interface vlan 1002
vrf member Tenant-2
ip forward
mtu 9216
no ip redirects
no shut
end
copy run start

This concludes the scenario.

Configure Host Connections

Value Proposition: In this scenario, we will configure Server-2 and Server-3. It is the same configuration that
has been used for years with normal Ethernet switches. The goal here is to verify that the servers can ping
their gateway and each other.

Configure Server-2 and Server-3 Access

Procedure

On Leaf-3, enter the following commands to configure Server-2 and Server-3 access:
Leaf-3:
Leaf-3# configure
int ethernet1/3
switchport
switchport mode access
switchport access vlan 101
spanning-tree port type edge
int ethernet1/4

Cisco VXLAN Configuration v1

25
 Scenarios
Verification

switchport
switchport mode access
switchport access vlan 102
spanning-tree port type edge
end
copy run start

Verification
Procedure

On Server-2, enter the following command to verify the configuration.

cisco@server-2:~$ ping 192.168.101.1
PING 192.168.101.1 (192.168.101.1): 56 data bytes
64 bytes from 192.168.101.1: seq=1 ttl=255 time=5.086 ms
64 bytes from 192.168.101.1: seq=2 ttl=255 time=3.703 ms
^C
--- 192.168.101.1 ping statistics ---
3 packets transmitted, 2 packets received, 33% packet loss
round-trip min/avg/max = 3.703/4.394/5.086 ms
cisco@server-2:~$ ping 192.168.102.30
PING 192.168.102.30 (192.168.102.30): 56 data bytes
64 bytes from 192.168.102.30: seq=1 ttl=63 time=6.976 ms
64 bytes from 192.168.102.30: seq=2 ttl=63 time=8.191 ms
^C
--- 192.168.102.30 ping statistics ---
3 packets transmitted, 2 packets received, 33% packet loss
round-trip min/avg/max = 6.976/7.583/8.191 ms
cisco@server-2:~$

Configure Server-1 Access (including vPC Configuration)

Procedure

Before the host ports can be configured, the vPC domain must be built. The end goal here is to create a vPC
to Server-1. The end goal is to show redundancy and how to configure it.
Leaf-1:
Leaf-1# configure
feature vpc
feature lacp
vrf context vpc-pka
address-family ipv4 unicast
exit
exit
interface ethernet1/5
no switchport
vrf member vpc-pka
Warning: Deleted all L3 config on interface Ethernet1/5

Cisco VXLAN Configuration v1

26
 Scenarios
Verification

ip address 192.168.0.0/31
no shutdown
vpc domain 10
peer-keepalive destination 192.168.0.1 source 192.168.0.0 vrf vpc-pka
peer-switch
peer-gateway
ip arp synchronize
exit
interface ethernet1/6-7
switch
switch mode trunk
channel-group 100 mode active
no shutdown
exit
interface port-channel 100
vpc peer-link
end
copy run start

Leaf-2:
Leaf-2# configure
feature vpc
feature lacp
vrf context vpc-pka
address-family ipv4 unicast
exit
exit
interface ethernet1/5
no switchport
vrf member vpc-pka
Warning: Deleted all L3 config on interface Ethernet1/5
ip address 192.168.0.1/31
no shutdown
vpc domain 10
peer-keepalive destination 192.168.0.0 source 192.168.0.1 vrf vpc-pka
peer-switch
peer-gateway
ip arp sync
ip arp synchronize
exit
interface ethernet 1/6-7
switch
switch mode trunk
no shut
channel-group 100 mode active
interface po100
interface port-channel 100
vpc peer-link
end
copy run start

Verification
Procedure

Enter the following command on Leaf-1 to verify the configuration.

Cisco VXLAN Configuration v1

27
 Scenarios
Configure Server-1 Ports

Leaf-1# show vpc

Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 10
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : primary
Number of vPCs configured : 0
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
Auto-recovery status : Disabled
Delay-restore status : Timer is off.(timeout = 30s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Operational Layer3 Peer-router : Disabled
Virtual-peerlink mode : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
1 Po100 up 1,101-102,201-202,1001-1002
Leaf-1#

Configure Server-1 Ports

Procedure

Enter the following commands to configure Server-1 ports.

Leaf-1, Leaf-2:
Leaf-1# configure
interface ethernet1/3
switchport
switchport mode access
switchport access vlan 101
channel-group 101 mode active
no shutdown
exit
interface port-channel101
vpc
spanning-tree port type edge
end
copy run start

Cisco VXLAN Configuration v1

28
 Scenarios
Verification

Verification
Procedure

Enter the following commands to verify the configuration.

Leaf-1# show port-channel summary
Flags: D - Down P - Up in port-channel (members)
I - Individual H - Hot-standby (LACP only)
s - Suspended r - Module-removed
b - BFD Session Wait
S - Switched R - Routed
U - Up (port-channel)
p - Up in delay-lacp mode (member)
M - Not in use. Min-links not met
--------------------------------------------------------------------------------
Group Port- Type Protocol Member Ports
Channel
--------------------------------------------------------------------------------
100 Po100(SU) Eth LACP Eth1/6(P) Eth1/7(P)
101 Po101(SU) Eth LACP Eth1/3(P)
Leaf-1# show vpc 101
vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
101 Po101 up success success 101
Please check "show vpc consistency-parameters vpc <vpc-num>" for the
consistency reason of down vpc and for type-2 consistency reasons for
any vpc.
Leaf-1#
ubuntu@server-1:~$ ping 192.168.101.1
PING 192.168.101.1 (192.168.101.1) 56(84) bytes of data.
64 bytes from 192.168.101.1: icmp_seq=2 ttl=255 time=4.13 ms
64 bytes from 192.168.101.1: icmp_seq=3 ttl=255 time=4.25 ms
^C
--- 192.168.101.1 ping statistics ---
3 packets transmitted, 2 received, 33.3333% packet loss, time 2003ms
rtt min/avg/max/mdev = 4.134/4.192/4.250/0.058 ms
ubuntu@server-1:~$ ping 192.168.101.20
PING 192.168.101.20 (192.168.101.20) 56(84) bytes of data.
64 bytes from 192.168.101.20: icmp_seq=1 ttl=64 time=31.9 ms
64 bytes from 192.168.101.20: icmp_seq=2 ttl=64 time=13.9 ms
^C
--- 192.168.101.20 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 13.881/22.901/31.921/9.020 ms
ubuntu@server-1:~$ ping 192.168.102.30
PING 192.168.102.30 (192.168.102.30) 56(84) bytes of data.
64 bytes from 192.168.102.30: icmp_seq=1 ttl=62 time=16.7 ms
^C
--- 192.168.102.30 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 16.742/16.742/16.742/0.000 ms
ubuntu@server-1:~$

This concludes the scenario.

Cisco VXLAN Configuration v1

29
 Scenarios
External Routing

External Routing
Value Proposition: In this scenario, you will set up external routing.

Set up Interfaces on Leaf-4 to reach the WAN Router

Procedure

Enter the following commands to set up the interfaces on Leaf-4 to reach the WAN router.
Leaf-4:
Leaf-4# configure
int ethernet1/3
no switchport
no shutdown
exit
int ethernet1/3.10
encapsulation dot1q 10
vrf member Tenant-1
ip address 172.16.1.0/31
no shutdown
exit
int ethernet1/3.20
encapsulation dot1q 20
vrf member Tenant-2
ip address 172.16.2.0/31
no shutdown
exit
end
copy run start

Note You can ignore the warnings you receive.

Verification
Procedure

Enter the following command to verify the configuration.

Leaf-4# ping 172.16.1.1 vrf Tenant-1
PING 172.16.1.1 (172.16.1.1): 56 data bytes
64 bytes from 172.16.1.1: icmp_seq=0 ttl=254 time=4.855 ms
64 bytes from 172.16.1.1: icmp_seq=1 ttl=254 time=3.288 ms
64 bytes from 172.16.1.1: icmp_seq=2 ttl=254 time=3.428 ms
64 bytes from 172.16.1.1: icmp_seq=3 ttl=254 time=3.683 ms
64 bytes from 172.16.1.1: icmp_seq=4 ttl=254 time=3.603 ms
--- 172.16.1.1 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 3.288/3.771/4.855 ms

Cisco VXLAN Configuration v1

30
 Scenarios
Configure BGP Peering To WAN Router

Leaf-4# ping 172.16.2.1 vrf Tenant-2

PING 172.16.2.1 (172.16.2.1): 56 data bytes
64 bytes from 172.16.2.1: icmp_seq=0 ttl=254 time=3.467 ms
64 bytes from 172.16.2.1: icmp_seq=1 ttl=254 time=3.543 ms
64 bytes from 172.16.2.1: icmp_seq=2 ttl=254 time=2.902 ms
64 bytes from 172.16.2.1: icmp_seq=3 ttl=254 time=2.269 ms
64 bytes from 172.16.2.1: icmp_seq=4 ttl=254 time=2.926 ms
--- 172.16.2.1 ping statistics ---
5 packets transmitted, 5 packets received, 0.00% packet loss
round-trip min/avg/max = 2.269/3.021/3.543 ms
Leaf-4#

Configure BGP Peering To WAN Router

Procedure

Enter the following commands to configure BGP peering to the WAN router.
Leaf-4:
Leaf-4# configure
router bgp 65001
vrf Tenant-1
neighbor 172.16.1.1
remote-as 65002
address-family ipv4 unicast
exit
exit
exit
vrf Tenant-2
neighbor 172.16.2.1
remote-as 65002
address-family ipv4 unicast
end
copy run start

Verification on Leaf-4
Procedure

Enter the following commands on Leaf-4 to verify the configuration.

Leaf-4# show bgp vrf Tenant-1 ipv4 unicast summary
BGP summary information for VRF Tenant-1, address family IPv4 Unicast
BGP router identifier 172.16.1.0, local AS number 65001
BGP table version is 65, IPv4 Unicast config peers 1, capable peers 1
6 network entries and 11 paths using 724 bytes of memory
BGP attribute entries [8/1312], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [6/24]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
172.16.1.1 4 65002 8 5 65 0 0 00:01:08 1

Cisco VXLAN Configuration v1

31
 Scenarios
Configure BGP to Filter Host Routes to WAN Router

Leaf-4# show bgp vrf Tenant-2 ipv4 unicast summary

BGP summary information for VRF Tenant-2, address family IPv4 Unicast
BGP router identifier 172.16.2.0, local AS number 65001
BGP table version is 52, IPv4 Unicast config peers 1, capable peers 1
3 network entries and 7 paths using 424 bytes of memory
BGP attribute entries [4/656], BGP AS path entries [1/6]
BGP community entries [0/0], BGP clusterlist entries [6/24]
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
172.16.2.1 4 65002 9 4 52 0 0 00:01:00 1
Leaf-4# show bgp vrf Tenant-1 ipv4 unicast neighbors 172.16.1.1 routes
Peer 172.16.1.1 routes for address family IPv4 Unicast:
BGP table version is 65, Local Router ID is 172.16.1.0
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2
Network Next Hop Metric LocPrf Weight Path
*>e172.16.3.0/24 172.16.1.1 0 0 65002 i
Leaf-4# show bgp vrf Tenant-2 ipv4 unicast neighbors 172.16.2.1 routes
Peer 172.16.2.1 routes for address family IPv4 Unicast:
BGP table version is 52, Local Router ID is 172.16.2.0
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup, 2 - best2
Network Next Hop Metric LocPrf Weight Path
*>e172.16.3.0/24 172.16.2.1 0 0 65002 i
Leaf-4#

Configure BGP to Filter Host Routes to WAN Router

Procedure

Enter the following commands to configure BGP to filter host routes to the WAN router.
Leaf-4:
Leaf-4# configure
ip prefix-list NOHOSTS seq 5 permit 0.0.0.0/0 le 31
route-map EBGP-PEER permit 5
match ip address prefix-list NOHOSTS
route-map EBGP-PEER deny 90
exit
router bgp 65001
vrf Tenant-1
neighbor 172.16.1.1
address-family ipv4 unicast
route-map EBGP-PEER out
exit
exit
exit
vrf Tenant-2
neighbor 172.16.2.1
address-family ipv4 unicast
route-map EBGP-PEER out
end
copy run start

Cisco VXLAN Configuration v1

32
 Scenarios
Verification on Server-4

Verification on Server-4
Procedure

Enter the following commands to verify the configuration.

cisco@server-4:~$ ping -c 3 192.168.101.10
PING 192.168.101.10 (192.168.101.10): 56 data bytes
64 bytes from 192.168.101.10: seq=0 ttl=61 time=17.855 ms
64 bytes from 192.168.101.10: seq=1 ttl=61 time=16.179 ms
64 bytes from 192.168.101.10: seq=2 ttl=61 time=17.638 ms
--- 192.168.101.10 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 16.179/17.224/17.855 ms
cisco@server-4:~$ ping -c 3 192.168.101.20
PING 192.168.101.20 (192.168.101.20): 56 data bytes
64 bytes from 192.168.101.20: seq=0 ttl=61 time=14.793 ms
64 bytes from 192.168.101.20: seq=1 ttl=61 time=14.926 ms
64 bytes from 192.168.101.20: seq=2 ttl=61 time=16.284 ms
--- 192.168.101.20 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 14.793/15.334/16.284 ms
cisco@server-4:~$ ping -c 3 192.168.102.30
PING 192.168.102.30 (192.168.102.30): 56 data bytes
64 bytes from 192.168.102.30: seq=0 ttl=61 time=13.634 ms
64 bytes from 192.168.102.30: seq=1 ttl=61 time=14.018 ms
64 bytes from 192.168.102.30: seq=2 ttl=61 time=14.232 ms
--- 192.168.102.30 ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 13.634/13.961/14.232 ms
cisco@server-4:~$

This concludes the scenario.

Cisco VXLAN Configuration v1

33
 Scenarios
Verification on Server-4

Cisco VXLAN Configuration v1

34
 CHAPTER 3
Appendix
• Device Toubleshooting, on page 35

Device Toubleshooting
On occasion, a Nexus 9K device may crash due to the Nexus 9000v’s highly demanding IO and available
dCloud environment resources.
For this reason, we have implemented an Out-of-Band method of accessing the serial consoles for the Nexus
9K devices using a guacamole webapp running in a server inside the session called web-consoles.
Google Chrome on the remote desktop is configured to open up the web-consoles webapp and all lab devices’
serial ports that have been configured within the guacamole application.

If a device has crashed in your session, use the following procedure to recover the failed node.

Procedure

Step 1 first click the connection of the device name that crashed (for example leaf1_cli).

Cisco VXLAN Configuration v1

35
 Appendix
Device Toubleshooting

Cisco VXLAN Configuration v1

36
Appendix
Device Toubleshooting

Step 2 This places you in rommon mode, where you will see the Loader > prompt.

Step 3 Once you are in rommon mode (Loader> prompt), enter the boot bootflash:nxos.9.3.6.bin command
and press <Enter>.

Step 4 Wait for the switch to finish booting. Once the device finishes booting, you will see the login prompt for the
device (for example Leaf-1).

Step 5 Log in to the device. You can continue the demonstration where you left off prior to the device crash.

Cisco VXLAN Configuration v1

37
 Appendix
Device Toubleshooting

Cisco VXLAN Configuration v1

38
 CHAPTER 4
What’s Next?
Now that you have touched the surface of what VXLAN can do, take a look at some of the videos on Cisco
Live’s Website. Search for sessions “BRKDCN-3378” and “BRKDCN-3040”.
After you have a firm grasp on the technology, we recommend that you automate the building of VXLAN
Fabrics using DCNM. There are several YouTube videos demonstrating this process.

Cisco VXLAN Configuration v1

39
 What’s Next?

Cisco VXLAN Configuration v1

40