Nexus 7000

virtual Port-Channel
Best Practices & Design
Guidelines

Technical Marketing
Data Center Switching Technology Group August 2010
version 1.3

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 1

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 2

 Feature Overview & Terminology
vPC Definition

 Allow a single device to use a port

channel across two upstream switches
 Eliminate STP blocked ports
 Uses all available uplink bandwidth
 Dual-homed server operate in active-
active mode Logical Topology without vPC
 Provide fast convergence upon
link/device failure
 Reduce CAPEX and OPEX
 Available on current and future hardware
for M1 and D1 generation cards.

Logical Topology with vPC

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 4
 Feature Overview & Terminology
vPC Terminology  vPC peer – a vPC switch, one of a pair
 vPC member port – one of a set of ports (port
channels) that form a vPC
vPC peer- vPC peer-link
keepalive link
 vPC – the combined port channel between the
vPC peers and the downstream device
CFS protocol  vPC peer-link – Link used to synchronize state
between vPC peer devices, must be 10GbE
vPC peer  vPC peer-keepalive link – the keepalive link
between vPC peer devices, i.e., backup to the
vPC vPC peer-link
vPC
vPC
member  vPC VLAN – one of the VLANs carried over
member
port
port the peer-link and used to communicate via vPC
with a peer device.
 non-vPC VLAN – One of the STP VLANs not
carried over the peer-link
vPC
 CFS – Cisco Fabric Services protocol, used for
non-vPC
device state synchronization and configuration
validation between vPC peer devices

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 5

 vPC Design Guidance
& Best Practices

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 6

 Building a vPC Domain
Configuration Steps
Following steps are needed to build a vPC (Order does Matter!)
1. Configure globally a vPC domain on both vPC devices
2. Configure a Peer-keepalive link on both vPC peer switches (make sure is operational)
NOTE: When a vPC domain is configured the keepalive must be operational to allow a vPC
domain to successfully form.
3. Configure (or reuse) an interconnecting port-channel between the vPC peer switches
4. Configure the inter-switch channel as Peer-link on both vPC devices (make sure is operational)
5. Configure (or reuse) Port-channels to dual-attached devices
6. Configure a unique logical vPC and join port-channels across different vPC peers

vPC peer- vPC peer-link

keepalive link

vPC peer

Standalone
Port-channel vPC vPC member port
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 7
 Building a vPC Domain
Peer Link
 Definition:
Standard 802.1Q Trunk
vPC peer-link
Can Carry vPC and non vPC VLANs*
Carries Cisco Fabric Services messages (tagged as CoS=4 for
reliable communication)
Carries flooded traffic from a vPC peer
Carries STP BPDUs, HSRP Hellos, IGMP updates, etc.
 Requirements:
Member ports must be 10GE interfaces one of the N7K-
M132XP-12 modules
Peer-link are point-to-point. No other device should be inserted
between the vPC peers.
 Recommendations (strong ones!)
Minimum 2x 10GbE ports on separate cards for best resiliency.
Dedicated 10GbE ports (not shared mode ports)
*It is Best Practice to split vPC and non-vPC
VLANs on different Inter-switch Port-Channels.
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 8
 Building a vPC Domain
Peer Link with Single 10G Module

 Common Nexus 7000 configuration:

1x 10G, 7x 1G cards
 vPC recommendation is 2 10G cards
 Potential problem occurs if Nexus 7000 is L3 boundary with
single 10G card
 Use Object Tracking Feature available in 4.2
 More information (links from Nexus 7000 wiki and CCO):

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/vPC_Single_10G_module.pptx

http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/interfaces/configuration/
guide/if_vPC.html#wp1529488

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 9

 Building a vPC Domain
Peer Link with Single 10G Module – Object Tracking
Scenario:
 vPC deployments with a single N7K-M132XP-12
card, where core and peer-link interfaces are
localized on the same card.
 This scenario is vulnerable to access-layer isolation
if the 10GE card fails on the primary vPC.
e1/… e1/… e1/… e1/…
vPC Object Tracking Solution:
e1/… vPC PL e1/…
 Leverages object tracking capability in vPC (new L3
e1/… e1/…
CLI commands are added). L2 vPC PKL
 Peer-link and Core interfaces are tracked as a list e2/… e2/…
vPC vPC
of boolean objects. Primary Secondary
 vPC object tracking suspends vPCs on the
impaired device, so traffic can get diverted over the
remaining vPC peer.

rhs-7k-1(config-vpc-domain)# track <object>

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 10

 Building a vPC Domain
Peer-Keepalive (1 of 2)
 Definition:
 Heartbeat between vPC peers
vPC peer-
 Active/Active (no Peer-Link) detection keepalive link
 Messages sent on 2 second interval
 3 second hold timeout on peer-link loss
 Fault Tolerant terminology is specific to VSS and deprecated in vPC.
 Packet Structure:
 UDP message on port 3200, 96 bytes long (32 byte payload), includes
version, time stamp, local and remote IPs, and domain ID.
 Keepalive messages can be captured and displayed using the onboard
Wireshark Toolkit.
 Recommendations:
 Should be a dedicated link (1Gb is adequate)
 Should NOT be routed over the Peer-Link
 Can optionally use the mgmt0 interface (along with management traffic)
 As last resort, can be routed over L3 infrastructure

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 12

 Building a vPC Domain
Peer-Keepalive (2 of 2)
Cautions/Additional Recommendations:
 When using supervisor management interfaces to carry the vPC peer-
keepalive, do not connect them back to back between the two switches.
 Only one management port will be active a given point in time and a
supervisor switchover may break keep-alive connectivity
 Use the management interface only if you have an out-of-band
management network (management switch in between).
Management Standby Management
Management Network Interface
Switch Active Management
vPC_PK Interface
vPC_PK

vPC_PL

vPC1 vPC2

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 13

 Building a vPC Domain
vPC Member Port

 Definition:
Port-channel member of a vPC peer.
 Requirements:
Configuration needs to match other vPC
peer’s member port config.
In case of inconsistency a VLAN or the
entire port-channel may suspend (i.e. MTU
mismatch).
Number of member ports on both vPC
peers is not required to match. vPC
member
port
Up to 8 active ports between both vPC
peers (16-way port-channel can be build
with multi-layer vPC)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 14

 Building a vPC Domain
VDC Interaction
 vPC works seamlessly in any VDC based environment.
 One vPC domain per VDC is supported, up to the maximum number of VDCs
supported in the system.
 It is still necessary to have a separate vPC peer-link and vPC Peer-Keepalive
Link infrastructure for each VDC deployed.
Can vPC run between VDCs on the same switch?
 This scenario should technically work, but it is NOT officially supported and
has not been extensively tested by our QA team.
 Could be useful for Demo or hands on, but It is NOT recommended for
production environments. Will consolidate redundant points on the same box
with VDCs (e.g. whole aggregation layer on a box) and introduce a single point
of failure.
 ISSU will NOT work in this configuration, because the vPC devices can NOT
be independently upgraded.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 15

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 16

 Attaching to a vPC domain
The One and Only Rule…

ALWAYS
dual attach devices
to a vPC Domain!!!

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 17

 Attaching to a vPC Domain
IEEE 802.3ad and LACP
 Definition:
Port-channel for devices for devices dual-attached to
the vPC pair.
Provides local load balancing for port-channel
members
STANDARD 802.3ad port channel
 Access Device Requirements
STANDARD 802.3ad capability
LACP Optional
vPC
 Recommendations: vPC
Regular
 Use LACP when available for better failover and mis- member
Port-
port
channel
configuration protection port

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 18

 Attaching to a vPC Domain
”My device can’t be dual attached!”
Recommendations (in order of preference):
1. ALWAYS try to dual attach devices using vPC (not applicable for routed links).
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-
active scenarios. Ensures full redundant active/active paths through vPC.
CONS: None
2. If (1) is not an option – connect the device via a vPC attached access switch (could use VDC to create a
“virtual access switch”).
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with vPC dual-
active scenarios. Availability limited by the access switch failure.
CONS: Need for an additional access switch or need to use one of the available VDCs. Additional
administrative burden to configure/manage the physical/Virtual Device
3. If (2) is not an option – connect device directly to (primary) vPC peer in a non-vPC VLAN * and provide
for a separate interconnecting port-channel between the two vPC peers.
PROS: Traffic diverted on a secondary path in case of peer-link failover
CONS: Need to configure and manage additional ports (i.e. port-channel) between the Nexus 7000
devices.
4. If (3) is not an option – connect device directly to (primary) vPC peer in a vPC VLAN
PROS: Easy deployment
CONS: VERY BAD. Bound to vPC roles (no role preemption in vPC) , Full Isolation on peer-link failure
when attached vPC toggles to a secondary vPC role.

* VLAN that is NOT part of any vPC and not present on vPC peer-link
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 19
 Attaching to a vPC Domain
vPC and non-vPC VLANs (i.e. single attached .. )
P S P S
P S P S

1. Dual Attached 2. Attached via VDC/Secondary Switch

Orphan
Ports
P S
P S
P S
P S

P Primary vPC
P
S Secondary vPC
S

3. Secondary ISL Port-Channel 4. Single Attached to vPC Device

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 20
 Attaching to a vPC Domain
”My device only does STP!”
Recommendations (in order of preference):
1. ALWAYS try dual attach devices using vPC
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with
vPC dual-active scenarios. Ensures full redundant active/active paths through vPC.
CONS: None
2. If (1) is not an option – connect the device via two independent links using STP. Use non-
vPC VLANs ONLY on the STP switch.*
PROS: Ensures minimal disruption in case of peer-link failover and consistent behavior with
vPC dual-active scenarios. Ensures full redundant Active/Active paths on vPC VLANs.
CONS: Requires an additional STP port-channel between the vPC devices. Operational
burden in provisioning and configuring separate STP and vPC VLAN domains. Only
Active/Standby paths on STP VLANs.
3. If (2) is not an option – connect the device via two independent links using STP. (Use vPC
VLANs on this switch)
PROS: Simplify VLAN provisioning and does not require allocation of an additional 10GE
port-channel.
CONS: STP and vPC devices may not be able to communicate each other in certain failure
scenarios (i.e. when STP Root and vPC primary device do not overlap). All VLANs carried
over the peer-link may suspend until the two adjacency forms and vPC is fully
synchronized".
* Run the same STP mode as the vPC domain. Enable portfast/port type edge on host facing ports
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 21
 Attaching to a vPC Domain
vPC and non-vPC VLANs (STP/vPC Hybrid)
Non vPC port-channel

P S SR PR
P S SR PRS
P
P S

1. All devices Dual Attached via vPC 2. Separate vPC and STP VLANs

SR PR
SR PR
P S
P S
P Primary vPC
P
S Secondary vPC
S
PR Primary STP Root
PR
SR Secondary STP Root
SR

3. Overlapping vPC and STP VLANs

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 22
 Attaching to a vPC Domain
16-way Port-Channel (1 of 2)

 Multi-Layer vPC can join 8 active

ports port-channels in a unique 16-
way port-channel*
 vPC peer side load-balancing is Nexus
LOCAL to the peer 7000

 Each vPC peer has only 8 active 16-way port

channel
links, but the pair has 16 active load
balanced links Nexus
5000

* Possible with any device supporting

vPC/MCEC and 8-way active port-channels

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 23

 Attaching to a vPC Domain
16-way Port-Channel (2 of 2)

 16 active ports between 8

active port-channel devices
and 16 active port-channel
devices?
Nexus
 vPC peer side load-balancing 7000
is LOCAL to the peer
 Each vPC peer has only 8 16-port port-channel
active links, but the pair has Nexus
16 active load balanced links 5000
to the downstream device
supporting 16 active ports
 D-series N7000 line cards will
also support 16 way active
port-channel load balancing, Nexus 5000 16-port port-channel
providing for a potential 32 support introduced in 4.1(3)N1(1a)
way vPC port channel! release

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 24

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 25

 Layer 3 and vPC
Recommendations
 Use separate L3 links to hook up routers to a vPC domain is still standing.
 Don’t use L2 port channel to attach routers to a vPC domain unless you can statically route to
HSRP address
 If both, routed and bridged traffic is required, use individual L3 links for routed traffic and L2 port-
channel for bridged traffic

Switch Switch

Po2 Po2

7k1 7k2
L3 ECMP
Po1

Router Router
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 26
 Layer 3 and vPC
What can happen… (1 of 3)

vPC view Layer 2 topology Layer 3 topology

7k vPC
7k1 7k2 7k1 7k2

R
R
R
R could be any router, Port-channel looks like a Layer 3 will use ECMP
L3 switch or VSS single L2 pipe. Hashing for northbound traffic
building a port-channel will decide which link to
chose

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 27

 Layer 3 and vPC
What can happen… (2 of 3)

1) Packet arrives at R
2) R does lookup in routing table and sees 2 equal S
paths going north (to 7k1 & 7k2) Po2
3) Assume it chooses 7k1 (ECMP decision)
4) R now has rewrite information to which router it
needs to go (router MAC 7k1 or 7k2)
5) L2 lookup happens and outgoing interface is port-
channel 1
6) Hashing determines which port-channel member is 7k1 7k2
chosen (say to 7k2)
7) Packet is sent to 7k2 Po1
8) 7k2 sees that it needs to send it over the peer-link
to 7k1 based on MAC address

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 28

 Layer 3 and vPC
What can happen… (3 of 3)

9) 7k1 performs lookup and sees that it

needs to send to S S
Po2
10) 7k1 performs check if the frame came
over peer link & is going out on a vPC.
11) Frame will only be forwarded if outgoing
interface is NOT a vPC or if outgoing
vPC doesn’t have active interface on
other vPC peer (in our example 7k2)
7k1 7k2

Po1

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 29

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 30

 Spanning Tree Recommendations
Overview – STP Interoperability
 STP Uses:
• Loop detection (failsafe to vPC)
• Non-vPC attached device
• Loop management on vPC addition/removal
 Requirements:
• Needs to remain enabled, but doesn’t dictate vPC member
port state
• Logical ports still count, need to be aware of number of
VLANs/port-channels deployed!
 Best Practices:
• Not recommended to enable Bridge Assurance feature on
vPC channels (i.e. no STP “network” port type). Tracked by
CSCsz76892. vPC STP
vPC is running to
• Make sure all switches in you layer 2 domain are running manage loops outside of
with Rapid-PVST or MST (IOS default is non-rapid PVST+), vPC’s direct domain, or
to avoid slow STP convergence (30+ secs)
before initial vPC
• Remember to configure portfast (edge port-type) on host configuration
facing interfaces to avoid slow STP convergence (30+
secs)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 31

 Spanning Tree Recommendations
Port Configuration Overview N
N
Network port
E Edge or portfast port type
E
- Normal port type
-
B BPDUguard
B
R Rootguard
Data Center Core R
L Loopguard
L

Primary Secondary
vPC vPC
vPC
HSRP
HSRP Domain HSRP
HSRP Layer 3
ACTIVE
ACTIVE STANDBY
STANDBY
Aggregation
N N Secondary
Primary
Primary N N
Secondary
Root Root
Root
Root
- - - - -
- - -
Layer 2 (STP + Rootguard)
R - R - R- -R R -R - R - R-
R R R R R R R R

-
Access -

- - L
- - L

E E E E E
E E E E E
B B B B B
B B B B B Layer 2 (STP + BPDUguard)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 32

 Spanning Tree Recommendations
STP interaction on double failure

 On a peer-link and peer-keepalive

symultaneous failure, Active/Active mode
may occur
BP Us
 Both vPC peers forward BPDUs with same DU D

BPDU
P
s B

Us
bridge IDs (As of 4.2(x)), this resolves the Us BP

BPD
D DU

s
need to disable the etherchannel guard BP s

feature on downstream devices.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 33

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 34

 N Network port

Data Center Interconnect E

-
N

E
Edge or portfast port type
Normal port type
Multi-layer vPC for Agg and DCI B
-

B
BPDUguard
F BPDUfilter
F
R Rootguard
R

DC 1 vPC domain 11
Long Distance
vPC domain 21 DC 2

CORE
- F F -
- - F F - -
- -
N N
N N

N N
N N

- - F F - -
- - F F - - R
R
- R R-
R - R
- - R R- -

AGGR
-
N N N N
N N N N

- - vPC domain 10 vPC domain 20 - -

R- R - - -
R R R R
R R
Key Recommendations

ACCESS
- -
-  vPC Domain id for facing vPC layers should be different -
E  No Bridge Assurance on interconnecting vPCs E
E
B
B
 BPDU Filter on the edge devices to avoid BPDU propagation B
E
B
 No L3 peering between DCs (i.e. L3 over vPC)

Server Cluster Server Cluster

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 35
 Data Center Interconnect
Encrypted Interconnect
DC-1 DC-2
Nexus 7010 Nexus 7010

vPC vPC
CTS Manual Mode
(802.1AE 10GE line-rate
encryption)
No ACS is required

Nexus 7010 Nexus 7010

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 36

 Data Center Interconnect
References

 Validated TrustSec between Nexus

7000 connected back to back.
 Validated TrustSec across EoMPLS
cloud with ASR 1000 routers and DCI Dark Fiber
Catalyst 6500s terminating EoMPLS.
 Cisco Multi-Platform DCI WebPage:
http://wwwin.cisco.com/marketing/datacenter/solutions/laun
ches/dci/index.shtml

 More information on Cisco TrustSec:

http://bock-bock.cisco.com/wiki/N7K:tech_resources:security

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 37

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 38

 HSRP with vPC
FHRP Active/Active
 Support for all FHRP protocols in
Active/Active mode with vPC
HSRP/VRRP HSRP/VRRP
 No additional configuration is “Active”:
Active for
“Standby”:
Active for
required shared L3 MAC shared L3 MAC

 No changes to the HSRP control L3

Plane behavior. L2
 Gateway Mac address information
synchronized via CFS on the
“Standby” HSRP router.
 Primary vPC device and “Active”
HSRP Router can be placed on
different peers.
 In an active/active 2-router vPC
scenario default HSRP timers can
be used as best practice with no
significant convergence degradation
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 39
 HSRP with vPC
Do NOT use HSRP Object Tracking
Cautions:
 It is not recommended to use HSRP link tracking with vPC
 Reason: vPC will not forward a packet back on a vPC once it has crossed the peer-link, except
in the case of a remote member port failure

L3 CORE

ACTIVE HSRP STANDBY HSRP

GW
GW VLAN 100, 200 GW L2/L3
Aggregation

VLAN 100 VLAN 200

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 40

 HSRP with vPC
L3 Backup Routing
 Use an OSPF point-to-point adjacency (or equivalent L3 protocol) between the vPC peers to establish a L3 backup path to the Core
through in case of uplinks failure
 A single point-to-point VLAN/SVI will suffice to establish a L3 neighborship.

OSPF
OSPF

OSPF

VLAN 99
L3 OSPF
OSPF
L2
Primary Secondary
vPC vPC

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 41

 HSRP with vPC
Dual L2/L3 Pod Interconnect
Scenario:
 Provide L2/L3 interconnect between L2
Pods, or between L2 attached Datacenters
(i.e. sharing the same HSRP group).
 A vPC domain without an active HSRP
instance in a group would not able to forward
traffic.
Multi-layer vPC with single HSRP:
 L3 on the N7K supports Active/Active on oneActive Standby Listen Listen
pair, and still allows normal HSRP behavior
on other pair (all in one HSRP group)
 L3 traffic will run across Intra-pod link for non
Active/Active L3 pair

 More info for supported multi-router HSRP configurations at:

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/HSRP_and_vPC.pptx
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 42
 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 43

 vPC and Services
vPC Services Integration
 Services deployed as part of
Catalyst 6500 Service chassis
 Investigation ongoing with
standalone services (ASA, ACE)
 Appliance based services that do
not support port-channel may L3
require additional peer-link L2
connections to deal with the
additional traffic forced across
the peer-link
 More information will be posted
to the vPC wiki page as soon as
more scenario are verified.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 44

 vPC and Services
Catalyst 6500 Services Chassis w. Services VDC Sandwich
Two Nexus 7000 Virtual Device Contexts used to “sandwich”
services between virtual switching layers
• Layer-2 switching in Services Chassis with transparent
services
• Services Chassis provides Etherchannel capabilities for
interaction with vPC
• vPC running in both VDC pairs to provide Etherchannel for
both inside and outside interfaces to Services Chassis

Design considerations:
• Access switches requiring services are connected to sub-
aggregation VDC
• Access switches not requiring services may be connected to
aggregation VDC
• May be extended to support multiple virtualized service
contexts by using multiple VRF instances in the sub-
aggregation VDC

Design Cautions:
• Be aware of the Layer 3 over vPC design caveat. If Peering at
Layer 3 is required across the two vPC layers an alternative
solution should be explored (i.e. using STP rather than vPC to
attach service chassis)
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 45
 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 46

 vPC Latest Enhancements
Summary
Several enhancements to vPC:
 vPC Object Tracking
 vPC Peer-Gateway
 vPC Delay Restore
 Multi-layer vPC with single HSRP group
 vPC unicast ARP handling
 vPC Exclude Interface-VLAN
 vPC single attached device Listing
 vPC Convergence and Scalability
For more details:
 vPC 4.2 Enhancements Presentation
http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/vPC_Ankara_New_Features_Overview.ppt
 4.2 Release Notes
http://www.cisco.com/en/US/docs/switches/datacenter/sw/4_2/nx-os/release/notes/42_nx-
os_release_note.html#wp218085

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 47

 vPC Latest Enhancements
vPC Peer-Gateway for NAS interoperability
Local Routing for peer
Scenario: router –mac Traffic

 Interoperability with non RFC compliant

features of some NAS devices (i.e. NETAPP
Fast-Path or EMC IP-Reflect)
 NAS device may reply to traffic using the vPC PL
L3
MAC address of the sender device rather
than the HSRP gateway. vPC PKL
L2
 Packet reaching vPC for the non local
Router MAC address are sent across the
peer-link and can be dropped if the final
destination is behind another vPC.
vPC Peer-Gateway Solution:
 Allows a vPC switch to act as the active
gateway for packets addressed to the peer
router MAC (CLI command added in the
vPC global config)

N7k(config-vpc-domain)# peer-gateway
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 48
 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 49

 In-Service Software Upgrade (ISSU)
vPC System Upgrade/Downgrade
4.1(3)
4.2(1) 4.1(3)
4.2(1)
 ISSU is still the recommended system
upgrade in a multi-device vPC environment
 vPC system can be independently upgraded
with no disruption to traffic. 4.1(3)
4.2(1)
 Upgrade is serialized and must be run one at
the time (i.e. config lock will prevent
synchronous upgrades)
 Configuration is locked on “other” vPC peer
during ISSU.

Begin End Caveats

4.1(x) 4.2(x) None
4.2(x) 4.1(x) None

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 50

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 51

 4.2(1) vPC Enhancements
Convergence Topology 20 flows @1000 pps

OSPF L3 Core
OSPF
Nexus 7000

4 E2
/1 /1
N7K-1 E2 4 N7K-2 L2/L3
OSPF
OSPF Aggregation
Po10
Nexus 7000 vPC

16-way port-channel 4-way port-channel

Po160 Po20
L2 Access
Nexus 5000

vPC Peer Link LACP

Channel (2x10 GigE)

vPC Peer-Keepalive (GigE) 20 flows @1000 pps 20 flows @1000 pps

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 52

vPC on Nexus 7000
Convergence Numbers
Failover case Failure Convergence Time
Topology
Failure Restoration

Failure of 4.1(4) 4.1(4)

P S
secondary vPC
North-Bound: ~700 ms North-Bound: ~3 sec
peer*
South-Bound: ~2.5 sec South-Bound: ~3.4 sec
4.2(1) 4.2(1)
North-Bound: ~50 ms. North-Bound: 100 – 900 ms
South-Bound: ~100 ms South-Bound: 1.2 -2 s
Failure of a 4.1(4) 4.1(4)
P S
primary vPC peer*
North-Bound: ~150 ms North-Bound:~4.5 secs
South-Bound: ~3 sec South-Bound: ~5 secs
4.2(1) 4.2(1)
North-Bound: ~50 ms North-Bound: ~400 ms-1.5 s
South-Bound: ~100 ms South-Bound: ~1.5 s
Failover of the 4.1(4) 4.1(4)
vPC Peer Link P S
North-Bound: ~1.3 s North-Bound: ~900 ms
South-Bound: ~1.8 s South-Bound: up to 10+ s (CSCsz88998)
4.2(1) 4.2(1)
North-Bound: 100-300 ms North-Bound: 150 - 900 ms
South-Bound: 50-500 ms South-Bound: ~ 900 ms–1.5 s

NOTE: Convergence numbers may vary depending on the specific configuration (i.e. scaled
number of VLANs/SVIs or HSRP groups) and traffic patterns (i.e. L2 vs L3 flows).
© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 53
 vPC on Nexus 7000
Scalability Number Improvements
Release Supported Scalability

4.1(5) 192 vPC’s (2-port) with the following,

200 VLANs
200 HSRP Groups
40K MACs & 40K ARPs
10K (S,G) w. 66 OIFs (L3 sources)
3K (S,G) w. 34 OIFs (L2 sources)

Latest 256 vPC’s (4-port) with the following,

Ankara 260 VLANs
4.2(1) 200 SVI/HSRP Groups
40k MACs & 40K ARPs
10K (S,G) w. 66 OIFs (L3 sources)
3K (S,G) w. 64 OIFs (L2 sources)
NOTE: Supported numbers of VLANs/vPCs are NOT related to an hardware or software limit but reflect what
has been currently validated by our QA. The N7k BU is planning to continuously increase these numbers as
soon as new data-points become available. Please contact the ask-nexus7000-pm or ask-nexus7000-tme
alias if you have particular VPC scaling requirements.

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 54

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 55

 vPC Hands-on Lab Information
On Demand vPC Lab Overview

N7K-Aggr N7K-Aggr

N7K-1 N7K-2
POD 1-2 VPC POD 1-2 VPC
Pod 1 Pod 2

Pod 1 Pod 2

 Instructor-led hands-on lab N7K-3

POD 3-4 VPC
N7K-4
POD 3-4 VPC

introducing the vPC (virtual Port-

Pod 4
channel) feature for the Nexus 7000. Pod 3

 Participants exposed to the

configuration of vPC with NX-OS. N7K-7
POD 5-6 VPC
N7K-8
POD 5-6 VPC
 Lab needs to be manually booked
Pod 6
through Nexus 7000 TMEs. Pod 5

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 56

 vPC Hands-on Lab Information
vPC Lab Logistics and Timing
 The vPC Laboratory consists of 6 independent PODs.

 A group of 2 students is assigned to each Pod.

 Each student will configure a vPC peer device.

 PODs are logically independent. Two adjacent PODs are physically

bound to the same Nexus. Virtual Device Contexts (VDCs) are used to
define logically independent devices on the same Nexus 7010 box.

 The vPC Lab session is expected to be completed in around two hours.

 More details on Lab Contacts and Lab Guide on the wiki:(check Lab
Materials section):

http://bock-bock.cisco.com/wiki/N7K:tech_resources:vpc

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 57

 Agenda
 Feature Overview & Terminology
 vPC Design Guidance & Best Practices
Building a vPC domain
Attaching to a vPC domain
Layer 3 and vPC
Spanning Tree Recommendations
Data Center Interconnect (& Encryption)
HSRP with vPC
vPC and Services
vPC latest enhancements
ISSU

 Convergence and Scalability

 vPC Hands-on Lab Information
 Roadmap and Reference Material

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 58

 Roadmap and Reference Material
vPC Plan of Action

Bogota Cairo Delhi Future

• vPC scalability, • vPC scalability, • vPC scalability, • vPC scalability,
new data-point new data-point new data-point new data-point
targets: 50 targets: 768 targets: 2000 targets: 3072
vPCs-2Ports and edge vPC-2ports FEX hosts- FEX hosts-
1000 VLANs and 300 VLANs 2ports and 300 2ports and 200
VLANs VLANs
300 vPCs-4ports • vPC over D1
and 300 VLANs ports • PVLANs over
• 16-port vPC on vPC
• Enhanced vPC D1 modules with • Config sync for
dual Active N5K downstream vPC
support • Port-Security • vPC for FEX
over vPC Host Ports

CCd and ECd Not CCd Not CCd Not CCd

1HCY’10 2HCY’10 1HCY’11

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 59

 Roadmap and Reference Material
vPC/VSS Interop Test Details
Physical Logical

L3 Core

E2
/1
4
/1

4
E2

N7K-1 N7K-2
L2/L3 Aggregation

Nexus 7000 vPC

E1 Po10 26
/2 E 1/
5
E1/26 E1/25
Po100 Po100
2
2/ Te2
Te1/2/1 e 1/ /2/ Te2/2/1
T 2

6K-1
6K-2 L2 Access vPC Peer Link LACP
Channel (2x10 GigE)
6500 VSS
vPC Peer-
Keepalive (GigE)

VSS VSL Channel

(2x10 GigE)

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 60

 Roadmap and Reference Material
vPC/VSS Interop Test Details

 The following scenarios were tested:

• VSS and vPC member failover and convergence
• Dual active scenarios and behavior
• Best practice guidelines for STP, L3 (NSF), Multicast
 Catalyst 6500/Nexus 7000 interoperability:
• Multiple ports per chassis act as one larger ether-channel
• More info:
http://bock-bock.cisco.com/wiki/N7K:tech_resources:vpc:vss_a
nd_vpc

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 61

 Roadmap and Reference Material
Other Solution Tests

 Enterprise Solutions Engineering:

http://www.cisco.com/en/US/docs/solutions/Enterprise/Data_Center/DC_3_0/DC-3_0_IPInfra.html

 vMotion in and between Datacenters:

http://bock-bock/wiki/CSSTG_TechMarketing:VMware-Cisco-vMotion

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 62

 Roadmap and Reference Material
New vPC Documentation and References

 vPC 4.2(1) Enhancements/Features:

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/vPC_Ankara_New_Features_Overview.ppt

 vPC 4.2(1) Convergence numbers:

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/vPC_failover_convergence_brief_Ankara.pptx

 On Demand GOLD Lab for vPC Hands-On:

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/NXOS_Virtual_Port_Channel_Lab_v1.pdf

 Recommendations for vPC deployment with single 10 GE Module:

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/vPC_Single_10G_module.pptx
 Configuration Guide for Object Tracking Feature:
http://www.cisco.com/en/US/partner/docs/switches/datacenter/sw/4_2/nx-os/interfaces/configuration/guide/if_vPC.html#wp1530133

 vPC L3 white Paper:

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/vPC_L3_Interactions.doc

 vPC Best Practice Paper:

http://bock-bock.cisco.com/wiki_file/N7K:tech_resources:vpc/vPC_Best_Practices.doc

© 2009 Cisco Systems, Inc. All rights reserved. Cisco Confidential 63