Lesson 3| Configuring Layer 2 Switching Features Overview Layer 2 switching is a critical aspect of the data center network. To support the requirements of high-availability clusters and workload mobility, VLANs often need to be stretched across many different switches. To ensure that the foundation of the data center infrastructure is sound, Cisco Nexus switches support a wide range of features that help to scale, manage, and. secure the Layer 2 switched network. Objectives Upon completing this lesson, you will be able to configure Layer 2 switching features to support network requirements when given an implementation plan. You will be able to meet these objectives: Identify how to configure basic interface parameters on the Cisco Nexus 5000 and 7000 Series switch interfaces and Cisco Nexus 5500 Platform switch interfac m= Identify the differences between the Layer 2 switching features of the Cisco Nexus 5000 and 7000 Series switches and the Cisco Nexus 5500 Platform switches Identify how to configure VLANs on Cisco Nexus switches = Identify how to use and configure the STP extensions on Cisco Nexus switchesBasic Interface Parameters This topic identifies how to configure basic interface parameters on the Cisco Nexus 5000 and. 7000 Series switch interfaces as well as the Cisco Nexus 5500 Platform switch interfaces, Layer 2 and Layer 3 Interfaces + All physical Ethemet interfaces on a Cisco Nexus switch are designated as interface ethernet sloV/port regardless of interface type and speed Taltan) show interface ethernet 1/T Bthemeti/i se op ardware: 10000 Ethernet, address: 0026.9908.a942 (bia cBéc.7566.4-0c) Miu 1800 bytes, Bw 10000000 Rese, ‘LY 10 woes reliability 255/255, txload 1/285, rdoad 1/255 meapeul ation AREA foll-dipler, 10 Gb/2, nadia type is 106 output enivtod> Nexus 5500/7000 support Layer 3 interfaces, in addition to Layer 2 3) w cmtaen —S|_LayerSiteram shgwimerace | 7 Tayer Dances wiataces, seth oun iy tf range) meatenpart mde access ieracerange ah . Tay ard ink nae, Interfacegroun. Cisco Nexus Operating System (NX-OS) Software supports the following types of interfaces: © Physical: Ethernet (10/100/1000/10G) Logical: PortChannel, loopback, null, switch virtual interface (SVI), tunnel, subinterface . = In-Band: Sup-eth0, Sup-cored . Management: Management, Connectivity Management Processor (CMP) All Ethernet interfaces are named “Ethernet.” There is no differentiation in the naming, convention for different speeds. ‘The show interface command displays the operational state of any interface, including the reason why that interface might be down, Interface Ranges and Groups When configuring multiple interfaces with the same parameters, you can use the interface range feature rather than configuring each interface singularly. The interface range configuration mode allows you to configure multiple interfaces with the same configuration parameters. After you enter interface range configuration mode, all command parameters that you enter are attributed to all interfaces within that range until you exit interface range configuration mode. You enter a range of interfaces using hyphens (-) and commas (,). Hyphens separate contiguous and commas separate discontiguous interfaces. When you enter discontiguous . you must enter the media type for each interface. 278 __ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncCisco Nexus 5500 Platform switch and Cisco Nexus 7000 Series switch interfaces may operate as either Layer 2 switch ports or Layer 3 routed ports. Using the no switchport command while in interface configuration mode sets the interface or range of interfac for Layer 3 operation. Issuing the switehport command followed by the switehport mode access or switehport mode trunk commands sets the interface for Layer 2 operation. Note The defauit mode of operation for all ports on a Cisco Nexus 7000 Series switch is Layer 3 ‘mode. If you prefer that port default to be Layer 2 mode, use the system default switchport command to change this behavior. ‘©2012 Cisco Systems, Inc {isco Nexus Switch Feature Configuration 2.79Shared vs. Dedicated Mode + Some Cisco Nexus 7000 10 Gigabit Ethemet interfaces operate in either shared or dedicated mode + For example N7K-MI32XP.12 VO module * Dedicated mode Only first interface in port group can be configured for dedicated mode Al other interfaces in the port group must be shut down Whi [contig)) interface ethemet 1/17, ethemet 1/19, e i/21, © 1/3 ite {oon igri trangeyf ahwedown Wk toon ig-it-ronge}¥ antertace ethernet 1/17 Nk Ltooat igri }4 no ahuedown + Shared mode Default setting Reversal to shared mode: interne gihemee TAT, © 718, © WH, @ TT Cisco Nexus 7000 Series switch 10 Gigabit Ethemet interfaces on the N7K-M132XP-12(L) VO ‘modules are arranged into port groups that are serviced by a port group ASIC. There are eight port groups on a N7K-M132XP-12(L) I/O module, arranged as shown in the table. Port Group Interfaces Port Group 1 Interfaces 1,3, 5, and 7 Port Group 2 Interfaces 2,4, 6, and 8 Port Group 3 Interfaces 8, 11, 13, and 15 Port Group 4 Interfaces 10, 12, 14, and 16 Port Group 5 Interfaces 17, 19, 21, and 23 Port Group 6 Interfaces 18, 20, 22, and 24 Port Group 7 Interfaces 25, 27, 29, and 31 Port Group 8 Interfaces 26, 28, 30, and 32 The port group ASIC provides 10 Gb/s of throughput to each port group. The interfaces in these port groups may operate in either a shared or dedicated mode. When they operate in shared mode, all four interfaces within the port group are active and share the 10 Gb/s of throughput. When they operate in dedicated mode, only the first interface within cach port group is active, and the other three are disabled. Shared mode is typically used for server access, where full and continuous 10 Gb/s of uplink bandwidth may not be required. Dedicated mode is typically used for switch-to-switeh uplinks and connections. ‘The bottom configuration in the figure shows the configuration steps to revert a range of interfaces to the shared mode. Note ‘The show membership. jerface ethernet X/Y capabi ‘command shows you the port group 80 __ Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncUnidirectional Link Detection (UDLD) * Detects unidirectional links by combining Layer 1 and Layer 2 mechanisms + When detected: shut down port and (optionally) generate syslog message + fnot detected: risk of bridging loop as two adjacent ports would be designated Root bridge ‘Bdge promty:25476 fxdge prionty 32768, © ecg pts Braige prot ‘2768 Unidirectional Link Detection (UDLD) gives devices the ability to detect unidirectional links within the network. When a unidirectional link is detected, UDLD shuts down the affected LAN port and alerts the user. Unidirectional links can cause various problems, including, spanning-tree topology loops. UDLD works with Layer | protocols to determine the physical status of a link. At Layer 1, autonegotiation manages physical signaling and fault detection. At Layer 2, UDLD performs tasks that autonegotiation cannot perform. These tasks include detecting the identities of neighbors and shutting down misconnected LAN ports. When autonegotiation and UDLD are both enabled, Layer 1 and Layer 2 detection functions work together to prevent physical and logical unidirectional connections and the malfunctioning of other protocols. A unidirectional link occurs when two-way traffic is suddenly reduced to traveling in a single direction. Ifa strand from a fiber pair is disconnected, autonegotiation ensures that the link becomes suspended. In this case, the logical link is undetermined, and UDLD takes no action. If both fibers are working normally at Layer |, UDLD determines whether both fibers are connected correctly and whether traffic is flowing bidirectionally between the two neighbors. This task cannot be performed by autonegotiation because autonegotiation is restricted to Layer 1 The switches periodically transmit UDLD packets to neighbor devices on LAN ports with UDLD enabled. If the packets are echoed back without a specific acknowledgment (echo), the link is then marked as unidirectional and the port is shut down. Devices on both ends of the link. must support UDLD for the protocol to successfully identify and disable unidirectional links. UDLD uses a special MAC address: 0100.0CCC.CCCC, ‘©2012 Gisco Systems, (isco Nexus Switch Feature Configuration 2-81Configuring UDLD 1. Enable UDLD in normal mode for all fiber-optic interfaces. 2. Enable aggressive mode for all fiber-optic interfaces (optional) ~ Whon aport stops receiving UDLD frames, it tries to reostablish the UDLD connection & times, then the port is disabled 3. Modify individual interfaces (optional) sable, re-enable, or enable using aggressive mode 4. View UDLD neighbors (optional) writch contig) ¥ feature wala @ vitch (contig) # wdtd aggressive @ sviteh config) | Sntertace ethernet 2/2, ethernet 2/ Svitch (contig it} wild disable avitcht show udld neighbors omz234230 1 Ethemet2/1 bidirectional jmz23230 1 Ethemet2/3 bidirectional ‘To use UDLD on the Ciseo Nexus switches, enable the UDLD feature by using the feature udld command, After globally enabling all 10-Gb (fiber) interfaces, run UDLD automatically. However, for the 1-Gb (copper) interfaces, UDLD must be manually enabled per each interface, UDLD supports two operational modes—normal mode, which is the default, and aggressive ‘mode, which must be specifically enabled. UDLD aggressive mode can only be used on point-to-point links between network devices that are capable of supporting this mode. When a port on a bidirectional link stops receiving UDLD packets, UDLD tries to reestablish the connection with the affected neighbor. UDLD disables the port after eight failed retr UDLD configuration commands are as follows: = feature udld: Enables the UDLD feature = udld aggressive: Enables aggressive mode globally = interface ype slot/port: Enters the interface subconfiguration mode = udld {enable | disable aggressive}: Configures the UDLD mode on the interface When UDLD is configured globally, the following must be taken into consideration: = All 10-Gb (fiber) interfaces run UDLD automatically. © For 1-Gb (copper) interfaces, you must manually enable UDLD per each interface. 282 __ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncPort Profiles Groups of commands can be configured on interfaces through a port, profile ‘Separate types of profiles exist for Ethernet, VLAN, and PortChannel interfaces. Port profiles can be inherited ina hierarchical manner. Port profile type and interface mode (Layer 2 orLayer 3) need to match in order to inherit a profile. Salven (ontigy) pore-protile type ethernet SERVERS ovitch (conFig~port=prot)t switehport Gritch contig-post-prat) | Spanning’ ree port type edge Siltch (contig-port-prot}# ewitelpert mede access wich loin peyote type etemet Wun saMERS Bitten (eontig-pore-protft svieenpOrt access vlan 10 Profignheriepotio switch (confiq-port~prot)4 inherit port-profile SERVERS Saee Sania pote itch (contig) | interface ethernet 1/1 WEB SERVERS and ovitch (conid~L1)1 inberit. port-profile NEB-SERVERS SERVERS ‘On Cisco Nexus switches, you can create a port profile that contains many interface commands and then apply that port profile to a range of interfaces. Each port profile can be applied only to a specific type of interface. The supported interface types are Ethernet, VLAN, ot PortChannel interfaces, Note When you choose Ethernet as the interface type, the port profile is in the default mode, Which is Layer 3. Enter the switchport command to change the port profile to Layer 2 mode. You inherit the port profile when you attach the port profile to an interface or range of interfaces. When you attach—or inherit—a port profile to an interface or range of interfaces, the system applies all the commands in that port profile to the interfaces. Note To apply the commands in the port profile to the interface, the port profile needs to be enabled through the state enabled command. By default, port profiles are not enabled, Additionally, you can have one port profile inherit another port profile, which allows the initial port profile to assume all of the commands of the second inherited port profile that do not conflict with the initial port profile. Four levels of inheritance are supported, except for the switchport private-vlan mapping and private-vlan mapping commands, which support only one level of inheritance. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2-83Verifying Port Profiles Verify port profile configuration, inheritance, and evaluated configuration ort-proti le WES-SERVERS ‘ype: Ethernet Gaeersption: ‘Seaton enabled config at tributes: Seitenpore Sritehport access vlan 10 sie Sees ae Inter aoesto which the pote] hs boon applied Behernet3/4 Seren one output emitted Use the show port-profile command to display information about the configured port profiles on the device. If the command is used without any additional parameters, it displays all configured port profiles. Further command options can be used to gather more specific information. The following options can be used: = show port-profile expand-interface: This option shows the expanded interface configuration for each interface that has a port profile applied to it, Output can be limited to a specific port profile. = show port-profile usage: This option shows which interfaces have a specific port profile applied to them. The name keyword can be used to limit the output to a specific port profile. 284 __ Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncCisco Nexus Unified Port Technology + Configure a physical port as one of: = 1110-Gigabit Ethernet ~ Fibre Channel over Ethernet (FCoE) 41-,2:, 4, 8 Gigabit native Fibre Channel port + Available on Cisco Nexus 5500 Platform switches, ‘Cisco NX-OS Release 5.0(3)N1(1b) or later ~ Cisco Nexus §548UP and 5596UP Switches and expansion modules * Aspects of unified fabric: Unified platform (same platform architecture and softw are for LAN and SAN) Untied device (cabling the same device) Unified wire (convergence on a single CNA and cable) Nexus: 0 Gigabit Ethemet 00 i Naive C( 1-2-4 gab Beginning with Cisco NX-OS Release 5.0(3)N1( Ib), Cisco introduced the Cisco Nexus unified port technology. Cisco Nexus unified ports allow you to configure a physical port on a Cisco Nexus 5500 Platform switch as a 1/10-Gigabit Ethernet, Fibre Channel over Ethernet (FCoE), or I-, 2+, 4+, or 8-Gigabit native Fibre Channel port Currently, most networks have two types of switches for different types of networks. For example, LAN switches carry Ethernet traffic up to Cisco Catalyst switches, and SAN switches carry Fibre Channel traffic from servers to Cisco MDS switches. With unified port technology, you can deploy a unified platform, unified device, and unified wire approach. Unified ports, allow you to move from an existing segregated platform approach—where you choose LAN and SAN port options—to a single unified fabric that is transparent and consistent with existing, practices and management software. A unified fabric includes the following = Unified platform: Uses the same hardware platform and the same software code level and certifies it once for your LAN and SAN environments. = Unified device: Runs LAN and SAN services on the same platform switch, The unified device allows you to connect your Ethernet and Fibre Channel cables to the same device. = Unified wire: Converges LAN and SAN networks on a Adapter (CNA) and connects them to your server. ingle Converged Network A unified fabric allows you to manage Ethernet and FCoE featurs existing Cisco tools, ‘The new Cisco Nexus 5S48UP Switch and the Cisco Nexus 5596UP Switch provide bu Unified port technology. In addition, a new unified port expansion module and two Layer 3 modules increase the benefits ofa deployed unified fabric: independently by using the in ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2-85Fabric Extenders « The Cisco Nexus 2000 Series Fabric Extenders serve as remote VO modules of a Cisco Nexus 5000/5500 or 7000 Switch: ~ Managed and configured fromparent switch + Together, parent switches and Cisco Nexus 2000 Series Fabric Extender combine benefits of top-of rack cabling with end-of-ow management => iy Cisco Nexus 2000 Series Fabric Extenders (FEXs) can be deployed together with Cisco Nexus 5000 or Cisco Nexus 7000 Series switches to create a data center network that combines the advantages of a top-of-rack (ToR) design with the advantages of an end of row (EoR) design. Dual-redundant Cisco Nexus 2000 Series FEXs are placed at the top of each rack. The uplink ports on the Cisco Nexus 2000 Series FEXs are connected to a Cisco Nexus 5000 or 7000 Series switch that is installed in the EoR position, From a cabling standpoint, this design isa ToR design. The cabling between the servers and the Cisco Nexus 2000 FEX is contained within the rack. Only a limited number of cables need to be run between the racks to support the 10 Gigabit Ethernet connections between the Cisco Nexus 2000 Series FEXs and the Cisco Nexus switches in the EoR position, From a network deployment standpoint, however, this design is an EoR design. The FEXs act as remote I/O modules for the Cisco Nexus switches, which means that the ports on the Cisco Nexus 2000 Series FEX act as ports on the associated switch, In the logical network topology, the FEXs disappear from the picture, and all servers appear as directly connected to the Cisco Nexus switch. From a network operations perspective, this design has the simplicity that is normally associated with EoR designs. All the configuration tasks for this type of data center design are performed on the EoR switches. There are no configuration or software maintenance tasks that are associated with the FEXs 286 __ Implementing Cisco Data Genter Untied Fabric (OGUFI) v5.0 (© 2012 Cisco Systems, IncFEX Deployment Models FEXs can be deployed using three different models: ‘1. Straight-through FEXusing static pinning (discussed here) 2. Straight-through FEX.using dynamic pinning Uses PortChannels; discussed in next lesson 3. Active-active FEXusing vPC = Uses Port Channels and vitual Port Channels; discussed in next lesson Straight trough DyramePnig © eRe e © reweaawe i ug 9 =, '9 3 mus 50005500 ‘o901500015800 News 50005500 ‘There are three deployment models that are used to deploy FEXs together with the Ciseo Nexus 5000 and Cisco Nexus 7000 Series switches: m= Straight-through using static pinning: In the straight-through model, each FEX is connected to a single Cisco Nexus switch. The single switch that the FEX is connected to exclusively manages the ports on that FEX. Static pinning means that each downlink server port on the FEX is statically pinned to one of the uplinks between the FEX and the switch, ‘Traffic to and from a specific server port always uses the same uplink. This model is, discussed in this lesson. = Str sing dynamic pinning: This deployment model also uses the straight- through connection mode! between the FEXs and the switches. However, there is no static relationship between the downlink server ports and the uplink ports. The ports between the FEX and the switch are bundled into a port channel, and traffic is distributed across the uplinks based on the port channel hashing mechanism, Port channels and this FEX deployment model are discussed in the “Configuring Port Channels” lesson. = Active-active FEX using virtual port channel (vPC): In this deployment model, the FEX is dual-homed to two Cisco Nexus switches. vPC is used on the link between the FEX and the pair of switches. Traffic is forwarded between the FEX and the switches based on vPC forwarding mechanisms. vPC and this FEX deployment model are discussed in the “Configuring Port Channels” lesson. Note Cisco Nexus 7000 Series switches currently support only straight-through deployment using dynamic pinning. Static pinning and active-active FEX are currently supported only on Cisco Nexus 5000 Series switches. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2.87FEX with Static Pinning « Static pinning statically maps server-FEX downlink ports to the uplink ports that connectthe FEXto the parent switch + Port mapping depends on the number of uplink ports that are used and the number of downlinks on the FEX + When an uplink port fails, all downlink ports pinned toit are disabled Oversubscription ratio is preserved ~ Single-homed servers lose connectivity halides = Duathoned serversfail over tothe other NC 3a In static pinning mode, the server ports on the Cisco Nexus 2000 Series FEX are statically pinned to one of the uplink ports. For example, when a Cisco Nexus 2000 Series FEX with 48 Gigabit Ethernet server ports is deployed in static pinning mode using four 10 Gigabit Ethernet, uplink ports to the Cisco Nexus 5000 Series switch, 12 server ports will be pinned to each uplink port. Ports 1-12 are pinned to the first uplink, ports 13-24 to the second uplink, ports, 25-36 to the third uplink, and ports 37-48 to the fourth uplink. This results in an oversubscription ratio of 1.2:1, because a group of 12 Gigabit Ethernet server ports shares the bandwidth of one 10 Gigabit Ethernet uplink. Ifone of the uplinks between the Cisco Nexus 2000 Series FEX and the Cisco Nexus 5000 Series switch fails, the FEX will disable the server ports that are pinned to that uplink port. example, ifthe fourth uplink fails, server ports 37-48 will be disabled, Servers that are connected to these ports will see the associated Ethernet link go down. If the servers are dual- homed and use some form of network interface card (NIC) redundancy, then this mechanism will be triggered, and the server will fail over to the other NIC. A single-homed server will simply lose connectivity if it is connected to one of the ports that are pinned to the failed uplink port. The oversubscription ratio on the other ports remains unchanged. The oversubscription rate remains unchanged because each of the other three groups of 12 Gigabit Ethernet ports is still sharing the same 10 Gigabit Ethernet uplink port as before. For 288 __ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncConfiguring FEX with Static Pinning All FEXconfiguration performed on parent switch: 1. Enable the FEX feature: 2. Configure the FEXinstance number 3. Define number of uplinks used for static pinning 4 5 Set FEXFabric mode Associate the ports with the FEX THRTnriay) feature fox @) vox (eontig) fox 21 @ ex (contig-fox}¥ deseription “FE 111, rack 1° GK config-fox) pinning max-tinks ¢ Sunge’in'vas Links will catse trattic ateruption © 6K (contig) Anterface ethernet 1/1-4 WK (contig-Lt-rango) switehport mode fex-tabric @ vo contig-it-rangey# fax associate 112 @ Cee ro ces All configuration and discovery for the Cisco Nexus 2000 Series FEX is performed from the parent switch and involves these steps: Step1 Enable the FEX feature (for Cisco Nexus 5000 Series switch and 5500 Platform switch). The equivalent installation and enabling occurs on Cisco Nexus 7000 Series switches in a virtual device context (VDC). ue the fex chassis-number sis number may be Step2 Create the FEX instance, To create an FEX instan command from within the global configuration context. The cha any integer from 100-199. Step 3 Once the FEX instance has been created, the configuration context changes to FEX configuration mode, where a description may be added. The pinning max-links /-4 command binds the 48 server-facing ports to the uplink ports (up to four statie ports may be activated), according to the following max-links argument. AAIL48 server-facing ports will be pinned to a single active uplink port (interface Ethemet CN/1/1-48), where CN = chassis number. pi pinning max-links 2 | All 48 server-facing ports will be pinned to two active uplink pors (interface Ethernet GN/1/1-24 assigned to the first active uplink port and interface Ethemet CN/1/25-48 assigned to the second active uplink port). jing max-links 3 | All 48 server-facing ports willbe pinned to three active uplink ports (interface Ethernet CN/1/1-16 assigned to the fist active uplink port, interface Ethernet CN/1/17-32 assigned to the second active uplink port, and interface Ethernet CN/1/33-48 assigned to the third active uplink port) pinning max-links 4 | All 48 server-facing ports will be pinned to all four uplink ports (Ethernet CN/1/1-12 assigned to first active uplink port, Ethernet CN/1/13-24 assigned to the second active uplink port, Ethernet CN/1/25-36 assigned to third active uplink port, and Ethernet CN/1/37-48 assigned to fourth active uplink port) Step4 Configure the interface mode by using the switehport mode fex-fabrie command. Step 5 Associate a parent switch interface to a Cisco Nexus 2000 Series FEX uplink port fora specific FEX by using the fex associate chassis-numher command. ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2-69VLAN Configuration This topic identifies how to configure VLANs on Cisco Nexus switches. VLANs + Cisco Nexus switches support up to 4094 VLANs in each VDC ~ h accordance with the EEE 802.10 standard - 81 VLANs in the high end of the VLAN range are reserved for internal use by the system and cannot be used + VLANs in a VDC are isolated from VLANs in other VOCs + Support for VLAN Trunking Protocol (VTP) hh Cisco NX-OS release 5.1(1) and later - VIP v1/2in server, clent, transparent , and off modes; VIP pruning TWik-l(eonfigy# wlan 20 NIK-1(contig-vian}# exit NUKL (config) # ewitehte vde Red NIk-1-Red¥ contig NIK-1-Red (contig) ¥ vlan 20 Nik-1-Red (cont iq-vlan}# Layer 2 ports on Cisco Nexus switches can be configured as access ports or 802.1Q trunk ports By default, they are configured as access ports. A switch port belongs to a VLAN. Unicast, broadcast, and multicast packets are forwarded and flooded only to end stations in that VLAN. Each VLAN is considered a logical network. Packets that are destined for a station that does not belong to the same VLAN must be forwarded through a router. ‘The Cisco Nexus 7000 Series switches support up to 4094 VLANs, which are organized into ranges: = VLAN L: The default VLAN cannot be modified or deleted. = VLAN 2-1005: Normal VLAN that can be created, used, modified, and deleted. = VLAN 1006-4094: Extended VLANs that can be created, named, and used. The state of these VLANS is always active, and the VLAN is always enabled and cannot be shut down, = VLAN 3968-4047 and 4094: Allocated for internal use only VLANs 3968-4047 and 4094 are reserved for internal use in each VDC for features that need to use internal VLANs for their operation—for example, multicast and diagnos Due to the use of VDCs, a VLAN number ean be reused in different VDCs because each VDC isa separate virtual device, The maximum number of VLANS in all VDCs is 16,000. VLAN Trunking Protocol (VTP) is supported in Cisco NX-OS Release 5.1(1) and later. Supported VTP features include VTP v1/2 in the server, client, transparent, and off mod well as VTP pruning. Implementing Cisco Data Genter Unified Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncPrivate VLANs + Private VLANs can be used to implement Layer? isolation within a single VLAN and associated subnet. + The primary VLAN represents the VLAN and associated subnet tothe rest of the network, + Secondary VLANs isolate hosts within the VLAN. Private VLAN. ‘ThisPVLAN isconfigured with tree distinct secondary VLANs. Deploying private VLANs (PVLANS) in an enterprise data center environment provides an effective means of sparing IP address space and controlling Layer 2 access to servers and. devices residing within the server farm. The Layer 2 isolation that is provided by PVLANs an excellent way to supplement Layer 3 security that is already used to protect a particular server farm subnet, ‘Two major benefits of deploying PVLANs are conserving IP address space and providing isolation for servers residing in the same subnet. Two VLAN concepts that are associated with PVLAN configuration are primary and secondary VLANs. Secondary VLANs consist of isolated VLANs and community VLANs, Servers residing in an isolated VLAN can only ‘communicate through the primary VLAN are isolated at Layer 2 from any other servers that are configured in the same or any other isolated VLANs. Servers that are part of a community VLAN can communicate at Layer 2 with all other servers residing in the same community VLAN. However, they must still communicate with other devices or servers through the primary VLAN. Any servers or applications that communicate using Layer 2 protocols such as multicast should be placed in the same community VLAN. As previously stated, all traffic to and from the isolated and community VLANSis first forwarded through the primary VLAN. Each primary VLAN is associated with a promiscuous port. Therefore, each isolated and community VLAN must be mapped to a primary VLAN. A promiscuous port can be configured either as a standard promiscuous port, which is the PVLAN equivalent of an access port, or as a promiscuous trunk port. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2.09Community VLAN + Secondary community VLANs can be used to create subgroups within the primary VLAN. * There is Layer2 connectivity within each community VLAN, but not between community VLANs. PromiscuousPort Community VLANA Community VLANB t =A Ports in community VLANAcan —‘t___. portsin different community talkto other portswithin the VLANscannotcommunicate ‘same community VLAN, without going through the promiscuousport In cases where similar systems do not need to interact directly, PVLANs provide additional protection at a Layer 2 level. PVLANs are an association of primary and secondary VLANs. A primary VLAN defines the broadcast domain to which secondary VLANs are associated. ‘The secondary VLANs can be either isolated VLANs or community VLANs. Hosts on isolated VLANs communicate only with the associated promiscuous ports in a primary VLAN, while hosts on community VLANs communicate among themselves and with the associated promiscuous ports. ‘To use PVLANS, the private VLAN feature must first be enabled. After there are operational ports in a PVLAN, that feature cannot then be disabled. The private VLAN feature permits partitioning of a Layer 2 broadcast domain on a VLAN into subdomains while still using the same Layer 3 subnet Community VLANs are ports within a community VLAN that can communicate with each other but cannot communicate with ports in other community VLANs or any isolated VLANs at the Layer 2 level. A PVLAN host port is either a community PVLAN port or an isolated PVLAN port, depending on the type of secondary VLAN with which it is associated 2-100 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncIsolated VLAN * Anisolated secondary VLAN creates a subgroup within the primary VLAN in which all hosts are isolated from each other at Layer 2. PromiscuousPort QOOQCIE!@GeQee t Ltt I Portsinisolated VLANAcannottalkto ____ communicate with othersscondary otherpotsin isolated VLANA. ‘VLANs through the promiscuousport, APVLAN can only have one isolated VLAN. A secondary isolated VLAN creates a subgroup within the primary VLAN that isolates hosts from each other within that secondary VLAN. Ports within an isolated VLAN cannot communicate with each other at a Layer 2 level. Any port that is associated with the isolated VLAN has complete Layer 2 isolation from other ports within the same PVLAN domain, except that it can communicate with associated promiscuous ports. PVLANS block all traffie to isolated ports except traffic from promiscuous ports. ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2107Promiscuous Port + Promiscuous ports provide outside connectivity for the secondary VLAN. + Traffic from a promiscuous port is sent to all ports in the associated secondary VLANs, and traffic from all ports in the secondary VLANs is sent to the promiscuous port. Promiscuous # Port Community Community Isolated Community VLANA VLANC VLAND VLANB Promiscuous ports belong to the primary VLAN. The promiscuous port can communicate with all ports, including the community and isolated host ports that belong to the secondary VLANs ‘sociated with the promiscuous port and ports that are associated with the primary VLAN. Within a primary VLAN, there can be several promiscuous ports. Each promiscuous port can have several secondary VLANs, or no secondary VLANS, associated with that port. A secondary VLAN can be associated with more than one promiscuous port as long as the promiscuous port and secondary VLANS are within the same primary VLAN. This option might be used for load-balancing or redundancy purposes. If you have secondary VLANs that are not associated with any promiscuous port, these secondary VLANs cannot communicate with the outside world, PVLANS only control Layer 2 connectivity within the VLAN. ACLs can be used to control the traffic that passes between these VLANs at Layer 3. 2102 _ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncConfiguring Private VLANs 1. Enable the PVLAN feature 2. Create primary VLAN 3. Create secondary VLANs of the appropriate types. 4. Associate secondary PVLANs with the primary PVLAN Tf pesvate-stan peimary@) van 12 Youcanaddor Privetervlan areociation 100-109! remove assovates WANS by using add shor vlan private-vian ‘angremove ‘ne arte Rey words ag, cmay ine When configuring a PVLAN, the private VLAN feature must first be enabled. You can then start using the commands to configure primary and secondary VLANS. To configure a VLAN as a primary VLAN, first create the VLAN, and then configure it as a primary VLAN Next, the secondary VLANs must be created and designated as secondary PVLANS. A secondary PVLAN must be configured either as type isolated or type community. To configure a range of VLANsas secondary PVLANS, use the vlan vian-range command. ‘The secondary PVLANs must be associated with the primary PVLAN in VLAN configuration mode, Use the following guidelines when associating secondary VLANs with a primary VLAN: m= The secondary_vian_list parameter can contain multiple community VLAN IDs. m= The secondary_vlan_list parameter can contain multiple isolated VLAN IDs, although it is common to have only a single isolated VLAN. m= Enter a secondary_vlan_list value or use the add keyword with a secondary _vlan_list value to associate secondary VLANs with a primary VLAN, Use the remove keyword with a secondary _vlan_list value to clear associations between secondary VLANS and a primary VLAN. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2103Associating Layer 3 Interface + Promiscuous ports can provide outside connectivity for Layer 2 switched traffic. + To provide outside connectivity for Layer 3 switched traffic from the PVLAN, associate the secondary VLANs with the SVI for the primary VLAN. witah (coaTlg)¥ Feature interface-vian Te nada: en suteh (config) # Snterface vlan 142 Sitch (config i0}# peiwate-vien mapping 100-103 Ingress Layer3 agnntched tat se Promiscuous ports or promiscuous trunk ports provide Layer 2 switched connectivity to the outside world, either to a network device—such as a switch, router, or firewall—or to a specific host, such as a backup server. When PVLANS are implemented on a Layer 3 switch, such as the Cisco Nexus 7000 Series switch, itis also possible to provide Layer 3 switched connectivity to the rest of the network via an SVI. To allow the secondary VLANs to use the SVI for the primary VLAN as a Layer 3 gateway to other subnets, itis necessary to associate the secondary VLANS with the SVI for the primary VLAN. Consider the following guidelines when mapping secondary VLANs to the Layer 3 VLAN interface of a primary VLAN: = The private-vlan mapping command only affects PVLAN ingress traffic that is Layer 3 switched. = Enter a secondary_vlan_list parameter, or use the add keyword with a secondary_vlan_list parameter to map the secondary VLANs to the primary VLAN. Use the remove keyword with a secondary_vlan_list parameter to clear the mapping, between secondary VLANs and the primary VLAN. The example shows how to permit routing of secondary VLAN ingress traffic from PVLANs 100-103 and VLAN 142. 2-104 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncConfiguring Layer 2 Host Port in PVLAN + Configure a Layer 2 port as a member ofa community orisolated VLAN. Sitcileontige ich aatetpert ovitch (contignif}) switehport mode private-rlan host mitch con Fig-if)¥ meitehport peivate-vian host-asscciation 142 101, mitch (conFig-if)¥ show interface ethemet 2/3 ewitehport — [Camin\y Wan tame: Ethernet 2/3 ininistrative Mode: peivate-vian host Gperstional Moder op ‘Aiministrative Trunking Encapsulation: negotiate Ragotiation of Trunking: Off Recess Mode VLAN: 1 (defaule) Trunking Native Mode VLAN: 1 (default) Rininietrative private-vian Bost-association: 142 (WLANOL42) 101 (VIAOI01) Adninsetrative private-vien mapping: none Qperational private-vlan: none To configure a Layer 2 port as a host port, use the switehport mode private-vlan host command, To associate the port with a primary and secondary VLAN, use the switehport private-vian host association command. Whether this port is a community port or an isolated port is determined by the PVLAN type of the secondary VLAN that is assigned to the port. This figure shows how to configure interface Ethernet 2/3 as a Layer 2 host port in a PVLAN. ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2105Configuring Layer 2 Port as Promiscuous Port + Configure a Layer 2 port as a promiscuous port. Wii (soatig)) interface ethernet 2/4 Wet (sone tet ewsechpere Ki (sone ign) Switotport mode private-vian promiscuous WK (oontig-Lt} 1 Switelport private-vian mapping 142. 100-103 WA (cone Lg-L#)4 show Antextace ethemet 2/4 awitahport ama: eens 2/4 Setechport’ Enable Ranintserative node: promiacious Grosset node: op Saninsserative Trunking Eocapaslation: negotiate fmgotiation of trunking: off otese bode vente 1 (east ©) [Trunking Native Mode VLAN: ? (default) inins erative Private-vian host-association: none nsnsseentive pesvace-vien mappings 242 (VEa¥01 2) 100 (VEAMOI00) 101 epesst ions privatacvan: sone ‘To configure a port as a promiscuous port, use the switchport mode private-vlan host command. To map the primary and secondary VLANS to the promiscuous port, use the switchport private-vlan mapping command PVLAN, ‘The figure shows how to configure interface Ethernet 2/4 as a promiscuous port in a 2-106 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncRapid Per VLAN Spanning Tree Plus (Rapid PVST+) * Cisco Nexus switches nun the Rapid Per VLAN Spanning Tree Plus (Rapid PVST+) protocol by default for all VLANs. + Rapid PVST+ uses a separate instance of the 802. 1w RSTP protocol for each VLAN. ‘Switch Primary link fails FSTP falover occurs Spanning Tree Protocol (STP) 802.1D was designed at a time when recovering within a minute after an outage was considered adequate. However, with the advent of Layer 3 switching in LAN environments, bridging and switching methods are now competing with routed solutions, such as Open Shortest Path First (OSPF) and Enhanced Interior Gateway Routing Protocol {EIGRP) to provide alternate paths more quickly than was previously possible. Cisco has enhanced the original 802.1 specification with extensions such as UplinkFast, BackboneFast, and PortFast to accelerate the convergence time of a bridged network. The disadvantage of these solutions is that they are proprietary solutions and require additional configuration to tune their performance. Rapid Spanning Tree Protocol (RSTP) IEEE 802.1w represents an evolution of the 802.1D standard, The 802.1D terminology remains basically unchanged in 802.1, as do most parameters, thereby making it easier for users to configure the new protocol. In most cases, RSTP performs better than Cisco proprietary extensions without necessary additional configuration, RSTP 802.1w is also capable of reverting to 802.1D in order to interoperate with legacy bridges on a per-port basis. Reversion for legacy bridges loses the convergence benefits that were introduced by 802.1. Per VLAN Spanning Tree Plus (PVST+) allows the definition of one spanning-tree instance per VLAN. Normal PVST* relies on the use of the older $02.1D STP to reconverge the STP domain in the case of link failures. Rapid Per VLAN Spanning Tree (Rapid PVST) allows the use of 802.w with Cisco PVST in order to provide a much faster convergence per VLAN. With Rapid Per VLAN Spanning Tree Plus (Rapid PVST+), each STP instance uses the 802.Iw algorithm to reconverge the network following link failure. Note Within a VDC, you can run either Rapid PVST+ or Multiple Spanning Tree (MST) but not both simultaneously. ‘©2012 Gisco Systems, (isco Nexus Switch Feature Configuration 2107RSTP Interoperation with 802.1D Although Cisco Nexus switches cannot operate in classical 802.1D mode, RSTP can interact with legacy STP bridges 1, Switeh Agonds RSTPBPDUS Switch B that Switch B drops '202.10 Enabled 2. Switch B doesnot getany valid BPDUs <0 itsends out itsown 802, 10 PDUs. 3, Switch Aspes an 802.10 ‘teh on the networkand ravertsto 602.1D made. Omran @xx108r0u Although Cisco Nexus switches cannot run in classical 802.1D mode, RSTP can interoperate with legacy STP protocols. However, the fast convergence benefits of RSTP are lost when interacting with legacy bridges. Each port maintains a variable defining the protocol or mode in order to run on a corresponding segment. A migration delay timer of three seconds is also started when the port comes up. When this timer is running, the current mode (STP or RSTP) that is associated with the port is, locked. After the migration delay expires, the port adopts the mode of the next bridge protocol data unit (BPDU) that it receives. Ifthe port changes its operating mode as a result of receiving a BPDU, the migration delay is restarted to limit the frequeney of possible mode changes. Legacy STP bridges ignore RSTP BPDUs and drop them. The legacy STP bridge assumes that there are no other bridges on the segment and starts sending out inferior $02.1 D-format BPDUs. Upon receiving these legacy BPDUs, RSTP bridges wait for twice the hello time before changing to 802.1D mode on that port only. As a result, the legacy 802.1D bridge begins to receive BPDUs that it can understand. Note If the legacy STP bridge is removed from the segment, the RSTP bridge continues to run legacy STP on that port. This situation occurs because the RSTP bridge has no way of knowing that the legacy bridge has been removed from the segment. Manual intervention is. required to restore the abilty of a port to detect the current protocol. When a port is in legacy 802.1D mode, it is also able to process topology change notification (TCN) BPDUs and BPDUs with the topology change (TC) or topology change acknowledgment (TCA) bit set Implementing Cisco Data Genter Unified Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncConfiguring Rapid PVST+ ARapid PVST* instance is automatically created when a VLAN is configured. sites (contigevlan) ¥ name Sales Prioity 32778 This bridge is the root Hello Tina 2 anc Max Age 20 aac Forward Delay 15 sec Priority 32778 (priority 22762 ays-id-ext 10) Hello Tine 2 sec Max Age 20 sec Forward Delay 15 sec interface Role Sts Cort rio. Nr Type weit ‘The figure shows the configuration steps to create and name a VLAN. The show command ‘output displays that the spanning-tree version is RSTP. The output also displays the root ID, bridge ID, and port state for each VLAN. RSTP is the default spanning-tree version for Cisco NX-OS Software. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2109Mutiple Spanning Tree Protocol + MSTallows VLANS to be load-balanced across different spanning-tree topologies. + MST calculates the spanning tree topology for a group of VLANS rather than per VLAN, making it more scalable than Rapid PVST+. WANA trerig th ANB true fh AWA back pah ‘The problem with running a single instance of STP is that any blocked link is unable to actively participate in the forwarding of data, The blocked link then becomes a wasted resource that is used for redundancy purposes only. Rapid PVST+ solves this issue by running a separate spanning-tree instance for each VLAN. MST is defined in 802.1 and is designed to support multiple instances of spanning tree over VLAN trunks. MST permits the mapping of multiple VLANs into a single spanning-tree instance, with each instance supporting a spanning-tree topology independent of other spanning-tree instances. This architecture provides multiple forwarding paths for data traffic and enables load balancing while simultaneously reducing the number of spanning-tree instances required to support many VLANs. MST further improves the fault tolerance of the network, as a failure in one instance or forwarding path does not affect other instances or forwarding paths. MST uses the RSTP mechanisms for each instance to provide rapid spanning-tree convergence through explicit handshaking, thereby eliminating the 802.1D forwarding delay while quickly transitioning root bridge ports and designated ports to the forwarding state. MST improves spanning-‘ree operation and maintains backward compatibility with the following = The original $02.1D STP = Existing Cisco proprietary Multi-Instance STP (MISTP) ing Cisco PVST+ = Exis = Rapid PVST+ 2110 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncMST Regions and Instances + MSTregion is a collection of interconnected switches with the same MST configuration. ‘The following should be the same for all switches in an MST region: Region name Revision number VLAMto-instance mappings For switches to participate in MST instances, their MST configuration information must be consistent. A collection of interconnected switches with the same MST configuration constitutes an MST region. The coi iguration includes the name of the region, the revision number, and the VLAN-to- MST instance assignment mapping. A region can have one or multiple members with the same MST configuration. Each member must be capable of processing 802.1w BPDUs. There is no limit to the number of MST regions in a network, Note ‘Although multiple MST regions can interact with each other, it is not recommended to partition the network into many regions. Each device can support up to 65 MST instances (MSTIs)—including Instance 0—in a single MST region. Instances are identified by any number in the range from 1 to 4094. The system reserves Instance 0 for a special instance, which is the Internal Spanning Tree (IST). By default, all VLANs are assigned to this instance. You can assign a VLAN to only one MST instance at a time. ‘The MST region appears as a single bridge to adjacent MST regions and to other Rapid PVST+ regions and 802.1D STPs, ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2111MST Configuration 4. Configure MST region parameters. 2. Exit from the configuration context to make changes effective 3. Change the spanning tree mode to MST. instance 1 vlan 100-199 instance 2 vlan 200-299 wet tconeig-not¥ anit @ WK contig) apanning-tene mode mat © This figure describes the proper configuration steps to set the MST configuration parameters and then to enable MST as the active spanning-tree mode. Note ‘Changes that are made in spanning-tree MST configuration mode are not applied until the ‘exit command is issued. To exit MST configuration made without applying the changes, use the abort command. 2-112 _ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncSTP Extensions This topic identifies how to use and configure the STP extensions on the Cisco Nexus switches, Spanning Tree Protocol Extensions Cisco NX-OS Software STP extensions: + STP edge port (PortFast) BPDU filtering BPDU guard Root guard Loop guard Bridge Assurance Cisco has added extensions to STP that enhance loop prevention, protect against user configuration errors, and provide better control over the protocol parameters. The available extensions are spanning-tree edge ports (previously known as PortFast), BPDU filtering, BPDU guard, loop guard, root guard, and Bridge Assurance, All of these extensions can be used with both Rapid PVST+ and MST. Many of these features can be applied either globally or on specified interf ‘©2012 Cisco Systems, Inc {isco Nexus Switch Feature Configuration 2113Spanning Tree Edge Port STP edge port + Reduces the time to transition a port connected to a host to the forwarding state after linkup + Also known as PortFast men ahostconnects, the swith portmaves |] An STP edge port mov esstmighttothe Uyough al STP states before fowarcing. || forwarding state elminatng a 30-secand delay. Configuring a Layer 2 access port as a spanning-tree edge port causes the port to bypass the listening and learning states and enter the forwarding state immediately. This feature was formerly known as PortFast, but the name was changed to spanning-tree edge port in order to conform to the RSTP standard naming convention for this feature. Spanning-tree edge ports are typically deployed on Layer 2 access ports that are connected to a single workstation or server. This design allows those devices to connect to the network immediately without waiting for STP convergence to take place. Interfaces that are connected to a single workstation or server are not expected to receive BPDUs, and it should be safe to transition these ports to the forwarding state. When configured as a spanning-tree edge port, a port is still running STP. A spanning-tree edge port can immediately transition to the blocking state if necessary—for example, upon receipt of a BPDU. Note ‘Spanning-tree edge port configuration is used to minimize the time that access ports must wait for STP convergence to occur and, therefore, should only be used on access ports. If you enable the spanning-tree edge port feature on a port that is connected to a switch, you might inadvertently create a temporary bridging loop. 2114 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncConfiguring STP Extensions + Cisco NX-OS syntax is similar to Cisco IOS syntax in most cases + Examples: 4. BEDU guard 2. ‘spanning-tree port type edge’ (nstead of spanning-ree portfast’) 3. Root guard TET (Saiah apanning-trwe port type edge Beduguard default @ 1K (conf ight Antertace athernett/2 NIK (ont igri 0)" epenning-tree port type edge ‘mening: Edge port type (portfast) should only be enabled on ports connactad a single host. Connecting hubs, concentrators, switches, bridyes, ete... 1 this interface when edge port type (portfast) is enabled, can cause temporary eidging Loops Use with CAUTION Ihige Port Type (Portfast) has beon configured on Ethernet i/1 but will only Thave effect whan the interface is in a non trunking node 10K-1 (cont ig) 1 Sntestace ethernett/2 IK-L toca ig-t)¥ apanniog-teee guard root © ‘The syntax that is used to configure the spanning-tree extensions in Cisco NX-OS Software that runs on Cisco Nexus switches is very similar to the syntax that is used in the Cisco IOS Software that runs on Cisco Catalyst switches, ‘The most important exception is the spanning-tree edge port feature, which was formerly known as PortFast. This change in naming is reflected in the command syntax. The Cisco NX- OS syntax to enable this feature is spanning-tree port type edge, while the Cisco IOS syntax, to enable this feature is spanning-tree portfast. ‘The figure shows an example configuration that enables the BPDU guard feature for all spanning-tree edge ports, configures interface Ethernet 1/1 as a spanning-tree edge port, and enables the root guard feature on interface Ethernet 1/2. ‘©2012 Cisco Systems, Inc, {isco Nexus Switch Feature Configuration 2115Why Bridge Assurance? * Normally, STP BPDUs flow from the root to the leaves of the tree. + When a non-designated, non-root port stops receiving BPDUs, it will become designated and transition to the forwarding state. + Aswitch might stop sending BPDUs due to a control plane failure condition while the data plane is still active. + This can cause a bridging loop. ‘The figure shows a normal STP topology and normal STP behavior, including a root bridge. A ‘malfunctioning switch that stopped sending any BPDUs (shown at the upper right of the graphic) could cause the neighboring switches to move a blocking port to non-blocking. In this situation, the malfunctioning switch can create a bridging loop in the network, 2116 Implementing Cisco Data Genter Untied Fabric (OGUFI) v5.0 (© 2012 Cisco Systems, IncBridge Assurance * Bridge Assurance prevents bridging loops caused by STP failures. + Bridge Assurance alters the behavior of STP. + BPDUs are sent on all ports that have Bridge Assurance enabled. + BPDUs are used as a hello protocol to detect protocol failure. Bridge Assurance is used to protect against certain problems that can cause bridging loops in the network. Specifically, Bridge Assurance can be used to protect against unidirectional link failure or other software failure. Bridge Assurance can also be used to protect against situations where a device continues to forward data traffic when it is no longer running STP. Note Bridge Assurance is supported only by Rapid PVST+ and MST. Bridge Assurance is enabled by default in Cisco NX-OS and can only be disabled globally. Bridge Assurance can only be enabled on point-to-point STP links: Both ends of the link must be enabled for Bridge Assurance. If they are not, the adjacent port is blocked. When Bridge Assurance is enabled, BPDUs are sent on all operational network ports, including alternative and backup ports, for each hello time period. If the port does not receive a BPDU for ed period, then the port moves into the blocking state and cannot be used for the root port calculation, After it receives a BPDU, it resumes normal spanning-tree operation ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2117Bridge Assurance Operation + When a port that has Bridge Assurance enabled stops receiving BPDUs, it will mark the port as inconsistent instead of moving to forwarding No bridging foop occurs ~ The function of Bridge Assurance’s simiar to loop guard + Only use one ofthe twomechanisms on the same port Say Metunctioning Veer 2 BRIGOE ASSUUNGE B100t- Briaje Anmoniee bloating pore Etamet2/@ VIANITOD With Bridge Assurance enabled, even a malfunctioning switch in the network does not create a bridging loop. When the potential loop is identified, Bridge Assurance puts the port into a Bridge Assurance inconsistent state. 2118 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncConfiguring Bridge Assurance + The bridge assurance feature is enabled globally. = itis enabled by defaut itch (eontlgy epamning tree bridge assurance + However, only ports of type network are enabled for Bridge Assurance. + spanning-tree port type network command enables Bridge Assurance ona specific port cilteh (ontlay) ntarface ethernet 1/3 mritch (con€igyif} spanning-tewe port type network criteh contiq-it} 4 show spanning-tree interface ethernet 1/2 ven Role Sts Cost Prior Type Bridge Assurance needs to be enabled globally in order to enable it at the interface level. However, it is already enabled globally by default. To disable Bridge Assurance, you can use the no spanning-tree bridge assurance command. When Bridge Assurance is enabled globally, all interfaces included in spanning-tree will have BA enabled. However, the default port type of an interface is normal. To enable Bridge Assurance on an interface, use the spanning-tree port type network command. Note The spanning-tree port type network command should always be configured on both sides of a link to prevent a port from going into blocking because itis not receiving BPDUs from the neighbor. ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2119Summary This topic summarizes the key points that were discussed in this lesson. Summary + Cisco Nexus interfaces support many modes, such as Layer 2 access. mode, Layer 2 trunk mode, Layer 3 mode, Cisco FEXmode, Cisco Adapter FEXmode, as well as additional features, such as port profiles. The Cisco Nexus 5000 and 7000 Series switches and Cisco Nexus 5500 Platform switches support extensive but slightly different sets of Layer 2 features. Rapid PVST* is the default spanning-tee mode on Cisco Nexus switches, and MST is used to scale spanning-tree domains. The Cisco NXOS Software supports a wide range of spanning-tree extensions, such as STP edge port, BPDU filtering, BPDU guard, root guard, loop guard, and Bridge Assurance. References For additional information, refer to these resources: ‘© To leam more about configuring Cisco Nexus 2000 FEX, refer to Cisco Nexus 2000 Series Fabric Extender Software Configuration Guide at this URL: http://www cisco.com/en/US/does/switches/datacenter/nexus2000/sw/configuration/guide/t el_6_0/b_Configuring_the Cisco_Nexus 2000_Series Fabric_Extender_rel_6_0.html ‘© To leam more about configuring Cisco Adapter FEX, refer to Cisco Nexus 5000 Series NX- OS Adapter-FEX Software Configuration Guide, Release 5.1(3)NI(1) at this URL: http://www cisco.com/en/US/does/switches/datacenter/nexus5000/sw/adapter- fex/513_nl_I/b_Configuring_Cisco_Nexus_5000_Series_Adapter- FEX_rel_ 5 13 NILhtml 2-120 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncLesson 4| Configuring PortChannels Overview Cisco PortChannel is one of the core technologies that are used in Ethernet-t Cisco PortChannel is used to bundle multiple physical links into a single logical link, which improves resiliency and optimizes bandwidth utilization on the links A limitation of regular port channel is that it only allows the aggregation of links between two devices, The virtual port channel (vPC) technology that is used by the Cisco Nexus 5000 and 7000 Series switches enables Multichassis EtherChannels (MECs) to be formed between a network device and two separate physical chassis. vPC technology allows logical loop-free Layer 2 topologies to be created, which prevents Spanning Tree Protocol (STP) from blocking any of the ports in the network topology. This type of design combines high availability with increased bandwidth between the access and aggregation lay. ers, Cisco Nexus 2000 Fabric Extender (FEX) technology can be deployed together with Cisco Nexus 5000 or Cisco Nexus 7000 Series switches in order to create a data center network that combines the advantages of a top-of-rack (ToR) design with the advantages of an end-of-row (EoR) design. Enhanced vPC combines two vPC topologies: hosts dual-homed to two FEXs and FEXs dual- homed to two Nexus 5500 Switches. Objectives Upon completing this lesson, you will be able to evaluate how port channels and vPCs should be used to improve the solution and then configure the features. You will be able to meet these objectives: = Identify where port channels and vPCs could be used to improve reliability m= Identify how to configure port channels on the Cisco Nexus switches = Identify the architecture and components of vPCs = Explain how to configure vPCs on the Cisco Nexus switches = Explain how to configure the Ci 5000 or 7000 Series switeh -0 Nexus 2000 Series FEX connected to a Cisco Nexus = Explain how to configure the Enhanced vPCs on a Cisco Nexus 5000 Series switchUsing Port Channels and vPCs This topic identifies where port channels and vPC’ could be used to improve reliability. Ethernet Port Channel * Multiple physical links combined into a single logical link Link redundaney Load balancing based on header hashing Links in a port channel need to be terminated on a single peer device Based on IEEE 802.3AD Often used in aggregation and core layers Static or dynamic configuration Dynamic negotiation by Link Aggregation Control Protocol (LACP) Physical View Logical View PortChannel is one of the core technologies that are used in Ethernet-based networks. To add resiliency against link failures and to increase the available bandwidth between two devi ‘multiple physical links can be provisioned between the devices. However, without PortChannel, control plane protocols, such as STP, or routing protocols will treat the links as individual links. In the case of STP, the result is blocked ports. Although the additional links add resiliency, the available bandwidth between the two devices is not increased. PortChannel technology combines the physical links into a single logical link, which is port channel. Control plane protocols, stich as STP and routing protocols, treat the port channel asa single link. Spanning tree will not block the links that are part of the port channel, and routing protocols only form a si adjacency across the port channel. ‘Traffic that is switched or routed to a port channel interface is balanced across the individual physical links through a hashing mechanism. The hashing mechanism uses a selection of the fields in the packet headers as input. This process ensures that packets with the same header will be forwarded on the same physical link to prevent packet reordering. A port channel can either be defined statically or negotiated dynamically by using Link Aggregation Control Protocol (LACP). Cisco Nexus Operating System (NX-OS) Software performs a compatibility check when adding ports to a port channel so as to ensure that the port can participate in the port channel aggregation. Therefore, it is important that all physical ports that participate in a port channel are configured identically. LACP, which is described in the 802.1AX standard, can be used to dynamically negotiate the aggregation of multiple links into port channel and to detect failure conditions. A major restriction of PortChannel technology is that it is inherently limited to the aggregation of a number of links that run between the same two devices. Implementing Cisco Data Genter Unified Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncPort Channel Layer 2 and Layer 3 Interfaces + Layer 2 port channels in access or trunk mode + Layer3 port channel interfaces May have subintorfaces May have a static MAC address configured + Otherwise, the MAC of the firstchannel member to come up aN ee WANT ocho (area) (He) ome] ke envi Ena ewe En Ha You can classify port channel interfaces as Layer 2 interfaces or, in the case of the Cisco Nexus 5500 Platform and 7000 Series switches, Layer 3 interfaces. In addition, you can configure Layer 2 port channels in either access or trunk mode, Layer 3 port channel interfaces have routed ports as channel members and may have subinterfé ces, You can configure a Layer 3 port channel with a static MAC address. If you do not configure this value, the Layer 3 port channel then uses the router MAC of the first channel member to come up, On the Cisco Nexus 7000 Seri device context (VDC). witches, all ports in a port channel must be in the same virtual ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2123Port Channel Load Balancing Destination MAC address = eS Source MAC adress 500016500 Source and destination MAC adcress News 7000 1Bactivetnks Series modules NNewus 7000 active and 8 stanaty Source and destination IP acarers Mseres teks (Cisco NXOS Destination TCPFUDP port number eee aL Teen Source TCP/UDP port number Source and destinatn TCPLIOP portnumter Destination Padeross Source IP adress ayn stot uber etsy inks The Cisco Nexus switches support the bundling of up to 16 ports into a port channel. The ‘maximum number of ports in a channel depends on the exact switch hardware and software combination, On the MI-Series modules on the Cisco Nexus 7000 Series switches, the ‘maximum is eight active links per port channel. Beginning with Cisco NX-OS Release 5.1, you can bundle up to 16 active ports simultaneously into a port channel on the FI series modules on the Cisco Nexus 7000 Series switch. On the Cisco Nexus 5000 Series switches, you can bundle up to 16 active links into a port channel. ‘The Cisco Nexus switch load-balances all traffic that is switched or routed to a port channel interface across.all operational individual physical links by hashing the various header fields in a frame into a numerical value that selects one of the links in the channel. This process ensures that packets with the same header will be forwarded on the same physical link in order to prevent packet reordering. The load-balancing mechanism is performed in the hardware and enabled by default. The load-balancing method can either be applied to all port channels on a specified module (Cisco Nexus 7000 Series switch) or to the entire switch (Cisco Nexus 5000 and 7000 Series switches and Cisco Nexus 5500 Platform switch). If'a per-module load- balancing method is configured, it takes precedence over the switchwide setting. ‘You can configure the switch to use one of the following load-balancing methods: = Destination MAC address Source MAC address © Source and destination MAC addresses Destination IP address Source IP address Source and destination IP addr Source TCP or UDP port number Destination TCP or UDP port number ses Source and destination TCP or UDP port numbers 224 Implementing Cisco Data Genter Unified Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncVirtual Port Channel (vPC) + Port channel extension Port channels terminated on different physical devices Resiloncy against device failures Mutiple physical switches appear as single logical switch othe peer device + Loop-free logical topologies with full physical redundancy + Use cases: ‘A. Duabuplink Layer 2 access: B. Server dual-horring ©. Active-active Fabric Extenders (FEX) With the increased use of virtualization technologies in data centers, and even across data center locations, organizations are shifting from a highly scalable Layer 3 network model to a highly scalable Layer 2 model. This shift is causing changes in the technologies that are used to manage large Layer 2 network environments. These changes include migration away from STP as a primary loop-management technology and toward new technologies, such as vPCs. ‘The biggest limitation of classic PortChannel is that the port channel operates only between two devices. In large networks, the support of multiple devices together is often a design requirement to provide some form of hardware failure alternate path. This alternate path is often connected in a way that would cause a loop, thereby limiting the benefits that are gained with port channel technology to a single path, To address this limitation, the Cisco NX-OS Software platform provides a technology called vPC. Although a pair of switches acting as a VPC peer endpoint looks like a single logical entity to the port channel-attached devices, the two devices that act as the logical port channel endpoint are still two separate devices. The vPC solution combines the benefits of hardware redundancy with the benefits of port channel loop management. The three main use eases of the vPC technology are as follows: switch such as a Cisco Nexus 5000 uch as Cisco Nexus 7000 = Dual-upli Series switch Series switches. k Layer 2 access: In this dual-homed to a pa rio, an ace r of distribution switches, = Server dual-homing: In this case, a server is connected via two interfaces to two separate access switches. m= Active-active Fabrie Extenders: In this topology, a Cisco Nexus 2000 Fabrie Extender is dual-homed to a pair of Nexus switches. ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2125vPC Enhancement vs. STP * Without PC Premary Secondary Root Root «STP blocks redundant uplink - VLAN-based load balancing Loop resolution relies on STP Protocol failure can cause complete netw orkmeltdow n + With vec. No blocked uplinks Lower oversubscriotion Hash-based BtherChannel load balancing Loop-free topology STP is used in case of keepalive ‘and VPC peer link simutaneous falure In early Layer 2 Ethernet network environments, it was necessary to develop protocol and. control mechanisms that limited the disastrous effects of a topology loop in the network. STP ‘was the primary solution to this problem, providing a loop detection and loop management for Layer 2 Ethernet networks. This protocol has gone through a number of enhancements and extensions. While STP scales to very large network environments, it still has one suboptimal principle: To break loops in a network, only one active path is allowed from one device to another. This principle is true regardless of how many actual connections might exist in the network, ‘The other main benefit of migration to an entirely port channel-based loop-management ‘mechanism is that link recovery is potentially much faster. STP can recover from a link failure in approximately six seconds, while an entirely port channel-based solution has the potential for failure recovery in less than one second, 2126 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncvPCs at Muttiple Layers + VPC supported on Cisco Nexus 5000/5500/7000 switches * PC can be deployed in multiple layers of the data center simultaneously ‘Server to access ‘Access to agaregation + Separate vPC configured at each level + Known as dual-sided vPC YPC is supported on Cisco Nexus 5000 and 7000 Series switches as well as Cisco Nexus 5500 Platform switches. The benefits that are provided by the vPC technology apply to any Layer 2 switched domain. Therefore, vPC is commonly deployed in both the aggregation and access layers of the data center. YPC can be used to create a loop-free logical topology between the access and aggregation layer switches, which increases the bisectional bandwidth and improves network stability and convergence. vPC can also be used between servers and the access layer switches in order to enable server dual-homing with dual-active connections, When the switches in the access and aggregation layers both support vPC, a unique vPC can be created at each layer. To implement this environment, you need to configure two separate vPC domains at the access and distribution layers. The layer would typically consist of Cisco Nexus 5000 Series switches or Cisco Nexus 5500 Platform switches and the distribution layer of the Cisco Nexus 5500 Platform switch or the Cisco Nexus 7000 Series switch, This scenario is commonly referred to as “dual-sided vPC, ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2127Configuring Port Channels This topic identifies how to configure port channels on the Cisco Nexus switches. Channel Modes tne Passive (LACP) Responds to LACP packets that it receives + Does not initiate LACP negotiation Active (LACP) —« Intiates negotiations with other ports by sending LACP packets (On (static) + Does not send any LAGP packets + Does not join any LACP channel groups + Becomes an individual link w th that interface Porchannel results: Ee fen Passive 98 0K rom OK OK On 98 8 Individual interfaces in port channels are configured with channel modes. When you run static port channels, with no protocol, the channel mode is always set to on. After you enable LACP. ¢globally on the device, you enable LACP for each channel by setting the channel mode for each interface to active or passive. You can then configure either channel mode for individual links in the LACP channel group. ‘The following table describes the channel modes. Channel Mode Description Passive This LACP mode places a por into a passive negotiating state, in which the port responds to LACP packets that it receives but does not intiate LACP negotiation, Active This LACP mode places a por into an active negotiating state, in which the Por initiates negotiations with other ports by sending LACP packets, On Al static port channels—that is, those that are not running LACP—remain in this mode. If you attempt to change the channel mode to active or passive before enabling LACP, the device returns an error message. You enable LACP on each channel by configuring the interface in that channel for the channel mode as either active or passive. When LAGP attempts to negotiate with an interface in the “on” state, it does not receive any LACP packets and becomes an individual link with that interface. It does not join the LACP channel group. Both the passive and active modes allow LACP to negotiate between ports in order to determine if they can form a port channel. This is based on criteria such as the port speed and the trunking state. The passive mode is useful when you do not know whether the remote system, or partner, supports LACP. ‘©2012 Cisco Systems, Inc, {isco Nexus Switch Feature Configuration 2131Ports can form an LACP port channel when they are in different LACP modes as long as the ‘modes are compatible, as in these examples: = A port that is in active mode can form a port channel suc in active mode. fully with another port that is = A port that is in active mode can form a port channel with another port that is in pas mode. ive © A port that is in passive mode cannot form a port channel with another port that is also in passive mode because neither port will initiate negotiation. = A port that is in “o * mode is not running LACP. 2-132 _ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncConfiguring Layer 2 Port Channels 1. Static configuration Layer2 interface in trunking mode LACP-based configuration Layer2 interface in access mode Tultch (contig) tnterface ethernet 1/25, ethernet 1/27 mitch (config-tfmeangel ¥ amitenpect. switch (contig-it-rangel ¥ éhanpel-group 2 o mitch (contig) | interface poct-channel 1 Suitch (oontig-it}¥ awitokport mode tank mitch (contigit}t Cather layer 2 contiguration.> fitch (contig) # feature 1acp switch (contig) | interface ethernet 1/29, ethernet 1/31 peitch contigeit-range) P awitehport, suite (contig-it-range ¥ enennei-group aay switch (config-if)}4 switchport access vlan 10 e TUG Sel ts) Cathe igor cecgereton > Configuration of port channels commonly consists of two elements—configuring the physical ports and configuring the port channel interface. The physical ports need to be assigned to a channel group, which then bundles the ports together. A channel group always has an associated port channel interface, which has the same number as the channel group number. You can create the port channel interface before you assign the physical interfaces to a channel ‘group. If you do not create the port channel interface beforehand, it is automatically created ‘when you assign the first physical interface to the channel group. After the interfaces have been successfully bundled into a channel group with an associated port channel number, you can then configure Layer 2 or Layer 3 settings on the port channel interface. Settings that are configured on the port channel will be inherited by the physical interfaces that are members of the associated channel group. If you create a port channel interface before assigning any interfaces to a channel group, itis, important that the configuration of the port channel interface be compatible with the configuration of the physical member interfaces that you assign to the channel group. ‘The figure shows two examples of Layer 2 port channel configuration: = The first example shows how to create a static port channel. In this case, the ports are configured for the “on” mode. The port channel is ereated statically without any negotiation, = The second example shows how to configure a port channel that uses LACP to aggregate the links, The ports are configured for active mode and therefore initiate negotiations with other ports by sending LACP packets. ‘©2012 Cisco Systems, Inc, (isco Nexus Switch Feature Configuration 2133Configuring Layer 3 Port Channels 1. Static configuration 2. LACP-based configuration “itch (config) ¥ interface ethernet 2/1, ethernet 2/3 ech (ontign # interface port-chamer 3 @ site (con fig-if}t ap aedswes 10-21-1/24 Suitch (ontig-if}4 other tayer 3 configura tion.> ‘suitoh (config) ¥ feature 1aop Sitch footy # interface ethernet 2/5, ay) | enables ‘suite foonfig-ie-range ¥ channel-growp. LAGE Siitoh foonfigeit}t Sp addeese 10.2.2.2/24 mitch (emfig-ifh) Cother Layer 3 configuration.» ‘The examples in the figure show how to configure Layer 3 port channels. The IP address for a port channel should always be configured on the port channel interface. ‘When you add an interface to a channel group, the software checks certain interface attributes to ensure that the interface is compatible with the channel group. For example, you cannot add a Layer 3 interface to a Layer 2 channel group. The Cisco NX-OS Software also checks a number of operational attributes for an interface before allowing that interface to participate in the port channel aggregation. Use the show port-channel compatibility-parameters command, which is described later, to see the complete list of compatibility checks that Cisco NX-O8 Software uses. 2134 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncVerifying Port Channels Flags: 9 "Dom "Gp in port-charnel (members) 3 individual ~ Hot-standby (ACP only) 2 Suepended! T ~ module rendved 5 Switched R - Routed 1 _ op tport-chamnes) poly) Bth WOME ——Btha/25(e)ena/27(@) Po2iso) —Rth_=—= ACP Bthi/29(P) Ethd/31(P) Po3(RO) th NONE ehZ/L(@)—BEIR/3) Posimo) eth TACe = Etnay/ste)—Etn2/7(2) Routoa (Ryana taheAGP swiched(S) PCs ‘method ‘The show port-channel summary command can be used to verify the status of the configured port channels on the switch. ‘©2012 Cisco Systems, Inc {isco Nexus Switch Feature Configuration 2135Configuring and Verifying Port Channel Load Balancing 1. Per-switch port channel load-balancing hash 2. Per-module port channel load-balancing hash ~ Cisco Nexus 7000 Series switch platforms only 3. Verify the port channel load-balancing configuration use: Sitch (ntl) port-Ghanmal load-balance souros-dast port oe ovitch (config) port-chamet load-belance source-dast-sp-port-vlen module jvitcut show port-charnel Load-batance —@ ‘System! source dest port Fort Channel toad-Balancing Addresses Used Per-Protocel: [Defaut agoathim Non-IP; source dest “mac Ps source dest TE! source -dest-port eouree-deat-ip source-dast-mc Nom: source-testmac To configure the port channel load-balancing options, use the port-channel load-balance ethernet command, The exact options for this command ean vary by platform and Cisco NX- OS Software release. The first example shows how to set the per-switch load-balancing option to include UDP and ‘TCP port numbers. This command could be configured on a Cisco Nexus 5000 or 7000 Series, switch of on a Cisco Nexus 5500 Platform switch, jancing hash on a specific module of a Cisco hows how to set the load: ‘The second example s Nexus 7000 Series switch. ‘The show port-channel load-balance command, which is shown in the third example, can be used to verify the load-balancing hash that is currently used. Note Per-module PC load balancing is platform-specific. Please check the release notes or a configuration guide. 2-136 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncvPC Architecture ‘This topic identifies the architecture and components of vPCs. ‘Combined port chanel between the vPC peers ‘VPC capatic downsteamdevice Apat of vPC-endblod swihes YPC peeriink Caries contol taficbetween vPC peers {GiscoFabric Protocol for stale eyndhronizton andcoriguatn Services \liaton Between vEC peers YPC peer Routed ink carving hearbeatpackesforactve- keepalvelink activedetecton. PC member One of the ports atforns a vPC. port YPC domain Pair of vPCpeersandassocieted VPC components COphandevice Device canectedta vPC peer an nonyPC ink Ophanpot Porton avPC peer thatcanness to an orphan fovice, Aso used Tora vPC mombor port on PC peer thathas ist comectviy tothe aherpoce A pair of Cisco Nexus switches that uses vPC present themselves to other network device single logical Layer 2 switch, However, the two switches remain two separately managed switches with independent management and control planes. The vPC architecture includes modifications to the data plane of the switches in order to ensure optimal packet forwarding, VPC architecture also includes control plane components to exchange state information between the switches and allow the two switches to appear as a single logical Layer 2 switch to the downstream devices. ‘The vPC archit ture consists of the following component = VPC peers: The core of the vPC architecture is a pair of Cisco Nexus switches acts as a single lo chassis using MEC. witches. This pair of I switch, which allows other devices to connect to the two m= PC peer link: The vPC peer link is the most important connectivity element in the vPC system. This link is used to create the illusion of a single control plane by forwarding bridge protocol data units (BPDUs) and LACP packets to the primary vPC switch from the secondary vPC switch. The peer link is also used to synchronize MAC address tables between the vPC peers and to synchronize Internet Group Management Protocol (IGMP) 'sfor IGMP snooping. The peer link provides the necessary transport for multicast and for the traffic of orphaned ports. In the case of a vPC device that is also a Layer 3 switch, the peer link also carries Hot Standby Router Protocol (HSRP) packets, co Fabric Services protocol is reliable messaging protocol that is designed to support rapid stateful configuration message passing and synchronization. The vPC peers use the Cisco Fabric Services protocol to synchronize data plane information and implement necessary configuration checks. vPC peers must synchronize the Layer 2 Forwarding table between the vPC peers. This way, if one vPC peer learns a new MAC address, that MAC address is also programmed on the L2F table of ‘©2012 Cisco Systems, Inc, (isco Nexus Switch Feature Configuration 2137the other peer device. The Cisco Fabric Services protocol travels on the peer link and does not require any configuration by the user. To help ensure that the peer link communi for Cisco Fabric Services over Ethernet is always available, spanning tree has been modified to keep the peer-link ports always forwarding. The Cisco Fabric Services over Ethernet protocol is also used to perform compatibility checks in order to validate the compatibility of vPC member ports to form the channel, to synchronize the IGMP snooping status, to monitor the status of the vPC member ports, and to synchronize the Address Resolution Protocol (ARP) table. tion = YPC peer keepalive link: The peer keepalive link is logical link that often runs over an out-of-band (OOB) network. The peer keepalive link provides a Layer 3 communications path that is used as a secondary test in onder to determine whether the remote peer operating properly. No data or synchronization traffic is sent over the vPC peer keepalive link—only IP packets that indicate that the originating switch is operating and running, VPC. The peer keepalive status is used to determine the status of the vPC peer when the YPC peer link goes down. In this scenario, it helps the vPC switch to determine whether the peer link itself has failed or whether the vPC peer has failed entirely. = YPC: A vPC is an MEC, a Layer 2 port channel that spans the two vPC peer switches. The downstream device that is connected on the vPC sees the vPC peer switches as a single logical switch. The downstream device does not need to support vPC itself. The downstream device then connects to the vPC peer switches using a regular port channel, which can either be statically configured or negotiated through LACP. © yPC domain: The vPC domain includes both vPC peer devices, vPC peer keepalive link, vPC peer link, and all port channels in the vPC domain that are connected to the downstream devices. A numerical vPC domain ID identifies the vPC. You can have only ‘one vPC domain ID on each devi = YPC member port: This is a port on one of the vPC peers that is a member of one of the PCs configured on the vPC pee = Orphan device: The term “orphan device” refers to any device that is connected to a vPC domain using regular links instead of connecting through a vPC. © Orphan port: The term “orphan port” refers to a switch port that is connected to an orphan device. The term is also used for vPC ports whose members are all connected to a single VPC peer. This situation can occur if a device that is connected to a vPC loses all its connections to one of the vPC peers. 2-138 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncvPC Data Plane for Local Traffic + VPC peer link carries only: YPC control traffic Flooded traffic (broadcast, muiticast, unknown unicast) Traffic for orphan ports + MAC address learning replaced with Cisco Fabric Senices-based MAC address leaming ‘Only for vPCs. Non-vPC ports use regular MAC address learning + Frames arriving at peer switch on peer link cannot exiton vPC. member port YPCs are specifically designed to limit the use of the peer link to switch management traffic as well as the occasional traffic flow from a failed network port. To begin, the peer link carries vPC control traffic, such as Cisco Fabric Services over Ethernet, BPDUs, and LACP messages. In addition, the peer link carries traffic that needs to be flooded, such as broadcast, multicast, and unknown unicast traffic. The peer link also carries traffic for orphan ports. ‘The term “orphan port” is used for two types of ports. One type of orphan port is any Layer 2 port on a vPC peer switch that does not participate in vPC. These ports use normal switch forwarding rules, and traffic from these ports can use the VPC peer link as a transit link to reach orphan devices that are connected to the other vPC peer switch. The other type of omphan port is port that isa member of a vPC but for which the peer switch has lost all the associated vPC member ports. When a vPC peer switch loses all member ports for a specific VPC, it forwards traffic that is destined for that vPC to the vPC peer link. In this special case, the vPC peer switch will be allowed to forward the traffic that is received on the peer link to one of the romaining active vPC member ports. To implement the specific vPC forwarding behavior, it is necessary to synchronize the Layer 2 Forwarding tables between the vPC peer switches through Cisco Fabric Services instead of ‘depending on the regular MAC address learning. Cisco Fabrie Services-based MAC address learning applies to vPC ports only and is not used for non-vPC ports. ‘One of the most important forwarding rules for vPC is that a frame that enters the vPC peer switch from the peer link cannot exit the switch from a VPC member port. This principle prevents frames that are received on a vPC from being flooded back onto the same vPC by the other peer switch. The exception to this rule is traffic that is destined for an orphaned vPC member port. ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2139vPC Data Plane for External Traffic 1. Inbound traffic destined for a WC iss forwarded on a local \PC member port whenever possible 2. Outbound traffic actively Layer3.core forwarded by all FHRP routers whenever possible 3. Benefits - Traffic avoids peer ink if possible, which creates a scalable solution Peer link capacity does not need to scale inearly withthe nurber of vPCs ‘The use of the vPC bandwidth is also minimal when traffic is exchanged with external networks. Whenever a vPC peer switch needs to forward inbound traffic for a VPC, it forwards it to a local vPC port if possible. Only if it has no active vPC member ports for the vPC does it then forward it across the vPC peer link to the other vPC peer switch Aggregation switches using vPCs commonly use a First Hop Redundancy Protocol (FHRP), such as HSRP, Gateway Load Balancing Protocol (GLBP), or Virtual Router Redundancy Protocol (VRRP) for default gateway redundancy. The normal forwarding behavior of these protocols has been enhanced with the peer gateway feature in order to allow them to interoperate with vPCs. Normally, only active FHRP routers forward traffic for the virtual default gateway MAC address. For vPCs, the forwarding rules have been enhanced to allow a nonactive FHRP router to forward frames that are destined for the FHRP virtual MAC address. However, the primary FHRP device is still responsible for responding to ARP requests, even though the secondary FHRP device forwards the data trafic ‘The result of the enhanced vPC forwarding behavior is that the vPC peer link does not carry VPC traffic unless a vPC has lost all its ports on one of the peer devices. Thus, there is no direct need to scale the bandwidth on the vPC peer link as you deploy more vPCs on a pair of vPC switches. However, the operation of the vPC peer link is vital to the operation of vPC. Therefore, the vPC peer link should consist of at least two dedicated 10 Gigabit Ethernet links. ‘These two links should be terminated on different 1/0 modules if possible. It is also recommended to avoid the use of orphan devices with vPC, if possible. Traffic from orphan ports may need to be forwarded across the peer link and must be taken into account when scaling peer link capacity. Also, orphan devices may experience traffic disruption in specific vPC failure scenarios. 2140 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncvPC Control Plane + Cisco Fabric Services over Ethemet(FSoE) is used to synchronize vPC control plane information MAC address learning IGNP snooping ~ ‘Configuration consistency checking Te VPC member portstatus Exige ARP cache (configurable) + Dsables by default + One switch is elected primary other secondary Role determines behavior during peer ink failure Primary sw tchis leading for STP on vPOs Non-pre-emplive election * Single logical entity n LACPand STP ‘Toneighbor devices connected ona vPC Cisco Fabric Services over Ethernet is used as the primary control plane protocol for vPC. Cisco Fabric Services over Ethemet performs several functions: m= yPC peers must synchronize the Layer 2 MAC address table between the VPC peers. If one ¥PC peer learns a new MAC address on a vPC, that MAC address is also programmed on the Layer 2 Forwarding table of the other peer device for that same vPC. This MAC address learning mechanism replaces the regular switch MAC address learning mechanism and prevents traffic from being forwarded across the vPC peer link unnecessarily, m= The synchronization of IGMP snooping information is performed by Cisco Fabric Services. Layer 2 Forwarding of multicast traffic with vPC is based on modified IGMP snooping, behavior that synchronizes the IGMP entries between the vPC peers. In a vPC implementation, IGMP traffic entering a vPC peer switch through a vPC triggers hardware programming for the multicast entry on both vPC member devices. = Cisco Fabric Services is also used to communicate essential configuration information to ensure configuration consistency between the peer switches. Similar to regular port channels, vPCs are subject to consistency checks and compatibility checks. During a compatibility check, one vPC peer conveys configuration information to the other vPC peer in order to verify that vPC member ports can actually form a port channel. In addition to compatibility checks for the individual vPCs, Cisco Fabrie Services is also used to perform consistency checks for a set of switchwide parameters that need to be configured consistently on both peer switches Cisco Fabric Services is used to track vPC status on the peer. When all vPC member ports on one of the vPC peer switches go down, Cisco Fabrie Services is used to notify the vPC peer switch that its ports have become orphan ports and that traffic that is received on the peer link for that vPC should now be forwarded to the vPC. m= Layer 3 vPC peers may be configured to synchronize their respective ARP tables. This feature is disabled by default and can be enabled by using the ip arp synchronize command. If enabled, this feature helps ensure faster convergence time upon a vPC switch reload, When two switches are reconnected afier a failure, they use Cisco Fabric Services. to perform bulk synchronization of the ARP table. ‘©2012 Cisco Systems, Inc, (isco Nexus Switch Feature Configuration 2141Between the pair of vPC peer switches, an election is held to determine a primary and secondary vPC device. This election is non-pre-emptive. The vPC primary or secondary role is primarily a control plane role that determines which of the two switches will primarily be responsible for the generation and processing of spanning-tree BPDUs for the vPCs Note ‘The vPC peer-switch option allows both the primary and secondary devices to generate PDUs for vPCs independently. The two switches will use the same spanning-tree bridge 1D to ensure that devices connected on a vPC still see the vPC peers as a single logical switch. This option is discussed later in the lesson. Both switches actively participate in traffic forwarding for the vPCs. However, the primary and secondary roles are also important in certain failure scenarios, most notably in a peer link ilure. When the vPC peer link fails but the vPC peer switches determine through the peer keepalive mechanism that the peer switch is still operational, then the operational secondary switch suspends all vPC member ports. The secondary role also shuts down all switch virtual interfaces (SVIs) associated with any VLANs that are configured as allowed VLANS for the vPC peer link For LACP and STP, the two vPC peer switches present themselves asa single logical switch to devices connected on a vPC. For LACP, this result is accomplished by generating the LACP system ID from a reserved pool of MAC addresses, which are then combined with the vPC domain ID. For STP, the behavior depends on the use of the peer-switch option. If the peer- switch option is not used, the vPC primary is responsible for generating and processing BPDUS and uses its own bridge ID for the BPDUs. The secondary role relays BPDU messages but does not generate BPDUs itself for the vPCs. When the peer-switch option is used, both the primary and secondary switches send and process BPDUs. However, they use the same bridge ID to present themselves as a single switch to devices connected on a vPC. a2 Implementing Cisco Data Genter Unified Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncvPC Guidelines Attribute ei ‘Switch types. ‘The swiich type mustbe the same Forexamplepair50005000 oF 55005500 is supported, but rot 5000-5500, The vPC has tobe bul between the samelincard modules Link speed \VPC peer inks must consttof 10/10/10 GigabtEthemet pors. PC koepalve ‘Avod running vPC keopaveover vPC peer ink VPC peerlnk ‘Atleast two 10-Gigabit Ettenetinteaces. voc + VPC cannotstretch aaossmutipleVDCs ona single swith, + Each VDC win vPC requires sown vPGpeer ink and VPC poerkeepalve ink [Number of VPC peers AvPC domaincannotcorsist of more than twopeer swiches. ‘Number of vBCsper You cannot corfgure more than one vPC domain por swich ot VOC. ‘Switch Routing + Dynamicroutng tovPC peersacessa vPCoracross the vPC per ink isnot supported. + State routing actossa vPCto an FHRPaddrosses is supported, { Bynamcroutng scrossa vPC betwoan two Layer 3 wdheshatare not patiipaingn vPCis suppor. + YPC member ors mistbe on same ine cardtype cn both swiches.2. Mia, Fret MoM@etc Consider these guidelines and limitations when deploying vPCs: You must pair Cisco Nexus switches of the same type. For example, you can deploy vPC on a pair of Cisco Nexus 5000 Series switches or Cisco Nexus 5500 Platform switches but not on a combination of them. A vPC peer link must consist of Ethernet ports with an interface speed of 10 Gb/s or higher. It is recommended to use at least two 10 Gigabit Ethernet ports in dedicated mode on two different YO modules. vPC keepalive should not run across a vPC peer link, A vPC is a per-VDC function on the Cisco Nexus 7000 Series switches. A vPC can be configured in multiple VDCs, but the configuration is entirely independent. A separate vPC peer link and vPC peer keepalive link are required for each of the VDCs. vPC domains cannot be stretched across multiple VDCs on the same switch, and all ports for a given vPC must be in the same VDC. A vPC domain by definition cons yPC domain ID. 1 domain, s of a pair of switches that are identified by a shared not possible to add more than two switches or VDCs to a vPC Only one vPC domain ID can be configured on a single switch or VDC. It for a switch or VDC to participate in more than one vPC domain. not poss A vPC isa Layer 2 port channel. vPC does not support the configuration of Layer 3 port channels. Dynamic routing from the vPC peers to routers connected on a vPC is not supported. It is recommended that routing adjacencies are established on separate routed links. Static routing to FHRP addresses is supported. The FHRP enhancements for vPC enable routing to a virtual FHRP address across a vPC. A vPC can be used as a Layer 2 link to establish a routing adjacency between two external routers. The routing restrictions for vPCs only apply to routing adjacencies between the VPC peer switches and routers that are connected on a vPC. ‘©2012 Cisco Systems, Inc, {isco Nexus Switch Feature Configuration 2143Configuring vPC This topic explains how to configure vPCs on Cisco Nexus switches. vPC Configuration Procedure 1 2 3, 4 5. 6 7. 8. 9. Configure vPC domain Choose peer keepalive option Configure peer keepalive link Configure VPC peer link Configure vPCs Optimize vPC—peer gateway (optional) Optimize veC—peer switch (optional) Verify brief vPC (optional) Verify VPC consistency parameters (optional) Follow these steps to implement a vPC: step 1 step 2 Step 3 Step 4 Step 5 Step 6 step 7 Step 8 step 9 Enable the vPC feature and configure the vPC domain ID on both switches. Choose a peer keepalive deployment option. Establish the vPC peer keepalive link. Configure the vPC peer link. This step completes the global vPC configuration on both vPC peer switches. Configure individual vPCs to downstream devi Optionally, enable the peer gateway feature to modify the FHRP operation. Optionally, enable the peer switch feature to optimize the STP behavior with vPCs. Optionally, verify operation of the PC. Optionally, verify vPC consistency parameters. 2144 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncStep 1: Configure vPC Domain * VPC domain groups switches participating in the VPC Container for global vPC parameters + Automatic generation of vPC systemMAC address, Derived from vPC domain D By vPC peers, + Domain IDs must be unique in a contiguous Layer 2 domain. Renin) Rates vee witeh (ean ¢Lg-yperdomai ny VPC systan MAG mitch show vee role ross derived from domamniO ‘The vPC domain defines the pair of vPC peer switches that participate in vPC. When you enter the vPC domain ID, you enter a subconfiguration mode where you can then configure additional global parameters for the vPC domain. ‘The vPC domain ID is a value between I and 1000 that uniquely identifies the vPC switch pair ‘The vPC peer devices use the vPC domain ID that you configure to automatically assign a unique vPC system MAC address. Each vPC domain has a unique MAC address that is used as. a unique identifier for the specific vPC-related operation. Although the devices use the vPC system MAC addresses only for link-scope operations, such as LACP, it is recommended that you create each vPC domain within the contiguous Layer 2 network with a unique domain ID. You can also configure a specific MAC address for the vPC domain rather than having Cisco NX-OS Software assign the address. ‘The example in the figure shows how to configure and verify the vPC domain. These commands are used: m= feature ype: This command enables the vPC feature. = ype domain domain-id: This command configures the domain ID. The sime domain ID must be used on both yPC peer switches in the vPC domain. is command shows the result of the vPC role election, and it also shows the system MAC address that is derived from the vPC domain ID. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2145Step 2: Choose Peer Keepalive Link Deployment Peer keepalive link + Detects and resolves roles ifa dual-active condition occurs * Out-of-band heartbeat between vPC peers EToun bone oermerrry Cx Dedicated Use a dedicated port and Use a dedicated routed port in a non-mgmt VLAN separate VRF (a Gigabit Ethernet Port port is sufficient) 008 mgmt Use the 0OB management —_Use the OOB management interface mam0 interface mant0 * band Use an in-band Layer 3 Use an upstream Layer 3 network network *Donotuse crosseaties w.connectmgm ‘The peer keepalive link provides an OOB heartbeat between the VPC peer switches, which is used to detect and resolve a dual-active condition when the vPC peer link fails. The peer keepalives are IP-based and can be routed across an IP network if required. Because itis vital that peer keepalives never be earried on the VPC peer link, these recommendations are made for deployment of the peer keepalive infrastructure: = For Cisco Nexus 7000 Series switches, itis recommended that you create a separate virtual routing and forwarding (VRF) instance specifically for the vPC peer keepalives. By signing a specific routed port to this VRF, you can ensure that the peer keepalive traffic is always carried on that link and never carried on the peer link, Because this link carries only keepalives, a Gigabit Ethernet port is sufficient for this link, Also, the port that is used for the peer keepalive link should ideally be terminated on a different /O module than the links that form the peer link. If itis not possible to allocate a dedicated port for peer keepalives, the OOB management network can be used. However, in this ease, it is important that the management ports on both supervisors are connected to the OOB management network. Do not use Ethernet crossover cables to connect the management ports on the vPC peers to each other back-to-back. To do so will cause the peer keepalive Tink to fail on supervisor switchover. [Pneither of these options is available, an upstream Layer 3 network in the core or aggregation layer of the data center could be used for the peer keepalives. = For the Cisco Nexus 5000 Series switches, the recommendations are slightly different: tis, recommended to use the OOB management interface mgmt 0 for peer keepalives if possible. If this option is not available, a dedicated port with an associated VLAN and SVI should be used. IF it is also not possible to dedicate a separate port for the peer keepalives, then an in-band Layer 3 network can be used. However, you should take care that the VLAN associated with the peer keepalive connection is not allowed on the vPC peer link if this option is used. 2-146 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncStep 3: Configure Peer Keepalive Link * Should never be in VLAN carried on the PC peer link + The management VRF is used by default Sinan (angi we-aomsIny | pear kaepalive destination witch) show vpe peer-keepative YC keep-alive statue peer is alive “-ecr is alive for (231) seconds, (92) msec 2011.01.31 22:08:24 874 me en/27 2011.01.31 22:08:25 185 me Nexus 7000 example using VFS By default, the vPC peer keepalive packets are routed in the management VRF and use the OB mgmt 0 interface. You can configure the vPC peer link to use a different VRF, but you should take care that the peer keepalive traffic is not routed across the vPC peer link. ‘The example in the figure shows how to configure and verify the vPC peer keepalive link. ‘These commands are used: = peer-keepalive destination jp-address [souree ip-address] [vrf {name | management}]: This command specifies the destination IP address for the vPC peer keepalive link. By default, this IP address is resolved in the management VRP, but other VRFs can be specified. If'a VRF other than the management VRE is used, the source IP address should also be specified because the source IP address defaults to the management IP address. Additional options can be added to this command to change the timers and quality of service (QoS) values that are used by default = show ype peer-keepalive: This command can be used to verify the status of the vPC peer keepalive link. ‘©2012 Cisco Systems, (isco Nexus Switch Feature Configuration 2147Step 4: Configure Peer Link + WC peer link Carries data and control traffic betw een peer switches + Recommendations: Port channel of atleast twodedicated 10 Gigabit Ethernet Trunk mode (Only trangport of VPC VLANs Teitch [contlgh) intestate port channel 20 mivon (contigy ie)? maeenpere mode nun sriteh (contig-if}) gatempost trunk allowed vlan 100-105 swith leone ie | tative vian 100 aviteh teontig-i¥ ct type network ‘The vPC peer link carries essential vPC traffic between the vPC peer switches, The VPC peer link is a port channel, which should consist of at least two dedicated 10 Gigabit Ethernet links. ‘These links should be terminated on two different /O modules ‘The vPC peer link should be configured as a trunk, The allowed VLAN list for the trunk should be configured in such a way that only vPC VLANs (VLANs that are present on any vPCs) are allowed on the trunk. It is not recommended to carry non-vPC VLANs on the vPC peer link, because this configuration could evere traffic disruption for the non-vPC VLANS if the VPC peer link fails. use It is recommended that you enable Bridge Assurance on the vPC peer link and use Unidirectional Link Detection (UDLD) to protect against unidirectional link failures. ‘The example in the figure shows how to configure the vPC peer link. The primary command that is used in this example is the ype peer-link command, which assigns an existing port channel interface as the vPC peer link. The example also shows the configuration of the recommended best practices. 2148 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncStep 5: Configure vPCs * Port channel on both vPC peer switches + Port channels and physicalinterfaces compatible on both peers + Binding through PC number Must be unique forthe vPC domain Several vPC numbers can exist per domain Combines. port channels on peer switches into a vPC. Galtchh config)) antardece ethernet 1/3 suiteha contig-ity ty meitenatsont gif SuitehB ont ig“if-range] channel-group 7 mode active ouitend (sontigit-range] Smearface port-channel 7 ritchie tsnt n= ae = “* F* ‘Once the vPC domain has been properly established, the individual vPCs can be configured. To configure a vPC, a port channel must be configured on both vPC peer switches. These two port channels must then be associated with each other by assigning a vPC number to the port channel interfaces. The VPC port number is unique for the vPC within the vPC domain and must be identical on the two peer switch As with regular port channels, vPC member ports should have a compatible and consistent configuration. You should ensure that the configurations on vPC member ports are not only compatible on a single switch but also between peer switches. ‘The example in the figure shows how to use the vpe number command to combine two existing, port channel interfaces on the two vPC peer switches into a single vPC. The example also shows the use of the channel-group command to create the port channels that are combined into a vPC. In the example, the vPC is configured as a trunk. This configuration is optional. A YPC could also be configured as an access port, for example, ifit is connected to a dual-homed server. ‘©2012 Cisco Systems, Inc, {isco Nexus Switch Feature Configuration 2149Step 6: Optimize vPC Peer Gateway + Peer gateway feature Allows a vPC swtchto act as active gatew ay for traffic to peer router MAC ~ Forwards local traffic to VPC. node and avoids use of the peer link Interoperable with NAS and load balancers « ICMP redirects are disabled for SVis associated with PC VLANs ‘a Geieah conta F we domain 10 mitch [conti y-vperdoneia | peer-gatawny ‘You can configure vPC peer devices to act as the gateway, even for packets that are destined to the vPC peer device MAC address, ‘The vPC peer gateway capability allows a vPC switch to act as the active gateway for packets that are addressed fo the router MAC address of the vPC peer. This feature enables local forwarding of such packets without the need to cross the vPC peer link. In this scenario, the feature optimizes the use of the peer link and avoids potential traffic loss in FHRP scenarios. ‘The peer gateway feature must be configured on both primary and secondary vPC peers and be nondisruptive to the operations of the device or to the vPC traffic. The vPC peer-pateway feature can be configured globally under the vPC domain submode. When this feature is enabled, IP redirects are automatically di that are associated with a vPC VLAN. This avoids generation of IP redirect m packets that are switched through the peer gateway router Note Packets arriving at the peer gateway vPC device will have their Time to Live (TTL) decremented, so packets carrying TTL = 1 may be dropped in transit because of TTL ‘expiration. This fact needs to be taken into account whan the peer gateway feature is enabled and particular network protocols sourcing packets with TTL = 1 operate on a vPC VLAN. 2-150 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncStep 7: Optimize vPC Peer Switch + VPC peer switch feature ~ VPC primary and secondary are both otdevices Different STP behavieron vPC and nonPC ports + On vPC ports BPDUsoriginatedby primary and secondary deviceswith same designated bridge ID + On non-vPC ports Maintain local bidge ID instoadof the vPCbridgeID ‘Advertise Bridge IDof the vPC system asthe root ume + Better convergence senvige0 During vPC primary switch fllure andrecovery sate AvoidsRSTP 9ync + No need forpinning the STP root to vPC primary switch witch conti) wpe domain 10 mteh (oon Lq-vpedomat futbol leon Fag vpe-domai 22:46:08 NTK-2-poas, ‘SGmp-2-viePIERDWITCH CONFIG ENARLED: vPC peer-awitch ‘configuration is enabled. Please nake sure to configure spennlng tase "eldge” priority on pes ecommanded ‘The peer switch option optimizes the behavior of spanning tree with vPCs: m= The vPC primary and secondary are both root devices and both originate BPDUs. = The BPDUs originated by both the vPC primary and the vPC secondary have the same designated bridge ID on vPC ports. = The BPDUs originated by the vPC primary and secondary devices on non-vPC ports maintain the local bridge ID instead of the vPC bridge ID and advertise the bridge ID of the ¥PC system as the root. ‘The peer switch option has these advantages: It reduces the traffic loss upon restoration of the peer link after a failure. It reduces the disruption that is associated with a dual-active failure, whereby both vPC members become primary. Both devices keep sending BPDUs with the same bridge ID information on vPC member ports. This prevents the port channel STP consistency feature from potentially disabling the port channel on an attached device. = Itreduces the potential loss of BPDUs if the primary and secondary roles change. ‘The example in the figure shows how to configure the peer-switch feature by using the peer- switch command. In addition to enabling the peer-switch feature, you should also set the be possible spanning tree bridge priority value on both peer switches. This setting forces the vPC switch pair to become the root of the spanning tree for the vPC VLANs. ‘©2012 Cisco Systems, {isco Nexus Switch Feature Configuration 2151Step 8: Verify Basic vPC geen Taetgetce tee) canegtcy art Syed onatniny aac Poor gateway enabied Beas Default seting woul Taber Se te contiouet Show -Dsabied Several commands can be used to verify the operation of vPC. The primary command to be used in initial verification is the show ype brief command. This command displays the vPC domain ID, the peer-link status, the keepalive message status, whether the configuration consistency is successful, and whether a peer link has formed. The command also displays the status of the individual vPCs that are configured on the switch, including the result of the consistency checks 2-152 _ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncStep 9: Verify VPC Consistency Parameters i a a aT ‘ype 1: vec wil be muspondad in case of mismatch SEE er megtce Se i cumt oie Jovstend show vp consistency parameters ype Local anspeer valves = ‘must match ype toon vase D8, F-20-4m-benac} [ith 0-23-tm-bene) io'ctvs ies cal sumpended Via ii ver If the show ype brief command displays failed consistency checks, you can use the show vpe consistency-parameters command to find the specific parameters that caused the consistency check to fail. The global option on this command allows you to verify the consistency of the global parameters between the two peer switches. The ype or interface options can be used to verify consistency between the port channel configurations for vPC member ports Afier you enable the vPC feature and configure the peer link on both vPC peer devices, Cisco Fabric Services messages provide a copy of the configuration on the local vPC de configuration to the remote vPC peer device. The system determines whether any of the crucial configuration parameters differ on the two devices. The parameters must be configured identically or the vPC moves into “suspend” mode. The per-interface parameters must be consistent per interface, and the global parameters must be consistent globally: m= Port channel mode: on, off, or active = Link speed per channel = Duplex mode per channel = Trunk mode per channel, including native VLAN, VLANs allowed on trunk, and the tagging of native VLAN traffic = STP mode = STP region configuration for Multiple Spanning Tree (MST) = Enabled or disabled state per VLAN = STP global settings, including Bridge Assurance setting, port type, and loop guard settings m= STP interface settings, including port type, loop guard, and root guard =| Maximum transmission unit (MTU) ‘©2012 Gisco Systems, (isco Nexus Switch Feature Configuration 2183Configuring the FEX This topic explains how to configure the Cisco Nexus 2000 Seri FEX connected to a Cisco ‘Nexus 5000 or 7000 Series swi FEX Deployment Models Fabric Extenders (FEXs) can be deployed using three different models: 1, Straight-through FEXusing static pinning (discussed previously) 2. Straight-through FEXusing dynamic pinning Uses port channets 3. Active-active FEXusing VPC Uses port channels and virtual port channels nae ‘Siaightihvough emery © rami pes © wtwe-actve ve. Nexus 00/5500 ‘Nexus $900/55007000 Nexus $000/5500/7000 There are three deployment models that are used to deploy FEXs together with the Cisco Nexus 5000 and 7000 Series switches: = Straight-through using static pinning: In the straight-through model, each FEX is connected to a single Cisco Nexus switch. The single switch that the FEX is connected to exclusively manages the ports on that particular FEX. Static pinning means that each downlink server port on the FEX is statically pinned to one of the uplinks between the FEX and the switch. Traffic to and from a specific server port always uses the same uplink. This model uses neither port channels nor vPCs and was explained in the “Configuring Layer 2 Switching Features lesson, = Straight-through using dynami ing: This deployment model also uses the straight- through connection model between the FEXs and the switches. However, there is no static, relation between the downlink server ports and the uplink ports. The ports between the FEX and the switch are bundled into a port channel, and traffic is distributed across the uplinks based on the port channel hashing mechanism. = Active-aetive FEX using vPC: In this deployment model, the FEX is dual-homed to two Cisco Nexus switches. vPC is used on the link between the FEX and the pair of switches. Traffic is forwarded between the FEX and the switches based on vPC forwarding mechanisms. Note ‘The second and third models are discussed in this topic. 2154 _ Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncDynamic Pinning » Port channel between Cisco Nexus switch and FEX2000 * Trafic distribution across the uplinks determined through port channel hashing + Failure scenarios: A. One or few uplinks fail ‘Server dow nlinks are not disabled eS nonsense © a korn tan Sonn womccney = YG + Dual-homed servers fail over to other NCIFEX * Oversubscription ratio changes: In dynamic pinning mode, the server ports are not statically pinned to an uplink port, but all server ports share the combined bandwidth of the uplink ports. This is achieved by configuring ‘port channel between the Cisco Nexus 2000 Series FEX and the Cisco Nexus 5000 Series switch. Instead of statically assigning traffic from specific server ports to the uplink port that they are pinned to, traffic is now distributed over the uplinks based on the port channel load- balancing hash algorithm. Ifone of the uplinks between the Cisco Nexus 2000 Series FEX and the Cisco Nexus 5000 Series switch fails, the FEX will not disable any server ports because there is no longer a direct relationship between the uplink ports and the server ports. Instead, the traffic of the 48 server ports is now distributed over the remaining three 10 Gigabit Ethernet uplinks. The servers that are connected to the Cisco Nexus 2000 Series FEX will not register the failure and will keep forwarding traffic on the NIC that is connected to the FEX. Single-homed servers will not lose connectivity when using dynamic pinning, Their traffic is simply redistributed over the remaining uplinks. Dual-homed servers will not fail over to the redundant NIC, However, this means that the oversubscription ratio for the remaining uplink ports changes. The oversubscription ratio for the remaining ports in the example is 48:30 = 1.6:1, which represents a 33-percent increase in traffic on each uplink port. Ifthe uplinks are already running close to maximum utilization, it may cause traffic from all servers to be dropped, thereby degrading performance for all servers. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2155Configure Dynamic Pinning Enable the FEX feature and define the FEX instance number For dynamic pinning, setthe number of uplinks to 1 Configure “fex-fabric” mode on ports connecting to FEX Associate the ports with the channel group Associate the port channel interface with the FEX Gaicch contig) Seatare fox fSiiten feoneu) # fax 21 muiteh (configrfex) 1 daeersption "HEX 122, rack 2, top” Javitch (eontig-fox) 4 pinning max-Links 2 Stange ‘in max-Ainke Wil1 cause craffic disruption, miss tte oped tment a ee a tr arasie @ Cena Sees itch contig-te)v fer ssebeiate iat @ Follow these steps when implementing dynamic pinning: Step1 Enable the FEX feature and define the FEX instance number Step2 For dynamic pinning, set the number of uplinks to I. The pinning max-links parameter is set to one, because all of the ports are pinned to the port channel interface instead of the individual physical interfaces. Step3 Configure fex-fabrie mode on ports connecting to a FEX. Step 4 Associate the ports connecting to an FEX with the channel group. step5 A \ciate the port channel interface with the FEX. Note The pinning maxctinks command isnot required on the Cisco Nexus 7000 Series switches because only dynamic pinning is supported 2-156 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncActive-Active FEX + FEXdual-homed to two Cisco Nexus 5000/5500 switches + Highest availability of FEXbased solutions Protection against failures of uplinks, FEX, and switch * PC as FEX uplink connection PC ports on FEX configured consistently on both sw itches Consistency checks as for regular vPCs + Configuration synchronization feature Automatic configuration synchronization Available on Cisco Nexus 5000 switches we ead Nexus 5000/6500 In the active-active FEX deployment, the Cisco Nexus 2000 Series FEX is controlled and configured by two Cisco Nexus 5000 Series switches. Both switches must be configured in a consistent manner, and vPC has to be set up to combine the FEX uplinks into a single port channel. Because a vPC-based port channel is used between the FEXs and the switches, dynamic pinning is automatically used. Traffic is balanced across the FEX uplinks based on port channel load balancing, When one of the FEX uplinks is lost, traffic will be balanced across the remaining uplinks Because vPC is used between the FEXs and the switches, vPC cannot be used between the FEXs and the servers. This means that port channels cannot be configured between the server and the access switches. Active/standby and transmit load balancing can still be used as NIC teaming options to attain high availability and some extent of load balancing, In the active-active FEX model, single-homed servers will maintain connectivity if one of the Cisco Nexus switches fails. Thus, this design increases the availability of single-homed servers to level that is comparable to that of a single-homed server that is connected to a chassis based switch with dual-redundant supervisors. Note The two Cisco Nexus switches are configured independently. Therefore, configuration changes that are made to ports on a dual-homed Cisco Nexus 2000 Series FEX have to be ‘manually synchronized between the two Cisco Nexus switches. To ease this administrative burden, the Cisco Nexus 5000 Series switch supports a configuration synchronization feature called switch profiles, which were discussed earlier in the lesson. ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2187Active-Active FEX Configuration Procedure Enable FEXfeature and define the FEX instance number For active-active FEX set the number of uplinks to 1 Configure “fex-fabric’ mode on ports connecting to FEX Associate the ports with the channel group Enable and configure vPC on both switches | a Dorrain vec Peer keepalive link sooiiauston Peer link Configure the port channel connected to the FEX ‘Same VPC number on both switches Association with FEX Configure ports on FEX Bode baween PC, vec and Fex FEXports ‘The initial part of an active-active FEX configuration is similar to a straight-through configuration with dynamic pinning. The FEX is created and the fabric ports are configured and combined in a channel group. All these commands, shown in Steps 1-4, have to be executed on both switches. Next, vPCs should be configured. A vPC domain, including a peer keepalive link and peer link, should be created. This is shown in Step 5. Next, the port channels that are associated with the FEX fabric ports are combined into a vPC, shown in Step 6. Once the vPC between the FEX and the switches has successfully formed, the ports on the FEX will be visible on both switches and can be configured in Step 7. Effectively, each of the individual ports on the FEX is treated as a vPC on the Cisco Nexus switches. Therefore, the same consistency checks are applied to the ports on the FEX as for regular yPCs. This means that any configuration change that is made on a port on the FEX should be made consistently on both of the vPC peer switches. 2-158 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncActive-Active FEX Configuration ‘intaacainie Will cates e [Sore se mane orca] cng) interac thar 137-20 iret petite oe err 3) iq) ineartace etharaee apcag: a 2 MEE 2 Tis)d intartece ports 1 @ FO pox irkpor charm 1 cit te eR Gre = fis tects gore 2 emmanne | Se = Oe ig) ipearface ethernet 131/1/2 ise aa ade ea) cnt lvieye ansecipore’ scoeea van 20 (cineca Lae age teg8 aaltepet actens van 0 od (ervoors ent FEX shot) This figure illustrates an active: previous procedure. jive FEX configuration example that was described in the ‘©2012 Cisco Systems, Inc, {isco Nexus Switch Feature Configuration 2159Configuring Dynamic Pinning on Nexus 7000 1. Install the FEX feature set in the default VDC 2. Enable or disable the FEX feature per VDC. Dofaut is alowed 3. FEX fabric interfaces must be members ofa port channel Cisco Nexus 7000 switches only support dynamic pinning Tk contig) ) Snaeail fostare-sot for Nik ewitehto” we RED (contig) Geature-set fox fox tal SVT Goecription "FEx 141, rack 4, top” snterfece ethernet 1/1-2, ethernet 1/9-10 NK ReD(cont ig) Sntarface port-chanpel & NieReDicontig-ith few associate 141 Configuration of an FEX on a Cisco Nexus 7000 Series switch is slightly different from the configuration on a Cisco Nexus 5000 Series switch. Partially, this is caused by the VDC-based architecture of the Cisco Nexus 7000 Series switches. Before a FEX can be configured in a VDC, the services that are required by the FEX feature need to be installed in the default VDC. To enable the use of the FEX feature set, use the install feature-set fex command in the default VDC. After the PEX feature set has been installed in the default VDC, the feature set can be enabled in any VDC by using the feature-set fex command, It is possible to restrict the use of the FEX feature set to specific VDCs only. By default, all VDCs can enable the FEX feature set once it has been installed in the default VDC. If you want to disallow the use of FEXs in a specific VDC, you can use the no allow feature-set fex command in VDC configuration mode for that particular VDC. Another difference with the FEX configuration on the Cisco Nexus 5000 Series switches is that the Cisco Nexus 7000 Series switches only support dynamic pinning, which makes it unnecessary to specify the maximum number of pinning interfaces by using the pinning max- inks command. 2160 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncConfiguring Enhanced vPCs This topic explains how to configure the Enhanced vPCs on a Cisco Nexus 5000 Series switch Enhanced vPC + Combination of two supported vPC topologies: Dual-homed connection of a host to two fabric extenders (FEXs) Dual-homed connection of an FEX to twoswitches + Enhanced vPC (two-layer VPC) ~ All paths from hosts to FEXs and then to switches are active ‘Supported on Nexus $500 (release 5.1(3)N1(1) or later) and any Nexus 2000 Supports Layer 3 on Nexus 5500 Fox tore a maze 7 a = vee vc = £ vec vro pa + 4 wee: ‘The Enhanced vPC feature, known as two-layer vPC, combines two dual-homing in one solution ve: = Dual-homed connection of a host to two FEXs = Dual-homed connection of an FEX to two switches ‘The combined topology is shown in the figure. With Enhanced vPC, all available paths from the hosts to the FEXs and from the FEXs to the ve and carry Ethernet traffic, maximizing the available bandwidth and providing redundancy at both levels. Enhanced vPC is supported on the Cisco Nexus 5500 Platform switch running NX-OS Release 5.13)N1(1) ora later release. Enhanced vPC can be deployed with any Cisco Nexus 2000 Series Fabric Extender. Enhanced vPC is compatible with Layer 3 features on the switch. 2164 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncNon-Recommended Topologies {. Dual-homed server connected to a pair of FEXs that connect to a single switch Not recommended despite FEX redundancy 2. Multinomed server connected by a port channel to more than two FEXs hncreased complexity withlitle benefit Enhanced vPC does not support the following topologies: = A dual-homed server that is connected to a pair of FEXs that connects to a single switeh: Although this topology becomes a functioning system when one switch has failed, itis not recommended in normal operation. = A multihomed server that is connected by a port channel to more than two FEXs: This topology results in increased complexity with little benefit. 2-166 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, IncEnhanced vPC Configuration Procedure 1. Enable and configure vPC on both switches, Domain Peer keepalve link Peer link 2. Configure port channels from the first FEX “fex-fabric" mode on ports connecting to FEX VPC nunber Associate the ports with the channel group 3. Configure port channels from the second FEX same as above) 4, Configure a host port channel on each FEX Perform these tasks to implement Enhanced vPC on Cisco Nexus 5500 Platform switches 1. Enable and configure vPC on both switches. The vPC parameters include the domain ID, peer keepalive link, and peer link. Configure port channels from the first FEX. Within this task, you need to configure fex- fabric mode on ports connecting to the FEX, define the vPC number, and associate the ports with the channel group. 3. Configure port channels from the second FEX. The individual steps are identical to those in ‘Task 2. If you configure the enhanced vPC for Fibre Channel over Ethernet (FCoE) traffic, associate the first FEX to one switch, then associate the second FEX to the other switch. 4, Configure a host port channel on each FEX 2-168 Implementing Cisco Data Genter Untied Fabric (OCUFI v5.0 (© 2012 Cisco Systems, IncEnhanced vPC Configuration aztsceseht/1"2 SIDE Sige te ek oS Nesbo- (Snr i fes9 itergace eenay3-a SS ae oe Nesoo-fconr ato te Satara Conti pa area Went eal FE Nesoeafcourdy-ie}9 aterm NSS (Surlg-10}4 Poe aaeactate so Es Raeaee BEGLAIET Beas coyee [Snr STE 1 stttiece nate tate —t ana exanaienre Te] ee ISM RRS ha ASiSanee —[eedmanroe BSE NEME-EN Hote wes tm 8 ‘The configuration example depicts some of the components that are required for an Enhanced vPC solution: = The port channel that is used for the peer link: In this case, its ID is = The port channel that is used for the links connecting the first parent switch to the first FEX: In this case, its ID is “101,” and it is configured for a vPC with the same number and associated with an FEX of the same number = The port channel that is used for the links connecting the first parent switch to the second FEX: In this case, its ID is “102,” and it is configured for a vPC with the same number and associated with an FEX of the same number. m= The port channel that groups the links connecting the host to the switeh fabric: In this, case, the host has two interfaces that are connected to the first two Ethemet ports on the first FEX (Ethernet 101/1/1-2) and two interfaces that are connected to the first two Ethernet ports on the second FEX (Ethernet 102/1/1-2). The port channel grouping these links has an ID of “2.” ‘©2012 Gisco Systems, {isco Nexus Switch Feature Configuration 2169Summary This topic summarizes the key points that were discussed in this lesson. Summary Port channels and vPCs improve network availability and optimize bandwidth usage. Channel groups are used to create a port channel interface. WC enables logical loop-free, dual-home topologies. ‘AvPC domain consists of two Cisco Nexus switches connected through a peer link, which is protected by a peer keepalive link. Port channels can be used to connect FEXS to Cisco Nexus switches using dynamic pinning (port channel) and active-active FEX (VPC), Enhanced PC combines two VPC topologies—hosts dual-homed to two FEXs and FEXs dual-homed to two Nexus 5500 Platform switches. References For additional information, refer to these resources: To leam more about configuring Cisco Nexus 2000 Series FEX, refer to Cisco Nexus 2000 Series Fabric Extender Software Configuration Guide at this URL: http://www cisco.com/en/US/does/switches/datacenter/nexus2000/sw/configuration/guide/t el_6 0/6 Configuring the Cisco Nexus 2000 Series Fabric Extender rel_6 O.html To lea more about configuring port channels, yPCs, and enhanced vPCs on Cisco Nexus 5000 Series switches and Cisco Nexus 5500 Platform switches, refer to Cisco Nexus 5000 Series NX-OS Layer 2 Switching Configuration Guide, Release 5.1(3)N2(1) at this URL: http:/www cisco.com/en/US/docs/switchesdatacenter/nexusS000/sw/layer2/513_N2_1/b Cisco_nSk_Layer2_Config_$13_N2_1 html To leam more about configuring port channels and vPCs on Cisco Nexus 7000 Series switches, refer to Cisco Nexus 7000 Series NX-OS Interfaces Configuration Guide, Release 6. at this URL: http://www.cisco.com/en/US/does/switches/datacenter/sw/6_x/nx- os/interfaces/configuration/guide/if_preface.html 2170 Implementing Cisco Data Genter Untied Fabric (OGUFI v5.0 (© 2012 Cisco Systems, Inc