Data Center Networking Technologies

Speaker Name: ABDULSALAM ALSHAQHAA

February 2020
Chapters to be covered.

Chapter 1. Data Center Networking

Chapter 2. Management and Monitoring of Cisco Nexus Devices
Chapter 3. Unified Fabric Overview

2
Chapter 1. Data Center Networking

3
Data Center Networking

• Data centers are designed to host critical computing resources in a centralized

place.

• Resources can be utilized from a campus building of an enterprise network, from a

branch office over the wide area network (WAN), and from the remote locations
over the public Internet.

• Network plays an important role in the availability of data center resources.

4
Port Channel
• A port channel is an aggregation of multiple physical interfaces that create a
logical interface.
• On the Nexus 5000 Series switch, you can bundle up to 16 links into a port
channel. On the Nexus 7000 Series switch, you can bundle 8 active ports on an M-
Series module and up to 16 ports on the F-Series module.
• When a port channel is created, you will see a new interface in the switch
configuration. This new interface is a logical representation of all the member
ports of the port channel.

5
Benefits of Port Channel
• Increased capacity: By combing multiple Ethernet links into one logical link, you can
increase the capacity of the link.

• High availability: Multiple Ethernet links are bundled together to create a logical link. In case
of a physical link failure, the port channel continues to operate even if a single member link is
alive.

• Load balancing: Switch distributes traffic across all operational interfaces in the port channel.
This enables you to distribute traffic across multiple physical interfaces, increasing the
efficiency of network.

• Simplified network topology: Simplify network topology by aggregating multiple links

into one logical link. If you have multiple links between a pair of switches, STP blocks all
the links except one. In the case of port channels, multiple links are combined into one
logical link; therefore, it simplifies the network topology by avoiding the STP calculation.
6
Link Aggregation Control Protocol (LACP)
• LACP provides a standard mechanism to ensure that peer switches exchange
information and agree on the necessary details to bundle ports into a port channel.

• LACP must be enabled on switches at both ends of the link.

• Protocol ensures that both sides of the link have compatible configuration (speed,
duplex, flow control, allowed VLAN) to form a bundle.

• On Nexus 7000 switches, LACP enables you to configure up to 16 interfaces into

a port channel.

7
Configuring Port Channel
Port channel configuration on the Cisco Nexus switches includes the following steps:

1. Enable the LACP feature. This step is required only if you are using Active
mode or Passive mode.

2. Configure the physical interface of the switch with the channel-group command
and specify the channel number. You can also specify the channel mode On,
Active, or Passive within the channel-group command. This command
automatically creates an interface port channel with the number that you
specified in the command.

3. Configure the newly created interface port channel with the appropriate
configuration, such as description, trunk configuration, and allowed VLANs.

8
Configuring Port Channel

9
Verifying Port Channel Configuration
• Several show commands are available on the Nexus switch to check port channel
configuration.
• These commands are helpful to verify port channel configuration and troubleshoot
the port channel issue:

10
Virtual Port Channel
• A vPC enables the extension of a port channel across two physical switches.

• Two switches work together to create a virtual domain, so a port channel can be
extended across the two devices within this virtual domain.

• Member ports in a virtual port channel can be from two different network switches.

• In Layer 2 network design, Cisco vPC technology allows dual-homing of a

downstream device to two upstream switches.

• The upstream switches present themselves to the downstream device as one switch
from the port channel and STP perspective.
11
Virtual Port Channel
• The limitation of the classic port channel is that it operates between only two
devices. In large networks with redundant devices, the alternative path is often
connected to a different network switch in a topology that would cause a loop.
• vPC addresses this limitation by allowing a pair of switches acting as a virtual
endpoint, so it looks like a single logical entity to port channel–attached devices.

12
Feature Overview
vPC Terminology

13
Benefits of vPC
vPC provides the following benefits:

■ Enables a single device to use a port channel across two upstream switches
■ Eliminates STP blocked ports
■ Provides a loop-free topology
■ Uses all available uplink bandwidth
■ Provides fast convergence in the case of link or device failure
■ Provides link-level resiliency
■ Helps ensure high availability

14
Configuration steps of vPC
• Configuration of vPC must be done on both peer switches. Make sure that peer
switches are connected to each other via physical links that can support the
requirements of a vPC peer link and that a Layer 3 path is available between the
two peer switches.

vPC configuration on the Cisco Nexus switches includes the following steps:
1. Enable the vPC feature on both peer switches using the following command:
feature vpc
2. Create the vPC domain and assign a domain ID to this domain. This domain
ID must be unique and should be configured on both vPC switches. Use the
following command to create the vPC domain:
vpc domain <domain-id>
15
Configuration steps of vPC
3. Establish peer keepalive connectivity. A peer keepalive link must be
configured before you set up the vPC peer link. The peer keepalive link can be in
any VRF, including management. Use the following command under the vPC
domain to configure the peer keepalive link:
peer-keepalive destination <remote peer IP> source <local IP> vrf mgmt.

4. Create a peer link. The vPC peer link is configured in a port channel
configuration. This port channel must be trunked to enable multiple VLANs
over the vPC. Use the following commands under the port channel to configure
the peer link:
Interface port-channel <peer link number>
switchport mode trunk
vpc peer-link
16
Configuration steps of vPC
5. Create the vPC itself. Create a regular port channel and add the vpc
command under the port channel configuration to create a vPC. This
configuration must be the same on both peer switches.
Interface port-channel <vPC number>
vpc <vPC number>

17
Configuration steps of vPC

18
Verification of vPC

19
FabricPath
• Server virtualization and seamless workload mobility is a leading trend in the data
center.

• A large-scale, highly scalable, and flexible Layer 2 fabric is needed in most data
centers to support distributed and virtualized application workloads.

• To understand FabricPath, it is important to have knowledge about following

limitations of STP:
 Lack of multipathing
 Inefficient path selection
 Slow convergence
 Protocol safety
 Limited scalability
20
What Is FabricPath?
• FabricPath network is created by connecting a group of switches into an arbitrary
topology, combining them in a fabric using a few simple commands.
• IS-IS Protocol is used to provide fabric-wide intelligence and tie the elements
together. There is no need to run STP within the FabricPath network.
• FabricPath can be deployed in any arbitrary topology; however, a typical spine
and leaf topology is used in the data center.
FabricPath Topology

21
Benefits of FabricPath
FabricPath converts the entire data center into one big switching fabric. Benefits of
Cisco FabricPath technology are as follows:

1. Operational simplicity.
2. Flexibility
3. High performance
4. Reliability based on proven technologies
5. High scalability and high availability
6. Efficiency

22
Components of FabricPath
• To understand FabricPath control and data plane operation, it is important to
understand the components of FabricPath.

23
Components of FabricPath (Contd..)
• Spine switches: In a two-tier architecture, spine switches are deployed to provide
connectivity between leaf switches. Spine switches act as the backbone of the
switching fabric. The role of spine switches is to provide a transit network
between the leaf switches.
• Leaf switches: Leaf switches provide access layer connectivity for servers and
other access layer devices. The hosts on two leaf switches use spine switches to
communicate with each other.
• FabricPath network: A network that connects the FabricPath switches is called a
FabricPath network. Typical FabricPath network is built using a spine and leaf
topology. In FabricPath network, packets are forwarded based on a new header,
which includes information such as Switch ID and hop count. FabricPath header is
added to the Layer 2 packet, when a packet enters the FabricPath network at the
ingress switch, and it is stripped when the packet leaves the FabricPath network at
the egress switch.
24
Components of FabricPath (Contd..)
• Classic Ethernet (CE): Conventional Ethernet using transparent bridging and
running STP is referred to as Classic Ethernet. Nexus switches can be part of both
a FabricPath network and a Classic Ethernet network at the same time, with some
ports participating in FabricPath and some ports in a Classic Ethernet network.
• Core port: Ports that are part of a FabricPath network are called core ports.
FabricPath core ports always forward Ethernet frames encapsulated in a
FabricPath header. Ethernet frames transmitted on core ports always carry an
IEEE 802.1Q VLAN tag and can be considered as a trunk port. Only FabricPath
VLANs are allowed on core ports.
• Edge port: Edge ports are switch interfaces that are part of Classic Ethernet.
These interfaces behave like normal Ethernet ports, and you can connect any
Ethernet device to these ports. Because these ports are part of Classic Ethernet,
MAC learning is performed as usual, and packets are transmitted using standard
IEEE 802.3 Ethernet frames.
25
Components of FabricPath (Contd..)
• FabricPath VLANs: The VLANs that are allowed on the FabricPath network are
called FabricPath VLANs. When you create a VLAN, by default it operates in
Classic Ethernet (CE) mode. To allow a VLAN over the FabricPath network, you
must go to the VLAN configuration and change the mode to FabricPath. An edge
port can carry both CE as well as FabricPath VLANs. Therefore, devices
connected on an edge port in the FabricPath VLAN can send traffic to a remote
host over the FabricPath network.
• FabricPath IS-IS: FabricPath switches run a Shortest Path First (SPF) routing
protocol based on standard IS-IS to build their forwarding table similar to a Layer
3 network. This link state protocol is the core of the FabricPath control plane.
• Dynamic Resource Allocation Protocol: DRAP is an extension to FabricPath IS-
IS that ensures network-wide unique and consistent Switch IDs and FTAG values.

26
Components of FabricPath (Contd..)
• Switch table: The switch table provides Layer 2 routing information based on
Switch ID. It contains Switch IDs and next-hop interfaces to reach the switch.
• MAC address table: This table contains MAC address entries and the source
from where these MAC addresses are learned. The source could be a local switch
interface or a remote switch. If a MAC address table has a local switch interface,
the packet is delivered using Classic Ethernet format. In a case in which a MAC
address table is pointing to a remote switch, a lookup is done in the switch table to
find the next-hop interface. The packet is then encapsulated in the FabricPath
header and sent to the next hop within the FabricPath network.
• FabricPath topology: Similar to STP, in which a set of VLANs could belong to
one or a different instance, in FabricPath a set of VLANs that have the same
forwarding behavior will be mapped to a topology or forwarding instance, and
different topologies could be constructed for a different set of VLANs. This could
be used for traffic engineering, administrative purposes, or security .
27
Configuring FabricPath
FabricPath can be configured using a few simple steps. You perform these steps on
all FabricPath switches. These steps involve enabling the FabricPath feature,
identifying FabricPath interfaces, and identifying FabricPath VLANs. Before you
start FabricPath configuration, ensure the following:

■ You have Nexus devices that support FabricPath.

■ The system is running at minimum NX-OS 5.1.1 (Nexus 7000)/NX-OS 5.1.3
(Nexus 5500) software release.
■ You have the appropriate license for FabricPath. The Enhanced Layer 2
license is required to run FabricPath.
■ If required, install the license using the following command: install license
<filename>
■ Install the FabricPath feature set by issuing install feature-set fabricpath.
28
Configuring FabricPath (Contd..)
The FabricPath configuration steps are as follows:
1. Before you start configuring the FabricPath on a switch, the FabricPath
feature set must be enabled. To enable the FabricPath feature set, use the following
command in switch global configuration mode:
feature-set fabricpath
2. Define FabricPath VLANs: Identify the VLANs that will be extended
across the FabricPath network. By default, all the VLANs are in Classic Ethernet
mode. You can change the VLAN mode to FabricPath by going into the
VLAN configuration. Use the following command to change the VLAN mode for
a range of VLANs:
vlan <range>
mode fabricpath

29
Configuring FabricPath (Contd..)
3. Identify FabricPath interfaces: Identify the FabricPath core ports. These
ports connect to other FabricPath switches to establish IS-IS adjacency.
There is no need to configure the IS-IS protocol for FabricPath. Configuring a
port in FabricPath mode automatically enables the IS-IS protocol on the port. The
following command is used to configure an interface in FabricPath mode:
interface <name>
switchport mode fabricpath

• After performing this configuration on all FabricPath switches in the topology,

FabricPath devices will form IS-IS adjacencies. The unicast and multicast routing
information is exchanged, and switches will start forwarding traffic

30
Configuring FabricPath (Contd..)

31
Verifying FabricPath

32
Chapter 2. Management and Monitoring of
Cisco Nexus Devices

33
Operational Plane of Nexus

34
Nexus Management and Monitoring Features
• The Nexus platform provides a wide range of management and monitoring
features. Following figure shows an overview of the management and monitoring
tools:

35
Out-of-Band Management
• In a data center network, out-of-band (OOB) management means that
management traffic is traversing through a dedicated path in the network. The
purpose of creating an out-of-band network is to increase the security and
availability of management capabilities within the data center.

• OOB management enables a network administrator to access the devices in the

data center for routine changes, monitoring, and troubleshooting without
dependency on the network build for user traffic. The advantage of this approach
is that during a network outage, network operators can reach the devices using this
OOB management network.

• Dedicated switches, routers, and firewalls are deployed to create an OOB

network. Network is dedicated for management traffic only and is isolated from
the user traffic. Network provides connectivity to management tools, management
port of network devices, and ILO ports of servers. 36
Out-of-Band Management (contd..)
Cisco Nexus switches can be connected to the OOB management network using the
following methods:
■ Console port
■ Connectivity management processor (CMP)
■ Management port

37
Console Port
• The console port is an asynchronous serial port used for initial switch
configuration. It provides an RS-232 connection with an RJ-45 interface. It is
recommended to connect all console ports of the devices to a terminal or comm
server router such as Cisco 2900 for remotely connecting to the console port of
devices. The remote administrator connects to the terminal server router and
performs a reverse telnet to connect to the console port of the device. Following
Figure shows remote access to the console port via OOB network.

38
Connectivity Management Processor
For management high availability, the Cisco Nexus 7000 series switches have a
connectivity management processor that is known as the CMP. CMP provides OOB
management and monitoring capability independent from the primary operating
system. The CMP enables lights-out remote monitoring and management of the
Cisco Nexus 7000 Series system, its supervisor module, and all other modules
without the need for separate terminal servers. Following are key features of CMP:
■ Provides a dedicated operating environment independent of the switch
processor.
■ Provides monitoring of supervisor status and initiation of resets.
■ Provides complete visibility of the system during the entire boot process.
■ Provides the capability to initiate a complete system restart.
■ Provides access to critical log information on the supervisor.
■ Provides the capability to take complete console control of the supervisor.
39
Connectivity Management Processor (contd..)
• Following figure shows how to connect to the CMP from the active supervisor
module.

40
Management Port (mgmt0)
• The management port of the Nexus switch is an Ethernet interface to provide
OOB management connectivity. It enables you to manage the device using an
IPv4 or IPv6 address. The management port is also known as the mgmt0 interface.
This port is part of management VRF on the switch and supports speeds of
10/100/1000 Ethernet.

41
In-Band Management

• In-band management utilizes the same network as user data to manage network
devices. In this method, switches are configured with an in-band IP address on a
routed interface, SVI, or loopback interface that is reachable via the same path as
user data. The network administrator manages the switch using this in-band IP
address. There is no need to build a separate OOB management network. If the
network is down, in-band management will not work either.

42
Simple Network Management Protocol
• Cisco NX-OS supports Simple Network Management Protocol (SNMP) to
remotely manage the Nexus devices.
• SNMP works at the application layer and facilitates the exchange of management
information between network management tools and the network devices, such as
switches and routers.
• Network administrators use SNMP to remotely manage network performance,
troubleshoot faults, and plan for network growth. There are three components of
the SNMP framework:
■ SNMP manager: A network management system (NMS) that uses the SNMP protocol to
manage the network devices.
■ SNMP agent: A software component that resides on the device that is managed. To
enable the SNMP agent, you must define the relationship between
manager and agent.
■ MIB: The collection of managed objects on the SNMP agent.
43
SNMP Notifications (contd..)
• In case of a network fault or other important event, the SNMP agent can generate
notifications without a polling request from the SNMP manager. NX-OS can
generate the following two types of SNMP notifications:

■ Traps: An unacknowledged message sent from the agent to the SNMP

managers listed in the host receiver table. These messages are less
reliable because there is no way for an agent to discover whether the
SNMP manager has received the message.
■ Informs: A message sent from the SNMP agent to the SNMP manager, for
which the manager must acknowledge the receipt of the message.
These messages are more reliable because of the acknowledgement
of the message from the SNMP manager. If the Cisco Nexus device
never receives a response for a message, it can send the inform
request again.
44
NX-OS SNMP Security Models and Levels

45
Remote Monitoring
• Remote monitoring (RMON) is an industry standard remote network
monitoring specification, developed by the IETF. RMON allows
various network console systems and agents to exchange network-
monitoring data.

• The Cisco NX-OS supports RMON alarms, events, and logs to

monitor a Cisco Nexus device. In a Cisco Nexus switch, RMON is
disabled by default, and no events or alarms are configured. You can
enable RMON and configure your alarms and events by using the CLI
or an SNMP based network management station.

46
Syslog
• Syslog is a standard protocol defined in IETF RFC 5424 for logging system
messages to a remote server.
• Nexus switches support logging of system messages. This log can be sent to the
terminal sessions, a log file, and syslog servers on remote systems.
• Nexus switch can be configured to send syslog messages to a maximum of eight
syslog servers.
System Message Severity Levels

47
Role-Based Access Control

• To manage a Nexus switch, users log in to the switch and perform tasks as per
their role in the organization. For example, a network operations center (NOC)
engineer might want to check the health of the network, but he is not permitted to
make changes to the switch configuration.

• Role-based access control (RBAC) provides a mechanism to define the amount of

access that each user has when the user logs in to the switch.

48
User Roles
• User roles contain rules that define the operations allowed on the
switch. These roles are assigned to users so they have limited access to
switch operations as per their role.
• Each user can have multiple roles, and each role can contain multiple
rules. You can also define roles to limit access to specific virtual
storage area networks (VSANs), VLANs, and interfaces of a Nexus
switch. Following are default Roles on Nexus 7000:

49
Rules
• A rule is the basic element of a role that defines the switch operations that are
permitted or denied by a user role. You can apply rules for the following
parameters:

■ Command: NX-OS command or group of commands defined in a regular

expression.
■ Feature: All the commands associated with a switch feature. You can
check the available feature names for this command using the
show role feature command.
■ Feature group: Default or user-defined group of features. Enter the show role
feature group command to display the default feature groups
available for this parameter.

50
Chapter 3. Unified Fabric Overview

51
Unified Fabric Overview

• Unified Fabric is a holistic network architecture that uniquely integrates

networking, storage, security, compute elements, and orchestration elements, all
designed to seamlessly operate in physical, virtual, and cloud environments.

• By unifying and consolidating infrastructure components, Cisco Unified Fabric

delivers flexibility and consistent architecture across a diverse set of
environments.

52
Multilayer and Spine-Leaf Clos Architecture

53
Challenges of Today’s Data Center Networks
• Increased volumes of application data led to significant network and storage
traffic growth across the infrastructure, which in turn had resulted in greater
reliance on the network.
• Often times application performance is measured along and coupled with the
performance characteristics of the network.
• Different types of application traffic carry different characteristics. Client-to-
server and server-to-server transactions usually involve short and bursty-in-nature
transmissions.
• Bandwidth availability and network latency are significant influencers on the
overall application performance, but so is the ability of applications to handle
packet drops.

54
Cisco Unified Fabric Principles
• Cisco Unified Fabric is a multidimensional approach where architectures,
features, and capabilities are combined with concepts of convergence,
scalability, intelligence, high availability, security, and so on.

• It enables optimized resource utilization, faster application rollout,

greater application performance, and overall lower operating costs, while
greatly increasing network business value.

• Cisco Unified Fabric creates a foundational platform for a true

multiprotocol environment on a single unified network infrastructure,
which breaks organizational silos while reducing technological
complexity.
55
Convergence of Network and Storage
• Convergence properties of the Cisco Unified Fabric architecture are the melding
of the SAN with a local-area network (LAN) delivered on top of enhanced
Ethernet fabric.

• End-to-end converged Unified Fabric provides the utmost advantages to the

organizations. However, at the same time, it allows the flexibility of integrating
into existing non-converged infrastructure, providing investment protection for the
current network equipment and technologies. Both the Cisco MDS 9000 family of
storage switches and the Cisco Nexus 5000/6000/7000 family of Unified Fabric
network switches have features that facilitate network convergence.

56
Convergence of Network and Storage (contd..)
Following figure depicts bridging between FCoE and traditional Fibre Channel
fabric, where FCoE servers connect to Fibre Channel storage arrays leveraging the
principles behind bidirectional bridging on the Cisco Nexus 5000/6000 series
switches:

57
Convergence of Network and Storage (contd..)
• Ultimately, SAN and LAN convergence does not
have to happen overnight, and Cisco Unified Fabric
architecture is flexible to allow gradual, and many
times non-disruptive, migration from disparate
isolated environments to unified network
infrastructure.

• Traditionally, server network interface cards (NICs)

and HBAs are used to provide connectivity into
isolated LAN and SAN environments. With
convergence of the two, NICs and HBAs make way
for a new type of adapter called converged network
adapter (CNA). Following figure depicts elements of
CNA.
58
Thank You

59