Advanced Data Center Virtualization: BRKDCT-3831

BRKDCT-3831
14488_04_2008_c2 © 2008 Cisco Systems, Inc. All rights reserved. Cisco Public 1
Advanced Data Center

Virtualization
BRKDCT-3831
BRKDCT-3831
© 2006, Cisco Systems, Inc. All rights reserved. 1

Presentation_ID.scr
Before We Get Started
Intermediate level session focused on data center

virtualization technologies and solutions, including both
front-end and back-end networks as well as server
virtualization
Prerequisites: being familiar with the basic LAN and
SAN design models as well as server virtualization
technologies
Other recommended sessions
BRKDCT-2866: Data Center Architecture Strategy and Planning
BRKDCT-2840: Data Center Networking: Taking Risk Away
from Layer 2 Interconnects
BRKDCT-1898: FCoE: The First 30 Feet of FC
BRKDCT-3831
Agenda
Data Center Virtualization Front-End Virtualization
Overview VLAN VRF VDC VSS VPNs

Front-End
Front-End Data Center

Virtualization Virtual Network Services
Core Layer Virtual
Virtual
Virtual Virtual Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Server Load Balancing
Security Services
Access Layer
Back-End
Virtual SANs/Unified IO
Server Virtualization
Back-End Virtualization VSANs vHBA CNA FCoE
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
End-to-End Management
VFrame Data Center
BRKDCT-3831

Presentation_ID.scr
Virtualization—Definition
(Well, One of Them)
Virtualization
Is the Pooling and Abstraction of
Resources and Services in a Way
That Masks the Physical Nature and
Boundaries of Those Resources and
Services from Their Users
BRKDCT-3831
What Is Network Virtualization?
Virtualization: One to many

One network supports many virtual networks
Data Center Front-End Network/LAN

BRKDCT-3831

Presentation_ID.scr
Virtualization: One to many

One network supports many virtual networks
Outsourced Merged New Segregated Department
IT Department Company (Regulatory Compliance)
Data Center Front-End Network/LAN

BRKDCT-3831

Virtualization: Many to one
One network consolidates many physical networks
Security Network
Guest/Partner Network
Backup Network
Out-of-Band Management Network
Data Center Network

BRKDCT-3831

Presentation_ID.scr
Virtualization: Many to 1
One network consolidates many physical networks
Data Center Network

BRKDCT-3831
“Network Virtualization” in the Data Center

One Term, Many Contexts
Virtual connectivity services Consolidated

Service Modules
Data Center
IP/MPLS, L3 VPN, VRFs
L2 VPNs, VFIs, PW
Virtualized front-end
Network
VLANs, PVLANs, VRF lite, VDC Front-
End
Virtual intelligent services
(Firewall, SLB, SSL, L4–7, etc.)
Compute virtualization
Clustering, GRID, virtualization
Service Modules
software (hypervisor-based) Servers Storage

Area
Virtualized storage Network
Virtual HBAs, CNAs
Virtual SANs (VSANs) Storage
Network-hosted storage
virtualization software
BRKDCT-3831

Presentation_ID.scr
Virtualized Data Center Infrastructure
DC Core
Gigabit Ethernet
Nexus 7000 WAN 10 Gigabit Ethernet
10GbE Core 10 Gigabit DCE
IP+MPLS WAN 4/8Gb Fiber Channel
Agg Router 10 Gigabit FCoE/DCE
DC Aggregation
Nexus 7000 SAN A/B
Cisco Catalyst 6500 10GbE Agg MDS 9500
10GbE VSS Agg Cisco Catalyst Storage Core
DC Services 6500
DC Services
DC Access
FC
Cisco Cisco Catalyst CBS 3100 Nexus 7000 Nexus 5000 CBS 3100 MDS 9500
Catalyst 6500 49xx Blade End-of-Row Rack MDS 9124e Storage
Rack Blade
End-of-Row
1GbE Server Access 10GbE and
10GbE and4/8Gb
4Gb FC Server
ServerAccess
Access Storage
10Gb FCoE Server Access
BRKDCT-3831
Agenda

Front-End

Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831

Presentation_ID.scr
VRF Overview
What Is a VRF (Virtual Routing and Forwarding)?
Typically all route processes

and static routes are Global Routing Table
populating one routing table
All interfaces are part of the
global routing table
router eigrp 1
network 10.1.1.0 0.0.0.255
!
router ospf 1
network 10.2.1.0 0.0.0.255 area 0
!
router bgp 65000
neighbor 192.168.1.1 remote-as 65000
!
ip route 0.0.0.0 0.0.0.0 140.75.138.114
BRKDCT-3831
VRF Overview
What Is a VRF (Virtual Routing and Forwarding)?
VRFs allow dividing up your routing

table into multiple virtual tables Global Routing Table
Routing protocol extensions allow
binding a process/address family to
a VRF
Interfaces are bound to a VRF using
ip vrf forwarding <vrf-name>
router eigrp 1
network 10.1.1.0 0.0.0.255
!
router ospf 1 vrf orange
network 10.2.1.0 0.0.0.255 area 0
!
router bgp 65000
address-family ipv4 vrf blue
…
!
ip route vrf green 0.0.0.0 0.0.0.0 …
BRKDCT-3831

Presentation_ID.scr
VRF Overview
Route Targets
VRF Export 3:3 Export 3:3 VRF

Import 3:3 Import 3:3
Export 2:2 Export 2:2
VRF Import 1:1 Export 3:3 Import 1:1 VRF
Import 3:3
Import 2:2
Export 1:1
VRF VRF
Red: Any-to-Any
Blue: Hub-and-Spoke
Import/export routes to/from MP-BGP updates

Globally significant—creates the VPN
Allows hub and spoke connectivity (central services)
BRKDCT-3831
Shared Services Extranet VPN

Multiple-Box Extranet Implementation
Export 3:3 VRF Export 3:3

VRF
Export 2:2 Export 2:2
VRF VRF
VRF Import 3:3

Import 2:2
Export 1:1
Shared
Bidirectional Communication Services
Between All VRFs and
Central Services VRF
Central services routes imported No routes exchanged between
into both VRF red and blue (1:1) blue/red
Central VRF imports routes for No transitivity: imported routes
blue and red subnets (3:3, 2:2) are not “reexported”
Æ Blue and red remain isolated
BRKDCT-3831

Presentation_ID.scr
Data Center as a Shared Service
on an Extranet VRF
Red VPN Blue VPN Internet Module
Blue VPN
Red VPN
WAN/Branch ISP1
Virtualized
Campus/MAN
DNS,
CAC
MAN
ISP2
DC
Core
Blue VRF
Red VRF
Shared Services
L3 interface Without
VRF-Enabled
.1Q with VRF-enabled VLANs
L3 Interface with VRF-Enabled
BRKDCT-3831
Agenda

Front-End

Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831

Presentation_ID.scr
Virtual Device Contexts at Nexus 7000
VDC Architecture
Virtual Device Contexts Provides Virtualization at the Device Level Allowing
Multiple Instances of the Device to Operate on the Same Physical Switch at
the Same Time
L2 Protocols L3 Protocols L2 Protocols L3 Protocols
VLAN Mgr UDLD OSPF GLBP VLAN Mgr UDLD OSPF GLBP
VLAN Mgr UDLD BGP HSRP VLAN Mgr UDLD BGP HSRP
LACP CTS EIGRP VRRP LACP CTS EIGRP VRRP

…
IGMP 802.1x PIM SNMP IGMP 802.1x PIM SNMP
RIB RIB RIB RIB
Protocol Stack (IPv4/IPv6/L2) Protocol Stack (IPv4/IPv6/L2)

VDC1 VDCn
Infrastructure
Kernel
Nexus 7000 Physical Switch
BRKDCT-3831
Virtual Device Contexts

Properties of the VDC
The Hardware Is Shared Across the VDCs but from the User,
Configuration and Management Perspective, the VDC Should Appear
as a Standalone Device
Each VDC treated as standalone device with limited resources
Each VDC uniquely identified by ID or name
Each VDC has unique MAC address assigned to identify VDC
Shared processor, shared linecards, and dedicated interfaces
Per VDC role-based management allows per VDC admin
configuration and management
Software fault isolation for protocol processes within the VDC
BRKDCT-3831

Presentation_ID.scr
VDC Fault Domain
A VDC Builds a Fault Domain Around All Running Processes Within That
VDC—Should a Fault Occur in a Running Process, It Is Truly Isolated from
Other Running Processes and They Will Not Be Impacted
VDC A VDC B Fault Domain
Process ABC
Process ABC
Process DEF
Process DEF
Process XYZ
Process XYZ Process “DEF” in
… … VDC B Crashes
Process DEF in VDC

A Is Not Affected and
Protocol Stack Protocol Stack Will Continue to Run
VDCA VDCB Unimpeded
Infrastructure
Kernel
Physical Switch
BRKDCT-3831

VDC Configuration
A VDC Is Created in the Following Manner—This Example Creates a VDC Called CiscoLive 2008
switch# conf t
switch(config)# vdc CiscoLive2008
switch(config-vdc)# show vdc
vdc_id vdc_name state mac

------ -------- ----- ----------
1 switch active 00:18:ba:d8:4c:3d
2 CiscoLive2008 active 00:18:ba:d8:4c:3e
switch(config-vdc)# show vdc detail

vdc id: 1
vdc name: switch
vdc state: active
vdc mac address: 00:18:ba:d8:4c:3d
vdc ha policy: RESET
vdc id: 2
vdc name: CiscoLive2008
vdc state: active
vdc mac address: 00:18:ba:d8:4c:3e
vdc ha policy: BRINGDOWN
BRKDCT-3831

Presentation_ID.scr
VDC Resource Assignment
The Default Resource Allocation Can Be Changed from the CLI—An Example Follows…
switch(config)# vdc CiscoLive2008

switch(config-vdc)# limit-resource vlan minimum 32 maximum 4094
switch(config-vdc)# show run | begin vdc
<snip>
vdc CiscoLive2008 id 2
template default
hap bringdown
limit-resource vlan minimum 32 maximum 4094
limit-resource span-ssn minimum 0 maximum 2
limit-resource vrf minimum 16 maximum 8192
limit-resource port-channel minimum 0 maximum 256
limit-resource glbp_group minimum 0 maximum 4096
<snip>
This Example Shows How the Minimum Number of VLANs Allocated to the CiscoLive 2008
VDC Is Changed from 16 to 32…
BRKDCT-3831

Resource Templates
Resource Templates Are Another Option for Assigning a Resource Allocation to Each VDC—
An Example of This Is Shown Below…
switch(config)# vdc resource template N7Kswitch

switch(config-vdc-template)# limit-resource vlan minimum 32 maximum 256
switch(config-vdc-template)# limit-resource vrf minimum 32 maximum 64
switch(config-vdc-template)# exit
switch(config)# vdc CiscoLive2008 template N7Kswitch
switch(config-vdc)# show vdc resource template
template ::N7Kswitch
--------
Resource Min Max
---------- ----- -----
vrf 32 64
vlan 32 256
template ::default
--------
Resource Min Max
---------- ----- -----
glbp_group 0 4096
port-channel 0 256
span-ssn 0 2
vlan 16 4094
vrf 16 8192
switch(config-vdc)#
BRKDCT-3831

Presentation_ID.scr
VDC and Interface Allocation
Ports Are Assigned on a per VDC

VDC VDC
Basis and Cannot Be Shared
A Across VDCs C
32-Port
10GE
Module
Once a Port Has Been Assigned to a

VDC VDC
VDC, All Subsequent Configuration Is
B Done from Within That VDC… C
BRKDCT-3831

VDC Resource Utilization (Layer 2)
Layer 2 Learning with Multiple Active VDCs Also Has an Impact on Resource
Utilization—MAC Addresses Learnt in a VDC Are Only Propagated to Other
Linecards When That Linecard Has a Port in That VDC
Switch Fabric
X
Linecard 1 Linecard 2 Linecard 3

MAC Table MAC Table MAC Table
MAC “A” MAC “A”
1/1 1/2 1/3 1/4 2/1 2/2 2/3 2/4 3/1 3/2 3/3 3/4
VDC
VDC
VDC
VDC
VDC
VDC
VDC
20
10
30
20
30
10
20
MAC Address A MAC “A” Is Propagated to Linecard 2 and 3 but Only

Linecard 2 Installs MAC Due to Local Port Being In VDC 10
BRKDCT-3831

Presentation_ID.scr
When Only the Default VDC Is Active, the FIB and ACL TCAM on Each
Linecard Is Primed with Forwarding Prefixes and Policies Associated with
That Default VDC as Shown Below
Linecard 1 Linecard 2 Linecard 3 Linecard 4 Linecard 5 Linecard 6 Linecard 7 Linecard 8

FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM
128K 128K 128K 128K 128K 128K 128K 128K
ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM
64K 64K 64K 64K 64K 64K 64K 64K
BRKDCT-3831

When Physical Port Resources Are Split Between Multiple VDCs, Then Only
Linecards That Have Ports Associated with a Given VDC Have Local TCAMs
Primed with FIB and Policy Information
Let’s See How This Setup Impacts TCAM Resource Allocation on the Same
Chassis Assuming the Following Breakup Shown Below
VDC Number Number of Routes Number of ACEs Allocated Linecards
10 100K 50K Linecard 1 and 2
20 10K 10K Linecard 1, 2, 3, 5
30 90K 40K Linecard 3 and 5
BRKDCT-3831

Presentation_ID.scr
FIB and ACL TCAM
VDC 10 VDC 20 VDC 30 Resources Are More
Effectively Utilized
Linecard 1 Linecard 2 Linecard 3 Linecard 4 Linecard 5 Linecard 6 Linecard 7 Linecard 8

FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM FIB TCAM
128K 128K 128K 128K 128K 128K 128K 128K
ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM ACL TCAM
64K 64K 64K 64K 64K 64K 64K 64K
BRKDCT-3831
Agenda

Front-End

Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831

Presentation_ID.scr
Common Data Center challenges
Traditional Data Center Designs Are Requiring Ever Increasing Layer 2
Adjacencies Between Server Nodes Due to Prevalence of Virtualization
Technology. However, They Are Pushing the Limits of Layer 2 Networks,
Placing More Burden on Loop-Detection Protocols Such as Spanning Tree…
FHRP, HSRP, VRRP

Spanning Tree
Policy Management
L2/L3 Core
Single Active Uplink

per VLAN (PVST), L2
Reconvergence, L2
Excessive BPDUs Distribution
Dual-Homed Servers
to Single Switch,
Single Active Uplink
per VLAN (PVST), L2
Reconvergence
L2 Access
BRKDCT-3831
Virtual Switch System at Data Center

A Virtual Switch-Enabled Data Center Allows for Maximum Scalability so
Bandwidth Can Be Added When Required, but Still Providing a Larger
Layer 2 Hierarchical Architecture Free of Reliance on Spanning Tree…
Single Router Node,

Fast L2 Convergence,
Scalable Architecture
L2/L3 Core
Dual Active Uplinks,

Fast L2 Convergence,
Minimized L2 Control L2
Plane, Scalable Distribution
Dual-Homed
Servers, Single
Active Uplink per
VLAN (PVST), Fast L2 Access
L2 Convergence
BRKDCT-3831

Presentation_ID.scr
Introduction to Virtual Switch
Concepts
Virtual Switch System Is a New Technology Break Through for the
Cisco Catalyst 6500 Family
BRKDCT-3831
Virtual Switch Architecture

Forwarding Operation
In Virtual Switch Mode, While Only One Control Plane Is Active,
Both Data Planes (Switch Fabrics) Are Active, and as Such, Each
Can Actively Participate in the Forwarding of Data
Switch 1—Control Plane Active Switch 2—Control Plane Hot Standby
Virtual Switch Domain
Switch 1—Data Plane Active Switch 2—Data Plane Active
Virtual Switch Domain
BRKDCT-3831

Presentation_ID.scr
Virtual Switch Architecture
Virtual Switch Link
The Virtual Switch Link Is a Special Link Joining Each Physical Switch Together—It Extends
the Out of Band Channel Allowing the Active Control Plane to Manage the Hardware in the
Second Chassis
The Distance of VSL Link Is Limited Only by the Chosen

10 Gigabit Ethernet Optics. VSLs Can Carry Regular Data
Traffic in Addition to the Control Plane Communication.
BRKDCT-3831
EtherChannel Concepts
Multichassis EtherChannel (MEC)
Prior to Virtual Switch, EtherChannels Were Restricted to Reside Within the
Same Physical Switch. In a Virtual Switch Environment, the Two Physical
Switches Form a Single Logical Network Entity—Therefore EtherChannels
Can Now Also Be Extended Across the Two Physical Chassis
Virtual Switch Virtual Switch
LACP, PAGP, or ON EtherChannel

Modes Are Supported…
Regular EtherChannel on Multichassis EtherChannel Across

Single Chassis Two VSL-Enabled Chassis
BRKDCT-3831

Presentation_ID.scr
EtherChannel Concepts
EtherChannel Hash for MEC
Deciding on Which Link of a Multichassis EtherChannel to Use in a
Virtual Switch Is Skewed in Favor Towards Local Links in the Bundle—
This Is Done to Avoid Overloading the Virtual Switch Link (VSL) with
Unnecessary Traffic Loads
Blue Traffic Destined Orange Traffic

for the Server Will Destined for the
Result in Link A1 in Server Will Result in
the MEC Link Bundle Link B2 in the MEC
Being Chosen as the Link Bundle Being
Destination Path… Chosen as the
Destination Path…
Link A1 Link B2
Server
BRKDCT-3831
MEC—Layer 3 Packet Flow

Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
U3
U1 U2 U4 U5
Po1 Po2
Switch 1 Po1 and Po2 Are Switch 2

Layer3 MECs
Port 1 Po1 Members—U1, U3 Port 2
Po2 Members—U2,U4,U5
Switch 1 Forwards an IP Packet

A Through Po1. Virtual Switch Learns B
the IP Route Through Po2.
BRKDCT-3831

Presentation_ID.scr
Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
U3
U1 U2 U4 U5
Po1 Po2
Switch 1 Switch 2
Port 1 Core1 Receives the Packet Through Port 2

U1 Based on the RBH Chosen on
Switch 1.
A Core1 Does an IP Lookup and
B
Selects the Port-Channel Po2.
BRKDCT-3831

Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
U3
U1 U2 U4 U5
Po1 Po2
Switch 1 Switch 2
Port 1 Port 2
Lookup for Po2 Selects the Member

U2 for All the RBH Values.
A B
Packet Exits via U2.
BRKDCT-3831

Presentation_ID.scr
Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
U3
U1 U2 U4 U5
Po1 Po2
Switch 1 Switch 2
Port 1 Lets SHUTDOWN the Port U2, Port 2

Turning MEC into a Regular Port-
Channel with Members U4 and U5.
A Lookup for Po2 on Core 1 Selects
B
the VSL Port-Channel as Exit Point.
BRKDCT-3831

Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
U3
U1 U2 U4 U5
Po1 Po2
Switch 1 Switch 2
Port 1 Port 2
Lookup for Po2 on Core 2 Selects

A U4 (or) U5 as Exit Point Based Upon B
the RBH Value for the Flow.
BRKDCT-3831

Presentation_ID.scr
Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
U3
U1 U2 U4 U5
Po1 Po2
Switch 1 Switch 2
Port 1 Now, “no shut” U2 and Shut Down Port 2

U1. Po2 Is a MEC Again. Traffic
Enters Core2 Through U3. Lookup
A for Po2 on Core 2 Selects U4 (or) B
U5 as Exit Point Based Upon the
RBH Value for the Flow.
BRKDCT-3831

Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
3rd
U3
U1 U2 U4
2nd
Po1 Po2
Switch 1 Switch 2
Port 1 Port 2
1st) A Transmits Packet to B.
1st 2nd) Switch 1 Forwards Packets Out
of Po1.
A 3rd) Core1 Receives the Packet.
B
Core1 Learns A Is on Port 1.
BRKDCT-3831

Presentation_ID.scr
Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
4th
U3
U1 U2 U4
Po1 Po2
Switch 1 Switch 2
Port 1 Port 2
4th) Core1 Performs Lookup on B

Core1 Floods Packet Due to Miss
A Flood Index Selects Port 2 and VSL
B
MEC LTL Index Selects U2
BRKDCT-3831

Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1
5th
U3
U1 U2 U4
Po1 Po2
Switch 1 Switch 2
Port 1 Port 2
A 5th) S2 Receives Packet from U2 B

S2 Transmits Packet Out Port2 to B
BRKDCT-3831

Presentation_ID.scr
Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1 A port1
U3
U1 U2 U4
Po1 Po2
Switch 1 Switch 2
Port 1 C2 Receives Packet from VSL Port 2

C2 Learns A Is on Port 1
C2 Performs Lookup for B
C2 Floods Due to Miss
A Flood Excludes U4 Since It Is a Multichassis B
Bundle and Packet Came from VSL
BRKDCT-3831

Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1 A port1
B port2
3rd
U3
U1 U2 U4
2nd
Po1 Po2
Switch 1 Switch 2
Port 1 Port 2
1st) B Transmits a Packet to A. 1st
2nd) Virtual Switch Receives the Packet
A Through U4. B
3rd) C2 Receives the Packet. C2 Learns B
Is on Port 2.
BRKDCT-3831

Presentation_ID.scr
Virtual Switch
Core 1 Core 2
VSL
(C1) (C2)
A port1 A port1
B port2
4th
U3
U1 U2 U4
Po1 Po2
5th
Switch 1 Switch 2
Port 1 4th) C2 Performs Lookup for A and Port 2

Selects Port1
Port1 LTL Index Selects U3
A C2 Transmits the Packet B
5th) S1 Receives the Packet and
Transmits It to A on Port 1
BRKDCT-3831
Hardware Requirements
VSL Hardware Requirements
The Virtual Switch Link Requires Special Hardware as Noted Below…
BRKDCT-3831

Presentation_ID.scr
Hardware Requirements
Other Hardware Considerations
12.2 (33) SXH
BRKDCT-3831
Virtual Switch System at Data Center

Benefits
BRKDCT-3831

Presentation_ID.scr
Agenda
Front-End
Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831
Aggregation Services Design Options

DC Core Gigabit Ethernet
Nexus 7000 WAN 10 Gigabit Ethernet
10GbE Core 10 Gigabit DCE
IP+MPLS WAN 4/8Gb Fiber Channel
Agg Router 10 Gigabit FCoE/DCE
DC Aggregation
Nexus 7000 SAN A/B
Cisco Catalyst 6500 10GbE Agg MDS 9500
10GbE VSS Agg Cisco Catalyst Storage Core
DC Services 6500
DC Services
DCEmbedded
Access Service Modules One-Arm Service Switches
FC
Cisco Cisco Catalyst CBS 3100 Nexus 7000 Nexus 5000 CBS 3100 MDS 9500
Catalyst 6500 49xx Blade End-of-Row Rack MDS 9124e Storage
Rack Blade
End-of-Row
1GbE Server Access 10GbE and
10GbE and4/8Gb
4Gb FC Server
ServerAccess
Access Storage
10Gb FCoE Server Access
BRKDCT-3831

Presentation_ID.scr
ACE Virtual Partitioning
System Separation for Server Load Balancing and SSL
Multiple Virtual Systems
One Physical Device (Dedicated Control and Data Path)
100% 25% 25% 15% 15% 20%
Traditional Device Cisco Application Infrastructure Control

Single configuration file Distinct context configuration files
Single routing table Separate routing tables
Limited RBAC RBAC with contexts,
roles, domains
Limited resource allocation
Management and data
resource control
Independent application rule sets
Global administration and
monitoring
BRKDCT-3831
ACE Virtual Partitions

Resource Control
Guaranteed resource levels for each context with support for

oversubscription
Guaranteed Guaranteed
Rates Memory
Bandwidth Access lists

Data connections/sec Regular expressions
Management connections/sec # Data connections
SSL bandwidth # Management connections
Syslogs/sec #SSL connections
# Xlates
# Sticky entries
BRKDCT-3831

Presentation_ID.scr
Firewall Service Module (FWSM)
Virtual Firewalls
Core/Internet Core/Internet
Cisco Cisco
Catalyst Catalyst
6500 MSFC 6500 MSFC
VLAN 10
VLAN 10 VLAN 20 VLAN 30
VFW VFW VFW VFW VFW VFW

FW SM FW SM
VLAN 11 VLAN 21 VLAN 31 VLAN11 VLAN 21 VLAN 31
A B C A B C
e.g., Three customers Æ three security contexts—scales up to 250

VLANs can be shared if needed (VLAN 10 on the right-hand side example)
Each context has its own policies (NAT, access-lists, fixups, etc.)
FWSM supports routed (Layer 3) or transparent (Layer 2) virtual firewalls at the
same time
BRKDCT-3831
FWSM—Virtual Firewall Resource Limiter

In system mode, classes can be defined
Individual contexts are then mapped to classes
Within a class, limits can be applied to specific resources such as:
(use “show resource types” for up-to-date list)
Conns CPS Conns Connections Xlates

Fixups Fixups/sec Hosts Hosts MAC-entries
Syslogs Syslogs/sec IPSec IPSec Mgmt Tunnels ALL
SSH SSH Sessions
Rate Limited Telnet Telnet Sessions
Absolute Limits
Limits specified as integer or %; 0 means no limit

Resources can be oversubscribed: e.g., class assigns max 10% of
resources, but 50 contexts are mapped to it
BRKDCT-3831

Presentation_ID.scr
Data Center Virtualized Services
Combination Example
VRF VRF VRF VRF

“Front-End” VRFs (MSFC)
v5 v6 v7 v8
1 3 4 Firewall Module Contexts
v107 v108
v105
2 3 4
ACE Module Contexts
v206 v207 v208
VRF
“Back-End” VRFs (MSFC)
BU-1 BU-2 BU-3 BU-4

v2081
v2082 Server Side VLANs
v105 v206 v207 v2083
...
* vX = VLAN X
**BU = Business Unit
BRKDCT-3831
Virtualized Services
Example: Modules and VLANs Association
cse-6509a# show module 7
Mod Ports Card Type Model Serial No. ACE/Admin# show vlans
--- ----- -------------------------------------- ------------------ ----------- Vlans configured on SUP for this module
7 6 Firewall Module WS-SVC-FWM-1 SAD0930052K vlan1301-1310 vlan1401-1410
Mod MAC addresses Hw Fw Sw Status ACE/Admin#
--- ---------------------------------- ------ ------------ ------------ -------
7 0014.a90c.987a to 0014.a90c.9881 3.0 7.2(1) 3.2(0)67 Ok
Mod Online Diag Status FWSM# show vlan
---- ------------------- 1201-1210, 1301-1310
7 Pass FWSM#
cse-6509a#
cse-6509a# show module 4
Mod Ports Card Type Model Serial No.
--- ----- -------------------------------------- ------------------ -----------
4 1 Application Control Engine Module ACE10-6500-K9 MSFC
SAD102905V2
Mod MAC addresses Hw Fw Sw Status
--- ---------------------------------- ------ ------------ ------------ -------
4 000a.b870.e43a to 000a.b870.e441 1.1 8.6(0.252-En 3.0(0)A1(4a) Ok
Mod Online Diag Status
---- -------------------
4 Pass vlan-group1
cse-6509a#
svclc multiple-vlan-interfaces
firewall multiple-vlan-interfaces FWSM
svclc vlan-group 1 1201-1210 vlan-group2

svclc vlan-group 2 1301-1310
svclc vlan-group 3 1401-1410 ACE
firewall module 7 vlan-group 1,2
svclc module 4 vlan-group 2,3
vlan-group3
BRKDCT-3831

Presentation_ID.scr
Example: Modules and VLANs Association (Cont.)
FWSM# FWSM/admin# show run | i Vlan
admin-context admin interface Vlan1210
! interface Vlan1310
svclc multiple-vlan-interfaces context admin
firewall multiple-vlan-interfaces allocate-interface Vlan1210 FWSM/INTERNET# show run | i Vlan
allocate-interface Vlan1310 interface Vlan1201
config-url disk:/admin.cfg interface Vlan1301
svclc vlan-group 1 1201-1210 ! interface Vlan1302
svclc vlan-group 2 1301-1310 context INTERNET
svclc vlan-group 3 1401-1410 allocate-interface Vlan1201 FWSM/INTRANET# show run | i Vlan
firewall module 7 vlan-group 1,2 config-url disk:/INTERNET.cfg
svclc module 4 vlan-group 2,3 !
context INTRANET
allocate-interface Vlan1205
allocate-interface Vlan1305
config-url disk:/INTRANET.cfg
ACE/Admin# ACE/INTERNET1# show run | i vlan

Generating configuration....
context INTERNET1 interface vlan 1301
description *** INTERNET (WEB TIER) interface vlan 1401
allocate-interface vlan 1301
allocate-interface vlan 1401 ACE/INTERNET2# show run | i vlan
! Generating configuration....
context INTERNET2 interface vlan 1302
description *** INTERNET (APPLICATION TIER) interface vlan 1402
allocate-interface vlan 1402 ACE/INTRANET# show run | i vlan
! Generating configuration....
context INTRANET interface vlan 1305
description *** INTRANET interface vlan 1405
BRKDCT-3831
Cisco ACE and FWSM Virtualized
Online Bank
Application
App Has
(SSL Offloading Capacity Microsoft
Required) Available Outlook
Ideal
Isolation
Cisco ACE
and
Cisco FWSM
Virtual Machines Virtual Machines
Bank Micro Bank Micro

Oracle Oracle
Apps soft Apps soft
ESX Server
BRKDCT-3831

Presentation_ID.scr
Agenda
Front-End
Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831
Increasing HA in the Data Center

Common NIC Teaming Configurations
AFT—Adapter Fault Tolerance SFT—Switch Fault Tolerance ALB—Adaptive Load Balancing
Default GW Default GW Default GW

10.2.1.1 10.2.1.1 10.2.1.1
HSRP HSRP HSRP
Heartbeats
Heartbeats
Heartbeats
Eth0: Active Eth1: Standby Eth0: Active Eth1: Standby Eth0: Active Eth1-X: Active
IP=10.2.1.14 IP=10.2.1.14
IP=10.2.1.14 IP=10.2.1.14
MAC =0007.e910.ce0f MAC =0007.e910.ce0e
MAC =0007.e910.ce0f MAC =0007.e910.ce0f
One Port Receives, All Ports Transmit
On Failover, Src MAC Eth1 = Src MAC Eth0 On Failover, Src MAC Eth1 = Src MAC Eth0
Incorporates Fault Tolerance
IP Address Eth1 = IP Address Eth0 IP Address Eth1 = IP Address Eth0
One IP Address and Multiple MAC Addresses
Note: NIC manufacturer drivers are changing and may operate differently. Also, server OS
have started integrating NIC teaming drivers which may operate differently.
Note: You can bundle multiple links to allow generating higher throughputs between servers
and clients.
BRKDCT-3831

Presentation_ID.scr
Virtual Switch System
Deployment Scenario at Data Center Access Layer
BRKDCT-3831
Enhanced Ethernet: PFC and DCBCXP

Priority Data Center Bridging
Flow Control Capability eXchange Protocol
Transmit Queues Receive Buffers
Ethernet Link
One One
Two Two Nexus
Nuova
5000
Switch
Three Three
Four Four Eight
Virtual
Five Five Lanes
Six STOP PAUSE Six
Seven Seven
Eight Eight
Handshaking Negotiation for:

Enables lossless fabrics CoS BW management
for each class of service Priority Flow Control (PFC)
PAUSE sent per virtual lane Congestion management (BCN/QCN)
when buffers limit exceeded Application (user_priority usage)
Logical link down
BRKDCT-3831

Presentation_ID.scr
Nexus 5000 Ethernet Host Virtualizer
Eliminates need for spanning Ethernet Host Virtualizer
tree protocol on uplink bridge
ports
LAN
Reduces CPU load on upstream
switches
Allows multiple active uplinks MAC

A
MAC
B
from nexus 5000 switch to Active-Active
network
Nexus
Doubles effective bandwidth 5000
vs. STP MAC MAC

A B
Prevents loops by pinning a
MAC address to only one port
Completely transparent to next
hop switch
BRKDCT-3831
Pinning
Border interface
Server interface
BRKDCT-3831

Presentation_ID.scr
Agenda
Front-End
Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831
Server Virtualization Scenarios

App App
Hardware-based virtualization Guest Guest
OS OS
Software-based virtualization Virtualization
Software
Hosted (application virtualization) Host Operating System
Hypervisor X86 Hardware
Full virtualization (binary translation)

Para-virtualization (OS assisted)
Hardware-assisted virtualization (Intel VT-x/AMD-V)
App App
Guest Guest Mgmt

OS OS Partition
Hypervisor
X86 Hardware
BRKDCT-3831

Presentation_ID.scr
Software-Based Virtualization
(Examples)
Hypervisor Hypervisor
Full Virtualization Para-Virtualization Application Virtualization
Examples Examples Examples

VMware ESX server Xen (with traditional VMware server
hardware)
Microsoft HyperV VMware workstation
Oracle VM server
Xen (with AMD-SVM
or Intel VM-T)
Virtuallron
(hardware-assisted)
BRKDCT-3831
VMware ESX Architecture in a Nutshell
Mgmt
Network Production
Network
VM Kernel
App. App. App. Network
Console
OS
OS OS OS
Virtual
Machines
VM Virtualization Layer
Physical Hardware
y
CPU or …
em
M
ESX Server Host
BRKDCT-3831

Presentation_ID.scr
VMware Networking Components
Per ESX Server Configuration VMs vSwitch
VMNICS =
Uplinks
vNIC vSwitch0
VM_LUN_0007
vmnic0
VM_LUN_0005
vNIC
vmnic1
Virtual Ports
BRKDCT-3831
VMware Networking Components (Cont.)
BRKDCT-3831

Presentation_ID.scr
vSwitch Overview Software
implementation of
ESX an Ethernet switch
Service Server
VM1 VM2 Console How is it like a
switch:
-MAC addr forwarding
Virtual NIC’s VLAN segmentation
How is it different:
-No need to learn
VMkernel
NIC VSwitch A
X VSwitch B
MAC addresses – it
knows the address of
No Trunk the connecting vNIC’s
Btwn vSwitch VMkernel -No participation in
spanning tree
No Loop
X X Physical NIC’s
No Loop
In ESX Physical
Without a bridging VM Switches
BRKDCT-3831
vSwitch Forwarding Characteristics
Forwarding based on MAC address (no learning): If

traffic doesn’t match a VM MAC is sent out to vmnic
VM-to-VM traffic stays local
Vswitches TAG traffic with 802.1q VLAN ID
vSwitches are 802.1q-capable
vSwitches can create EtherChannels
BRKDCT-3831

Presentation_ID.scr
VMware Best Practices:
VST is Preferred
BRKDCT-3831
Meaning of NIC Teaming in VMware
ESX Server NIC Cards
vSwitch Uplinks
vmnic0 vmnic1 vmnic2 vmnic3
NIC Teaming
NIC Teaming
This Is Not NIC Teaming

vNIC vNIC vNIC
vNIC
vNIC
ESX Server Host
BRKDCT-3831

Presentation_ID.scr
Meaning of NIC Teaming in VMware (2)
Teaming is Configured at
The vmnic Level
This is NOT Teaming
BRKDCT-3831
Agenda

Front-End

Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831

Presentation_ID.scr
Virtual Storage Area Network
Deployment
Department A
Consolidation of SAN islands
Increased utilization of fabric ports with SAN Islands
just-in-time provisioning
Deployment of large fabrics
Dividing a large fabric in smaller VSANs
Disruptive events isolated per VSAN
RBAC for administrative tasks
Zoning is independent per VSAN
Department B Department C
Advanced traffic management
Defining the paths for each VSAN
Virtual SANs
VSANs may share the same EISL (VSANs)
Cost effective on WAN links
Department A
Resilient SAN extension Department B
Standard solution Department C

(ANSI T11 FC-FS-2 section 10)
BRKDCT-3831
VSAN Advantages for Consolidation

SAN Islands Consolidated SANs
Backup OLTP Backup VSAN
Overlay Isolated Virtual

E-Mail Fabrics (VSANs) on Same
Physical Infrastructure
OLTP VSAN
E-Mail VSAN
Attribute
More Number of SAN Switches Fewer
No Share Disk/Tape Yes
No Share DR Facilities Yes
Complex SAN Management Simple
Support Virtualization
Very hard Easy
and Mobility
BRKDCT-3831

Presentation_ID.scr
VSAN Technology
The Virtual SANs Feature
Consists of Two Primary
Functions Fibre Channel
Services for
Blue VSAN
VSAN Header Is
Hardware-based isolation of Removed at Fibre Channel
Services for
Egress Point
tagged traffic belonging to Red VSAN
Cisco MDS 9000
different VSANs Family with VSAN Trunking
Service E_Port
Create independent instance (TE_Port)
of fiber channel services for Enhanced ISL (EISL)

Trunk Carries
each newly created VSAN— Tagged Traffic from
Multiple VSANs
services include: Trunking
VSAN Header Is E_Port
Added at Ingress (TE_Port)
Point Indicating Fibre Channel
Membership Services for
Blue VSAN
No Special Fibre Channel
Support Required Services for
by End Nodes Red VSAN
BRKDCT-3831
Inter VSAN Routing

Similar to L3 interconnection
between VLAN Engineering
VSAN_1
VSAN-Specific
Disk
Allows sharing of centralized

storage services such as tape
libraries and disks across IVR
VSANs—without merging
separate fabrics (VSANs)
Network address translation
allow interconnection of IVR
VSANs without a predefined Tape
VSAN_4
addressing schema (Access
Marketing via IVR)
VSAN_2
HR
VSAN_3
BRKDCT-3831

Presentation_ID.scr
Agenda
Front-End
Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831
N-Port ID Virtualization (NPIV)

Mechanism to assign multiple N_Port_IDs to a single N_Port
Allows all the access control, zoning, port security (PSM) be
implemented on application level
Multiple N_Port_IDs are allocated in the same VSAN
Application Server FC Switch
E-Mail Email I/O

N_Port_ID 1
Web Web I/O F_Port

N_Port_ID 2
File Services File Services I/O

N_Port_ID 3
BRKDCT-3831

Presentation_ID.scr
NPIV Usage Examples
Virtual Machine Aggregation ‘Intelligent Pass-Thru’
FC FC FC FC
FC FC FC FC
NPV Edge
Switch
FC
NP_Port
NPIV-Enabled HBA
F_Port F_Port
BRKDCT-3831
NPIV Configuration Example

NPIV Is Enabled Switchwide
with the Command:
npiv enable
Notice that a F-port supports
multiple logins
BRKDCT-3831

Presentation_ID.scr
Virtual Servers Share a Physical HBA
A zone includes the physical HBA
and the storage array
Access control is demanded to storage
Servers
Virtual
array “LUN masking and mapping”, it is

based on the physical HBA pWWN and
it is the same for all VMs
The hypervisor is in charge of the
mapping, errors may be disastrous
Storage Array
Hypervisor
MDS9000 (LUN Mapping and Masking)
Mapping
FC
HW
pWWN-P FC
pWWN-P
Single Login on a Single Point-to-Point Connection FC Name Server

Zone
BRKDCT-3831
Virtual Server Using NPIV and

Storage Device Mapping
Virtual HBAs can be zoned individually
“LUN masking and mapping” is based on
the virtual HBA pWWN of each VMs
Servers
Virtual
Very safe with respect to

configuration errors
Only supports RDM
Available in ESX 3.5
MDS9000 Storage Array

Hypervisor
Mapping Mapping Mapping Mapping FC
FC FC FC FC
To pWWN-1
pWWN-1 pWWN-2 pWWN-3 pWWN-4 To pWWN-2
pWWN-P To pWWN-3
HW
pWWN-1
pWWN-P FC pWWN-2 To pWWN-4
pWWN-3
pWWN-4
Multiple Logins on a Single Point-to-Point Connection FC Name Server
BRKDCT-3831

Presentation_ID.scr
N-Port Virtualization (NPV): An Overview
NPV-Core Switch (MDS or Third-Party Switch with NPIV Support)
FC FC
Solves the Domain-id

Explosion Problem 10.1.1 20.2.1
F-Port
VSAN 10
VS
AN
5 15
N
NP-Port V SA Can Have Multiple
Uplinks, on Different
VSANs (Port Channel Up to 100
MDS 9124 and Trunking in a NPV Switches
MDS 9134 Later Release)
Server Port (F)

Cisco MDS
in a
Blade Chassis 10.5.2 10.5.7
FC
Blade Server 1
Target
Blade Server 2 NPV Device
20.5.1 Initiator
Uses the Same Domain(s) as
the NPV-Core Switch(es) (No FL Ports)
Blade Server n
BRKDCT-3831
Domain ID Scalability:
NPV Solves the Issue
Blade Chassis SAN
Fabric
Server 1
FC Blade
…
Switch 1
…
Server 2
…
FC Blade
Switch 2 F-Port F-Port
…
Server N
NPIV-Enabled Switch
e.g., MDS Switch
N-Ports F-Ports NP Ports F-Ports
Eliminates Domain ID for MDS FC switch in blade enclosures—HBA model

Server ports automatically assigned to NP ports (load balancing algorithm)
Need to configure the same VSAN between NP ports and core F-ports
When F-trunking will be available, the limitation of single VSAN per link will
go away
BRKDCT-3831

Presentation_ID.scr
VMware Support
Nested NPIV FLOGI/FDISC Login Process
NPV-Core Switch
When NP port comes up on a NPV
edge switch, it first FLOGI and
PLOGI into the core to register into
the FC name server FCNS
pWWN1, pWWN2
pWWN3,pWWN4
End devices connected on NPV F F

edge switch does FLOGI but NPV
switch converts FLOGI to FDISC NP NP
command, creating a virtual
NPV Edge Switch
PWWN for the end device and
allowing to login using the physical F F
NP port
NPIV capable devices connected
on NPV switch will continue FDISC
login process for all virtual PWWN FC FC FC FC FC FC FC FC
FC FC
which will go through same NP
port as physical end device
BRKDCT-3831
FlexAttach
Flexibility for Adds, Moves, and Changes
Blade Server FlexAttach (based on WWN NAT)

…. Each blade switch F-Port assigned a
Blade N
Blade 1
Blade
New
virtual WWN
Blade switch performs NAT operations
FlexAttach on real WWN of attached server
No Blade NPV
Switch Config
Change Benefits
No SAN reconfiguration required when
new blade server attaches to blade
No Switch
Zoning SAN switch port
Change
Provides flexibility for server
administrator, by eliminating need for
coordinating change management with
networking team
No Array
Configuration Storage
Change Reduces downtime when replacing
failed blade servers
BRKDCT-3831

Presentation_ID.scr
FlexAttach—Since SANOS 3.2(2)
FlexAttach Point (Virtual PWWN)
Creation of virtual PWWN on NPV switch F-port
Zone vPWWN to storage
LUN masking is done on vPWWN
Can swap blade server or replace physical HBA
No need for zoning modification
No LUN masking change required
Automatic link to new PWWN
No manual relinking to new PWWN is needed
Before After
FC1/1 vPWWN1 FC1/1 vPWWN1

PWWN 1
PWWN 2
Server 1 Server 1
BRKDCT-3831
VMotion and Virtual HBAs

VM Migration with Emulex HBA
Server 1 Out Move Selected Apps,

Dynamic migration relocates of Resources FC Access to Server 2
VMs to available resources
By operator
Automatic load balancing
HA and DR A B C D E B
Enhanced VMotion in ESX 3.5 Hypervisor Hypervisor
Tear down initial virtual port NPIV HBAs NPIV HBAs
Reregisters same address on
another server
Enhanced VMotion preserves
access configuration
Zoning
LUN masking VSANs
VSAN selective routing
Fabric QoS priority level
BRKDCT-3831

Presentation_ID.scr
Validated Solution from
Cisco, Emulex, and VMware
Cisco MDS directors and

switches with NPIV
(SAN OS 3.0 and later)
Emulex 4G HBAs
VMware ESX 3.5
Jointly tested and
validated by three
companies
BRKDCT-3831
Agenda

Front-End

Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831

Presentation_ID.scr
Unified I/O (FCoE)
Fewer HBA/NICs per Server
FC HBA SAN (FC)

LAN (Ethernet)
FC HBA SAN (FC) CNA SAN (FCoE)
NIC LAN (Ethernet)
CNA
NIC LAN (Ethernet)
CNA = Converged Network Adapter

BRKDCT-3831
Fiber Channel over Ethernet:

How It Works
Direct mapping of fiber channel over Ethernet

FC-4 FC-4
CRC
EOF
SOF
FC-3 FC-3 FC Frame

FC-2 FC-2
FC-1 FCoE Mapping Ethernet Ethernet Ethernet

MAC Header Payload FCS
FC-0 PHY
(a) Protocol Layers (b) Frame Encapsulation
Leverages standards-based extensions to Ethernet to

provide reliable I/O delivery
Priority flow control
FCoE Traffic
10GE Lossless
Data Center Bridging Capability Ethernet
Link Other Networking
eXchange Protocol (DCBCXP) Traffic
BRKDCT-3831

Presentation_ID.scr
FCoE Enablers
10 Gbps Ethernet
Lossless Ethernet
Matches the lossless behavior guaranteed in FC by B2B credits
Ethernet jumbo frames

Max FC frame payload = 2112 bytes
Normal Ethernet Frame, Ethertype = FCoE

Same as a Physical FC Frame
Ethernet
Header
Header
Header
FCoE
CRC
EOF
FCS
FC
FC Payload
Control Information: Version, Ordered Sets (SOF, EOF)

BRKDCT-3831
Encapsulation Technologies
Operating System/Applications
SCSI Layer
FCP iSCSI FCP FCP FCP SRP
FCIP iFCP
TCP TCP TCP
IP IP IP FCoE
FC Ethernet IB
1, 2, 4, (8), 10 Gbps 1, 10 . . . Gbps 10, 20 Gbps
BRKDCT-3831

Presentation_ID.scr
Encapsulation Technologies
FCP layer is untouched

Allows same
OS/Applications management tools for
SCSI Layer fiber channel
FCP Allows same fiber
channel drivers
Allows same multipathing
FCoE software
E. Ethernet Simplifies certifications

1, 10 . . . Gbps Evolution rather
than revolution
BRKDCT-3831
Unified I/O Use Case

Today
LAN SAN A SAN B

Management
FCoE
Ethernet
FC
BRKDCT-3831

Presentation_ID.scr
Unified I/O Use Case
Unified I/O
Unified I/O
LAN SAN B Reduction of server adapters
SAN A
Fewer cables
Management
Simplification of access
layer and cabling
Gateway-free implementation—
fits in installed base of existing
LAN and SAN
FCoE L2 multipathing access—
Switch
distribution
Lower TCO
Investment protection
(LANs and SANs)
Consistent operational model
FCoE One set of ToR switches
Ethernet
FC
BRKDCT-3831
CNA: I/O Consolidation Adapter

10 GbE/FCoE
Off the shelf NIC and HBA
ASICs from: Qlogic, Emulex
Dual 10 GbE/FCoE ports
Support for native drivers

and utilities
Customer certified stacks
Replaces multiple adapters

per server
10 GbE FC
Consolidates 10 GbE and FC
on a single interface
Minimum disruption in existing
customer environments
PCIe Bus
Designed Multiplexer and FCoE Offload Protocol Engine

BRKDCT-3831

Presentation_ID.scr
FCoE Software Stack
FCoE Software Stack
Supported on Intel Oplin
10 GbE Adapters
Software upgraded turns
10 GbE adapter into FCoE adapter
Software
Software implementation
Initiator and target mode
FCP, FC class 3
Fully supports Ethernet pause
frames (per priority pause)
Supported OS
Linux: Red Hat and SLES
Hardware
Windows
“Free” access to the SAN

Website: www.Open-FCoE.org L2 Ethernet NIC
Announcement is: http://lkml.org/lkml/2007/11/27/227
BRKDCT-3831
CNAs: View from Operating System
Standard drivers
Same management
Operating system sees:
2 x 10 Gigabit
Ethernet adapter
2 x 4 Gbps fiber
channel HBAs
BRKDCT-3831

Presentation_ID.scr
IO Consolidation
Connecting LAN and SAN on a
Single Physical Link
virtual-ethernet interface (veth) SAN A SAN B LAN
Paired with host’s Ethernet device

Configuration point for all Fiber
Ethernet
Channel
Ethernet features Forwarding
Forwarding
virtual-fc interface (vfc)
Paired with host’s HBA device vfc veth
Configuration point for all vig mux

fiber channel features
virtual-interface-group (vig) Ethernet
Logical representation of a switch port

Consists of one veth and one vfc
Configured online or offline mux
Bound to physical switch port for host0 eth0

deployment SCSI IP
EtherChannel post FCS
BRKDCT-3831
IO Consolidation: Interface Configuration

Create virtual-interface-group and bind to physical interface
switch(config)# interface vig 20
switch(config-if)# bind Ethernet 1/1
Configure virtual-ethernet and virtual-fc
switch(config-if)# interface veth 20/1
switch(config-if)# interface vfc 20/1
veth20/1 vfc20/1 veth30/1 vfc30/1
vig20 vig30
Eth1/1 Eth1/33
BRKDCT-3831

Presentation_ID.scr
Agenda
Front-End
Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831
SAN-Based Storage (Block)

Virtualization
Production
A SCSI operation from the
host is mapped in one or
more SCSI operation to
the SAN-attached storage
This mapping function is
enable by a network resource Virtual 9 GB
Volume
Centralized management
Highly scalable Virtualization
(Volume Management)
Works across
heterogeneous arrays
Example: LUN concatenation
4 GB 5 GB
Storage Pool
BRKDCT-3831

Presentation_ID.scr
Block Level Virtualization
Is Enhanced by VSANs
Volume management functionality are provided by

the intelligent storage network
The volume management functionality
Exposes a virtual target to the host to provide storage capacity
Accesses the storage by mean of a virtual initiator
The architecture relies heavily on the VSAN underlying

infrastructure to provide the desired level of isolation
High performances are achieved by processing in
software the SCSI control path and using application
specific hardware to process the SCSI data path
BRKDCT-3831
Distributed Storage Virtualization

on VSANs
Front-End
Front-end VSANs Host-1 Host-3
VSAN10 VSAN 20
Virtual targets
Virtual volumes Virtual Target1 Virtual Target2
VSAN 10 VSAN 20
Virtual initiators Virtual Virtual
Volume1 Volume2
Back-end VSAN
Virtual Virtual
Initiator Initiator
Zoning connects real VSAN 50 VSAN 50
Fabric
initiator and virtual
target or virtual initiator
and real storage
Back-End
Storage Storage
Zones Array Array
Storage
VSAN 50
BRKDCT-3831

Presentation_ID.scr
Sample Use: Seamless Data Mobility
Works across
heterogeneous arrays
Nondisruptive to
application host
Can be utilized for
“end-of-lease” storage Virtualization
migration Mobility
Movement of data
from one tier class
to another tier
Tier1 Tier2
BRKDCT-3831
Agenda

Front-End

Core Layer Virtual
Virtual
Virtual
Virtual
Firewall
Firewall
Firewall SLB SSL
SSL
SSL
VDC Context
Context
Context
11 1
Context
29
Context
Context
Context
33175
Aggregation Layer
Virtual Machines
VSS
Security Services
Access Layer
Back-End
SAN
HBA Virtual Storage
Unified IO (FCoE)
Storage
VFrame Data Center
BRKDCT-3831

Presentation_ID.scr
Cisco VFrame Data Center:
Network-Driven Service Orchestration
Coordinated Provisioning and Reuse of
Physical and Virtualized
Compute, Storage, and Network Resources
Operational cost
savings VFrame Data Center
Faster and simpler

service orchestration
Robust virtualization
scale-out
FC FC
VM VM FC
Hypervisor
Compute Pool Network Pool Storage Pool

BRKDCT-3831
Adopting VFrame DC Today

Addressing Today’s Challenges While Building SOI Foundation
1. Categorize physical resources into service views
2. Ensure design consistency with standardized infrastructure templates
3. Automate physical provisioning for server virtualization environments
4. Reduce break-fix server support costs with rapid recovery from shared pool
5. Recover failed service with rapid local disaster recovery
6. Provide policy-based dynamic capacity on-demand for applications
Slow
Policy
Application
Performance
Server Service View
X Application
Degradation
V V V V
or Failure
X Rapidly
Network Service View
VFrame DC FC
Configure
FC FC FC
New
FC
FC V V V V Application
Hypervisor Hypervisor Environment
SAN NAS Application
Traditional
Storage silos
Service View Service 1
BRKDCT-3831

Presentation_ID.scr
Design to Operate Workflow for SOI
Logical, Structured for Ease of Use
Design
Service
Template
Firewall LANs L4–L7 Server Boot OS/ SAN

Discover I/O Application Infrastructure
Resources
Firewall Switch Port VIPs, LB Image Mgmt, Zones,

Deploy Selection, Config Policies Remote Boot, VSANs, LUNs,
Firewall VLANs, DHCP, VM Mappings NFS Volumes
Service Chaining, Trunks, SVIs
Networks Firewall Rules
Operate Automated Failover Policy-Based Resource Optimization
Policies Management Integration thru API Service Maintenance
BRKDCT-3831
Data Center Virtualization

via the Network
Service Orchestration
FC
End-to-End Service Provisioning

FC
Client Security App LAN Servers SAN Storage

Delivery
BRKDCT-3831

Presentation_ID.scr
Q and A
BRKDCT-3831
Recommended Reading
Continue your Cisco Live

learning experience with further
reading from Cisco Press
Check the Recommended
Reading flyer for suggested
books
Available Onsite at the Cisco Company Store

BRKDCT-3831

Presentation_ID.scr
Complete Your Online
Session Evaluation
Give us your feedback and you could win Don’t forget to activate
fabulous prizes. Winners announced daily. your Cisco Live virtual
account for access to
Receive 20 Passport points for each session all session material
evaluation you complete. on-demand and return
for our live virtual event
Complete your session evaluation online now in October 2008.
(open a browser through our wireless network Go to the Collaboration
to access our portal) or visit one of the Internet Zone in World of
stations throughout the Convention Center. Solutions or visit
www.cisco-live.com.
BRKDCT-3831
BRKDCT-3831

Presentation_ID.scr

Advanced Data Center Virtualization: BRKDCT-3831

Uploaded by

Copyright:

Available Formats

Advanced Data Center Virtualization: BRKDCT-3831

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Advanced Data Center Virtualization: BRKDCT-3831

Uploaded by

Copyright:

Available Formats

BRKDCT-3831

Advanced Data Center

© 2006, Cisco Systems, Inc. All rights reserved. 1

 Intermediate level session focused on data center

Overview VLAN VRF VDC VSS VPNs

 Front-End Data Center

© 2006, Cisco Systems, Inc. All rights reserved. 2

What Is Network Virtualization?

 Virtualization: One to many

Data Center Front-End Network/LAN

© 2006, Cisco Systems, Inc. All rights reserved. 3

 Virtualization: One to many

Virtual Virtual Virtual

Data Center Front-End Network/LAN

What Is Network Virtualization?

Out-of-Band Management Network

Data Center Network

© 2006, Cisco Systems, Inc. All rights reserved. 4

Data Center Network

“Network Virtualization” in the Data Center

 Virtual connectivity services Consolidated

software (hypervisor-based) Servers Storage

© 2006, Cisco Systems, Inc. All rights reserved. 5

Overview VLAN VRF VDC VSS VPNs

 Front-End Data Center

© 2006, Cisco Systems, Inc. All rights reserved. 6

 Typically all route processes

 VRFs allow dividing up your routing

© 2006, Cisco Systems, Inc. All rights reserved. 7

VRF Export 3:3 Export 3:3 VRF

 Import/export routes to/from MP-BGP updates

Shared Services Extranet VPN

Export 3:3 VRF Export 3:3

VRF Import 3:3

© 2006, Cisco Systems, Inc. All rights reserved. 8

Overview VLAN VRF VDC VSS VPNs

 Front-End Data Center

© 2006, Cisco Systems, Inc. All rights reserved. 9

L2 Protocols L3 Protocols L2 Protocols L3 Protocols

LACP CTS EIGRP VRRP LACP CTS EIGRP VRRP

RIB RIB RIB RIB

Protocol Stack (IPv4/IPv6/L2) Protocol Stack (IPv4/IPv6/L2)

Virtual Device Contexts

© 2006, Cisco Systems, Inc. All rights reserved. 10

VDC A VDC B Fault Domain

Process DEF in VDC

Virtual Device Contexts

vdc_id vdc_name state mac

switch(config-vdc)# show vdc detail

© 2006, Cisco Systems, Inc. All rights reserved. 11

switch(config)# vdc CiscoLive2008

Virtual Device Contexts

switch(config)# vdc resource template N7Kswitch

© 2006, Cisco Systems, Inc. All rights reserved. 12

Ports Are Assigned on a per VDC

Once a Port Has Been Assigned to a

Virtual Device Contexts

Linecard 1 Linecard 2 Linecard 3

MAC “A” MAC “A”

Intermediate level session focused on data center

Front-End Data Center

Virtualization: One to many

Virtualization: One to many

Virtual connectivity services Consolidated

Front-End Data Center

Typically all route processes

VRFs allow dividing up your routing

Import/export routes to/from MP-BGP updates

Front-End Data Center

Front-End Data Center

Guaranteed resource levels for each context with support for

Bandwidth Access lists