Advanced SAN Services

BRKSAN-3707
Mike Dunn
Network Consultant
July 14, 2011

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

What Are the Top Storage Initiatives?

Consolidation

Deduplication

Backup Redesign

Technology Refresh

Tiered Storage Build Out

Tiered Storage Build Out

Technology Refresh

Archiving

Virtualization Adoption

Consolidation

Archiving

Reporting

Disaster Recovery

Virtualization Adoption

Data Migration

Thin Provisioning

Improving Performance

Data Migration

Improving Forecasting

Backup Redesign
0%

10%

20%

30%

40%

50%

0%

10%

20%

30%

40%

50%

Source : TIPs Storage Research 2010

Source : TIPs Storage Research 2009

Top Five Solution Initiatives:

BRKSAN-3707

Deduplication
Technology Refresh
Tiered Storage Build Out
Archiving
Consolidation

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Agenda
SAN Consolidation with Virtualization
Inter-VSAN Routing (IVR)
N-Port Virtualizer (NPV) / NPIV
FlexAttach

Tiered Storage and Backup Design

Data Mobility Manager (DMM)

Storage Media Encryption (SME)

SANTap

Fibre Channel over Ethernet (FCoE)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

SAN Consolidation with

Virtualization

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

The Result of SAN Consolidation May Require

Resource Sharing
Isolated fabrics hosted on different SAN
switches

Physical
SAN

Physical
SAN

Physical
SAN

Application isolation is important

Virtual SANs (VSAN) are virtual fabrics

providing logical separation

VSAN

Physical
Islands

VSAN

VSAN

Consolidated and manageable logical SANs

Traffic Isolation

Virtual
Fabrics

Inter-VSAN Routing (IVR) route traffic

between VSANs to achieve cross-fabric
connectivity
Consolidated SANs still isolated
Only relevant fabric events are propagated
Enables resource sharing

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

VSAN

VSAN

VSAN

Routed
Virtual
Fabric
Cisco Public

Inter-VSAN Routing (IVR)

Switch Virtualization
IVR Is Needed When:

pwwnH

A Fabric

B Fabric

Domain 10

Domain 20

IVR Enabled
Switches

Feature IVR Enable

No other devices can or need

to communicate

B
Domain 50

Domain 60
Backup VSAN

Certain devices (hosts and

tapes) are in different VSANs
for isolation reason and they
need to communicate at the
same time

IVR Configuration Includes:

VSAN Topology
Zones

(V200)

pwwnT

IVR Provides Isolation and Resource Sharing

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

IVR Use Case

Common Resource Sharing
Common Physical Fabric

Overlay data replication fabrics

Tape Media Server
on common physical fabric

Tape Media Server

Finance
SAN

Tape Media Server

Engineering
SAN

HR
SAN

BRKSAN-3707

TAPE
SAN

2011 Cisco and/or its affiliates. All rights reserved.

No need for separate pair of switches

for each replication connection
Use one VSAN per replication
connection

Share the common resources

among different replication
VSANs
Share common SAN Extension
circuits amongst multiple virtual
fabrics

Cisco Public

IVR VSAN Topology

Defines Routing Topology Scope
pwwnH

Manually define the fabric

routing scope
List of VSANs to be used for IVR on
each switch

OLTP VSAN
(VSAN 100)
Domain 10
Email VSAN
(VSAN 300)

swwn1 vsan-range 100,200

Database
VSAN
(VSAN 500)

Contains one entry for each

IVR-enabled switch

swwn1

Domain 60

Same set of entries on all the

switches
List contains only VSANs included
in topology

Needs to be activated

Backup VSAN
(VSAN 200)

Topology map is calculated which is

used for routing
pwwnT
A Fabric Shown, Repeat for B Fabric

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

IVR VSAN Topology

All IVR-Enabled Switches Need VSAN Topology
pwwnH

All the IVR-enabled switches

should have same number
of entries

OLTP VSAN
(VSAN 100)

All the switches should

contain topology entries for
other IVR-enabled entries
swwn1 vsan-range 100,200
swwn2 vsan-range 100,200
swwn1

swwn2

Backup VSAN
(VSAN 200)

pwwnT
A Fabric Shown, Repeat for B Fabric
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

IVR Zoneset and Zones

Defines IVR Zoning
pwwnH
IVR_zone1
member pwwnH
member pwwnT

OLTP VSAN
(VSAN 100)

Establishes communication
across VSANs
Similar to regular zones
Extends zoning across VSAN
boundary

Domain 10

Domain 60

ivr_zoneset1
ivr_zone1
member pwwnH vsan 100
member pwwnT vsan 200

Needs to be activated once

IVR adds zones to relevant VSANs

All the IVR enabled switches

need to have same IVR
zones

Backup VSAN
(VSAN 200)

One IVR Zoneset per topology

pwwnT

IVR_zone1
member pwwnH
member pwwnT

A Fabric Shown, Repeat for B Fabric

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

10

IVR Virtual Switch (Domain) and Device

Representing Native Switch/Device
pwwnH
10.1.2

Virtual Switch (domain)

OLTP VSAN
(VSAN 100)

Representation of a switch (domain)

in another VSAN

Virtual Device
Representation of a device another
VSAN

Domain 10

Device Advertisement
Domain 60

Process of instantiating the

domain/device in another VSAN

pwwnH
10.1.2

Domain 10

pwwnT
60.1.2
Backup VSAN
(VSAN 200)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

11

Inter-VSAN Routing (IVR)

VSAN Topology

swwn1 vsan-range 100,200

Step by Step

IVR zones

Domain 60

ivr_zone1
pwwnh vsan 100
pwwnt vsan 200

pwwnH
10.3.4

OLTP VSAN
(VSAN 100)

Switches are
virtualized with
original domain ID

No FCID
Translation

60.1.2

Domain 10

Domain 60

Devices are
virtualized with
original FCID
IVR routes between
VSANs
Domain IDs have to
be unique across all
the VSANs

Backup VSAN
(VSAN 200)

pwwnT
60.1.2
Domain 10

10.3.4
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

12

IVR Network Address Translation (NAT)

No Unique Domain Requirement
VSAN 200 Is Represented as
Domain 30 in VSAN 100

Domain 30
OLTP VSAN
(VSAN 100)

pwwnH
10.3.1

30.3.2

10.1.2, 70.1.2}
->
{30.3.2, 10.3.1}

{10.3.1, 30.3.2}
->
{70.1.2, 10.1.2}

Domain 10
Domain 10

Backup VSAN
(VSAN 200)

pwwnT
10.1.2

Note same domain ID in

both VSANs
With IVR NAT, each
VSAN is represented in
another VSAN using a
domain ID

Switches are virtualized

using domain assigned
to its native VSAN
No unique domain
restriction since the
FCIDs are translated

Domain 70

VSAN 100 Is Represented as

Domain 70 in VSAN 200
BRKSAN-3707

70.1.2

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

13

IVR NAT Domain and FCID Persistency

Ensure Domains/FCIDs Are Persistent
native-vsan 200 domain 30
pwwn 33:44:55:66:77:88:99:1 fcid 30.3.2
OLTP VSAN
(VSAN 100)

Domain 30

pwwnH
10.3.1

The Virtual Domain ID and

FCID can change

30.3.2

When Devices/switches go down/up

Some HP-UX and AIX servers

need static FCIDs

Persistent Virtual Domain

Domain 10

Same Virtual domain can be assigned

to a virtual switch

Persistent Virtual FCID

Domain 10

Same FCID is assigned to the virtual

device
Domain has to be persistent

Backup VSAN
(VSAN 200)

Domain 30

pwwnT
50.1.2

Domain 70

70.1.2
native-vsan 100 domain 70
BRKSAN-3707

pwwn 11:22:33:44:55:66:77:88
2011 Cisco and/or its affiliates. Allfcid
rights 70.1.2
reserved.

Cisco Public

14

IVR VSAN Automatic Topology

Eliminates Manual Configuration Topology
pwwnH
10.1.2

Enable auto topology on one

switch

OLTP VSAN
(VSAN 100)

No manual configuration of VSAN list is

needed
Requires Cisco Fabric Services (CFS)
distribution enabled on all switches

Email VSAN
(VSAN 300)

Domain 10

Relevant local VSANs on a IVR

enabled switch are used

Database VSAN
(VSAN 500)

ivr vsan-topology auto

Domain 60

swwn1 vsan-range 100,200

swwn2 vsan-range 100,200

Only member VSAN and transit VSANs

are used

Backup VSAN
(VSAN 200)

pwwnT
60.1.2

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

15

IVR Across Remote Data Centers

Edge VSAN Extends Through the WAN Connection
pwwnH

Consolidate many physical

fabrics into VSANs through
SAN extensions

OLTP VSAN
(VSAN 100)

swwn1 vsan-range 100,200

swwn1

IVR Enabled Switch

SONET/SDH
DWDM
CWDM
IP (Metro Eth)

Remote Backup
VSAN
(VSAN 200)

Events are propagated across

geographically separated
fabrics
The edge VSAN events will be
carried over the WAN/MAN

swwn2

pwwnT
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

16

IVR Transit VSAN

Separate VSAN for the WAN/MAN Connection
pwwnH

The VSAN on the transit

path between members
VSANs

OLTP VSAN
(VSAN 100)

swwn1 vsan-range 100,300

swwn2 vsan-range 200,300

Transit VSAN
(VSAN 300)

swwn1 vsan-range 100,300

swwn2 vsan-range 200,300

IVR enabled on Transit VSAN

border switches
swwn1

SONET/SDH
DWDM
CWDM
IP (Metro Eth)

Only relevant and required

events are carried over transit
VSAN
Isolates Edge VSANs from any
transit events in transit VSAN
Network

swwn2

Backup VSAN
(VSAN 200)

pwwnT
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

17

IVR Summary
Used for Consolidation of Fabric and Sharing of resources
Can Isolate Fabrics when Traversing Data Centers
Recommend use of Transit VSANs over WANs

IVR NAT allows duplication of Domain IDs

All switches in IVR Domain must be configured

Use Auto Topology for simpler topologies
Use Manual Configuration for complex networks

Persistent DomainID and FCIDs are available for HP-UX and AIX
deployments

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

18

SAN Design Overview

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

19

Typical SAN Design

Core Edge Is Typical SAN Design
Option
Storage on High Density Directors on
the Core

Servers at the Edge:

End / Middle of Row with Directors
Top of Rack with Fabric Switches

Blade Chassis with Blade Switches

B
A

Total Number of Devices: 1,728

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

20

Alternate SAN Design

Edge - Core Edge Option
Storage on Directors Separate
Attached to Core

Core Directors Provide Routing and

Services

B A

Servers at the Edge:

End / Middle of Row with Directors
Top of Rack with Fabric Switches

Blade Chassis with Blade Switches

Total Number of Devices: 4,240

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

B
A

21

SAN Explosion with Fabric and Blade

Switches
Scalability
Each fabric/blade Switch uses a single
Domain ID
Theoretical maximum Domain IDs is 239
per VSAN
Supported number of domains is typically
smaller ~ 40 (depends on storage
vendor)

Manageability
More FC domains / switches to manage
Shared management of blade switches
between storage and server
administrators

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

22

Top-of-Rack Design with Fabric Switches

Rack Mounted Servers
SAN A

Top of Rack Design

Number of Dual Attached Servers

SAN B

448

MDS 91x4
Number of FC Switches per Fabric

MDS 9148

29

A
B

14 Racks

32 Dual
Attached
Servers per
Rack
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

23

SAN Virtualization with N-Port

Virtualizer (NPV)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

24

NPIV (N_Port Identifier Virtualization)

Multiple Logins on a Physical Port

T11 standard
Assigning multiple FC IDs to a single N_Port
Uses FDISC to get additional FCIDs
Shares the physical port but separate logins
Server

3 Logins
Login1 FCID=1.1.1
FC

FLOGI
FDISC

Login2 FCID=1.2.1

F_Port

FDISC
Login3 FCID=1.2.3
N_Port Controller

3 FCIDs

HBA

BRKSAN-3707

MDS 9000

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

25

N-Port Virtualizer (NPV)

Switch Mode Aggregating Multiple HBA Logins
NPV Core Switch

NPV is a switch mode

Switch acts like a NPIV (N-Port ID Virtualization) host
(conceptually an HBA aggregator)

Domain ID 10

NPV switch uplink is no longer an ISL (NP-port)

E

fc1/3

Changing from switching mode to NPV mode is disruptive

ISL NPIV

NP Link
E

fc1/2

NPV switch does not use a Domain ID

NP

Simplified Management
Fewer FC domains/switches in the fabric

Domain Id 20
Blade/
Fabric
Switch

NPV
F
fc1/1

BRKSAN-3707

NPV enabled switch is now managed like a NPIV enabled

host
Enables management by server administrators

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

26

Comparison Between NPIV and NPV

NPIV (N-Port ID Virtualization)

Used by HBA and FC switches

Enables multiple logins on a

single interface

Allows SAN to control and

monitor virtual machines (VMs)

Used for VMWare, MS Virtual

Server and Linux Xen
applications

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

NPV (N-Port Virtualizer)

Used by FC (MDS 9124, 9134

and 9148), FCOE switches
(Nexus 5K), blade switches and
Cisco UCS Fabric InterConnects
(UCS6100)

Aggregate multiple
physical/logical logins to the core
switch

Addresses the explosion of

number of FC switches

Used for server consolidation

applications

Cisco Public

27

Top-of-Rack Design with Fabric Switches

Using NPV
Rack Mounted Servers
SAN A

Top of Rack Design

Number of Dual Attached Servers

SAN B

448

MDS 91x4
Number of FC Switches per Fabric

MDS 9148

A
B

14 Racks

32 Dual
Attached
Servers per
Rack
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

28

NPV Auto Load Balancing

Automatic Balancing of Server Loads on NP Links

Blade Server Chassis

Uniform balancing of server loads on

NP links

Blade 4

Blade 3

Blade 2

Blade 1

Server loads are not tied to any uplink

Benefit
Optimal uplink bandwidth utilization

Balanced
Load on NP
Links

1
3

2
4

SAN

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

29

Uplink Failure Means Traffic Disruption and

Downtime to Servers
Blade Server Chassis

External link failure brings down

server connectivity

Blade 4

Blade 3

Blade 2

Blade 1

Applications are disrupted

X X
NPV

SAN

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

30

F-Port Port Channel

Enhance NPV Uplink Resiliency
F-Port PortChannels
Core Director

F-Port Port
Channel

Storage

Blade System
Blade N

Similar to ISL portchannels in FC

and EtherChannels in Ethernet

Blade 2

SAN

Link failures do not affect the

server connectivity

Blade 1

N-Port

Benefits

F-Port

High-Availability - no disruption if
cable, port, or line cards fail

interface port-channel 1
channel mode active
no shut

Optimal bandwidth utilization &

higher aggregate bandwidth with
load balancing

interface fc1/1
channel-group 1

No application disruption

interface fc1/2
channel-group 1

BRKSAN-3707

Bundle multiple ports in to 1

logical link

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

31

F-Port Trunking
VSAN Consolidation on NP Uplinks
F-Port Trunking
Uplinks carry multiple VSANs
NPV

F-Port Trunking
on
F-Port Channel

Benefits

Core Director
Storage

Blade System
VSAN 1

Blade N
Blade 2

VSAN 2

Separate management domains

SAN

Traffic Isolation and ability to host

differentiated services on blades

Blade 1
VSAN 3

N-Port

Extend VSAN benefits to Blade

servers

F-Port

Interface fc1/1
trunk mode on
trunk allowed-vsan 1-3
Interface port-channel 1
trunk mode on
trunk allowed-vsan 1-3

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

32

Deploying NPV Using NPV Wizard

Enable NPV on
Blade Switches

Enable NPIV
on SAN Core

Pair NPV Switches

and SAN Core

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

33

Deploying NPV Using NPV Wizard

Setup
Connections

Setup VSAN

Apply the
Configuration

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

34

Nested NPIV
Connecting NPIV Capable Hosts to NPV
NPV-Core Switch

Two levels of NPIV usage

From server to first level switch (NPV)

From NPV to the core SAN

F

Virtual servers connected to the NPV

devices

F
NPIV

Servers Supporting NPIV

NP P2

P1 NP

VMware ESX using RDM mode

NPV Edge Switch

F

NPIV
P3 = vP1

vP2
vP3
vP4
BRKSAN-3707

P4 = vP5

vP6
vP7
vP8
2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

35

Reference

NPV Scalability
Switching
Mode

NPV Mode

Logins per Port

42

114

Logins per Port-Group

168

114

Logins per MDS 9124

1,008

684

Logins per MDS 9134

1,680

1,140

Logins per MDS 9124e

1,008

684

840

570

2,016

1,368

Blade 4

1
3

2
4

Number of Logins

Logins per Port

126 Gen 1/2

250 Gen 3

Logins per Line Card

400 Gen1/2
800 Gen 3

Logins per Switch

2,000

Logins per Physical Fabric

10,000

BRKSAN-3707

Blade 3

Logins

Blade 2

Login per MDS 9148

Blade 1

Logins per IBM Blade Switch

Server Chassis

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

SAN

36

NPV Summary
Multiple FCIDs to a single port
NPV is a switch mode, switch acts like an NPIV aggregator
Solves the domain ID explosion
Simplifies fabric management
F-Port Channel provides failover and load-balancing
Wizard setup for simple configuration

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

37

Server Virtualization Adoption

Challenges for SAN
Administrators

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

38

Server Virtualization and SAN

Virtual Machines (VMs)
Do not need pWWNs (no SAN identity)
Have no explicit fabric login

Meaning Virtual Machines without SAN identity

SAN administrator cannot control them
SAN administrator cannot monitor them
SAN administrator cannot ensure service levels to them

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

39

NPIV Allows SAN Administrators to

Control and Monitor VMs
NPIV gives Virtual Servers SAN identity
Designed for virtual server environments Linux on zSeries, VMware

Allows SAN control of VMs

Zoning and LUN Masking at VM level

Multiple applications on the same port can use different IDs

Better utilization of the server connectivity

Monitor the VMs using Flowstats

Virtual Servers

Email
Web
Print

Control and
Monitor VMs
in the SAN

Zone_Email
vpwwn1
pwwnD1
Zone_Web
vpwwn1
pwwnD1
Zone_Print
vpwwn1
pwwnD1

LUN1(pwwnD1)

vpwwn1 FCID=1.1.1
FC

vpwwn2 FCID=1.1.2

LUN2 (pwwnD2)

F_Port
LUN3(pwwnD3)

vpwwn2 FCID=1.1.3

N_Port Controller
HBA

BRKSAN-3707

MDS 9000
2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

40

Server Virtualization with

FlexAttach

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

41

Server and SAN Administrator Coordination

Leads to Inefficient Operations
Blade Server Chassis
pwwn1

Blade N

Failed
Blade

Blade 1

HBA/Server
Failure

pwwn2

Configurations use port WWN (pWWN)

Needs co-ordination between

server and SAN administrators

F Port

Replacement of failed server

NP Port
F Port
SAN Zoning
Change

Server/HBA failures need SAN

and array configuration change

Zone myZone
member pwwn1
member pwwn2
member pwwnD

Generally needs a change-window

SAN

Storage
Array
Configuration
(LUN Masking)
Change

BRKSAN-3707

pwwn1
LUN 0
LUN1
pwwn2
LUN0
LUN2

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

42

Cisco FlexAttach
Flexibility for Server Mobility
Blade Server Chassis
pwwn3

How FlexAttach Works?

pwwn1 pwwn2

Each F-port is assigned a virtual WWN

Blade N

Blade2

Blade 1

Virtual
pWWNs

Blade server assumes virtual WWN of the

port it is connected to

vpwwn1 vpwwn2 F Port

Flex Attach
on
NPV

NP Port
F Port
No SAN Zoning
Change

Zone myZone
member vpwwn1
member vpwwn2
member pwwnD

SAN

Benefits
Flexibility for Server Mobility - Adds, Moves
and Changes
No SAN re-configuration required

Eliminate need for SAN and server team to

coordinate changes

Works only in NPV mode

Storage
No Array
Configuration
Change

BRKSAN-3707

vpwwn1
LUN 0
LUN1
vpwwn2
LUN0
LUN2

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

43

FlexAttach
Rewrites Real pWWN to Virtual Port WWN
Core Switch (MDS or 3rd Party Switch with NPIV Support)

1.

Interface fc1/1 is
FlexAttach enabled and
assigned a port wwn
vpwwn1

2.

Server S1 does FLOGI

to interface fc1/1

3.

pwwn1 FLOGI is
rewritten to use vpwwn1
FLOGI

4.

vpwwn1 FLOGI is
converted to FDISC and
registered with FC Name
Server

Server S1 Is Known by
vpwwn1 in the SAN

F
Port WWN of S1= vpwwn1

NP
NPV
FlexAttach

pwwn Rewrite Rules

F fc1/1 vpwwn1
Port WWN of S1 = pwwn1

pwwn1

Server S1
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

44

Zero Touch Replacement

Same Port Replacement
New
Blade

Blade Server

Blade Server

Blade N

Blade 2

Blade 1

Blade N

Blade 2

Blade 1

No change needed in
SAN or on blades

No explicit SANServer Admin

interactions needed

vpwwn1
NPV

NPV

Zone myZone
member vpwwn1
member vpwwn2
member pwwnD

Replace the failed server

onto the same port

SAN Core

Storage

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

45

Replace to a Spare Server

No Need for Physical Replacement
Blade Server

Blade Server

vpwwn1

vpwwn1
fc1/1

Spare
Blade

Blade 2

Blade 1

Blade N

Blade 2

Blade 1

Move to a spare
server

fc5/10
NPV

NPV

Zone myZone
member vpwwn1
member vpwwn2
member pwwnD

Move the virtual

pWWN to the new
port
No physical
replacement

No explicit SANServer Admin

interactions
needed
SAN Core

Benefit
Flexibility for server mobility
across different Blade
chassis/Racks

Storage

Highly Scalable solution

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

46

SAN Pre-Provisioning Independent of Servers

No Change When the Servers Are Ready to Be Deployed
New
Blade

New
Blade

Pre-provision SAN
for ordered
servers
Blade Server

Blade Server

Blade N

Blade 2

Blade 1

vp1

Blade N

.
vp2

NPV

NPV

No need for servers

to arrive
Use the FlexAttach
generated virtual
pWWNs
Use planned change
control for SAN
change

No explicit SANServer Admin

interactions
needed

Configure SAN for

vp1 and vp2
SAN Core
Zone myZone
member vp1
member vp2
member pwwnD

Storage

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

47

FlexAttach Summary
Virtualize HBA (WWNs) to a switchport
Assigns a virtual WWN to a switchport
Eliminates server and storage admin coordination for changes
Allows flexibility for server moves
Eases replacement of servers and HBAs
Pre-provision SAN ports and storage in advance of servers

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

48

Agenda
SAN Consolidation with Virtualization
Inter-VSAN Routing (IVR)
N-Port Virtualizer (NPV) / NPIV
FlexAttach

Tiered Storage and Backup Design

Data Mobility Manager (DMM)

Storage Media Encryption (SME)

SANTap

Fibre Channel over Ethernet (FCoE)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

49

Tiered Storage / Backup

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

50

Service Level Requirements for Application

Data Vary
Classify and manage data
according to application need

Classification Criteria
Access Times/Performance
Application Availability
Recovery Time and Point

Cost of Storage

Optimize the resource

utilization

Source : EMC / HDS

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

51

Data Classification to Create Storage Tiers

Tier1
Moving the data to the
different tiers using
Mission
Critical
Data

Data Mobility Manager

(DMM)

Onsite

Tier2
Minutes to
Hours
Availability

Onsite CDP

Network based mirroring

using

SANTap based solution

OffSite CDP

Tier3
Hours to
Day
Availability

Onsite Tape/VTL
OffSite
Tape/VTL

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Encrypt the data for

compliance using

Storage Media
Encryption (SME)

52

Implementing Tiered Storage

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

53

Data Migration

Application Servers
Oracle

Clearcase

Exchange

Facilitates moving of data from

an Existing Storage Pool to a
New Storage Pool via the SAN
Why is it done?
To Upgrade, Consolidate or
Replace existing storage

SAN Fabric

How often?
Typically every 3 years upon
lease expiry for a single
Storage Array

Existing
Storage

BRKSAN-3707

New
Storage

2011 Cisco and/or its affiliates. All rights reserved.

Ongoing activity at the IT

Department level considering
the number of Storage Arrays
present

Cisco Public

54

Data Migration Techniques Available

Servers

Servers

SAN Fabric

Servers

SAN Fabric

Existing
Storage

New
Storage

Server/Software Based

SAN Fabric

Existing
Storage

New
Storage

Storage Array Based

Online data migration
No host software or agents.

No additional h/w
No re-wiring
Cons

Cons

Throughput limited by host bandwidth

Large CPU cycles consumed
Longer Migration time
Clustered environments not supported

New
Storage

Appliance Based
Pros

Pros

Pros

BRKSAN-3707

Existing
Storage

Cons
Virtualizes the source disk
(PWWNs change)
LUN mapping/masking handling
Re-configuration/Reboot of all
hosts accessing this target.

Vendor lock-in
License

2011 Cisco and/or its affiliates. All rights reserved.

No host software required

More scalable

Cisco Public

55

Ciscos Data Migration

Application Servers
Oracle

Clearcase

Exchange

SAN-based Data Migration

Advantages
SAN moves the data

No server software required

No Virtualization layer in the SAN
SAN Fabric

Scalable

Referred to as
Cisco MDS Data Mobility Manager [DMM]

Existing
Storage

BRKSAN-3707

New
Storage

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

56

Cisco MDS Data Mobility Manager (DMM)

SAN based Data Migration

Switch: MDS 9509/9513/9216/9222i
Line card : 32 Port Storage Services Module (SSM) or
MSM 18+4

SOLARIS1-SRVR

Fabric A

MSM

Fabric B

MSM

Existing
Storage

BRKSAN-3707

New
Storage

Online/Offline Data Migration

Transparent Insertion/Removal via MDS

FC Redirects

Sync Data Migration

Heterogeneous Array Migration
Dual Fabric
Unequal LUN Migration
Rate Adjusted Migration
Server/Storage based migration
Delayed Cut-Over
4.1 TB/Hour data movement rate
GUI for configuration/status

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

57

Cisco MDS DMM Online Mode

Introduce Cisco MDS Data Mobility Engine
(SSM / SN) into fabric

SOLARIS1-SRVR

No re-configuration/re-wiring of the existing SAN

Enable via Configuration GUI

Fabric

Server I/O continues transparently to the

Existing Disks

MSM

Uses Ciscos FC Redirect Feature

No SAN-based Volume Management

Existing
Storage

BRKSAN-3707

New
Storage

Data Mobility Engine moves data in the

background
Heterogeneous Storage Arrays
Disable via Configuration GUI

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

58

What Is FC-Redirect?
A target centric re-direct based transport is a
low level infrastructure used for transportation
only

Application Server

Re-writing only the FC SID & DID.

FC-Redirect
Traps and
Sends the
Packets to MSM

MSM

MSM Sends
Packets to Both
Old and New
Old Array
Array

BRKSAN-3707

DMM
Programs FCRedirect to
Send Traffic
Destined to
Old Array to
MSM

New Array

Seamless integration of one or more

intelligent services in a fabric for a specific
Host & Disk (I_T) pair.
No re-wiring or re-configuring existing Hosts
& Disks.
No Splitting of fabrics into multiple VSANs.
Operate in a heterogeneous switch
environment
Disk must be attached to a FC-Redirect aware
MDS, Host & MSM can be located anywhere
in the fabric.
Inter-switch communication is done over the
existing CFS infrastructure.

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

59

How DMM Works

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

60

DMM Method 1
The Server has one path to the existing
storage through each Fabric
SOLARIS1-SRVR

In a Dual Fabric topology, a DMM Job

requires 2 DMM modules one in each Fabric
SAN A

SAN B

IP

MSM

MSM

The DMM module in each Fabric must support the VSAN of

interest in that fabric

The 2 DMM modules in a Dual-Fabric DMM

Job establish a IP connection between them
In each fabric, the server I/Os are redirected
through the DMM module in that fabric

DMM performs the data copy from the

Existing to the New storage using one of the
fabrics.
Existing
Storage

New
Storage

For a Dual Fabric Multi VSAN topology, the

DMM module in each fabric will handle
multiple VSANs
In Each Fabric, all paths from the Server to
the Existing storage will be redirected through
a single DMM module

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

61

Cisco MDS DMM Algorithm

Server

Dealing with Server IOs

While (Regions Left) {
Writes to Migrated
Select
Region;
Area a
Are
Mirrored
Writes
to Being
Copy
Region;

}
Migrated

Being Migrated
To Be Migrated

Existing
Storage
LUN
BRKSAN-3707

Writes to To Be
Migrated Area Are
Written to Existing
Storage Only

Server Reads Are

Read from Existing
Storage Only

New
Storage
LUN
2011 Cisco and/or its affiliates. All rights reserved.

Migrated Area Are

Queued Temporarily
(Until Region Has
Been Migrated)

Cisco Public

62

DMM Method 2 - Async

Supports Dual Fabric Topology

One MSM in each Fabric

SOLARIS1-SRVR

SAN A

MSM performs

SAN B

MRL

MRL

MSM

MRL Bitmap

MSM

Modified Region Log (MRL) update

for each WRITE I/O
Passes the WRITE I/O to the
Existing Storage

Aggregation of MRL bitmap

Data movement for Migration

Existing
Storage

New
Storage

Server HBA Port, Existing/New

Storage Port needs to be in the
same VSAN (per Fabric)

Eliminates the mirror latency for

Server WRITEs
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

63

Cisco Data Mobility Manager

Async Mode
Server

Dealing with Server IOs

Modified Region Log [MRL]

Mark All Regions in MRL Dirty

Writes Are
to
While (MRL Regions
Left)Written
{
Existing Storage Only
Select
a Region;
MRL
Entry Is Updated
forRegion;
Each Write Issued
Copy

Reads Are
ClearServer
MRL Region
Read from Existing
Storage Only
Multiple Passes of
MRL Done Until All
Regions Are Clear
For Cut-Over Last
MRL Pass Done with
the LUN in the Offline
Mode

Existing
Storage
LUN
BRKSAN-3707

New
Storage
LUN
2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

64

DMM Method 2 for Data Center Migration

New Storage in the Remote Data
Center needs to be in the same
VSAN as Server/Existing Storage

SOLARIS1-SRVR

SAN A

MRL

MSM

SAN B

MRL

MSM

Production SAN or VSAN need to

span across Local/Remote Data
Centers
Deployed topology
Production SAN/VSANs constrained
within a Data Center
Replication or Migration SAN/VSAN span
Data Centers

Existing
Storage

Local
Data Center

FCIP
Cloud

Remote
Data Center
New
Storage

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

65

Three SAN/VSAN Topology

SAN A and SAN B - Production

SOLARIS1-SRVR

Contains: Server/Existing Storage

SAN A

SAN B

Traffic: Server to Existing Storage

LUN
Dual Fabric within Data Center
New Storage not visible

Replication SAN
Contains Existing/New Storage
Existing
Storage

Traffic: via Replication SAN to New

Storage
FC, DWDM or FCIP links

Replication SAN

Local Data Center

Remote Data Center

New
Storage
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

66

DMM Method 3
3 MSMs per DMM Job

SOLARIS1-SRVR

MSM 1 / 2 in the Production SAN

SAN A

SAN B

Keeps track of Server Write I/Os via

Modified Region Log [aka MRL] bitmap
Sends the MRL bitmap over IP to MSM 3

MSM 1

MSM 2

MSM 3 in the Replication SAN

Merges the MRL bitmap from MSM 1/ 2
Performs data movement from Existing to
New Storage based on the merged MRL
bitmap

Existing
Storage

MSM 3

Replication SAN

Merged
MRL

Replication SAN
Connected via DWDM links within the same
Metro Area

Local Data Center

Connected via FCIP links across continents

Remote Data Center
New
Storage
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

67

Deployment Guidelines
Do not add the same initiator/target port pair into more than one migration
job simultaneously.
When using multipath ports, the server must not send simultaneous I/O
write requests to the same LUN from both multipath ports. The first I/O
request must be acknowledged as completed before initiating the second
I/O request.

DMM is not compatible with LUN zoning.

DMM is not compatible with inter-VSAN routing (IVR). The server and
storage ports must be included in the same VSAN.
DMM is not compatible with SAN device virtualization (SDV).
DMM does not support migration to a smaller destination LUN.

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

68

Cisco MDS DMM Licenses

There Are Two Types of DMM Licenses:
Permanent License: This license (also called End User license) is only available
to end users that will be deploying DMM for their own data migration needs. The
permanent license may not be used by users that expect to use the MDS platform
(with the SSM / SN card) to sell migration services to other users.
180-day License: This license is a time-based license that is available to service
provider users that expect to sell MDS platform-based migration services. Users
that qualify for the permanent license may purchase the 180-day License if they
so choose to do so.

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

69

Cisco DMM Recap

Method 1
2 SAN topology
Server, Existing/New Storage connected to each SAN and are in the
same VSAN
Server WRITE I/Os mirrored to Existing/New Storage in both SANs
Data Movement performed in one of the SANs

Method 2 - Async
2 SAN topology
Server, Existing/New Storage connected to each SAN and are in the
same VSAN
MRL bitmap tracks Server WRITE I/Os in both SANs
Data Movement performed in one of the SANs
Method 3 Data Center Migration
3 SAN topology : 2 Production SANs and 1 Replication/Migration SAN
Server and Existing Storage connected to the Production SAN
Existing/New Storage connected to Replication/Migration SAN
MRL bitmap tracks Server WRITE I/Os in the Production SAN
Data Movement performed in the Replication SAN
Built on Method 2
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

70

Storage Media Encryption

(SME)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

71

Cisco SME - Secure, Integrated Solution

Application
Server
Name: XYZ
SSN: 1234567890
Amount: $123,456
Status: Gold

Encrypts media for SAN attached tapes,

virtual tape libraries and disk arrays
Uses IEEE AES-256 encryption
Disk XTS, Tape GCM

Key Management
Center (KMC)
TCP/IP

Encrypt

CC EAL-3 and FIPS 140-2 certified switch

Solution includes Cisco KMC for

provisioning and key management
Integration with RSA Key Manager

@!$%!%!%!%%^&
Name: XYZ
*&^%$#&%$#$%*!^
SSN: 1234567890
Amount: $123,456
@*%$*^^^^%$@*)
%#*@(*$%%%%#@
Status: Gold

Handles traffic from any VSAN in fabric

2H CY2011

Tape
Devices

Virtual Tape Disk

Array
Library

BRKSAN-3707

Compresses tape data equal or better

than tape drives

Offline data recovery tool decrypts tape

without MDS 9000 using Linux server

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

72

Delivering Encryption as a SAN Service

MDS9200
Series

Name: XYZ
SSN: 1234567890
Amount: $123,456
Status: Gold

MDS 9500
Series

Storage Media
Encryption Service

@!$%!%!%!%%^&
*&^%$#&%$#$%*!^
@*%$*^^^^%$@*)
%#*@(*$%%%%#@

1. Insert Cisco MSM-18/4 or SSN16 modules or MDS 9222i switches

2. SME is a licensed feature
3. Enable Cisco SME and setup encryption service
4. Provision encryption for specific storage devices
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

73

Cisco SME - Scalable, Highly Available

Media Servers

Integrates transparently in MDS fabrics

using FC-redirect
Allows rapid deployment
No SAN re-configuration or re-wiring

Provision as a simple, logical process of

selecting what to encrypt
MSM-18/4

Provision at the data center level and not

at the module level

MSM-18/4

Modular, clustered solution offers highly

scalable and reliable performance
Up to 4 switches and 32 encryption units
Support dual fabric configurations
Disk Arrays, Tape Drives and VTLs

Automatically load balances

Redirects traffic if a failure occurs

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

74

Cisco SME Disk Data Flow

Dual-Fabric cluster

Host

Traffic encrypted on all paths

Operations
Data Preparation
Rekey
MSM-18/4

MSM-18/4

Modes
Offline

Disk Array
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

75

Wizard-Based Provisioning

Wizard 1 Creating a Cluster

Selects Encryption Modules
Wizard 2 - Adding a Tape Group
Defines Key Management Policies
Selects Media Servers
GeneratesCisco
and Stores
SME Master Key
SpecificsisDevices
ready ! to Encrypt Tape
Volumes On

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

76

SME Key Management

Cisco Key
Management Center RSA Key Manager

Cisco KMC provisions and transports

keys securely
No new software required, based on Cisco
Fabric Manager

Application Servers

Managed through web browser interface

Provides essential key management

functions:
Archiving, replicating, recovering, and
purging media keys
MSM-18/4
Fabric A

Disk Arrays,
Tape Drives
and VTL

BRKSAN-3707

MSM-18/4

Logging Cisco SME transactions

Fabric B

Accommodates single and multiple site

environments

Integration with RSA Key Manager

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

77

Master Key Protection

Smart Cards

Advanced

Level of Security

Smart Cards with Recovery Shares for Each

Master Key Where M of N Recovery Officers
Are Required to Recover a Master Key

Standard
Smart Cards with All Master Keys
No Recovery Shares

Options:
2 of 3
2 of 5
3 of 5

Basic
A file with all master keys
Master keys encrypted with a password
Regular backup and archive

Complexity
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

78

SME Tape Key Hierarchy

Master key resides in smart
cards
Master Key

Cisco Key
Management
Center

Quorum (M out of N) of smartcards

required to recover a master key
Recovery shares accomplish
secret sharing

Tape Volume
Group Key

Tape Key

Keys reside in clear-text only

within crypto boundary on
switch module
Unique key per tape, or per tape
volume group
Media keys wrapped by master
key before storage or transport
to Cisco key management
center
Option to store tape keys on
tape media

Tape Key
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

79

Secure System Architecture

Hardware and software architecture designed to meet FIPS140-2
level two certification requirements
Tamper-evident: attempts to tamper with system are immediately
visible
Strong, standard AES-256 modes of encryption
Smart cards available for master key protection

Critical security parameters and media keys never leave

system unencrypted
Role-based access control (RBAC) secures management
Enforces SME-specific roles

AAA server support allows centralized user authentication and accounting

(auditing)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

80

Roles and Identities

SME Storage Admin, Key
Management

SME Recovery Officer

Per-VSAN role-based access

control (RBAC) limits
management scope

Responsible for any recovery

function requiring
a master key

SME Storage Administrator is

responsible for managing tape
devices and volume groups

Quorum of recovery officers

needed to perform recovery
procedures (default is two out of
five)

SME Key Management role is

responsible for key
import/export, archiving, etc.

Security operations (SecOp)

staff may assume this role

Fully Integrated with MDS CLI and

GUI RBAC (TACACS+, RADIUS)
SME Design Guide White Paper:
http://www.cisco.com/en/US/prod/collateral/ps4159/ps6409/ps4358/design_guide_c07-464433.html
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

81

SME Disk

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

82

SME Disk Overview

SOLARIS1-SRVR

Available NX-OS 5.2(1), 2nd Half 2011

SME Node

SME Cluster

18+4 MSM/9222i

Fabric B

Fabric A

SME Node
Module 2

SME Node
Module 9

Encryption

Encryption

SME Node
Module 4

SME Node
Module 8

Encryption of Data flowing between

Servers and Storage

Dual Fabric Topology

Encryption performed on all the Fabric

paths

Supports SME Clusters

Supports SME Key Management

EMC1454-ES

Dual Fabric Data Center SAN Topology

Clear Text I/O

Encrypted I/O

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

83

SME Disk Configuration Model

Disk
SRVR1

SRVR2

Disk1,Disk 2,Disk3,Disk4
Multiple accessible paths [I,T,L]
HB HC

HA

Disk1:[HA, TA], [HB, TB]

HD

Disk2:[HC, TA], [HD, TB]

Fabric A

Disk3:[HC, TA], [HD, TB]

Fabric B

Disk4:[HA, TA], [HB, TB]

SME Node

SME Node

A crypto disk has the following components:

Diskgroup name:admin assigned
Disk name: admin assigned

Zero or more paths

SRVR1 LUN Map
Server Ports->HA,HB
LUN 0 : Disk 1
LUN 1 : Disk 4

Disk Group

Disk 1

Disk 3

Disk 2

Disk 4

Storage Array

State: CLEAR, CRYPTO etc

SRVR2 LUN Map

Server Ports->HC,HD
LUN 0 : Disk 2
LUN 1 : Disk 3

Zero or one active key in the KMC

Zero or more archived keys in the KMC

Disk Group
A administrative label used to group a collection of
crypto disks
Recorded as part of the crypto disk name in KMC

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

84

SME Disk Key Hierarchy

Smart Card
Master Key

A Two-Tier Hierarchy Is Used by Cisco

SME Disk
LUN key encrypts data on the disk
These keys are unique for each LUN

LUN Key

Stored in the KMC, encrypted by the

Master Key

Master key encrypts LUN keys

LUN

Generated when a cryptographic cluster

is created
There is a unique master key for each
cluster

SME Key Management White Paper:

http://www.cisco.com/en/US/prod/collateral/ps4159/ps6409/ps4358/white_paper_c11-462423_ps6028_Products_White_Paper.html
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

85

Shared Infrastructure Between SME-T and SME-D

Cluster infrastructure remains unchanged and the existing cluster configuration
commands can be used as is for creating an SME cluster for disks
The crypto node configuration remains unchanged and the existing sme interface
config commands can be used as is for SME Disks
The KMC configuration remains unchanged. However there are backend changes due
to differences in usage of KMC for tapes vs disks
The discovery of IT nexus pair for SME cluster remains unchanged. However there are
backend changes due to differences in handling of the discovery of tapes vs disks
In short the existing SME configuration guide titled Cisco MDS 9000 Family Storage
Media Encryption Configuration Guide should suffice for the above components

As per design and implementation (pending testing) SME Tape and SME Disk should
be able to co-exist in the same SME cluster with the following underlying
understanding:
SME tape backup group and SME diskgroup share the same name space, i.e. a disk group cannot have the
same name as tape backup group name
An IT nexus will either have all tape devices or all disk devices

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

86

Cisco SME Review (1)

Architecture and Flexibility
In addition to SME, SSN-16 can support multiple applications
Services Oriented SAN solution, not just a point product

SAN-level Provisioning and automatic load-balancing

Automatic assignment of flows to service engines, no static/manual configuration
required

High-availability of encryption engines

Integrated Clustering and HA

N+1 availability; in case of a failure, any available engines in the fabric picks up the
load

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

87

Cisco SME Review (2)

Linear Scalability
Up to 10 SSN16s in MDS 9513 and up to 40 encryption engines in a SAN. Adding an
engine linearly increases capacity and throughput

Key Management
Integrated free key management solution as well as support for external enterprise
key manager

Mix of SME Tape and SME Disk on the same SSN16

Save cost by eliminating the need for separate hardware

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

88

SANTap

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

89

SANTap
Enables appliance-based
storage applications without
compromising SAN integrity

Initiator

Initiator Target I/O

About SAN Tap

MDS delivers a copy of primary
I/O to an appliance

SAN

Copy of
Primary
I/O

Appliance

Appliance provides the storage

application

Examples of applications include

Continuous Data Protection (CDP),
replication, etc.

Key customer benefits

Target

Preserve integrity, availability, and

performance of primary I/O
No service disruption
Investment protection

= SAN Tap
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

90

Ease of Deployment
MDS9200
Series

MDS 9500
Series

SANTap Service
Appliance

Initiator

Target

Insert Cisco MSM-18/4, MDS 9222i switches, or SSM Module

No rewiring required
The hosts and targets do not have to be connected to MSM
No need to reconfigure hosts and targets

- The hosts continue to see the same WWNs for storage

- The targets continue to see the same WWNs for host

SANTap is a licensed feature

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

91

SANTap at Work

SANTap mirrors
write I/Os to RPA
Host VSAN

RecoverPoint
Appliance

RecoverPoint
Appliance

WAN
Remote Site

Local Site

SAN

Target VSAN

SANTap out-ofband fabric

splitting preserves:

LUN

Production
LUN

BRKSAN-3707

Local
CDP
Copy

Local
CDP
Journal

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

I/O integrity

I/O availability

I/O performance

92

SANTap Configuration
Host pWWN =
10:00:00:00:c9:a5:a6

DVT pWWN =
50:00:1f:e1:50:3b:09
Host
VSAN

Host VSAN contains host

pWWN and Data Virtual Target
(DVT) pWWN
DVT is real pWWN of target
port

Copy of
Primary
I/O

Target
VSAN

Appliance

Target VSAN contains target

pWWN and Virtual Initiator (VI)
VI is real pWWN of host port

Host VI pWWN =
10:00:00:00:c9:a5:a6
Target pWWN =
50:00:1f:e1:50:3b:09

No need for devices to move to

other switch/ports to work with
SANTap

= SAN Tap
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

93

SANTap and RecoverPoint Data Flow

Local Data Center

Remote Data Center

Host

Host

LOCAL FLOW
1.
Write I/O is sent to MSM module
2.
Write I/O is then forward to both local Storage Array
and local Appliance
3.
Both local Storage Array and local Appliance
acknowledge Write I/O back to the MSM
4.
Once MSM receives both acknowledgements, then
sends acknowledgment to Application Server

SANTap
SANTap

WAN
2

Array

BRKSAN-3707

Appliance

Appliance

REMOTE FLOW
1.
I/O is sent through the WAN to remote Appliance
2.
I/O is then sent to replication LUN(s) through the MSM
3.
I/O is then acknowledged back to the Remote
Appliance
4.
Remote Appliance then sends acknowledgement
back to Primary Data Center Appliance through the
WAN
2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

Array

94

SANTap Summary
Appliance-based storage application
MDS deliver a copy of I/O to the appliance
Enables Continuous Data Protection and Recovery
Copy of I/O is not in primary data path
No SAN re-wiring or reconfiguration required to implement

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

95

Agenda
SAN Consolidation with Virtualization
Inter-VSAN Routing (IVR)
N-Port Virtualizer (NPV) / NPIV
FlexAttach

Tiered Storage and Backup Design

Data Mobility Manager (DMM)
Storage Media Encryption (SME)
SANTap

Fibre Channel over Ethernet (FCoE)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

96

FCoE

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

97

FCoE: Consolidation Highway

I/O Consolidation
Consolidates separate LAN, SAN, and server cluster network environments into a
unified fabric.

Multi core CPU architectures driving increased network bandwidth

demands
Virtual Machines driving increased I/O connections and bandwidth
Fibre Channel Prevalent Storage Solution
Same operational model as today

Incremental Implementation
Start at the Edge

Leverage FC tools investment and management applications

Low latency 10GE affordability (even optics)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

98

Server Connectivity: Today

SAN A

SAN B
10GE
Backbone

10GE

4/8 Gbps FC

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

99

Server Connectivity: Unified with FCoE

SAN A

SAN B
10GE
Backbone

Nexus

10GE FCoE

10GE
4/8 Gbps FC
BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

100

Session Summary
SAN Consolidation with Virtualization
Inter-VSAN Routing (IVR)
N-Port Virtualizer (NPV) / NPIV
FlexAttach

Tiered Storage and Backup Design

Data Mobility Manager (DMM)

Storage Media Encryption (SME)

SANTap

Fibre Channel over Ethernet (FCoE)

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

101

Q&A

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

102

Other Sessions
BRKSAN-1121: SAN Core Edge Design Best Practices
BRKSAN-2047: FCOE Design, Operation, and Management Best
Practices

BRKSAN-3123: Storage Cloud Concept and Design

BRKSAN-2704: SAN Extension Design and Operation
BRKDCT-1044: FCoE for the IP Network Engineer

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

103

Additional Information
Cisco Storage Networking
http://www.cisco.com/go/storagenetworking

Cisco Data Center Networking

http://www.cisco.com/go/datacenter

Storage Network Industry Association (SNIA)

http://www.snia.org

Internet Engineering Task ForceIP Storage

http://www.ietf.org/html.charters/ips-charter.html

ANSI T11Fibre Channel

http://www.t11.org/index.htm

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

104

Recommended Reading
Continue your Cisco Live learning
experience with further reading from
Cisco Press
Check the Recommended Reading
flyer for suggested books

Available Onsite at the Cisco Company Store

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

105

Complete Your Online

Session Evaluation
Receive 25 Cisco Preferred Access points for each session
evaluation you complete.
Give us your feedback and you could win fabulous prizes.
Points are calculated on a daily basis. Winners will be notified
by email after July 22nd.

Complete your session evaluation online now (open a browser

through our wireless network to access our portal) or visit one
of the Internet stations throughout the Convention Center.
Dont forget to activate your Cisco Live and Networkers Virtual
account for access to all session materials, communities, and
on-demand and live activities throughout the year. Activate
your account at any internet station or visit
www.ciscolivevirtual.com.

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

106

Visit the Cisco Store for

Related Titles
http://theciscostores.com

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

107

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

108

Thank you.

BRKSAN-3707

2011 Cisco and/or its affiliates. All rights reserved.

Cisco Public

109