V11.

cover

Front cover
Course Guide
Storage Area Networking Fundamentals
Course code AHQVSN7 ERC 12.0

IBM Systems Software Education

January 2017 edition
Notices
This information was developed for products and services offered in the US.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative
for information on the products and services currently available in your area. Any reference to an IBM product, program, or service is not
intended to state or imply that only that IBM product, program, or service may be used. Any functionally equivalent product, program, or
service that does not infringe any IBM intellectual property right may be used instead. However, it is the user's responsibility to evaluate
and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this
document does not grant you any license to these patents. You can send license inquiries, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive, MD-NC119
Armonk, NY 10504-1785
United States of America
INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY
KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some jurisdictions do not allow disclaimer
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein;
these changes will be incorporated in new editions of the publication. IBM may make improvements and/or changes in the product(s)
and/or the program(s) described in this publication at any time without notice.
Any references in this information to non-IBM websites are provided for convenience only and do not in any manner serve as an
endorsement of those websites. The materials at those websites are not part of the materials for this IBM product and use of those
websites is at your own risk.
IBM may use or distribute any of the information you provide in any way it believes appropriate without incurring any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other
publicly available sources. IBM has not tested those products and cannot confirm the accuracy of performance, compatibility or any other
claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those
products.
This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible,
the examples include the names of individuals, companies, brands, and products. All of these names are fictitious and any similarity to
actual people or business enterprises is entirely coincidental.
Trademarks
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corp., registered in many
jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM
trademarks is available on the web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml.
© Copyright International Business Machines Corporation 2001, 2017.
This document may not be reproduced in whole or in part without the prior written permission of IBM.
US Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
V11.0
Contents

TOC

Contents
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

Course description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Agenda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii

Unit 1. SAN concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-1

Unit objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-2
SAN concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-3
SAN concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-4
50 years of storage technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-5
Storage challenges in business . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-6
Storing data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-7
RAID 1 mirrored array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-8
RAID 0 striped array . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-9
RAID 5 block-level stripe with distributed parity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-10
RAID 10 mirror with stripe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-11
RAID50 distributed parity with striping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-12
Storage system components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-13
SAN concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14
Small Computer System Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-15
Parallel SCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-16
Storage architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-17
Direct attached storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-18
Direct attached storage data access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-19
Direct attached storage issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-20
Network attached storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-21
Network attached storage data access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-22
Storage area network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-23
Storage area network data access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-24
SAN usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-25
SAN benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-26
Storage architecture components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-27
SAN concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-28
Logical protocol and physical interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-29
Physical interfaces and topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-30
SAN attached storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-32
SAN with other technologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-33
Fibre Channel enables SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-34
Fibre Channel layered architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-35
SAN infrastructure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-36
SAN fabric interconnect components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-37
Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-38
Review questions (1 of 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-39
Review questions (2 of 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-40
Review questions (3 of 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-41
Unit summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1-42

Unit 2. Fibre Channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-1

Unit objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-2

© Copyright IBM Corp. 2001, 2017 iii

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Contents

TOC Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-3

Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-4
Fibre Channel introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-5
Fibre Channel characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-6
Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-8
Parallel SCSI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-9
Parallel SCSI limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-10
SCSI-3 interface evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-12
Fibre Channel advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-13
Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-14
Fibre Channel structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-15
Compare Fibre Channel to OSI layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-16
Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-17
FC-0 layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-18
FC-0 layer Open Fibre Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-19
Fiber optic technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-20
Optical cable options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-21
FC-0 layer interface to host bus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-22
Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-23
FC-1 layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-24
8b/10b encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-25
8b/10b encoding tables and rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-26
Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-27
FC-2 layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-28
Ordered sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-29
Fibre Channel frame structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-30
Frame, sequence, and exchange . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-32
Fibre Channel flow control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-33
Buffer-to-buffer credit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-34
End-to-end credit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-35
Flow control performance considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-36
Fibre Channel classes of service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-37
Class 1 service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-38
Class 2 service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-39
Class 3 service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-40
Specific class: Class F . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-41
Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-42
FC-3 layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-43
Fibre Channel topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-45
FC-4 layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-46
Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-47
Review questions (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-48
Review questions (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-49
Unit summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2-50

Unit 3. SAN design and topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-1

Unit objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-2
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-3
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-4
SAN components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-5
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-7
Fibre Channel topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-8
Point-to-point topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-9
Arbitrated loop topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-10
Switched fabric topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-11
Single switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-13

© Copyright IBM Corp. 2001, 2017 iv

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Contents

TOC Cascade and ring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-14

Full mesh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-15
Core-edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-16
Edge-core-edge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-17
Comparison between FC topologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-18
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-19
Common switch ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-20
Expansion switch ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-22
Diagnostic ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-23
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-24
Name and addressing convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-25
Worldwide name addressing scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-26
WWNN and WWPN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-27
Port address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-28
24-bit port address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-29
24-bit port loop address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-30
IBM b-type address modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-31
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-32
Port initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-33
Fabric login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-34
Port login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-35
Process login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-36
Fabric services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-37
Name Server database . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-39
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-40
Fibre Channel zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-41
Zoning concepts and terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-42
Zone example configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-43
Port zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-44
WWN zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-45
LUN masking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-46
Zoning best practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-47
SAN design and topology topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-48
N_Port ID Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-49
N_Port Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-50
Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-51
Review questions (1 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-52
Review questions (2 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-53
Review questions (3 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-54
Review questions (4 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-55
Unit summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3-56

Unit 4. IBM Fibre Channel b-type switches and directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-1

Unit objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-2
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-3
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-4
Fibre Channel switches and directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-5
Planning for FC switches and directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-6
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-7
IBM System Storage SAN b-type product portfolio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-8
Naming convention: IBM and Brocade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-10
BROCADE ASIC overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-11
SAN Director configuration component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-12
SAN768B-2 configuration overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-13
SAN384B-2 configuration overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-14
Control processor blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-15

© Copyright IBM Corp. 2001, 2017 v

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Contents

TOC Core router blade . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-16

Inter chassis link example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-17
Port blades options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-18
Blade architecture example: FC16-48 blade on SAN768B-2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-19
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-20
SAN switches and directors setup overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-21
SAN switch management interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-22
Management interfaces using IP network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-23
Log in to the switch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-24
Useful switch OS commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-25
Display switches in the fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-26
Display switch and port status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-27
Display switch operating parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-28
Switch port ID format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-29
Display nodes in a given name server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-31
Web Tools interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-32
Fabric tree and switch view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-33
Name Server view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-34
Name Server entry detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-35
Switch view SAN768B-2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-36
Director high availability CP status . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-37
Switch events log . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-38
Port admin view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-39
Basic performance monitoring options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-40
Fabric Watch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-41
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-42
Authority for administration and zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-43
Switch configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-44
Switch information report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-45
Network configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-46
Firmware download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-47
Hot code activation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-48
SNMP administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-50
License administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-51
Port administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-52
User administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-53
Configure switch parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-54
Configuration upload/download . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-55
RADIUS Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-56
Trace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-57
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-58
Switch: Port interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-59
Switch port initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-60
E_Port functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-61
F_Port functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-62
FL_Port functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-63
Name Server: Distributed model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-64
Name Server: Functional view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-65
Name Server: Switch view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-66
Dual fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-67
Multi-switch fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-68
Fault-tolerant fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-69
Multiple inter-switch links . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-70
Cross-connected fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-71
Exercise: To cascade or not? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-72
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-73

© Copyright IBM Corp. 2001, 2017 vi

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Contents

TOC Routing mechanisms: Fabric shortest path first . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-74

Fabric shortest path first . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-75
FSPF: Link cost metric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-76
FSPF: Load sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-77
Display all paths to domains: topologyShow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-78
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-79
Inter-switch link trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-80
Enabling ports for trunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-81
Trunking: CLI output examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-82
Identify route using pathinfo command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-83
Domain 1 islShow and trunkShow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-84
Identify route GUI: Host to device (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-85
Identify route GUI: Host to device (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-86
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-87
Fibre Channel switch zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-88
Zoning concepts and terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-89
Zone example configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-90
Zoning implementation steps in a nutshell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-91
Zoning administration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-92
Create new alias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-93
Create new zone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-94
Search for member in selection list . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-95
Create a zone configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-96
Zone configuration action options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-97
View effective configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-98
Zoning: CLI command examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-99
Adding a new switch to the fabric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-100
Merging fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-101
Domain overlap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-102
Splitting fabrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-103
SAN extension switches over IP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-104
IBM b-type switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-105
IBM Network Advisor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-106
IBM Network Advisor features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-107
IBM Network Advisor benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-108
IBM Network Advisor dashboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-109
Widgets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-110
Discovery tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-111
SAN tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-112
IP tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-113
Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-114
Review questions (1 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-115
Review questions (2 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-116
Review questions (3 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-117
Review questions (4 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-118
Unit summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4-119

Unit 5. Cisco MDS Fibre Channel switches and directors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-1

Unit objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-2
Cisco MDS Fibre Channel switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-3
Cisco MDS Fibre Channel switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-4
Cisco MDS 9000 family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-5
Cisco MDS 9000 Enterprise SAN environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-6
Cisco MDS 9700 series modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-8
MDS terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-9
Cisco-specific Fibre Channel port types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-10

© Copyright IBM Corp. 2001, 2017 vii

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Contents

TOC Cisco MDS Fibre Channel switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-11
Management interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-12
CLI command hierarchy and structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-13
Basic system configuration: Switch setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-14
Role-based access for CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-17
Port interfaces and configure port examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-20
Cisco MDS Fibre Channel switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-24
Cisco Data Center Network Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-25
Install Cisco Data Center Network Manager . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-26
Cisco DCNM login . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-27
Cisco DCNM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-28
Cisco Device Manager launch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-29
Cisco Device Manager summary tab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-30
DCNM switch physical view . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-31
Cisco MDS Fibre Channel switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-32
VSAN introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-33
Cisco virtual SAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-34
VSANs and trunking TE_Ports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-35
Create a VSAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-36
Assign ports to VSAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-37
Cisco MDS Fibre Channel switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-38
DCNM ISL merge analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-39
ISL connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-40
Trunking and port channel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-41
Define a port channel (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-42
Define a port channel (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-43
Cisco MDS Fibre Channel switches and directors topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-44
Zoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-45
Zone configuration components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-46
Define WWPN resource zone aliases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-47
Define other resource zone aliases (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-48
Define other resource zone alias (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-49
Define a zone (1 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-50
Define a zone (2 of 2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-51
Define a zoneset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-52
Activate a zoneset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-53
Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-54
Review questions (1 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-55
Review questions (2 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-56
Review questions (3 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-57
Review questions (4 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-58
Unit summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5-59

Unit 6. SAN over Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-1

Unit objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-2
SAN over Ethernet topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-3
SAN over Ethernet topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-4
Ethernet concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-5
Ethernet operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-7
SAN over Ethernet topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-8
TCP/IP concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-9
TCP protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-11
IP protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-13
SAN over Ethernet topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-14
iSCSI introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-15
iSCSI frame encapsulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-16

© Copyright IBM Corp. 2001, 2017 viii

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Contents

TOC iSCSI protocol layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-17

iSCSI components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-19
iSCSI name discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-20
Performance recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-22
iSCSI initiator comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-23
iSCSI benefits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-24
SAN over Ethernet topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-25
Converged Enhanced Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-26
Priority-based Flow Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-27
Enhanced Transmission Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-28
Data Center Bridging Exchange protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-29
SAN over Ethernet topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-30
Fibre Channel over Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-31
FCoE topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-32
FCoE protocol layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-33
FCoE packet comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-34
FCoE components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-35
FCoE fabric components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-36
FCoE initialization flow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-38
Keywords . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-39
Review questions (1 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-40
Review questions (2 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-41
Review questions (3 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-42
Review questions (4 of 4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-43
Unit summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6-44

Appendix A. Review answers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A-1

© Copyright IBM Corp. 2001, 2017 ix

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Trademarks

TMK

Trademarks
The reader should recognize that the following terms, which appear in the content of this training
document, are official trademarks of IBM or other companies:
IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business
Machines Corp., registered in many jurisdictions worldwide.
The following are trademarks of International Business Machines Corporation, registered in many
jurisdictions worldwide:
AIX 6™ AIX® Express®
FICON® Initiate® NetView®
Power Systems™ Power® Redbooks®
System Storage DS® System Storage® System z®
Tivoli® z/OS® 400®
Linear Tape-Open, LTO and Ultrium are trademarks of HP, IBM Corp. and Quantum in the U.S. and
other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other
countries, or both.
Java™ and all Java-based trademarks and logos are trademarks or registered trademarks of
Oracle and/or its affiliates.
UNIX is a registered trademark of The Open Group in the United States and other countries.
VMware and the VMware “boxes” logo and design, Virtual SMP and VMotion are registered
trademarks or trademarks (the “Marks”) of VMware, Inc. in the United States and/or other
jurisdictions.
Other product and service names might be trademarks of IBM or other companies.

© Copyright IBM Corp. 2001, 2017 x

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Course description

pref

Course description
Storage Area Networking Fundamentals

Duration: 4 days

Purpose
IBM offers a comprehensive portfolio of SAN switches, storage, software, services, and solutions to
reliably bring information to people in a cost effective way. IBM provides flexible, scalable, and open
standards-based business-class and global enterprise-class storage networking solutions for the
on demand world.
This course provides an overview of storage network and data center networking technology. It
reviews SAN concepts, Fibre Channel architecture, SAN topologies, IBM b-type offerings, IBM
offerings from Cisco, and SAN over Ethernet architectures, such as iSCSI and FCoE.

Audience
This is a base course for individuals who are involved in the planning, installing, configuring, and
upgrading of IBM Storage Systems.

Prerequisites
The following course, or equivalent knowledge, is required prior to this course:
• IBM Storage Fundamentals (SS01G)

Objectives
After completing this course, you should be able to:
• Classify the components of a SAN infrastructure
• Appraise the Fibre Channel architecture and terms
• Classify the three major components of a SAN
• Differentiate the features of the IBM Storage System b-type switch offerings
• Differentiate the features of the IBM offered Cisco SAN switches and directors
• Evaluate the implementations of iSCSI and FCoE

© Copyright IBM Corp. 2001, 2017 xi

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Course description

pref
Contents
SAN concepts
Fibre Channel
SAN design and topology
IBM Fibre Channel b-type switches and directors
Cisco MDS Fibre Channel switches and directors
SAN over Ethernet
Exercises

© Copyright IBM Corp. 2001, 2017 xii

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Agenda

pref

Agenda

Note

The following unit and exercise durations are estimates, and might not reflect every class
experience.

Day 1
(00:30) Welcome
(02:00) Unit 1 - SAN concepts
(02:00) Unit 2 - Fibre Channel
(01:30) Unit 3 - SAN design and topology (1 of 2)

Day 2
(01:00) Unit 3 - SAN design and topology (2 of 2) (continued)
(05:30) Unit 4 - IBM Fibre Channel b-type switches and directors

Day 3
(00:15) Exercise 0 - Lab introduction
(00:30) Exercise 1 - Switch management review
(00:45) Exercise 2 - Zone configuration using Web Tools
(00:45) Exercise 3 - Configure a Spectrum Virtualize storage system
(01:00) Exercise 4 - Zone configuration with CLI
(00:45) Exercise 5 - SAN Domain Fabrics
(00:30) Exercise 6 - Boot from SAN
(02:00) Unit 5 - Cisco MDS Fibre Channel switches and directors (1 of 2)

Day 4
(00:30) Unit 5 - Cisco MDS Fibre Channel switches and directors (2 of 2) (continued)
(02:00) Unit 6 - SAN over Ethernet
(00:30) Exercise 7 - Cisco MDS basic configuration
(00:30) Exercise 8 - Cisco management tool installation
(00:30) Exercise 9 - VSAN configuration
(00:30) Exercise 10 - Define zone configuration on Cisco
(00:30) Exercise 11 - Configuring the DS3524 storage device
(00:30) Exercise 12 - iSCSI configuration

© Copyright IBM Corp. 2001, 2017 xiii

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Unit 1. SAN concepts

Estimated time
02:00

Overview
The Storage Networking Industry Association (SNIA) defines the storage area network (SAN) as a
network whose primary purpose is the transfer of data between computer systems and storage
elements. A SAN consists of a communication infrastructure, which provides physical connections.
It also includes a management layer, which organizes the connections, storage elements, and
computer systems so that data transfer is secure and robust.
This unit defines the components of a storage system and reviews the types of storage network
technologies. SAN topologies and SAN infrastructure components are discussed.

How you will check your progress

• Review questions

References
SG24-5470 Introduction to Storage Area Networks and System Networking
http://www.ibm.com/systems/storage/san/

© Copyright IBM Corp. 2001, 2017 1-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Unit objectives
• Evaluate the components of a storage system
• Classify the components of a SAN infrastructure
• Differentiate between DAS, NAS, and SAN technologies
• Discuss SAN implementation benefits
• Differentiate between storage network topologies

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-1. Unit objectives

© Copyright IBM Corp. 2001, 2017 1-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN concepts
• Storage system components
• Storage system interconnection
ƒ DAS, NAS, SAN
• Storage area network
ƒ SAN topology and technology

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-2. SAN concepts

© Copyright IBM Corp. 2001, 2017 1-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN concepts
• Storage system components
• Storage system interconnection
– DAS, NAS, SAN
• Storage area network
– SAN topology and technology

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-3. SAN concepts

© Copyright IBM Corp. 2001, 2017 1-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

50 years of storage technology

IBM 726 Tape Unit IBM RAMAC 350

700 GB 3592 Media 300 GB 15 K FC Disk
4.9”
5.7”
4”
4.3”
SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-4. 50 years of storage technology

In 1952, the IBM 726 Tape unit operated at 7,500 characters per second which was 56 times faster
than the rate of a punch card reader. With a working density of 100 bits per inch on a half-inch wide
tape, a 10.5 inch diameter reel could hold the equivalent of more than 35,000 punched cards.
In 2002, the IBM Ultrium LTO (Linear Tape-Open) tape cartridge was 1,000 times faster, and able to
store more information in one inch of tape (without data compression) than the 726 tape drive was
capable of storing on over 500 feet of tape. LTO storage provides more than several hundred
terabytes (TB) of uncompressed capacity in a fully automated IBM tape library and can fit it in the
space equivalent to eight five-drawer file cabinets. The IBM Ultrium LTO tape cartridge records
more than 124 Kbits per inch. Densities of over 250 Kbits per inch are feasible in the future.
In 1956, the first disk drive, the IBM 350 disk storage unit, was announced by IBM as part of the
IBM 305 RAMAC computer system. The IBM 350 Disk Storage Unit consisted of 50 twenty-four
inch diameter magnetic disks unit, an electronic and pneumatic controlled access mechanism, and
a small air compressor. The IBM 350 was 60 inches long, 68 inches high, and 29 inches deep. The
IBM 350 capacity was 5 million characters.
Disk capacities today are reaching into the terabyte range on a single drive and in a fraction of the
space that was taken by the IBM 350.

© Copyright IBM Corp. 2001, 2017 1-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Storage challenges in business

• Data explosion
ƒ Management and analysis
• Business continuity
ƒ High availability and disaster recovery solutions
• Consolidation
ƒ Simplify architecture and maximize performance
• Ease of use
ƒ Simplified management for complex storage infrastructure

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-5. Storage challenges in business

Data storage requirements continue to grow. According to estimates, the volume of business data
worldwide across all companies doubles every 1.2 years. Managing and analyzing this big data
pool is a major business concern.
Access to storage in the event of a device or data center outage is critical in today’s continuous
availability computing environment.
Consolidation and virtualization of storage is part of a general trend to make more efficient use of
data center storage in order to accommodate increasing volumes of data and reduce costs.
Techniques such as de-duplication, thin provisioning, and automatic tiered storage can be
employed to help.
A complex storage networking infrastructure can be challenging if there are multiple management
tools required for the various resources.

© Copyright IBM Corp. 2001, 2017 1-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Storing data
• HDD can store gigabytes
of data
• Data unavailable if HDD
is not accessible
• Secondary HDD provides
a backup copy location
A

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-6. Storing data

Data is stored on a hard disk drive (HDD) on which can be read and written. Depending on the
methods that are used to run those tasks, and the HDD technology on which they were built, the
read and write function can be faster or slower. Today, hundreds of gigabytes can be stored on a
single HDD which allows one to keep all the data we can ever imagine. Even though this approach
seems to bring only advantages so far, what happens if for any reason the HDD cannot be
accessed?
A solution might be to have a secondary HDD and to manually copy the primary HDD to the
secondary HDD. Immediately the data can be viewed as being safe. But, how often must the
manual copies be done so as not to lose data and to keep it as up-to-date as possible? To keep it
as current as possible, every change would require another copy. The amount of data copied must
also be decided to be either the entire HDD or just what changed.

© Copyright IBM Corp. 2001, 2017 1-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

RAID 1 mirrored array

• RAID 1 technology provides
automatic data redundancy
• RAID 1 writes data to each A
of two HDDs B
• RAID 1 array also known RAID Controller
C
as mirrored disk
• RAID 1 improves read
performance by using
both HDDs
A A

B B

C C

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-7. RAID 1 mirrored array

Data needs to be copied every time that it changes to provide a reliable fault tolerant system. It also
cannot be done in a manual way. A Redundant Array of Independent Disks (RAID) controller can
maintain disks in synchronization and can also manage all the writes and reads (I/O) to and from
the disks. Disks in a RAID configuration are called a disk array.
RAID 1 consists of an exact copy (or mirror) of a set of data on two or more disks. A classic RAID 1
mirrored pair contains two disks. This configuration offers no parity, striping, or spanning of disk
space across multiple disks, since the data is mirrored on all disks belonging to the array, and the
array can only be as big as the smallest member disk.
RAID 1 configurations can improve availability in that if one HDD fails, then the data copy on the
other HDDs is still accessible. Although RAID 1 has the advantage of data redundancy in the event
of a disk failure, it does have disadvantages in that only half of the total HDD capacity is available
and that the write performance is slower since it must do multiple writes operations each time.

© Copyright IBM Corp. 2001, 2017 1-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

RAID 0 striped array

• RAID 0 implements a striped disk array.
• Each block of data is written to a separate disk drive.
Disk 1 Disk 2 Disk 3

Block A Block B Block C

Block D Block E Block F
Block G Block H Block I
Block J Block K Block L

• Same size required on all disk drives:

ƒ 40 GB + 40 GB + 40 GB = Logical disk of 120 GB
ƒ 40 GB + 20 GB + 40 GB = Logical disk of 60 GB
• One disk failure results in loss of all data in the array.

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-8. RAID 0 striped array

RAID 0 splits data evenly across two or more disks, without parity information, redundancy, or fault
tolerance. Since RAID 0 provides no fault tolerance or redundancy, the failure of one drive will
cause the entire array to fail. Therefore, as a result of having data striped across all disks, the
failure will result in total data loss. RAID 0 is normally used to increase performance, although it can
also be used as a way to create a large logical volume out of two or more physical disks.
A RAID 0 setup can be created with disks of differing sizes, but the storage space added to the
array by each disk is limited to the size of the smallest disk. For example, if a 120 GB disk is striped
together with a 320 GB disk, the size of the array will be 240 GB (120 GB × 2).

© Copyright IBM Corp. 2001, 2017 1-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

RAID 5 block-level stripe with distributed parity

• Most common secure RAID level.
• Read data transactions are faster.
Disk 1 Disk 2 Disk 3 Disk 4

Block A Block B Block C Parity ABC

Block D Block E Parity DEF Block F
Block G Parity GHI Block H Block I
Parity JKL Block J Block K Block L

• Minimum of three disks for a RAID 5 array

ƒ Three drives of 100 GB = 200 GB available
ƒ 10 drives of 100 GB = 900 GB available.
• Write data transactions are slower due to parity calculation

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-9. RAID 5 block-level stripe with distributed parity

RAID 5 consists of block-level striping with distributed parity. The parity information is distributed
among the drives. It requires that all drives but one be present to operate. Upon failure of a single
drive, subsequent reads can be calculated from the distributed parity such that no data is lost.
RAID 5 requires at least three disks. RAID 5 can also be defined with a spare disk. If one of the
disks in the array fails, then the blocks on the failed disk are rebuilt on the spare.
Read performance is increased since all RAID members participate in serving of the read requests.

© Copyright IBM Corp. 2001, 2017 1-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

RAID 10 mirror with stripe

• A stripe of mirrors
• Each RAID 1 cluster has a minimum of two drives
RAID 0

RAID 1 RAID 1

Disk 1 Disk 2 Disk 3 Disk 4

Block A Block A Block B Block B

Block C Block C Block D Block D
Block E Block E Block F Block F
Block G Block G Block H Block H

• Minimum of four drives to implement

• Fault-tolerance improved

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-10. RAID 10 mirror with stripe

RAID 10, also called RAID 1+0 and sometimes RAID 1&0, is a stripe of mirrors achieving both
replication and sharing of data between disks. The usable capacity of a RAID 10 array is the same
as in a RAID 1 array made of the same drives, in which one half of the drives is used to mirror the
other half.
RAID 10 is a RAID 0 array of mirrors and requires a minimum of four drives. Implementations
supporting two disks such as Linux RAID 10 offer a choice of layouts. Arrays of more than four
disks are also possible. Under certain circumstances, it can sustain multiple simultaneous drive
failures (1 per RAID 1 cluster).
In most cases RAID 10 provides better throughput and latency than all other RAID levels except
RAID 0 (which wins in throughput). Thus, it is the preferable RAID level for I/O-intensive
applications such as database, email, and web servers, as well as for any other use requiring high
disk performance.

© Copyright IBM Corp. 2001, 2017 1-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

RAID50 distributed parity with striping

• Combines RAID 5 with RAID 0
ƒ Distributed parity with striping
• Each RAID cluster has minimum
of three drives
• Minimum of six drives to implement RAID Controller
• Offers high reliability and performance

RAID 0

RAID 5 RAID 5

Block A Block B Parity AB Block C Block D Parity CD

Block E Parity EF Block F Block G Parity GH Block H
Parity IJ Block I Block J Parity KL Block K Block L

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-11. RAID50 distributed parity with striping

RAID 50 has features of both RAID 5 (distributed parity) and RAID 0 (striping). It is composed of
two groups of RAID 5 (containing at least three disks in each group) and each group uses
distributed parity bit operation. These two groups of disks form a RAID 0 array, realizing cross-disk
extraction of data. RAID 50 provides reliable data storage and excellent overall performance and
supports larger volumes. If two physical disks (one in each array) fail at the same time the data can
be recovered. In RAID 50, at least six disk drives are needed. It is most suitable for applications
which need high reliability storage, high read-write speed, and high data transmission performance.

© Copyright IBM Corp. 2001, 2017 1-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Storage system components

Processors

RAID
Controllers RAID Controller RAID Controller
RAID 0

RAID 5 RAID 5
HDDs

Block A Block B Parity AB Block C Block D Parity CD

Block E Parity EF Block F Block G Parity GH Block H
Parity IJ Block I Block J Parity KL Block K Block L

Network
Interface HBA
- FC
SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-12. Storage system components

In order to build a basic independent storage system there are two additional components to
include. One of the components is a CPU which processes all the required instructions to allow
data to flow. Adding only one CPU creates a single point of failure (SPOF) and so there are two
CPUs that are added. Two CPUs (or controllers) also provide the ability for load balancing and
therefore improving performance.
Another component needed to build an independent storage system is to support communications
with other systems in a network. Therefore, a minimum of two network interfaces, known as Host
Bus Adapters (HBA), are included to avoid a SPOF. A single host server can access both HBAs to
increase availability and performance, but would need a multipath driver to manage the data
access.
Finally, all these hardware components are placed into a single cabinet, called an enclosure.
Availability can be enhanced for the disk drives by implementing features such as hot-swap disks,
hot spare RAID disks, and cache memory.

© Copyright IBM Corp. 2001, 2017 1-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN concepts
• Storage system components
• Storage system interconnection
– DAS, NAS, SAN
• Storage area network
– SAN topology and technology

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-13. SAN concepts

© Copyright IBM Corp. 2001, 2017 1-14

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Small Computer System Interface

• ANSI standards for physical transfer of data between computers and
peripheral devices
• Defines commands, protocols, and electrical / optical interfaces
• Addressing includes SCSI ID and LUN
• Bus architecture maximum limits are 8 or 16 devices
• SCSI initiator sends commands to SCSI target

Bus
HBA Controller
6-25 M LUN LUN LUN
Host
Initiator Target LUN=(disk/tape)

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-14. Small Computer System Interface

Small Computer System Interface (SCSI) is a set of standards for physically connecting and
transferring data between computers and peripheral devices. The SCSI standards define
commands, protocols, and electrical and optical interfaces. SCSI is most commonly used for hard
disk drives and tape drives, but it can connect a wide range of other devices, including scanners
and CD drives, although not all controllers can handle all devices.
Each device on the SCSI bus is assigned a unique SCSI identification number or ID. Devices can
encompass multiple logical units, which are addressed by logical unit number (LUN). Simple
devices have just one LUN, more complex device might have multiple LUNs.
On a parallel SCSI bus, a device (for example, host adapter, disk drive) is identified by a “SCSI ID”,
which is a number in the range 0–7 on a narrow bus and in the range 0–15 on a wide bus.
The SCSI standards also include an extensive set of command definitions. In SCSI terminology,
communication takes place between an initiator and a target. The initiator sends a command to the
target, which then responds.

© Copyright IBM Corp. 2001, 2017 1-15

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Parallel SCSI

Bus
HBA Controller
6-25 M LUN LUN LUN
Host
Initiator Target LUN=(disk/tape)

SCSI bus type 8-bit width 16-bit width

(narrow) (wide)
SCSI-1 5 MB/s
SCSI-2 Fast 10 MB/s 20 MB/s
SCSI-3 Ultra 20 MB/s 40 MB/s
SCSI-3 Ultra2 40 MB/s 80 MB/s
SCSI-3 Ultra3 (Ultra160) 160 MB/s
SCSI-3 Ultra320 320 MB/s
SCSI-3 Ultra640 640 MB/s

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-15. Parallel SCSI

SCSI is a fully documented ANSI standard. The first SCSI standard was created in 1986 and is
called SCSI-1. Several iterations of the standard have occurred due to technological advances and
marketplace demands. These changes have resulted in SCSI-2 and SCSI-3.
The SCSI architecture defines both software and hardware physical transport. The software
protocol specifies the commands and the controls needed for data exchange. The original physical
transport was a parallel cable that transmitted 8 bits of data in parallel. Over time, the quest for
higher bandwidth brought about wider cables that deliver 16 bits or 32 bits of data in parallel.
A SCSI bus is connected to the host with a host bus adapter (HBA). The HBA is often called the
SCSI adapter or SCSI controller. Typically, it is the initiator of SCSI commands. An initiator
determines the task to be performed on the target which is the device controller.
Up to 16 device controllers, or targets, can be attached to the same bus, one of which is the host
HBA. A target responds to the initiator requests and carries out the tasks. The SCSI bus address for
each device is referred to as the SCSI-ID.
SCSI devices can assume the role of an initiator or that of a target. More devices are made with the
capability to assume either role. This facilitates the development of outboard copy applications.

© Copyright IBM Corp. 2001, 2017 1-16

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Storage architectures
• Direct attached storage
ƒ Dedicated storage
• Network attached storage
ƒ Share files over TCP/IP
• Storage area networks
ƒ Centrally managed, shared storage
ƒ Dedicated storage network

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-16. Storage architectures

Direct attached storage (DAS) is for customers who prefer dedicated storage at a lower cost than a
SAN.
Network attached storage (NAS) is for customers in midrange environments who want to share files
on an existing TCP/IP network.
Storage area network (SAN) is for customers who want to centrally manage shared storage in a
dedicated network.

© Copyright IBM Corp. 2001, 2017 1-17

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Direct attached storage

• Storage is accessed directly from the server host
ƒ Internal or external from server
• Data access is at block level
• The server communicates directly
to the DAS storage device
• Storage device is addressed
using SCSI ID Block Block Block
level level level

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-17. Direct attached storage

Direct-attached storage (DAS) is storage that is directly accessible only from the host to which the
DAS is attached. The most familiar example of DAS is the internal hard drive in a laptop or desktop
PC. DAS devices can be housed inside a server host or outside a server (as is the case with
external hard drives and storage appliances). Multiple systems can use the same DAS device, as
long as each PC or server has a separate connection to the storage device.
In a block level storage device, raw storage volumes are created. Then the server-based operating
system connects to these volumes using their particular file system software and uses them as
individual hard drives.

© Copyright IBM Corp. 2001, 2017 1-18

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Direct attached storage data access

• Application requests file I/O
• File system software translates request
• Commands are sent to the physical storage Application
device to retrieve the data
ƒ SCSI architecture and commands are File system
employed in most server based systems
Device driver
• Data blocks are retrieved from
storage device SCSI bus

• Communications is through direct

connection to the storage device
ƒ Internal or external Block I/O

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-18. Direct attached storage data access

Data I/O requests start out as a file I/O requests from the application on a server. The request is
handled by the file system software in the server. The file I/O request is translated to block I/O
requests. SCSI commands are delivered directly to the physical storage device in order to complete
the I/O request.
DAS communications is done by using block level requests directly between the application server
and the storage device through either external or internal cabling to the storage device.

© Copyright IBM Corp. 2001, 2017 1-19

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Direct attached storage issues

• Islands of storage
ƒ Inefficient resource usage
ƒ Difficult to scale capacity

• SCSI limitations
ƒ Cable, device addressing, performance

• Difficult to manage overall storage

ƒ Different file systems, backup/recovery, data sharing

• Inconsistent copies of information

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-19. Direct attached storage issues

As server farms grow, DAS storage becomes more challenging to manage. Storage usage can vary
significantly from server to server causing inefficient resource usage and difficulty to scale the
require capacity. The SCSI implementation for storage can be limiting because of cabling, device
addressing, and performance. Storage formats can vary due to differing files systems which can
impact procedures for data sharing and backup/recovery. Servers that need to access common
data could be subject to inconsistent copies of information between them.

© Copyright IBM Corp. 2001, 2017 1-20

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Network attached storage

• Storage is accessed using specialized appliances over a TCP/IP
network
• Client/server model is employed
ƒ Examples include NFS and CIFS
• Supports multiple clients
• Data access is at file level
• The server communicates using TCP/IP File
to the NAS storage device level
• NAS appliance accesses storage local
to the system
• NAS gateway accesses storage located
external to the system
• Storage device is addressed using TCP/IP socket

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-20. Network attached storage

Network attached storage (NAS) is a file-level computer data storage server connected to a TCP/IP
network providing data access to a heterogeneous group of clients. NAS is specialized for serving
files either by its hardware, software, or configuration. It is often manufactured as a computer
appliance.
NAS systems contain one or more hard drives, often arranged into RAID arrays. Network-attached
storage removes the responsibility of file serving from other servers on the network. File-based
protocols such as Network File System (NFS), popular on UNIX systems, or Server Message
Block/Common Internet File System (SMB/CIFS), for Microsoft Windows systems, are used by
NAS systems.
A NAS system is a streamlined file server whose storage is either integrated in the server itself
(NAS appliance) or whose storage is located elsewhere, such as on a SAN (NAS gateway).

© Copyright IBM Corp. 2001, 2017 1-21

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Network attached storage data access Application

• Application requests file I/O File system
• File system software translates request NFS/CIFS
ƒ File system redirects through NFS/CIFS
TCP/IP
• TCP/IP packet is delivered to NAS server NIC
• NFS/CIFS redirects to file system software
File I/O
• Commands are sent to the physical storage IP Network
device to retrieve the data
ƒ SCSI architecture and commands are NIC
employed in most server based systems TCP/IP
• Data blocks are retrieved from the NFS/CIFS
storage device
File system
• Communications is through a TCP/IP
connection to the file server Device driver
I/O HBA

Block I/O

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-21. Network attached storage data access

Data I/O requests start out as a file I/O requests from the application on a server. The request is
handled by the file system software in the server. The file I/O request is redirected to the CIFS or
NFS client software. A TCP/IP packet is created and delivered to the NAS server. The CIFS or NFS
server software passes the request to the NAS file system. The file I/O request is translated to
block I/O requests. SCSI commands are delivered directly to the physical storage device in order to
complete the I/O request. The data is repackaged through CIFS or NFS to deliver it back to the
requesting application server.
NAS communications is done by using file level requests sent across a TCP/IP network between
the application server and the NAS server. The NAS server then issues a block level requests
directly to the storage device through either external or internal cabling to the storage device.

© Copyright IBM Corp. 2001, 2017 1-22

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Storage area network

• Storage is accessed directly through a specialized, high speed, highly
available network from the host
• Data access is at block level
• The server communicates using a SAN
protocol to the storage device
ƒ Generally, Fibre Channel protocol
• Supports any to any connection
• Storage device is addressed using
worldwide name
ƒ Fibre Channel based
Block
level

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-22. Storage area network

Storage area network (SAN) is a separate network that is dedicated to storage and uses mostly
Fibre Channel to provide scalable bandwidth and flexible connectivity. A SAN allows an any-to-any
connection across the network, by using interconnect elements such as switches and directors. It
eliminates the traditional dedicated connection between a server and storage, and the concept that
the server effectively owns and manages the storage devices. It also eliminates any restriction to
the amount of data that a server can access, currently limited by the number of storage devices that
are attached to the individual server.
This concept enables storage devices and servers to be interconnected by using similar elements,
such as LANs and wide area networks (WANs). SANs also enable new network architectures
where multiple hosts access multiple storage devices that are connected to the same network.
Fibre Channel devices are uniquely identified with a 64 bit address called the worldwide name
(WWN).

© Copyright IBM Corp. 2001, 2017 1-23

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Storage area network data access

• Application requests file I/O
• File system software translates request
Application
• Commands are sent to the SAN storage controller
to retrieve the data File system
ƒ SCSI architecture and commands are Device driver
encapsulated in Fibre Channel protocol
FC HBA
• Storage controller retrieves data blocks from
storage device
ƒ Data returned to server using Fibre Channel SAN Network
protocol
• Communications is through SAN network
connection to the storage device Block I/O

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-23. Storage area network data access

Data I/O requests start out as a file I/O requests from the application on a server. The request is
handled by the file system software in the server. The file I/O request is translated to block I/O
requests. SCSI commands are encapsulated in Fibre Channel protocol and delivered to the storage
controller. The storage controller accesses the blocks of data from the physical storage device that
represent the file I/O request and communicates back to the server by way of the SAN network in
order to complete the I/O request.
SAN communications is based on block level requests and can be view as an extension to the
storage bus concept.
Fibre Channel protocol is not the only protocol type possible in a SAN network, but it is currently the
most widely used.

© Copyright IBM Corp. 2001, 2017 1-24

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN usage
• Infrastructure simplification
ƒ Consolidation
ƒ Virtualization
ƒ Automation
ƒ Integration
• Information lifecycle management
• Business continuity

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-24. SAN usage

The key benefits that a storage area network (SAN) might bring to a highly data-dependent
business infrastructure can be summarized into three concepts: Infrastructure simplification,
information lifecycle management, and business continuity.
Simplified storage environments have fewer elements to manage. This type of environment leads
to increased resource utilization, simplifies storage management, and can provide economies of
scale for owning disk storage servers. These environments can be more resilient and provide an
infrastructure for virtualization and automation.
A SAN implementation makes it easier to manage the information lifecycle because it integrates
applications and data into a single-view system, in which the information resides. This single-view
location can be managed more efficiently.
By deploying a consistent and safe infrastructure, SANs make it possible to meet any availability
requirements including continuous operations, data security, regulatory requirements, and disaster
recovery.

© Copyright IBM Corp. 2001, 2017 1-25

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN benefits
• Improvements to application availability
ƒ Storage is independent of applications and accessible through multiple data
paths for better reliability, availability, and serviceability
• Higher application performance
ƒ Storage processing is offloaded from servers and moved onto a separate
network
• Centralized and consolidated storage
ƒ Simpler management, scalability, flexibility, and availability
• Data transfer and vaulting to remote sites
ƒ Remote copy of data that is enabled for disaster protection and against
malicious attacks
• Simplified centralized management
ƒ Single image of storage media simplifies management

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-25. SAN benefits

• Improvements to application availability: Storage is independent of applications and accessible

through multiple data paths for better reliability, availability, and serviceability.
• Higher application performance: Storage processing is offloaded from servers and moved onto
a separate network.
• Centralized and consolidated storage: Simpler management, scalability, flexibility, and
availability.
• Data transfer and vaulting to remote sites: Remote copy of data that is enabled for disaster
protection and against malicious attacks.
• Simplified centralized management: Single image of storage media simplifies management.

© Copyright IBM Corp. 2001, 2017 1-26

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Storage architecture components

Server Network Storage

DAS • Server HW • Direct cable • Controller + storage
• Device driver devices

NAS • Server HW • TCP/IP net • Controller + storage

• NAS client SW • ETH cables devices
• NIC • Switch/Rtr • NAS server SW
• NIC

SAN • Server HW • SAN switch • Controller + storage

• HBA • SAN cables devices
• Multipath SW • HBA

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-26. Storage architecture components

Storage architecture components include requirements for the server, the network (or
infrastructure), and the storage system.
For a DAS environment, the server requires the device driver for the specific storage system to
which it will connect. The infrastructure is an appropriate cable depending on if the storage device
is internal or external to the server. The storage system requires a controller and the storage device
(disk or tape), which may be housed together in the storage system.
For a NAS environment, the server requires the selected NAS client protocol software support,
such as NFS or CIFS, as well as an interface card for network access. The infrastructure is based
on TCP/IP which includes Ethernet cables and a set of switches and routers to support the
communications. The storage system requires a controller, the storage device, the NAS server
protocol software support, and an interface card for network access.
For a SAN environment, the server requires an interface card, called a host bus adapter (HBA), for
network access and a multipath software support for the storage system to provide redundant path
support. The infrastructure includes Fibre Channel switches and cabling to attach all the resources.
The storage system(s) requires a controller, the storage device, and an interface card (HBA) for
network access.

© Copyright IBM Corp. 2001, 2017 1-27

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN concepts
• Storage system components
• Storage system interconnection
ƒ DAS, NAS, SAN
• Storage area network
ƒ SAN topology and technology

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-27. SAN concepts

© Copyright IBM Corp. 2001, 2017 1-28

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Logical protocol and physical interface

• Host initiates I/O commands
• Storage device executes I/O commands
• Data result is returned to the host
• Interface between host and storage device consists of:
ƒ Data transfer protocol commands
ƒ Physical transport protocol

Host Device

SCSI Commands (logical protocol)

Disk
command Wiring (physical
Commands interface)
(logical protocol) Tape
Wiring (physical interface)
Initiate commands Execute commands

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-28. Logical protocol and physical interface

A host initiates an I/O command to access data which is transported to the external storage device.
The storage device executes the I/O command and returns the resulting data to the host over the
same transport.
The data communication interface between the host and the storage device is composed of two
components:
• A logical, or software protocol that specifies the commands and the controls that are needed for
data exchange.
• A physical, or hardware wiring protocol that defines the command transport mechanisms.
The original SCSI physical transport was a parallel copper cable that transmitted 8 bits of data in
parallel. Over time, wider parallel cables and fiber optics were integrated to improve performance
and addressability.

© Copyright IBM Corp. 2001, 2017 1-29

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Physical interfaces and topologies

SCSI BUS#, SCSI ID, LUN#

Open
systems SCSI BUS
zSystem
server

Initiator Target Target zSystem zSystem

LUNs LUNs ESCON or

FICON
Director

Controller Controller
AIX SSA loop

Devices

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-29. Physical interfaces and topologies

The device network technology defines the topology of the physical interface and controls factors
such as the bandwidth, the total number of devices, and the maximum distance supported.
Up to 16 device controllers, or targets, can be attached to the same parallel SCSI bus. However,
one of the controllers is the host SCSI adapter. A target responds to the initiator (host adapter)
requests and carries out the I/O tasks. The SCSI bus address for each device is referred to as the
SCSI ID. Each target (the peripheral controller) can control more than one device. These devices
are represented by logical unit numbers (LUNs). A LUN is a physical or logical peripheral device,
addressable through a target, or peripheral controller. A target accepts SCSI commands from the
initiator and forwards them to the identified LUN for execution. The LUN has the logic necessary for
the execution of SCSI commands. The number of LUNs associated with a target is
device-dependent. Also, the maximum LUNs a target can support varies with the operating system
platform.
A SCSI host initiator preparing to use a SCSI bus, chooses the target by changing the voltage on
the data wire representing that target ID. Thus, the SCSI ID is not part of the SCSI command or
data. The LUN number, which designates a specific logical volume that is associated with a target
ID, is carried in SCSI commands.
Parallel SCSI data transfer rates can vary from 5 to 640 MBps, depending on the level of the SCSI
protocol and the width of the bus cable. The maximum cable length is 25 meters.

© Copyright IBM Corp. 2001, 2017 1-30

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty
Serial Storage Architecture (SSA) was designed by IBM as an alternative storage interface to
parallel SCSI. Higher protocol layers were based on the SCSI-3 standard. An SSA connection
consists of two ports capable of four simultaneous conversations for an aggregated transmission
rate of 160 MBps. SSA nodes can be 25 meters apart.
System z servers use ESCON for an I/O network interface. One of the key elements of ESCON is
the ESCON Director, an I/O switch capable of providing dynamic, nonblocking, any-to-any
connectivity for up to 60 fiber optic links operating at 200 Mb/s. The links can be up to 36 miles (60
kilometers) apart. The data transfer rate for ESCON is 17 MBps. FICON replaces the network
interface of ESCON. FICON is Fibre Channel based and supports 100MBps or 200MBps data
transfer rate. The FICON link distance is 10 kilometers but can be extended to 20 kilometers with
an RPQ. With repeaters, the distance can extend to 100 kilometers.

© Copyright IBM Corp. 2001, 2017 1-31

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN attached storage

SCSI BUS#, SCSI ID, LUN#

Open
systems SCSI BUS
Host2
server Direct-attached storage

Initiator Target Target Host1 Host3

LUNs LUNs
SAN
Fabric
• Same SCSI commands used
• Physical transport changes RAID
CACHE
RAID
CACHE
ƒ Serial technology Controller Controller
ƒ SCSI target ID is arbitrary

Devices

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-30. SAN attached storage

SCSI over Fibre Channel, also known as Fibre Channel Protocol (FCP), is the primary protocol in
SANs. With FCP, the same SCSI commands are used (the logical protocol), but the parallel SCSI
bus (physical interface) is replaced by using serial technology.
SCSI commands contain LUN numbers but not SCSI IDs. Target IDs are used to identify controllers
on the SCSI bus by varying the voltage on the data wire. By changing the physical transport to
Fibre Channel, the SCSI commands do not use or need SCSI IDs. For compatibility, a single
arbitrary target ID per Fibre Channel port is presented by adapter drivers to preserve SCSI host
software interfaces.

© Copyright IBM Corp. 2001, 2017 1-32

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN with other technologies

Ethernet

DAS DAS NAS SAN iSCSI FCoE

Internal External Client Initiator Initiator Initiator

NAS iSCSI FCoE

Server Target Target

NAS iSCSI FCoE

Gateway Gateway Gateway

Fibre Channel SAN

SAN SVC Extension Extension

Target Virtualize switch IP switch

Remote SAN

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-31. SAN with other technologies

The storage area network (SAN) model of storage attachment is an extension of the direct-attach
storage model. In the SAN model, the objective is to connect heterogeneous hosts to a set of
intelligent storage elements. The SAN is a separate, dedicated network which is created to offload
data and storage management traffic from the primary network.
This SAN infrastructure provides the foundation to facilitate solutions such as device pooling,
centralized storage management, high-availability clustering, LAN-free backup, and server-less
backup.
The SAN is a separate network requiring hub and switches, adapters, and cables. Therefore, the
cost of implementation must be weighed against the potential network throughput and increased
productivity that might offset the cost of the infrastructure.

© Copyright IBM Corp. 2001, 2017 1-33

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Fibre Channel enables SAN

Channels Networks
• Connection service • Connectionless
ESCON
• Physical circuits • Logical circuits
• HW reliable transfers • Unreliable transfers
• High speed • High connectivity
• Low latency SCSI • Higher latency
• Short distance • Longer distance
• Hardware intensive • Software intensive

FCP FICON
Fibre Channel
• High data rates – up to 16 Gbps • Low latency – dedicated bandwidth
• Circuit- and packet-switched • High connectivity – 24-bit address
• Reliable transfers – BER<10-12 • Media options – copper or fiber
• High data integrity – error detect • Long distance – 10 km
SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-32. Fibre Channel enables SAN

The Fibre Channel standard, which is defined by the American National Standards Institute (ANSI
X3T11), provides interconnection options among a wide array of heterogeneous hosts and storage
elements.
Fibre Channel is an unification of the best characteristics of two technologies:
• Channels: Designed to perform high-speed data transfer between a host and its storage
elements in a static environment.
• Networks: Designed to facilitate data exchange among multiple independent communicating
agents in a dynamic peer-to-peer environment.
When the SCSI protocol is used over Fibre Channel, it is called Fibre Channel protocol (FCP).
Fiber Connection (FICON), an enhancement to ESCON, is Fibre Channel-based and is the
prerequisite for the IBM System z family to fully participate in a heterogeneous SAN.
Fibre Channel integrates both the high-speed data transfer characteristics of channels and the
dynamic interconnectivity characteristics of networks. In addition, it includes definitions for using
fiber optics technology to address the distance constraints that are typically found in SCSI
environments. Fibre Channel is the technology of the SAN infrastructure.

© Copyright IBM Corp. 2001, 2017 1-34

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Fibre Channel layered architecture

FC-4 Protocol Mapping Upper

Layers
FC-3 Common services

FC-2 Framing protocol / Flow control

FC-1 Transmission protocol Physical

Layers
FC-0 Physical Interface / Media

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-33. Fibre Channel layered architecture

Fibre Channel supports a data transfer rate up to 1600 Mbps and a distance of 10 kilometers per
link (without extended distance transceivers). Both fiber optic and copper interfaces are supported.
As with all networking protocols, the Fibre Channel architecture is structured in layers.
The physical layers (FC-PH) are:
• FC-0: Defines physical media and transmission rates. These include cables, connectors,
drivers, transmitters, and receivers.
• FC-1: Defines data encoding schemes that are used to synchronize data for transmission.
• FC-2: Defines the framing protocol and flow controls. It also defines the physical topologies,
which are point-to-point, arbitrated loop, and switched fabric.
The upper layers are:
• FC-3: Defines common services.
• FC-4: Defines the upper layer protocol mapping. Protocols such as FCP (SCSI over Fibre
Channel), FICON, and IP, are mapped to the Fibre Channel transport service. The protocol
mapping aspect of Fibre Channel allows the architecture to be compatible with preexisting
standards and enables interoperability with already installed systems and software.

© Copyright IBM Corp. 2001, 2017 1-35

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN infrastructure

Storage Disk storage

Tape drive/library

Intelligent storage server

FC RAID controller

SAN Fabric FC Switch FC Hub

FC Director
Router

Servers
Server Server Server
Server
Software Applications Services

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-34. SAN infrastructure

The building blocks to create a storage area network include:

• Storage: Various storage elements, typically including intelligent or smart storage servers.
Intelligence is defined as having at least a storage control processor (controller) with built-in
algorithms such as cache management.
• SAN Fabric: Network-like interconnect components, such as FC switches, directors, and hubs
that are tailored to manage storage traffic. SAN routers provide iSCSI to FC storage routing and
SAN distance extensions.
• Servers: Host systems installed with host bus adapters that support the Fibre Channel
architecture.
• Software: Host software or server applications with the ability to support SAN-facilitated
applications such as device pooling or LAN-free backup.
• Services: In-house or contracted consulting services that are used to design, plan, and
implement a SAN. Support and education are also key aspects of this building block.

© Copyright IBM Corp. 2001, 2017 1-36

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

SAN fabric interconnect components

• Enterprise SAN Directors
– IBM System Storage SAN768B-2
– IBM System Storage SAN384B-2
– Cisco MDS 9710 Multilayer Director for IBM System Networking
– Cisco MDS 9706 Multilayer Director for IBM System Storage
– Cisco MDS 9513 Multilayer Director for IBM System Storage
– Cisco MDS 9506 Multilayer Director for IBM System Storage
• Mid-range SAN switches
– IBM System Networking SAN96B-5
– IBM System Storage SAN48B-5
– Cisco MDS 9148S 16G Multilayer Fabric Switch for IBM System Storage
– Cisco MDS 9148 for IBM System Storage for IBM System Storage
• Entry SAN switches
– IBM System Storage SAN24B-4 Express
– IBM System Networking SAN24B-5
• Specialty switches
– IBM System Storage SAN42B-R extension switch
– Cisco MDS 9250i Multiservice Fabric Switch for IBM System Storage
– IBM System Storage SAN06B-R multiprotocol router

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-35. SAN fabric interconnect components

IBM SAN products and solutions provide integrated SMB and enterprise SAN solutions with
multi-protocol local, campus, metropolitan and global storage networking.
Switches and directors allow Fibre Channel devices to be connected (cascaded) together,
implementing a switched fabric topology between them. The switch intelligently routes the frames
from the initiator to the target and operates at full Fibre Channel bandwidth.

© Copyright IBM Corp. 2001, 2017 1-37

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Keywords
• Array • NAS gateway
• Redundant Array of Independent • Storage area network
Disks • Worldwide name
• Stripe, mirror, parity • Block level request
• Host Bus Adapter • File level request
• Single Point of Failure • Initiator
• Enclosure • Target
• Small Computer System • SCSI bus
Interface
• SSA loop
• Logical Unit Number
• ESCON / FICON
• Direct attached storage
• SAN Fabric
• Network attached storage
• Fibre Channel protocol
• NAS appliance
• Switches and Directors

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-36. Keywords

© Copyright IBM Corp. 2001, 2017 1-38

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Review questions (1 of 3)
1. What is a key design element of an intelligent storage
system?
a. Offer flexible networking options
b. Eliminate the single point of failure
c. Increase the storage capacity
d. Provide a faster processor

2. What are components of the SAN Fabric in a SAN

infrastructure?
a. Hosts and HBAs
b. Storage systems and controllers
c. Switches and directors
d. Thin provisioning and copy services

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-37. Review questions (1 of 3)

Write your answers here:

© Copyright IBM Corp. 2001, 2017 1-39

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Review questions (2 of 3)
3. Which storage network technology uses a client/server
model?
a. DAS
b. NAS
c. SAN
d. SNA

4. True or False: SAN implementation is simpler to manage

and scale because of the consolidated storage.

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-38. Review questions (2 of 3)

Write your answers here:

© Copyright IBM Corp. 2001, 2017 1-40

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Review questions (3 of 3)
5. The Fibre Channel SAN topology can be viewed as an
extension to (blank).
a. The SCSI bus
b. The SSA loop
c. The ESCON
d. The token ring

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-39. Review questions (3 of 3)

Write your answers here:

© Copyright IBM Corp. 2001, 2017 1-41

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 1. SAN concepts

Uempty

Unit summary
• Evaluate the components of a storage system
• Classify the components of a SAN infrastructure
• Differentiate between DAS, NAS, and SAN technologies
• Discuss SAN implementation benefits
• Differentiate between storage network topologies

SAN concepts © Copyright IBM Corporation 2001, 2017

Figure 1-40. Unit summary

© Copyright IBM Corp. 2001, 2017 1-42

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Unit 2. Fibre Channel

Estimated time
02:00

Overview
Although a storage area network can be built successfully without using Fibre Channel technology,
the mainframe ESCON/FICON homogeneous SAN environment being a case in point, SANs in the
open systems environment are typically deployed within the Fibre Channel framework.
Fibre Channel is a communication and interconnection technology that enables up to 16 million
hosts and storage devices to form a network and exchange data at high speeds across extended
distances.
One defining characteristic of Fibre Channel is that it allows many existing, well-known, and
long-implemented channel and networking protocols to run over the same physical interface and
media. Consequently, there is a rich set of standards and terminology associated with its
architecture.
The purpose of this unit is to provide an overview of the Fibre Channel architecture, describe its
concepts, and define several Fibre Channel terms.

How you will check your progress

• Review questions

References
SG24-5470 Introduction to Storage Area Networks and System Networking
http://www.ibm.com/san IBM SAN home page
http://www.fibrechannel.org Fibre Channel Industry Association (FCIA)
http://www.t11.org Technical Committee T11 home page

© Copyright IBM Corp. 2001, 2017 2-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Unit objectives
• Appraise the Fibre Channel architecture and terms
• Classify selected Fibre Channel features to their associated Fibre
Channel architecture layer
• Contrast the SCSI limitations to the Fibre Channel benefits
• Differentiate the features between multimode and single mode fiber
• Discuss purpose and function of the 8bit/10bit encoding in Fibre
Channel
• Distinguish between the Fibre Channel terms of transmission word,
frame, sequence, and exchange
• Explain Fibre Channel flow control with its use of buffer credits and
service classes

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-1. Unit objectives

© Copyright IBM Corp. 2001, 2017 2-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
• FC-2
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-2. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
• FC-2
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-3. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel introduction

• Fibre Channel is predominant architecture on SAN
• Technology standard is built for high speed
• Fibre Channel carries various traffic
ƒ Intelligent Peripheral Interface (IPI)
ƒ Internet Protocol (IP)
ƒ Fibre Channel connection (FICON)
ƒ Fibre Channel Protocol (FCP for SCSI)

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-4. Fibre Channel introduction

Fibre Channel is a multi-layered network, which is based on a series of American National

Standards Institute (ANSI) standards that define characteristics and functions for moving data
across the network.
Fibre Channel (FC) is the predominant architecture upon which SAN implementations are built.
Fibre Channel allows data to be transferred at extremely high speeds. There are many products on
the market that take advantage of the high-speed, high-availability characteristics of the Fibre
Channel architecture.
Fibre Channel was developed through industry cooperation, unlike Small Computer System
Interface (SCSI), which was developed by a vendor and submitted for standardization afterward.
Fibre Channel is an architecture that is used to carry intelligent peripheral interface (IPI) traffic,
Internet Protocol (IP) traffic, Fibre Channel connection (FICON) traffic, Fibre Channel Protocol
(FCP for SCSI) traffic. The Fibre Channel transport can carry traffic with varied protocols.
FICON is the standard protocol for z/OS and will replace all Enterprise Systems Connection
(ESCON) environments over time, where FCP is the standard protocol for open systems. Both
protocols use Fibre Channel architecture to carry the traffic.

© Copyright IBM Corp. 2001, 2017 2-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel characteristics

• Combines features of I/O channels and computer networks
• Offers high performance for large data transfers
• Uses optical fiber or copper cabling
• Transfers data using serial transmission
• Provides reliable transmission of data
• Delivers data in structured frames
• Supports delivery of varied information
ƒ Data, video, and audio
• Uses existing device oriented command sets
ƒ SCSI
• Supports a large address pool for resources
ƒ 15 million, theoretically

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-5. Fibre Channel characteristics

Fibre Channel is an open, technical standard for networking which incorporates the channel
transport characteristics of an I/O bus, with the flexible connectivity and distance characteristics of
a traditional network. With the channel-like qualities, hosts and applications see storage devices as
though they are locally attached storage. With the network characteristics, it can support multiple
protocols, a broad range of devices, and it can be managed as a network. Fibre Channel can use
either optical fiber (long distance) or copper cable links (short distance at low cost).
Like other networks, information is sent in structured packets or frames, and data is serialized
before transmission. But, unlike other networks, the Fibre Channel architecture includes a
significant amount of hardware processing to deliver high performance. Fibre Channel uses a serial
data transport scheme in a single data line to achieve high data rates. Serial transfer by its very
nature, does not suffer from the problem of skew, so speed and distance are not restricted as with
parallel data transfers. Fibre Channel can operate over longer distances, both natively and by
implementing cascading, and even longer with the introduction of repeaters. Campus SANs can be
interlinked to build enterprise-wide SANs.
Fibre Channel transport layers are protocol independent, enabling the transmission of multiple
protocols.

© Copyright IBM Corp. 2001, 2017 2-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty
Using a credit-based flow control approach, Fibre Channel is able to deliver data as fast as the
destination device buffer is able to receive it. Low transmission overhead enables high sustained
utilization rates without loss of data.

© Copyright IBM Corp. 2001, 2017 2-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
• FC-2
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-6. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Parallel SCSI

Bus
HBA Controller
6-25 M LUN LUN LUN
Host
Initiator Target LUN=(disk/tape)

SCSI bus type 8-bit width 16-bit width

(narrow) (wide)
SCSI-1 5 MB/s
SCSI-2 Fast 10 MB/s 20 MB/s
SCSI-3 Ultra 20 MB/s 40 MB/s
SCSI-3 Ultra2 40 MB/s 80 MB/s
SCSI-3 Ultra3 (Ultra160) 160 MB/s

SCSI-3 Ultra320 320 MB/s

SCSI-3 Ultra640 640 MB/s

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-7. Parallel SCSI

SCSI is a fully documented ANSI standard. The first SCSI standard was created in 1986 and is
called SCSI-1. Several iterations of the standard have occurred due to technological advances and
marketplace demands. These changes have resulted in SCSI-2 and SCSI-3.
The SCSI architecture defines both software and hardware physical transport. The software
protocol specifies the commands and the controls needed for data exchange. The original physical
transport was a parallel cable that transmitted 8 bits of data in parallel. Over time, the quest for
higher bandwidth brought about wider cables that deliver 16 bits or 32 bits of data in parallel.
A SCSI bus is connected to the host with a host bus adapter (HBA). The HBA is often called the
SCSI adapter or SCSI controller. Typically, it is the initiator of SCSI commands. An initiator
determines what task needs to be performed and which target, that is, device controller, is to
perform the task.
Up to 16 device controllers, or targets, can be attached to the same bus, one of which is the host
HBA. A target responds to the initiator requests and carries out the tasks. The SCSI bus address for
each device is referred to as the SCSI-ID.

© Copyright IBM Corp. 2001, 2017 2-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Parallel SCSI limitations

• Scalability
• Reliability and availability
HBA LUN LUN LUN
• Speed and latency
ƒ Cycle time
Host
ƒ Skew
• Distance
• Device sharing

Initiator Target
Cycle time

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-8. Parallel SCSI limitations

Scalability limitations - The amount of data that is available to the server is determined by the
number of devices which can attach to the bus and by the number of buses that are attached to the
server. Up to 15 devices can be attached to a server on a single SCSI bus.
Reliability and availability limitations - Access to data on a SCSI bus may be lost if there is a failure
to a SCSI connection to the disks, a reconfiguration of the disks, or servicing of a disk device. This
loss is because all of the devices in the string must be taken offline.
Speed and latency limitations - The data rate of the SCSI bus is determined by the number of bits
transferred and the bus cycle time (measured in megahertz (MHz)). Although decreasing the cycle
time increases the transfer rate, it might also reduce the distance over which the data can be
successfully transferred. The physical transport was originally a parallel cable that transmitted 8 bits
in parallel. Later implementations widened the parallel transfers to 16-bit paths (SCSI wide) to
achieve higher bandwidths. Skew is the propagation delay in sending data bits in parallel and may
cause the transferred data bits not to arrive at the target device at the same time, or in order. Arrival
of data occurs during a small window of time, depending on the transmission speed and the
physical bus length. To minimize the skew, the distance between devices and the initiating server is
reduced.

© Copyright IBM Corp. 2001, 2017 2-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty
Distance limitations - The maximum length of the SCSI bus, ranging from 1 m to 25 m, includes all
attached devices. These limitations might severely restrict the total GB capacity of the disk storage
which can be attached to an individual server. Faster speed means shorter distance.
Device sharing limitations - Many applications require the system to access several devices, or for
several systems to share a single device. SCSI can enable multiple servers or devices to share the
same bus, known as a multi-drop configuration. To avoid signal interference and data corruption, all
unused ports on a parallel SCSI bus must be properly terminated. Incorrect termination can result
in transaction errors or failures. Normally, only a single server can access data on a specific disk
with a SCSI bus. In a shared bus environment, all devices cannot transfer data at the same time.
SCSI uses an arbitration protocol to determine which device can gain access to the bus, which
occurs before and after every data transfer on the bus. While arbitration takes place, no data
movement can occur, which reduces bandwidth utilization.

© Copyright IBM Corp. 2001, 2017 2-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

SCSI-3 interface evolution

Parallel Ultra Ultra2 Ultra3

SCSI SCSI SCSI

SCSI -3 FCP

Serial SSA

IEEE
1394

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-9. SCSI-3 interface evolution

The most notable feature in the SCSI-3 Parallel Environment is the data transfer speeds associated
with Ultra, Ultra2, and Ultra3 SCSI.
SCSI-3 serial defines the SCSI mapping for three interfaces:
Fibre Channel (FC): The physical transport of parallel SCSI is replaced with the Fibre Channel
physical interface. The term FCP is used to describe the transport of SCSI commands and data
using Fibre Channel protocol.
Serial Storage Architecture (SSA): Designed by IBM as an alternative storage interface to parallel
SCSI, SSA was made available to the industry as an open standard in 1991. An SSA connection
consists of two ports capable of four simultaneous conversations for an aggregated transmission
rate of 160 MBps. SSA nodes can be 25 meters apart.
IEEE 1394: Better known by the name Firewire which is Apple Computer's implementation, defines
a peripheral bus with the objective of low cost but with the associated lower transfer speed and
delays. However, it does incorporate isochronous services used to deliver voice and video data.

© Copyright IBM Corp. 2001, 2017 2-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel advantages

Channel Network Fibre Channel

Short distance Long distance Long distance

Low latency Higher latency Low latency

Reliable transfers Unreliable transfers Reliable transfer

High connectivity High connectivity

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-10. Fibre Channel advantages

Fibre Channel combines the best attributes of both channel and network topologies.
Channel: A channel is a static, directly attached data pipe designed to transmit data between a host
and a fixed number of peripheral devices. As such it gives minimal decision-making tasks in
fulfilling its role of a high-speed and reliable data delivery medium. The most common channel
protocol in the open systems environment is SCSI. In the zSeries environment, it is ESCON.
Network: A network is designed to be dynamic. It automatically adjusts to environment changes to
support a varying number of nodes and their dynamic connections. As a result, it has more
decision-making tasks that must be performed on the fly. Many of these tasks are done in the
software, making networks slower than channels. The flexibility in connectivity leads to greater
scalability and supports greater distance than channels.
Fibre Channel incorporates the high-speed attribute of channels and the dynamic connectivity
attribute of networks. Fibre Channel provides relief to SCSI protocol limitations where the limitations
of distance and connectivity have been a barrier to business growth. In the System z area, Fibre
Channel delivers higher I/O performance. It also alleviates the constraints associated with the
ESCON environment, such as the limits of a maximum of 256 channels and 1024 devices per link
for a given system.

© Copyright IBM Corp. 2001, 2017 2-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
• FC-2
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-11. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-14

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel structure

Channels Networks

FC 4 IPI
IPI SCSI
SCSI HiPPI
HiPPI FICON
SBCCS 802.2
802.2 IP
IP ATM

FC 3 services
Common Services

FC 2 Framing
Framing Protocol
protocol / Flow Control
control

FC -PH
FC 1 Encode / Decode

FC 0 Physical Interface / Media

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-12. Fibre Channel structure

Fibre Channel (FC) is broken up into a series of five layers. The concept of layers allows the
development of one layer to remain independent of the adjacent layers. Although a Fibre Channel
contains five layers, those layers follow the general principles that are stated in the ISO/OSI model.
The series of five layers that make up a Fibre Channel, can be categorized into these two groups:
• Physical and signaling layers
• Upper layers

© Copyright IBM Corp. 2001, 2017 2-15

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Compare Fibre Channel to OSI layers

Application
Presentation
Presentation
Session
FC 4 Application
Application Interfaces
Interfaces Transport

FC 3 Common Services
Network
FC 2 Framing Protocol / Flow Control
DataLink
FC 1 Encode / Decode
Physical
FC 0 Physical Link
OSI Reference Model

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-13. Compare Fibre Channel to OSI layers

Fibre Channel structure is based on the networking architecture and can be viewed as a hierarchy
of functions similar to other networking protocols.
The Open System Interconnect (OSI) model represents its network architecture in seven layers.
Fibre Channel consists of five layers. While it does not map directly to the OSI model, its lower
layers relate closely to the lower layers of OSI.
• FC-0 and FC-1 are similar to the physical layer of the OSI model.
• FC-2 is similar to what other protocols define as a media access control (MAC) layer, which is
viewed as the lower half of the DataLink layer. Fibre Channel, however, does not define the
concept of a MAC.
• FC-3 is not really a layer at all. It is still a largely undefined set of services for devices having
more than one port. An example is striping, where data is transmitted out of all ports at the
same time in order to increase bandwidth.
• FC-4 defines how other well-known higher layer protocols are mapped onto and transmitted
over Fibre Channel. Thus, one can roughly associate it with the transport layer of the OSI
model.

© Copyright IBM Corp. 2001, 2017 2-16

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
– Physical link
• FC-1
• FC-2
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-14. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-17

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

FC-0 layer
• Defines the physical link in the system
ƒ Cables, fiber optics, connectors, pinouts
ƒ Optical and electrical parameters
ƒ Data transmitters and receivers
ƒ Bandwidth

FC 0 Physical Interface / Media

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-15. FC-0 layer

FC-0 is the lowest Fibre Channel layer and defines the physical link between two Fibre Channel
ports (transmit and receive), including the cabling types, connectors, optical and electrical
parameters for a variety of data rates.
The system bit error rate (BER) at the supported media and speeds is less than 10-12 . The physical
level is designed for the use of large number of technologies to meet the widest range of system
requirements. An end-to-end communicating route may consist of different link technologies to
achieve the maximal performance and price efficiency.
Fibre Channel supports copper and optical cabling. Copper cables are used for interconnecting
storage devices over short distances, while optical cables are used for all connections over longer
distances due to its noise immunity. Optical cables include single-mode and multi-mode fibers.
Copper cables include shielded twisted pair and coaxial. Direct attached cables are copper cables
with attached transceivers at each end.

© Copyright IBM Corp. 2001, 2017 2-18

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

FC-0 layer Open Fibre Control

Optical Optical
TX RX Fibre
Fibre
Channel Open Open Channel
Controller Fibre Fibre Controller
Control Control
Optical Optical
RX TX

Link

• FC 0 specifies a safety system for an open fiber link condition

ƒ Receiver detects the open fiber condition
ƒ Transmitter pulses a signal at a low duty cycle
ƒ When the path is restored, both ports receive the pulsing signals
After a double-handshake process, connection is restored within a few seconds.

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-16. FC-0 layer Open Fibre Control

The FC-0 layer specifies a safety system called the Open Fibre Control system (OFC) for SW laser
data links, since the optical power levels exceed the limits defined by the laser safety standards. If
an open fiber condition occurs in the link, the receiver side port detects it and pulses its laser
transmitter at a low duty cycle that meets the safety requirements. The receiver side of the other
port (at the other end of the fiber) detects this pulsing signal and also pulses its laser transmitter at
a low duty cycle. When the open fiber path is restored then both ports receive the pulsing signals
and after a double handshaking procedure the connection is automatically restored within a few
seconds.

© Copyright IBM Corp. 2001, 2017 2-19

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fiber optic technology

Multimode fiber Single mode fiber
Multiple paths for the light to travel Single path for the light to travel

LED
or
Light SX LX
emitting Laser Laser
diode
Core diameter
8.3 micron
Core diameter
50 or 62.5 micron
Cladding diameter
125 micron
Cladding diameter
125-micron Outer coating diameter
250-micron
Outer coating diameter
250-micron

SC duplex single mode

SC duplex multimode

LC duplex single mode

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-17. Fiber optic technology

Transmission of signals over fiber uses the phenomenon of total internal reflection to transmit data
with rays of light through a strand of glass. This strand is called an optical fiber. Total internal
reflection occurs because the cladding that surrounds the core of the optical fiber has a lower index
of refraction than the core. This difference in index of refraction causes the light to be contained on
the fiber as it travels from source to destination. In optical fiber, mode refers to the propagation of
light rays.
Multimode fiber (MM) has a core whose diameter is large compared to the wavelength of the light
signal, and many modes of light within the light beam are propagated at different angles. The
modes of light tend to lose shape as they traverse the fiber. This loss of shape is called dispersion
and limits the distance. The light source for multimode fiber is shortwave laser (SW).
Single-mode fiber (SM) has a core whose diameter is small and only one mode of light is
propagated. Longwave laser (LW) is the light source used. The combination of longwave laser and
the propagation of only a single mode of light provide long transmission distances.
A micron is one millionth of a meter.

© Copyright IBM Corp. 2001, 2017 2-20

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Optical cable options

Media type Speed (MB/s) Transmitter Distance
1310 nm LW 2 m to 10 km
100
1550 nm LW 2 m to 50 km
1310 nm LW 2 m to 10 km
200
1550 nm LW 2 m to 50 km
Single-mode fiber
400 1310 nm LW 2 m to 2 km
800 1310 nm LW 2 m to 10 km
1310 nm LW 0.5 m to 10 km
1600
1490 nm LW 0.5 m to 2 km
100 0.5 m to 300 m
200 0.5 m to 300 m
Multimode fiber 400 850nm SW 0.5 m to 150 m
800 0.5 m to 150 m
1600 0.5 m to 100 m

• Fibre Channel over copper cable

ƒ Distance limit of approximately 13 - 30 meters over twisted pair.

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-18. Optical cable options

Fibre Channel supports both copper and fiber optics media. Due to its electromagnetic interference
(EMI) noise immunity, fiber allows longer connectivity distances than copper.
A transceiver is used to bring the data bit stream from cables to the circuit board of the host bus
adapter or the device controller adapter. The transceiver is a device that converts one form of signal
to another for both transmission and reception. With fiber optics, optical pulses are converted to
electrical signals and vice versa. The transceiver can reside on the host bus adapter (HBA), or
device controller adapter, or a separate connector. A cable is attached to the transmitter (TX) of a
transceiver on one end, and a receiver (RX) of another transceiver on the other end.
Devices with signaling rates of 1Gb (a data transfer rate of 100 MBps) transmit over distances of up
to 30 meters for copper, or up to 10 kilometers for fiber. Some director vendors offer ports that
deliver 10Gb data transfer rate. These ports are designed to deliver high-bandwidth data transfer
between two connecting directors. Extended distance transceivers increase the transmission
distance to up to 100 kilometers.

© Copyright IBM Corp. 2001, 2017 2-21

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

FC-0 layer interface to host bus

(Not part of FC-0)

Host bus adapter (HBA)
Serialize/Deserialize
(SERDES) TX
FC Topology Support RX
TX
ULP Device Drivers RX
Interface to Host Bus

Host bus

Host CPU

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-19. FC-0 layer interface to host bus

A Fibre Channel Host Bus Adapter (HBA) provides the interface and the bus to deliver data from
external storage devices to the host bus. The HBAs can have a single or multiple transceivers. The
transceiver can be integrated with the HBA or it can be supplied by a GigaBit Interface Converter
(GBIC), a Small Form-factor Pluggable (SFP), or a Direct Attached Cable (DAC).
A Fibre Channel HBA supports either the full Fibre Channel protocol or a subset. For example, an
HBA might support Fibre Channel arbitrated loop, but not the Fibre Channel switch functionality.
Besides topology support, the HBA implements the entire stack of the Fibre Channel protocol by
providing the necessary device driver needed by the host operating system.
Many HBAs utilize highly integrated Fibre Channel Application-Specific Integrated Circuits (ASICs)
to process the Fibre Channel protocol and to manage the I/O buffering with the host. An ASIC is a
microelectronic integrated circuit (chip) designed and customized for a specific function. It offers
advantages, such as size, power consumption, and speed over implementing standard discrete
general-purpose electronic solutions that utilize numerous standard electronic chips and
components.
The HBA implements the entire Fibre Channel stack and is not only an FC-0 function.

© Copyright IBM Corp. 2001, 2017 2-22

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
– Transmission character encoding
• FC-2
– Frame format
– Flow control
– Classes of service
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-20. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-23

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

FC-1 layer
• Data transmission functions
ƒ Adaptive 8-bit /10-bit data transmission code scheme
í 64b/66b encoding for 10Gb and 16Gb links
í Special characters
í Error control
ƒ Ordered sets for link control
ƒ Link level protocol implementation

FC 1 Encode / Decode

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-21. FC-1 layer

The FC-1 layer defines the transmission protocol including serial encoding and decoding rules,
special characters, and error control. The information transmitted over a fiber is 8 bits that are
encoded into a 10 bit transmission character. The reason to use a transmission code is to improve
the transmission characteristic of information across a fiber. The transmission code must be DC
balanced to support the electrical requirements of the receiving units. Because serial data
transports have only two leads, transmit and receive, clocking is not possible by using a separate
line. Serial data must carry the reference timing, which means that clocking is embedded in the bit
stream. Fibre Channel uses a byte-encoding scheme and clock and data recovery (CDR) logic to
recover the clock. From this recovery, it determines the data bits that comprise bytes and words.
The transmission characters ensure that short run lengths and enough transitions are present in
the serial bit stream to make clock recovery possible.
Four encoded transmission characters, or 40 bits, is called a transmission word. Special
transmission words, called ordered sets, are designed to have special meaning in transmission link
control. Ordered sets are used to identify the start and end of data frames, to transmit low-level
status and command information, and to maintain link activity during periods of idleness.
For 10Gb or 16Gb links, the encoding scheme used is 64b/66b.

© Copyright IBM Corp. 2001, 2017 2-24

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

8b/10b encoding
H G F E D C B A
0 1 0 1 1 0 0 0 x’58’

1 1 0 0 0 0 1 0
D 24 . 2

5b/6b Encoder 5b/6b Encoder

1 1 0 0 1 1 0 1 0 1 If RD = -1

0 0 1 1 0 0 0 1 0 1 If RD = +1

A B C D E i F G H j
1 1 0 0 1 1 0 1 0 1 Disparity = +2
or
0 0 1 1 0 0 0 1 0 1 Disparity = -2
Transmission character

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-22. 8b/10b encoding

To transfer data over a high-speed serial interface, the data is encoded before transmission and
decoded upon reception. The encoding process ensures that sufficient clock information is present
in the serial data stream. This information allows the receiver to synchronize to the embedded clock
information and successfully recover the data at the required error rate. This 8b/10b encoding finds
errors that a parity check cannot.
This scheme is called 8b/10b encoding because it refers to the number of data bits input to the
encoder and the number of bits output from the encoder. The format of the 8b/10b character is of
the format Ann.m where A represents D for data or K for a special character, nn is the decimal
value of the lower 5 bits (EDCBA), “.” is a period, and m is the decimal value of the upper 3 bits
(HGF).
The residual effect on the stream to the number of zero and one bits transmitted is maintained as
the running disparity (RD) and the effect of skew is balanced by the choice of encoding. When a 6-
or 4-bit code is used that has a non-zero disparity (count of ones minus count of zeros; that is, −2 or
+2), the choice of positive or negative disparity encodings must be the one that toggles the running
disparity. In other words, the non zero disparity codes alternate.

© Copyright IBM Corp. 2001, 2017 2-25

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

8b/10b encoding tables and rules

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-23. 8b/10b encoding tables and rules

8b/10b coding is DC-free, meaning that the long-term ratio of ones and zeros transmitted is exactly
50%. To achieve this, the difference between the number of ones transmitted and the number of
zeros transmitted is always limited to ±2, and at the end of each symbol, it is either +1 or −1. This
difference is known as the running disparity (RD).
For each 5b/6b and 3b/4b code with an unequal number of ones and zeros, there may be two bit
patterns that can be used to transmit it: one with two more “1” bits, and one with all bits inverted and
thus two more zeros. Depending on the current running disparity of the signal, the encoding engine
selects which of the two possible six- or four-bit sequences to send for the given data. Obviously, if
the six- or four-bit code has equal numbers of ones and zeros, there is no choice to make, as the
disparity would be unchanged.
16Gb Fibre Channel uses 64b/66b encoding.

© Copyright IBM Corp. 2001, 2017 2-26

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
• FC-2
– Frame format
– Flow control
– Classes of service
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-24. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-27

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

FC-2 layer
• Provides transport methods to determine:
ƒ Topologies based on the presence/absence of the fabric
ƒ Classes of service
ƒ Frame format and structure
í Sequence and exchange identifiers
ƒ Segmentation and reassembly

FC 2 Framing
Framing Protocol
protocol / Flow Control
control

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-25. FC-2 layer

The FC-2 layer is the network layer that supports the framing and signaling protocol. This layer
supports the various topologies by adapting the data formats to support point-to-point, arbitrated
loop, or switched fabrics. Classes of service accommodates different application needs and offers
various options for communication and flow control. Data frames are formed at this layer to manage
the delivery of the information initiator to target. The frame format includes controls to associate
multiple frames of data and to manage segmentation and reassembly of the data.
Ordered sets provide the availability to obtain bit and word synchronization, which also establishes
word boundary alignment.

© Copyright IBM Corp. 2001, 2017 2-28

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Ordered sets
• Four byte transmission words containing data and special characters
that have special meaning
ƒ Always begins with special character K28.5
• Frame delimiters
ƒ Start-of-Frame and End-of-Frame
• Primitive signals
ƒ Idle and Receive Ready
• Primitive sequences
ƒ Offline, Not Operational, Link Reset, and Link Reset Response

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-26. Ordered sets

Ordered Sets are four byte transmission words containing data and special characters which have
a special meaning. Ordered Sets provide the availability to obtain bit and word synchronization,
which also establishes word boundary alignment. An Ordered Set always begins with the special
character K28.5. Three major types of Ordered Sets are defined by the signaling protocol.
The Frame delimiters (the Start-of-Frame (SOF) and End-of-Frame (EOF) Ordered Sets) are
Ordered Sets which immediately precede or follow the contents of a Frame. There are multiple SOF
and EOF delimiters defined for the Fabric and N_Port Sequence control.
Two Primitive Signals (Idle and Receiver Ready (R_RDY)) are Ordered Sets designated by the
standard to have a special meaning. An Idle is a Primitive Signal transmitted on the link to indicate
an operational Port facility ready for Frame transmission and reception. The R_RDY Primitive
Signal indicates that the interface buffer is available for receiving further Frames.
A Primitive Sequence is an Ordered Set that is transmitted and repeated continuously to indicate
specific conditions within a Port or conditions encountered by the receiver logic of a Port. When a
Primitive Sequence is received and recognized, a corresponding Primitive Sequence or Idle is
transmitted in response. Recognition of a Primitive Sequence requires consecutive detection of
three instances of the same Ordered Set. The Primitive Sequences supported by the standard are
Offline (OLS), Not Operational (NOS), Link Reset (LR) and Link Reset Response (LRR).

© Copyright IBM Corp. 2001, 2017 2-29

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel frame structure

SOF Frame content EOF
4 bytes 28 – 2140 bytes 4 bytes

Frame header Frame Payload CRC

24 bytes 0 – 2112 bytes 4 bytes

Byte1 Byte2 Byte3 Byte4

R_CTL Destination_ID
Reserved Source_ID
Type Frame Control
SEQ_ID DF_CTL Sequence Count
Originator X_ID Responder X_ID
Parameters

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-27. Fibre Channel frame structure

Each frame begins and ends with a frame delimiter. The frame content includes a frame header, the
frame payload, and a CRC to detect transmission errors. Fibre Channel protocol restricts the frame
payload to a maximum of 528 transmission words, or 2112 bytes.
The frame header identifies the source and destination ports, control information to manage the
frame, and association with sequences and exchanges.
• Routing control (R_CTL) identifies the type of information contained in the payload and where in
the destination node it should be routed.
• Destination_ID (D_ID) / Source_ID (S_ID) contains the destination/source port address pair of
the frame.
• Type identifies the protocol of the frame content for data frames, such as SCSI, or a reason
code for control frames.
• Frame control (F_CTL) contains control information that relates to the frame content.
• Sequence_ID (SEQ_ID) is assigned by the sequence initiator and is unique for a specific D_ID
and S_ID pair while the sequence is open.
• Data field control (DF_CTL) specifies whether there are optional headers present at the
beginning of the data field.

© Copyright IBM Corp. 2001, 2017 2-30

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty
• Sequence count (SEQ_CNT) identifies the position of a frame within a sequence and is
incremented by one for each subsequent frame in the sequence.
• Originator ID (OX_ID) / Responder ID (RX_ID) identifies the exchange ID assigned by the
originator/responder.
• Parameters specifies relative offset for data frames, or information specific to link control
frames.

© Copyright IBM Corp. 2001, 2017 2-31

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Frame, sequence, and exchange

• Transmission word
– 4 bytes

• Frame
– 9 to 537 transmission words

• Sequence
– Multiple associated frames

• Exchange
– Multiple associated sequences

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-28. Frame, sequence, and exchange

A transmission word is the smallest transmission unit that is defined in Fibre Channel. This unit
consists of four transmission characters. When information that is transferred is not an even
multiple of 4 bytes, the framing protocol adds fill bytes. The fill bytes are stripped at the destination.
Frames are the building blocks of Fibre Channel. A frame is a string of transmission words that are
prefixed by a SOF delimiter and followed by an EOF delimiter. Fibre Channel places a restriction on
the length of the data field of a frame, at 528 transmission words, which is 2112 bytes.
Larger amounts of data are transmitted in several frames which is called a sequence. The
information in a sequence moves in one direction, from a source port to a destination port. Various
fields in the frame header are used to identify the beginning, middle, and end of a sequence. Other
fields in the frame header are used to identify the order of frames, in case they arrive out of order at
the destination.
An entire transaction between two ports is made up of sequences that are administered by an even
larger unit that is called an exchange. Two other fields in the frame header identify the exchange ID.
An exchange is responsible for managing a single operation that might span several sequences,
possibly in opposite directions. The source and destination can have multiple exchanges active at a
time.

© Copyright IBM Corp. 2001, 2017 2-32

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel flow control

• Fibre Channel has a built-in flow control.
ƒ A device transmits frames to another device.
í Only when the other device is ready to accept them
ƒ Devices communicate to each other before sending data during login.
í Frame credit is established between devices.
• Fibre Channel uses two types of flow control:
ƒ Buffer-to-buffer credit
ƒ End-to-end credit

Store

Credit Transaction

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-29. Fibre Channel flow control

In Fibre Channel, data is sent in frames. As the frames are transmitted, the devices need to
temporarily store the frames as they arrive. The data frames must be stored until they are
assembled in sequence and then delivered to the upper layer protocol. Without a way to manage
the flow of frames, it is possible to inundate and overwhelm a target device with frames. There must
be a mechanism to stop this from happening. The ability of a device to accept a frame is called its
credit. This credit refers to as the number of buffers (its buffer credit) that a node maintains for
accepting incoming data.
Credit values are exchanged between devices during login, so each knows how many frames the
other can receive. After enough frames have been transmitted and credit runs out, no more frames
can be transmitted until the destination device indicates it has processed one or more frames and is
ready to receive new ones.
Fibre Channel uses two types of flow control, buffer-to-buffer and end-to-end.

© Copyright IBM Corp. 2001, 2017 2-33

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Buffer-to-buffer credit
• Maximum number of frames a port can support
ƒ Between adjacent ports
í Node port and fabric switch port
í Direct attached node ports
ƒ Both ports on the link exchange receive credit values
í This value becomes the BB_Credit value for the other port.
ƒ Receiver ready indicates availability to receive another frame

BB_Credit BB_Credit

Host node Fabric Storage node

Switch

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-30. Buffer-to-buffer credit

Buffer-to-buffer credits are the maximum number of frame transfers that a port can support. During
login, node ports (N_Ports) and fabric switch ports (F_Ports) at both ends of a link establish its
buffer-to-buffer credit (BB_Credit). Each port states the maximum BB_Credit that they can offer and
the lower of the two is used. The BB_Credit value does not have to be the same between the two
ports.
To accomplish successful flow control, the BB_Credit_CNT is initialized to 0 during login. Each time
that a port sends a frame, it increments the BB_Credit_CNT by one. When it receives a receiver
ready (R_RDY) indication from the adjacent port, it decrements the BB_Credit_CNT by one. At
certain times, the BB_Credit_CNT might become equal to the BB_Credit of the receiving port. If this
happens, the transmitting port must stop sending frames until the respective count is decremented.

© Copyright IBM Corp. 2001, 2017 2-34

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

End-to-end credit
• Maximum number of frames between one node to another
• Concept is very similar to buffer-to-buffer flow control.
ƒ Both node ports exchange receive credit values
í This value becomes the EE_Credit value for the other port.
ƒ Acknowledgment indicates availability to receive another frame

EE_Credit

Host node Fabric Storage node

Switch

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-31. End-to-end credit

End-to-end credits are the maximum number of frame transfers that a node port can support from
another node port. End-to-end flow control is not concerned with individual links, but rather the
source and destination N_Ports. The concept is very similar to buffer-to-buffer flow control.
To accomplish successful flow control, the EE_Credit_CNT is initialized to 0 during login. Each time
that a port sends a frame, it increments the EE_Credit_CNT by one. When it receives an
acknowledgment (ACK) from the destination port, it decrements the EE_Credit_CNT by one. At
certain times, the EE_Credit_CNT might become equal to the EE_Credit of the receiving port. If this
happens, the transmitting port must stop sending frames until the respective count is decremented.

© Copyright IBM Corp. 2001, 2017 2-35

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Flow control performance considerations

• Ensure that there are sufficient receive buffers
• Long links may hold many packets in transit
ƒ More frames could be sent before sender is notified by receiver
• Review the default buffer and maximum buffer credit values for each
switch

Sending Receiving
Node Node
Send more frames!

Receive
Buffer

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-32. Flow control performance considerations

The number of buffers has importance in overall performance. Enough buffers are needed to
ensure that the transmitting port can continue sending frames without stopping in order to be able
to use the full bandwidth. Having enough buffers is especially true with distance. At 1 Gbps, a frame
occupies about 75 m - 4 km of fiber, which depends on the size of the data payload. In a 100 km
link, many frames can be sent before the first one reaches its destination. You need an ACK back to
start replenishing the EE_Credit or an R_RDY indication to replenish the BB_Credit.
Consider frames with 2 KB of data. These frames occupy approximately 4 km of fiber. About 25
frames can be sent before the first frame arrives at the far end of the 100 km link. Another 25
frames can be sent before the first R_RDY or ACK indication is received. Therefore, a size of at
least 50 buffers is needed to allow for non-stop transmission at a 100 km distance with frames of
this size. If the frame size is reduced, more buffers would be required to allow non-stop
transmission. In brief, the buffer credit management is critical in case of long-distance
communication, hence the appropriate buffer credit allocation is important to obtain optimal
performance. Inappropriate allocation of the buffer credit might result in a delay of transmission
over the Fibre Channel link. As a preferred practice, always refer to the default buffer and maximum
buffer credit values for each model of switch from each vendor.

© Copyright IBM Corp. 2001, 2017 2-36

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel classes of service

• Applications require different delivery services and guarantees
• Fibre Channel standards define various service classes
ƒ Class 1 - Dedicated connection, full bandwidth, acknowledged
ƒ Class 2 - Connectionless node, multiplexed, acknowledged
ƒ Class 3 - Datagram connectionless, multiplexed, not acknowledged
ƒ Class 4 - Dedication connection, virtual circuit, acknowledged
ƒ Class 5 - Isochronous, no buffering, not clearly defined
ƒ Class 6 - Dedicated connection, multicast, full bandwidth, acknowledged
ƒ Class F- Connectionless switch, multiplexed, acknowledged

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-33. Fibre Channel classes of service

Fibre Channel defines several communication strategies called classes of service. Applications
might require different levels of service and guarantees with respect to delivery, connectivity, and
bandwidth. Some applications will need to have bandwidth dedicated to them for the duration of the
data exchange. An example of this would be a tape backup application. Other applications might be
bursty in nature and not require a dedicated connection, but they might insist that an
acknowledgment is sent for each successful transfer.
The Fibre Channel standards provide different classes of service to accommodate the applications
needs.

© Copyright IBM Corp. 2001, 2017 2-37

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Class 1 service
• Dedicated connection is established between two node ports.
– Provides acknowledged service
– Ensures that frames are received in order
– Full bandwidth is reserved for the connection
– Bandwidth contenders are blocked during the connection

Initiator Fabric Responder

Node Switch Node

Frame

ACK

Frame

ACK

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-34. Class 1 service

Class 1 service establishes a dedicated connection between source and destination node through
the fabric during the transmission. It provides acknowledged service and only uses receive ready
service during the connection request. This class of service ensures that the frames are received
by the destination device in the same order in which they are sent. This class reserves full
bandwidth for the connection between the two devices. It does not provide for a good utilization of
the available bandwidth, since it is blocking another possible contender for the same device.
Because of this blocking and the necessary dedicated connections, class 1 is rarely used.

© Copyright IBM Corp. 2001, 2017 2-38

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Class 2 service
• Connectionless communications between two node ports
– Provides end to end acknowledged service
– Provides flow control between fabric and node
– Does not ensure that frames are received in order
– Frames can be multiplexed across fabric to several destination nodes

Initiator Fabric Responder

Node Switch Node

Frame
R_RDY R_RDY

ACK
R_RDY
R_RDY

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-35. Class 2 service

Class 2 is a connectionless, acknowledged service. Class 2 makes better use of available

bandwidth since it allows the fabric to multiplex several messages on a frame-by-frame basis.
Frames can be multiplexed between a source device and one or more destination devices.
Flow-control support is provided for both end-to-end, between the communicating devices, and at
link level between the Fabric and communicating devices. As frames travel through the fabric they
can take different routes, so class 2 service does not guarantee in-order delivery. Class 2 relies on
upper layer protocols to take care of frame sequence. The use of acknowledgments reduces
available bandwidth, which needs to be considered in large-scale busy networks.

© Copyright IBM Corp. 2001, 2017 2-39

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Class 3 service
• Connectionless communications between two node ports
– Does not provide end to end acknowledged service
– Provides flow control between fabric and node
– Does not ensure that frames are received in order
– Bandwidth is multiplexed across nodes

Initiator Fabric Responder

Node Switch Node

Frame
R_RDY R_RDY

Frame
R_RDY R_RDY

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-36. Class 3 service

Class 3 is a datagram connectionless service. There is no dedicated connection in class 3 and the
received frames are not acknowledged. Flow-control support is provided only at the link level
between the Fabric and communicating devices. It optimizes the use of fabric resources, but it is up
to the upper layer protocol to ensure that all frames are received in the correct order. The upper
layer protocol also is responsible to request the retransmission of missing frames from the source.
Class 3 is a commonly used class of service in Fibre Channel networks.

© Copyright IBM Corp. 2001, 2017 2-40

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Specific class: Class F

• Connectionless communications between two switch ports
– Provides end to end acknowledged service
– Provides flow control between fabric switches
– Does not ensure that frames are received in order
– Frames can be multiplexed across fabric to several destination switches

Fabric Fabric Fabric

Switch Switch Switch

Frame
R_RDY R_RDY

ACK
R_RDY
R_RDY

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-37. Specific class: Class F

Class F service is defined in the Fibre Channel Switched Fabric (FC-SW) and the FC-SW-2
standard for use by switches that communicate through inter-switch links (ISLs). It is a
connectionless service with notification of non-delivery between fabric expansion ports (between
switches) that are used for control, coordination, and configuration of the fabric. Class F is similar to
class 2. The main difference is that class 2 deals with node ports (N_Ports) that send data frames,
while class F is used by fabric expansion ports (E_Ports) for control and management of the fabric.

© Copyright IBM Corp. 2001, 2017 2-41

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
• FC-2
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-38. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-42

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

FC-3 layer
• Provides common services for advanced features
ƒ Multicast
í Single transmission to multiple destination ports
ƒ Hunt groups
í Associate multiple ports to the same alias
ƒ Striping
í Use multiple node ports in parallel

FC 3 Common Services
services

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-39. FC-3 layer

The FC-3 layer is a single point of the Fibre Channel stack through which all traffic flows. It is
responsible for defining services that are common to all ports on a Fibre Channel node, regardless
of what protocols they support. It also provides common services to multiple ports on a fabric.
Multicasting allows a message to be sent to multiple destination ports simultaneously. The
message can be broadcast to either all the ports on the node, a set of ports, a set of nodes, or to
the entire fabric.
Hunt groups allow all or a set of ports of a Fibre Channel node to be assigned an alias identifier.
The group of ports that share a common alias identifier is known as a hunt group. During
transmission, if the originating transmitter port is busy, the information can be forwarded to any of
the ports that belong to the hunt group. This service makes the transmission faster and more
efficient by reducing the latency that is generated while waiting for the originating port to be free.
Striping allows the user of multiple ports of a node and multiple links to send a single information
unit in parallel across the links. The simultaneous use of multiple ports and links to transmit
information greatly increases the speed of transmission. It also helps to exploit the maximum
available bandwidth.
FC-3 defines common services for all node ports, which include:
• Login server - Provides a means for nodes to establish sessions for communications.

© Copyright IBM Corp. 2001, 2017 2-43

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty
• Name server - Provides a directory of N_Ports within a fabric.
• Alias server - Used to assign alias IDs to multicast and hunt groups.

© Copyright IBM Corp. 2001, 2017 2-44

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Fibre Channel topics

• Fibre Channel introduction
• SCSI limitations
• Fibre Channel layers
• FC-0
• FC-1
• FC-2
• FC-3
• FC-4

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-40. Fibre Channel topics

© Copyright IBM Corp. 2001, 2017 2-45

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

FC-4 layer
• Defines the application interface to execute over Fibre Channel
• Supported channel and network protocols include
ƒ Intelligent Peripheral Interface
ƒ Small Computer System Interface
ƒ High Performance Parallel Interface framing protocol
ƒ Single Byte Command Code Set mapping
ƒ Fibre Connection
ƒ IEEE 802.2
ƒ Internet Protocol
ƒ Asynchronous Transfer Mode

Channels Networks

FC 4 IPI SCSI HiPPI

HiPPI FICON 802.2 IP ATM

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-41. FC-4 layer

The FC-4 layer provides the application-specific protocols. Fibre Channel is equally adept at
transporting both the network and channel information and allows both protocol types to be
concurrently transported over the same physical interface.
Through mapping rules, a specific FC-4 describes how upper layer protocol (ULP) processes of the
same FC-4 type interoperate.
A channel example is FCP. This protocol is used to transfer SCSI data over Fibre Channel. A
networking example is sending IP packets between the nodes. FICON is another ULP in use for
mainframe systems. FICON is a contraction of Fibre Connection and refers to running ESCON
traffic over Fibre Channel.

© Copyright IBM Corp. 2001, 2017 2-46

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Keywords
• SCSI limitations • Ordered set
• Host bus adapter • 8b/10b encoding
• SCSI-3 serial • Running disparity
• FC-PH • Frame delimiters
• FC upper layers • Frame
• FC-0 / FC-1 / FC-2 • Sequence / Exchange
• Open Fibre Control • BB_Credit / EE_Credit
• Multimode fiber • Classes of service
• Single mode fiber • FC-3 / FC-4
• Longwave laser • Hunt groups
• Shortwave laser • Striping
• GBIC /SFP / DAC • FICON
• Transmission character
• Transmission word

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-42. Keywords

© Copyright IBM Corp. 2001, 2017 2-47

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Review questions (1 of 2)
1. Which of the following are benefits of Fibre Channel?
a. Long distance
b. Low latency
c. Reliable transfer
d. High connectivity
e. All the above

2. Which Fibre Channel layer provides a safety system for

open fiber systems?

3. Which Fibre Channel layer provides common services?

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-43. Review questions (1 of 2)

© Copyright IBM Corp. 2001, 2017 2-48

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Review questions (2 of 2)
4. True or False: Fibre Channel service class 3 uses end-to-
end buffer credits for flow control.

5. What is the purpose of the Fibre Channel 8b/10b encoding?

a. Synchronize the clocking between sender and receiver
b. Ensure data arrives in correct order
c. Establish end point connections
d. Recover data errors

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-44. Review questions (2 of 2)

© Copyright IBM Corp. 2001, 2017 2-49

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 2. Fibre Channel

Uempty

Unit summary
• Appraise the Fibre Channel architecture and terms
• Classify selected Fibre Channel features to their associated Fibre
Channel architecture layer
• Contrast the SCSI limitations to the Fibre Channel benefits
• Differentiate the features between multimode and single mode fiber
• Discuss purpose and function of the 8bit/10bit encoding in Fibre
Channel
• Distinguish between the Fibre Channel terms of transmission word,
frame, sequence, and exchange
• Explain Fibre Channel flow control with its use of buffer credits and
service classes

Fibre Channel © Copyright IBM Corporation 2001, 2017

Figure 2-45. Unit summary

© Copyright IBM Corp. 2001, 2017 2-50

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Unit 3. SAN design and topology

Estimated time
02:30

Overview
This unit introduces and describes the many components needed to build a SAN environment. It
presents the major components of a SAN, reviews several SAN topologies, defines the usage of
various switch port types, describes the SAN addressing structures, and discusses the SAN port
login process. It also compares the implementations of NPIV and NPV.

How you will check your progress

• Review questions

References
SG24-5470 Introduction to Storage Area Networks and System Networking
http://www.ibm.com/san IBM SAN home page
http://www.fibrechannel.org Fibre Channel Industry Association (FCIA)
http://www.t11.org Technical Committee T11 home page

© Copyright IBM Corp. 2001, 2017 3-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Unit objectives
• Classify the three major components of a SAN
• Differentiate between the various SAN topologies and their features
• Distinguish the features and uses of the common and expansion SAN
switch ports
• Contrast the structure and the use of the worldwide name with the Fibre
Channel port address
• Describe the port login process of a node into a SAN switch
• Examine the use of worldwide name and domain-port number for
defining zones in a SAN switch
• Compare the implementation of NPIV with NPV

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-1. Unit objectives

© Copyright IBM Corp. 2001, 2017 3-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-2. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-3. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN components
Linux
System z
• SAN connectivity Power
Systems
System i
• SAN storage
• SAN servers
ƒ Multipath SW UNIX Windows

Director
Switch Router

Storage Storage Storage

Device Device Device

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-4. SAN components

The three major components of a SAN include connectivity, storage, and servers.
Connectivity of the storage and server components is typically Fibre Channel. SANs interconnect
the storage interfaces together into many network configurations and across longer distances.
The SAN storage is freed from a particular server bus and instead is directly attached to the
network. The storage is externalized and can be functionally distributed across the organization.
The SAN also enables the centralization of storage devices and the clustering of servers. A SAN
infrastructure provides enhanced network availability, data accessibility, and system manageability.
It is important to remember that a good SAN begins with a good design.
The server infrastructure is the underlying reason for all SAN solutions. This infrastructure includes
a mix of server platforms such as Microsoft Windows, UNIX (and its various versions), and z/OS.
Initiatives such as server consolidation and e-business make the importance of storage in the
network greater and increase the need for SANs.
Multipathing software in a server provides the SAN with an improved level of fault-tolerance and
performance because it provides more than one physical path between the server and storage. In
the case of IBM, the IBM Subsystem Device Driver (SDD) is used and has the following benefits:
• Enhances data availability
• Dynamic input/output (I/O) load-balancing across multiple paths

© Copyright IBM Corp. 2001, 2017 3-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty
• Automatic path failover protection
• Concurrent download of licensed machine code
When considering how many paths should be configured to each volume, never exceed the
supported level that is given by the storage device. When zoning is implemented to a storage
device, the number of paths must be decided.

© Copyright IBM Corp. 2001, 2017 3-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-5. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Fibre Channel topologies

• Fibre Channel supports three topologies
ƒ Point-to-point:
í Two devices are directly connected.
í This is the simplest topology, with limited connectivity.
ƒ Arbitrated loop
í All devices are in a loop or ring.
Similar to token-ring networking.
ƒ Switched fabric
í All devices or loops of devices are connected to Fibre Channel switches.
Similar conceptually to Ethernet implementations.

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-6. Fibre Channel topologies

Fibre Channel provides three topology options for connecting devices: point-to-point, arbitrated
loop, and switched fabric. A switched fabric is the most commonly encountered.
Topology and it has sub-classifications of topology.

© Copyright IBM Corp. 2001, 2017 3-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Point-to-point topology
• Consists of only two Fibre Channel devices connected directly together.
• The transmit fiber of one device connects to the receive fiber of the
other device.
• There is no sharing of the media with other devices
• Allows the devices to enjoy the total bandwidth of the link.
• A simple link initialization is required before communications can begin.

Storage System

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-7. Point-to-point topology

A point-to-point connection (FC-P2P) is the simplest topology. It is used when there are exactly two
nodes, and future expansion is not predicted. There is no sharing of the media, which allows the
devices to use the total bandwidth of the link. A simple link initialization is needed before
communications can begin.
Fibre Channel is a full duplex protocol, which means both paths transmit data simultaneously. As an
example, Fibre Channel connections based on the 1 Gb standard are able to transmit at 100 MBps
and receive at 100 MBps simultaneously. Again, as an example, for Fibre Channel connections
based on the 2 Gb standard, they can transmit at 200 MBps and receive at 200 MBps
simultaneously. This extends to 4 Gb, 8 Gb, and 16 Gb technologies as well.

© Copyright IBM Corp. 2001, 2017 3-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Arbitrated loop topology

• Cost-effective way to connect up to 127 ports in a single network.
• Media is shared among the devices.
– Limits device access
– Arbitrated loop is not a token-passive scheme.
• Device must arbitrate and gain control of the loop before transmitting
data
– Essentially a point-to-point communication exists between these two devices.
– All other devices in between simply repeat the data.

Storage System

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-8. Arbitrated loop topology

Fibre Channel Arbitrated Loop (FC-AL) is a loop of up to 126 nodes that is managed as a shared
bus. Traffic flows in one direction, carrying data frames and primitives around the loop.
Using arbitration protocol, a single connection is established between a sender and a receiver, and
a data frame is transferred around the loop. When the communication comes to an end between
the two connected ports, the loop becomes available for arbitration and a new connection might be
established. Loops can be configured with hubs to make connection management easier. A
distance of up to 10 km is supported by the Fibre Channel standard for both of these configurations.
However, latency on the arbitrated loop configuration is affected by the loop size.

© Copyright IBM Corp. 2001, 2017 3-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Switched fabric topology

• Dedicated links between device node and switch
• Adding devices increases bandwidth
• Fabric topology may consist of one or more switches
ƒ Provides path redundancy
ƒ Provides scalability up to 16 million ports

Switch/Director

Storage System

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-9. Switched fabric topology

Fibre Channel Switched Fabric (FC-SW) applies to switches and directors. A Fibre Channel fabric
is one or more fabric switches in a single and sometimes extended configuration. Switched fabrics
provide full bandwidth for each port. That is in contrast to the shared bandwidth for each port in
arbitrated loop implementations.
One of the key differentiators is that if a device is added into an arbitrated loop, then the shared
bandwidth is further divided. However, in a switched fabric, adding a device or a new connection
between existing ones actually increases the bandwidth. For example, an eight-port switch
(assume that it is based on 8 Gb technology) with three initiators and three targets, can support
three concurrent 800 MBps conversations or a total of 2,400 MBps throughput. This equates to
4,800 MBps, if full-duplex applications were available.
This configuration is one of the major reasons why arbitrated loop is considered a legacy SAN
topology. A switched fabric is usually referred to as a fabric.
In terms of switch interconnections, the switched SAN topologies can be classified as the following
types:
• Single switch topology
• Cascaded and ring topology
• Full mesh topology

© Copyright IBM Corp. 2001, 2017 3-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty
In terms of a tiered approach, the switched fabric can be further classified with the following
topology:
• Core-edge topology
• Edge-core-edge topology

© Copyright IBM Corp. 2001, 2017 3-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Single switch
• Simplest design
• No inter-switch links

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-10. Single switch

The single switch topology has only one switch and has no inter-switch links (ISLs). It is the
simplest design for infrastructures which do not need any redundancy. A single switch can vary
from a single 16 port switch to connect a few servers and a couple of storage devices, to a large
256 port director that connects a large enterprise together. Because of the issues of it introducing a
single point of failure, this topology is rarely used.
A variation on a single switch is two switches. If all the servers and storage devices have dual
paths, and include a multi-pathing failover capability, then a very resilient network can be built with
two single switch fabrics. Every device is connected to both fabrics, so if any part of one fabric goes
down, the other fabric will continue supporting communications until the problem is fixed.

© Copyright IBM Corp. 2001, 2017 3-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Cascade and ring

• Cascade
ƒ Switches are connected with ISLs in a queued fashion

• Ring
ƒ Similar to cascade, but first and last switch are also connected

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-11. Cascade and ring

A cascade topology is where the switches are simply inter-connected in a queue. A cascade design
does not need many ISLs. An issue with this design is having to go through several hops to connect
devices at either end of the queue, thereby increasing network latency. The main issue with a
cascade design is that if a switch fails, then some switches will not be able to communicate with
each other. For that reason it is rarely seen these days.
A ring topology is just a cascade fabric with the end switches connected together to form a ring. In
order to add another switch, the ring must be broken. However, if one switch fails, then the other
switches can still communicate with each other.

© Copyright IBM Corp. 2001, 2017 3-14

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Full mesh
• Every switch connected to every other switch

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-12. Full mesh

A full mesh topology is where every switch is connected to every other switch with an ISL. The
advantage of this approach is that any device can be connected to any open port in the fabric, and
know that it can connect to any other device after just one switch hop. The big disadvantage is
scalability. When adding a new switch, it must be connected to every other switch in the fabric. It is
not suitable for low port count switches because a large number of the ports would be used for
ISLs.

© Copyright IBM Corp. 2001, 2017 3-15

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Core-edge
• Storage devices connect to core switches
• Server devices connect to edge switches
Storage
Node

Core

Edge

Server
Node

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-13. Core-edge

A core-edge topology is where the servers are connected to the edge fabric and the storage is
connected to core switches. Core-edge topology is a logical progression from full mesh as it
reduces the requirement for many ISLs while preserving the one switch hop rule. It uses a high
performance director for the core switch which is connected directly to high performance servers
and storage. Appliances that need lesser performance are connected to the core by slower edge
switches.
As the fabric grows in the number of switches one needs to design a plan for failover and path
redundancy. Every server can be connected to two different switches providing two independent
paths through the SAN to the storage. The host servers must have multi-pathing software that can
automatically failover if a path fails and can load balance when two paths are available. 
With a complex fabric, the switches need to be connected correctly, have unique domain IDs, and
be time synchronized. This can be done using fabric management software, but the Fibre Channel
standard allows for a 'Principal Switch‘, or master switch, to manage the network.

© Copyright IBM Corp. 2001, 2017 3-16

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Edge-core-edge
• Storage and server devices connect to edge switches
• Edge switches connect through core switches
Storage
Node

Edge

Core

Edge

Server
Node

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-14. Edge-core-edge

An edge-core-edge topology is where the server and storage are connected to the edge fabric and
the core switch connectivity is used only for scalability in terms of connecting to edge switches. This
configuration expands the SAN traffic flow to long distance by dense wavelength division
multiplexing (DWDM), connecting to virtualization appliances, and encryption switches. Also, the
servers can be isolated to one edge and the storage can be at the other edge which helps with
management.

© Copyright IBM Corp. 2001, 2017 3-17

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Comparison between FC topologies

Attribute Point-to-Point Arbitrated Loop Switched Fabric
Maximum Ports 2 127 16,000,000

Maximum bandwidth 2 x Link Rate 2 x Link Rate (# of ports) x Link Rate

Address Size N/A 8-bit AL_PA 24-bit Port ID

Address Assignment N_Port Login Loop init. And Fabric Login Fabric Login

Concurrent Connection 1 1 Switch Ports /2

Link Fails Loop Fails unless Switch and port link fails
Effect of Port Failure
bypassed

Concurrent Maintenance Link Down Might disrupt entire loop Switch and port link down

Expansion Additional P2P links Attach loop to Fabric Expand Fabric

Add redundant P2P Use dual loops Use redundant switches

Redundancy
link

All All (all devices must be All

Link Rates Supported
same)

CoS supported All 1, 2 and 3 All

Frame delivery In order In order Not guaranteed

Access to Medium Dedicated Arbitrated Dedicated

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-15. Comparison between FC topologies

The visual shows a comparison between the different Fibre Channel topologies.

© Copyright IBM Corp. 2001, 2017 3-18

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-16. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-19

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Common switch ports

Storage
NL_Port Device
NL_Port
N_Port
L / FL_Port
Switch
F_Port U_Port
Switch
F_Port

Storage
N_Port Storage Device
Device NL_Port
NL_Port
TL_Port
Switch
G / F_Port

N_Port

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-17. Common switch ports

The following list provides the various kinds of common Fibre Channel port types and their purpose
in switches, servers, and storage:
• N_Port (Node) is a port that is not loop capable. It is a host end port that is used to connect to
the fabric switch.
• NL_Port (Node loop) is a port that is loop capable. It is used to connect an equipment port to the
fabric in a loop configuration through an L_Port or FL_Port.
• F_Port (Fabric) is a port that is connected to an N_Port point-point to a switch.
• FL_Port (Fabric loop) is a fabric port that is counted to a loop device. It is used to connect an
NL_Port to the switch in a public loop configuration.
• L_Port (Loop-capable) is a loop-capable node or switch port.
• TL_Port (Translative loop) is a Cisco specific port type that is connected with non-fabric aware,
private loop devices. A TL_Port makes a select set of fabric devices appear as private devices
on the loop and makes it also appear to a device on the private loop that they are talking to an
on-loop device when in fact they are talking to an off-loop device. It makes devices on the
private loop appear as public devices to rest of the fabric. Mechanisms independent of nature of
fabric device – N_Port, public NL_Port, or another private NL_Port made visible through
translation.

© Copyright IBM Corp. 2001, 2017 3-20

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty
• G_Port (Generic) is a port that can operate as either an E_Port or an F_Port. A port is defined
as a G_Port after it is connected but has not received a response to loop initialization or has not
yet completed the link initialization procedure with the adjacent Fibre Channel device.
• U_Port (Universal) is a more generic switch port than a G_Port. It can operate as either an
E_Port, F_Port, or FL_Port. A port is defined as a U_Port when it is not connected or has not
yet assumed a specific function in the fabric.

© Copyright IBM Corp. 2001, 2017 3-21

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Expansion switch ports

VE / VEX_Port VE / VEX_Port
Multiprotocol FCIP tunnel Multiprotocol
Router Router
E / EX_Port E / EX_Port

E / EX_Port E / EX_Port

Cisco TE_Port
Switch Switch
TE_Port
F_Port F_Port
Cisco
N_Port N_Port N_Port Switch
Storage Storage
Device Device

VSAN1 VSAN2

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-18. Expansion switch ports

The following ports are found in a multi-switch fabric where switches are interconnected by way of a
Fibre Channel link:
• E_Port (expansion) is a port used to connect to the E_Port of another switch to enlarge the
switch fabric. When E_Ports between two switches form a link, that link is referred as an
inter-switch link or ISL.
• EX_Port is a type of E_Port used to connect a multiprotocol router to a fabric switch. An
EX_Port follows standard E_Port protocols, and supports FC-NAT, but does not allow fabric
merging across EX_Ports.
• VE_Port (Virtual) is a virtual port that emulates an E_Port over an FCIP link. VE port
connectivity is supported over point-to-point links.
• VEX_Port is a routed VE_Ports, just as EX_Ports are routed E_Ports. VE_Ports and VEX_Ports
have the same behavior and functionality.
• TE_Port (Trunking) is a Cisco specific port that provides not only standard E_Port functions, but
allows for routing of multiple VSANs (virtual SANs). All frames are transmitted in EISL frame
format, which contains VSAN information. Interconnected switches use the VSAN ID to
multiplex traffic from one or more VSANs across the same physical link.

© Copyright IBM Corp. 2001, 2017 3-22

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Diagnostic ports
• Brocade
– D_Port
í Diagnostics include loopback plug, loopback cable, ISL test
• Cisco
– SD_Port
í Dedicated port connection to a SPAN analyzer tool
– ST_Port
í Dedicated port for remote SPAN communication
í For multi-switched fabrics

*SPAN - Switch Port Analyzer

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-19. Diagnostic ports

• D_Port (Diagnostic) is a Brocade diagnostic port type which can be enabled only on the 16
Gbps b-type switches with Fabric Operating System 7.0. This system uses the Spinfab test and
performs electrical loop back, optical loop back, measures link distance, and also stress tests
with a link saturation test. Spinfab is a switch command that runs a functional test of
switch-to-switch ISL cabling and trunk group operation.
• SD_Port (SPAN destination) is a Cisco Switch Port Analyzer (SPAN) diagnostic port that is used
for diagnostic capture with a connection to SPAN network analyzer. SD_Ports do not receive
frames. They merely transmit a copy of the source traffic.
• ST_Port (SPAN tunnel) is the Cisco port type for Remote Switch Port Analyzer (RSPAN)
monitoring in a source switch. This switch is a dedicated port that is used for RSPAN analysis,
and is not connected to any other device.

© Copyright IBM Corp. 2001, 2017 3-23

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-20. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-24

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Name and addressing convention

64-bit Port WWN

24-bit Port address

Port Name Host HBA (Node HBA)

N_Port ID

Port Name
N_Port ID

Node Name
64-bit Node WWN
64-bit Port WWN
24-bit Port address

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-21. Name and addressing convention

To participate in the Fibre Channel environment, each node is assigned a 64-bit node name by the
manufacturer. The manufacturer registers with the IEEE to obtain a globally unique name which is
referred to as the worldwide name (WWN). Thus each node has a unique node WWN, also known
as worldwide node name (WWNN).
A port within a node is called a node port (N_Port) and is assigned a 64-bit port name by the
manufacturer, typically using the same format as the node WWN. Therefore, each port has its own
port WWN, also known as worldwide port name (WWPN). WWNs provide a mechanism to uniquely
identify every node and every port within a node in the SAN fabric.
The Fibre Channel frame header contains source and destination port addresses that are only 24
bits long. So, besides the port WWN, each port is assigned a 24-bit port address, known as the
N_Port ID or Fibre Channel ID (FCID). A switch assigns this 24-bit port address when an attaching
port performs a fabric login.

© Copyright IBM Corp. 2001, 2017 3-25

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Worldwide name addressing scheme

Company ID Vendor Specific Info

10 : 00 : 08 : 00 : 5A : D0 : 97 : 9B
20 : 00 : 08 : 00 : 5A : D0 : 97 : 9B

Company ID Vendor Specific Info

5 0 : 05 : 07 : 6 3 : 00 : D0 : 97 : 9B Newer
Scheme
6 0 : 05 : 07 : 6 3 : 00 : D0 : 97 : 9B

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-22. Worldwide name addressing scheme

All Fibre Channel devices have a unique identity called the worldwide name (WWN). This is similar
to the way all Ethernet cards have a unique media access control (MAC) address.
Each node N_Port will have its own WWPN, but it also possible for a device with more than one
Fibre Channel adapter to have its own WWNN as well. Thus, for example, a storage system has its
own WWNN as well as incorporating the WWNNs of the adapter within it. This means that a soft
zone can be created using particular adapters.
There are two formats of the WWN as defined by the IEEE. The original format contains
hexadecimal values of either a x’10’ or x’20’ in the first 2 bytes followed by the vendor identifier in
the next 3 bytes. Vendor-specific information is contained in the following fields. The newer
addressing scheme contains hexadecimal values of either a x’5’ or a x’6’ in the first half-byte
followed by the vendor identifier in the next 3 bytes. The vendor-specific information is then
contained in the following fields. The Organizationally Unique Identifier (OUI) is 3 bytes long and is
used to ensure globally unique names. The position of the OUI is dependent on the WWN format
designator.
Both of these formats are currently in use and depend on the hardware manufacturer standards to
follow either of the formats. However, the vendor ID and company ID are assigned unique by the
IEEE standards and each vendor and their identifier can be found in the following text file:
http://standards.ieee.org/develop/regauth/oui/oui.txt

© Copyright IBM Corp. 2001, 2017 3-26

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

WWNN and WWPN

Host server Switch

1 WWNN per HBA 1 WWNN per chassis

1 WWPN per port 1 WWPN per port

Storage system Tape system

1 WWNN per controller 1 WWNN per drive

1 WWPN per port 1 WWPN per drive

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-23. WWNN and WWPN

A worldwide node name (WWNN) is a globally unique 64-bit identifier that is assigned to each Fibre
Channel node or device. For host servers, the WWNN is unique for each host bus adapter (HBA),
and in a case of a server with two HBAs, they have two WWNNs. For a SAN switch, the WWNN is
common for the chassis. For storage, the WWNN is common for each controller unit of midrange
storage. And, in a case of high-end enterprise storage, the WWNN is unique for the entire array.
For tape devices, each tape drive has a unique WWNN.
A worldwide port name (WWPN) is a unique identifier for each Fibre Channel port of any Fibre
Channel device. For host servers, the WWPN is unique for each port of the HBA. For a switch, the
WWPN is unique for each port in the chassis. For storage, each port has an individual WWPN. For
tape devices, each tape drive has a unique WWPN.

© Copyright IBM Corp. 2001, 2017 3-27

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Port address
• Each node port is dynamically assigned a 24-bit port address
ƒ Assigned during fabric login
• Supports up to 16 million port addresses in one switched fabric
• 24-bit port address is also called N_Port ID, or FCID
• Used in frame routing through the fabric
ƒ Smaller frame header speeds up routing process
• WWN and N_Port ID association is maintained in a simple name server
ƒ SNS service available in each fabric switch
ƒ Similar in concept to MAC and IP address association

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-24. Port address

The administered name, or WWN, is not used for transporting frames across the network. In
addition to a Fibre Channel name, a communicating device is dynamically assigned a 24-bit port
address, or N_Port ID, that is used for frame routing. Port addresses are assigned dynamically at
fabric login. With a 24-bit addressing scheme, this allows for up to 16 million addresses.
The 24-bit addresses of two communicating partners are embedded in the frame header for both
the destination identifier (D_ID) and source identifier (S_ID).
In fabric environments, the switch is responsible for assigning a 24-bit address to each device as it
logs on. The switch maintains a table of the device's WWN and the assigned 24-bit address in the
Simple Name Server (SNS).

© Copyright IBM Corp. 2001, 2017 3-28

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

24-bit port address

Domain Area Node
8-bit 8-bit 8-bit

Fabric Loop ID AL_PA

16-bit 8-bit

• Domain represents the address of the switch itself

ƒ 239 unique domains
• Area represents F_Ports within a switch domain or FL_Ports on a loop
• Node represents a single N_Port within a switched topology or NL_Port
on a loop
ƒ Normally x'00' in a switched fabric
ƒ Features may exploit this field
í Shared area addressing

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-25. 24-bit port address

The most significant byte of the port address is the Domain ID. This byte is the address of the
switch itself. A domain ID is a unique number that identifies the switch or director to a fabric. Each
manufacturer has a range of numbers and a maximum number of domain IDs that can be used in a
fabric. Because some of the domain addresses are reserved, such as for broadcast, there are only
239 domain addresses available. This number means that there can theoretically be as many as
239 switches in the SAN environment.
The Area ID is used to identify the individual FL_Ports supporting loops, or it can be used as the
identifier for a group of F_Ports (for example, a card with more ports on it). This means that each
group of ports has a different area number, even if there is only one port in the group.
The Node ID identifies attached N_Ports or NL_Ports. This also called the Port ID.
Each switch in a fabric is assigned one or more unique Domain_IDs using a two-step process. First,
one switch, called Principal Switch, is selected from the switches of a Fabric. Then, the Principal
Switch assigns Domain_IDs to the other switches of the Fabric. Address assignment within a
domain is performed by the switch to which that Domain_ID is granted.

© Copyright IBM Corp. 2001, 2017 3-29

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

24-bit port loop address

Domain Area Node
8-bit 8-bit 8-bit

Fabric Loop ID AL_PA

16-bit 8-bit

• NL_Port address modes

– Private loop
í No switch fabric access
í Fabric loop ID has value of x'0000'
– Public loop
í Enabled to access switch fabric
í Fabric loop ID has unique value for the loop
• AL_PA ID
– Value acquired during initialization
– Maximum per loop is 127

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-26. 24-bit port loop address

An NL_Port, like an N_Port, has a 24-bit port address. If no switch connection exists, the two upper
bytes of this port address are zeros (x’00 00’) and the loop is referred to as a private loop. The
devices on the loop have no connection outside the loop. If the loop is attached to a fabric and an
NL_Port supports a fabric login, the upper 2 bytes are assigned a positive value by the switch. This
mode is called a public loop.
Because fabric-capable NL_Ports are members of both a local loop and the greater fabric
community, a 24-bit address is needed as an identifier. In the case of a public loop assignment, the
value of the upper 2 bytes represents the loop identifier. This ID is common to all NL_Ports on the
same loop that have logged in to the fabric.
In both public and private arbitrated loops, the last byte of the 24-bit port address is the arbitrated
loop physical address (AL_PA). The AL_PA is acquired during initialization of the loop and might, in
the case of a fabric-capable loop device, be modified by the switch during login.
The total number of the AL_PA IDs available for arbitrated loop addressing is 127. This number is
based on the requirements of 8b/10b running disparity between frames.

© Copyright IBM Corp. 2001, 2017 3-30

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

IBM b-type address modes

• Native mode Domain ID1

Storage FCID Port 16 port Port FCID

Device 011200 2 Switch 3 011300

• Core mode
Domain ID1

Storage FCID Port 256 port Port FCID

Device 010200 2 Switch 3 010300

• Shared addressing mode

Domain ID1

Storage FCID Port 512 port Port FCID

Device 010200 2 Switch 259 010380

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-27. IBM b-type address modes

IBM b-type Fibre Channel switches offer three different addressing modes:
• Native mode is used in traditional switches which support a maximum of 16 ports. The area part
of the fabric address always has a prefix of 1 and is from x’10’ to x’1F’ (maximum of 16 ports).
The fabric addressing format that is used is DD1A00.
• Core mode is used to support a maximum of 256 ports per domain/switch. The area part of the
fabric address is from x’00’ to x’FF’ (maximum of 256 ports). The fabric addressing format that
is used for this mode is DDAA00.
• Shared addressing mode is used when more than 256 ports are used in the same
domain/switch. This mode is used in directors with high port density. The port addressing in
these directors use the Area ID plus bits from the Node ID (such as x’80’) for port numbers
above 255.
The IBM OEM agreement with Brocade is referred to as b-type.

© Copyright IBM Corp. 2001, 2017 3-31

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-28. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-32

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Port initialization
• Detect port type
• Negotiate link speed
• Initialize port to active state
• Fibre Channel fabric login
ƒ Fabric login
ƒ Port login
ƒ Process login

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-29. Port initialization

At a high level, port initialization starts with port type detection. What follows is speed and active
state detection where the speed is negotiated according to the device that is connected, and then
the port initializes to an active state. In this active state, every F/FL_Port that has an N/NL_Port that
is connected, as well as the Extended Link Service (ELS) and the Fibre Channel Common
Transport (FCCT) protocol, are used for further switch-port to node-port communication. After this
initialization completes, data can flow.
The following three login types are available for fabric devices:
• Fabric login (FLOGI) – results in N_Port ID, or FCID, assigned to the node
• Port login (PLOGI) – results in being registered in the switch to communicate to other nodes
• Process login (PRLI) – results in active communication between two nodes

© Copyright IBM Corp. 2001, 2017 3-33

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Fabric login

Domain ID1

Switch
FLOGI request Fabric
login
FCID assigned
x’FFFFFE’

Storage
Device

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-30. Fabric login

Fabric login (FLOGI) is an extended link service command that sets up a session between two
participants. A session is created between an N_Port or NL_Port and the switch.
An N_Port sends a FLOGI frame that contains its Node Name, its N_Port Name, and service
parameters to the Fabric Login server at a well-known address of 0xFFFFFE.
The switch accepts the login and returns an accept (ACC) frame to the sender. If some of the
service parameters that are requested by the N_Port or NL_Port are not supported, the switch sets
the appropriate bits in the ACC frame to indicate this status.
NL_Ports derive their AL_PA during the loop initialization process (LIP). The switch then decides
whether it accepts this AL_PA, if it does not conflict with any previously assigned AL_PA on the
loop. If not, a new AL_PA is assigned to the NL_Port, which then causes the start of another LIP.

© Copyright IBM Corp. 2001, 2017 3-34

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Port login

Domain ID1

Switch
FLOGI request Fabric
login
FCID assigned
x’FFFFFE’
PLOGI request Directory service
x’FFFFFE’

Storage
Device

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-31. Port login

Port login (PLOGI) is used to establish a session between two N_Ports, and is necessary before
any upper level commands or operations can be performed. During the port login, two N_Ports
(devices) swap service parameters and make themselves known to each other. The device
information is registered with a simple name server called the Directory server at a well-known
address of 0xFFFFFC. The device might register values for all or some of its objects, but the most
useful include the following objects:
• 24-bit port address
• 64-bit port name
• 64-bit node name
• Buffer-to-buffer credit capability
• Maximum frame size
• Class-of-service parameters
• FC-4 protocols that are supported
• Port type
When the communication parameters and identities of other devices are discovered, they are able
to establish logical sessions between devices (initiator and targets).

© Copyright IBM Corp. 2001, 2017 3-35

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Process login

Domain ID1

Switch
FLOGI request Fabric
login
FCID assigned
x’FFFFFE’
PLOGI request Directory service
x’FFFFFE’

PRLI request

Storage
Device

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-32. Process login

Process login (PRLI) is used to set up the environment between related processes on an
originating N_Port and a responding N_Port. A group of related processes is collectively known as
an image pair. The processes that are involved can be system processes and system images, such
as mainframe logical partitions, control unit images, and FC-4 processes. Use of process login is
optional from the perspective of the Fibre Channel FC-2 layer. However, it might be required by a
specific upper-level protocol, as in the case of SCSI-FCP mapping.

© Copyright IBM Corp. 2001, 2017 3-36

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Fabric services
• Management services
ƒ Used to pass SAN-wide data to a management suite
• Time services
ƒ Used to manage expiration times
• Directory services
ƒ Node registry during port login
• Fabric login server
ƒ Node communicates here during fabric login
• Fabric controller
ƒ Manages and communicates state changes of nodes

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-33. Fabric services

Fabric services are implemented by switches and directors that participate in the SAN. A node can
use the switching device to which it is connected. These services are addressed by FC-2 frames
and are accessed by so called well-known addresses.
Management server (at port address 0xFFFFFA) is an in-band fabric service that allows data to be
passed from the device to management platforms. This data includes such information as the
topology of the SAN. A critical feature of this service is that it allows management software access
to the simple name server (SNS), bypassing any potential block that is caused by zoning. This
means that a management suite can have a view of the entire SAN.
Time services (at port address 0xFFFFFB) are provided to serve time information that is sufficient
for managing expiration time.
Directory services, or simple name server, (at port address 0xFFFFFC) is maintained and updated
by all switches in the fabric. After a node successfully logs in to the fabric, it performs a PLOGI to
register itself and pass on critical information such as class-of-service parameters, WWN, port
address, and the upper layer protocols that it can support.
Fabric login server (at port address 0xFFFFFE) is where a node communicates to do a fabric login.
Fabric controller (at port address 0xFFFFFD) is where nodes register with a state change
registration (SCR) frame. The fabric controller maintains the fabric state with all of the registered

© Copyright IBM Corp. 2001, 2017 3-37

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty
device details and alerts registered devices with a registered state change notification (RSCN).
This alert is sent whenever there is any device that is added or removed, there is a zone change, a
switch IP, or name change, and so on.

© Copyright IBM Corp. 2001, 2017 3-38

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Name Server database

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-34. Name Server database

The name server function in the switch provides the ability to reach a node port attached to a fabric,
to register its attributes, and to query the existence of other node ports and their attributes. A
database is maintained by the name server containing node WWN, port WWN, 24-bit port address,
class of service, protocols supported, IP addresses, and port characteristics for each participant.
Most responses to name server queries will be limited to whatever meets the query criteria and to
the zone of the originating port of the query.

© Copyright IBM Corp. 2001, 2017 3-39

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-35. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-40

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Fibre Channel zoning

• Managed and enforced within the fabric switches
• Provides logical grouping of devices within a SAN fabric
• Enables secure sharing of SAN resources
ƒ Nodes with no zones in common are invisible to each other

SAN Fabric

Storage Storage Storage

Device Device Device

Zone A Zone B

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-36. Fibre Channel zoning

Zoning provides access control between storage devices and host server groups in a switched
fabric. Creating zones increases network security and prevents data loss or corruption. Only the
members of the same zone can communicate within that zone. All other attempts from outside are
rejected.
Zoning can be considered as a security feature, and not just for separating environments.
Zoning can be used for test and maintenance purposes. For example, few enterprises mix their test
and maintenance environments with their production environment. Within a fabric, they can easily
separate the test environment from the production bandwidth allocation on the same fabric by using
zoning.
Zones can be implemented in two ways; Port zoning and WWN zoning. A group of associated
zones is called a zone configuration, or zone set.

© Copyright IBM Corp. 2001, 2017 3-41

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Zoning concepts and terminology

Zone 1 Zone
Zone 1 Zone
Zone 1
Zone 1 Zone
Zone 1 Configuration

• Multiple members per • Only one active at a time

zone configuration
• Zone members include:
ƒ WWNN
ƒ WWPN
ƒ Alias
ƒ Domain ID and Port address

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-37. Zoning concepts and terminology

Zone configurations (or zone sets) consist of one or more zones. A zone configuration can be
activated or deactivated as a single entity across all switches in the fabric, but only one zone
configuration can be activated at any time. Zone configurations can be activated without needing to
power down the switch.
Zones can be members of more than one zone configuration. A zone consists of multiple zone
members. Members in a zone can access each other. Members in different zones cannot access
each other. A zone consists of members that can be specified using WWNN, WWPN, Domain ID
and switch port number, and a predefined alias.

© Copyright IBM Corp. 2001, 2017 3-42

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Zone example configuration

Server 5

SAN Fabric

Storage Storage Storage

Device Device 3 Device

Zone A Zone B

Zone Configuration1

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-38. Zone example configuration

The example shows a zone configuration that contains two zones without overlapping zone
members. When this zone configuration is activated, nodes in Zone A are invisible to nodes in Zone
B, and vice versa.
There are two devices that are not zoned. Their operation depends on the switch default zoning
settings which can either be open or closed. It is recommended to set the default zone setting to no
access.
Assuming a default zone that is set for no access, server 5 is not included in either zone and is not
available to any device (zoned or not). Similarly storage device 3 which is not part of either Zone A
or Zone B is also not capable of communicating with other devices. In other words Server 5 and
storage device 3 will not be able to communicate with one another.

© Copyright IBM Corp. 2001, 2017 3-43

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Port zoning

1 2 3

Storage SAN Storage

Device 4 Fabric 7 Device

5 6

Storage Storage
Device Device

Zone1 – port1, port4, port5

Zone2 – port2, port6
Zone3 – port3, port6, port7

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-39. Port zoning

Port zoning (also called hard zoning) utilizes physical ports to define security zones. The members
of a zone are physical ports on the fabric switch. The members are defined by specifying the switch
Domain ID and the switch port number. A device’s access to data is determined by what physical
port to which they are connected. With port zoning, zone information must be updated any time a
device changes switch ports.
One of the disadvantages of hardware zoning is that devices must be connected to a specific port.
The whole zoning configuration can become unusable when the device is connected to a different
port. In cases where the device connections are not permanent, the use of WWN zoning is likely to
simplify the configuration. If possible, the designer can include some unused ports in a zone.
Therefore, in the event of a particular port failure, the cable can be moved to a different port in the
same zone. This means that the zone would not need to be reconfigured.

© Copyright IBM Corp. 2001, 2017 3-44

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

WWN zoning

alex ben chuck

Storage
Device
dave gary
Storage SAN
Device Fabric
Alias WWPN
alex 50:05:76:ab:cd:22:03:65
ed frank ben 50:05:76:ab:cd:12:06:92
Storage Storage chuck 50:05:76:ab:cd:24:05:94
Device Device
dave 50:05:76:ab:cd:20:09:91
ed 50:05:76:ab:cd:23:05:93
Zone1 – alex, dave, ed
Zone2 – ben, frank frank 50:05:76:ab:cd:02:05:94
Zone3 – chuck, frank, gary gary 50:05:76:ab:cd:20:08:09

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-40. WWN zoning

WWN zoning (also called soft zoning) uses the simple name servers in the switches to either allow
or block access to particular World Wide Names (WWNs) in the fabric. When a port contacts the
name server, the name server replies only with information about the ports in the same zone as the
requesting port.
The members of the zone can be defined by using the WWNNs and WWPNs. Zoning software can
also allow you to create symbolic names for the zone members and for the zones themselves. A
member can belong to multiple zones.
There is no need to worry about the physical connections to the switch. In using WWNs for the
zone members, even when a device is connected to another physical port, it remains in the same
zoning definition. A major advantage of WWN zoning is the ability to re-cable the fabric without
having to redo the zone definition.
WWN zoning is susceptible to unauthorized access, as the zone can be bypassed if an attacker is
able to spoof the World Wide Name of an authorized HBA.

© Copyright IBM Corp. 2001, 2017 3-45

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

LUN masking
• Authorization process done
at the device HBA Server A Server B Server C
ƒ Device HBA defines and
enforces the access policies
• Does not replace zoning
SAN
Fabric

Storage
Device

LUN A LUN B LUN C

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-41. LUN masking

It is possible for more than one host to see the same storage device or LUN. This is potentially a
problem, both from a practical and a security perspective.
Logical Unit Number Masking, or LUN masking, is an authorization process that makes a Logical
Unit Number available to some hosts and unavailable to other hosts. LUN masking is mainly
implemented at the host bus adapter (HBA) level. When LUN masking is implemented at the
storage controller level, the controller itself enforces the access policies to the device and as a
result it is more secure. The critical point here is that the allocation of a LUN to a host connection is
made by hiding devices that are not assigned.
LUN masking is not an alternative to zoning. Instead, zoning should be used in conjunction with
LUN masking.

© Copyright IBM Corp. 2001, 2017 3-46

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Zoning best practices

• Implement zoning
• Set default zone policy to deny
• Define only one initiator per zone
• Define tape and disk devices in separate zones
• Define alias names for devices
• Document zones and zone configurations
• Use WWPN rather than WWNN for devices

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-42. Zoning best practices

Zoning is the most common management activity in a SAN. To create a solid foundation for a new
SAN, adopt a set of best practices to ensure that the SAN is secure, stable, and easy to manage.
• Implement zoning, even if LUN masking is being used.
• Set the default zone policy to deny or no access.
• Define a single initiator (HBA) per zone.
• Define tape and disk traffic in separate zones.
• Device aliases or FC aliases should be used to simplify management whenever possible.
Consider names that are meaningful to be able to locate and identify the resource such as rack,
resource type, and port number.
• Keep documented backups of zone members and zones within zone sets. Ensure that the
saved location is accessible even if the SAN fails.

© Copyright IBM Corp. 2001, 2017 3-47

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

SAN design and topology topics

• SAN components
• SAN topologies
• SAN switch ports
• Fibre Channel addressing
• Fibre Channel login process
• SAN zoning
• SAN port virtualization

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-43. SAN design and topology topics

© Copyright IBM Corp. 2001, 2017 3-48

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

N_Port ID Virtualization
• Without NPIV
Server Switch
WWPN HBA F_Port

FCID
• With NPIV

Server Switch
HBA F_Port
WWPN
WWPN NPIV
WWPN support
WWPN
FCID
FCID
FCID
FCID

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-44. N_Port ID Virtualization

Normally, an N_Port would have a single N_Port_ID associated with it. This N_Port_ID is a 24-bit
address assigned by the Fibre Channel switch during the FLOGI process. The N_Port_ID is not the
same as the World Wide Port Name (WWPN), although there is typically a one-to-one relationship
between WWPN and N_Port_ID. Thus, for any given physical N_Port, there would be exactly one
WWPN and one N_Port_ID associated with it.
What N_Port ID Virtualization (NPIV) does is allow a single physical N_Port to have multiple
WWPNs, and therefore multiple N_Port_IDs, associated with it. After the normal FLOGI process,
an NPIV-enabled physical N_Port can subsequently issue additional commands to register more
WWPNs and receive more N_Port_IDs (one for each WWPN). The Fibre Channel switch must also
support NPIV, as the F_Port on the other end of the link would “see” multiple WWPNs and multiple
N_Port_IDs coming from the host and must know how to handle this behavior.
Once all the applicable WWPNs have been registered, each of these WWPNs can be used for SAN
zoning or LUN presentation. There is no distinction between the physical WWPN and the virtual
WWPNs.

© Copyright IBM Corp. 2001, 2017 3-49

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

N_Port Virtualization
No
Domain ID

Server Switch
WWPN HBA F_Port

FCID
Domain ID
Server Switch
WWPN HBA F_Port NP_Port F_Port

FCID NPIV
support
Server FCID
WWPN HBA F_Port FCID
FCID
NPV
FCID
enabled

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-45. N_Port Virtualization

While NPIV is primarily a host-based solution, N_Port Virtualization (NPV) is primarily a

switch-based technology. It is designed to reduce switch management and overhead in larger SAN
deployments. Consider that every Fibre Channel switch in a fabric needs a different domain ID, and
that the total number of domain IDs in a fabric is limited. In some cases, this limit can be fairly low
depending upon the devices attached to the fabric. The problem, though, is that one might need to
add Fibre Channel switches in order to scale the size of your fabric. There is therefore an inherent
conflict between trying to reduce the overall number of switches in order to keep the domain ID
count low while also needing to add switches in order to have a sufficiently high port count. NPV is
intended to help address this problem.
NPV introduces another type of Fibre Channel port, the NP_Port. The NP_Port connects to an
F_Port and acts as a proxy for other N_Ports on the NPV-enabled switch. Essentially, the NP_Port
“looks” like an NPIV-enabled host to the F_Port on the other end. An NPV-enabled switch will
register additional WWPNs (and receive additional N_Port_IDs) using NPIV on behalf of the
N_Ports connected to it. The physical N_Ports do not have any knowledge this is occurring and do
not need any support for it. It is all handled by the NPV-enabled switch.
Clearly, the upstream Fibre Channel switch must support NPIV, since the NP_Port looks and acts
like an NPIV-enabled host to the upstream F_Port. Additionally, because the NPV-enabled switch
now looks like an end host, it no longer needs a domain ID to participate in the Fibre Channel
fabric. Using NPV, you can add switches and ports to your fabric without adding domain IDs.

© Copyright IBM Corp. 2001, 2017 3-50

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Keywords
• Multipathing • Worldwide port name
• IBM Subsystem Device Driver • N_Port ID
• point-to-point connection • Fibre Channel ID
• Fibre Channel Arbitrated Loop • Domain ID
• Fibre Channel Switched Fabric • Area ID
• single switch topology • Node ID
• inter-switch links • Fabric login
• cascade topology • Port login
• ring topology • Process login
• full mesh topology • Fabric services
• core-edge topology • Zones
• edge-core-edge topology • Zone configurations
• N_Port • Port zoning
• F_Port • WWN zoning
• E_Port • LUN masking
• Worldwide name • N_Port ID Virtualization
• Worldwide node name • N_Port Virtualization
SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-46. Keywords

© Copyright IBM Corp. 2001, 2017 3-51

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Review questions (1 of 4)
1. What are the three major components of a SAN?

2. Which SAN topology never uses switch technology?

3. What switch topology requires ISLs to every other switch in

the fabric?
a. Ring
b. Full mesh
c. Core-edge
d. Edge-core-edge
e. Loop

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-47. Review questions (1 of 4)

© Copyright IBM Corp. 2001, 2017 3-52

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Review questions (2 of 4)
4. Which SAN switch port type represents a host end port?
a. N_Port
b. F_Port
c. E_Port
d. VE_Port

5. Which SAN switch port type is used to connect to another

SAN switch?
a. N_Port
b. F_Port
c. E_Port
d. D_Port

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-48. Review questions (2 of 4)

© Copyright IBM Corp. 2001, 2017 3-53

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Review questions (3 of 4)
6. True or False: The source ID in a Fibre Channel frame is
the worldwide node name sending system.

7. True of False: The login that results in the assignment of the

port address is called the PLOGI.

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-49. Review questions (3 of 4)

© Copyright IBM Corp. 2001, 2017 3-54

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Review questions (4 of 4)
8. True or False: A device can participate in multiple SAN
zones.

9. True or False: LUN masking cannot be implemented at the

same time with port zoning.

10. True or False: An NPV enabled switch is assigned a switch

domain ID when inserted into a fabric.

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-50. Review questions (4 of 4)

© Copyright IBM Corp. 2001, 2017 3-55

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 3. SAN design and topology

Uempty

Unit summary
• Classify the three major components of a SAN
• Differentiate between the various SAN topologies and their features
• Distinguish the features and uses of the common and expansion SAN
switch ports
• Contrast the structure and the use of the worldwide name with the Fibre
Channel port address
• Describe the port login process of a node into a SAN switch
• Examine the use of worldwide name and domain-port number for
defining zones in a SAN switch
• Compare the implementation of NPIV with NPV

SAN design and topology © Copyright IBM Corporation 2001, 2017

Figure 3-51. Unit summary

© Copyright IBM Corp. 2001, 2017 3-56

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Unit 4. IBM Fibre Channel b-type

switches and directors
Estimated time
05:30

Overview
The foundational component of a Fibre Channel SAN infrastructure is the Fibre Channel switch or
director. It is the switch's implementation of Fibre Channel services that makes storage networking
a reality.
This unit examines the functional characteristics associated with the IBM Storage System b-type
SAN switches and directors. The steps involved to set up and use the SAN switches are discussed.
Planning and implementation considerations associated with cascading and zoning in multi-switch
configurations are also examined.

How you will check your progress

• Review questions

References
53-1002920-02 Brocade Fabric OS Administrator's Guide
53-1002921-02 Brocade Fabric OS Command Reference Manual
53-1002934-02 Brocade Web Tools Administrator's Guide
SG24-8186 IBM b-type Gen 5 16 Gbps Switches and Network Advisor
http://www.ibm.com/san IBM SAN home page
http://www.brocade.com Brocade home page

© Copyright IBM Corp. 2001, 2017 4-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Unit objectives
• Classify IBM Storage System b-type switches as Enterprise, Midrange,
Entry, and Multiprotocol
• Differentiate the features of the IBM Storage System b-type switch offerings
• Discuss setup considerations for the IBM Storage System b-type SAN
switches
• Employ Web Tools for basic SAN switch administration
• Describe SAN switch functions related to ports, name server, and multi-
switch fabrics
• Explain basic operations of Fabric Shortest Path First routing algorithm
• Recognize the benefits of an inter-switch link trunk environment
• Formulate the procedure to define a SAN zone configuration
• Discuss features of IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-1. Unit objectives

© Copyright IBM Corp. 2001, 2017 4-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-2. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-3. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Fibre Channel switches and directors

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-4. Fibre Channel switches and directors

Despite some interoperability issues, there are many common characteristics and functions in the
offering of various switch manufacturers, for example:
• Self-configuring generic ports (G_Ports) or universal ports (U_Ports).
• Usage of hot-pluggable SFPs.
• Usage of Application-Specific Integrated Circuits (ASICs) to control switch and port logic.
Some manufacturers, such as Brocade and Cisco, provide the capacity to allow fabric-aware
devices to access private loop devices.
The IBM System Storage SAN switches and directors are manufactured by Brocade
Communications Systems (b-type). IBM also has a reseller agreement with Cisco Systems for the
Cisco MDS SAN solutions.

© Copyright IBM Corp. 2001, 2017 4-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Planning for FC switches and directors

Cable length? Number of ports?

SFP: Longwave or shortwave?

Desktop or rack? Switch names and IDs?

Ethernet IP address?
Firmware download

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-5. Planning for FC switches and directors

Some initial planning items associated with implementing a Fibre Channel switch or director
include:
• Where will it physically be installed? Which room? Is it a table top, rack, or floor model? What
about power supply requirements?
• What kind of cable and length is required?
• How many ports are required? Will long-wave (LW), short-wave (SW), or copper connectors be
used?
• Do the hosts and storage devices have the proper level of software and firmware? Has the
vendor's support matrix been reviewed?
• Have the required Fibre Channel host bus adapters and device drivers been identified?
• Will the switch be connected to a public or private communications network? Are IP addresses
and switch naming standards in place?

© Copyright IBM Corp. 2001, 2017 4-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-6. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM System Storage SAN b-type product portfolio

Enterprise

SAN768B-2 (2499-816) SAN384B-2 (2499-416)

Mid Range

SAN48B-5 (2498-F48) SAN96B-5 (2498-F96 & 2498-N96)

Entry

SAN24B-4 Express (2498-B24 & 2498-24E) SAN24B-5 (2498-X24)

Protocol
Multi-

SAN06B-R (2498-R06) SAN42B-R (2498-R42)

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-7. IBM System Storage SAN b-type product portfolio

The IBM System Storage SAN b-type family of switches and directors are used to interconnect
multiple heterogeneous host servers with storage servers and devices, creating a storage area
network (SAN).
Enterprise SAN directors include the IBM Storage System SAN768B-2 which provides up to 512
16-Gb Fibre Channel (FC) ports using 64-port 16 Gb Fibre Channel blades. The IBM System
Storage SAN384B-2 provides up to 256 16-Gb FC ports using 64-port 16 Gb Fibre Channel blades.
The UltraScale Inter-Chassis Links (ICLs) can connect up to nine SAN768B-2 and SAN384B-2
chassis in a full mesh topology or up to 10 chassis in a core-edge topology.
Midrange SAN switches include the IBM System Storage SAN48B-5 which provides up to 48
16-Gb FC ports supporting 2, 4, 8, 10, and 16 Gb speeds or FICON connections. It supports 24, 36,
and 48-port configurations. The IBM System Storage SAN96B-5 provides up to 96 16-Gb FC ports
supporting 2, 4, 8, 10, and 16 Gb speeds. It supports 48, 72, and 96-port configurations.
Entry SAN switches include the IBM System Storage SAN24B-4 Express which provides up to 24
8-Gb FC ports supporting 1, 2, 4, and 8 Gb speeds. It supports 8, 16, and 24-port configurations.
The IBM System Storage SAN24B-5 provides up to 24 16-Gb FC ports supporting 2, 4, 8, and 16
Gb speeds. It supports 12 and 24-port configurations.
Multiprotocol switches include the IBM System Storage SAN06B-R extension switch which
provides up to 16 8-Gb FC ports and up to 6 1-Gb Ethernet ports. Fibre Channel on IP (FCIP) is

© Copyright IBM Corp. 2001, 2017 4-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty
supported. The IBM System Storage SAN42B-R extension switch provides up to 24 16-Gb FC
ports, up to 16 1/10Gb Ethernet ports, and up to 2 40-Gb Ethernet ports. Fibre Channel on IP
(FCIP) is supported.
All these switch models support hot code activation. Interface to the switch for administrative
purposes, such as setting up zoning or obtaining operational statistics, is through an Ethernet port.
A command-line interface (CLI) is available. The Web Tools, a standard base feature, is a web
browser-driven application that can also be used to manage the switch. The Web Tools software
resides in the switch and is delivered as part of the switch microcode.

© Copyright IBM Corp. 2001, 2017 4-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Naming convention: IBM and Brocade

IBM name IBM MTM Brocade name

IBM System Storage SAN768B-2 2499-816 Brocade DCX 8510-8

IBM System Storage SAN384B-2 2499-416 Brocade DCX 8510-4

IBM System Storage SAN96B-5 2498-F96 / N96 Brocade 6520

IBM System Storage SAN48B-5 2498-F48 Brocade 6510

IBM System Storage SAN24B-4 Express 2498-B24 Brocade 300

IBM System Storage SAN24B-5 2498-X24 Brocade 6505

IBM System Storage SAN06B-R 2498-R06 Brocade 7800

IBM System Storage SAN42B-R 2498-R42 Brocade 7840

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-8. Naming convention: IBM and Brocade

The table shows the naming convention of the IBM b-type switches followed by the machine names
in IBM and Brocade.

© Copyright IBM Corp. 2001, 2017 4-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

BROCADE ASIC overview

Primary ASIC responsibilities:
• Routing Fibre Channel frames within the chassis or switch
• Connecting front end (user ports) with back-end system ports
• Manages buffer credit for flow control
IBM SAN switch model ASIC type
IBM System Storage SAN768B-2 Condor3
IBM System Storage SAN384B-2 Condor3
IBM System Storage SAN96B-5 Condor3
IBM System Storage SAN48B-5 Condor3
IBM System Storage SAN24B-5 Condor3
IBM System Storage SAN24B-4 GoldenEye2
IBM System Storage SAN42B-R Condor3
IBM System Storage SAN06B-R GoldenEye2
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-9. BROCADE ASIC overview

ASIC: GoldenEye 2
Speed: 8 Gb
Ports per ASIC: 24
ASIC: Condor3
Speed: 16 Gb
Ports per ASIC: 48

© Copyright IBM Corp. 2001, 2017 4-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SAN Director configuration component

IBM System Storage IBM System Storage

SAN768B-2 SAN384B-2

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-10. SAN Director configuration component

SAN768B-2 features
• A 14U, 8-slot vertical card cage
• Up to 512 ports (equivalent to 640 with UltraScale ICLs) at 16Gb in a single chassis
• 10.2Tb chassis bandwidth
▪ 8.2Tb Fibre Channel/FICON ports / 2.0Tb UltraScale ICL bandwidth
• 512Gb bandwidth per slot
• Two active/passive control processor (CP) blades
• Two active/active core switching (CR16-8) blades
SAN384B-2 features
• A 8U, 4-slot horizontal card cage (9U includes bottom exhaust shelf)
• Up to 256 ports (equivalent to 320 with UltraScale ICLs) at 16Gb in a single chassis
• 5.1Tb chassis bandwidth
▪ 4.1Tb Fibre Channel/FICON ports / 1.0Tb UltraScale ICL bandwidth
• 512Gb bandwidth per slot
• Two active/passive control processor (CP) blades
• Two active/active core switching (CR16-4) blades

© Copyright IBM Corp. 2001, 2017 4-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SAN768B-2 configuration overview

• 12 slot chassis
ƒ 8 port blades
í Slots 1-4 and 9-12
ƒ 2 Control Processor blades
í Slots 6 and 7
ƒ 2 Core blades
í Slots 5 and 8

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-11. SAN768B-2 configuration overview

The core frame routing functionality is handled by the core blades. Port blades are available in 24,
32, 48, and 64-port configurations.
The blades supported by the SAN768B-2 include the 16 Gb 32-port, 48-port Fibre Channel blades,
the 16 Gb 64-port Fibre Channel blade, 8 Gb 64-port Fibre Channel blade, 8 Gb Enhanced
Extension blade, and the FCoE 10 Gb Ethernet 24-port blade.

© Copyright IBM Corp. 2001, 2017 4-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SAN384B-2 configuration overview

• 8 slot chassis
ƒ 4 port blades
í Slots 1-2 and 7-8
ƒ 2 Control Processor blades
í Slots 4 and 5
ƒ 2 Core blades
í Slots 3 and 6

Slot 1

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-12. SAN384B-2 configuration overview

The core frame routing functionality is handled by the core blades. Port blades are available in 24,
32, 48, and 64-port configurations.
The blades supported by the SAN384B-2 include the 16 Gb 32-port, 48-port Fibre Channel blades,
the 16 Gb 64-port Fibre Channel blade, 8 Gb 64-port Fibre Channel blade, 8 Gb Enhanced
Extension blade, and the FCoE 10 Gb Ethernet 24-port blade.

© Copyright IBM Corp. 2001, 2017 4-14

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Control processor blade

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-13. Control processor blade

The SAN768B-2 has two control processor (CP8) blades in slots 6 and 7. Each CP blade has a
USB port, an RS-232 console port, two IP network ports, and dual processors. The USB port only
supports Brocade branded USB drives and can be used for firmware download, support save,
configuration upload, and configuration download.

© Copyright IBM Corp. 2001, 2017 4-15

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Core router blade

• CR16-8 for SAN768B-2 CR16-8 CR16-4
ƒ 16 QSFP ICL ports per CR
í Equivalent of 64 x 16 Gb ports
ƒ 4 Condor3 16 Gb ASICs
• CR16-4 for SAN384B-2
ƒ 8 QSFP ICL ports per CR
í Equivalent of 32 x 16 Gb ports
ƒ 2 Condor3 16 Gb ASICs
• Max connect 10 chassis
• Max trunk group for ICL
ƒ 4 QSFPs

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-14. Core router blade

The CR16-8 blade and the CR16-4 blade provide the core routing of frames either from blade to
blade or from SAN768B-2 to SAN768B-2 (or SAN384B-2) through an ICL cable.
The CR16-8 blade has four Condor3 ASICs and 16 ICL ports. In one chassis the SAN768B-2 has
32 available ICL ports. The CR16-4 blade has two Condor3 ASICs and 8 ICL ports. In one chassis
the SAN384B-2 has 16 available ICL ports.
Each ISL port delivers 64 Gb (4 x 16 Gb) bandwidth.

© Copyright IBM Corp. 2001, 2017 4-16

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Inter chassis link example

• Up to 1152 x 16 Gb ports
• Up to 25 Tb total bandwidth

SAN768B-2

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-15. Inter chassis link example

The visual shows three SAN768B-2 directors connected with ICL in a full mesh topology. There is a
minimum of 256 Gb bandwidth between chassis (12:1 over-subscription). More ICLs can be added
as needed for additional bandwidth and trunking (up to 3:1 over-subscription).

© Copyright IBM Corp. 2001, 2017 4-17

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Port blades options

FC16-32 FC16-48 FC8-64 FC16-64

FCoE10-24 FX8-24

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-16. Port blades options

For both directors (SAN768B-2 and SAN384B-2) in the family, there are the following blade options:
• FC16-32: 32 x 16 Gb FC ports with Condor3 ASICs (512 Gb to core) and 1:1 over-subscription
to backplane
• FC16-48: 48 x 16 Gb FC ports with Condor3 ASICs (512 Gb to core) and 1.5:1
over-subscription to backplane
• FC8-64: 64 x 8 Gb FC ports with Condor2 ASICs (256 Gb to core) and 2:1 over-subscription to
backplane and no FICON support
• FC16-64: 64 x 16 Gb FC ports with Condor3 ASICs (512 Gb to core) and 2:1 over-subscription
to backplane and no FICON support
• FX8-24: 12 x 8 Gb FC ports + 10 x 1 GbE ports + optional 1 x 10 GbE port, max 4 blades per
chassis, FCIP extension blade
• FCoE10-24: 24 x 10 Gb CEE ports, max 1 blade per chassis and no FICON support,
(SAN768B-2 only)

© Copyright IBM Corp. 2001, 2017 4-18

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Blade architecture example: FC16-48 blade on SAN768B-2

CR16-8
FC16-48 C3
User facing
ports
C3
C3
24 x 16 Gb C3
C3
768 Gb

C3
24 x 16 Gb C3
C3
C3

1.5:1 max
C3
oversubscription 512 Gb
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-17. Blade architecture example: FC16-48 blade on SAN768B-2

The FC16-48 is a 48-port blade that can operate on two Condor3 ASICs. Operating at speeds of 2,
4, 8, 10, and 16 Gb, this blade supports F, E, D, M, and EX ports. It provides a 24:16, or 1.5:1
subscription at 16 Gb speeds.

© Copyright IBM Corp. 2001, 2017 4-19

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-18. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-20

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SAN switches and directors setup overview

• Switch setup:
ƒ Verify complete shipment, unpack
ƒ Insert SFPs/GBICs
ƒ Install additional power supply
ƒ Power on self test (POST)
ƒ Set IP address
ƒ Attach to LAN
ƒ Activate Web Tools
ƒ Download firmware and update switch
ƒ Set time and switch name and number
ƒ Connect FC cables and apply labels

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-19. SAN switches and directors setup overview

The visual highlights a list of setup steps.

The Fibre Channel cables attaching hosts and storage devices to the switch are not connected until
after the switch has been set up and configured.

© Copyright IBM Corp. 2001, 2017 4-21

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SAN switch management interfaces

• Command-line interface (Telnet, SSH, Secure Telnet)
• Web Tools (SSL option)
• Fabric Manager (optional)

Fabric Manager

• Multiple fabrics
• Switch groups
• Port groups

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-20. SAN switch management interfaces

Management interfaces to the switched fabric environment:

• The command-line interface (CLI) to manage a switch is accessed using the switch serial port,
Telnet, secure Telnet, or Secure Shell (SSH).
• The Web Tools GUI provides a web-based interface to manager and monitor an entire fabric of
switches.
• The IBM Fabric Manager is an optional application that integrates the management of multiple
fabrics. Multiple switches and ports belonging to one or more fabrics can be grouped for
management purposes (such as downloading firmware to a group of switches).

© Copyright IBM Corp. 2001, 2017 4-22

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Management interfaces using IP network

IBM
Fabric Manager
Telnet Secure
Telnet Web Tools Multiple fabrics
GUI Switch groups
SSH CLI HTTP SSL
Port groups

IP network SNMP v1, v3

NTP RADIUS
server
Serial
connection CLI

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-21. Management interfaces using IP network

The tsclockserver command enables the switch time to be set by a specified NTP (Network Time
Protocol) server.
Fabric OS supports RADIUS authentication, authorization, and accounting services (AAA). When
configured for RADIUS, the switch becomes a Network Access Server (NAS) that acts as a
RADIUS client. Authentication records are stored in the RADIUS host server database.
Fabric OS supports secure sockets layer (SSL) protocol, which provides secure access to a fabric
through web-base management tools such as Web Tools. SSL support is a standard Fabric OS
feature. Refer to the Brocade OS Procedures Guide for details to implement SSL.

© Copyright IBM Corp. 2001, 2017 4-23

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Log in to the switch

Default values:
• Login: admin
• Password: password

Consistent time across switches:

• NTP server, or
• Fabric Manager set time

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-22. Log in to the switch

The default user ID is admin and the default password is password. The switch's prompt consists of
the switch name and the user ID of the logged in user.
Multiple levels of the firmware can coexist in the same fabric. Refer to the product support website
for the latest information on firmware levels. Use the version command to display the current level
of the firmware version running in the switch.
The admin account allows read/write interactions with the switch. The user account allows read
interactions with the switch. Up to 15 additional accounts can be created per switch. Two additional
accounts, root and factory, are designed for support and service and should not be used under
normal circumstances. There are several other role-based access user accounts that can be used
which define capabilities based on the assigned role.
The tsclockserver command is used to identify an external NTP server for time synchronization.
The timer server only needs to be configured on one switch in the fabric. The default is local time. If
configured, the NTP server configuration is propagated to the other switches in the fabric.
The Fabric Manager Set Time interface can also be used to set a consistent time for all the
switches within a fabric.

© Copyright IBM Corp. 2001, 2017 4-24

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Useful switch OS commands

• Show commands • Help commands
ƒ chassisShow ƒ help
ƒ configShow í diagHelp
ƒ fabricShow í fcipHelp
ƒ islShow í routeHelp
ƒ licenseShow í secHelp
í zoneHelp
ƒ nsallShow
ƒ nsShow
ƒ portShow
ƒ switchShow
ƒ zoneShow

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-23. Useful switch OS commands

A rich set of commands is available to interact with the switch operating system. These commands
complement the web browser interface.
When the help command is used without any operands, it displays the entire inventory of
commands in alphabetical order. At the end of the list are additional commands that can be used to
display a subset of related commands. For example, diagHelp displays a list of interface.
The commands are documented in the Command Reference manual of the version of the switch
firmware.

© Copyright IBM Corp. 2001, 2017 4-25

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Display switches in the fabric

fabricShow
DAZZ16A:admin> fabricShow
Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
-------------------------------------------------------------------------
2: fffc02 10:00:00:60:69:e2:01:38 9.11.199.7 0.0.0.0 "Meteor"
3: fffc03 10:00:00:60:69:80:43:b6 9.11.195.215 0.0.0.0 "SecFab1"
4: fffc04 10:00:00:60:69:80:43:b7 9.11.195.216 0.0.0.0 "SecFab3"
6: fffc06 10:00:00:05:1e:34:11:8e 9.11.198.5 0.0.0.0 "DAZZ16A"
7: fffc07 10:00:00:05:1e:34:11:9a 9.11.198.254 0.0.0.0 "DAZZ16B"
9: fffc09 10:00:00:05:1e:34:10:5a 9.11.198.56 0.0.0.0 "DAZZ8A"
10: fffc0a 10:00:00:05:1e:34:10:5d 9.11.199.3 0.0.0.0 "DAZZ8B"
11: fffc0b 10:00:00:60:69:90:03:60 9.11.193.223 0.0.0.0 "IOTL128"
12: fffc0c 10:00:00:60:69:50:03:fd 9.11.193.220 0.0.0.0 "IOTL118"
13: fffc0d 10:00:00:60:69:50:03:e5 9.11.193.216 0.0.0.0 "IOTL119"
14: fffc0e 10:00:00:60:69:50:04:a7 9.11.193.217 0.0.0.0 "IOTL120"
15: fffc0f 10:00:00:60:69:c0:06:d5 9.11.192.74 0.0.0.0 "Mojo1"
16: fffc10 10:00:00:60:69:10:6f:4a 9.11.193.234 0.0.0.0 "IOTL104"
18: fffc12 10:00:00:60:69:30:06:07 9.11.193.222 0.0.0.0 "IOTL127"
19: fffc13 10:00:00:05:1e:34:11:ad 9.11.198.138 0.0.0.0 "DAZZ16C"
29: fffc1d 10:00:00:05:1e:34:13:5c 10.150.10.71 0.0.0.0 "SANBladeFC1"
30: fffc1e 10:00:00:05:1e:34:13:4a 10.150.10.72 0.0.0.0 >"SANBladeFC2"

The Fabric has 17 switches

> = Principal Switch

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-24. Display switches in the fabric

The fabricShow command displays the member switches of the fabric.

• Switch ID: The switch domain ID and embedded port_ID.
• Worldwide Name: The node WWN of the switch.
• Enet IP Addr: The switch Ethernet IP address.
• FC IP Addr: The switch Fibre Channel IP address.
• Name: The switch symbolic name. The > identifies the principal switch of the fabric.
A preferred principal switch can be identified using the fabricPrincipal command.

© Copyright IBM Corp. 2001, 2017 4-26

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Display switch and port status

DAZZ16A:admin> switchshow switchShow
switchName: DAZZ16A
switchType: 66.1
switchState: Online
switchMode: Native
switchRole: Subordinate
switchDomain: 6
switchId: fffc01
switchWwn: 10:00:00:05:33:9f:0f:2d
zoning: ON (Primary_Dazzler_Meteor)
switchBeacon: OFF
FC Router: OFF
Allow XISL Use: OFF
LS Attributes: [FID: 128, Base Switch: No, Default Switch: Yes, Address Mode 0]

Index Port Address Media Speed State Proto

==================================================
0 0 010000 id N8 Online FC F-Port 21:00:00:24:ff:4e:8c:ba
1 1 010100 id N8 Online FC F-Port 21:00:00:24:ff:4e:8c:bb
2 2 010200 id N8 Online FC F-Port 10:00:00:00:c9:e3:cc:12
3 3 010300 id N8 Online FC F-Port 10:00:00:00:c9:e3:cc:13
4 4 010400 id N8 Online FC F-Port 20:54:00:80:e5:37:de:16
5 5 010500 id N8 Online FC F-Port 20:64:00:80:e5:37:de:16
.
.
.
16 16 011000 id N8 No_Light FC

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-25. Display switch and port status

The switchShow command displays the switch and port status.

• The switch, DAZZ16A, is a subordinate switch with a domain ID of 6 (decimal).
• Zoning is active and the activated zoning policy is Primary_Dazzler_Meteor.
• The switch beacon feature is currently turned off.
• The ports have negotiated speeds of 8 Gb (N8).

© Copyright IBM Corp. 2001, 2017 4-27

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Display switch operating parameters

DAZZ16A:admin> configshow
RSCN.end-device.TransmissionMode:0
configShow
alpaList:1
cer.internal_port_code:1
diag.loopID:125
diag.mode.burnin:0
diag.mode.esd:0
diag.mode.lab:0
diag.mode.mfg:0
diag.postDisable:0
diag.retryDisable:0
diag.test.crossPort.passes:5000
diag.test.passes:0
diag.test.portLoopback.passes:1000
diag.test.silkScreen.passes:180
diag.test.spinSilk.passes:120
ether.link.mode:AUTO
fabric.domain:6
fabric.ididmode:0
fabric.ops.BBCredit:16
fabric.ops.E_D_TOV:2000
fabric.ops.R_A_TOV:10000
fabric.ops.dataFieldSize:2112
fabric.ops.mode.fcpProbeDisable:0
fabric.ops.mode.isolate:0
fabric.ops.mode.longDistance:0
fabric.ops.mode.noClassF:0
fabric.ops.mode.pidFormat:1
fabric.ops.mode.tachyonCompat:0
PID Format = 0, 1, 2
fabric.ops.mode.unicastOnly:0
fabric.ops.mode.useCsCtl:0
fabric.ops.mode.vcEncode:0
fabric.ops.vc.class.2:2
fabric.ops.vc.class.3:3
Type <CR> to continue, Q<CR> to stop:

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-26. Display switch operating parameters

The configShow command is used to display the switch system configuration settings.
The pidFormat field is defaulted to 1, that is, the entire area field of the 24-bit port address is being
used to denote the switch port number.
Earlier switch models (S16, S08, and early shipments of F16) have a 0 in the pidFormat field. That
is, the upper half of the area field in the 24-bit port address has a constant value of 1.
If these switches are to be connected to a fabric running with pidFormat set to 1, then ensure the
pidFormat of the joining switch is set to 1 (that is, respond with “1” to the pidFormat option when the
configure command is executed).

© Copyright IBM Corp. 2001, 2017 4-28

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Switch port ID format

L3F3S1:admin> switchdisable

L3F3S1:admin> configure
24-bit Port ID
Switch Domain
Switch Domain Area
Area Port
Port
Configure... 8-bit
8 - bit 8-bit
8 - bit 8-bit
8 - bit

Fabric parameters (yes, y, no, n): [no] y Fabric Loop

Fabric LoopIDID AL_PA
AL_PA
16-bit
16 - bit 8-bit
8 - bit
Domain: (1..239) [21]
BB credit: (1..27) [16]
R_A_TOV: (4000..120000) [10000]
E_D_TOV: (1000..5000) [2000]
Data field size: (256..2112) [2112]
Sequence Level Switching: (0..1) [0]
Disable Device Probing: (0..1) [0]
Suppress Class F Traffic: (0..1) [0]
SYNC IO mode: (0..1) [0] PID formats:
VC Encoded Address Mode: (0..1) [0]
Switch PID Format: (0..2) [0] 1 0 = Native PID (16 ports)
Per-frame Route Priority: (0..1) [0] 1 = Core Switch PID (256 ports)
Long-Distance Fabric: (0..1) [0]
2 = Extended Edge PID (256 ports)
Committing configuration...done.
0x102d7970 (tShell): Jul 8 16:36:39
WARNING CONFIG-PIDCHANGE_CORE, 3, Switch PID format changed to Format 1
('Core PID Format').

L3F3S1:admin> switchenable

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-27. Switch port ID format

The 24-bit port address format is controlled by the Switch PID Format configuration parameter. The
default value of “1” provides addressability for up to 256 ports per switch domain and is
recommended. The area field represents the switch port number in hexadecimal. For example, port
15 (counting from 0) is x'0F', port 31 is x'1F', port 127 is x'7F', and port 255 is x'FF'.
Native PID (PID format = 0) might be found in older switches models (S08, S16, F16). It added an
x'10' to the port number. The area field represents the switch port number plus x'10' - for example,
port 15 is x'1F'. When possible, disable the switch to reset the PID format value to “1”.
Extended Edge PID (PID format = 2) is designed to accommodate the native PID format scheme
and at the same time, support switches containing up to 256 ports. It can be used to avoid host
reboots for those host operating systems that have dependencies on the 24-bit FC port address.
PID format = 2 is implemented such that:
• For port counts of 16 or less, Native PID Format has the same value as Extended Edge PID
Format (port values ranging from x'10' to x'1F'); for example, port 15 is x'1F'.
• For port counts greater than 16, an x'10' is still added to each physical slot (switch blade) but
the last slot wraps back to x'00'. For example, a switch domain of 64 ports would start with port
IDs of x'10' to x'1F' for ports 0 -15, x'20' to x'2F' for ports 16 to 31, x'30' to x'3F' for ports 32 to
47, and x'00' to ‘0F' for ports 48 to 63.

© Copyright IBM Corp. 2001, 2017 4-29

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty
All switches in the same fabric must run with the same PID format value, otherwise the fabric
segments.
To set the PID format, disable the switch with the switchdisable command, then issue the
configure command, enter y to Fabric parameters to change the PID format. Issue the
switchenable command to return to normal processing. The switch domain ID can be set at the
same time.

© Copyright IBM Corp. 2001, 2017 4-30

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Display nodes in a given name server

ATSSAN_fabric2:admin> nsshow nsShow
The Local Name Server has 6 entries {
Type Pid COS PortName NodeName TTL(sec)
N 021a00; 3;21:00:00:eO:8b:02:1b:69;20:00:00:e0:8b:O2:1b:69; na
Fabric Port Name: 20:0a:00:60:69:10:3a:ce
N 021b00; 2,3;l0:00:00:00:c9:20:d8:5f;l0:00:00.00:c9:20.d8:5f; na
Fabric Port Name: 20:0b:00:60:69:10:3a:ce
N 021c00; 2,3;l0:00:00:00:c9:22:ae:a7;20:00:00:00:c9:22:ae:a7; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0c:00:60:69:10:3a:ce
N 021d00; 2,3;l0:00:00:00:c9:22:72:48;20:00:00:00:c9:22:72:48; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0d:00:60:69:10:3a:ce
N 021e00; 2,3;l0:00:00:00:c9:22:af:a3;20:00:00:00:c9:22:af:a3; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0e:00:60:69:10:3a:ce
N 021f00; 2,3;l0:00:00:00:c9:22:af:ae;20:00:00:00:c9:22:af:ae; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0f:00:60:69:10:3a:ce
}

ATSSAN_fabric2:admin> nsallshow
nsAllShow
14 Nx_Ports in the Fabric {
011a00 011b00 011c00 011d00 011e00 011f00 021a00 021b00
021c00 021d00 021e00 021f00 031700 0318ef
}
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-28. Display nodes in a given name server

The nsShow command displays entries in the local name server of the switch.
• Type: N for N_Port, NL for NL_Port, or U for unknown.
• Pid: 24-bit port ID.
• COS: Classes of service supported by the device.
• PortName: Port WWN of the device.
• NodeName: Node WWN of the device.
• TTL: Time-to-live (in seconds) for cached entries from other name servers or N/A (not
applicable) for local entries.
• FC4s: Upper level protocols supported by the device
• Fabric Port Name: Port WWN of port to which the device is attached.
The nsAllShow command displays the 24-bit port identifiers of all devices connected to the fabric.

© Copyright IBM Corp. 2001, 2017 4-31

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Web Tools interface

• Management views: • Configuration:
ƒ Switch view ƒ Switch admin
ƒ Port view ƒ Port admin
ƒ Name server view ƒ Zone admin
ƒ Performance monitor view
ƒ Fabric watch view

Windows
Zone 1
IBM Zone 3

Sun Zone
2
Web browser interface

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-29. Web Tools interface

Web Tools is an embedded graphical user interface (GUI) that enables administrators to monitor
and manage single or small fabrics, switches, and ports. Web Tools is launched directly from a web
browser, or from the Network Advisor.
A limited set of features is accessible using Web Tools without a license, and is available free of
charge. Additional switch management features are accessible using Web Tools with the Enhanced
Group Management (EGM) license. Beginning with Fabric OS version 6.1.1, some Web Tools
capabilities are moved from Web Tools to Network Advisor. Review the Web Tools Administrator’s
Guide for more details.

© Copyright IBM Corp. 2001, 2017 4-32

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Fabric tree and switch view

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-30. Fabric tree and switch view

To launch Web Tools, enter the switch DNS name or IP address in the URL field of a web browser.
If multiple switches are connected to form one fabric, regardless of which switch is the launching
point, all switches within the same fabric are accessible without launching another Web Tools
browser session.
Java Plug-in must be installed on the workstation to be able to launch the switch explorer.
The Switch View tab lists switches that are in the fabric tree, a graphical view of the switch front
panel, and a listing of selected switch information. Selecting a switch in the fabric tree causes the
detailed information associated with that switch to be displayed.
Navigation controls are displayed at the top of the screen.
With the exception of some diagnostic and configuration settings, most CLI commands have a
functional equivalent in Web Tools.

© Copyright IBM Corp. 2001, 2017 4-33

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Name Server view

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-31. Name Server view

The Name Server tab displays the name server database. This data represents all name server
entries for the fabric as viewed by the selected logical switch.
The total number of ports in the fabric that are currently logged in to the Name Server is displayed
at the top of the Name Server table.
Selecting an entry (highlighted entry) causes the Detail view and the Accessible Devices buttons
to be functional.

© Copyright IBM Corp. 2001, 2017 4-34

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Name Server entry detail

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-32. Name Server entry detail

The Detail View button of a selected entry in the Name Server table displays information for the
chosen device in a display window for ease of reading.
Similarly, the Accessible Devices button of the selected entry in the Name Server table displays
other devices in the same zone as the selected device.

© Copyright IBM Corp. 2001, 2017 4-35

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Switch view SAN768B-2

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-33. Switch view SAN768B-2

Switch entries in the Fabric Tree are cursor sensitive. Additional information is displayed when the
cursor is placed over the switch name. Placing the cursor over a port displays the port number,
type, and status. If the cursor is placed on a blade, then the blade ID is displayed.
The Active CP indicator is lit on the CP that is currently active. Detailed status of the CPs can be
determined by selecting the HA button in Switch view. The haShow CLI command provides similar
information. The color of the HA button indicates the overall status of the switch.

© Copyright IBM Corp. 2001, 2017 4-36

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Director high availability CP status

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-34. Director high availability CP status

The status of both CPs, including the firmware levels, are displayed by the CP tab. A non-disruptive
CP failover is only possible when all the fabric services have been synchronized between the two
CPs. The current status is displayed in the HA status field at the top of this window.
If the HA Status field displays HA enabled, Heartbeat up, HA State not in Sync, the
Synchronize Services button will proceed to complete a synchronization of services.
The Initiate Failover button will initiate a CP failover.

© Copyright IBM Corp. 2001, 2017 4-37

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Switch events log

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-35. Switch events log

The Switch Events tab on the Switch View displays the error event log of the selected switch. To
sort the events by column, click a column header. To resize a column, drag the column dividers. For
example, drag the Message column to see the entire message.
The CLI errShow command shows the log entries one entry at a time (per command prompt). The
CLI errDump command dumps the switch error event log.

© Copyright IBM Corp. 2001, 2017 4-38

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Port admin view

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-36. Port admin view

The Port Admin tab from the Switch view displays general information, SFP information, and
statistics associated with a selected port. Port numbers are listed in a tree structure down the left
side of the panel and allow another port to be selected without returning to the Switch view.
The port information is automatically updated when the view is opened and is periodically refreshed
while the view remains open.
The information displayed is similar to the portStatsShow CLI command.

© Copyright IBM Corp. 2001, 2017 4-39

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Basic performance monitoring options

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-37. Basic performance monitoring options

The Performance Monitor contains a collection of graphs on the display panel, or canvas. The
graphs are sized and loaded on the canvas based on the number of performance monitoring
options selected.
Access them through the Monitor > Performance Monitor options. Basic monitoring options,
available as part of the standard feature of the switch, are:
• Port Throughput: Displays the performance of a port based on the 4-byte transmission words
received and transmitted.
• Switch Aggregate Throughput: Displays the aggregated performance of all ports of the switch.
• Switch Throughput Utilizations: Displays the port throughput of each port at the time the sample
is taken.
• Port Error: Displays the CRC errors for a given port.
• Switch Percent Utilization: Displays the percentage of usage of a chosen switch at the time the
sample is taken.
• Port Snapshot Error: Displays the CRC error count between sampling periods for all the ports
on a switch.

© Copyright IBM Corp. 2001, 2017 4-40

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Fabric Watch

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-38. Fabric Watch

Fabric Watch is an optional licensed feature that monitors the performance and status of switches.
Fabric Watch tracks a number of SAN fabric elements, events, and counters. For example, Fabric
Watch monitors the following:
• Fabric resources, including fabric reconfigurations, zoning changes, new logins, domain ID
changes, E_Port failures, and segmentation changes
• Switch environmental functions, such as temperature, flash, CPU and memory usage, along
with security violations
• Port state transitions, errors, and traffic information for multiple port classes.
To use Fabric Watch, you must have the Fabric Watch license or Fabric Vision license installed on
the switch.

© Copyright IBM Corp. 2001, 2017 4-41

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-39. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-42

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Authority for administration and zoning

• Administrative authority level required to access:
ƒ Administrative pages
ƒ Zoning pages

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-40. Authority for administration and zoning

To administer switch configuration functions, including zoning configuration, the administrator must
have administrative privileges. The more restrictive level, user privilege, cannot perform zone
configuration and switch administration functions, but can view the information.

© Copyright IBM Corp. 2001, 2017 4-43

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Switch configuration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-41. Switch configuration

The Switch tab in the Switch Admin view is used to manage basic switch setup such as assign
switch name, assign switch domain ID, enable or disable the switch, configure email DNS server,
and generate a switch status report.
The View Report button provides a printable (CLI-like) report for the switch.
Buttons at the lower right side of the page can be used to Apply changes to save, Close to exit the
Switch Admin view, or Refresh to obtain latest switch information.
For each tab of the Switch Admin window, a message area at the bottom of the administrative
interface displays the status of the changes that were applied to the switch.

© Copyright IBM Corp. 2001, 2017 4-44

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Switch information report

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-42. Switch information report

The View Report button from the Switch tab obtains a printable (CLI-like) report for the switch.
The Switch Information Report contains:
• List of switches connected to the fabric (fabricShow) and firmware level.
• List of inter-switch links for this switch (islShow)
• List of ports (switchShow)
• Name Server data (nsAllShow and nsShow)
• Zoning information (cfgShow)
• SFP module type and serial number for each of the ports (sfpShow)
A more comprehensive dump of switch status for debugging purposes is provided by the
supportShow command.

© Copyright IBM Corp. 2001, 2017 4-45

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Network configuration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-43. Network configuration

The Network tab configures the IP networking addresses.

The Syslog IP address is used for sending events using the syslog protocol to a host. If messages
are not to be sent, enter none or leave blank. The IP address is the server that is running the
syslogd (syslog daemon) process. Syslogd is available on most UNIX systems that read and
forward system messages to the appropriate log files or users, depending on the system
configuration. When one or more IP addresses are configured, the switch forwards error log entries
to the syslogd on the specified servers.
Up to six Syslog IP addresses can be configured. The Add, Remove, and Clear All buttons
provide management of the Syslog IP list.
The message area at the bottom of the window can be expanded or compressed by clicking the up
or down arrows in the upper-left corner of the message area.

© Copyright IBM Corp. 2001, 2017 4-46

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Firmware download

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-44. Firmware download

The Firmware tab is used to download firmware upgrades to the switch. Firmware can be
accessed using the network or the USB.
When USB is selected, only the firmware path or directory name can be specified. The default path
for Windows and Linux is /usb/usbstorage/brocade/firmware/<version>. No other fields on
the tab are available. The USB button is available if the USB is present on the switch.
When Network is selected, there are fields to enter the host name or IP address, user name,
password, and fully qualified path to the file release.plist. Protocol access options include FTP
and SCP.
About halfway through the download process, after the firmware key is downloaded to the switch,
connection to the switch is lost and Web Tools invalidates the current session. Web Tools
invalidates all windows because upfront login is always enabled and cannot be disabled. Close all
Web Tools windows and log in again.

© Copyright IBM Corp. 2001, 2017 4-47

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Hot code activation

Two code partitions per CP

Primary Secondary
Current New Firmware download
level level

Primary Secondary
Current New Auto reboot
level level

Primary Secondary
New current New current Auto commit
level level

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-45. Hot code activation

Switches running FOS V4.1 or higher support hot code activation, except the SAN06B-R router.
Each switch CP has two partitions: primary and secondary. Firmware download can automatically
reboot and then automatically commit the firmware so that both partitions are updated.
Code is downloaded to the secondary partition. After the download, the partitions are swapped so
that the next reboot invokes the downloaded or new level of code. The firmwareDownload
command prompts for input to determine whether the reboot is to occur after the download and if
the new code should be automatically committed after reboot.
Additional CLI commands include:
• firmwareShow: Displays FOS versions in both partitions.
• firmwareRestore: Restores the old active firmware image.
• firmwareCommit: Copies the updated firmware to the other partition and commits both partitions
(after the system reboot has taken place).
• firmwareDownloadStatus: Displays the event log that records the current progress and status of
the current firmwareDownload command.
For directors, both CPs are upgraded as part of the download. When the firmwareDownload
command is submitted, the firmware is downloaded to the standby CP. The standby CP then forces

© Copyright IBM Corp. 2001, 2017 4-48

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty
a CP failover and firmware is downloaded to the new standby CP. The new standby CP is rebooted
and the firmwareCommit command is executed on both CPs.
The firmwareDownload command also allows the option to upgrade only one CP instead of both.

© Copyright IBM Corp. 2001, 2017 4-49

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SNMP administration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-46. SNMP administration

Every switch contains an embedded SNMP agent and MIBs. The agent can receive queries from
one or more management stations, and can send traps to up to six management stations.
The SNMP tab is used to set SNMP options. The configuration includes both SNMPv1 and
SNMPv3 parameters, and access control list.
In order for the switches to send SNMP traps, the CLI snmpMibCapSet command should be issued
first. This enables the Management Information Bases (MIBs) on all switches to be monitored. The
snmpConfig command is the CLI interface to configure the SNMP environment.

© Copyright IBM Corp. 2001, 2017 4-50

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

License administration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-47. License administration

The License tab is used to view a list of installed license keys and features. It is also used to add or
remove licenses. Two examples of licensed features are Fabric Watch and Ports on Demand.
Licenses are based on the selected features and the switch WWNN. So if multiple switches are to
receive licenses, ensure that the correct license is applied to the appropriate switch.

© Copyright IBM Corp. 2001, 2017 4-51

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Port administration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-48. Port administration

The Port Admin tab to manage port attributes.

Each port is selectable in the tree to the left. General information is displayed for the selected port.
The Actions pull-down provide options to modify the selected port including Rename the port, set
Persistent Enable/Disable, and set Port Trunking.

© Copyright IBM Corp. 2001, 2017 4-52

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

User administration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-49. User administration

In addition to the root, factory, admin and user accounts, several additional accounts can be
created with selected default or user-defined roles. Default Fabric OS roles include Admin,
BasicSwitchAdmin, FabricAdmin. Operator. SecurityAdmin, SwitchAdmin, User, and ZoneAdmin.
The switchAdmin role was introduced with FOS v5. It has similar functions as the admin role but
cannot change the zoning configuration or perform user account management.
FOS supports up to two concurrent sessions for the Admin role and up to four sessions for each of
the other role types.
CLI commands include roleConfig to manage user-defined roles, classConfig to view RBAC
information about each category or class of commands, and userConfig to assign user-defined
roles to a user account.

© Copyright IBM Corp. 2001, 2017 4-53

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Configure switch parameters

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-50. Configure switch parameters

The Configure tab provides similar function as the CLI configure command. The following actions
that are controlled from this tab include fabric parameters, virtual channel parameters, arbitrated
loop parameters, systems services, Class-Specific Control (CSCTL) QoS mode services, and
firmware signed services.
Many of the values are shaded in gray to indicate the read-only attribute. To modify the value, the
switch must first be disabled.

© Copyright IBM Corp. 2001, 2017 4-54

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Configuration upload/download

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-51. Configuration upload/download

The Upload/Download subtab of the Configure tab allows the switch configuration to be uploaded
to a host computer for backup.

© Copyright IBM Corp. 2001, 2017 4-55

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

RADIUS Configuration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-52. RADIUS Configuration

Fabric OS supports Active Directory, RADIUS, and TACACS+ for authentication, authorization, and
accounting services (AAA). When configured for RADIUS, the switch becomes a Network Access
Server (NAS) that acts as a RADIUS client. Authentication records are stored in the RADIUS host
server database.
Login and logout account name, assigned role, and time accounting records are centralized and
stored in the RADIUS server.
Configure at least two RADIUS servers for availability. Local switch authentication can be enabled
so that if no connection to RADIUS servers, the switch uses local authentication.
The AAA Service tab and the aaaConfig CLI command control AAA interfaces. AAA services are
disabled by default. Refer to the Brocade Fabric OS Procedures Guide for detailed implementation
steps.

© Copyright IBM Corp. 2001, 2017 4-56

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Trace

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-53. Trace

A trace dump is a snapshot of the running behavior within the switch. The dump can be used by
product support personnel for debugging purposes. For example, a trace dump can be created
each time a certain error message is logged to the system error log. Product developers can then
examine the events that led up to the message.
Tracing is set on by default. As software on the switch executes, the trace information is placed into
a circular buffer in system RAM. Periodically, the trace buffer is frozen and saved. This saved
information is a trace dump. A trace dump is generated when a traceDump command is manually
issued, a critical-level LOG message occurs, a particular LOG message occurs (the traceTrig
command is used tailor the conditions), a kernel panic occurs, or the hardware watchdog timer
expires.
The trace dump is maintained on the switch until either it is uploaded to the FTP host or another
trace dump is generated (that is, it overrides the previous dump). The Trace tab can be used to
enable a trace dump to be uploaded to a configured FTP host. The Auto FTP Upload option
enables a trace dump to be automatically uploaded when the dump is generated.
The supportSave command can be used to package all error logs, the supportShow command
output, and the trace dump, and move the data to an FTP server.

© Copyright IBM Corp. 2001, 2017 4-57

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-54. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-58

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Switch: Port interfaces

N_Port

N_Port
Switch/Director Switch/Director
F_Port F_Port
E_Port E_Port
N_Port F_Port Public Loop
FL_Port

Storage System NL_Port

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-55. Switch: Port interfaces

The following ports are defined by Fibre Channel:

• E_Port (Expansion Port): Is the connection between two Fibre Channel switches. When
E_Ports between two switches form a link, that link is referred as an Inter-Switch Link or ISL.
• EX_Port: Is the connection between a Fibre Channel router and a Fibre Channel switch. On the
side of the switch it looks like a normal E_Port, but on the side of the router it is a EX_Port.
• F_Ports (Fabric Port): Is a Fabric connection in a switched fabric topology. An F_Port is not loop
capable.
• FL_Port (Fabric Loop Port): Is the fabric connection in a public loop for an arbitrated loop
topology. Note that a switch port might automatically become either an F_Port or an FL_Port
depending on what is connected.
• L_Port (Loop Port): Is the loose term used for any arbitrated loop port. NL_Port or FL_Port.
• N_Port (Node Port): Is the node connection pertaining to hosts or storage devices in a
point-to-point or switched fabric topology.
• NL_Port (Node Loop Port): Is the node connection pertaining to hosts or storage devices in an
arbitrated loop topology.

© Copyright IBM Corp. 2001, 2017 4-59

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Switch port initialization

U_Port What is plugged into the port?

FL_Port Loop device?

G_Port

E_Port F_Port
Another switch? Point-to-point device?

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-56. Switch port initialization

Universal ports allow any device to be connected to any port of the switch. During port initialization,
the determination of port type is automatic. From the initial U_Port state, port initialization
progresses through the following stages when something is plugged into the port:
1. If the attached device is a loop device, the port state is set to FL_Port.
2. Otherwise, the port state transitions to G_Port and port initialization continues.
3. If the attached device is another switch, the port becomes an E_Port.
4. Otherwise, the port state is set to F_Port.
To allow LTO generation 2 tape drives to function as N_Ports instead of NL_Ports, use the
portCfgGport command to set the attaching switch port to function as a G_Port; thus bypassing
loop port initialization.

© Copyright IBM Corp. 2001, 2017 4-60

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

E_Port functions
• Connects two switches to create a fabric
• Uses virtual channels within the ISL for performance

E E

ISLs

E E

Switch Switch

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-57. E_Port functions

The E_Port allows two switches to be connected to form a fabric. The link that connects the two
switches is called ISL (inter-switch link).
For performance, each ISL supports eight virtual channels (VC).
• VC 0: For link control frames, operates with priority 0.
• VC 1: For Class 2 ACKs and link control frames, operates with priority 1.
• VC 2 through VC 5: For data frames traffic, operates with priority 2.
• VC 6: For multicast traffic, operates with priority 3.
• VC 7: For broadcast traffic, operates with priority 3.
Default settings and priorities for virtual channels are already optimized for performance. While they
can be changed, the recommendation is to use the shipped defaults.

© Copyright IBM Corp. 2001, 2017 4-61

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

F_Port functions
• Connects N_Ports to fabric:
ƒ Assigns 24-bit Port_ID
ƒ Supports Class 2 and Class 3 services
ƒ Registers N_Port to name server

Server

N_ Port

F_ Port
Switch

F_ N_

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-58. F_Port functions

The F_Port provides the means for node ports to connect to a fabric. It is the conduit for frame
delivery to connected nodes.
During the fabric login (FLOGI) of a device, the Fabric F_Port Login Server automatically registers
to the Name Server the following attributes associated with the device:
• 64-bit Node WWN
• 64-bit Port WWN
• 24-bit Port ID
• Port type
• Classes of service

© Copyright IBM Corp. 2001, 2017 4-62

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

FL_Port functions
• Connects FC_AL Ports to fabric:
ƒ Registers public and private devices to name server
NL_Port
ƒ Provides translative mode
í Public host <=> Private device

FL_Port
ƒ Supports QuickLoop
í Private host <=> Fabric device Switch

Hub
NL_Port

NL_Port

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-59. FL_Port functions

The FL_Port enables a loop of devices to be connected to the fabric and share the same fabric
port. It is the conduit for frame delivery to the connected loop device. The FL_Port supports
standard FC-AL processing. It has an AL_PA value of x'00' and always wins loop arbitration. The
embedded port occupies AL_PA x'01', leaving room for 125 additional devices that could potentially
be attached to the same loop.
Like the F_Port, the FL_Port also automatically registers device attributes to the Name Server
during the fabric login (FLOGI) of the device. Both public and private devices are automatically
registered.
Two additional functions are provided by the FL_Port to support loop devices: Translative Mode
and QuickLoop. Both functions provide enhance fabric attachment support of legacy private loop
devices.

© Copyright IBM Corp. 2001, 2017 4-63

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Name Server: Distributed model

Name Server db Name Server db

Switch Switch
Distributed
Name Server
Switch
Switch
(FC - CT)

Name Server db db
Name Server
Fabric

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-60. Name Server: Distributed model

Each b-type switch or director contains a Name Server that maintains a directory of all devices
locally attached to that switch and functions as the Name Server for these attached devices.
Functionally the Name Server is implemented as a distributed model so that devices attached to
the fabric (regardless of which switch), appears to be in the same Name Server database.
When a query request from a device requires data from a remote switch, Name Server-to-Name
Server communication occurs to obtain data from the remote switch. This communication is
transparent to the requester.
Communication among Name Servers is based on the FC-CT (Fibre Channel Common Transport)
standard. Directory entries obtained from a remote Name Server database are cached in the local
Name Server database for 15 minutes.

© Copyright IBM Corp. 2001, 2017 4-64

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Name Server: Functional view

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-61. Name Server: Functional view

When the Name Server table is displayed using the Web Tools, the data is derived from the Name
Server databases of all the switches in the fabric.
The Name Server function in the switch OS provides the ability for node ports attached to a fabric to
register its attributes and query the existence of other node ports and their attributes.
A database containing node WWNs, port WWNs, 24-bit port addresses, classes of service,
protocols supported, IP addresses, and port characteristics for each participant is maintained by the
Name Server.

© Copyright IBM Corp. 2001, 2017 4-65

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Name Server: Switch view

ATSSAN_fabric2:admin> nsshow
The Local Name Server has 9 entries {
Type Pid COS PortName NodeName TTL(sec)
*N 011f00; 2,3;10:00:00:00:c9:22:ae:68;20:00:00:00:c9:22:ae:68; 885 Remote entry
FC4s: FCP
PortSymb: [28] "EMULEX FIREFLY "
Fabric Port Name: 20:0f:00:60:69:10:41:b6
N 021a00; 3;21:00:00:e0:8b:02:1b:69;20:00:00:e0:8b:02:1b:69; na
Fabric Port Name: 20:0a:00:60:69:10:3a:ce
N 021b00; 2,3;10:00:00:00:c9:20:d8:5f;10:00:00:00:c9:20:d8:5f; na
Fabric Port Name: 20:0b:00:60:69:10:3a:ce
N 021c00; 2,3;10:00:00:00:c9:22:ae:a7;20:00:00:00:c9:22:ae:a7; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0c:00:60:69:10:3a:ce Local entries
N 021d00; 2,3;10:00:00:00:c9:22:72:48;20:00:00:00:c9:22:72:48; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0d:00:60:69:10:3a:ce
N 021e00; 2,3;10:00:00:00:c9:22:af:a3;20:00:00:00:c9:22:af:a3; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0e:00:60:69:10:3a:ce
N 021f00; 2,3;10:00:00:00:c9:22:af:ae;20:00:00:00:c9:22:af:ae; na
FC4s: FCP [EMULEX FIREFLY ]
Fabric Port Name: 20:0f:00:60:69:10:3a:ce
*N 031700; 2,3;10:00:00:00:c9:24:60:0b;50:05:07:63:00:c0:0b:96; 885
FC4s: FCP Remote entries
PortSymb: [28] "IBM 2105F20 1003"
Fabric Port Name: 20:07:00:60:69:10:39:6a
*NL 0318ef; 3;20:01:00:60:45:16:15:2f;10:00:00:60:45:16:15:2f; 885
FC4s: FCP
PortSymb: [28] "PATHLGHTSAN Gateway 329V"
Fabric Port Name: 20:08:00:60:69:10:39:6a
}

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-62. Name Server: Switch view

The nsShow command displays the local Name Server's database. Cached entries from other
switches have an “*” prefix in the Type field and a numerical value (in seconds) in the TTL
(time-to-live) field.

© Copyright IBM Corp. 2001, 2017 4-66

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Dual fabric

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-63. Dual fabric

Multiple switches can be used to enhance availability. The simplest implementation of multiple
switches is to have two switches that are not interconnected. When switches are not
interconnected, a dual fabric (two fabrics) is created.
In this implementation each host device is connected to both switches. If one switch fails, data is
automatically rerouted to the other switch. This would still require host or device driver software to
recognize path failure and failover to the alternate path on the redundant switch.
This implementation is referred to as a dual fabric or a redundant fabric environment.

© Copyright IBM Corp. 2001, 2017 4-67

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Multi-switch fabric

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-64. Multi-switch fabric

A complex fabric can be made with interconnected switches and directors, and can even span a
LAN/WAN connection. The challenge is to route the traffic while maintaining reliability with a
minimum of overhead, latency, and to prevent out-of-order delivery of frames.
Cascaded switches is used to refer to interconnected switches. Multiple interconnected switches
can form a large logical any-to-any fabric allowing a device to access any other device connected
to any one of the switches.
A set of interconnected switches where each switch is directly connected to all the other switches,
is called a fully meshed fabric.

© Copyright IBM Corp. 2001, 2017 4-68

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Fault-tolerant fabric

Switch Switch Switch Switch

k
L in e r
a i lov
F

Switch Switch

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-65. Fault-tolerant fabric

A partially meshed network provides multiple data paths among its participants. A partially meshed
topology can be arranged such that if any link or switch fails, the remaining switches can still
communicate.
A fully meshed fabric is a meshed fabric where any one switch is a single hop from any other
switch. This minimizes latency across the fabric, and if any link fails, all switches can still
communicate with each other with a guaranteed hop count not to exceed two. A fully meshed fabric
keeps traffic through switches (hops) to a minimum, but greatly reduces the number of ports
available for other devices as the number of switches increases in the fabric.

© Copyright IBM Corp. 2001, 2017 4-69

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Multiple inter-switch links

Switch 1 Data Traffic Switch 2

Data Traffic

Data Traffic
Data Traffic

Switch 3

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-66. Multiple inter-switch links

The two purposes of inter-switch links (ISLs) are:

• Expand the number of ports available in a fabric by connecting switches.
• Provide redundant paths between two switches for availability and performance.
Multiple ISLs operate concurrently between any two switches in the fabric, allowing multiple
redundant paths to be defined. All the ISLs carry traffic. In the event of an ISL failure, routing is
automatically reconfigured so that another ISL would be used.

© Copyright IBM Corp. 2001, 2017 4-70

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Cross-connected fabric

Edge Edge
switches switches

Cross-connect
switches

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-67. Cross-connected fabric

A switch might be used as a core switch. That is, it is used only to connect to other switches. More
switches can be added to the fabric non-disruptively allowing the configuration to expand in concert
with the SAN requirements of the installation.
A highly available, scalable fabric can be built with multiple core or cross-connected switches which
are used for redundancy. Edge switches are used to connect host systems and storage.
This design can scale in terms of both port density and fabric throughput. If more devices need to
be connected to the fabric, more edge switches can be added. Adding more cross-connected
switches can increase performance by providing additional data paths and distributing the workload
among server and storage edge switches.

© Copyright IBM Corp. 2001, 2017 4-71

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Exercise: To cascade or not?

Server 2 Server 3 Server 4 Server 5

8-port
switch

Disk Storage
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-68. Exercise: To cascade or not?

Five servers are connected to an 8-port switch. Each server requires a sustained bandwidth of
approximately 45 MBps.
To accommodate this traffic at the storage level, three links are connected to the storage array. All
three links can each carry traffic at 100 MBps. All servers must go through a switch. No redundancy
is required.
1. A sixth server, also requiring a maximum bandwidth of 45 MBps, needs to be connected to the
disk storage. How could this be achieved?
2. Assume that the sixth server requires a maximum bandwidth of 70 MBps. How would this
change the configuration?

© Copyright IBM Corp. 2001, 2017 4-72

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-69. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-73

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Routing mechanisms: Fabric shortest path first

• Provides dynamic selection to the fastest path or least number of hops
path
ƒ Every link between switches has an assigned cost value
í Link cost is based on switch latency, link speed, and congestion
ƒ The total cost of every path between every switch is computed
ƒ The least cost path is the selected path between any two switches
• Each switch maintains a topology database
ƒ Includes link costs for each switch
• FSPF protocol defines mechanisms to communicate changes in the
fabric
ƒ Path costs are recalculated when fabric changes occur
í When add/remove ISLs or switches in the fabric

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-70. Routing mechanisms: Fabric shortest path first

According with the FC-SW-2 standard, fabric shortest path first (FSPF) is a link state path selection
protocol. The concepts used in FSPF were first proposed by Brocade, and have since been
incorporated into the FC-SW-2 standard. Since then it has been adopted by most, if not all,
manufacturers.
This protocol keeps track of the state of the links on all switches in the fabric. It also associates a
cost with each link. The protocol computes paths from a switch to all other switches in the fabric by
adding the cost of all the links traversed by the path and choosing the path that has the lowest total
cost. In order for the protocol to work, the cost must be a positive integer number.

© Copyright IBM Corp. 2001, 2017 4-74

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Fabric shortest path first

(2 Hops)

(3 Hops)

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-71. Fabric shortest path first

Fabric Shortest Path First (FSPF) route selection protocol automatically computes the shortest
equivalent paths between any two switches in a fabric when switches are powered up, and
programs the hardware routing tables in the switches accordingly. If multiple equivalent paths are
available between two switches, the traffic load is shared among these paths.
The routing is changed in the event of a failure, or if a new ISL is added that provides an equal or
better path to some destinations.
FSPF is resilient to failures, automatically computing an alternate path around a failed link (typically
in less than 1 second).

© Copyright IBM Corp. 2001, 2017 4-75

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

FSPF: Link cost metric

1 Gbps
2 Gbps

(2 Hops)
Cost metric = 2000

(3 Hops)
Cost metric = 2000

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-72. FSPF: Link cost metric

Each hop is assigned a link cost metric value. For a 1 Gb link, the value is 1000 and for a 2 Gb link,
the value is 500.
If there are two routes with equal cost metric values, then the traffic load sharing algorithm assigns
one server connected to the switch to one route path, and the other server connected to the same
switch to the other available route path.

© Copyright IBM Corp. 2001, 2017 4-76

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

FSPF: Load sharing

(2 Hops)

(2 Hops)

Loading Sharing

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-73. FSPF: Load sharing

If two equivalent paths of two hops each are available to the destination switch both routes are
used to share the traffic load. Each server is assigned its own route path.
All switch models support port-based routing. This means that the routing path is chosen for an
incoming frame based only on the incoming port and the destination domain. The dynamic load
sharing feature (DLS) can be enabled to balance the load across the available output ports within
the domain.
Two additional routing policy options include:
• Device-based routing: The routing path is based on the FC address of the source device (S_ID)
and the destination device (D_ID), improving path utilization for better performance.
• Exchange-based routing: The routing path is based on the S_ID, D_ID, and the FC originator
exchange ID (OXID), optimizing path utilization for best performance. This is the default.
The aptPolicy command displays and sets the switch routing policy.

© Copyright IBM Corp. 2001, 2017 4-77

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Display all paths to domains: topologyShow

topologyShow

ATSSAN_fabric1:view> topologyShow
4 domains in the fabric; Local Domain ID: 1
Domain Metric Hops Out Port In Ports Flags Name
-----------------------------------------------------------------
2 2000 2 0 0x0000a800 D "ATSSAN_fabric2"
1 0x00005000 D

3 1000 1 0 0x0000f802 D "ATSSAN_fabric3"

4 1000 1 1 0x0000f801 D "ATSSAN_fabric4"

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-74. Display all paths to domains: topologyShow

The topologyShow command displays the fabric topology as it appears to the local switch. It
displays all the paths, from this switch to other domains in the fabric. For each destination domain,
it identifies the output E_Port from this switch and the number of hops required to get to that
domain.
Each hop has a cost metric of 1000 for 1 Gb links and a cost metric of 500 for 2 Gb links by default.
The metric value can be changed with the linkCost command. All paths discovered by FSPF path
selection protocol are identified as dynamic (D in the Flags column).

© Copyright IBM Corp. 2001, 2017 4-78

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-75. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-79

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Inter-switch link trunking

A 2.0 Gb A 2.0 Gb
Congestion
B 1.5 Gb A, E B 1.5 Gb
B
C 0.5 Gb C C 0.5 Gb
D
D 1.0 Gb D 1.0 Gb
Individual 2Gb links
E 2.0 Gb E 2.0 Gb

A 2.0 Gb A 2.0 Gb
B 1.5 Gb Single B 1.5 Gb
C 0.5 Gb C 0.5 Gb
D 1.0 Gb 8 Gb trunk D 1.0 Gb
E 2.0 Gb E 2.0 Gb

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-76. Inter-switch link trunking

Although FSPF-compliant switches ensure fixed routing paths and guarantee that all frames are
delivered in order, congestion occurs if the aggregation of the data stream exceeds the capacity of
one of the ISLs in the path.
For example, four untrunked ISLs have a maximum capacity of 2 Gb each, which provides a
potential maximum throughput of 8 Gb. Yet due to load-sharing path assignments, the two 2-Gb
data streams could both be assigned to the first of the four ISLs. Thus, the aggregated throughput
of the four ISLs is: 2 Gb + 1.5 Gb +.5 Gb + 1 Gb, for a total of 5 Gb. The two 2-Gb data streams are
competing for the same congested path.
ISL Trunking enables four 2-Gb ISLs to function as one logical 8 Gb path, known as a trunking
group. The 8 Gb trunk provides more bandwidth than is needed for all the attached nodes. ISLs
within the same trunk group must operate at the same speed. The ISL trunking feature enhances
switch-to-switch performance while simplifying management and improving reliability.

© Copyright IBM Corp. 2001, 2017 4-80

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Enabling ports for trunking

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-77. Enabling ports for trunking

Trunking is enabled on a port or switch basis with either the Web Tools or using the CLI. The Port
Admin tab provides a method to enable trunking on a port using the Web Tools. Select the action
button to be able to enable or disable trunking for the selected port.

© Copyright IBM Corp. 2001, 2017 4-81

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Trunking: CLI output examples

3800_21:admin> fabricShow
Switch ID Worldwide Name Enet IP Addr FC IP Addr Name
-----------------------------------------------------------------------
9: fffc09 10:00:00:60:69:50:11:53 10.255.255.209 0.0.0.0 "3800_9"
21: fffc15 10:00:00:60:69:50:10:85 10.255.255.221 0.0.0.0 "3800_21"
109: fffc6d 10:00:00:60:69:20:54:e4 10.255.255.109 0.0.0.0 >"2400_109"
The Fabric has 3 switches

3800_21:admin> islShow
1: 4 -> 4 10:00:00:60:69:50:11:53 3800_9 sp: 2G bw: 4G TRUNK
2: 9 -> 1 10:00:00:60:69:20:54:e4 2400_109 sp: 1G bw: 1G

3800_21:admin> trunkshow
1: 4 -> 4 10:00:00:60:69:50:11:53 deskew 15 MASTER
5 -> 5 10:00:00:60:69:50:11:53 deskew 16

3800_21:admin> topologyShow
3 domains in the fabric; Local Domain ID: 21

Domain Metric Hops Out Port In Ports Flags Bandwidth Name

-----------------------------------------------------------------------
9 500 1 4 0x00000600 D 4 (Gbs) "3800_9"
109 1000 1 9 0x00000410 D 1 (Gbs) "2400_109"

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-78. Trunking: CLI output examples

Observe the outputs of the various CLI commands.

© Copyright IBM Corp. 2001, 2017 4-82

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Identify route using pathinfo command

DAZZ16A:admin> pathinfo 10,7,9 -r

Host
7
Target port is Embedded 6
Hop In Port Domain ID (Name) Out Port BW Cost DAZZ16A
----------------------------------------------------------
0 E 6 (DAZZ16A) 8 8G 500 8
1 64 1 (Meteor) 80 4G 500
2 4 10 (DAZZ8B) E - -
Reverse path
3 E 10 (DAZZ8B) 4 4G 500 64
4 80 1 (Meteor) 64 8G 500
5 8 6 (DAZZ16A) E - - 1
Meteor
80

DAZZ16A:admin> islshow
1: 8-> 64 10:00:00:60:69:e2:01:38 Meteor sp: 2G bw: 8G TRUNK
DAZZ16A:admin> trunkshow
1: 8 -> 64 10:00:00:60:69:e2:01:38 deskew 15 MASTER 4
9 -> 65 10:00:00:60:69:e2:01:38 deskew 16
10 -> 66 10:00:00:60:69:e2:01:38 deskew 15 10
11 -> 67 10:00:00:60:69:e2:01:38 deskew 16 DAZZ8B
9
Device
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-79. Identify route using pathinfo command

The pathinfo command can be used to display detailed routing information from a source port on
the local switch to a destination port on another switch. The exact path, including intermediate
switches are displayed. The -r operand causes the reverse path to be displayed as well. Statistics
such as bytes per second and frames per second can also be displayed.
The example shows the route from source domain 6 port 7 to destination domain 10 port 9 and the
reverse path.
The islshow command issued from switch DAZZ16A displays that there is only one ISL on this
switch.
The trunkshow command displays that there are actually four ISLs forming the ISL trunk.

© Copyright IBM Corp. 2001, 2017 4-83

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Domain 1 islShow and trunkShow

Meteor:admin> islshow Host

1: 1-> 3 10:00:00:60:69:50:04:a7 IOTL120 sp: 2G bw: 4G TRUNK 7
2: 16-> 2 10:00:00:60:69:50:03:fd IOTL118 sp: 2G bw: 4G TRUNK
3: 32-> 4 10:00:00:05:1e:34:11:ad DAZZ16C sp: 2G bw: 2G TRUNK 6
4: 33-> 4 10:00:00:05:1e:34:10:5a DAZZ8A sp: 2G bw: 2G TRUNK
5: 34-> 1 10:00:00:60:69:50:03:e5 IOTL119 sp: 2G bw: 2G TRUNK DAZZ16A
6: 48-> 8 10:00:00:05:1e:34:11:9a DAZZ16B sp: 2G bw: 2G
7: 49-> 9 10:00:00:05:1e:34:11:9a DAZZ16B sp: 2G bw: 2G 8
8: 50-> 6 10:00:00:60:69:c0:06:d5 Mojo1 sp: 2G bw: 2G TRUNK
9: 64-> 8 10:00:00:05:1e:34:11:8e DAZZ16A sp: 2G bw: 8G TRUNK
10: 80-> 4 10:00:00:05:1e:34:10:5d DAZZ8B sp: 2G bw: 4G TRUNK
11: 97-> 14 10:00:00:60:69:10:6f:4a IOTL104 sp: 1G bw: 1G
12: 98-> 16 10:00:00:60:69:90:03:60 IOTL128 sp: 1G bw: 1G
13: 99-> 17 10:00:00:60:69:90:03:60 IOTL128 sp: 2G bw: 2G TRUNK 64
14: 109-> 5 10:00:00:05:1e:34:55:78
15: 111-> 13 10:00:00:05:1e:34:55:6c
pulsar3
Pulsar4
sp:
sp:
2G
2G
bw:
bw:
2G
2G
TRUNK
TRUNK
1
16: 112-> 0 10:00:00:60:69:80:43:b6 SecFab1 sp: 2G bw: 8G TRUNK Meteor
80
Meteor:admin> trunkshow

7:64 -> 8 10:00:00:05:1e:34:11:8e deskew 16 MASTER

67 -> 11 10:00:00:05:1e:34:11:8e deskew 15
4
66 -> 10 10:00:00:05:1e:34:11:8e deskew 16
65 -> 9 10:00:00:05:1e:34:11:8e deskew 17 10
DAZZ8B
8:80 -> 4 10:00:00:05:1e:34:10:5d deskew 16 MASTER
81 -> 5 10:00:00:05:1e:34:10:5d deskew 15 9
Device
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-80. Domain 1 islShow and trunkShow

The trunkshow output data for trunking group 7 from switch Meteor should be consistent with the
trunkshow output from switch DAZZ16A.
Trunking group 8 shows the two ISLs forming the trunk between switches Meteor and DAZZ8B.

© Copyright IBM Corp. 2001, 2017 4-84

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Identify route GUI: Host to device (1 of 2)

Domain 6 to Domain 1 Host

7
6
DAZZ16A
8

64
1
Meteor
?

?
10
DAZZ8B
9
Device
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-81. Identify route GUI: Host to device (1 of 2)

Detailed FSPF routing data for input ports attached to a given switch domain can be found using
the GUI in the Routing tab of the Administration notebook.
For example, if a host attached to domain 6 port 7 needs to communicate with a device in domain
10, its FSPF assigned route has two hops with a cost metric of 1000.
The first hop for port 7 is to exit domain 6 using output port 8 to arrive at domain 1 port 64.
Upon arrival at domain 1, the routing is managed by the FSPF routing mechanism on that switch.
Thus, to determine the next hop, the FSPF routing data from domain 1 needs to be examined.

© Copyright IBM Corp. 2001, 2017 4-85

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Identify route GUI: Host to device (2 of 2)

Domain 1 to Domain 10 Host

7
6
DAZZ16A
Slot 7 Port 0 = port 64 8

64
1
Meteor
80

4
10
DAZZ8B
9
Device
IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-82. Identify route GUI: Host to device (2 of 2)

The FSPF assigned router for the host attached to domain 6 port 7 to communicate with a device
attached to domain 10 has two hops. After the first hop, the traffic has arrived at domain 1 using
port 64.
Port 64 is actually slot 7 port 0 on the switch. For slot 7 port 0 to get to domain 10, its FSPF
assigned route has one hop with a cost metric of 500 (which means the cost metric from domain 6
to domain 1 was also 500).
Slot 7 port 0 (port 64) will exit domain 1 using output port 80 to hop to domain 10 port 4. Upon
arrival at domain 10, frames destined for port 9 of that domain are routed to that port by the domain
10 switch.

© Copyright IBM Corp. 2001, 2017 4-86

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-83. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-87

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Fibre Channel switch zoning

• Fibre Channel SAN zoning
ƒ Create SAN zones
í Zones can overlap
ƒ Control data access
í Nodes with no zones in common are invisible to each other

• Zoning management
ƒ Administrator sets policy
í Dynamic configuration
í Fabric enforces zoning Windows
Zone 1

Sun Zone
2

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-84. Fibre Channel switch zoning

The IBM System Storage b-type SAN switches provide fabric enforced zoning. Zones can be hard
(hardware enforced) or soft (name server advisory). In a hard zone, members are specified by
physical domain ID and port number. In a soft zone, at least one member is specified logically by a
WWN.
For hard zones, the switch hardware ensures that there is no data transfer beyond members of the
defined zone. Hard zoning provides the greatest security possible.

© Copyright IBM Corp. 2001, 2017 4-88

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Zoning concepts and terminology

Zone 1 Zone
Zone 1 Zone
Zone 1
Zone 1 Zone
Zone 1 Configuration

• Multiple members per • Only one active at a time

zone configuration
• Zone members include:
ƒ WWNN
ƒ WWPN
ƒ Alias
ƒ Domain ID and Port address

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-85. Zoning concepts and terminology

A zone is a group of devices attached to the fabric. It consists of members that can be specified by:
• Node worldwide name of the device
• Port worldwide name of the port within a device
• Domain ID and port number where the device is attached
• Alias (a name assigned to a device or group of devices for ease of reference).
One or more zones make up a zone configuration. The zone configuration reflects the zoning policy
to be implemented for the fabric. Devices can belong to one or more zones. Zones can belong to
one or more zone configurations. Multiple zone configurations can be defined.
At a given point in time, only one zone configuration is enabled. This enabled configuration is
referred to as the effective configuration. The cfgSave command is used to save the defined
configuration and the enabled configuration in non-volatile memory.

© Copyright IBM Corp. 2001, 2017 4-89

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Zone example configuration

Server 5

SAN Fabric

Storage Storage Storage

Device Device 3 Device

Zone A Zone B

Zone Configuration1

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-86. Zone example configuration

The example shows a zone configuration that contains two zones without overlapping zone
members. When this zone configuration is activated, nodes in Zone A are invisible to nodes in Zone
B, and vice versa.
There are two devices that are not zoned. Their operation depends on the switch default zoning
settings which can either be open or closed. It is recommended to set the default zone setting to no
access.
Assuming a default zone that is set for no access, server 5 is not included in either zone, and is not
available to any device (zoned or not). Similarly, storage device 3 which is not part of either Zone A
or Zone B is also not capable of communicating with other devices. In other words Server 5 and
storage device 3 will not be able to communicate with one another.

© Copyright IBM Corp. 2001, 2017 4-90

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Zoning implementation steps in a nutshell

Identify resource access requirements

Step 1: among server and storage ports

Step 2: Planning Step 3: Implementation

a. Determine zone a. Create Alias

configuration Relate to ports

b. Define zones b. Create zone

Add members
c. Identify members
of each zone c. Create zone configuration
Add zones

d. Enable zone configuration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-87. Zoning implementation steps in a nutshell

Planning for the zoning environment is a top-down process.

1. Identify the logical relationships that exist among the servers and storage ports of the fabric.
2. Translate the logical relationships of fabric devices into a zone configuration and identify
members for each zone.
Implementing the zoning environment is a bottoms-up process.
1. Create the alias definitions (optional).
2. Create the zones to define zone members (which can be specified with aliases).
3. Create the zone configurations to contain zones.
4. Activate the zone configuration.
The default zoning on Fabric OS 7.0+ is set to ‘noaccess’. The CLI command defzone activates or
deactivates the default zone.

© Copyright IBM Corp. 2001, 2017 4-91

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Zoning administration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-88. Zoning administration

The Zone Admin option in the Monitor pull-down menu in Web Tools is used to access zone
configuration panels. A user with administer privilege level is required. The Zone admin interface
consists of the following functions:
• Zone alias settings
• Zone settings
• Zone configuration settings
Zoning configuration can also be done using the CLI. An operator with user privilege level might
view zoning data using the CLI, but might not perform any updates.
Zone members can be specified by WWNs or Domain ID and port number. When zoning is defined
using Web Tools, the presentation of zone member selection can be viewed in two ways:
• Fabric view: Displays the physical hierarchy of the current switch (slots and ports) and devices
within the fabric.
• Devices only view: Displays a flat list of attached and imported physical devices by WWN.

© Copyright IBM Corp. 2001, 2017 4-92

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Create new alias

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-89. Create new alias

The Alias tab is used to create, delete, and rename aliases using the buttons across the top row.
Another method is to select the New option and select New Alias.
The example shows an alias named AliasTest that is consisted of a member representing port 16
on domain 1.
Add a port to the alias by selecting a port in the Member Selection List (left window) and clicking the
arrow that points to the Alias Members (right window). Similarly, a WWN is added to the alias by
selecting that WWN and clicking the arrow.
To remove a member from an alias, highlight the member entry in the right window and click the
arrow that points towards the left window. The Delete button in the top row will delete the alias that
is in the Name window.

© Copyright IBM Corp. 2001, 2017 4-93

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Create new zone

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-90. Create new zone

The Zone tab is used to create, delete, and rename zones, and to add or delete members of zones.
Another method is to select the New option and select New Zone.
The technique to add members to a zone or update a zone is similar to creating or updating an
alias.
The ports, WWNs, and aliases are listed in the Member Selection List so that they can be
highlighted and added to the Zone Members window.

© Copyright IBM Corp. 2001, 2017 4-94

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Search for member in selection list

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-91. Search for member in selection list

Use the Search Member option from the Edit pull-down menu to search for a member in the
Member Selection List window. If found, the member is highlighted in the selection list.
The Add WWN, Delete WWN, and Replace WWN menu options allow a given WWN to be added,
deleted, or replaced in one or more zones in the zoning database.

© Copyright IBM Corp. 2001, 2017 4-95

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Create a zone configuration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-92. Create a zone configuration

The Zone Config tab is used to create zone configurations, place zones into zone configurations,
and rename or delete zone configurations. Another method is to select the New option and select
New Zone Config.
After a zone configuration name is created, zones can be selected from the Member Selection List
and added to the zone configuration.
When a zone configuration is created or changed, it is not in effect until the zone configuration is
enabled.

© Copyright IBM Corp. 2001, 2017 4-96

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Zone configuration action options

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-93. Zone configuration action options

The Zoning Actions pull-down menu provides several options. The Choose Zone Config to be
enabled option is used to activate a previously created zone configuration. A dialog box appears to
allow selection of the desired configuration to activate. These zoning definitions are also saved in
the zoning database. The Disable Zoning option disables the zoning configuration that is currently
activated. A dialog box provides a warning before the configuration is disabled. The zoning
definitions are also saved in the zoning database. The Save Config option is used to save the
zoning definitions without activating any zoning policy. The Clear All option is used to delete all
zoning definitions. The cleared configuration is saved. Any enabled configuration will be disabled.
The Set Default Mode option sets the default zone to either No Access or All Access.

© Copyright IBM Corp. 2001, 2017 4-97

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

View effective configuration

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-94. View effective configuration

The zone configuration in effect identified in the lower right corner. The Print Effective Zone
Configuration option on the Print pull-down menu provides a display of the content of the effective
zone configuration. There is a Print button on the pop-up window in order to send the information to
a printer. An alternative to display the effective zone configuration is to click the Printer icon in the
lower right corner of the Zone Administration screen.

© Copyright IBM Corp. 2001, 2017 4-98

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Zoning: CLI command examples

Switch115:sanadmin aliCreate "DISK_NT", "1,1"

Switch115:sanadmin aliCreate "DISK_AIX", "1,9; 1,10"
Switch115:sanadmin aliCreate "NT1", "1,3"
Switch115:sanadmin aliCreate "NT2", "1,4"
Switch115:sanadmin aliCreate "AIX1", "1,11; 1,12"
Switch115:sanadmin aliCreate "AIX2", "1,13"
Switch115:sanadmin aliCreate "TAPE", "1,8"
Switch115:sanadmin zoneCreate "Zone_AIX", "DISK_AIX; AIX1; AIX2"
Switch115:sanadmin zoneCreate "Zone_AIX_TAPE", "DISK_AIX; AIX1; AIX2; TAPE"
Switch115:sanadmin zoneCreate "Zone_NT", "DISK_NT; NT1; NT2"
Switch115:sanadmin zoneCreate "Zone_NT_TAPE", "DISK_NT; NT1; NT2; TAPE"
Switch115:sanadmin cfgCreate "BKP_AIX_cfg", "Zone_AIX_TAPE; Zone_NT"
Switch115:sanadmin cfgCreate "BKP_NT_cfg", "Zone_NT_TAPE; Zone_AIX"
Switch115:sanadmin cfgShow "BKP_AIX_cfg"
cfg: BKP_AIX_cfg Zone_AIX_TAPE; Zone_NT

Switch115:sanadmin zoneShow "Zone_AIX_TAPE"

zone: Zone_AIX_TAPE
DISK_AIX; AIX1; AIX2; TAPE

Switch115:sanadmin aliShow "TAPE"

alias: TAPE 1,8

Switch115:sanadmin cfgEnable "BKP_AIX_cfg"

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-95. Zoning: CLI command examples

The following commands are available to define and manage the zoning environment using the
line-mode command interface:
• cfgCreate: Creates a new zone configuration
• cfgAdd: Add a zone to a configuration
• cfgDelete: Deletes a zone configuration
• cfgRemove: Removes a zone from a configuration
• cfgShow: Print the specified zone definition, otherwise all zone configurations are printed
• zoneAdd: Adds a member to a zone
• zoneCreate: Creates a zone
• zoneDelete: Deletes a zone
• zoneRemove: Removes a member from a zone
• zoneShow: Shows zone definition

© Copyright IBM Corp. 2001, 2017 4-99

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Adding a new switch to the fabric

Alias data
Zone data
Zone Cfg data
Alias data
Zone data
Principal Switch Zone Cfg data

Alias data
Zone data
Zone Cfg data

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-96. Adding a new switch to the fabric

When a brand new switch from the factory is connected to a fabric, all zone configuration data is
automatically copied from the fabric to the new switch by one of its neighboring switches. If a zone
configuration is enabled in the fabric, the same configuration is enabled on the new switch.
After this operation, a cfgShow command will display the same zoning output from the new switch
as it does for the rest of the fabric. Likewise, for the zoning data viewed using the Web Tools.

© Copyright IBM Corp. 2001, 2017 4-100

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Merging fabrics

Alias data 1 Alias data 2

Zone data 1 Zone data 2
Zone Cfg data 1 Zone Cfg data 2

Principal Switch

Alias data 2
Zone data 2
Zone Cfg data 2
Alias data 1
Zone data 1
Zone Cfg data 1

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-97. Merging fabrics

When two fabrics are being merged and both fabrics have identical zone configuration data and the
same configuration is enabled, then the fabrics are joined with the same active zone configuration.
If the fabrics have different zone configurations, the two sets of information will be merged, if
possible. Otherwise, the ISL is disabled to prevent the merge and prevent data corruption. Merging
is not possible in the following situations:
• Zoning is enabled for both fabrics, and the enabled configurations are different (configuration
mismatch).
• The name of a zone object in one fabric is used for a different type of zone object in the other
fabric (type mismatch).
• The definition of a zone object in one fabric is different from its definition in the other fabric
(content mismatch).

© Copyright IBM Corp. 2001, 2017 4-101

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Domain overlap
• Added switch has the same domain ID as another in the fabric
• Reconfiguring switch is necessary.
• Enter the following CLI commands
ƒ switchdisable
ƒ configdefault
ƒ cfgclear
ƒ switchenable
ƒ reboot

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-98. Domain overlap

When a switch with its own zoning configuration is being merged into an existing fabric and the
switch domain ID overlaps other switches in the fabric a segmented fabric results.
Two commands are available to reset and clear the configuration data of the switch.
• configDefault is used to reset switch configuration parameters back to the factory defaults.
• cfgClear is used to clear or erase zoning configuration data.
After the zoning configuration is cleared, the switch will merge into the existing fabric. A new
domain ID will automatically be assigned by the fabric to the switch. Or if desired, the configure
command can be used to manually assign a domain ID.

© Copyright IBM Corp. 2001, 2017 4-102

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Splitting fabrics

Alias data 1 Alias data 1

Zone data 1 Zone data 1
Zone Cfg data 1 Zone Cfg data 1

Principal Switch

Alias data 1 Alias data 1

Zone data 1 Zone data 1
Zone Cfg data 1 Zone Cfg data 1

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-99. Splitting fabrics

If an ISL failure causes a fabric to split into two separate fabrics, each new fabric retains the original
zone configuration.
When the ISL is replaced and no changes were made to the zone configuration in either of the new
fabrics, the two fabrics will merge back into one single fabric.
If changes have been made to either zone configuration, then the rules for merging two fabrics
apply.

© Copyright IBM Corp. 2001, 2017 4-103

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SAN extension switches over IP

• IBM System Storage SAN06B-R extension switch
• IBM System Storage SAN42B-R extension switch

FCIP
FC-FC Routing

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-100. SAN extension switches over IP

The IBM System Storage SAN06B-R extension switch and the IBM System Storage SAN42B-R
extension switch accelerates and optimizes replication, backup, and data migration over any
distance using Fibre Channel (FC) and Fibre Channel over IP (FCIP) networking technologies.
A wide range of System Storage midrange and enterprise storage area network (SAN)
infrastructure simplification and business continuity solutions can be created with an IBM System
Storage SAN extension switch. Infrastructure simplification solutions include disaster tolerance
over metropolitan and global IP networks with System Storage disk arrays, tape libraries, and IBM
Tivoli Storage Manager data protection software. Separate SAN islands can also be consolidated
using FC routing.

© Copyright IBM Corp. 2001, 2017 4-104

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM b-type switches and directors topics

• FC switches and directors introduction
• IBM b-type SAN switch portfolio
• Setup IBM B-type SAN switches
• Web Tools administration pages
• SAN switch functions
• Fabric Shortest Path First
• Inter-switch link trunks
• SAN zone configuration
• IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-101. IBM b-type switches and directors topics

© Copyright IBM Corp. 2001, 2017 4-105

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM Network Advisor

• Comprehensive management for data, storage, and converged
networks
ƒ High level overview of network
ƒ High level overview of network performance
ƒ Device status
ƒ Access to reports
ƒ Device configuration
ƒ System logs
• Licensed versions
ƒ IBM Network Advisor Professional
ƒ IBM Network Advisor Professional Plus
ƒ IBM Network Advisor Enterprise

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-102. IBM Network Advisor

IBM Network Advisor Software management tool provides comprehensive management for data,
storage, and converged networks. It includes an intuitive interface, and provides an in-depth view of
performance measures and historical data. It receives SNMP traps, syslog event messages,
customizable event alerts, and contains the Advanced Call Home feature to automatically collect
diagnostic information and send notifications to IBM Support for faster fault diagnosis and isolation.
This single application can deliver end-to-end visibility and insight across different network types. It
supports Fibre Channel SANs, including Gen 5 Fibre Channel platforms, IBM FICON, and IBM
b-type SAN FCoE networks. This tool also supports comprehensive lifecycle management
capabilities across different networks through a simple, seamless user experience.
Network Advisor Professional Edition supports up to 2 SAN fabrics, up to 1000 SAN devices, and
up to 15 SAN switches and access gateways.
Network Advisor Professional Plus supports up to 36 SAN fabrics, up to 5000 devices, and up to 40
SAN switches and access gateways.
Network Advisor Enterprise edition supports up to 100 SAN fabrics, up to 40,000 devices, and up to
400 SAN switches and access gateways.

© Copyright IBM Corp. 2001, 2017 4-106

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM Network Advisor features

• Maximizes administrative productivity
ƒ Enables automation of repetitive tasks
• Key features
ƒ Performance monitoring and management
ƒ Zoning
ƒ Configuration management
ƒ Virtual Fabrics
ƒ Port groups
ƒ FCIP management support for extension platforms
ƒ Virtualization support
ƒ Simplified HBA and CNA management

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-103. IBM Network Advisor features

IT organizations are able to analyze performance trends over time to maximize asset utilization and
aid in proactive capacity planning. They can perform common tasks across groups of switches in a
single operation, including firmware downloads, fabric configuration setup, backup, or restoration of
critical configuration or performance data, collection of diagnostic data for troubleshooting. Key
features include:
• Performance monitoring and management
• Zoning
• LSAN zoning
• Configuration management
• Virtual Fabrics
• Port groups
• FCIP management support for extension platforms
• Virtualization support
• Simplified HBA and CNA management

© Copyright IBM Corp. 2001, 2017 4-107

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM Network Advisor benefits

• Increase performance and reliability
ƒ Offers proactive performance analysis and troubleshooting capabilities on
SAN
• Simplify operations
ƒ Provides centralized, end-to-end network management of SANs
• Simplify network management
ƒ Manages storage and data networks through a single application
• Exploit Fabric Vision technology integration
ƒ Provides greater flexibility, visibility, and insight across environments
• Reduce total cost of ownership (TCO)
ƒ Provides proactive performance analysis and troubleshooting capabilities

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-104. IBM Network Advisor benefits

IBM Network Advisor Software management tool provides comprehensive management for data,
storage, and converged networks. This single application can deliver end-to-end visibility and
insight across different network types. It supports Fibre Channel SANs, including Gen 5 Fibre
Channel platforms, IBM FICON, and IBM b-type SAN FCoE networks. This tool also supports
comprehensive lifecycle management capabilities across different networks through a simple,
seamless user experience.
IBM Network Advisor provides customizable dashboards to help monitor the network.

© Copyright IBM Corp. 2001, 2017 4-108

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IBM Network Advisor dashboard

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-105. IBM Network Advisor dashboard

The dashboards (which can be found in the Dashboard tab of the Performance Dashboard
window) provide a high-level overview of the network and the current state of the management
devices. You can use the dashboards to easily check the status of the devices in the network. The
dashboards also provide several features to help you quickly access reports, device configurations,
and system logs.
The Performance Dashboard was introduced with IBM Network Advisor V12.0.3. It provides a
high-level overview of performance in the network. You can use the dashboard to easily check the
performance of devices in the network. The Performance dashboard also provides several features
to help you quickly access performance metrics and reports.
Dashboard widget items are selectable. Just double or right-click the widget item, and additional
information and options are displayed within the dashboard.

© Copyright IBM Corp. 2001, 2017 4-109

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Widgets

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-106. Widgets

The Dashboard contains over 40 different widgets for various network device and port measures
across SAN, IP, and wireless networks. The widgets are customizable and users configure them.
Most widgets allow users to drill down on specific aspects of the widget to reveal additional
information. Users can leverage out-of-box, default dashboard configurations or create customized
views of specific widgets. The management application provides the following preconfigured status
widgets:
• Bottlenecked ports: Table view of the bottlenecked ports and the number of violations for each
bottlenecked port in the SAN
• Events: Bar chart view of events grouped by severity and range
• Host adapter inventory: Stacked bar chart view of host adapters grouped by selected category
• SAN Inventory: Stacked bar chart view of FC devices grouped by operational status and
selected category
• SAN Status: Pie chart view of FC devices categorized by operational status
• Status: List view of various status attributes
• VM Alarms: Table view of alarms received from vCenter products

© Copyright IBM Corp. 2001, 2017 4-110

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Discovery tab

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-107. Discovery tab

If an IP or SAN device is not discovered (No Products Present), a Discover link displays on the
Dashboard to provide access to the Discover Setup dialog box.
The discovery switch is a switch in the fabric that uses in-band communication to obtain fabric-wide
information about the name server, zoning, and fabric membership from all other switches. There
must be at least one discovery switch present in all fabrics. The presence of a discovery switch
provides significant help in improving the scalability of the application. The discovery switch is also
referred to as a seed switch. The seed switch must be running a supported FOS and must be HTTP
reachable.

© Copyright IBM Corp. 2001, 2017 4-111

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

SAN tab

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-108. SAN tab

The SAN tab displays a product list on the left, a connectivity map in the center, and a master log,
legend, and mini-map at the bottom. Status icons are overlaid on product icons. The worst-case
status is rolled up to closed objects on the product list to ensure that they are flagged.

© Copyright IBM Corp. 2001, 2017 4-112

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

IP tab

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-109. IP tab

The IP tab displays a product list on the left, a connectivity map in the center for the selected
topology (IP, L2, Ethernet, or VLAN), a master log, and mini-map at the bottom.

© Copyright IBM Corp. 2001, 2017 4-113

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Keywords
• IBM System Storage SAN b-type • Fabric Switch/Directors
• Inter-chassis links • Port functions
• Control processor blade • Name server
• Core routing blade • Zone alias
• Web Tools • Zone
• Fabric Manager • Zone Configuration
• Switch view • Network Advisor
• Fabric Watch • Dashboard
• Multi-switch fabric • Widgets
• Fabric Shortest Path First • SAN tab
• Trunking • IP tab

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-110. Keywords

© Copyright IBM Corp. 2001, 2017 4-114

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Review questions (1 of 4)
1. True or False: All IBM SAN b-type switches support Fibre
Channel link speeds up to 16 Gb.

2. Which Fibre Channel b-type SAN switch model supports

multiprotocol and distance extension?
a. SAN42B-4
b. SAN48B-5
c. SAN42B-R
d. SAN96B-5

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-111. Review questions (1 of 4)

Write your answers here.

© Copyright IBM Corp. 2001, 2017 4-115

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Review questions (2 of 4)
3. How many port blades can be accommodated by the IBM
System Storage SAN768-2?
a. 2
b. 4
c. 8
d. 12

4. Which SAN switch administration account is suitable for one

to view without configuration authority?
a. Admin
b. User
c. Root
d. factory

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-112. Review questions (2 of 4)

Write your answers here.

© Copyright IBM Corp. 2001, 2017 4-116

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Review questions (3 of 4)
5. True or False: The Web Tools switch administration is found
in the Switch View tab.

6. What is the initial port state in SAN switch port initialization?

a. G_Port
b. U_Port
c. E_Port
d. F_Port

7. True or False: Four inter-switch links each running at 4 Gb

between the same two SAN switches always functions as
one logical 16 Gb path.

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-113. Review questions (3 of 4)

Write your answers here.

© Copyright IBM Corp. 2001, 2017 4-117

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Review questions (4 of 4)
8. Which of the following enables path selection in a multi-
switch Fibre Channel environment?
a. ISL
b. ICL
c. FSPF
d. FCCT

9. True or False: IBM Network Advisor can support both IP and

SAN Networks.

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-114. Review questions (4 of 4)

Write your answers here.

© Copyright IBM Corp. 2001, 2017 4-118

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 4. IBM Fibre Channel b-type switches and directors

Uempty

Unit summary
• Classify IBM Storage System b-type switches as Enterprise, Midrange,
Entry, and Multiprotocol
• Differentiate the features of the IBM Storage System b-type switch
offerings
• Discuss setup considerations for the IBM Storage System b-type SAN
switches
• Employ Web Tools for basic SAN switch administration
• Describe SAN switch functions related to ports, name server, and multi-
switch fabrics
• Explain basic operations of Fabric Shortest Path First routing algorithm
• Recognize the benefits of an inter-switch link trunk environment
• Formulate the procedure to define a SAN zone configuration
• Discuss features of IBM Network Advisor

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Figure 4-115. Unit summary

© Copyright IBM Corp. 2001, 2017 4-119

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Unit 5. Cisco MDS Fibre Channel

switches and directors
Estimated time
02:30

Overview
This unit examines the functional characteristics associated with the IBM Storage Systems and
Cisco directors and switches. The features of selected directors are reviewed, steps involved to set
up are discussed, and SAN administration using Device manager and Data Center Network
Manager are introduced. Administration tasks are reviewed including the creation of a VSAN, an
ISL, a port channel, and a zone.

How you will check your progress

• Review questions

References
OL-6986 Cisco MDS 9500 Series Hardware Installation Guide
78-16946 Cisco MDS 9000 Family Quick Configuration Guide
OL-6973 Cisco MDS 9000 Family Configuration Guide
OL-6970 Cisco MDS 9000 Family Command Reference
OL-25174-01 Cisco DCNM Fundamentals Guide, Release 5.x
http://www.ibm.com/san IBM SAN home page
http://www.cisco.com Cisco home page

© Copyright IBM Corp. 2001, 2017 5-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Unit objectives
• Classify IBM offered Cisco MDS SAN switches and directors
• Differentiate the features of the IBM offered Cisco SAN switches and
directors
• Discuss setup considerations for the IBM offered Cisco SAN switches
• Employ Device Manager and Data Center Network Manager for basic
SAN switch administration
• Evaluate the use of a Virtual Storage Area Network
• Differentiate the use of an inter-switch link and a port channel
• Formulate the procedure to define a SAN zone configuration

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-1. Unit objectives

© Copyright IBM Corp. 2001, 2017 5-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS Fibre Channel switches and directors topics

• Cisco MDS family overview
• Initial setup configuration
• Device Manager and Data Center Network Manager
• Virtual Storage Area Network
• Inter-switch links and port channel
• Zoning

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-2. Cisco MDS Fibre Channel switches and directors topics

© Copyright IBM Corp. 2001, 2017 5-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS Fibre Channel switches and directors topics

• Cisco MDS family overview
• Initial setup configuration
• Device Manager and Data Center Network Manager
• Virtual Storage Area Network
• Inter-switch links and port channel
• Zoning

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-3. Cisco MDS Fibre Channel switches and directors topics

© Copyright IBM Corp. 2001, 2017 5-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS 9000 family

• MDS 9000 series of switches and directors
ƒ Based on Cisco SAN operating system (NX-OS / SAN-OS)
ƒ Managed by a comprehensive management platform
í Cisco Data Center Network Manager
ƒ Scalable architecture for all models of switch and director units
• Multiple layers of intelligent features
ƒ Ultra-high availability
ƒ Scalable architecture
ƒ Comprehensive security features
ƒ Ease of management
ƒ Advanced diagnostics and troubleshooting capabilities
ƒ Seamless integration of multiple technologies
ƒ Multi-protocol support

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-4. Cisco MDS 9000 family

Cisco positions its solution as a multilayer intelligent storage platform comprising:

• Integrated management: Multiple policy-based management options that integrate to existing
framework (like IP framework) implementations.
• Intelligent storage services: Network applications for volume management, data mobility, and
replication with open APIs.
• Intelligent network services: Intelligent infrastructure to enable scalable, secure, and stable
enterprise SAN consolidation to reduce cost.
• Multiprotocol: Flexible connectivity using FC, FCIP, FICON, and iSCSI to reduce cost and
integrate midrange storage into disaster recovery implementation.

© Copyright IBM Corp. 2001, 2017 5-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS 9000 Enterprise SAN environment

Cisco Data Center Network Manager

MDS 9513
MDS 9710

MDS 9506 MDS 9706

Cisco MDS Multilayer Directors

MDS 9396S

MDS 9148S MDS 9250i

Cisco MDS Multilayer Switches
Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-5. Cisco MDS 9000 Enterprise SAN environment

IBM and Cisco Systems have entered a reseller agreement that enables the Cisco MDS 9000
family of Fibre Channel solutions to join the IBM family of SAN connectivity offerings. The Multilayer
Data Center Switch (MDS) family include:
• Cisco MDS 9148S switch for IBM System Storage supports up to 48 x 16 Gb Fibre Channel
ports in a one rack unit (1U) switch.
• Cisco MDS 9396S switch for IBM System Storage supports up to 96 x 16 Gb Fibre Channel
ports in a two rack unit (2U) switch.
• Cisco MDS 9250i switch offers up to 40 x 16 Gb Fibre Channel ports, 2 x 1/10 Gigabit Ethernet
IP storage services ports, and 8 x 10 Gb Ethernet Fibre Channel over Ethernet (FCoE) ports in
a two rack unit (2U) switch.
Cisco Enterprise SAN Directors includes:
• Cisco MDS 9506 director supports up to 192 x 1/2/4/8Gb FC ports, up to 32 x 10 Gb FCoE
ports, and up to 64 x 1 Gb Ethernet ports in a 7 rack unit (7U) chassis. The Supervisor-2A is
installed in pairs. The Fibre Channel switching bandwidth capacity is 1.5 Tb.
• Cisco MDS 9513 director supports up to 528 x 1/2/4/8/10Gb Fibre Channel ports, up to 88 x 10
Gb FCoE ports, and up to 176 x 1 Gb Ethernet ports in a 14 rack unit (14U) chassis. The
Supervisor-2A is installed in pairs. The Fibre Channel switching bandwidth capacity is 8.4 Tb.

© Copyright IBM Corp. 2001, 2017 5-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
• Cisco MDS 9706 director supports up to 192 x 2/4/8/10/16Gb Fibre Channel ports, and up to
192 x 10 Gb FCoE in a 9 rack unit (9U) chassis. The Supervisor-1 is installed in pairs. The Fibre
Channel switching bandwidth capacity is 12 Tb.
• Cisco MDS 9710 director supports up to 384 x 2/4/8/10/16Gb Fibre Channel ports, and up to
384 x 10 Gb FCoE in a 14 rack unit (14U) chassis. The Supervisor-1 is installed in pairs. The
Fibre Channel switching bandwidth capacity is 24 Tb.
Centralized management of the fabric environment is provided by the Cisco Data Center Network
Manager which can manage Cisco MDS and Nexus platforms.

© Copyright IBM Corp. 2001, 2017 5-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS 9700 series modules

Supervisor-1 module

48-Port 16 Gb FC 48-Port 10 Gb FCoE

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-6. Cisco MDS 9700 series modules

The Multilayer Data Center Switch (MDS) 9700 series of modules include:
MDS 9700 Series Supervisor-1 module: In redundant configurations, the Supervisor-1 module has
the ability to automatically restart failed processes. If a supervisor module is reset, complete
synchronization between the active and standby supervisor modules helps ensure stateful failover
with no disruption of traffic.
• MDS 9700 Series 48-port 16 Gb Fibre Channel switching module is hot swappable and
compatible with 2/4/8Gb, 4/8/16Gb, and 10 Gb Fibre Channel interfaces.
• MDS 9700 Series 48-port 10 Gb FCoE switching module provides 10 Gb line-rate,
non-blocking, predictable performance across all traffic conditions for every FCoE port in the
chassis.
Some products might not be available as IBM has the right to withdraw products from marketing as
needed.

© Copyright IBM Corp. 2001, 2017 5-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

MDS terminology
MDS 9509 has nine line cards

16-port FC

32-port FC

8-port GbE

Supervisor

Interface FC8/16 is
line card 8

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-7. MDS terminology

A switching module, storage services module, IP storage module, or supervisor module is also
referred to as a line card.
The Supervisor is integrated into the base of the Cisco MDS switches.
Of the slots available to house nine line cards in the Cisco MDS director, two of the slots are
reserved for each of two Supervisors. One Supervisor is active, to other is in standby mode.
The ports on the 32-port switching modules are grouped into eight groups of four ports each. These
ports are numbered from 1 to 16 across, then the next row, going across again from 17 to 32.
Each switch contains one internal bootflash that resides in the Supervisor module or the switching
module. The directors contain an optional external CompactFlash called slot0. If present, it can be
used to store software images, logs, and dumps.

© Copyright IBM Corp. 2001, 2017 5-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco-specific Fibre Channel port types

N_Port

N_Port
Switch/Director Switch/Director

F_Port
F_Port
E_Port
E_Port
F_Port Public Loop
N_Port
FL_Port
TE_Port
Port Channel
Storage System NL_Port

TE_Port

SD_Port Private Loop

TL_Port
Tape Library
FC Analyzer
Switch/Director

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-8. Cisco-specific Fibre Channel port types

There are several Cisco-specific port types.

Trunking E_Ports (TE_Port) receive and transmit Extended ISL frames. An EISL frame has an
EISL header that contains VSAN information. An E_Port becomes a TE_Port when the port is
configured with trunking mode on. By default, a TE_Port carries traffic from all VSANs.
Translative loop ports (TL_Port) support private loop devices. It can support communication from a
private loop device and a device that is attached to any switch on the fabric, a device in any public
loop on the fabric, a device on any private loop on the fabric, or a device on the same private loop.
Switch Port Analyzer(SPAN) Destination ports (SD_Port) are used to transmit frames as replicated
traffic from other ports on the switch. An SD_Port can be connected to an FC analyzer for
monitoring and diagnostic purposes. The SPAN tunnel port (ST_Port) complements the SD_Port
and supports RSPAN (Remote SPAN) which means the SD_port can be monitoring traffic of a port
on a remote switch (a different domain than the domain that contains the SD_Port).
If the port mode is defined as auto, then it can operate as an F_Port, FL_Port, E_Port, or TE_Port
by auto-sensing during port interface initialization. If the port mode is defined as Fx, it can operate
in either F_Port or FL_Port mode. Generally, the default port mode is auto.

© Copyright IBM Corp. 2001, 2017 5-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS Fibre Channel switches and directors topics

• Cisco MDS family overview
• Initial setup configuration
• Device Manager and Data Center Network Manager
• Virtual Storage Area Network
• Inter-switch links and port channel
• Zoning

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-9. Cisco MDS Fibre Channel switches and directors topics

© Copyright IBM Corp. 2001, 2017 5-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Management interfaces
Cisco
Data Center
Telnet SSH Network Manager
Fabric view
GUI Device view
CLI
Summary view

SNMPv1, 2, or 3

IP network
NTP
RADIUS
server
Serial connection CLI

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-10. Management interfaces

Management interfaces to the MDS switches include a CLI using Telnet/SSH or a serial connection,
and a GUI using the Cisco Data Center Network Manager tool. The Network Manager might be
installed from the installation CD or might be downloaded from the Cisco website.
Optionally, additional security and authentication schemes can be configured.
• SSH (Secure Shell) can be used to authenticate communication between the switch (host) and
a management station (client), and encrypt communication traffic.
• RADIUS (Remote Authentication Dial-In User Security) is a protocol that is used for the
exchange of credentials between a RADIUS server and a management station (client device).
The MDS switches support RADIUS AAA (authentication, authorization, and accounting) to
verify identity, grant access, and log client actions.
• SNMPv3 has built-in security for user authentication and data encryption.
To achieve time synchronization of system clocks, the MDS switch supports the network time
protocol (NTP). The switches in client mode are configured with one or more NTP servers. The
servers act as the time source and receive client synchronization requests.

© Copyright IBM Corp. 2001, 2017 5-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

CLI command hierarchy and structure

EXEC mode
Switch prompt

show clear config terminal copy debug

Configuration mode

role username interface vsan zoneset ...

Configuration submode
fc

channel-group switchport ...

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-11. CLI command hierarchy and structure

The command-line interface has two main command execution modes; EXEC mode and
Configuration mode. While EXEC mode commands primarily facilitate the display of system
information, the configuration mode enables tailoring of system functions. Additional
function-unique tailoring options are executed in the configuration sub-mode of the given function.
The commands available are dependent upon the command execution mode. Commands are
organized hierarchically with commands that perform similar functions that are grouped under the
same level.
A question mark (?) is used to obtain the list of available commands and/or parameters for a given
mode or sub-mode. The exit command is used to return to the previous level in the command
hierarchy. The end command is used to return directly back to the EXEC mode switch prompt.

© Copyright IBM Corp. 2001, 2017 5-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Basic system configuration: Switch setup

Preparing to configure the switch:

Run the setup utility to:

• Set admin password

• Set SNMP management userid/password
• Set switch name
• Set switch IP address
• Enable Telnet
• Control default port state
• Control default zone behavior
Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-12. Basic system configuration: Switch setup

When the switch is accessed for the first time (through the serial console port), a setup utility needs
to be executed to tailor the IP address and other configuration information necessary for the switch
to be fully functional.

© Copyright IBM Corp. 2001, 2017 5-14

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
Save the configuration so that the tailored options are reflected the next time that the switch is
booted. Observe that in the subsequent example the switch prompt changed to the defined name of
the switch (MDS9216_1B) after the configuration was saved.
switch# setup
---- Basic System Configuration Dialog ----
This setup utility will guide you through the basic configuration of
the system. Use ctrl-c to abort configuration dialog at any prompt.
Basic management setup configures only enough connectivity for
management of the system.
Would you like to enter the basic configuration dialog (yes/no): y
Enter the password for "admin": admin
Create another login account (yes/no) [n]: n
Configure SNMPv3 Management parameters (yes/no) [y]: y
SNMPv3 user name [admin]: admin
SNMPv3 user authentication password: admin123
The same password will be used for SNMPv3 privacy as well.
Configure read-only SNMP community string (yes/no) [n]: n
Configure read-write SNMP community string (yes/no) [n]: n
Enter the switch name: MDS9216_1B
Continue with Out-of-band (mgmt0) management configuration? (yes/no) [y]: y
Mgmt0 IP address: 10.10.1.29
Mgmt0 IP netmask: 255.255.0.0
Continue with In-band (vsan1) management configuration? (yes/no) [n]: n
Enable IP routing? (yes/no) [y]: y
Configure static route? (yes/no) [y]: n
Configure the default network? (yes/no) [y]: n
Configure the default gateway? (yes/no) [y]: y
IP address of the default gateway: 10.10.1.100
Configure the DNS IP address? (yes/no) [n]: n
Configure the default domain name? (yes/no) [n]: n
Enable the telnet service? (yes/no) [y]: y
Enable the ssh service? (yes/no) [n]: n
Configure the ntp server? (yes/no) [n]: n
Configure default switchport interface state (shut/noshut) [shut]:
Configure default switchport trunk mode (on/off/auto) [on]:
Configure default port-channel auto-create state (on/off) [off]:
Configure default zone policy (permit/deny) [deny]:
Enable full zone set distribution (yes/no) [n]:
The following configuration will be applied:
username admin password admin role network-admin
snmp-server user admin network-admin auth md5 admin123 priv admin123
switchname MDS9216_1B
interface mgmt0
IP address 10.10.1.29 255.255.0.0
no shutdown
IP default-gateway 10.10.1.100
telnet server enable
no ssh server enable

© Copyright IBM Corp. 2001, 2017 5-15

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
system default switchport shutdown
system default switchport trunk mode on
system default port-channel auto-create
no zone default-zone permit vsan 1-4093
no zoneset distribute full vsan 1-4093
Would you like to edit the configuration? (yes/no) [n]: n
Use this configuration and save it? (yes/no) [y]: y
[########################################] 100%
MDS9216_1B#

© Copyright IBM Corp. 2001, 2017 5-16

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Role-based access for CLI

Switch prompt

1 config terminal 2
Configure a role with Configure a user ID and associate
name and access rules. it with the named role.

role username

rule
rule role
rule
rule

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-13. Role-based access for CLI

By default, two roles are defined in all switches:

• Network operator (read-only): Can view the configuration but cannot make any configuration
changes.
• Network administrator (read/write): Can execute all commands and make configuration
changes. Can also create and customize up to 64 additional roles.
Role-based access can limit switch access to the granularity defined by the rules that are assigned
by the administrator. The switch software allows the operation to progress only if the user has
permission to access a given command or set of parameters within a given command. In addition, it
can be tailored to permit access to a specific VSAN or set of VSANs.

© Copyright IBM Corp. 2001, 2017 5-17

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
The following example defines a role that is called no-configuration and associates that role to a
user whose name is user1. User1 cannot issue the config t command and is only limited to
issuing three parameters that are associated with the show command.
MDS9216_1B# config t
MDS9216_1B(config)# role name no-configuration
MDS9216_1B(config-role)# description non-admin group
MDS9216_1B(config-role)# rule 1 permit clear
MDS9216_1B(config-role)# rule 2 permit debug
MDS9216_1B(config-role)# rule 3 permit exec
MDS9216_1B(config-role)# rule 4 permit show feature version
MDS9216_1B(config-role)# rule 5 permit show feature environment
MDS9216_1B(config-role)# rule 6 permit show feature interface
MDS9216_1B(config-role)# exit
MDS9216_1B(config)# end
MDS9216_1B# show role name no-configuration
Role: no-configuration
Description: non-admin group
-----------------------------------------
Rule Type Command-type Feature
-----------------------------------------
permit clear *
permit debug *
permit exec *
permit show version
permit show environment
permit show interface
MDS9216_1B# config t
Enter configuration commands, one per line. End with CNTL/Z.
MDS9216_1B(config)# username user1 role no-configuration
warning: password for user:user1 not set. S/he can not log in currently
MDS9216_1B(config)# username user1 password Cisco expire 2003-12-25
MDS9216_1B(config)# end
MDS9216_1B# show user-account
user:admin
this user account has no expiry date
roles:network-admin
user:user1
expires on Thu Dec 25 00:00:00 2003
roles:no-configuration
login: user1
Password:
MDS9216_1B# config t
^
% invalid cmd detected at '^' marker.
MDS9216_1B# show ?
environment System environment information
interface Interface status and configuration
version Show the software version

© Copyright IBM Corp. 2001, 2017 5-18

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
MDS9216_1B# show version
Cisco Storage Area Networking Operating System (SAN-OS) Software
TAC support: http://www.cisco.com/tac
Copyright (c) 2002 by Cisco Systems, Inc. All rights reserved.
The copyright for certain works contained herein are owned by
Andiamo Systems, Inc. and/or other third parties and are used and
distributed under license.
Software
BIOS: version 1.0.3
loader: version 1.0(2)
kickstart: version 1.0(2)
system: version 1.0(2)
BIOS compile time: 11/18/02
kickstart image file is: bootflash:/boot-1.0.2
kickstart compile time: 1/13/2003 20:00:00
system image file is: bootflash:/isan-1.0.2
system compile time: 1/13/2003 20:00:00
Hardware
RAM 963124 kB
bootflash: 503808 blocks (block size 512b)
slot0: 0 blocks (block size 512b)
MDS9216_1B uptime is 0 days 7 hours 33 minute(s) 18 second(s)
Last reset at 556461 usecs after Fri Jan 11 20:56:47 1980
Reason: Reset Requested by CLI command reload
System version: 1.0(2)
MDS9216_1B#

© Copyright IBM Corp. 2001, 2017 5-19

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Port interfaces and configure port examples

Switch prompt

config terminal

interface

fc Port-channel
Configure bundled ISLs

switchport channel-group
Configure port mode,
speed, trunk mode, and
so on.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-14. Port interfaces and configure port examples

By default, ports are in administrative shutdown mode and need to be defined and enabled for
frame transfer.
Configure a given port for mode (port type), speed (can be set to auto-sense), and if trunking
should be enabled.

© Copyright IBM Corp. 2001, 2017 5-20

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
One or more ISLs between neighboring switches can be aggregated or bundled to form a higher
bandwidth logical ISL. This logical ISL is called a port channel, not a trunk group. Up to 128-port
channel groups can be defined in a switch.
login: admin
Password:
MDS9216_1B# config t
Enter configuration commands, one per line. End with CNTL/Z.
MDS9216_1B(config)# interface fc1/1 - 2
MDS9216_1B(config-if)# shutdown
MDS9216_1B(config-if)# switchport speed auto
MDS9216_1B(config-if)# switchport mode f
MDS9216_1B(config-if)# no shutdown
MDS9216_1B(config-if)# end
MDS9216_1B# show interface fc1/1
fc1/1 is up
Hardware is Fibre Channel
Port WWN is 20:01:00:05:30:00:6a:1e
Admin port mode is F
Port mode is F, FCID is 0x7b0100
Port vsan is 1
Speed is 2 Gbps
Receive B2B Credit is 16
Receive data field size is 2112
Beacon is turned off
5 minutes input rate 8 bits/sec, 1 bytes/sec, 0 frames/sec
5 minutes output rate 8 bits/sec, 1 bytes/sec, 0 frames/sec
26 frames input, 1556 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
26 frames output, 1572 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
1 output OLS, 1 LRR, 0 NOS, 0 loop inits
MDS9216_1B# config t
Enter configuration commands, one per line. End with CNTL/Z.
MDS9216_1B(config)# interface fc1/3
MDS9216_1B(config-if)# shutdown
MDS9216_1B(config-if)# switchport speed auto
MDS9216_1B(config-if)# switchport mode auto
MDS9216_1B(config-if)# no shutdown
MDS9216_1B(config-if)# end
MDS9216_1B# show interface fc1/3
fc1/3 is up
Hardware is Fibre Channel
Port WWN is 20:03:00:05:30:00:6a:1e
Admin port mode is auto, trunk mode is on
Port mode is FL, FCID is 0x7b0200

© Copyright IBM Corp. 2001, 2017 5-21

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
Port vsan is 1
Speed is 1 Gbps
Receive B2B Credit is 16
Receive data field size is 2112
Beacon is turned off
5 minutes input rate 96 bits/sec, 12 bytes/sec, 0 frames/sec
5 minutes output rate 80 bits/sec, 10 bytes/sec, 0 frames/sec
30 frames input, 3680 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
30 frames output, 3320 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 1 loop inits
0 output OLS, 0 LRR, 0 NOS, 0 loop inits
MDS9216_1B# config t
Enter configuration commands, one per line. End with CNTL/Z.
MDS9216_1B(config)# interface fc1/5 - 8
MDS9216_1B(config-if)# shutdown
MDS9216_1B(config-if)# switchport speed auto
MDS9216_1B(config-if)# switchport mode e
MDS9216_1B(config-if)# switchport trunk mode on
MDS9216_1B(config-if)# switchport trunk allowed vsan 1
MDS9216_1B(config)# interface fc1/8
MDS9216_1B(config-if)# no shutdown
MDS9216_1B(config-if)# end
MDS9216_1B# show interface fc1/8
fc1/8 is down (Link failure or not-connected)
Hardware is Fibre Channel
Port WWN is 20:08:00:05:30:00:6a:1e
Admin port mode is E, trunk mode is on
Port vsan is 1
Receive data field size is 2112
Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
0 frames output, 0 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
0 output OLS, 0 LRR, 0 NOS, 0 loop inits
MDS9216_1B# show interface fc1/5
fc1/5 is down (Administratively down)
Hardware is Fibre Channel
Admin port mode is E, trunk mode is on

© Copyright IBM Corp. 2001, 2017 5-22

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty
Port vsan is 1
Receive data field size is 2112
Beacon is turned off
5 minutes input rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
5 minutes output rate 0 bits/sec, 0 bytes/sec, 0 frames/sec
0 frames input, 0 bytes
0 discards, 0 errors
0 CRC, 0 unknown class
0 too long, 0 too short
0 frames output, 0 bytes
0 discards, 0 errors
0 input OLS, 0 LRR, 0 NOS, 0 loop inits
0 output OLS, 0 LRR, 0 NOS, 0 loop inits

© Copyright IBM Corp. 2001, 2017 5-23

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS Fibre Channel switches and directors topics

• Cisco MDS family overview
• Initial setup configuration
• Device Manager and Data Center Network Manager
• Virtual Storage Area Network
• Inter-switch links and port channel
• Zoning

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-15. Cisco MDS Fibre Channel switches and directors topics

© Copyright IBM Corp. 2001, 2017 5-24

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco Data Center Network Manager

Cisco Data Center Network Manager: Views

DCNM-SAN Device

Server Manager

Physical Switch
topology image

Logical fabrics Switch and physical

(VSANs) fabric summary

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-16. Cisco Data Center Network Manager

Cisco Data Center Network Manager (DCNM) is a management system for the Cisco Unified
Fabric. It enables you to provision, monitor, and troubleshoot the data center network infrastructure.
It provides visibility and control of the unified data center so that you can optimize for the quality of
service (QoS) required to meet service-level agreements.
DCNM-SAN Server provides centralized MDS management services and performance monitoring.
SNMP operations are used to efficiently collect fabric information.
DCNM-SAN Client provides Fibre Channel troubleshooting tools, in addition to complete
configuration and status monitoring capabilities for Cisco MDS 9000 Family switches and Cisco
Nexus 5000 Series switches.
Device Manager provides a graphic representation of a Cisco MDS 9000 Family switch chassis or
Cisco Nexus 5000 Series switch chassis, including the installed switching modules, the supervisor
modules, the status of each port within each module, the power supplies, and the fan assemblies.
The tables in the DCNM-SAN Information pane basically correspond to the dialog boxes that
appear in Device Manager. However, while DCNM-SAN tables show values for one or more
switches, a Device Manager dialog box shows values for a single switch.

© Copyright IBM Corp. 2001, 2017 5-25

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Install Cisco Data Center Network Manager

• Cisco Data Center Network Manager installation prerequisites:
ƒ Supervisor module installed on each switch to be managed
ƒ Supervisor module configured using the setup routine or CLI:
í IP address to the mgmt0 interface
í SNMPv3 user name and password
Same password for all the switches in the fabric
• How to access the Cisco Data Center Network Manager:
ƒ Install the DCNM web client software
ƒ Proxy server configuration
ƒ Log in and launch the applications

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-17. Install Cisco Data Center Network Manager

Before you can install the Cisco Data Center Network Manager, a supervisor module must be
installed on each switch that you want to manage, and the supervisor module must be configured
with a management IP address and SNMPv3 agent configured with credentials for access. Use the
same SNMPv3 ID and password on all fabric switches that will be managed. The DCNM web client
is Java based.

© Copyright IBM Corp. 2001, 2017 5-26

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco DCNM login

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-18. Cisco DCNM login

Once the DCNM web client installation is completed, you can start it by using a browser pointed to
the system where the web client was installed. The DCNM web client is Java based and SNMP
based. In the login window, the IP address is of the switch that is to be managed, or it is of the Data
Center Network Manager SAN server that manages the switch.
The Control panel provides ways to discover new fabrics and select a fabric to manage.

© Copyright IBM Corp. 2001, 2017 5-27

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco DCNM
Information
pane

Environment
Summary

Tree pane

Tree pane

Fabric pane
Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-19. Cisco DCNM

In addition to the menu bar, the toolbar on top, and the message bar on the bottom, the DCNM SAN
main window has three panes:
• Tree pane: Controls the display of the Logical Domains (VSANs) or Physical Domains (physical
fabric that is associated with the seed switch).
• Information pane: Displays detail information of whatever option is selected in the tree pane.
• Fabric pane: Displays the topology of the fabric, including switches, hosts, and storage (end
devices). It also has tabs for displaying log and event data.

© Copyright IBM Corp. 2001, 2017 5-28

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco Device Manager launch

Switch front panel

Code level
18-port FC + 4 1-GbE

48-port FC

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-20. Cisco Device Manager launch

To launch or install Device Manager, you can use a web browser. Provide the IP address of the
switch that you want to manage. When Device Manager is selected, it either runs or is installed.
When starting Device Manager, you are prompted for authentication to log in.
Upon successful login, the Device Manager application is started and user is presented with a
graphical representation of the physical switch.
The Device Manager has two tabbed views:
• Device view: Displays the front panel of the logged in switch.
• Summary view: Displays the port statistics of the ISLs, Fx, and Nx ports.
The menu bar at the top of the window provides access to detailed configuration and statistical
data. For the given switch in the visual, slot 1 is built in and contains 18 FC ports and 2 GbE ports.
Slot 1 also contains the Supervisor. The switch firmware level is displayed with the Supervisor
module. Slot 2 is available to house any MDS 9000 line card module.
The Device Manager for a switch can also be opened from the DCNM fabric pane. The Device
Manager window shows a graphical representation of the switch while displaying the power, fan
trays, switch modules, and respective ports installed.

© Copyright IBM Corp. 2001, 2017 5-29

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco Device Manager summary tab

Switch summary

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-21. Cisco Device Manager summary tab

The Device Manager Summary tab provides a detailed display of the interfaces of the given
switch. Utilization data for the ports can be displayed in terms of utilization percentages, bytes
transfers, packets transferred, and average packet size.
For the Fibre Channel interfaces, each logged in device is identified by its 24-bit FCID as well as its
port WWN. As of Cisco SAN OS v2, FCID values are persistent by default. That is, if the cable for
the device is moved to another port interface on the same switch, the 24-bit FCID for the device
stays constant after it logs in to the switch from the new port interface.

© Copyright IBM Corp. 2001, 2017 5-30

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

DCNM switch physical view

Switches in the same physical Fabric

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-22. DCNM switch physical view

Select Switches in the tree pane to provides a list of switches that are in the same physical fabric as
the seed switch (mds9222i-2)

© Copyright IBM Corp. 2001, 2017 5-31

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS Fibre Channel switches and directors topics

• Cisco MDS family overview
• Initial setup configuration
• Device Manager and Data Center Network Manager
• Virtual Storage Area Network
• Inter-switch links and port channel
• Zoning

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-23. Cisco MDS Fibre Channel switches and directors topics

© Copyright IBM Corp. 2001, 2017 5-32

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

VSAN introduction
• A method to allocate ports within a physical fabric
ƒ In order to create a virtual fabric
• SAN islands are virtualized onto a common SAN infrastructure
ƒ Traffic isolation by controlling each incoming and outgoing port
• VSAN on FC similar to VLAN on Ethernet
ƒ Maximum 256 VSANs per switch
í Default VSAN ID = 1, Isolated VSAN ID = 4094
ƒ 239 switches per VSAN
í FC_ID can be reused across multiple VSANs
ƒ Each frame uniquely identified
í VSAN_ID header is applied at the ingress port
í SAN/Priority in header for quality of service (QoS)
• Fabric services and gathered statistics are per VSAN

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-24. VSAN introduction

Virtual Storage Area Network (VSAN) functionality leverages the advantages of isolated SAN
fabrics with capacities that address the limitations of isolated SAN islands.
VSAN provides a method for allocating ports within a physical fabric to create virtual fabrics.
Independent physical SAN islands are virtualized onto a common SAN infrastructure. An analogy is
that VSANs on Fibre Channel (FC) networks are like VLANs on Ethernet networks.

© Copyright IBM Corp. 2001, 2017 5-33

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco virtual SAN

Windows AIX Windows AIX Windows AIX

zOS/FICON zOS/FICON zOS/FICON

V7000 V7000 V7000

DS8000 DS8000 DS8000

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-25. Cisco virtual SAN

Cisco Virtual SANs (VSANs) allow independent logical fabrics to be defined from a set of one or
more physical switches. The traditional definition of a fabric is a set of one or more interconnected
FC switches. The definition of a VSAN is a set of ports that spans across one or more FC switches.
A given switch port is assigned to one and only one VSAN.
Each VSAN or logical fabric is completely isolated from the other VSANs. It functions as a separate
and independent fabric with its own set of fabric services (for example, name server, zoning
services, routing services).
Associated with each VSAN is a name and a VSAN number. The number range is from 1 to 4094.
VSAN 1 is the default VSAN and is the only one enabled from the factory. All ports are initially
defined in the default VSAN. Up to 256 VSANs can be defined with a number range from 2 to 4093.
VSAN 4094 is the isolated VSAN. When a VSAN is deleted, all the device ports associated with the
deleted VSAN are transferred to the isolated VSAN to avoid inadvertent usage of those ports.
VSANs can increase the effective utilization of physical switch resources. It alleviates the need to
maintain multiple physically isolated fabrics to meet differing organizational or application needs.

© Copyright IBM Corp. 2001, 2017 5-34

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

VSANs and trunking TE_Ports

• Each VSAN uses an independent set of fabric services
ƒ Zone server database, name server database, and so on
• Hardware traffic isolation
ƒ No special configuration required for end nodes
ƒ A VSAN_ID header is added at ingress port
í EISL trunk carries tagged traffic for multiple VSANs
ƒ The VSAN_ID header is removed at egress port

FC Frame VSAN A FC Frame FC Frame

TE_Ports zOS/FICON
DS8000

Windows
V7000 FC Frame VSAN B FC Frame FC Frame

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-26. VSANs and trunking TE_Ports

Based on FC standards, an E_Port exists at each end of an ISL to extend traffic from one switch to
another. While Cisco supports the standards-based E_Port implementation, it also supports
Extended ISLs (EISLs) and TE_Ports at the end points of ISLs to facilitate the implementation of
VSANs.
Cisco treats ISLs as global resources available to all VSANs. Thus, by default, all ISLs are eligible
to carry traffic from all VSANs. An extra EISL frame header is prefixed to the standard FC frame to
differentiate traffic from one VSAN versus another.
Trunking is used to describe an ISL that carries EISL frame headers containing VSAN information.
E_Ports that carry traffic for one or more VSANs (E_Ports that handle EISL headers) are referred to
as trunking E_Ports or TE_Ports.
The aggregation of multiple ISLs into a single logical entity is referred to as a port channel group in
Cisco terms.

© Copyright IBM Corp. 2001, 2017 5-35

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Create a VSAN

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-27. Create a VSAN

The Create VSAN icon in DCNM opens a window to create a VSAN. The Create VSAN window
specifies the VSAN ID name, LoadBalancing properties, InterOperValue properties, and whether
the VSAN should be active or suspended. To enforce static domain IDs, check the Static Domain
IDs box. The created VSAN appears in DCNM but is down since no ports have been assigned to
the VSAN.

© Copyright IBM Corp. 2001, 2017 5-36

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Assign ports to VSAN

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-28. Assign ports to VSAN

Select the FC interfaces for the newly created VSAN by highlighting the entries. The Port VSAN cell
is double-clicked in order to change the VSAN ID to the ID for the new VSAN. After responding to a
warning message, the ports are now visible in the new VSAN and the new VSAN is operational
since active ports are present in the VSAN.

© Copyright IBM Corp. 2001, 2017 5-37

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS Fibre Channel switches and directors topics

• Cisco MDS family overview
• Initial setup configuration
• Device Manager and Data Center Network Manager
• Virtual Storage Area Network
• Inter-switch links and port channel
• Zoning

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-29. Cisco MDS Fibre Channel switches and directors topics

© Copyright IBM Corp. 2001, 2017 5-38

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

DCNM ISL merge analysis

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-30. DCNM ISL merge analysis

An Inter-Switch Link (ISL) is created when connecting an E_Port (expansion port) of one switch to
an E_Port on another switch. When multiple ISLs are used, these can be aggregated to become a
single logical ISL which, in Cisco terminology, is called a port channel.
Prior to establishing an ISL between two switches, you launch the Merge Analysis zone tool to
verify that the existing VSANs can merge successfully across the fabric and to avoid segmentation.
The IP addresses (or FQDNs) are entered along with the VSANs to analyze. After successful
merge analysis, the two switches are now ready to establish ISLs between them. With multiple
VSANs transferring on each ISL, it would establish EISL links using TE ports.

© Copyright IBM Corp. 2001, 2017 5-39

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

ISL connection

Switch1 Switch 2
Mds9222i-1 Mds9222i-2

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-31. ISL connection

Two switches can be connected using two ISLs. Depending on the trunk setting for the port, it
becomes either an E_Port or a TE_Port. In your example, both ports are TE_Ports.
The same is verified using the Device Manager of both the switches. After connecting the two
switches, DCNM shows the added switch and the ISLs in the graphical representation of the fabric.

© Copyright IBM Corp. 2001, 2017 5-40

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Trunking and port channel

• FC trunking
– Used for VSANs
– Trunking E_Port (TE_Port)
– Transmits and receives frames for multiple VSANs over a single physical link
í Link is configured as an extended ISL (EISL)
– Available for Fibre Channel and Gb Ethernet interfaces
• FC port channel
– Aggregates up to 16 ISL / EISL links into a single logical link
– Increases availability of the aggregate bandwidth
í Traffic is distributed among all functional links in the port channel
– Provides high availability
í Channel remains active as long as one of the links remains active
í Traffic is transparently distributed over the remaining links
– Available for Fibre Channel and Gb Ethernet interfaces

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-32. Trunking and port channel

FC trunking (also known as VSAN trunking), in Cisco terms, enables interconnect ports to transmit
and receive frames for multiple VSANs over the same E_Port. The link is configured as an
extended ISL (EISL) link using the EISL frame format. Trunking is normally enabled for all ports in
the switch but can be disabled on a port-by-port basis. If the port becomes operational as a trunking
E_Port, it is referred to as a TE_Port. If a port with trunking enabled is connected to a third-party
switch, it works as a normal E_Port.
FC port channel, in Cisco terms, is used to aggregate up to 16 ISL or EISL links into a single logical
link. The Fibre Channel ports can be any Fibre Channel ports in any Fibre Channel switch or line
card. The port channel feature increases the available aggregate bandwidth of the logical link since
the traffic is distributed among all functional links in the channel. It also provides high availability
since the channel remains active as long as at least one of the links forming it remains active and
the traffic is transparently distributed over the remaining links.
Since port channel can be built on EISL links, both trunking and port channel are supported
simultaneously.
Other Fibre Channel switch manufacturers use the term trunking to describe the aggregation of
several physical interfaces into a single logical interface.

© Copyright IBM Corp. 2001, 2017 5-41

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define a port channel (1 of 2)

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-33. Define a port channel (1 of 2)

Selecting the Port Channel icon launches the FC Port Channel Wizard. The first panel identifies
the switch pair that are to be linked by the FC port channel. The second panel provides a selection
of the ISLs to include in the port channel. The third panel provides attributes for the port channel.
Attributes include:
• Channel ID number for both switches
• Port VSAN which is a logical assignment of the port channel to a particular VSAN. There are no
functional implications.
• VSAN List which identifies to which VSANs the ISLs belong.
• Trunk Mode to enable trunking for the links in the port channel. Select auto if unsure whether
links are E_Ports versus TE_Ports.
• Force Admin, Trunk, Speed, and VSAN attributes to be identical to ensures that the same
parameter settings are used in all physical ports in the channel. If these settings are not
identical, the ports cannot become part of the port channel.
• The aggregated speed of the port channel.

© Copyright IBM Corp. 2001, 2017 5-42

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define a port channel (2 of 2)

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-34. Define a port channel (2 of 2)

The FC Port Wizard will pop up a warning message that the process of converting ports into a port
channel might be disruptive. Once complete, the port channel is activated and can be verified on
the DCNM window. Observe that the port channel link speed displays 16 Gb, since there are two
8-Gb ports in use.

© Copyright IBM Corp. 2001, 2017 5-43

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Cisco MDS Fibre Channel switches and directors topics

• Cisco MDS family overview
• Initial setup configuration
• Device Manager and Data Center Network Manager
• Virtual Storage Area Network
• Inter-switch links and port channel
• Zoning

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-35. Cisco MDS Fibre Channel switches and directors topics

© Copyright IBM Corp. 2001, 2017 5-44

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Zoning
• Administered from any switch in the fabric
ƒ All changes are automatically distributed to all other switches
• Zone criteria
ƒ Worldwide port name (WWPN)
ƒ Fabric Port WWN (fWWN)
ƒ FCID
ƒ FC alias
ƒ Domain ID
ƒ IP address
ƒ Interface
ƒ Alias name
í Simplifies zone management

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-36. Zoning

The Cisco MDS 9000 family supports zoning based on the following criteria:
• Worldwide port name (WWPN): The WWN of the Nx_Port (device) attached to the switch.
• Fabric Port WWN (fWWN): The WWN of the fabric port (port-based zoning).
• FCID: The FCID of the N_Port attached to the switch.
• FC alias: The alias used.
• Domain ID: Where the domain ID is the domain ID of a switch.
• IP address: Where the IP address of the devices is entered as a 32 byte dotted decimal
optionally specifying a subnet mask that includes all addresses in the specified subnet.
• Interface: Switch interface zoning is similar to port zoning and can be defined as a zone
member on both a local and a remote switch. This type of zoning is for iSCSI initiators.
The Cisco MDS 9000 family supports a default zone. All ports and WWNs not assigned to any zone
belong to the default zone. If zoning is not activated, all devices belong to the default zone. You can
control access between default zone members by default zone policy. This is both a per-switch
(defined at setup) and a per-VSAN setting. The default is deny, but can be changed using the
config command zone default-zone permit.

© Copyright IBM Corp. 2001, 2017 5-45

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Zone configuration components

• Alias
ƒ Alias members can be FC ID, fabric port WWN, or WWPN.
• Zones
ƒ Contains either Alias or FC ID, WWPN, WWPN, Fabric Port, and so forth.
ƒ Specifies access control, confining the specified members in a zone.
• Zoneset
ƒ Groups zones
ƒ Enforces the access control defined by each zone when activated

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-37. Zone configuration components

When creating storage zones, it is recommended that you use aliases since this eases
administration and troubleshooting, especially when the SAN environment increases in size.
When working with zonesets, it is crucial to understand that while you can create multiple zonesets
(and zones can be members of multiple zonesets), only one zoneset can be active at any given
time (for each VSAN).

© Copyright IBM Corp. 2001, 2017 5-46

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define WWPN resource zone aliases

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-38. Define WWPN resource zone aliases

Using aliases for zoned resources is preferred because it makes identifying the resources easier.
To set an alias with DCNM for a WWPN in a selected VSAN can be done from the Edit Local Full
Zone Database window. Right-click the FC-Aliases entry in the tree to insert an alias. A Create
Alias window will pop up where a name can be associated to a WWPN in the VSAN.

© Copyright IBM Corp. 2001, 2017 5-47

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define other resource zone aliases (1 of 2)

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-39. Define other resource zone aliases (1 of 2)

To set an alias with DCNM for other resources types can be done from the Edit Local Full Zone
Database window. Right-click the FC-Aliases entry in the tree view to insert an alias. A Create Alias
window will pop up where an empty alias name can be defined. Once the empty alias is in the list,
highlight it and select the insert option which will open the Add Member window. The Add Member
window provides a way to set an alias for any zone resource type.

© Copyright IBM Corp. 2001, 2017 5-48

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define other resource zone alias (2 of 2)

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-40. Define other resource zone alias (2 of 2)

When an entry in the Zone By section of the Add Member window is selected, then the Select End
Devices window is displayed to select the resource to associate with the alias. If, for example, the
FCID entry was selected in the Add Member window, then the alias association will be the FCID of
the resource. Once the alias is associated, the alias can be verified by selecting the defined alias in
the tree view.

© Copyright IBM Corp. 2001, 2017 5-49

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define a zone (1 of 2)

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-41. Define a zone (1 of 2)

Defining a zone on DCNM can be done from the Edit Local Full Zone Database window. Right-click
the Zones entry in the tree view to insert a new zone definition. A Create Zone window will pop up
where the zone name can be defined. Once the zone name is in the list, highlight it and select the
insert option which will open the Add Member window. The Add Member window provides a way to
identify the resources for the zone.

© Copyright IBM Corp. 2001, 2017 5-50

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define a zone (2 of 2)

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-42. Define a zone (2 of 2)

When an entry in the Zone By section of the Add Member window is selected, then a selection
window is displayed to select the resources to include in the zone. If, for example, the FC-Aliases
entry was selected in the Add Member window, then the Select Aliases window is displayed for
adding to the zone. Once the resources are associated to the zone, the zone can be verified by
selecting the defined zone in the tree view.

© Copyright IBM Corp. 2001, 2017 5-51

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Define a zoneset

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-43. Define a zoneset

Defining a zoneset on DCNM can be done from the Edit Local Full Zone Database window.
Right-click the Zonesets entry in the tree view to insert a new zone definition. A window will pop up
where the zoneset name can be defined. Once the zoneset name is in the tree view list, highlight it
and select the insert option which will open the Select Zone window. Select the zones to add to the
zoneset.

© Copyright IBM Corp. 2001, 2017 5-52

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Activate a zoneset

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-44. Activate a zoneset

To activate a zoneset, right-click the specific zoneset entry in the tree view to select the Activate
option. A Save Configuration window will pop up to identify tasks to run after the zoneset is
activated. The options include to save the running configuration to the startup configuration, and
alternatively to a configuration file. The activation and post-activation tasks can be monitored in the
Zone Log window.

© Copyright IBM Corp. 2001, 2017 5-53

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Keywords
• Multilayer Data Center Switch • Cisco Data Center Network
(MDS) Manager (DCNM) SAN server
• Cisco MDS switch/director • DCNM web client
• Switching module • Virtual Storage Area Network
• Line card (VSAN)
• supervisor • Trunking
• Trunking E_Ports (TE_Port) • Port channel
• Translative loop ports (TL_Port) • Merge Analysis
• SPAN Destination ports • Default zone
(SD_Port) • Zone resource alias
• Device Manager • Zone
• Zoneset

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-45. Keywords

© Copyright IBM Corp. 2001, 2017 5-54

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Review questions (1 of 4)
1. True or False: All IBM offered MDS directors support 16 Gb
Fibre Channel

2. True or False: The Cisco DCNM client must be installed on

the user's local system in order to manage a SAN switch.

3. True or False: A redundant Supervisor module in a Cisco

MDS director is optional.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-46. Review questions (1 of 4)

© Copyright IBM Corp. 2001, 2017 5-55

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Review questions (2 of 4)
4. True or False: Cisco Device Manager supports a multi-
switch fabric.

5. What name identifies the Cisco feature that supports the

building of isolated SANs?
a. CSAN
b. VSAN
c. isoSAN
d. Cisco SAN

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-47. Review questions (2 of 4)

© Copyright IBM Corp. 2001, 2017 5-56

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Review questions (3 of 4)
6. What is the Cisco port type needed to support the
transmission of multiple isolated SANs?
a. ST_Port
b. E_Port
c. VE_Port
d. TE_Port

7. True or False: A port channel is the aggregation of multiple

ISLs into one logical link.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-48. Review questions (3 of 4)

© Copyright IBM Corp. 2001, 2017 5-57

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Review questions (4 of 4)
8. True or False: Multiple zonesets can be activated at the
same time on a one VSAN.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-49. Review questions (4 of 4)

© Copyright IBM Corp. 2001, 2017 5-58

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 5. Cisco MDS Fibre Channel switches and directors

Uempty

Unit summary
• Classify IBM offered Cisco MDS SAN switches and directors
• Differentiate the features of the IBM offered Cisco SAN switches and
directors
• Discuss setup considerations for the IBM offered Cisco SAN switches
• Employ Device Manager and Data Center Network Manager for basic
SAN switch administration
• Evaluate the use of a Virtual Storage Area Network
• Differentiate the use of an inter-switch link and a port channel
• Formulate the procedure to define a SAN zone configuration

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Figure 5-50. Unit summary

© Copyright IBM Corp. 2001, 2017 5-59

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Unit 6. SAN over Ethernet

Estimated time
02:00

Overview
There are two standards for block storage protocols over Ethernet networks. FCoE was ratified in
2009, while iSCSI was ratified in 2004. While iSCSI simply encapsulates the SCSI protocol in IP,
FCoE operates lower in the network stack and to do so required many enhancements to Ethernet.
This unit provides some introductory basics for Ethernet and TCP/IP, followed by a discussion on
iSCSI. It presents the enhanced Ethernet features and a discussion on Fibre Channel over Ethernet
(FCoE).

How you will check your progress

• Review questions
• Machine exercises

References
SG24-5470 Introduction to Storage Area Networks and System Networking
SG24-6240 IP Storage Networking: IBM NAS and iSCSI Solutions
http://www.ibm.com/san IBM SAN home page
http://www.fibrechannel.org Fibre Channel Industry Association (FCIA)
http://www.t11.org Technical Committee T11 home page

© Copyright IBM Corp. 2001, 2017 6-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Unit objectives
• Describe basic concepts of Ethernet
• Describe basic concepts of TCP/IP
• Describe key features of the iSCSI implementation
• Distinguish the features of the Converged Enhanced Ethernet major
components
• Describe key features of the FCoE implementation
• Evaluate the implementations of iSCSI and FCoE

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-1. Unit objectives

© Copyright IBM Corp. 2001, 2017 6-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

SAN over Ethernet topics

• Ethernet basics
• TCP/IP basics
• iSCSI
• Converged Enhanced Ethernet
• Fibre Channel over Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-2. SAN over Ethernet topics

© Copyright IBM Corp. 2001, 2017 6-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

SAN over Ethernet topics

• Ethernet basics
• TCP/IP basics
• iSCSI
• Converged Enhanced Ethernet
• Fibre Channel over Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-3. SAN over Ethernet topics

© Copyright IBM Corp. 2001, 2017 6-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Ethernet concepts
• Bottom two layers in OSI model
ƒ DataLink layer Application
í Link layer control
í Media access control
Presentation
ƒ Physical layer Session
• Topology options Transport
ƒ Bus, star-wired
ƒ Switched Ethernet Network
• Single broadcast domain DataLink Ethernet
ƒ Local area network Layers
Physical

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-4. Ethernet concepts

Ethernet protocols are defined in the bottom two layers of the OSI network model. The DataLink
layer provides node-to-node data transfer, supporting a link between two directly connected nodes.
The DataLink layer is divided into two sublayers. The Media Access Control (MAC) layer is
responsible for controlling how devices in a network gain access to data and permission to transmit
it. The Logical Link Control (LLC) layer is responsible for identifying Network layer protocols and
then encapsulating them and controls error checking and packet synchronization. The physical
layer defines the electrical and physical specifications of the data connection, the transmission
mode (that is, simplex, half duplex, full duplex), and the network topology.
The bus topology is the simplest (and the traditional) topology. In this one-cable LAN, all
workstations are connected in succession, as a “bus” arrangement, on a single cable. All
transmissions go to all the connected workstations. Each workstation then selects the
transmissions it should receive based on the address information contained in the transmission.
In a star topology, all attached workstations are wired directly to a central hub that establishes,
maintains, and breaks connections between them (in the event of an error). The advantage of a star
topology is that it's easy to isolate a problem node. The disadvantage is that if the hub fails, the
entire system is compromised.
Switched Ethernet relies on centralized multiport switches to provide a physical link between
multiple LAN segments. Inside each switch, high-speed circuitry supports wire-speed virtual

© Copyright IBM Corp. 2001, 2017 6-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty
connections between all the segments for maximum bandwidth allocation on demand. Adding new
segments to a switch increases the aggregate network speed as it reduces overall congestion, so
Switched Ethernet provides superior configuration flexibility.
A broadcast domain is a logical division of a computer network, in which all nodes can reach each
other by broadcast at the DataLink layer. This is also known as a local area network (LAN).

© Copyright IBM Corp. 2001, 2017 6-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Ethernet operation
• Data is encapsulated by Ethernet protocols
• All speeds use same header structure (10 Mb – 10 Gb)
ƒ 10 Gb must be full duplex mode
ƒ Jumbo frames
í Up to 9000 bytes versus 1500 bytes
• Virtual LAN (VLAN)

Ethernet headers VLAN ID Payload data FCS

Ethernet frame

VLAN tagged

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-5. Ethernet operation

An Ethernet frame is preceded by a preamble and start frame delimiter (SFD), which are both part
of the Ethernet packet at the physical layer. Each Ethernet frame starts with an Ethernet header,
which contains destination and source MAC addresses as its first two fields. The middle section of
the frame is payload data including any headers for other protocols (for example, Internet Protocol)
carried in the frame. The frame ends with a frame check sequence (FCS), which is a 32-bit cyclic
redundancy check used to detect any in-transit corruption of data.
An Ethernet frame can transmit a payload size up to the MTU of 1500 bytes. Some
implementations of Gigabit Ethernet and other higher-speed variants of Ethernet support larger
frames, known as jumbo frames that can range up to 9000 bytes. A 10 Gb Ethernet must operate
as full duplex.
A virtual LAN (VLAN) is any broadcast domain that is partitioned and isolated in a computer
network at the DataLink layer. One link can be shared between switches that support VLAN
operation rather than have multiple links for each LAN broadcast domain. VLANs can be used to
partition a local network into several distinctive segments. The VLAN ID is an optional field in the
Ethernet header.

© Copyright IBM Corp. 2001, 2017 6-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

SAN over Ethernet topics

• Ethernet basics
• TCP/IP basics
• iSCSI
• Converged Enhanced Ethernet
• Fibre Channel over Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-6. SAN over Ethernet topics

© Copyright IBM Corp. 2001, 2017 6-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

TCP/IP concepts
• Foundation for Internet communications
• Protocol stack designed in layers
ƒ A functional change to a layer should not cause change to other layers
• Protocols defined through Request for Comments
OSI TCP/IP
Application
Presentation Application
Application layer
Session
Transport TCP UDP Transport layer
Network IP Internetwork layer
DataLink Network layer
Ethernet
Physical

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-7. TCP/IP concepts

The Transmission Control Protocol/Internet Protocol (TCP/IP) protocol suite is the engine for the
Internet and networks worldwide. Its simplicity and power has led to its becoming the single
network protocol of choice in the world today. The main design goal of TCP/IP was to build an
interconnection of networks, referred to as an internetwork, or Internet, that provided universal
communication services over heterogeneous physical networks. Like most networking software,
TCP/IP is modeled in layers. By dividing the communication software into layers, the protocol stack
allows for division of labor, ease of implementation and code testing, and the ability to develop
alternative layer implementations.
The application layer is provided by the program that uses TCP/IP for communication. An
application is a user process cooperating with another process usually on a different host in a
client/server relationship. Applications in the storage world would include iSCSI and Fibre Channel
on IP traffic.
The Transport layer provides the end-to-end data transfer by delivering data from an application to
its remote peer. Multiple applications can be supported simultaneously. The most-used transport
layer protocol is the Transmission Control Protocol (TCP), which provides connection-oriented
reliable data delivery, duplicate data suppression, congestion control, and flow control.
The Internetwork layer, also called the Internet layer or the network layer, provides the “virtual
network” image of an Internet. This layer shields the higher levels from the physical network

© Copyright IBM Corp. 2001, 2017 6-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty
architecture below it. Internet Protocol (IP) is the most important protocol in this layer. It is a
connectionless protocol that does not assume reliability from lower layers. IP does not provide
reliability, flow control, or error recovery. These functions must be provided at a higher level. IP
provides a routing function that attempts to deliver transmitted messages to their destination.
The network interface layer, also called the link layer or the data-link layer, is the interface to the
actual network hardware. TCP/IP specifications do not describe or standardize any network-layer
protocols per se; they only standardize ways of accessing those protocols from the internetwork
layer.
The Internet Protocol suite is still evolving through the mechanism of Request for Comments
(RFC). Some RFCs are described as information documents, while others describe Internet
protocols. RFC documents are available publicly and online and best obtained from the IETF
website:
• http://www.ietf.org
TCP/IP operates on other networks, but this discussion will be limited to Ethernet only.

© Copyright IBM Corp. 2001, 2017 6-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

TCP protocol
• Interfaces between applications and logical network
ƒ Logical port numbers identify the applications
• Connection oriented process
ƒ Connection made between two TCP ports
Socket address = IP address + TCP port
ƒ Requires acknowledgment from receivers
í If not, then TCP resends the information
• Stream transfer, reliability, and flow control
ƒ Receiver application expects to receive a stream
í TCP segments the stream and IDs each segment
Sequence number identifies each segment
í TCP delivers to IP to transmit in a frame
Segments received are acknowledge

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-8. TCP protocol

Each process that wants to communicate with another process identifies itself to the TCP/IP
protocol suite by one or more ports. A port is a 16-bit number used by the host-to-host protocol to
identify to which higher-level protocol or application program (process) it must deliver incoming
messages. TCP provides error recovery, flow control, and reliability.
TCP is a connection-oriented protocol. Two application processes communicate through TCP
sockets. The socket model provides a process with a full-duplex byte stream connection to another
process. If two processes are communicating over TCP, they have a logical connection that is
uniquely identifiable by the two sockets involved, that is, by the combination [TCP, local IP address,
local port, remote IP address, remote port]. Server processes are able to manage multiple
conversations through a single port.
From the application's viewpoint, TCP transfers a contiguous stream of bytes through the network.
The application does not have to bother with chopping the data into basic blocks or datagrams.
TCP does this by grouping the bytes into TCP segments, which are passed to the IP layer for
transmission to the destination. TCP assigns a sequence number to each byte transmitted, and
expects a positive acknowledgment (ACK) from the receiving TCP layer. If the ACK is not received
within a timeout interval, the data is retransmitted.
The receiving TCP uses the sequence numbers to rearrange the segments when they arrive out of
order, and to eliminate duplicate segments. The receiving TCP, when sending an ACK back to the

© Copyright IBM Corp. 2001, 2017 6-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty
sender, also indicates to the sender the number of bytes it can receive (beyond the last received
TCP segment) without causing overrun and overflow in its internal buffers.

© Copyright IBM Corp. 2001, 2017 6-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

IP protocol
• Internetwork protocol layer
ƒ Logical network process
• IP does not recover transmission errors
ƒ IP is called an ‘unreliable’ transport
• Address and routing processing
ƒ IP address options
í IPv4 or IPv6
ƒ Routers interconnect networks to form Internet
í In-flight packets might get discarded by router due to congestion
• IP datagram is encapsulated by physical transport media
ƒ Such as Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-9. IP protocol

IP is the protocol that hides the underlying physical network by creating a virtual network view. It is
an unreliable, best-effort, and connectionless packet delivery protocol. Note that best-effort means
that the packets sent by IP might be lost, arrive out of order, or even be duplicated. IP assumes
higher layer protocols will address these anomalies.
An IP address is 32 bits (IPv4) or 128 bits (IPv6) in length and represents the logical address of a
TCP/IP host in the Internet. Hosts that are considered to be in the same logical subnet should exist
in the same physical Ethernet broadcast domain.
Routers interconnect networks at the internetwork layer level and route packets between them. The
routers and the hosts must understand the addressing structure associated with the networking
protocols it supports and take decisions on whether, or how, to forward packets. Hosts and routers
each make routing decisions as where to send the packet. There are two types of IP routing: direct
and indirect. However, routers, can deal with traffic congestion by discarding packets. Higher layer
protocols are assumed to recover lost packets.
The unit of data that includes the IP headers is called the IP datagram. The IP datagram is
encapsulated by the physical transport layer (such as Ethernet).

© Copyright IBM Corp. 2001, 2017 6-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

SAN over Ethernet topics

• Ethernet basics
• TCP/IP basics
• iSCSI
• Converged Enhanced Ethernet
• Fibre Channel over Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-10. SAN over Ethernet topics

© Copyright IBM Corp. 2001, 2017 6-14

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

iSCSI introduction
• Transports SCSI over TCP/IP networks
Ethernet
ƒ Connection oriented TCP
• Enables block-level access iSCSI
• Transaction occurs between iSCSI initiator Initiator

and iSCSI target

iSCSI
Target

Applications
iSCSI
SCSI Gateway

FCP iSCSI
Fibre Channel SAN

TCP SAN SVC

Target Virtualize
IP
FC Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-11. iSCSI introduction

The Internet Small Computer System Interface (iSCSI) protocol is the encapsulation of the industry
standard SCSI protocol within TCP/IP packets. The iSCSI protocol provides a block-level storage
capability similar to Fibre Channel (FC) storage area network (SAN) technology, which is
essentially the same system of encapsulating the SCSI protocol within an external carrier. The
difference is that the iSCSI SAN uses Ethernet instead of Fibre Channel transport technology.
With an iSCSI network, an existing traditional Ethernet networking infrastructure can be used,
thereby reducing the costs for specialized storage area networking devices, software, and licenses.
iSCSI enables the use of existing TCP/IP infrastructure and addresses distance limitations (iSCSI
can also be used over the Internet). Because an iSCSI request is an encapsulation of a SCSI
request, the SCSI concept of command descriptor blocks (CDBs) is applicable to iSCSI. This
means that the disk drives in the SAN are presented over the existing Ethernet network to server
applications as though the disks are local to the physical server hardware.

© Copyright IBM Corp. 2001, 2017 6-15

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

iSCSI frame encapsulation

• Fibre Channel frame versus iSCSI frame
Fibre
FCP SCSI Data
Channel

Ethernet IP TCP iSCSI SCSI Data

CDB

iSCSI PDU

TCP segment

IP datagram

Ethernet frame

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-12. iSCSI frame encapsulation

The iSCSI Protocol Data Unit (PDU) is located in the data area of the TCP segment. The PDU
consists of an iSCSI header, where the data length is specified and iSCSI data is encapsulated and
transported within the TCP/IP packet. A PDU is not restricted to one TCP segment and can span
over more than one.
Otherwise, it is also possible to have more than one iSCSI PDU in a single TCP segment data area.
Each TCP segment is encapsulated within an IP datagram. TCP/IP is responsible for reassembling
the TCP segment in the correct order on the target side and delivering it to the iSCSI layer in the
same order in which it was sent. After arriving at the iSCSI target or initiator, it is opened and SCSI
data is revealed for storage processing.
The iSCSI protocol works effectively over Internet Protocol networks without needing any change
the TCP/IP protocol. However, iSCSI incurs more overhead when compared to Fibre Channel
protocol.

© Copyright IBM Corp. 2001, 2017 6-16

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

iSCSI protocol layers

Applications
SCSI
• Command descriptor block
FCP iSCSI • Packet data unit

TCP • TCP segment

IP • IP datagram
FC Ethernet • Ethernet frame

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-13. iSCSI protocol layers

The iSCSI layer interfaces with the standard SCSI access mechanism and provides the same
functionality as the physical SCSI Bus. It uses Transmission Control Protocol (TCP) for reliable
data transmission over potentially unreliable networks. TCP provides iSCSI with orderly transport of
SCSI commands that are spread across multiple TCP segments and sent to the target.
An iSCSI communication is always started by the initiator. When an application sends data to a
Logical Unit (LU), it invokes a “write” on the initiator APIs which send the request to the SCSI Layer.
The SCSI drivers at the SCSI layer creates a Command Descriptor Block (CDB) for the write
request and passes it on to the iSCSI protocol layer. CDBs define the type of SCSI operation, the
logical block address to start at, the length of data that is involved, and other control parameters.
The iSCSI protocol layer places the CDB in an iSCSI PDU and passes it on to the TCP layer. A
PDU contains both CDB and information. The TCP layer divides the iSCSI PDU into segments,
adds a TCP header, and passes the segments to the IP layer for addressing and routing processes.
The IP layer processes for logical addressing and routing, adds a header, and then sends the
datagram to the physical layer, such as Ethernet, for physical transportation to the target. The
physical layer adds a header (and optionally, a trailer) and sends the frame to the target.
At the iSCSI target end, the physical layer strips off the headers, and passes the datagram for IP
processing. From there the segment is passed to TCP for processing. TCP then passes the iSCSI
PDU to the iSCSI layer. The iSCSI layer extracts the SCSI CDB from the iSCSI PDU and sends it

© Copyright IBM Corp. 2001, 2017 6-17

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty
along with the related parameters and data to the SCSI target device. The SCSI target device will
then send the SCSI “Write” request along with the data to the logical unit.

© Copyright IBM Corp. 2001, 2017 6-18

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

iSCSI components
• iSCSI name represents the iSCSI Host
Initiator
iSCSI Initiator
ƒ Not the IP address iqn.2011-11.com.acme:host01
• iSCSI name represents the
IP Address IP Address
Target 10.1.1.1 10.2.2.1
ƒ Not the IP address or TCP port
ƒ Registered port is TCP 3260
10.1.1 10.2.2
í Can be changed Network Network
ƒ Each Target represents
a unique set of LUNs
IP Address IP Address
10.1.1.9 iSCSI 10.2.2.9
TCP port 3260 TCP port 3260
Device

iSCSI Target iSCSI Target

iqn.2011-11.com.acme:ST04 iqn.2011-11.com.acme:ST05

0 n 0 n
LUN LUN

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-14. iSCSI components

Both targets and initiators require names for the purpose of identification. In addition, names enable
iSCSI storage resources to be managed regardless of location address. An iSCSI node name is
also the SCSI device name of an iSCSI device. The iSCSI name of a SCSI device is the principal
object used in authentication of targets to initiators and initiators to targets. This name is also used
to identify and manage iSCSI storage resources.
The registered port for the iSCSI target is TCP port 3260. This port can be changed during
configuration.
iSCSI names are associated with iSCSI nodes, and not iSCSI network adapter cards, to ensure
that the replacement of network adapter cards does not require reconfiguration of all SCSI and
iSCSI resource allocation information.
The iSCSI initiator end is typically application servers or users, whereas iSCSI targets are typically
SAN access points or actual storage controllers.

© Copyright IBM Corp. 2001, 2017 6-19

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

iSCSI name discovery

• iSCSI Discovery
ƒ Configure target manually
ƒ Configure a default target, then request an iSCSI Names list
í Using separate SendTargets command
ƒ Use Service Location Protocol (SLP) to find targets
ƒ Query storage name server for accessible list of targets
• iSCSI Qualifier Name (IQN)
ƒ iqn.yyyy-mm.vendor_name_authority:unique_name
• Enterprise Unique Identifier (EUI)
ƒ eui.112233AABBCCDDEE
• Internet Storage Name Service (iSNS)
ƒ Provides discovery of targets and their IP addresses
ƒ Option that reduces need to do manual configuration

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-15. iSCSI name discovery

The iSCSI protocol enables a methodology for both naming and address of initiators and targets.
An iSCSI initiator can discover an iSCSI target by configuring the target’s address on the initiator,
by configuring a default target address on the initiator and the initiator connects to the target and
requests a list of iSCSI Names by using a separate SendTargets command, by issuing Service
Location Protocol (SLP) multicast requests to which the targets can respond, and by querying a
storage name server for a list of targets that it can access.
The iSCSI Qualified Name (IQN) is expected to be unique worldwide and in a recognizable human
readable format for the iSCSI nodes. It can be used by any organization that owns a domain name.
It also includes a date code that indicates the point in time at which the domain name was
registered, which is also included in the name in reverse format. The final field of the IQN (after the
colon) is a unique string that identifies the iSCSI device within the context of the domain owner. The
IQN can be up to 223 bytes long. iSNS is defined in RFC 3721.
The Enterprise Unique Identifier (EUI) is also referred to as EUI-64 for identifying the iSCSI nodes.
It includes an IEEE assigned Organizationally Unique Identifiers (OUI) for the manufacturer, which
is a specified value that is assigned by the IEEE registration authority. The EUI format has 16 hex
digits (64 bits) that are expected to be unique worldwide. The first 24 bits (112233 in the visual) are
the unique company ID (OUI). The subsequent 40 bits are generated exclusively by the
manufacturer. EUI is defined by the IEEE Registration Authority.

© Copyright IBM Corp. 2001, 2017 6-20

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty
The Internet Storage Name Service (iSNS) provides an option for iSCSI servers and clients to
discover targets and their IP addresses. The iSNS server can offer additional services, such as an
explicit initiator to target mappings for security. iSNS is an option for iSCSI devices to reduce the
need for manual configuration. iSNS is defined in RFC 4171.

© Copyright IBM Corp. 2001, 2017 6-21

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Performance recommendations
• Use jumbo IP packets
ƒ 1500 bytes versus 9000 bytes
• Ensure the network is set to full duplex
• Use a TCP Offload Engine (TOE)
ƒ Hardware assist
• Separate iSCSI SAN from ordinary user traffic
ƒ Physical LAN or VLAN
• Use Gigabit Ethernet
ƒ 1Gb or 10Gb

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-16. Performance recommendations

Set the Ethernet network to use jumbo packets. Most Ethernet equipment defaults to a frame size
of 1500 bytes. However, modern Ethernet equipment can offer the option of using jumbo frames
with 9000 bytes of information per frame. Packing six times as much information in each frame
improves performance. This change has to be applied to every component on the network path. A
single item set to 1500 byte frames can make the entire network default to the smaller frames.
Ensure the network is set to run in full duplex. Full duplex provides better performance than half
duplex in storage networks because it allows data to flow to and from the storage device
simultaneously. Some networks are set to half duplex for reliability or other reasons.
TCP offload engine (TOE) is a technology used in network interface cards (NIC) to offload
processing of the entire TCP/IP stack to the network controller. It is primarily used with high-speed
network interfaces, such as 1 Gb or 10 Gb Ethernet, where processing overhead of the network
stack becomes significant.
Separate regular LAN traffic from SAN traffic. Mixing SAN and regular traffic is certainly cheaper,
but it can lead to network congestion and sub-optimal performance. Analyze the traffic and consider
building a separate Ethernet network to support the iSCSI SAN.
Use Gigabit Ethernet for SAN traffic. Gigabit Ethernet has become more common and will offer the
best possible performance for iSCSI SAN. If you're installing a network just for the SAN, Gigabit
Ethernet is worth serious consideration.

© Copyright IBM Corp. 2001, 2017 6-22

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

iSCSI initiator comparison

Software only Hardware assisted Hardware only initiators
initiators Software initiators
Hardware required NIC TOE iSCSI HBA
Hardware definition Provides Ethernet A specialized NIC that An HBA that provides
Connectivity provides additional Ethernet connectivity and
functionality additional functionality.
Offloaded burden Physical and DataLink TCP/ IP, iSCSI read/write
communication physical, and DataLink processing,
communication. TCP/IP processing,
physical, and DataLink
communication
Burden managed by 9iSCSI protocol None
the server CPU management 9iSCSI protocol
9TCP/IP protocol management
management
Software-based
Yes Yes No
initiator required
iSCSI performance Adequate Good Best
Cost $ $$ $$$

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-17. iSCSI initiator comparison

Software only initiators are software-based iSCSI drivers with a standard network card. All iSCSI
and TCP processing occurs in the host CPU. Many operating systems provide software initiators as
a standard feature which minimizes costs.
Hardware assisted software initiators are software-based iSCSI drivers with a network card that
incorporates a TCP offload engine (TOE) to reduce the TCP processing load on the host’s CPU.
Many server network interface cards (NICs) include some level of TCP offload.
Hardware only initiators are host bus adapters that offloads both TCP and iSCSI processing. This
type of initiator reduces both the TCP and iSCSI processing load on the host CPU. Some iSCSI
HBAs can be used for both iSCSI and standard network traffic while others require a separate port
for non-iSCSI traffic.

© Copyright IBM Corp. 2001, 2017 6-23

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

iSCSI benefits
• Take advantage of existing Internet infrastructure
• Use existing Internet management facilities
• Address SCSI distance limitations
• Lower cost

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-18. iSCSI benefits

iSCSI devices attach to IP networks. This is attractive compared to Fibre Channel because of the
widespread use of IP networks. IP networks are already in place in most organizations and are
supported by existing IT skills. TCP/IP-based networks can potentially support much longer
distances than pure Fibre Channel SANs.
iSCSI is managed like any direct-attach SCSI device. iSCSI-connected disk volumes are visible to
attached processors. Compared to Fibre Channel SANs, iSCSI benefits from using IP networks for
which there are established network management tools and people skills, such as Tivoli NetView or
HP Openview. Such tools enable network administrators to coordinate provision of bandwidth
among users and applications, traffic management, and overall network operations. Training in new
networking skills is minimized.
IP networks offer the capability to easily extend beyond the confines of a LAN, to include
Metropolitan and Wide Area Networks (MANs and WANs). This gives greater flexibility, and at far
less cost and complexity, compared to the interconnection of Fibre Channel SANs over wide areas.
An iSCSI SAN is likely to have lower costs than a Fibre Channel SAN. For example, iSCSI network
adapters and cables cost far less than FC HBAs. In a very simple iSCSI implementation, it might be
that no additional purchases are needed in an existing setup.

© Copyright IBM Corp. 2001, 2017 6-24

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

SAN over Ethernet topics

• Ethernet basics
• TCP/IP basics
• iSCSI
• Converged Enhanced Ethernet
• Fibre Channel over Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-19. SAN over Ethernet topics

© Copyright IBM Corp. 2001, 2017 6-25

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Converged Enhanced Ethernet

• Also known as Data Center Bridging
• Consolidates various traffic types over Ethernet network
• Enables lossless traffic service on standard Ethernet
• Required for Fibre Channel over Ethernet implementations
• Implements multiple supporting standards
ƒ Priority-based Flow Control
ƒ Enhanced Transmission Selection
ƒ Data Center Bridging Exchange

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-20. Converged Enhanced Ethernet

Converged Enhanced Ethernet (CEE), or Data center bridging (DCB) refers to a set of IEEE open
standards developed to enable the convergence of various applications in data centers, such as
LAN, SAN, and HPC on a single physical link. CEE's primary focus is to consolidate the number of
cables and adapters connected to servers. Enterprise storage data requires lossless traffic service.
Converged Enhanced Ethernet (CEE) enhances the existing Ethernet standards to make them
lossless on a per priority based traffic model. Some of the major components to support CEE
includes the following:
• Priority-based Flow Control (PFC; IEEE 802.1Qbb) provides a link level flow control mechanism
that can be controlled independently for each frame priority. The goal of this mechanism is to
ensure zero loss under congestion in DCB networks.
• Enhanced Transmission Selection (ETS; 802.1Qaz) provides a common management
framework for assignment of bandwidth to frame priorities.
• Congestion Notification (IEEE 802.1Qau) provides end to end congestion management for
protocols that are capable of transmission rate limiting to avoid frame loss.
• Data Center Bridging Exchange Protocol (DCBX) is a discovery and capability exchange
protocol that is used for conveying capabilities and configuration of the above features between
neighbors to ensure consistent configuration across the network.

© Copyright IBM Corp. 2001, 2017 6-26

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Priority-based Flow Control

• Eliminates frame loss due to congestion
• Enables flow control per traffic class
• Extends the concept of IEEE 802.3x PAUSE mechanism
ƒ Stop sending data on a per priority basis
Transmit queues Receive buffers

0 0

1 1

2 STOP PAUSE 2

3 3

4 4

5 5

6 6

7 7

Ethernet link
SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-21. Priority-based Flow Control

Priority-based Flow Control (PFC; 802.1Qbb) provides a link level flow control mechanism that can
be independently controlled at a priority level and that can selectively pause different classes of
traffic. The goal is to ensure zero loss due to congestion in data center bridging networks. With
priority-based flow control, separate flow control mechanisms can be used for different traffic
classes.
The Ethernet use of IEEE 802.3x PAUSE for flow control enables feedback from a receiver to its
sender to communicate buffer availability. A receiver generates a MAC control frame to send a
PAUSE request to a sender when it predicts the potential for buffer overflow. Upon receiving a
PAUSE frame, the sender responds by stopping transmission of any new packets until the receiver
is ready to accept them again. The consequence is that the application of IEEE 802.3x PAUSE
makes an Ethernet segment unsuitable for carrying multiple traffic flows that might require different
quality of service (QoS).
IEEE 802.1Qbb PFC extends the basic IEEE 802.3x PAUSE semantics to individual priority groups,
enabling applications that require flow control to coexist on the same wire with applications that
perform better without it.

© Copyright IBM Corp. 2001, 2017 6-27

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Enhanced Transmission Selection

• Optimizes bandwidth management of virtual links
• ETS information exchanged in DCBX
• Guarantees bandwidth minimum value
Desired traffic ETS 10Gb link throughput
configuration
Priority
group 1 3 Gb 3 Gb 3 Gb
2 Gb
3 Gb 3 Gb 2 Gb
Priority
group 2 3 Gb 3 Gb 3 Gb 3 Gb
3 Gb 3 Gb 3 Gb

Priority 3 Gb 4 Gb 5 Gb
group 3 5 Gb
6 Gb
4 Gb
3 Gb
9 Gb 10 Gb 10 Gb

t1 t2 t3 t1 t2 t3

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-22. Enhanced Transmission Selection

Enhanced Transmission Selection (ETS; 802.1Qaz) enables a percentage of available bandwidth

on a link to be divided among specified priority groups. ETS specifies different amounts of link
bandwidth for different traffic types (such as for LAN, SAN, and management). ETS provides traffic
differentiation, such that multiple traffic classes can share the same consolidated Ethernet link
without impacting each other.
Priority groups are defined by Priority-based Flow Control (PFC) and represents a traffic class,
such as LAN, SAN, IPC, and management. Each priority group is assigned a priority group ID with
a value of either 15, or a number in the range of 0 through 7. All traffic with a certain PGID is
handled with the same priority and allocation rules. Lossless or lossy functionality is also defined for
each priority group. When a group does not fully utilize its bandwidth allocation, the unused
bandwidth is given to the other groups. PGID value of 15 is the highest priority traffic class.

© Copyright IBM Corp. 2001, 2017 6-28

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Data Center Bridging Exchange protocol

• Extension of Link Layer Discovery Protocol
• Exchanges priority map values
ƒ For FCoE and FCoE Initialization Protocol (FIP)
• Enables lossless behavior

End CEE/DCB
Node Switch

DCBX

DCBX

FIP discovery

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-23. Data Center Bridging Exchange protocol

The Data Center Bridging Exchange Protocol (DCBX) is a discovery and capability exchange
protocol responsible for the configuration of link parameters for Data Center Bridging functions. It
determines which devices support the enhanced functionalities that create the “DCB” cloud where
FCoE traffic can safely travel. It also allows for transparent pass-through to non-DCB traffic classes.
This protocol uses functions provided by Link Layer Discovery Protocol (LLDP; IEEE 802.1AB).

© Copyright IBM Corp. 2001, 2017 6-29

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

SAN over Ethernet topics

• Ethernet basics
• TCP/IP basics
• iSCSI
• Converged Enhanced Ethernet
• Fibre Channel over Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-24. SAN over Ethernet topics

© Copyright IBM Corp. 2001, 2017 6-30

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Fibre Channel over Ethernet

• FCoE is direct mapping of Fibre Channel upper layers onto Ethernet
• Aims to reduce cost through sharing of Ethernet infrastructure.
• Assumes lossless network where frames are never dropped
ƒ Does not use TCP and IP layers
• Standard specified by ANSI T11 committee within FC-BB-5.
• Ethernet enhancements are defined by IEEE standards
• Supported by the storage networking industry
ƒ Including IBM, Brocade, Cisco, QLogic, HP, and Intel

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-25. Fibre Channel over Ethernet

Fibre Channel over Ethernet (FCoE) is the encapsulation of Fibre Channel frames inside of
Ethernet frames. The primary goal of FCoE is to reduce cost by sharing the Ethernet infrastructure
which is already in place. FCoE assumes a lossless network and exploits the features of
Converged Enhanced Ethernet (CEE). FCoE does not use TCP/IP protocols to deliver data through
the network.
While the primary driving force behind FCoE is the current storage and SAN vendors, the standards
effort is currently handled by the ANSI T11 committee which is the same body that manages the
main Fibre Channel standards. FCoE is a subset of the T11 FC standard FC-BB-5. FCoE is
supported in the storage industry across a wide spectrum of vendors.

© Copyright IBM Corp. 2001, 2017 6-31

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

FCoE topology
• Typical data center networks
Ethernet Switches
LAN
NIC
Server
Fibre Channel Switches
SAN
HBA

• FCoE network
DCB Capable Ethernet Switches LAN

Server CNA FCoE

Forwarder

SAN

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-26. FCoE topology

In a traditional storage area network (SAN) and LAN environment, the server has an Ethernet
adapter for data traffic and a Fibre Channel host bus adapter (HBA) for storage traffic. With FCoE,
these two adapters are replaced by a Converged Network Adapter (CNA) that can service both
kinds of traffic.
A cable from the CNA connects to a lossless Ethernet (DCB capable) switch, which eventually
provides connectivity to a FCoE Forwarder (FCF) and storage devices. The FCF is the
communication bridge between the Enhanced Ethernet and an FC fabric.
From the Fibre Channel perspective, FCoE works like another flavor of a physical media that can
carry Fibre Channel frames between virtual Fibre Channel ports.

© Copyright IBM Corp. 2001, 2017 6-32

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

FCoE protocol layers

• FCoE bottom layer is CEE/DCB Ethernet
• Fibre Channel stack is unchanged by FCoE implementation
• FCoE does not use TCP/IP layers

Applications
SCSI
FCP FCP iSCSI

TCP
FCoE IP
FC Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-27. FCoE protocol layers

To send the packets over the network, changes are made to the protocol stack to accommodate the
Enhanced Ethernet. The bottom two layers of the FC protocol stack have been replaced with their
Ethernet protocol stack equivalents of the Media Access Control (MAC) and the Physical layers.
The FC-BB-5 project group members have the responsibility for ensuring that the existing FC stack
remains unaffected and that the work that has already gone into the existing FC stack is neither
undermined or lost.
FCoE, unlike iSCSI, does not use TCP/IP protocols. The mappings for transporting Fibre Channel
over Ethernet are provided in the FCoE layer of the stack. Because an Ethernet network can lose
frames, it is the extensions to the Ethernet that will allow FCoE to exhibit a lossless and full duplex
behavior when carrying Fibre Channel frames.
The FCoE layer interfaces with the portion of the Fibre Channel protocol layer FC-2 (FC-2V) that
supports a virtual Fibre Channel port, regardless of the used physical media.

© Copyright IBM Corp. 2001, 2017 6-33

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

FCoE packet comparison

Fibre
FCP SCSI Data
Channel

Ethernet IP TCP iSCSI SCSI Data

Ethernet FCoE FCP SCSI Data

CDB

FCP frame

FCoE frame

Ethernet frame

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-28. FCoE packet comparison

The standard Ethernet frame is a maximum of 1518 bytes and the maximum FC frame size of 2148
bytes. if one were to simply wrap the frames into an Ethernet frame, some form of fragmentation, or
segmentation, must occur due to the difference in maximum sizes (1518 compared to 2148).
Although this segmentation is possible, it is not desirable because it adds more processing
overhead to the operation. Conventionally, jumbo frames can carry up to 9000 bytes of payload, but
variations exist and one must be careful when using the term jumbo frames.
Although jumbo frames are not a standard, the quickest and simplest option is to require jumbo
frame support for every device in the FCoE network. “Baby” jumbo frames of approximately 2500
bytes are desirable.
FCoE encapsulates Class 2, 3, and F Fibre Channel frames into appropriate Ethernet frames.

© Copyright IBM Corp. 2001, 2017 6-34

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

FCoE components
• Converged network adapter (CNA)
• Converged cables
ƒ Optical
ƒ Twinax
• CEE/DCB Ethernet switches
ƒ Lossless Ethernet
ƒ Full duplex
ƒ Jumbo frame support
ƒ FCoE Forwarder (FCF)
í Provide fabric login and other Fibre Channel protocol services
í Encapsulate/de-encapsulate frames between CNA and SAN

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-29. FCoE components

Converged Network Adapter (CNA) is similar to an HBA or a NIC, but instead of handling either
Fibre Channel or IP, the CNA can handle both simultaneously. The CNA presents separate
networking and storage system interfaces to the operating system.
FCoE cable options include optical cabling generally found in Fibre Channel SANs and a newer
type of Twinax copper cabling. FCoE Twinax cables require less power and are less expensive, but
because their distance is limited to fewer than 10 meters, optical cabling is a likely choice.
The Ethernet infrastructure, over which FCoE will travel, must of the lossless nature. CEE/DCB
switches support this lossless behavior. Key features needed to support FCoE transmission include
Priority Flow Control (PFC) and Enhanced Transmission Selection (ETS). For performance
reasons, full duplex and jumbo frames are recommended.
Traffic between two FCoE end nodes (ENodes) is forwarded through one or more FCoE
Forwarders (FCFs). FCoE Forwarder (FCF) handles the login requests and provides the Fibre
Channel services typically associated with a Fibre Channel switch. FCFs can optionally provide the
means to encapsulate and de-encapsulate Fibre Channel frames that flow between the CNA and
the SAN.

© Copyright IBM Corp. 2001, 2017 6-35

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

FCoE fabric components

• FCoE_LEP represents the assigned MAC address
• Virtual switch ports are created by the FCoE Controller
ƒ VN_Port, VF_Port, and VE_Port
• FCF provides switching from Ethernet to Fibre Channel

Fibre Channel

E/F_Port E/N_Port

ENode
ENode FCF FC Device

VN_Port
VN_Port VF_Port

FCoE_LEP
FCoE_LEP FCoE_LEP
CNA port
CNA port CNA port
CNA port

CEE/DCB Ethernet

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-30. FCoE fabric components

The FCoE End Node (ENode) is an FC node that is associated with one or more Enhanced
Ethernet MACs, one or more FCoE Link End Points (FCoE_LEPs), and one or more VN_Ports.
Note that each Enhanced Ethernet MAC is coupled with an FCoE Controller. The FCoE_Controller
(which is in the ENode and the FCF) is responsible for the creation of VN_Ports, VF_Ports,
VE_Ports, and FCoE_LEPs. The FCoE Controller also performs the FCoE Initialization Protocol
(FIP).
The FCoE Forwarder (FCF) is a function that exists in a switch that has Ethernet ports and is
responsible for translating the FCoE frames between the Enhanced Ethernet and an FC SAN. On
the Enhanced Ethernet side, this function can be within a device, or it can be integrated into an
Enhanced Ethernet switch. On the FC SAN side, the native FC ports connect to a Fibre Channel
switch.
The FCoE Link End Point (FCoE_LEP) is the data forwarding component of an FCoE entity that
handles the encapsulation and de-encapsulation of the Fibre Channel frames, and handles the
exchange of encapsulated frames with another FCoE_LEP over the Ethernet network.
The Virtual N_Port (VN_Port) operates as an N_Port and is dynamically created on a successful
completion of the FCoE Initialization Protocol (FIP) FLOGI exchange.
The Virtual F_Port (VF_Port) communicates with one or more VN_Ports and is dynamically created
on a successful completion of the FCoE Initialization Protocol (FIP) FLOGI exchange.

© Copyright IBM Corp. 2001, 2017 6-36

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty
The Virtual E_Port (VE_Port) communicates with another VE_Port and is dynamically created on a
successful completion of the FCoE Initialization Protocol (FIP) Exchange Link Parameter (ELP)
exchange.

© Copyright IBM Corp. 2001, 2017 6-37

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

FCoE initialization flow

• FCoE Initialization Protocol (FIP) (Ethertype 8914)
– FIP discovery
– FCoE virtual link creation/maintenance
• VLAN discovery
– FIP VLAN information returned
• FCoE discovery
– FCF replies unicast FCF MAC address
• Fabric login
– ENode sends FLOGI
– FCF replies accept and assigned VN_Port MAC address
í Server Provided MAC Address (SPMA)
í Fabric Provided MAC Address (FPMA)
–Format XX:XX:XZ:YY:YY:YY
> XX:XX:XZ = FC-MAP (Z=domain ID, 0E:FC:00 is default)
> YY:YY:YY = FCID (assigned during FLOGI)

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-31. FCoE initialization flow

Before any FCoE frame is transported between virtual Fibre Channel ports, each FCoE device
must perform the functions of FCF detection, FCoE VLAN discovery, virtual link initialization, and
connection maintenance. These functions are coordinated by the FCoE Initialization Protocol (FIP).
All FIP frames use an Ether-type of x’8914’ and each ENode uses its original MAC address on all
FIP frames.
The Discovery phase can include a VLAN exchange and then an FCoE discovery exchange.
The Login phase creates a VN_Port on the ENode and associates it with the VF_Port in the FCF.
These virtual ports are then used for normal FCoE traffic between the ENode and the FCF.
VE_Ports, which are formed between two FCFs perform the same negotiations phases in a similar
way as the VN_Port and VF_Port pair. However, the virtual link connection is dependent on a
successful completion of a FIP ELP request instead of the FIP FLOGI request.
Server-Provided MAC Address (SPMA) is a MAC address that is issued in accordance with
Ethernet standards and set by the manufacturer at installation.
Fabric-Provided MAC Address (FPMA) is a fabric-unique address that is assigned by the FCF. The
low-order 24 bits are equivalent to the N_Port ID (FCID) assigned during fabric login, and the
high-order 24 bits are equal to the FCoE MAC address prefix (FC-MAP) associated with the fabric.

© Copyright IBM Corp. 2001, 2017 6-38

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Keywords
• DataLink layer • iSCSI initiator
• Media access control sublayer • iSCSI targets
• Logical link control sublayer • Internet Storage Name Service
• Physical layer • TCP offload engine
• Bus topology • Converged Enhanced Ethernet
• Star topology • Data center bridging
• Switched Ethernet • Priority-based Flow Control
• Broadcast domain • Enhanced Transmission Selection
• Jumbo frames • Data Center Bridging Exchange
• Virtual LAN Protocol
• Transmission Control Protocol/Internet • Fibre Channel over Ethernet
Protocol • Converged Network Adapter
• Transport layer • FCoE Forwarder
• Internetwork layer • FCoE End Node
• Connection-oriented • FCoE_Controller
• TCP sockets • FCoE Initialization Protocol
• TCP segments • FCoE Link End Point
• Routers • VN_Port, VF_Port , VE_Port
• iSCSI Protocol Data Unit • Server-Provided MAC Address
• iSCSI Qualified Name • Fabric-Provided MAC Address
SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-32. Keywords

© Copyright IBM Corp. 2001, 2017 6-39

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Review questions (1 of 4)
1. Which of the following is required to deliver multiple
broadcast domains across a single link on an Ethernet
network?
a. VLAN
b. Router
c. Hub
d. Bus

2. True or False: A TCP connection provides a flow control

mechanism for delivering data.

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-33. Review questions (1 of 4)

Write answers here.

© Copyright IBM Corp. 2001, 2017 6-40

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Review questions (2 of 4)
3. Which of the following is not included for the identification of
a TCP socket?
a. IP address
b. Port address
c. TCP protocol ID
d. MAC address

4. True or False: iSCSI implementations use TCP sockets

connections.

5. True or False: The iSCSI target name is associated with a

specific IP address.

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-34. Review questions (2 of 4)

Write answers here.

© Copyright IBM Corp. 2001, 2017 6-41

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Review questions (3 of 4)
6. Which of the following is a feature of Enhanced
Transmission Selection?
a. Enables flow control per traffic class
b. Optimizes bandwidth management of virtual links
c. Exchanges priority map values
d. Pre-requisite for iSCSI implementation

7. True or False: The TCP offload engine enhances the

network performance of FCoE.

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-35. Review questions (3 of 4)

Write answers here.

© Copyright IBM Corp. 2001, 2017 6-42

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Review questions (4 of 4)
8. Which address represents the FCoE link end point?
a. Network interface MAC address
b. IP address
c. Fabric-provided MAC address
d. Fibre Channel ID

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-36. Review questions (4 of 4)

Write answers here.

© Copyright IBM Corp. 2001, 2017 6-43

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Unit 6. SAN over Ethernet

Uempty

Unit summary
• Describe basic concepts of Ethernet
• Describe basic concepts of TCP/IP
• Describe key features of the iSCSI implementation
• Distinguish the features of the Converged Enhanced Ethernet major
components
• Describe key features of the FCoE implementation
• Evaluate the implementations of iSCSI and FCoE

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Figure 6-37. Unit summary

© Copyright IBM Corp. 2001, 2017 6-44

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Appendix A. Review answers

Unit 1, "SAN concepts"
Solutions for Figure 1-37, "Review questions (1 of 3)," on page 1-39

Review answers (1 of 3)
1. What is a key design element of an intelligent storage
system?
a. Offer flexible networking options
b. Eliminate the single point of failure
c. Increase the storage capacity
d. Provide a faster processor
The answer is Eliminate the single point of failure.

2. What are components of the SAN Fabric in a SAN

infrastructure?
a. Hosts and HBAs
b. Storage systems and controllers
c. Switches and directors
d. Thin provisioning and copy services
The answer is Switches and directors.

SAN concepts © Copyright IBM Corporation 2001, 2017

Solutions for Figure 1-38, "Review questions (2 of 3)," on page 1-40

© Copyright IBM Corp. 2001, 2017 A-1

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (2 of 3)
3. Which storage network technology uses a client/server
model?
a. DAS
b. NAS
c. SAN
d. SNA
The answer is NAS.

4. True or False: SAN implementation is simpler to manage

and scale because of the consolidated storage.
The answer is True.

SAN concepts © Copyright IBM Corporation 2001, 2017

Solutions for Figure 1-39, "Review questions (3 of 3)," on page 1-41

© Copyright IBM Corp. 2001, 2017 A-2

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (3 of 3)
5. The Fibre Channel SAN topology can be viewed as an extension
to the SCSI bus.
a. The SCSI bus
b. The SSA loop
c. The ESCON
d. The token ring
The answer is The SCSI bus.

SAN concepts © Copyright IBM Corporation 2001, 2017

Unit 2, "Fibre Channel"

Solutions for Figure 2-43, "Review questions (1 of 2)," on page 2-48

© Copyright IBM Corp. 2001, 2017 A-3

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (1 of 2)
1. Which of the following are benefits of Fibre Channel?
a. Long distance
b. Low latency
c. Reliable transfer
d. High connectivity
e. All the above
The answer is All the above.

2. Which Fibre Channel layer provides a safety system for

open fiber systems?
The answer is FC-0.

3. Which Fibre Channel layer provides common services?

The answer is FC-3.

Fibre Channel © Copyright IBM Corporation 2001, 2017

Solutions for Figure 2-44, "Review questions (2 of 2)," on page 2-49

© Copyright IBM Corp. 2001, 2017 A-4

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (2 of 2)
4. True or False: Fibre Channel service class 3 uses end-to-
end buffer credits for flow control.
The answer is False.

5. What is the purpose of the Fibre Channel 8b/10b encoding?

a. Synchronize the clocking between sender and receiver
b. Ensure data arrives in correct order
c. Establish end point connections
d. Recover data errors
The answers are Synchronize the clocking between sender and
receiver and Recover data errors.

Fibre Channel © Copyright IBM Corporation 2001, 2017

Unit 3, "SAN design and topology"

Solutions for Figure 3-47, "Review questions (1 of 4)," on page 3-52

© Copyright IBM Corp. 2001, 2017 A-5

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (1 of 4)
1. What are the three major components of a SAN?
The answer is connectivity, storage, and servers.

2. Which SAN topology never uses switch technology?

The answer is point-to-point topology.

3. What switch topology requires ISLs to every other switch in

the fabric?
a. Ring
b. Full mesh
c. Core-edge
d. Edge-core-edge
e. Loop
The answer is Full mesh.

SAN design and topology © Copyright IBM Corporation 2001, 2017

Solutions for Figure 3-48, "Review questions (2 of 4)," on page 3-53

© Copyright IBM Corp. 2001, 2017 A-6

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (2 of 4)
4. Which SAN switch port type represents a host end port?
a. N_Port
b. F_Port
c. E_Port
d. VE_Port
The answer is N_Port.

5. Which SAN switch port type is used to connect to another

SAN switch?
a. N_Port
b. F_Port
c. E_Port
d. D_Port
The answer is E_Port.

SAN design and topology © Copyright IBM Corporation 2001, 2017

Solutions for Figure 3-49, "Review questions (3 of 4)," on page 3-54

© Copyright IBM Corp. 2001, 2017 A-7

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (3 of 4)
6. True or False: The source ID in a Fibre Channel frame is
the worldwide node name sending system.
The answer is False.

7. True of False: The login that results in the assignment of the

port address is called the PLOGI.
The answer is False.

SAN design and topology © Copyright IBM Corporation 2001, 2017

Solutions for Figure 3-50, "Review questions (4 of 4)," on page 3-55

© Copyright IBM Corp. 2001, 2017 A-8

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (4 of 4)
8. True or False: A device can participate in multiple SAN
zones.
The answer is True.

9. True or False: LUN masking cannot be implemented at the

same time with port zoning.
The answer is False.

10. True or False: An NPV enabled switch is assigned a switch

domain ID when inserted into a fabric.
The answer is False.

SAN design and topology © Copyright IBM Corporation 2001, 2017

Unit 4, "IBM Fibre Channel b-type switches and directors"

Solutions for Figure 4-111, "Review questions (1 of 4)," on page 4-115

© Copyright IBM Corp. 2001, 2017 A-9

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (1 of 4)
1. True or False: All IBM SAN b-type switches support Fibre
Channel link speeds up to 16 Gb.
The answer is False.

2. Which Fibre Channel b-type SAN switch model supports

multiprotocol and distance extension?
a. SAN42B-4
b. SAN48B-5
c. SAN42B-R
d. SAN96B-5
The answer is SAN42B-R.

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Solutions for Figure 4-112, "Review questions (2 of 4)," on page 4-116

© Copyright IBM Corp. 2001, 2017 A-10

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (2 of 4)
3. How many port blades can be accommodated by the IBM
System Storage SAN768-2?
a. 2
b. 4
c. 8
d. 12
The answer is 8.

4. Which SAN switch administration account is suitable for one

to view without configuration authority?
a. Admin
b. User
c. Root
d. Factory
The answer is user.

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Solutions for Figure 4-113, "Review questions (3 of 4)," on page 4-117

© Copyright IBM Corp. 2001, 2017 A-11

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (3 of 4)
5. True or False: The Web Tools switch administration is found
in the Switch View tab.
The answer is False.

6. What is the initial port state in SAN switch port initialization?

a. G_Port
b. U_Port
c. E_Port
d. F_Port
The answer is U_Port.

7. True or False: Four inter-switch links each running at 4 Gb

between the same two SAN switches always functions as
one logical 16 Gb path.
The answer is False.

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Solutions for Figure 4-114, "Review questions (4 of 4)," on page 4-118

© Copyright IBM Corp. 2001, 2017 A-12

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (4 of 4)
8. Which of the following enables path selection in a multi-
switch Fibre Channel environment?
a. ISL
b. ICL
c. FSPF
d. FCCT
The answer is FSPF.

9. True or False: IBM Network Advisor can support both IP and

SAN Networks.
The answer is True.

IBM Fibre Channel b-type switches and directors © Copyright IBM Corporation 2001, 2017

Unit 5, "Cisco MDS Fibre Channel switches and directors"

Solutions for Figure 5-46, "Review questions (1 of 4)," on page 5-55

© Copyright IBM Corp. 2001, 2017 A-13

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (1 of 4)
1. True or False: All IBM offered MDS directors support 16 Gb
Fibre Channel
The answer is False.

2. True or False: The Cisco DCNM client must be installed on

the user's local system in order to manage a SAN switch.
The answer is False.

3. True or False: A redundant Supervisor module in a Cisco

MDS director is optional.
The answer is False.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Solutions for Figure 5-47, "Review questions (2 of 4)," on page 5-56

© Copyright IBM Corp. 2001, 2017 A-14

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (2 of 4)
4. True or False: Cisco Device Manager supports a multi-
switch fabric.
The answer is False.

5. What name identifies the Cisco feature that supports the

building of isolated SANs?
a. CSAN
b. VSAN
c. isoSAN
d. Cisco SAN
The answer is VSAN.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Solutions for Figure 5-48, "Review questions (3 of 4)," on page 5-57

© Copyright IBM Corp. 2001, 2017 A-15

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (3 of 4)
6. What is the Cisco port type needed to support the
transmission of multiple isolated SANs?
a. ST_Port
b. E_Port
c. VE_Port
d. TE_Port
The answer is TE_Port

7. True or False: A port channel is the aggregation of multiple

ISLs into one logical link.
The answer is True.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Solutions for Figure 5-49, "Review questions (4 of 4)," on page 5-58

© Copyright IBM Corp. 2001, 2017 A-16

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (4 of 4)
8. True or False: Multiple zonesets can be activated at the
same time on a one VSAN.
The answer is False.

Cisco MDS Fibre Channel switches and directors © Copyright IBM Corporation 2001, 2017

Unit 6, "SAN over Ethernet"

Solutions for Figure 6-33, "Review questions (1 of 4)," on page 6-40

© Copyright IBM Corp. 2001, 2017 A-17

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (1 of 4)
1. Which of the following is required to deliver multiple
broadcast domains across a single link on an Ethernet
network?
a. VLAN
b. Router
c. Hub
d. Bus
The answer is VLAN.

2. True or False: A TCP connection provides a flow control

mechanism for delivering data.
The answer is True.

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Solutions for Figure 6-34, "Review questions (2 of 4)," on page 6-41

© Copyright IBM Corp. 2001, 2017 A-18

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (2 of 4)
3. Which of the following is not included for the identification of
a TCP socket?
a. IP address
b. Port address
c. TCP protocol ID
d. MAC address
The answer is MAC address.

4. True or False: iSCSI implementations use TCP sockets

connections.
The answer is True.

5. True or False: The iSCSI target name is associated with a

specific IP address.
The answer is False.

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Solutions for Figure 6-35, "Review questions (3 of 4)," on page 6-42

© Copyright IBM Corp. 2001, 2017 A-19

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (3 of 4)
6. Which of the following is a feature of Enhanced
Transmission Selection?
a. Enables flow control per traffic class
b. Optimizes bandwidth management of virtual links
c. Exchanges priority map values
d. Pre-requisite for iSCSI implementation
The answer is Optimizes bandwidth management of virtual links.

7. True or False: The TCP offload engine enhances the

network performance of FCoE.
The answer is False.

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

Solutions for Figure 6-36, "Review questions (4 of 4)," on page 6-43

© Copyright IBM Corp. 2001, 2017 A-20

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0
Appendix A. Review answers

AP

Review answers (4 of 4)
8. Which address represents the FCoE link end point?
a. Network interface MAC address
b. IP address
c. Fabric-provided MAC address
d. Fibre Channel ID
The answer is Fabric-provided MAC address.

SAN over Ethernet © Copyright IBM Corporation 2001, 2017

© Copyright IBM Corp. 2001, 2017 A-21

Course materials may not be reproduced in whole or in part without the prior written permission of IBM.
V11.0

backpg

© Copyright International Business Machines Corporation 2001, 2017.