Elective (1)

Storage Technologies:
Section 2
Common Storage Protocols
www.huawei.com

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved.

 Contents
1. SCSI/iSCSI
2. SAS
3. FC/FCOE
4. PCIe
5. IB
6. CIFS/NFS
7. FTP/HTTP

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 2
SCSI And iSCSI in Storage

DAS

IP-SAN

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 3
What is SCSI ?

SCSI Device
Commands

SCSI System Model

SCSI Physical Link

Mapping

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 4
History of SCSI
SCSI-1 SCSI-2
 Released in 1983-1985.  Released in 1988-1994.
 Supports asynchronous and  Compatible with SCSI-1.
synchronous transmission.  Supports 16bit broadband.
SCSI-1  Supports up to 7 units of 8bit devices.  Highest speed of 16MBps.
 Highest speed of 5Mbps.  Still in use.
 6m maximum distance for 50pin data
cables.
 Obsolete. SCSI-3
SCSI-2  Becomes standard at 1993.
 Compatible with SCSI-2/SCSI-1
 Becomes a true system

SCSI-3 architecture.
 Supports FCP/IEEE1394 and
other types of media.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 5
SCSI Logical Topology

Initiator 1 Initiator 2 Initiator

Target

Target 1 Target 2 Target 3

LUN 0
LUN 2
LUN 5
LUN namespace Target device

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 6
SCSI Initiator Model

SCSI Device Driver Layer

SCSI Middle Layer

SCSI Link Layer

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 7
SCSI Target Model

Port Layer

Middle Layer

Device Layer

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 8
SCSI Protocol and Storage System
 SCSI protocol is the fundamental protocol between hosts and storage disks.
 DAS(Direct Attached Storage) uses SCSI protocol for host and storage device
interconnection.

SCSI BUS
Host Adapter

SCSI ID 7
Data/LBA BUS

SCSI Array
Control Signals SCSI Array
ID 0 ID 5

LUN 0 LUN 1 LUN 0

LUN 2 LUN 1

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 9
SCSI Protocol Addressing

Bus ID Differentiates between different SCSI Main Bus.

Device Differentiates between different

ID devices on the SCSI Bus.

Differentiates between the sub-

LUN devices between SCSI Device.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 10
The Birth of iSCSI
•SCSI allows smaller number of
connected devices.
•SCSI has very limited device
connection distance.

•SCSI Based On IP Network: iSCSI

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 11
What is iSCSI ?
SCSI Application (File System, Database)

SCSI Block Command SCSI Flow Command Other SCSI Command

SCSI Commands, Data and Status

iSCSI

TCP

IP

Ethernet Network

iSCSI Protocol Stack

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 12
iSCSI Initiator - Target Model
 Initiator iSCSI
 SCSI layer is responsible to generate the Initiator Target
CDB (Command Descriptor Block) and
SCSI SCSI
sends it to iSCSI.
 iSCSI is responsible to generate iSCSI
PDU(Protocol Data Unit), and send it to the iSCSI iSCSI
target over IP Network.

 Target TCP TCP

 iSCSI layer receives the PDU and sends the
CDB to the SCSI layer.
IP IP
 SCSI layer is responsible for interpreting the
meaning of the CDB and send responses if
required. Link Link

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 13
iSCSI System Architecture
 iSCSI nodes encapsulates the SCSI commands and data blocks into iSCSI PDU, then
sends it to the TCP/IP layer where the iSCSI PDU if further encapsulated into IP packets
suitable for transmission over IP network.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 14
SAS in Storage System
 In enterprise storage systems, SAS (Serial Attached SCSI) interface has already replaced
SCSI and SATA interface.
 SAS uses point to point architecture, and its performance can go up to 300MB/s, 600MB/s
or higher.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 15
Why is SAS Developed?
1. Parallel bus has reached its peak in development and reached its limit in
bandwidth.
2. Serial Bus technologies like Fibre Channel, InfiniBand, Ethernet has its
own disadvantages for application in storages:
a) FC: High costs, and suitable for complex networking and long
distance scenarios.
b) InfiniBand: Complex networking, very high costs.
c) iSCSI: Long time delay, slow transmission speed.

Serial Attached SCSI: SAS

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 16
What is SAS?
 SAS: Serial Attached SCSI, is the serial standard for SCSI bus protocol.
 SAS uses serial technology to obtain higher transmission rate and better expansion capability,
and it is compatible with SATA hard disks.
 SAS current transmission rates are up to 3Gbps, 6Gbps, 12Gbps or higher, and supports full-
duplex mode connection.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 17
SAS Protocol Layered Architecture

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 18
SAS Features
 SAS serial communication method allows multiple devices in multiple data paths to communicate at
full speed.
 SAS supports wide link that is formed through the binding of multiple narrow links.
 SAS architecture uses expander for interface expansion and has very good expansion capabilities.
 SAS uses Full-Duplex Mode in data communication.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 19
SAS Expansion Capabilities
 SAS architecture uses expander to perform interface expansion, and has very good
expansion capabilities. 1 SAS domain can connect up to the maximum of 16384 units of
storage devices.
Expander

Expander

Expander

SAS
RAID Expander * SAS/SATA Standard Single Port
Controller Connectors And Cables.
1

Expander

Expander

128

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 20
Principles of SAS Cable
 SAS cables commonly has 4 paths, and the speed of each path commonly used is 12GB/s.
 SAS devices are connected in the form of loops (also known as chains).
 The bandwidth of the cable is 4X12GB/s, which limits the number of disk devices in the SAS loop.
 The current best practice for the maximum number of hard disks in a SAS loop is 168 disks, which
means a maximum of 7 hard disk enclosures with 24 disk slots each to form a loop.

SAS Cable Connector is shown on the

picture on the right.

Mini SAS

High Density Mini SAS

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 21
Comparison Between SAS And Other
Transmission Technologies
Types of Fields Of
Main Advantages Main Disadvantages
Technology Application

ATA Low Cost Low Performance PC

Medium Drawbacks of Parallel Enterprise Grade

SCSI
Performance Technologies Storages
High Performance, High End
FC High Cost
High Reliability Storages
Low Cost, High Low Performance and Mid and Low End
SATA
Capacity Low Reliability Storages
High Performance, Mid and Low End
SAS Medium Cost
High Reliability Storages

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 22
Fibre Channel (FC) In Storages

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 23
What is FC ?

 FC is an abbreviation for
Fibre Channel. It is used for
the connection of servers
and shared storage devices,
and also for the internal
connection between storage
controllers and drives.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 24
FC Topologies
Point to Point Arbitration Loop FC Switched Network

Most Widely Adopted Technology

Only connects 2 devices Supports maximum of Supports maximum of 16

127 devices (Fiber million devices
(Direct Connection) Hub) (FC Switches)

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 25
FCoE Protocol
 It is a protocol that directly transmits FC signals over the enhanced lossless Ethernet
network.
 FCoE encapsulates the FC frame within an Ethernet Frame, allowing LAN and SAN
service traffic to be transmitted through the same Ethernet network

Ethernet Data
Link Layer
Frame
 IP Service

 FCoE (Block Storage)

 VoIP
 Video stream of VoIP

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 26
Differences Between FC and FCoE
 FCoE : Fibre Channel over Ethernet
 FCoE is not meant to replace the FC Differences in Protocol Stack
technology, but it is an expansion of
FC in different connection and
transmission layer.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 27
FC Connectors
3 ． ST Type 4 ． MT-RJ Type Connector
Connector

1 ． FC Type Connector

FC/PC Type FC/APC Type 5 ． LC Type Connector 6 ． MU Type Connector

2 ． SC Type Connector

SC/PC Type SC/APC Type

SC (Connects to GBIC)
LC is commonly used
(Connects using SFP)

 ESFP Type  SFP Type  GBIC Type

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 28
What Is PCIe ?
 PCI Express (also known as PCIe) is a high-performance, high-bandwidth serial
communications interconnection standard first introduced by Intel and later
developed by the Peripheral Component Interconnect Special Interest Group
(PCI-SIG) to replace bus-based communications Architectures such as: PCI, PCI
Extended (PCI-X), and Accelerated Graphics Port (AGP).

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 29
Why Use PCIe ?

 The main motivation for moving to PCIe is to achieve significantly higher

system throughput, scalability, and flexibility with lower production costs.
 The qualities mentioned above are almost impossible to achieve using
traditional bus based connections.

High-Performance, High-Bandwidth Serial

Communication Interconnection Standard:
PCI Express (PCIe)

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 30
PCIe Protocol Architecture
 Layers in PCIe Device Protocol Architecture are as follows:
 Physical Layer PCIe Device
 Data Link Layer
APP
 Transaction Layer
 Application Layer
Transaction

Physical
Data Link
Logical Sub-Block
Physical
Electrical Sub-Block

Tx Rx

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 31
InfiniBand (IB) in Storage Systems
IB in Converged Storage IB In Distributed Storage

Server IB Back End

IB Back End
Switch
Switch
ini
9000 9000 9000 9000
FCoE FC TOE 10 GE IB
…
tgt Front End Node Node Node Node
1 2 3 n
Storage Controller

ini Back End Distributed Cluster

SAS SAS
IB Front Client
End End
tgt Switches

Disk Switch Level Connection

Back End Network
Ini: Initiator Front End Network
Tgt ： Target

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 32
What Is IB ?
 IB (InfiniBand):
 InfiniBand technology is not used for general network connection. Its main design purpose is to solve server-side
connection problems.
 InfiniBand technology is used for communication between servers and servers (such as replication, distributed
work, etc.), between servers and storage devices (such as SANs and direct storage attachments), and between
servers and networks (such as LANs, WANs, and the Internet).

 InfiniBand features:
 Based on standard protocols
 High bandwidth, low latency
 Remote direct memory access
 Transport offload

 The key to the InfiniBand architecture is solving the bottleneck problem of the shared bus by adopting
a point-to-point switching architecture that is specifically designed to address fault tolerance and
scalability problems. By adding switches to the InfiniBand system, I/O system expansion can be easily
implemented, and thus allowing more terminal devices to access the I/O system.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 33
IB Layered Architecture

Application Layer

Transmission Layer

Network Layer

Link Layer

Physical Layer

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 34
IB Architecture (IBA)

 Components of InfiniBand Architecture(IBA):

 Node: Host Channel Adapter (HCA), Host Target Adapter(HTA).
 Network: Switch, Routers.
 Physical ： Links(Fiber or Cables), Repeaters.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 35
IB Interface
• Channel Adapter (CA) is divided into 2
different types as below: IB Switches:
• Host Channel Adapter (HCA): It is used
at the hosts side such as Mellanox HCA
products.
• Target Channel Adapter (TCA): It is
used mainly on the IB switches side or
serve as the interface on the storage
systems. IB Connectors:

Host Channel Adapter (HCA):

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 36
IB Signaling Mode

Transmission
SDR DDR QDR FDR EDR
Rate (Gbit/s)

1X 2.5 5 10 14 26
4X 10 20 40 56 104
8X 20 40 80 112 208
12X 30 60 120 168 312
 SDR ： single data rate
 DDR ： double data rate
 QDR ： quad data rate
 FDR ： fourteen data rate
EDR ： enhanced data rate
 HDR : High Data Rate
NDR : Next Data Rate

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 37
CIFS/NFS In Storage Systems
 The two most common network sharing protocols for NAS are: CIFS and NFS.
 CIFS (Common Internet File System): CIFS refers to the collective name of SMB (Server Message Block).
Network file sharing between Windows hosts is achieved by using Microsoft's own CIFS service.
 NFS (Network File System) is a network file system that uses NFS extensively for cloud computing and
databases. UNIX-like operating systems such as Linux/UNIX/AIX/HP-UX/Mac OS X uses NFS to provide
network file system storage services.

Host Storage

CIFS/NFS client
IP
Network

CIFS/NFS server

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 38
Working Principle of CIFS
Client End Server

SMB2 NEGOTIATE Request

Protocol Protocol Handshake
SMB2 NEGOTIATE Response Negotiation

SMB2 SESSION_SETUP Request

Session Security Authentication
SMB2 SESSION_SETUP Response Setup

SMB2 TREECONNECT Request

Tree Share Connection
SMB2 TREECONNECT Response Connection
……

Network File File Operation

Operation

SMB2 TREE_DISCONNECT Request Tree

Break Connection
SMB2 TREE_DISCONNECT Response Disconnection

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 39
NFS Working Principle

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 40
CIFS Typical Application Case - File
Sharing Service
 File sharing service scenarios applies to enterprise file servers, media assets, and other
application scenarios.
Office

IP Management
IP Windows IP
Performance Service Sharing User
IP IP IP Monitoring Management Management Management

NAS Service

LAN
DNS

Local
Users
AD Servers
Domain
Users Authentication Flow
Management Flow
Data Flow

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 41
NFS Typical Application Case:Shared
Storage In Cloud Computing
 Cloud Computing uses NFS servers as its internal shared storage:

Cloud Server Unified Storage

NFS client Internal
External IP
Firewall

IP Network NFS server

Network Cloud Server

Cloud Server

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 42
What is FTP ?
 FTP (File Transfer Protocol) is used to transfer files between a remote server
and a local host. It is a common protocol for transferring files on an IP network.
 The FTP protocol is an application layer protocol in the TCP/IP protocol suite.
It is used to transfer files between a remote server and a local client, and uses
TCP ports 20 and 21 for transmission. Port 20 is used to transmit data, and port
21 is used to transmit control messages. The basic operation of the FTP protocol
is described in RFC959.
 FTP works in two different modes:
 Active mode (PORT): The FTP server initiates a connection request when a data connection is established. This is not
applicable when the FTP client is in the firewall (for example, the FTP client is on a private network).
 Passive mode (PASV): The FTP client initiates a connection request when establishing a data connection. This is not
applicable when the FTP server restricts the client from connecting to its upper port (generally greater than 1024).

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 43
What is HTTP ?
 Hypertext transfer protocol (HTTP) is a data transfer protocol that specifies the rules for
the communication between the browser and the web server, and transmission of web
document via the Internet.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 44
FTP Typical Application Scenario - File
Uploads and Downloads

IP Network
FTP Client

FTP Server

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 45
HTTP Typical Application Scenario - Web
Access

WebDAV Clients
IP Network

Web Server
WebDAV is the extension of the
HTTP 1.0 protocol which allows
multiple users to edit and manage
files collaboratively.

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 46
 Thank You
www.huawei.com

Copyright © 2018 Huawei Technologies Co., Ltd. All rights reserved. Page 47