Object Storage

Content

1 Overview

2 Features

3 Application Scenarios

4 Products

2 Huawei Confidential
Storage Trends: Unstructured Data Surge
500
44 ZB Users: > 800 million
80%: unstructured data DAU: > 300 million
375
MAU: > 1.5 billion

250
• Videos require quick storage and retrieval
• There has been an explosion in users, videos, and
capacity
125
80% • Videos require tagging, categorization, and analytics
• User preferences are created from usage history for
customized video suggestions
• The most popular topics will be updated dynamically
20%
• Data is accessed with mobile apps
2009 2010 2011 2012 2013 2014 2015 2017 2017 2018 2019 2020

IDC report: 80% of the projected global volume of 44 ZB in Traditional NAS storage provides inadequate
2020 will be unstructured data performance, expansion, and retrieval. ByteDance has
 Unstructured data: Image, video, audio, radar, and sensor developed object storage for TikTok videos.
data
 Device connections: 10% of data
 User data analytics: 35% of data

3 Huawei Confidential
 Object Storage Harnesses Internet and Mass Data
Limitless scalability
Unstructured data surge • Storage for tens of billions of objects
• Linear expansion on demand for
capacity and performance
• Storage must provide large capacity and
scaling on demand for sudden growth
• Unstructured data must have metadata for
marking, classification, and analytics Customized attributes
• Identified by unique object IDs
• User-defined attribute fields

Object Object

Interactive online access Universal access

• Online device access anytime and
anywhere
Object storage
• Data centers must support access across
regions
• File systems (NAS) are designed for
human-machine interaction, making M2M
interactions, such as data sharing and
management, difficult Flexible connection
• REST API, HTTP, HTTPS

4 Huawei Confidential
Object Storage Surpasses NAS for Mass Data
NAS storage Object storage
Server

APP APP

Access
Shared directory Direct access
• Servers • Support for different
Directory • App access through Web IoT Mobil
devices
shared directory e

Protocols
NFS/CIFS/POSIX CIFS/NFS
• Complex protocols
HTTP/HTTPS HTTP/HTTPS (RESTful API)
• URL address
• Incompatibility • Simple flexibility
with different OSs

Networks
Interne
LAN Private Public/Private t
• Private connection
• Public/Private connection
• No Internet access
• Unrestricted networks

Bucket Bucket Bucket

Organization
Tree structure Flat structure
• Retrieval performance Object
• Simple file
of files or directory
directories
impacted by growth
• Better scalability

NAS storage: Local file directory Object storage: Function

5 Huawei Confidential
What Is Object Storage?
Object storage simplifies management and reduces costs by eliminating hierarchies for
unlimited cloud storage of unstructured data.
OID
System
Data Object metadat
a
User
Bucket Bucket Bucket
metadata

Item Description

Object Identifier
Object names
(OID)
• Data is stored as objects
System Object attributes, such as object size, access permissions,
• Data is in a flat address space metadata and creation time
• Bucket/Object
• Easy scalability User metadata Additional object information about applications

Data Service data such as text, images, or video

6 Huawei Confidential
Objects with Data and Metadata
File (Data)
 X-ray
User metadata
System metadata (object attributes)
 Filename: CA12345.JPG
 Created: June 2, 2009
System metadata  Last modified: July 4, 2009

User metadata (user-defined attributes)

 Name: XXXXX
File (Data) 

Category: Chest X-ray
Save time: XX years
 Doctor: Dr. XX
 Result: No abnormalities were found.

7 Huawei Confidential
Object
 History of Object Storage

1990s: Content addressable storage (CAS)

Storage to manage mass data

2006: Amazon releases AWS (S3)

Object storage with improved
Internet access
Industry standard

2012: Huawei releases OceanStor UDS

The first object storage from
Huawei

8 Huawei Confidential
Distributed Architecture Provides Superior Implementation

Superior distributed architecture provides object storage  Distributed storage

without additional gateways.
Object mapping for file systems is
eliminated to shorten access latency.

Gateway architecture Distributed architecture  Concurrent access

Distributed storage improves over
object gateways and disk array
Object gateway architecture by making the network
Object APIs fully redundant and eliminating single
points of failure for better concurrent
access.
Object APIs

File system  Scale-out

Capacity and performance increase
linearly with nodes, which are
Object storage unrestricted by file system size.
Disk array

9 Huawei Confidential
Amazon S3 Sets Mainstream Standard for APIs

 Amazon S3 is an open  OpenStack Swift is an object  CDMI is a cloud storage

protocol of Amazon AWS for storage API for OpenStack API from the Storage
object storage. It is the cloud computing Network Industry
industry standard for object Association (SNIA)
storage

10 Huawei Confidential
RESTful API for Object Storage

GET PUT
HTTP
POST DELETE
Clien Object storage
t
• Each Uniform Resource Identifier (URI) represents a resource, such as text, images,
songs, or services

• Clients operate the resource on the server side with four HTTP verbs:
 GET: Get a resource
 POST: Create or update a resource
 PUT: Update a resource
 DELETE: Delete a resource

11 Huawei Confidential
AWS S3 Provides the Standard Protocol for Object Storage

• Amazon S3 is an open protocol of Amazon Tenan

User
AWS for object storage t

• It is the industry standard for object storage

Bucket Bucket
• S3 simplifies the complex directory structure
of the file system with the bucket/object
Objec … Objec … Objec
model Object
t t t
• HTTP/HTTPS RESTful interfaces

Concept Description

Bucket A container for object storage which eliminates nesting and thus container hierarchies

Object Object data and metadata which are always stored in a bucket

12 Huawei Confidential
OpenStack Swift Parallels AWS S3
Account
 Swift is a distributed module of the cloud OS User 1
OpenStack. User 2
 Model with account, container, and object layers. A
container is equivalent to a bucket in AWS S3 Containe Containe
 HTTP/HTTPS RESTful interfaces r r
Objec … Objec … Objec
Object
t t t
Concept Description
The top level of the hierarchy. Service providers create accounts to establish user ownership of resources. The account defines a
Account namespace for containers. A container might have the same name in two different accounts.

Container Namespace creation for objects. An object with the same name in two different containers represents two different objects.

Object Content storage such as documents and images, as well as custom metadata

13 Huawei Confidential
 AWS S3 API: Requests

Format Example

PUT
object

GET
object

14 Huawei Confidential
Object Storage Interface Requires Customized Software or Tools
Usage of object storage APIs require software developed by users or an existing tool, such as
the S3 Browser or backup software.

Service host
Invokes Web
services
APP
User-developed Bucket
software Invokes local Bucket
APIs
Object
eSDK Object
Object

User Object Object Object

Object Data
Key

User-defined Metadata
metadata

Object storage

15 Huawei Confidential
Practice Accessing Object Storage with Tools
Exercise: Access an object of OceanStor 100D or the object storage of Huawei Cloud with the S3 Browser or online
tools.
1. Create a bucket in Huawei Cloud. 2. Put an object in the bucket.

3. View the bucket list.

4. View the object list.

16 Huawei Confidential
Content

1 Overview

2 Features

3 Application Scenarios

4 Products

17 Huawei Confidential
Global Namespace: Storage Access with Unified Domain Name
Object storage supports multiple regions and Availability Zones (AZs). Storage resources in different regions require
virtualization into a global namespace for domain name resolution, location services (LSs), and load balancing.

Global
Namespace

Region Region Region

1 2 N

LAN/WAN
AZ 1 AZ 2 AZ_N

Object
Cluster 1 Cluster 2 Cluster 3 Cluster Object
Object-based storage Object
Node 1 Node 1 Node 1
NNode 1 Object
Object
resource pool Object

Node 2 Node 2 Node 2 Node 2

…… …… …… ……
City A City B City C City D
Node N Node N Node N Node N

18 Huawei Confidential
Cross-Region Replication for Disaster Recovery

Cluster C

Cluster B
Bucket 3
Asynchronous data synchronization
Region 1 Region 2
Object storage
25/10GE
25/10GE
Bucket 1 Bucket 2
Cluster A Bucket 2
Object storage
… …
… …
Cluster A Cluster B
Bucket 1

Object storage Object storage

• The system asynchronously copies objects in bucket 1 of • Different regional clusters support asynchronous
a region to bucket 2 in another region using policies in replication with one-to-one or many-to-one
the global namespace. delivery for remote DR and local access.
• Cross-region transmission uses SSL encryption to
protect data security.

19 Huawei Confidential
Erasure Coding for Higher Disk Utilization

Distributed error tolerance with the highest utilization

Erasure coding (EC) provides cloud storage with
encoding and error tolerance. It replaces multiple
copies with data redundancy verification for cloud
storage pools to equal or raise reliability while
lowering costs with advanced technology
Data
fragments
Redundant
fragments
PP PP PP PP

• Disk utilization improves from 33% in three-copy

model to 80% in EC model, reducing hardware costs
and improving disk utilization by 40%
• Data availability can reach 99.9999% with different
hardware
... • Enterprise products for cloud storage generally
provide EC

20 Huawei Confidential
Multi-Tenancy Isolates Tenant Storage Data
Multi-tenancy isolates tenants so that they always have access to the specified performance levels and
capacity despite changing conditions.
Individual tenant data is isolated and remains invisible to other tenants.

Tenant Tenant B Tenant C

A
 Single system: Multi-client services reduce
initial investment.

 Online storage: One system is centrally

managed with logically isolated data.

 Transmission encryption: HTTPS encryption

Object storage and user authentication are supported to
Data center
protect data transmission.

21 Huawei Confidential
 Quotas Limit Occupied Resources
Quotas limit the resources occupied by buckets and tenants.

Object storage
100 TB
Quotas
• Buckets: Data writes to a bucket stop after it reaches its
maximum capacity.
• Tenants: Data writes to an account stop after all of its
Tenant Tenant Tenant buckets reach maximum capacity.
1 2Financial 3
General Personnel
20 TB manager
40 TB dept. 40 TB dept.
Resource statistics
• Buckets: Bucket size and the number of objects in the
10 TB bucket. Users can query their bucket resources.
• Accounts: Account quota, the number of buckets/objects,
and total capacity. Accounts support queries of account
resources.

2 TB 10 TB 8 TB 5 TB Notify
Employee A Employee B Employee C Employee D • Occupied resources generate alarms or events.
Bucket 1 Bucket 2 Bucket 3 Bucket 4
10 TB

22 Huawei Confidential
API Billing by Operation or Capacity

Billing services are mandatory for object storage and you only
pay for what you use.
Billing center
Object storage generally provides open APIs, such as SFTP,
which upper-layer billing uses to access object storage and
obtain data for fee calculation.

 There are two types of bills:

 Detailed records save the operations of end users, such
API Billing APIs as object downloads/uploads or bucket
creation/deletion. Recorded attributes include the
Object storage operation type and data capacity.
 Inbound bills record the resource utilization of end
users. Object storage saves a capacity record for each
uploaded object in the inbound bill file.

23 Huawei Confidential
Content

1 Overview

2 Features

3 Application Scenarios

4 Products

24 Huawei Confidential
Massive Resource Pools

Cloud
Cloud
Email Video backup
disks Cloud archiving Advantages

Seamless capacity expansion: Unlimited scalability is

Video available on demand with linear performance
surveillance Storage leasing
Music Images TV increases to match capacity.

Reliability: The EC algorithm provides high reliability

with redundancy, transmission encryption, and E2E
Multi-tenancy
data security.

Cross-site access: Data is remotely accessible.

Efficiency: M2M data management and storage

Object storage
support artificial intelligence and analytics to reduce
the cost and length of the design process.

25 Huawei Confidential
Backup and Archiving
Production Office
Online data

Advantages
Backup/Archive Backup/Archive
data data

Mass storage space with backup software creates

Application 1 Backup media server diverse backup solutions for mass data.

Resource allocation on demand saves space.

Production Office
Performance improves to match capacity
Online data
Object storage expansion and prevent performance bottlenecks
from increasing data.

Backup/Archive
Enhanced reliability is achieved with cluster
Backup/Archive
data data deployment, EC, and automatic fault detection and
recovery.

Application 2 Backup media server

26 Huawei Confidential
Content

1 Overview

2 Features

3 Application Scenario

4 Products

27 Huawei Confidential
Huawei OceanStor 100D: Fully Distributed System for Object,
Block, File, and HDFS Storage

Traditional
New apps
apps
Reliable, Elastic, Open

OpenStack

iSCSI SCSI NFS CIFS FTP HDFS NDMP S3/Swift

 Block, file, HDFS and object storage

Block File Object on demand
 Fully distributed architecture
OceanStor 100D  Versatile hardware platform
 4,096 nodes for storage at the
Versatile hardware platform exabyte-level

28 Huawei Confidential
Thank you. Bring digital to every person, home, and
organization for a fully connected,
intelligent world.

Copyright © 2020 Huawei Technologies Co., Ltd.

All Rights Reserved.

The information in this document may contain predictive

statements including, without limitation, statements regarding
the future financial and operating results, future product
portfolio, new technology, etc. There are a number of factors that
could cause actual results and developments to differ materially
from those expressed or implied in the predictive statements.
Therefore, such information is provided for reference purpose
only and constitutes neither an offer nor an acceptance. Huawei
may change the information at any time without notice.