INFINIBOX 4.0.

10
USER DOCUMENTATION

Last updated: 06/12/2018
 InfiniBox 4.0.10 User Documentation

Table of Contents
1 Introduction ..................................................................................................................3
2 Getting started ............................................................................................................15
3 Compression ...............................................................................................................45
4 Authentication and security .......................................................................................54
5 iSCSI connectivity .......................................................................................................76
6 Hosts and clusters.......................................................................................................83
7 Network connectivity and configuration.................................................................101
8 Hardware monitoring ...............................................................................................136
9 Pools ..........................................................................................................................148
10 Provisioning volumes ...............................................................................................160
11 Provisioning filesystems ...........................................................................................184
12 Consistency groups...................................................................................................228
13 Link ............................................................................................................................260
14 Replication ................................................................................................................268
15 Encryption at-rest .....................................................................................................306
16 InfiniSpares ...............................................................................................................308
17 Event notification......................................................................................................312
18 GUI and InfiniShell software upgrade......................................................................321
19 Metadata ...................................................................................................................323
20 Performance monitoring ..........................................................................................324
21 Quality of service.......................................................................................................362

 –  2
 InfiniBox 4.0.10 User Documentation

1 Introduction
INFINIDAT features high-end, hyper-scale storage with highly-dense capacity, high performance, and
infinite use cases. INFINIDAT offers you an environment-friendly system with low power consumption,
low CapEx & OpEx and unmatched TCO.

With more than 90 patent applications (many of which are approved), INFINIDAT is a global leader in the
areas of data protection, performance, automation, energy savings, ease of management, and scaling.

1.1 Why InfiniBox

Data grows faster than IT budget. It is not the production data that grows so fast, but rather the data
copies that support the production.

Along with the capital cost of usable capacity, the operational costs increase. Power consumption, floor
space and human capital are all part of the costs of owning and managing a storage system.

Costs are also measured by putting a price tag on how lengthy is the task of provisioning capacity or
creating snapshots; how many storage administrators are needed; what percentage of the
administration tasks can be automated; what is the cost of human mistake; are we safe from downtime;
and so on.

INFINIDAT offers a new paradigm in the storage – a paradigm that breaks the traditional connection
between storage capacity and the cost associated with it.

1.1.1 The challenges at hand

The ever-increasing demand for capacity arises from various aspects of the organization's needs. 

Backup - Local backup copies allow for recovering from data loss and logical data corruption
while remote backup copies are used for disaster recovery, audit and regulatory compliance.
Provisioning - Most organizations provision capacity in thick format. That is, the storage space is
purchased and allocated prior to the actual use. Organizations that have already moved to thin
provisioning (assessing the actual storage needs prior to purchasing it) find themselves in an
increasing risk of downtime, resulting from a lack of tools for capacity planning.
Data copies - Full copies of the production data consume lots of expensive storage capacity, as
well as computational resources that are required for moving them around.

The fast data growth caused by virtualization, increased use of BI and other data-intensive applications
has turned storage into the biggest IT spending (around 25% according to studies). 

• RAID - Most companies use RAID-1 or 5 for protecting their production data from a single failure
for their critical data. And RAID-6 for large drives to provide protection against most dual failures
but avoid using it for critical, due to its performance penalty.

Introduction  –  3
 InfiniBox 4.0.10 User Documentation

• SAS drives - High-performance drives are expensive, and provide smaller capacities. A
combination of SATA drives and flash-based SSD drives is both inexpensive and high in
performance. However, this combination's efficiency depends on the chosen RAID layout and
their ability to manage which data needs to be in which storage tier (pre-fetching).
• Data center - The bigger the data center, the bigger the power and cooling costs, as well as the
cost of floor space. The latter costs have skyrocketed in recent years.

This document explains INFINIDAT’s unique approach to solving all these challenges in a single, robust
storage architecture. 

1.2 InfiniBox terminology

• Capacity
• Pools 
• Volumes 
• Filesystems
• Consistency groups
• Network spaces
• Replication

1.2.1 Capacity

Term Meaning

System physical capacity The total physical capacity that can be allocated
within a single system.

System virtual capacity The maximum system capacity that can be defined
by pools. This capacity can be larger than the system
physical capacity.

System free virtual capacity The system virtual capacity minus the sum of the
pools virtual capacity.

Introduction  –  4
 InfiniBox 4.0.10 User Documentation

1.2.2 Pools 

Term Meaning

Pool physical capacity The maximum usable net capacity that can be
allocated to the pool entities.

Pool virtual capacity The limit on the size of all volumes (thin and thick
provisioned), excluding snapshots.

Pool allocated virtual space The sum of all volumes (thick and thin, excluding thin
snapshots). The allocated space cannot exceed the
pool virtual capacity.

Pool allocated physical space The sum of all thick volumes, plus the physical space
consumed by all the thin volumes and
snapshots. The used space cannot exceed the pool
physical capacity.

Pool free physical space The defined pool physical capacity minus the
allocated physical space.

Pool free virtual space The defined pool virtual capacity minus the allocated
virtual space.

Reserved space Automatically defined spare physical space for the

pool, to support pool physical space depletion
behavior.

Emergency buffer A policy for automatically extending of the pool

physical capacity in the event of pool physical space
depletion.

Limited state Upon further accepting writes, the pool enters a

Limited state where no new volumes can be created,
but the existing volumes can be written into.

Locked state If the pool can no longer expand, it enters a Locked

state and its entities can no longer accept writes.

Introduction  –  5
 InfiniBox 4.0.10 User Documentation

1.2.3 Volumes 

Term Meaning

Volume A set of consecutively addressed disk blocks that are

presented to the storage controller as a range of
consecutively numbered logical blocks with disk-like
storage and I/O semantics.

The InfiniBox volume accepts the following

attributes: capacity (size, used capacity, tree
allocation), thin provisioning, mapping to a host,
write protection, ability to use the system's SSD
drives.

Thick volume Reserved-space volume. Virtual and Physical space

allocation is done immediately as Thick Volume is
created, at the amount of the volume size.

Thin volume Shared-space volume. Thin volumes consume

physical space upon writes. The virtual space is
allocated upon volume creation to the amount of the
volume size.
The initially allocated physical space is 0 (zero). In a
case of space depletion, the allocation is not
guaranteed for thin volumes.

1.2.4 Filesystems

Term Meaning

Filesystem The filesystem controls the way data is stored on the

storage system by providing a name to each piece of
data (file). The file system allows to search for and
access the files.

1.2.5 Consistency groups

Term Meaning

Consistency Group (CG) Container of volumes

Introduction  –  6
 InfiniBox 4.0.10 User Documentation

Term Meaning

Group-consistent snapshot Snapshot operation that is performed on the

consistency group entities which ensures that write-
order-fidelity is maintained across all the objects in
the group

Snapshot Group (SG) Container of snapshots created through a group-

consistent snapshot

Replica consistency group Consistency group that belongs to a replica (either as

a source or as a target)

1.2.6 Network spaces

Term Meaning

WWPN World Wide Port Name. A globally unique ID for each

FC port. The equivalent of MAC address in Fiber
Channel.

WWNN World Wide Node Name. A globally unique ID for a

system / group of WWPNs.

Multipath driver Host-side driver that allows the host to leverage

more than 1 path to a LUN.

Node An InfiniBox server. Typically, there are 3 nodes in a

system.

Ethernet Port (Port) Physical Ethernet port (copper / optical).  

Ethernet Interface One or more Ethernet Ports that are grouped

together for the purpose of redundancy and/or
performance using LACP.

Service A network data service, e.g. NFS, iSCSI, Replication

IP address A group of floating IP addresses that will offer the

service (Each IP offers 1 service). It is recommended
that each Network Space will have 6 IP Addresses to
ensure even workload distribution.

Introduction  –  7
 InfiniBox 4.0.10 User Documentation

Term Meaning

Network A set of definitions for a network address space,

annotated using CIDR. Example: 10.0.0.0/24.

1.2.7 Replication

Source The InfiniBox system whose volume is replicated.

system

Target The InfiniBox system that the volume is replicated to.

system

Source A volume on the source system that is replicated (or planned to be replicated).
dataset

Target A volume on the target system that is the destination of the replicated data. A target volume
dataset that is part of a replica cannot accept host writes.

Replica An entity that matches a pair of source and target volumes and other settings essential for the
replication.

Local The InfiniBox system through which the user manages the replica.

Remote The InfiniBox system at the other end of the replica.

Both local and remote system may contain source and target datasets.

Link A bidirectional means of communication between local and remote systems.

The link is defined via Management.

Sync job The process of creating a snapshot of the source and delivering it to the target.

Sync The amount of time between two planned sync jobs.

interval

Introduction  –  8
 InfiniBox 4.0.10 User Documentation

1.3 How does InfiniBox work

• InfiniRAID
• Host Power Tools
• Multi-Protocol

INFINIDAT Storage System has been designed in order to meet the following goals:

• Cost - The cost of usable capacity, power consumption, cooling, floor space, human capital are all
part of the costs of owning and managing a storage system. Length and complexity of the task of
provisioning capacity or creating snapshots, cost of storage administrators, cost of manual labor
that could be automated, cost of  human error
• Reliability -  Multiple data protection layers
• Performance - Balance is loaded across all hardware components to achieve optimal utilization
• Redundancy - 3 Nodes in Active-Active-Active configuration, each Node has constant access to all
of the drives
• Availability - 
• No single point of failure
• Protection against double disk failure
• Fast rebuild in case of drive failure

1.3.1 InfiniRAID
As part of the process of destaging data from system cache to disk drives, every 64KB of data has an
added 4KB for metadata that is written in a RAID6-like or Dual Parity data protection scheme. Full
stripes of 1088KB (using 68KB stripe members) are spread across all the 480 disks within the system,
creating hundreds of thousands of independent RAID-Groups; and which provide key architectural
benefits for both performance and support for fast recovery from single or double disk failures.

If a drive fails, InfiniRAID immediately rebuilds the data into the spare capacity in the system, carefully
distributing the data across all the remaining disks, relieving the administrator from the need to
physically replace the disk for the process to complete.

Resuming protection after a disk failure is achieved so quickly as the failed disk was a member of 1000's
of independent RAID-groups, each spread over different disks. This means that all the disks participate
in the recovery process, minimizing the impact on each disk

Introduction  –  9
 InfiniBox 4.0.10 User Documentation

1.3.2 Host Power Tools

Host PowerToolsTM provides both storage and application administrators with the newest set of storage
management tools. Host PowerToolsTM helps the storage administrators by automating the mundane,
manual, time-consuming tasks, thus enabling the storage administrator to focus on other tasks, rather
than maintaining infrastructure.

INFINIDAT Host PowerToolsTM for VMware is an application that provides the VMware
administrator with provisioning capabilities by assisting and easing the process of volumes backup and
recovery.

1.3.3 Multi-Protocol
In an effort to reduce the storage sprawl, InfiniBox was designed to support multiple protocols in a
single system. The internal data structures allowing InfiniBox to scale to petabytes were designed to
allow storing both SAN volumes, NAS filesystems and object storage (RESTful API). As additional
protocols are added to InfiniBox stack, the focus on keeping the feature-set identical across all
protocols is maintained. This allows customers to choose the right protocol for their need without
compromising on functionality.

1.4 Scalable InfiniBox layouts

• Available layouts
• Common components
• Capacity
• Rack layout
• InfiniBox power feed
• Power Distribution Unit (PDU)
• PDU to ATS
• ATS to BBU
• PDU to Enclosure(s)
• BBU to Nodes
• Enclosures and drives
• BBUs
• Drive visibility
• The patch panel
• The Service Drawer
• Hardware terminology

Introduction  –  10
 InfiniBox 4.0.10 User Documentation

InfiniBox features off-the-shelf hardware components, allowing for a quick adoption of newer, faster
hardware components.

1.4.1 Available layouts

Common components
Each InfiniBox layout contains an array of 3 storage controllers (nodes), 3 BBUs and a service drawer.
The storage controllers are connected to each other via an InfiniBand network. Each storage controller
is equipped with RAM, Flash Cache and internal disk drives in various configurations.

Capacity
The InfiniBox layouts are equipped with 1,2,4 or 8 drive enclosures, each contains 60 disk drives. The
available sizes are 3,4,6 or 8 TB. InfiniBox offers a variety of capacity options, starting with a single
enclosure of 60 disk drives at the disk size of 3 TB, all the way up to 8 enclosures of 60 disk drives each,
at the disk size of 8 TB.

Rack layout
The following image displays the various InfiniBox rack layouts. 

Introduction  –  11
 InfiniBox 4.0.10 User Documentation

1.4.2 InfiniBox power feed

Power Distribution Unit (PDU)

The InfiniBox PDUs receive power from the data center in one of the following ways:

1. 2 PDU on each side of InfiniBox, that are fed from above the InfiniBox.

2. 2 PDU on each side that are fed from the floor.
3. 1 PDU on each side, that is fed from above, the PDUs are 3PH.

PDU to ATS
The PDUs feed the ATSs in either of the following configurations:

• PDU - 4 * 1PH / 2 * 3PH

• ATS - 3
• Connectivity 
• 2 PDUs connected to 1 ATS
• 2 PDUs connected to the other 2 ATS (each PDU is connected to both ATS)

ATS to BBU
Each of the ATS feeds each of the BBUs (a single ATS feeds a single BBU).

• ATS - 3
• BBU - 3
• Connectivity - An ATS per BBU

PDU to Enclosure(s)
Each enclosure is fed from 2 different PDUs.

BBU to Nodes
Each BBU feeds two Nodes. Each of the nodes is fed by 2 BBUs. Thus, in case of a single BBU failure, all
3 nodes remain acitve:

• 2 BBUs keep 3 Nodes active.

• 1 BBU keeps 2 Nodes active.

Introduction  –  12
 InfiniBox 4.0.10 User Documentation

Enclosures and drives

1,2,4 or 8 Enclosures, each contains 60 drives:

• Ultra-dense 4U
• 60 NL-SAS HDDs, hot-pluggable
• 2 x hot-pluggable, redundant IO controllers equipped with redundant 6Gbps, 2.0 SAS
connections
• 2 x hot-pluggable, redundant power and cooling modules
• Two independent AC power inlets

BBUs
Support the Nodes during power loss, allowing enough time for the notes to gracefully shutdown

• Provide power loss enough time for the Nodes to shut down gracefully in an event where the two
power sources are lost
• When power returns, the system is powered on automatically

Drive visibility
Each node has direct access to all drives.

1.4.3 The patch panel

The InfiniBox patch panel is located on the rear side of the rack. The patch panel layout includes 3 sets
of cable ports, one for each node.

Each of the nodes may include:

• 8 FC ports
• 4 Ethernet ports (copper or optics)
• 1 management port for the InfiniBox Management Suit
• Port 15 for connecting to the Remote Support appliance 

Introduction  –  13
 InfiniBox 4.0.10 User Documentation

1.4.4 The Service Drawer

The Service Drawer contains the following components:

• ATS
• Support Appliance (on a fit-PC)
• Internal system Ethernet switch

1.4.5 Hardware terminology

• PDU - Power Distribution Unit. An electricity appliance that receives power from the data center
and provides multiple power outputs to feed the InfiniBox appliances. 
• ATS - Automatic Transfer Switch. An electrical switch that selects the power source between two
PDUs to feed the BBU in such a way that each of the BBU is fed from redundant power sources.
The ATS is located within the Utility Drawer.
• BBU - Backup Battery Unit. Provides electrical power in case no power arrives from the data
center, allowing InfiniBox to gracefully shutdown and protect the user data.
• Node - A Storage Controller. The InfiniBox component that receives IO from the hosts and
controls the way they are stored. InfiniBox has 3 Nodes in Active-Active-Active formation for
maximum redundancy.
Nodes are fed from the BBU. Nodes may include SSD drives for Read Cache.
• Enclosure - A fully redundant storage enclosure that contains 60 drives.
The Enclosures are power consumers. Enclosures are fed from redundant PDUs.

1.5 InfiniBox rack safety notices

Although the InfiniBox rack is shipped and installed for you, prior to accepting the InfiniBox rack, please
read the attached Safety Notices.

• InfiniBox Safety Notices

Introduction  –  14
 InfiniBox 4.0.10 User Documentation

2 Getting started

2.1 Accessing the InfiniBox GUI

2.1.1 Accessing the InfiniBox Management Console

The InfiniBox Management Console (GUI) is a web application that is accessed from any browser. It
allows for accessing  (connecting to) InfiniBox.

2.1.2 Accessing the InfiniBox GUI

1. Access your InfiniBox from the following URL:
<InfiniBox>.<your-site>.com
The InfiniBox GUI opens on screen.

2.1.3 Finding your way around the GUI

The GUI screen has a menu on the left.

Dashboard view
In the image below, the GUI is opened on the Dashboard view. The Dashboard displays metrics on:

• The current IOPS and throughput

• System health
• Number of storage entities (pools, volumes and filesystems, hosts and clusters and replicas)
• Capacity
• Latest events

Getting started  –  15

 InfiniBox 4.0.10 User Documentation

Other views
Read on all other GUI view in the following help pages:

• Querying for pools and their properties

• Querying for volumes and their properties
• Creating a filesystem
• Creating a replica

2.2 Accessing the InfiniShell Console

2.2.1 About this task

The InfiniBox Management Console provides both Graphic and command-line UI for managing the
InfiniBox.

The command-line UI is called InfiniShell. It allows for accessing  (connecting to) InfiniBox.

2.2.2 Steps  
1. Select InfiniShell from the menu at the top right of the screen.

 
The InfiniShell console opens, displaying the command prompt:
admin@localhost>

Getting started  –  16

 InfiniBox 4.0.10 User Documentation

Alternately, you can access the InfiniShell console from the following URL:

<InfiniBox>.<your-site>.com /#/infinishell

2.3 Accessing InfiniBox via HTTPS only

• About this task
• GUI Instructions
• InfiniShell Instructions 
• Returned HTTP code

2.3.1 About this task

InfiniBox can be set to allow access via HTTPS only.

Upon setting the InfiniBox to allow access via HTTPS only, all access requests are redirected to SSL. The
setting takes place during runtime and is persistent across reboot and hot upgrade.

2.3.2 GUI Instructions

1. Click the Settings icon on the navigation tree. On the General tab, slide the Always redirect
to HTTPS slider.
2. Click Update.

2.3.3 InfiniShell Instructions 

1. To set InfiniBox to be accessed via SSL only, run the following infinishell command: 
config.system.set_ssl_redirection.

2.3.4 Returned HTTP code

The redirection returns HTTP code 307.

2.4 Command auto-completion

• Querying for a detailed output
• Controlling the columns of the displayed query output
• Sorting the query output

Getting started  –  17

 InfiniBox 4.0.10 User Documentation

• Displaying the output in CSV format

• Displaying the output in JSON format
• Querying for any string
• Querying for a single string
• Querying for more than one string
• Approving a warning message
• GUI shortcuts

InfiniShell provides on-line auto-completion help as-you-type.

2.4.1 Typing a command name + TAB

Type a part of the command name (and press TAB)

Type the following string

po
The string auto completes to
The first part of the command name, followed by a dot (.)
pool.
Meaning: all of the commands that start with "pool."

Type the command initial part (and press TAB)

Type the following string

pool.
The string auto completes to
All of the commands that start with "pool.":
Commands:
cache delete query resize unlock
create lock rename set_thresholds user_query

Type a command name (and press TAB)

Type
pool.create
The string auto completes to
All of the parameters that the command pool.create accepts.
• Mandatory parameters are not bracketed.
• Optional parameters are bracketed.
name physical_capacity [virtual_capacity] [ssd_cache] [auto_extend]

Getting started  –  18

 InfiniBox 4.0.10 User Documentation

Typing a query command + TAB

Type
pool.query
The string auto completes to
[name] [unit] [--detailed] [--columns] [--sort] [--format]

Querying for a detailed output

Displays the query results with all of the queried object’s fields:

pool.query pool=1855 --detailed

Name 1855
State NORMAL
Physical Total 20.00 TB
Physical Allocated 2.19 TB
Physical Free 17.80 TB
Virtual Total 20.00 TB
Virtual Allocated 2.19 TB
Virtual Free 17.80 TB
Emergency Buffer UNLIMITED
Used Emergency Buffer -
Warning Threshold 80%
Critical Threshold 90%
Volumes QoS Policy -
Filesystems QoS Policy -
Compressed yes
Data Reduction 1.00 : 1
SSD Cache Default yes

2.4.2 Controlling the columns of the displayed query output

Displays a list of attributes that can be included as columns in the query result:
pool.query --columns=

name, state, physical_total, physical_allocated, physical_free,

virtual_total, virtual_allocated, virtual_free, auto_extend,
used_emergency_buffer, warning_threshold, critical_threshold or
ssd_cache_default (multiple values, separated by commas)

2.4.3 Sorting the query output

Displays a list of attributes that the query can be sorted by:

Getting started  –  19

 InfiniBox 4.0.10 User Documentation

vol.query --columns=name,pool --sort=    

Names of sortable columns: e.g, size: sort by ascending order, -size: sort by descending order.
name, serial, size, wp, mapped, ssd_cache, created_at, updated_at, -name, -serial, -size, -wp,
-mapped, -ssd_cache, -created_at or -updated_at (multiple values, separated by commas)
-created_at -name -size -updated_at created_at name size updated_at
-mapped -serial -ssd_cache -wp mapped serial ssd_cache wp

Another example:
pool.query --sort=size

NAME STATE PHYSICAL TOTAL PHYSICAL ALLOCATED VIRTUAL TOTAL VIRTUAL ALLOCATED AUTO EXTEND
pool1 NORMAL 7.00 TB 0 7.00 TB 0 UNLIMITED
pool2 NORMAL 6.00 TB 0 6.00 TB 0 UNLIMITED
pool3 NORMAL 5.00 TB 0 5.00 TB 0 UNLIMITED

2.4.4 Displaying the output in CSV format

pool.query --format=csv

NAME,STATE,PHYSICAL TOTAL,PHYSICAL ALLOCATED,VIRTUAL TOTAL,VIRTUAL ALLOCATED,AUTO EXTEND

pool1,NORMAL,7.00 TB,0,7.00 TB,0,UNLIMITED
pool2,NORMAL,6.00 TB,0,6.00 TB,0,UNLIMITED
pool3,NORMAL,5.00 TB,0,5.00 TB,0,UNLIMITED

2.4.5 Displaying the output in JSON format

pool.query --format=json

Displays the output in JSON format:

{
"command": "pool.query --format=json",
"result": [
{
"allocated_physical_space": "0",
"allocated_virtual_space": "10.78 TB",
"max_extend": "DISABLED",
"name": "pool_acc3d07cb0b011e48cda00505699a571",
"physical_capacity": "10.99 TB",
"state": "NORMAL",
"virtual_capacity": "10.99 TB"
}

Getting started  –  20

 InfiniBox 4.0.10 User Documentation

]
}

2.4.6 Querying for any string

Querying for a single string

*.query --grep=STRING
Returns query results only for entities that contain the searched string.

pool.query --grep=er --columns=name

uberpool_0000
uberpool_0001

Querying for more than one string

*.query  --grep="STRING1|STRING2"
Returns query results for entities that contain either of the searched strings.

pool.query --grep=er|re --columns=name

replica_0000
replica_0001
uberpool_0000
uberpool_0001

2.4.7 Approving a warning message

Some InfiniShell commands require user approval. The approval message always provides a reason,
and can be either approved or rejected. For example, deleting a pool requires user approval:

pool.delete name=pool1
Deleting pool 'pool1' might prevent the pool's administrator from creating new volumes.
Are you sure? [y/n]

If the warning message is approved, the pool is deleted:

Are you sure? [y/n] y

Pool "pool1" deleted

If the warning message is disapproved, the command is aborted and the pool is not deleted:

Getting started  –  21

 InfiniBox 4.0.10 User Documentation

Are you sure? [y/n] n

Command aborted

2.4.8
GUI shortcuts
Press on a question mark (?) anywhere on the GUI to get a list of GUI shortcuts.

2.5 Confirming dangerous operations

• About this task
• Taxonomy of dangerous operations
• Deleting an entity that contains data

Getting started  –  22

 InfiniBox 4.0.10 User Documentation

• Deleting an entity that supports other entities

2.5.1 About this task

Dangerous operations are operations that create a risk of downtime or user interference. These
operations require explicit approval to make sure the user does not perform them by accident.

Taxonomy of dangerous operations

Deleting an entity that contains data

Upon an attempt to delete an entity, the user is asked to approve a warning message.

For example, deleting a dataset results with the following message:

Deleting an entity that supports other entities

Upon an attempt to delete an entity that is a requirement for another entity, an informational message
is displayed.

For example, deleting a link that has replica definitions that are using it, results with the following
message:

Getting started  –  23

 InfiniBox 4.0.10 User Documentation

2.6 Getting help on commands

2.6.1 Getting help on commands

The help command uses the following arguments:

• On – this argument gets a name of a command and returns the command syntax.
• Search – this argument gets any string of characters and returns a list of commands where it is
part of the command name or syntax.

See the following examples:

• help - The output of this command is a list of all of the InfiniShell command categories.
• help on= and TAB - The output of this command is a list of all of the categories and commands
for which help can run with the on argument. See the next example.
• help on=user - The output of this command lists categories and commands that have user in
their name.
• help search=role - Returns a list of categories and commands where role is part of either the
category, command name, or parameter.
• <command_name>? -  Use this form similarly to help on=<command_name>.  For
example: user.create?

2.7 InfiniBox Management Console

• InfiniShell
• InfiniBox GUI
• RESTful API

InfiniBox offers an extensive management suite for controlling the way your data is managed. The
InfiniBox Management Console is a web application that can be accessed from any browser.

The management suite comprises the following elements:

• InfiniShell
• InfiniBox GUI
• RESTful API

Getting started  –  24

 InfiniBox 4.0.10 User Documentation

2.7.1 InfiniShell
The InfiniBox CLI comes with a rich, context-aware auto-complete, and with a simple and easy to use
command structure that only takes administrators a few minutes to learn. With commands structured
the same way across the board, and simple use of wildcards to apply the same commands to multiple
objects, administrators love using InfiniShell to accelerate and simplify their work.

2.7.2 InfiniBox GUI

The InfiniBox GUI a modern and up-to-date, easy to use application that features a responsive layout
for multiple devices and resolutions. The entities commands are available through a right-click menu.
When searching for entities, the results are divided by category (actions, entities, etc.). All menus
support keyboard navigation, including global keyboard shortcuts and global search.

2.7.3 RESTful API

The InfiniBox Management Console is a web application that can be accessed from any browser.

2.8 InfiniBox management queries

• About this task
• Hardware queries
• Network queries
• config.ethernet.interface.query
• config.ethernet.port.query
• config.fc.port.query
• config.fc.switch.query
• Replica and link queries
• link.query
• replica.query
• pool.query

2.8.1 About this task

InfiniBox provides queries that list entities along with their properties. This page lists the available
queries with the available values of the state field.

2.8.2 Hardware queries

Available hardware queries are:

Getting started  –  25

 InfiniBox 4.0.10 User Documentation

• drive.query
• enclosure.query
• node.drive.query
• node.query

Available states:

• UNCONFIGURED - the enclosure is not configured to receive a disk at this slot

• MISSING - the enclosure is configured to receive a disk and there is not disk at the slot
• READY - the disk is in the slot and is not yet activated
• ACTIVE - the disk is in the slot and is activated
• FAILED - the disk is in the slot, was ACTIVE and is not deactivated

2.8.3 Network queries

config.ethernet.interface.query

• CONNECTED - the interface is connected to all of the ports

• DEGRADED - the interface is connected to some ports (not all of them)
• DISCONNECTED - the interface is not connected to any of the ports

config.ethernet.port.query

• This command will be removed in 3.0

config.fc.port.query

• OK - the port exists

config.fc.switch.query

• No states are available

2.8.4 Replica and link queries

link.query

• UP - the link is up, supported by the entire cluster and can support a service
• DOWN - the link is down and cannot support a service

Getting started  –  26

 InfiniBox 4.0.10 User Documentation

• DEGRADED - Either of:

• The link is up and can support a service, but it is not supported by the entire cluster
• The management link is down
• UNKNOWN

replica.query

• INITIAL_REPLICATION - the replica is correctly defined and is undergoing an Initialization phase in

which the local dataset is replicated to the target in order to create a consistent snapshot.
• IDLE - the replica is correctly defined, the Initialization phase ended successfully, and the replica
is ready for the next sync job.
• REPLICATING - the replica is undergoing a sync job. At the end of the sync job the replica returns
to an Idle state.
• AUTO_SUSPENDED - the link is down, or the target pool is not in Normal state, so a sync job does
not result with a consistent snapshot.
• SUSPENDED - the replica is manually suspended, and no new sync job will start.
• RECONNECTING
• CANCELED
• UNKNOWN

2.8.5 pool.query
• Normal - the pool is created with this state and remains in this state as long as it does not use
the emergency buffer.
• Limited - the pool enters a Limited state when it consumes its emergency buffer. The pool
returns to a Normal state if data is removed from it, or if its capacity is extended.
• Locked - the pool enters a Locked state if its emergency buffer is fully consumed. The pool
returns to a Normal state if its capacity is extended, or data is removed from it.

2.9 InfiniShell query options

• About this task
• Filtering a query
• Filtering the query result by any string of characters
• Filtering by a string of characters
• Filtering for query results that do not include a specific string

Getting started  –  27

 InfiniBox 4.0.10 User Documentation

• Sorting the query

• Sorting by numerical value
• Sorting by alphanumerical value
• Setting the displayed capacity unit
• Selecting the displayed columns
• Setting the results format
• CSV format
• JSON format
• Displaying a detailed output

2.9.1 About this task

InfiniShell provides options that help the user to control the way query result are displayed.

2.9.2 Filtering a query

InfiniBox queries can be filtered, so that only specific entities will be returned. The filtering options of
each query are available through running help on the query command: typing the command and
pressing TAB.

For example:

You can filter the volume query by:

• Name - the name of the volume

• Pool - the pool that the volume belongs to
• CG - the consistency group that the volume is a member of
• Mapped, thin, write protected - volumes attributes

vol.query

[name] [pool] [mapped] [thin]  

[--columns] [--format] [-d|--xdebug]
[cg] [unit] [write_protected] [--detailed]
[--sort] [--grep] [-y|--yes]

Getting started  –  28

 InfiniBox 4.0.10 User Documentation

2.9.3 Filtering the query result by any string of characters

Filtering by a string of characters

Example: vol.query --grep=replica

Filtering for query results that do not include a specific string

Examples:

• vol.query --grep=replica
• vol.query --inverted-grep=replica

2.9.4 Sorting the query

The query results can be sorted by the value of any of its fields.

Sorting by numerical value

Sorting the query results by used, from small to big.

vol.query --sort=used

NAME THIN USED POOL

vol_1 yes 980.02 MB replication-from-ibox1092
vol_2 yes 880.02 MB replica_from_ibox3008
vol_3 yes 763.69 MB replica_from_ibox3008
vol_4 yes 678.97 MB replica_from_ibox3008
...

Sorting the query results by used, from big to small.

vol.query --sort=-used

NAME THIN USED POOL

vol_4 yes 678.97 MB replica_from_ibox3008  
vol_3 yes 763.69 MB replica_from_ibox3008
vol_2 yes 880.02 MB replica_from_ibox3008
vol_1 yes 980.02 MB replication-from-ibox1092
...

Getting started  –  29

 InfiniBox 4.0.10 User Documentation

Sorting by alphanumerical value

Sorting the query results by volume name.

vol.query --sort=name

NAME THIN USED POOL

vol_1 yes 980.02 MB replication-from-ibox1092
vol_2 yes 880.02 MB replica_from_ibox3008
vol_3 yes 763.69 MB replica_from_ibox3008
vol_4 yes 678.97 MB replica_from_ibox3008
...

2.9.5 Setting the displayed capacity unit

You can select the unit in which capacity is displayed.

vol.query unit=
B, G, GB, GiB, T, TB, TiB or block
B G GB GiB T TB TiB block

2.9.6 Selecting the displayed columns

Select the columns that will be displayed on the query result.

vol.query --columns=name,used,pool
 
NAME USED POOL  
aixio001_1 0 aix  
aixio001_2 0 aix  
aixio001_3 0 aix  
aixio001_4 0 aix  
aixio001_5 0 aix  
aixio003_boot_1 0 aix  
aixio003_boot_2 0 aix  
aixio003_boot_3 0 aix  
aixio003_boot_4 0 aix  
aixio004-01 222.00 GB aix  
aixio004-02 444.00 GB aix  
aixio004-03 665.99 GB aix  

Getting started  –  30

 InfiniBox 4.0.10 User Documentation

aixio004-04 887.96 GB aix

...

2.9.7 Setting the results format

You can select between tabular, CSV and JSON formats.

(For tabular format, run the query without the format option.)

CSV format

vol.query --format=csv

NAME,THIN,SIZE,USED,ALLOCATED,TREE ALLOCATED,POOL,WP,MAPPED

aixio001_1,no,500.00 GB,0,500.00 GB,0,aix,no,yes

aixio001_2,no,500.00 GB,0,500.00 GB,0,aix,no,yes

aixio001_3,no,500.00 GB,0,500.00 GB,0,aix,no,yes

aixio001_4,no,500.00 GB,0,500.00 GB,0,aix,no,yes

aixio001_5,no,500.00 GB,0,500.00 GB,0,aix,no,yes

Getting started  –  31

 InfiniBox 4.0.10 User Documentation

JSON format

vol.query name=vol_fea63de9-6a2f-4f9d-8fa2-62aa34e6d855_replica --format=json

"command": "vol.query name=vol_fea63de9-6a2f-4f9d-8fa2-62aa34e6d855_replica --format=json",

"result": [

"allocated": "1.00 GB",

"mapped": "no",

"name": "vol_fea63de9-6a2f-4f9d-8fa2-62aa34e6d855_replica",

"pool": "replica_from_ibox3008",

"size": "1.00 GB",

"thin": "yes",

"tree_allocated": "0",

"used": "1.00 GB",

"write_protected": "no"

2.9.8 Displaying a detailed output

The detailed output displays more fields than the default tabular output.

Getting started  –  32

 InfiniBox 4.0.10 User Documentation

vol.query name=vol_fea63de9-6a2f-4f9d-8fa2-62aa34e6d855_replica --detailed

Name vol_fea63de9-6a2f-4f9d-8fa2-62aa34e6d855_replica

Serial 742b0f000000bbb0000000000000414

Thin yes

Size 1.00 GB

Used 1.00 GB

Allocated 1.00 GB

Tree Allocated 0

Pool replica_from_ibox3008

Write Protected no

Mapped no

Ssd Cache yes

Replication Role -

Created At 2015-09-27 09:59:28

Updated At 2015-10-19 12:33:45

2.10 InfiniShell session controls

2.10.1 Session controls

• %clear - Clears the screen.
• %csv - Displays the command output in CSV format. Enter the command again to exit from this
mode.
• %debug - Toggle debug prints. The command output is preceded by debugging messages.
• In addition, you can run individual commands in debug mode. For example:
vol.query name=v1 –xdebug
Or
vol.query name=v1 –d
• %logstart - Start command logger. Enter the command to start logging the session you are
currently in.

Getting started  –  33

 InfiniBox 4.0.10 User Documentation

• %logstart file=log.txt 
Logger activated 
You may provide values to the parameters (none of them is mandatory).
• The file parameter saves the log to a file with the specified name and sets the mode to
append.
• The mode parameter determines the way the log is saved into a file.
• Append mode keeps logging to an existing log file.
• Override mode overrides an existing log file.
• The output parameter logs the output of each entered command.
• The timestamp parameter adds a time stamp to each of the logged commands.
• %logstate - Display the command logger state. This command displays the state of the logging
(either on or off).
• %logstop - Stop the command logger. This command stops logging.
• %raw - Toggle raw API output. Enables displaying the JSON query of the commands output.
• %user - Displays the name of the active user.
• %version - Displays the InfiniShell version.

2.11 Using the InfiniBox API

• Accessing the API documentation from the management console
• API documentation in PDF format

InfiniBox provides a thoroughly documented set of RESTful APIs. The documentation is available both
on the management console and here, on this help article.

2.11.1 Accessing the API documentation from the management console

The API documentation is available on the following URL:

http://<your_system>/apidoc/

2.11.2 API documentation in PDF format

InfiniBox Management API 4.0.pdf

2.12 Using the InfiniSDK

• Downloading InfiniSDK
• InfiniSDK documentation

Getting started  –  34

 InfiniBox 4.0.10 User Documentation

InfiniBox provides an SDK, which is a comprehensive library for use with Python. InfiniSDK greatly
simplifies the creation of customized scripts. The compatibility with RESTful allows for using the API
with any language or application that supports the appropriate HTTP commands. However, we do
recommend that the SDK is used, as this will be updated as new features that are added to the product.

2.12.1 Downloading InfiniSDK

Download locations:

• GitHub
• PyPI

2.12.2 InfiniSDK documentation

• InfiniSDK User's Guide on readthedocs.org

2.13 Running InfiniShell from an interactive shell

• About this task

• Getting help
• Usage
• Options
• Examples
• Printing the InfiniShell version
• Executing infinishell commands without logging into the system first
• Creating a script file
• Logging into an InfiniBox system
• Content of the infinishellrc file
• How to generate this file
• Path on Windows OS
• The content of the file
• Password obfuscation
• To edit the password
• To obfuscate the password
• To view the password

Getting started  –  35

 InfiniBox 4.0.10 User Documentation

2.13.1 About this task

When running InfiniShell from outside the InfiniBox GUI, the following options are available.

2.13.2 Getting help

infinishell --help

2.13.3 Usage

infinishell [options] system[:port]

Getting started  –  36

 InfiniBox 4.0.10 User Documentation

2.13.4 Options

-h, --help show this help message and exit

-v, --version Print version and exit
-c CMD, --cmd=CMD Commands to execute (separated by semicolons)
-f FILE, --file=FILE Script file to execute
-d, --debug Enable debug prints
-r, --raw Display raw API responses instead of normal rendering
-u USERNAME, --user=USERNAME Login username
-p PASSWORD, --password=PASSWORD Login password
-H CMD Display syntax of the specified command
-l FILE, --log=FILE Log commands to the specified file (default: None)
--write-default-config Generate the default configuration file at "/home/<user>/.infinidat/infinishell/
infinishellrc"
--list-environment List environment variables
--color=WHEN Whether or not to output with color. Options are: always, never, auto (default: auto)
-k, --keep-going Keep going when administration errors occur (syntax errors always abort the
entire script)
--fail-fast Stop running script when the first administration error occurs (this is the default)
-q, --quiet No output apart from errors
--not-quiet Normal output, the opposite of --quiet (this is the default)
--with-paging Enable paging of large outputs (this is the default)
--no-paging Disable paging of large outputs
-y, --yes Automatically confirm prompts (of potentially destructive commands)
--no Automatically reject prompts (of potentially destructive commands) (this is the default)
--history-file=PATH Path of history file, for cross-session history of commands. By default this is
saved in InfiniShell's data directory, under a unique name generated off of the target system
address
--with-history Enable cross-session history of commands (this is the default)
--no-history Disable cross-session history of commands
--csv CSV output mode (optimize output for easy parsing, see RFC 4180)
--no-csv Disable CSV output mode (this is the default)
--json JSON output mode (optimize output for easy parsing, see RFC 4180)
--no-json Disable JSON output mode (this is the default)
--request-timeout=SECONDS Maximal amount of time to wait for management response (default:
30)
--with-ssl Encrypt connection using SSL or TLS (this is the default)
--no-ssl Do not encrypt connection

Please note that defaults may be overridden in the configuration file: "/home/<user>/.infinidat/
infinishell/infinishellrc". Also, some environment variables are available for your convenience. To
list them, use the --list environment option.

Getting started  –  37

 InfiniBox 4.0.10 User Documentation

Examples

Printing the InfiniShell version

• Parameters - None
• Example - infinishell -v
• Output - Infinidat LTD. Command Line Utility v2.0.0.26

Executing infinishell commands without logging into the system first

Execute infinishell commands separated by semicolons

• Option - -c 
• Example - infinishell -c "vol.delete name=v1" box-avi --user=admin --password=123456
• Output - Volume "v1" deleted

Creating a script file

Create a file with infinishell commands that will be run one-by-one.

• Options
• -f - specify the file name
• -k - do not exit the script file upon a failed execution of a command
• -y - approve all dangerous operations
• Example - infinishell -f shell-script box-avi --user=admin --password=123456 -y
• Output (some columns were removed)

NAME THIN SIZE USED

v1 no 10.00 GB 0
Volume "v2" created
NAME THIN SIZE USED
v1 no 10.00 GB 0
v2 no 10.00 GB 0

Logging into an InfiniBox system

• Parameters
• system
• user
• password
• Example - infinishell box-avi --user=admin --password=******

Getting started  –  38

 InfiniBox 4.0.10 User Documentation

• Output

Infinidat Ltd. Command Line Utility v2.0.0.26

For help on all commands, type 'help'.
For help on a specific command, type 'help on=vol.create' or 'vol.create?'
admin@box-avi>

2.13.5 Content of the infinishellrc file

How to generate this file

infinishell --write-default-config

Path on Windows OS

%PROFILE%\.infinidat\infinishell\infinishellrc

Getting started  –  39

 InfiniBox 4.0.10 User Documentation

The content of the file

{
"api_max_page_size": 1000,
"internal_log_backup_count": 10,
"color": "auto",
"keep_going": false,
"api_port": 80,
"enable_fs_commands": false,
"api_warnings": false,
"api_ssl": true,
"default_address": null,
"username": "admin",
"api_request_timeout_seconds": 30,
"api_default_page_size": 50,
"history_file": null,
"plugins_dir": null,
"csv_output": false,
"api_default_search_page_size": 200,
"password": "38a7d20a8700614715deoJNra5ud4a87ffb6354aaad4b9f8",
"internal_log_max_size_mb": 20,
"json_output": false,
"internal_log_file": null,
"quiet": false,
"paging": true,
"history": true,
"api_ssl_verify_certificate": false,
"log_file": null,
"auto_confirm_prompts": false,
"api_ssl_port": 443
}

Password obfuscation
When changing the password, the password is visible on the screen. However, upon the next look into
the configuration file, the password is no longer visible, as it is obfuscated.

To edit the password

$ nano ~/.infinidat/infinishell/infinishellrc
...

Getting started  –  40

 InfiniBox 4.0.10 User Documentation

$ cat ~/.infinidat/infinishell/infinishellrc | grep password

"password": "123456",

To obfuscate the password

• To make the password into obfuscated, launch InfiniShell, so configuration file will be updated
with the encrypted password.
• There is no to start a shell session against any system

To view the password

$ cat ~/.infinidat/infinishell/infinishellrc | grep password

"password": "38a7d20a8700614715deoJNra5ud4a87ffb6354aaad4b9f8",

2.14 Setting the message of the day

• About this task
• Approving the message of the day in order to proceed to the login screen
• Instructions for setting the message of the day

2.14.1 About this task

The message of the day is a configurable piece of configurable text that is displayed to the user on the
InfiniBox GUI. The user has to approve the message in order to proceed to the GUI login screen.

The maximum message length is 4,094 characters.

Getting started  –  41

 InfiniBox 4.0.10 User Documentation

Approving the message of the day in order to proceed to the login screen
1. When navigating to the InfiniBox GUI screen, the message of the day is displayed.

Getting started  –  42

 InfiniBox 4.0.10 User Documentation

2. Click Continue to proceed to the login screen.

2.14.2 Instructions for setting the message of the day

1. Log into InfiniBox GUI and click the Settings icon. The Setting screen opens,
displaying the General tab. 
2. Paste the message of the day text to the Message of the day text field (at the column to the left).
3. Click Update.
The message of the day is updated and will be displayed from now on to all users.

Getting started  –  43

 InfiniBox 4.0.10 User Documentation

2.15 A note on installing InfiniShell on a Windows client

• Download the installation package from the INFINIDAT repository.
• Note:
• If the installation file is not executed from the installation library, the infinishell.exe
file is not inserted into the installation library
• To insert the infinishell.exe file to its proper location, log out of Windows and log in
again 

Getting started  –  44

 InfiniBox 4.0.10 User Documentation

3 Compression

3.1 Overview of compression

• Data reduction
• When to compress
• Compression Methods
• Size of compressed data
• Terminology
• Accounting the savings
• Using capacity freed from data reduction
• Reduction ratio
• Default behavior
• Performance protection

3.1.1 Data reduction

Compression is an efficient way of reducing storage costs, slowing down the growing cost of storage,
but may also impact performance. In order to lower the performance impact, we are interested in when
is the data compressed and which method is used for compression.

When to compress
Storage arrays may compress the incoming data either before or after the write operation is
acknowledged to the host. When the data is compressed before the acknowledgement, the storage
array benefits from low cache usage, in the expense of the following:

• Performance – Compressing the data takes time, which means the host experiences higher
latency, which reduces its ability to take advantage of the storage array performance capabilities
• Increased cost of cache hits - data that was just written usually has a good chance of being read
in the short time after it was written (and often read several times). This yields the much desired
low-latency cache hit. However, if the data is compressed, it has to be decompressed anew for
each cache hit. The repeatedly decompression results in high that may undo the performance
gains that could have resulted from the low-latency cache hits.

To avoid these two considerable issues InfiniBox leverages a very large write cache that allows for
compressing data only during its destage to disk.

As a result, InfiniBox not only reduces the latency of writes, but in many cases avoids compression
altogether, further avoiding undesireable performance impact.

Compression  –  45
 InfiniBox 4.0.10 User Documentation

Compression Methods
Compression used to be a CPU-intensive task, but improvements in CPU architecture and compression
algorithms have reduced the CPU cycles required to compress data. 

InfiniBox uses LZ4, the most common algorithm to compress data. However the system was designed
to allow adding more compression types later, as more efficient compression algorithms become
available, or new CPU capabilities further reduce the CPU overhead of existing compression algorithms.

Size of compressed data

The InfiniBox case for not compressing the data until destaged is backed by the nature of the
compression algorithms.

Simply put, compression algorithms try to find strings of data that repeat as many times as possible
within the data and replace them with shorter strings, all the while keeping a dictionary of these
replacements.

During the decompression, each string that appears in the dictionary is replaced with the original
(longer) string, thus the original data is restored.

The more frequent the data string, the more effective the compression. When comparing the
compression of a string of 1 million identical characters to 1000 strings of 1000 identical characters
each, we find that contemporary algorithms keep one dictionary in the first case, compared to 1000
dictionaries in the later case. An improved compression algorithm may have to compare longer strings
of characters, possibly increasing the compression ratio in the expense of a performance impact. In
addition, the compression algorithms are susceptible to reach a plateu in their ability to improve both
performance and compression ratio.

As InfiniBox compresses 64KB of data at a time (compared to the commonly used 4KB-8KB) it reaches a
higher compression ratio at a lower performance impact for the same source data.

3.1.2 Terminology
Compressing data creates a need to show the capacity savings created by compression and zero
reclaim. InfiniBox 3.0 features a new capacity counter to volumes, filesystems and pools called "Data
reduction".

Data reduction is displayed as a ratio between the capacity that would have been consumed by the
entity, and the capacity that is actually consumed (including both capacity associated with the entity
itself and its snapshots).

3.1.3 Accounting the savings

While compression works on both thick and thin entities, its behavior is different in each case:

Compression  –  46
 InfiniBox 4.0.10 User Documentation

• Thin entities: All data written is compressed, savings from data reduction remains available in
the pool's physical capacity
• Thick entities: Data is written comressed, but the savings remains allocated to the entity (can't
be reused.) Only data in the entity's snapshots (which are thin) remains in the pool's physical
capacity.

In the following example, "vol1", a 1TB thin volume is mapped to a host. The host writes 800GB of data
(assumed to be 2:1 compressible), and fills 100 of the remaining 200GB with zeros:

Host reported capacity:

Result:

• Zeros are automatically reclaimed, so the 100GB of zeros does not consume space.
• The 800GB is compressed by a ratio of 2:1.
• The 1TB was reduced to 400GB on disk out of the 900GB the user wrote, yielding a 2.25:1 data
reduction ratio

Using capacity freed from data reduction

Once entities enjoy data reduction they effectively free capacity that needs to be reused. The amount of
capacity that is freed returns to the pool. Now, it is up to the storage admin to decide whether to:

• Leave the freed capacity in the pool, allowing any of the pool admins to use it

Compression  –  47
 InfiniBox 4.0.10 User Documentation

• Reduce the pool size, allowing the storage admin to reuse that capacity elsewhere.

Reduction ratio
Data reduction ratios vary, depending on the dataset.

For example, virtual machines and databases tend to include many zeros, increasing the data
reduction, while data encrypted before writing it to the storage does not compress well.

The maximum available reduction ratio is 16:1.

3.1.4 Default behavior

Compression per entity is inherited from its pool during creation time, and can be changed at any time.
By default all pools after upgrading to 3.0 are set to "compression=on", so all new entities are created
with compression (unless manually changed).

Existing entities in the pool (Created before 3.0) can be manually set to compress new data. 

When a new pool is created it inherits its compression behavior from a system default for compression
(set to "on", can be overridden).

3.1.5 Performance protection

InfiniBox employs multiple mechanisms to make sure performance is not degraded as a result of
compression:

Writes:

Any write to the storage is first accepted in memory to guarantee minimal latency, and only
compressed as data is destaged from RAM to persistent media.

Since many writes get overwritten while in the cache, this InfiniBox avoids a lot of unnecessary
compression.

Once the data is destaged, data is compressed in sections (sets of consecutive 64KB) increments,
creating a large enough dataset to achieve effective compression, but still keeping each sections
separated. This allows the system to retrieve any section without having to retrieve the entire
stripe.

Reads:

All reads from RAM and SSD don't require decompression as data is not compressed while in
cache. This maintains low latency.

Data retrieved from disk will require decompression when its loaded into the cache.

3.2 Accounting the compressed capacity

• The saved data

Compression  –  48
 InfiniBox 4.0.10 User Documentation

• Use-cases
• Introducing compression to an existing InfiniBox system
• Writing into a dataset
• Taking two snapshots
• Overwriting the dataset with 256KB
• Deleting a snapshot
• Unmapping blocks from the volume

3.2.1 The saved data

InfiniBox calculates the saved capacity on the dataset tree level. The capacity counters report on the
following:

• How much capacity is allocated to the dataset (in dark blue, on the image below)
• How much capacity is allocated to the dataset descendants (in light blue, on the image below)
• How much capacity is saved, due to the compression (in grey, on the image below)
• How much free capacity is there (the sum of the grey and white lines, on the image below)

3.2.2 Use-cases

Introducing compression to an existing InfiniBox system

• On an existing system, prior to the introduction of the compression feature, all pools and their
entities are not compressed.
• Upon upgrading to an InfiniBox software release that features compression, all existing pools
and their entities remain uncompressed
• When creating a new pool and marking it as compressed, it will be compressed
• The transition of an entity from a non-compressed pool to a compressed pool is described below.

Compression  –  49
 InfiniBox 4.0.10 User Documentation

Writing into a dataset

• Raw capacity - 4 * 64KB
• Compressed capacity - 64KB
• In this use case, the 256KB were compressed to 64KB
• Used capacity (allocated by the user) - 4 * 64 = 256KB
• Allocated - 64KB
• Tree allocated - 0
• Saved capacity - 256 - 64 = 192KB

Taking two snapshots

• Used capacity (allocated by the user) - 256KB
• Allocated - 64KB
• Tree allocated - 0
• Saved capacity - 192KB

Overwriting the dataset with 256KB

• Raw capacity - 256KB
• Compressed capacity - 128KB
• In this use case, the 256KB were compressed to 128KB
• Used capacity (allocated by the user) - 256KB
• Allocated - 128KB
• Tree allocated - 64
• The overwritten blocks of one of the snapshots
• Saved capacity - 192 + 128 = 320KB

Deleting a snapshot
• Used capacity (allocated by the user) - 256KB
• Allocated - 128KB 
• Tree allocated - 64
• The capacity is moved to the other snapshot
• Saved capacity - 192 + 128 = 320KB

Compression  –  50
 InfiniBox 4.0.10 User Documentation

Unmapping blocks from the volume

• Used capacity (allocated by the user) - 0
• The 256KB are no longer needed
• Allocated - 0 
• Tree allocated - 64
• remaining
• Saved capacity - 256 - 64 = 192KB

3.3 Compression limitations

• Temporal counters inaccuracy
• Compressing pre-existing data
• Writing non-compressed data on a compression-enabled dataset

3.3.1 Temporal counters inaccuracy

• Compression happens during destage, and so data in the write cache is not yet compressed (not
reflected by the volume / filesystem "savings" field.
• On a large system with 200TB of data, with a full cache (135GB) this is less than 0.07% of
inaccuracy (less for systems with more data)
• Since data in write cache is not compressed, the host write is acknowledged without delay

3.3.2 Compressing pre-existing data

• When upgrading a populated InfiniBox system to a software version that enables compression,
the already existing data is not automatically compressed
• Moving old volumes to new compressed volumes (for example, using VMotion) compresses the
data
• This limitation will be eliminated in a later release

3.3.3 Writing non-compressed data on a compression-enabled dataset

The InfiniBox system may write data uncompressed for a compression-enabled dataset, if:

• The system wrote the maximum amount of compressed data it can (for data that is highly
compressible).

Compression  –  51
 InfiniBox 4.0.10 User Documentation

This limit is not a hard number but rather calculated based on the memory consumed by the
metadata. We estimate average compression up to 1:3 will not consume all the memory.
• A specific volume / filesystem reached the ceiling compression ratio. Compression on that entity
will stop to allow other entities to keep compressing more data.
• Compression is ineffective (for example - data was encrypted by the host and doesn't compress) -
the system avoids compression (reduces CPU load)

3.4 Setting an entity to be compressed

• About this task
• Instructions
• A note on compressing a newly created filesystem

3.4.1 About this task

When creating a pool, volume or a filesystem, there is an option to set them to be compressed.

The default compression settings are as follows:

• Pool - the default compression value is set via the API:

• Request: /api/rest/config/mgmt/ 
• Parameter: pool.compression_enabled_default
• Volume & filesystem - follow the pool's settings
• Whenever the pool is set to no compression, the Volume Create and Filesystem Create
screen will have their default compression setting set to no

3.4.2 Instructions
The compression settings toggle is available on the following screens:

• Creating a pool
• Creating a volume
• Creating a filesystem

3.4.3 A note on compressing a newly created filesystem

When a new filesystem is created, it reserves 1.6MB for its metadata.
This part of the filesystem compresses very well, leading to the filesystem showing 16:1 compression
ratio.

Compression  –  52
 InfiniBox 4.0.10 User Documentation

• Note that a thick filesystem starts with 1:1 ratio even if compression is on.

3.5 Changing the provisioning type of a compressed dataset

• About this task

3.5.1 About this task

Compression applies to thin provisioned volumes and filesystems. When the provisioning type of a
compressed dataset changes from thin to thick, all of its pre-compressed size is allocated.

Compression  –  53
 InfiniBox 4.0.10 User Documentation

4 Authentication and security

4.1 Overview of user management

• InfiniBox user management
• Limits
• Role Based Access Control (RBAC)
• Available user roles
• Read-only
• Admin
• Pool admin
• Technician
• InfiniBox functionality for user roles
• User repositories
• Local users
• Setting user names and passwords
• Role management in external user repositories
• Definining an Active Directory domain as user repository
• Defining  OpenLDAP server as user repository 
• Authentication methods
• User authentication and role resolution process
• Authentication and Role resolution diagram

4.1.1 InfiniBox user management

The InfiniBox management interface is exposed in the form of a comprehensive REST API,GUI and CLI.
To prevent unauthorized access to the system configuration and monitoring capabilities, access to the
API requires a succesful authentication in the form of providing user and password credentials of a user
with a valid role in InfiniBox.

InfiniBox supports multiple types of user repositories which can be used for user authentication and
access rights resolution. InfiniBox supports using multiple user repositories concurrently with the ability
to define the credentials resolution across the repositories.

Administrative user repository types

• Local users - User repository defined and managed within the InfiniBox system itself

Authentication and security  –  54

 InfiniBox 4.0.10 User Documentation

• Active Directory (AD) - Microsoft Active Directory of a specific domain (representing multiple AD
servers)
• OpenLDAP - Open LDAP repository stored on an OpenLDAP server

Limits
• Max number of local users in the system: 50
• Max number of external user repositories: 10
• Max number of servers within an OpenLDAP repository: 3

4.1.2 Role Based Access Control (RBAC)

InfiniBox role-based access-control (RBAC) assigns authenticated users into predefined access
permission roles. The access rights of each of the roles are pre-configured, thus the extent of the user
actions is predictable. InfiniBox supports 4 role types, where each user must have a single role assigned
to it.

Available user roles

Read-only
A read-only user can query for information only. The query permissions are sufficient for carrying out
monitoring tasks, viewing the system health, events, capacity utilization, etc. The read-only user cannot,
however, make any changes to the system. 

Admin
The admin (system administrator) has permission for all InfiniBox software functionality, including
network administration, provisioning pools and entities, and creating other users. For creating other
users, see below.

Pool admin
The pool admin has admin rights for a specific pool (or multiple pools). Within this pool (or pools), the
pool admin can provision datasets, map them to hosts and take snapshots, but cannot create new
pools or change the definitions of existing pools.

The pool admin has read-only permissions outside its pool (or pools).

Technician
The technician is a role that is typically assigned to INFINIDAT technicians that take care of InfiniBox
hardware on the customer premise.

Authentication and security  –  55

 InfiniBox 4.0.10 User Documentation

The technician role has permissions that are identical to the read-only user, with additional access to
FRU API, CLI and GUI commands. 

The status of the InfiniBox system physical components is visible to all user roles (admin, pool admin,
read-only). However, only the technician has access to commands that are required for hardware
maintenance (for example, the deactivation of a faulty drive and the activation of the new drive that
replaces the faulty drive).

InfiniBox functionality for user roles

InfiniBox functionality Admin Pool Read Technicia

Admin Only n

Pool administration

Provisioning volumes, filesystems, consistency

groups and their snapshots

Exporting a filesystem

Assigning pools and volumes to a QoS policy

Host and cluster management

Event rules administration

Provisioning a replica

Network management

Hardware operations

User management

4.1.3 User repositories

Local users
InfiniBox provides you with a set of out-of-the-box local users. On top of these out-of-the-box users, you
can define more local users. All of the local users have to have one of the available user roles. 

Authentication and security  –  56

 InfiniBox 4.0.10 User Documentation

Setting user names and passwords

• Both user names and passwords are case-sensitive

• User names have to conform to the following guidelines:

• Maximum of 65 Latin characters, numbers, spaces, and the following symbols: "^&'@()[]$=!-#{}%.
+~_" (excluding quotation marks).

• Leading and trailing whitespace characters are stripped.

Role management in external user repositories

Unlike the local users repository, external repositories role assignment is done at the group level - i.e. a
group from AD or OpenLDAP repository can be assigned one of the access roles. Users belonging to
this group will be granted the role of the group.

Definining an Active Directory domain as user repository

Defining an Active Directory domain repository as a user repository in InfiniBox requires the following
information: 

• AD Domain name
• Repository Name
• Bind username and password
• Whether to use LDAPS
• Users (optional) - in order to limit InfiniBox access only to some of the repository users, specify
these users according to these attributes:
• User Class, Username Attribute, Users Base DN
• Groups (optional) - in order to limit InfiniBox access only to some of the repository user groups,
specify these users according to these attributes:
• Group Class, Group Name Attribute, memberof Attribute, Group Base DN

InfiniBox dynamically discovers all AD domain controllers by issuing a DNS query to the provided AD
domain, InfiniBox will then try to use the fastest responding domain controller for its AD queries.

Defining  OpenLDAP server as user repository 

Defining an Active Directory or LDAP server to work with InfiniBox requires the following information:

• Repository Name
• OpenLDAP Server(s) ip/hostname
• Use SSL - select whether to use LDAPS

Authentication and security  –  57

 InfiniBox 4.0.10 User Documentation

• Port
• Bind username and password
• Users (optional) - in order to limit InfiniBox access only to some of the repository users, specify
these users according to these attributes:
• User Class, Username Attribute, Users Base DN
• Groups (optional) - in order to limit InfiniBox access only to some of the repository user groups,
specify these users according to these attributes:
• Group Class, Group Name Attribute, memberof Attribute, Group Base DN

For resiliency, it is highly recommended to define multiple OpenLDAP servers managing the same user
repository, so that in case of a unavaliability of one of the OpenLDAP servers, InfiniBox can
transparently failover to use others.

4.1.4 Authentication methods
InfiniBox authenticates local users by comparing the username and password that the user provides
during the login to InfiniBox to the credentials that are stored on the InfiniBox.

An LDAP / Active Directory user does not authenticate directly to InfiniBox. The user authenticates to
LDAP / Active Directory, and the LDAP / Active Directory is defined as an InfiniBox user group. User
management is done in the user repository.  

For a broader discussion, see: InfiniBox Security Guide.

User authentication and role resolution process

When the user logs into InfiniBox and types the username and password, the username is resolved in
the following order:

1. InfiniBox checks whether the username belongs to a local user. If it is, InfiniBox checks whether
the password matches and the login succeeds
2. If the username does not belong to a local user, InfiniBox looks for it in the AD and LDAP that are
configured to work with InfiniBox. 
a. If the username belongs to any of the AD or LDAP and belongs to a user group that is
allowed to log into InfiniBox, the login succeeds.
b. The username is assigned with a user role that is determined by the user group that the
repository is mapped to.
c. In case that the username sppears in more than one repository, the username is resolved
according to the repository that ranks first in the order of resolution list. The order of
resolution is configurable.
3. The login fails in the following cases:

Authentication and security  –  58

 InfiniBox 4.0.10 User Documentation

a. The username does not belong to any local user, nor any AD or LDAP
b. The username belongs to an AD or LDAP that a configured to work with InfiniBox, but is
not a member of a group that is allowed to work with InfiniBox
c. The usename is listed as a local user, or belongs to the right group, but the provided
password is incorrect 

Authentication and Role resolution diagram

4.2 User management commands

• User commands
• Creating a new local user
• Creating an admin 
• Creating a PoolAdmin
• Querying for users

Authentication and security  –  59

 InfiniBox 4.0.10 User Documentation

• More user commands

• Pool Admin commands
• Switching among users
• login
• quit, exit
• User group commands
• Creating a user group

All of the commands are available both from the InfiniShell and the GUI.

A detailed explanation of the InfiniShell commands, along with their parameters, and syntax example, is
available on the InfiniShell Command Reference (here).

4.2.1 User commands

Creating a new local user

user.create

Set the user’s name, password, email address and role.

Creating an admin 
user.create name=pool-01 [email protected] role=PoolAdmin password=123456
The output is:

Local user "pool-01" created

Creating a PoolAdmin
user.create name=pool-02 [email protected] role=PoolAdmin
You are asked to provide a password for the user. The password you enter is not displayed on
screen.                    

Please enter a password for user "pool-02":

The output is:

Local user "pool-02" created

Authentication and security  –  60

 InfiniBox 4.0.10 User Documentation

All of the user’s attributes can be edited later on.

4.2.2 Querying for users

user.query

List existing local users along with their roles and the pools they have access to.

NAME ROLE POOLS

admin ADMIN <all pools>
infinidat INFINIDAT <all pools>
technician TECHNICIAN -

4.2.3 More user commands

• user.rename - Change local user name.
• user.change_password - Interactively reset password of local user. Enter the new password and
re-enter when prompted to do so.
• user.reset_password - Send password reset email to local user. 
• Follow the instructions on the email.
• Note: This command requires a verification of the INFINIDAT certificate. Hence, the
command is available when accessing InfiniBox from a shell and when using the GUI, but is
not accessible when using InfiniShell from a browser.
• user.change_email - Change local user email.
• user.change_role -  Change local user role.  Available roles: Admin, PoolAdmin. The availability
of these roles depends on the role of the user that runs this command.
• user.delete - Delete the local user. A user cannot delete its own account. Whenever you need to
rearrange your pool administrators, and to delete any of them, use the Admin user. You do not
have permissions to delete the Admin user.

4.2.4 Pool Admin commands

The PoolAdmin user is an administrator of specific pools. This user has admin privileges to specific
pools only. The user has read-only access to other pools. The PoolAdmin user is not predefined on the
InfiniBox.

• user.add_pool - Grant pool provisioning privileges to a pool administrator.

• This command operates on pool administrators only.
• Type:

Authentication and security  –  61

 InfiniBox 4.0.10 User Documentation

user.add_pool name=<PoolAdmin> pool=

 and TAB.
A list of available pool IDs is dispalyed on screen. Add the pools one-by-one:

user.add_pool name=admin1 pool=pool1

Following the confirmation message, the pool association with the pool admin is visible
via user.query.
• user.remove_pool - Revoke pool provisioning privileges from a pool administrator.

4.2.5 Switching among users

Use the following login commands to switch among users.

login
Use this command in order to switch the user with which you log into the InfiniBox.

For example, following the user.create command, you can test the access rights of the new user. From
the InfiniShell, log into InfiniBox as follows:

admin@localhost> login username=pool-01-admin

Enter the password.

The logged in user is displayed on the command prompt:

pool-01-admin@localhost>

quit, exit
Use these commands to close the InfiniShell session.

Refresh the browser to login again.

4.2.6 User group commands

A user group is an entity that maps an LDAP group to an InfiniBox user role.

Creating a user group

• Command name:

Authentication and security  –  62

 InfiniBox 4.0.10 User Documentation

user.group.create

• Example: 

user.group.create name=ldap1 role=Admin ldap=infinidat.com

dn=CN=Administrators,CN=Builtin,DC=infinidat,DC=com

4.3 Locking a local user

• About this task
• A note on predefined user
• Disabling a user
• Enabling a user 

4.3.1 About this task

This security feature allows the system admin to better control local accounts, refraining them from
accessing InfiniBox on demand. The system admin can lock any of the local users. The locked user is
still listed on the users list, but cannot log into the InfiniBox system until unlocked by the system admin.

• This operation is applicable only to local users.

• For more information on managing LDAP users, see here: LDAP and Active Directory integration.

A note on predefined user

This task allows to lock all predefined local users, including:

• INFINIDAT
• Technician

4.3.2 Disabling a user

1. Click the GUI Settings icon and then the Users tab.

Authentication and security  –  63

 InfiniBox 4.0.10 User Documentation

2. Check a user and select Disable user from the actions menu.

3. Approve the operation:

The user is marked as disabled.

4.3.3 Enabling a user 

1. Click the GUI Settings icon and then the Users tab.
2. Check a user and select Enable user from the actions menu.
The user is enabled. 

Authentication and security  –  64

 InfiniBox 4.0.10 User Documentation

4.4 Security mechanisms

• Authenticated session support
• Configuring the session auto-termination parameters
• Session identification
• Password change amidst a user session
• User role changes
• LDAP users
• Changes to LDAP users take effect only after a refresh
• Changing the password of an LDAP user
• Management node failover

InfiniBox features security mechanisms that:

• Increase the security on access to the storage

• Provide visibility to storage admins on who's connected to the storage system

4.4.1 Authenticated session support

InfiniBox supports authenticated sessions to prevent:

• Storing passwords in clear text

• Sending passwords over the wire for each request

Configuring the session auto-termination parameters

• Via the API:
• Request: api/rest/config
• Parameters: 
• mgmt/mgmt.session_expiry
• mgmt/mgmt.session_idle_timeout

• Via infinishell:
• config.system.set_session_timeout

4.4.2 Session identification

Each session will include the following data:

• session_id

Authentication and security  –  65

 InfiniBox 4.0.10 User Documentation

• username
• password
• creation time
• last request time

Password change amidst a user session

In case that the password changes amidst a user session, the session is updated and the user does not
have to re-login.

4.4.3 User role changes

Changes to the user role take effect immediately.

4.4.4 LDAP users

• For more information on LDAP users, see here: LDAP and Active Directory integration.

Changes to LDAP users take effect only after a refresh

The following changes to an LDAP user take effect only after the credentials are refreshed:

• Group assignment
• Account removal

Changing the password of an LDAP user

LDAP users changing passwords will require re-authenticate.

4.4.5 Management node failover

A management node failover requires all users to re-authenticate.

4.5 Setting the session timeout

• About this task
• Handling session expiry
• Idle session timeout
• Active session timeout
• Setting the session timeout parameters

Authentication and security  –  66

 InfiniBox 4.0.10 User Documentation

• Using the GUI

• Using InfiniShell
• Set both active and idle system timeout 
• Query for the active and idle system timeout
• Using the API
• Setting the active session timeout
• Required URL parameters
• Required JSON data
• Setting the idle session timeout
• Required URL parameters
• Required JSON data
• Getting the active session timeout
• Required URL parameters
• Required JSON data
• API endpoint 
• Returned code
• Getting the session idle timeout
• Required URL parameters
• Required JSON data

4.5.1 About this task

InfiniBox session timeout values are configurable from the GUI and CLI. These values apply to all of the
clients (GUI, CLI, API). 

• Idle session timeout - sets the session to expire in case of user inactivity
• The user's credentials are kept for the timeout period. Within this period, when opening
the GUI in the browser, there is no need to type the user and password.
• By default, the session terminates after 1 hour of inactivity
• Active session timeout - sets the session to disconnect the user 
• By default, the session expires after 24 hours

These values are configurable, as described below.

Authentication and security  –  67

 InfiniBox 4.0.10 User Documentation

4.5.2 Handling session expiry

Idle session timeout

When the session is about to expire due to inactivity, a message pops-up on screen.

1. To keep the session, click on Keep me signed in. 

The Idle Timeout counter is reset, allowing another hour of inactivity (1 hour by default, or the
amount of time configured by the user).

Active session timeout

When the session expires, the user returns to the login screen.

4.5.3 Setting the session timeout parameters

Using the GUI

1. Log into InfiniBox GUI and click the Settings icon. The Setting screen opens,
displaying the General tab. 
2. Set the session timeout parameters:
a. Idle session timeout
b. Active session timeout

Authentication and security  –  68

 InfiniBox 4.0.10 User Documentation

3. Click Update.

Using InfiniShell

Set both active and idle system timeout 

config.system.set_session_timeout active_session=1440 idle_session=60

Query for the active and idle system timeout

config.system.query

Authentication and security  –  69

 InfiniBox 4.0.10 User Documentation

Using the API

Setting the active session timeout

Required URL parameters

None

Required JSON data

The time in minutes.

"2880"

API endpoint

PUT api/rest/config/mgmt/mgmt.session_expiry

Returned code

{
"metadata": {
"ready": true
},
"result": 2880,
"error": null
}

Setting the idle session timeout

Required URL parameters

None

Required JSON data

The time in minutes.

"60"

API endpoint

Authentication and security  –  70

 InfiniBox 4.0.10 User Documentation

PUT api/rest/config/mgmt/mgmt.session_idle_timeout

Returned code

{
"metadata": {
"ready": true
},
"result": 120,
"error": null
}

Getting the active session timeout

Required URL parameters

None

Required JSON data

None

API endpoint 

GET api/rest/config/mgmt/mgmt.session_expiry

Returned code

{ "metadata": { "ready": true }, "result": 1440, "error": null }

Getting the session idle timeout

Required URL parameters

None

Required JSON data
None

API endpoint 

Authentication and security  –  71

 InfiniBox 4.0.10 User Documentation

GET api/rest/config/mgmt/mgmt.session_idle_timeout

Returned code 

{ "metadata": { "ready": true }, "result": 60, "error": null }

4.6 How to define an LDAP server

• About this task
• Instructions
• Instructions for testing the LDAP definition

4.6.1 About this task

Defining an LDAP server to work with InfiniBox allows LDAP users (users that are listed on the LDAP
server) to access InfiniBox.

4.6.2 Instructions
1. On the InfiniBox GUI, go to Settings, click the LDAP tab and click the Define button.
2. Set the following attributes:
a. Name - LDAP connection name as it will be displayed on the InfiniBox GUI. This name does
not have to be the actual LDAP server name.
b. Domain name - the name of the LDAP server or a domain of LDAP server
c. Use SSL - select whether to use LDAPS
d. Port
e. Bind username 
f. Bind password
3. (Optional) Click the Action button to change the preconfigured schema
4. Click Define Server
The server is defined.

Authentication and security  –  72

 InfiniBox 4.0.10 User Documentation

4.6.3 Instructions for testing the LDAP definition

1. From the LDAP tab, select an LDAP definition, click the Action button and select Modify
Configuration.
The Modify Active Directory Server screen opens. 
2. Click Test.
A success message is displayed on the screen. 

4.7 Working around an issue found when defining a user group

• The issue
• Root Cause
• Workaround
• Planned Fix

4.7.1 The issue

InfiniBox releases 4.0 and 4.0.1 introduce an issue with defining a user group for AD server with a
default scheme value.

Authentication and security  –  73

 InfiniBox 4.0.10 User Documentation

4.7.2 Root Cause

InfiniBox 4.0 and 4.0.1 set the value of the Group Name Attribute of the Active Directory repository
type to group rather than to cn, as was in previous InfiniBox releases.

When InfiniBox queries the LDAP server, group is not found.

Authentication and security  –  74

 InfiniBox 4.0.10 User Documentation

4.7.3 Workaround
To work around this issue, change the value of Group Name Attribute to cn and retry the user group
creation.

Through CLI:

admin@localhost> config.ldap.modify ldap=<LDAP-NAME> schema_group_attribute=cn

4.7.4 Planned Fix

This issue will be fixed in InfiniBox 4.0.10.

Authentication and security  –  75

 InfiniBox 4.0.10 User Documentation

5 iSCSI connectivity

5.1 Overview of iSCSI connectivity

• What is iSCSI
• Discovery
• Connecting to the storage
• iSCSI terminology
• Connecting a host to an iSCSI storage
• Alternative methods of discovering iSCSI targets
• Multiple protocol access to SCSI devices
• Physical networking considerations when using iSCSI
• Logical networking consideration when using iSCSI

5.1.1 What is iSCSI

iSCSI (Internet Small Computer Systems Interface) is an IP based transport protocol for SCSI. It allows
computers to connect to their SAN storage array without additional, dedicated Fibre Channel SAN
adapters, thus reducing the overall cost of the infrastructure.
When iSCSI configuration is in use, the host opens a TCP session that serves as a path for SCSI
commands from the host initiator to the storage target. Each of these TCP sessions is therefore
equivalent to a single Initiator-target path that would be created using zoning in a Fibre Channel
environment.
An iSCSI session can contain multiple TCP connections within a single session (called MC/S), which can
help in high latency network.
Since InfiniBox in optimized for use within LAN speeds and MC/S has an administrative overhead,
InfiniBox implements a one-ot-one relationship between connections and sessions to keep the
configuration simple, and so the words ‘session’ and ‘connection’ are used in this document
interchangeably.

Discovery
Before the host can open a connection to a storage array, it needs to first "learn" about the storage.
There are 2 methods of discovery:

• Static discovery - the user manually provides one of the target hostname / IP addresses. The host
will then connect to this IP to query the storage for its details (more below)
• Dynamic discovery - the host receives a hostname / IP address for an iSNS server which acts as a
mediator: The iSNS receives the configuration from the storage periodically, and provides it to the
host whenever the asked.

iSCSI connectivity  –  76

 InfiniBox 4.0.10 User Documentation

Dynamic discovery allows the host to discover multiple storage arrays at once, as well as receive
updates on new storage array as they are added to the environment.
Regardless of the discovery method, the host will get a lot of information about the target, such as the
list of IP addresses it can use for accessing the data and the preferred IO sizes andthe authentication
requirements. In addition, the host is required to authenticate and provide digests (checksums) for the
data.

Note that a complete manual configuration is also supported, but is not a common practice.

Connecting to the storage

At the end of the discovery, the session is closed and the host it not yet connected to the storage.
Connecting to the storage will create multiple TCP sessions from the host to the target IPs (1 session
per intiator-target pair is the default), which will allow IO to be sent / received. This is known as the "Full
Feature Phase" of the connection.

At this stage the host treats all iSCSI connections as it would Fibre Channel paths. The host starts
sending SCSI commands over these paths, and gets the list of LUNs available to the initiator(s). Further,
the host sends query commands on each path to get the details of each LUN. These details will be used
by the multipath driver to identify the group of devices as multiple paths to the same device.

5.1.2 iSCSI terminology

Term Description

RFC3720 / The iSCSI standards (original and updated)

RFC7143

iSCSI SCSI (Internet Small Computer System Interface) over TCP/IP

iQN iSCSI initiator identifier (iSCSI Qualified)

• Software initiator - an iQN per host

• Hardware initiaor - an iQN per HBA 

Session A connection between the iSCSI intiator and an iSCSI target (IP address)

a single I_T Nexus domain, connecting 1 Initiator to 1 target device

Multiple sessions An iSCSI host connects to all the target IPs in the Network space using a separate
session per IP (see What is a network space).

iSCSI connectivity  –  77

 InfiniBox 4.0.10 User Documentation

Term Description

CHAP Challenge Handshake Authentication Protocol, used for iSCSI authentication.

Types of CHAP authentication:

• Inbound - Initiator authenticated by the target in order to gain access to

the target
• Mutual - The target is also authenticated by the initator (after Inbound
authentication)

Digest  Additional CRC added to the iSCSI message in order to guarantee it arrived without
corruption.
There are separate digests for the header and data payload. 

ErrrorRecoveryLev Defines how the Initiator and target recover from errors.
el (ERL) 
InfiniBox supports ERL0

iSNS Internet Storage Name Service.

Allows clients to list targets in the network (Similar to a DNS service only for iSCSI
discovery).

5.1.3 Connecting a host to an iSCSI storage

The steps for connecting a host to an iSCSI storage (target) are:

1. installing an iSCSI initiator on the host (this can be a software initiator that comes with the OS or
an iSCSI HBA)
2. Discovering one of the iSCSI storage targets (IP addresses). 
This is an automated process where the host collects information from the storage. After the
host gets all the information, it closes the discovery session.
3. Connecting to all target IP addresses (a.k.a. full feature phase)
The host opens multiple TCP connections to the  iSCSI targets, commonly each of these
connections has a separate session, and each session is treated as a path to the storage.

At the end of this process, the connectivity between the host and storage may look like this:

iSCSI connectivity  –  78

 InfiniBox 4.0.10 User Documentation

Once the host completes the connection it can scan for SCSI devices and discover the LUNs it has
access to.

Alternative methods of discovering iSCSI targets

If iSNS is used, the host may discover multiple storage systems at once using a 3rd party iSNS server.

• iSNS serves as a directory where all iSCSI targets are registered and the clients can discover their
IP addresses in a single query.
• Once the host discovers the target, it follows a similar path to connect to the iSCSI target as
explained above.

5.1.4 Multiple protocol access to SCSI devices

InfiniBox allows hosts to access block devices over both iSCSI and FC protocols. InfiniBox support hosts
that have both Fibre Channel and iSCSI paths to LUNs.

This is a fast way to migrate between the 2 protocols, by allowing the same host to first connect using
another protocol and only then removing the paths of the old protocol.

iSCSI connectivity  –  79

 InfiniBox 4.0.10 User Documentation

5.1.5 Physical networking considerations when using iSCSI

InfiniBox allows to create a dual redundant network for multiple services. One such dual redundant
network serves iSCSI hosts. As It is a common best practice to avoid using multiple-port LACP when
connecting iSCSI hosts, setting the iSCSI network to be connected separately from all other networks,
allows iSCSI to utilize multi-pathing to leverage all available paths to the storage.

A separate, Single-port LACP is created on top of the physical network. In the following illustration, 2
switches are connected to InfiniBox. The infiniBox ports of each of the nodes serve either an FC switch
or an iSCSI switch.

• FC switch - connected to nodes 1 and 2

• Ports that connect to this switch a grouped in LACPs
• iSCSI switch - connected to node 3 only

5.1.6 Logical networking consideration when using iSCSI

The logical components that form the iSCSI service are seprated from logical components that serve
other networks.

iSCSI connectivity  –  80

 InfiniBox 4.0.10 User Documentation

5.2 iSCSI setup quick-start

Available on this document:

• InfiniBox 4.0 Best Practices Guide for Setting Up Services.pdf (the link takes you to the Support
site article where the PDF is available) 

The instructions that were provided for previous releases are obsolete.

5.3 iSCSI connectivity limits and limitations

• Limits
• Limitations

iSCSI connectivity  –  81

 InfiniBox 4.0.10 User Documentation

5.3.1 Limits
• Maximum supported host ports (iQNs): 2000

5.3.2 Limitations
• Authentication - only CHAP and Mutual CHAP are supported
• IP addresses - only IPv4 is supported
• IPSEC is not supported
• MCS (Multiple Connections per Session) - is not supported
• NetWork Space exclusivity - iSCSI requires a separate Network Space 

iSCSI connectivity  –  82

 InfiniBox 4.0.10 User Documentation

6 Hosts and clusters

6.1 Overview of hosts

• InfiniBox host connectivity
• Operations available on hosts
• Host PowerTools
• Host PowerTools compatibility and supported operating systems
• Best practices for integrations
• Connecting a host without using Host PowerTools
• Connecting a cluster of hosts to InfiniBox

6.1.1 InfiniBox host connectivity

A host is an entity that represents a physical, or a virtual host that volumes can be mapped to. InfiniBox
can be attached to various host systems by using its FC ports. Each InfiniBox node has 8 FC ports (a
total of 24 ports on the system). The host is a server that runs an operating system that is recognized by
InfiniBox and to which InfiniBox provides storage services.

6.1.2 Operations available on hosts

The following operations are available on hosts:

• Creating a host from InfiniBox

• Creating a host using Host PowerTools: see the documentation for each Host PowerTools
package further down this topic.
For example, for Linux, see:
• Host PowerTools installation instructions
• Provisioning volumes using Host PowerTools
• Adding a port to a host
• Mapping a volume to a host or a cluster
• Unmapping a volume from a host or a cluster
• Viewing the host connectivity status
• Setting the host security properties
• Deleting a host

Hosts and clusters  –  83

 InfiniBox 4.0.10 User Documentation

6.1.3 Host PowerTools

In order to attach the host to InfiniBox, there is a need to install an HostPowerTools package. More
information about Host PowerTools is available on the Host PowerTools section on the Support site:

• Host PowerTools for Linux and Windows

• Host PowerTools for VMware
• InfiniBox Storage Replication Adapter
• InfiniBox OpenStack Cinder Volume Driver
• INFINIDAT VSS Hardware Provider
• And many more

Host PowerTools compatibility and supported operating systems

This Support article specifies:

• InfiniBox compatibility with other INFINIDAT products

• Host PowerTools compatibility with operating systems
• Host PowerTools compatibility with VMware Products
• INFINIDAT OpenStack Cinder Plug-in compatibility with OpenStack
• And many more

This Support article provides an Interoperability Matrix.

Best practices for integrations

This support article specifies best practices for integration with SAP HANA, IBM SVC, and SQL Server.

Connecting a host without using Host PowerTools

Instructions are available on this Support site article.

6.1.4 Connecting a cluster of hosts to InfiniBox

A host cluster is a group of independent hosts that are working together as a single system to provide
high availability of services for clients. When a failure occurs on one host in a cluster, resources are
redirected and the workload is redistributed to the other hosts in the cluster. 
Host clusters are designed for applications that have long-running in-memory state or frequently
updated data. 

The hosts in the cluster share the same volumes mapping.  

• Creating a cluster

Hosts and clusters  –  84

 InfiniBox 4.0.10 User Documentation

• Adding a host to a cluster

• Mapping a volume to a host or a cluster

6.2 Creating a host

• About this task
• Prerequisites
• GUI instructions
• InfiniShell instructions
• See also

6.2.1 About this task

This task creates a host on the InfiniBox system.

6.2.2 Prerequisites
• Ports (iSCSI or FC, as needed)

6.2.3 GUI instructions

1. Click the Hosts & Clusters icon on the toolbar at the left of the InfiniBox Management Console
screen.
2. Select Host from the drop-down list at the right of the Create button.
The Create Host screen opens.
• Alternately, click the Hosts tab and click Create. 
3. Provide a name to the host.
4. Add ports (This is optional. The ports can be added later).
a. Select a Protocol Type for the port that will be added (FC or iSCSI). 
b. Provide ports to the host. 
i. For FC, choose between the following options:
1. Select from a list of unassigned ports
2. Insert the port manually
ii. For iSCSI, insert the IQN manually
5. Click Create.
The host is created. 

Hosts and clusters  –  85

 InfiniBox 4.0.10 User Documentation

6.2.4 InfiniShell instructions

Use the following commands:

• host.create - to create a port

• host.add_port - to add ports to the host
• host.initiator_query - to query for available initiators

6.2.5 See also

• Querying for mappings and for unmapped initiators

6.3 Adding a port to a host

• About this task
• Adding a port using Host PowerTools
• Prerequisites
• GUI instructions
• InfiniShell instructions

6.3.1 About this task

This task adds a port to a host from the InfiniBox GUI.

6.3.2 Adding a port using Host PowerTools

On the host side, the ports are automatically added and discovered. See "Registering the system"
on: Host PowerTools installation instructions.

6.3.3 Prerequisites
• Host
• Initiators - look for available ports: Querying for mappings and for unmapped initiators

6.3.4 GUI instructions

1. Click the Hosts & Clusters icon on the toolbar at the left of the InfiniBox Management Console
screen.
2. Select a Host.

Hosts and clusters  –  86

 InfiniBox 4.0.10 User Documentation

3. Click the Ports tab.

4. Click Add port
5. Select a Protocol Type 
6. Provide ports to the host 
a. For FC, you can choose between the following options:
i. Select from a list of unassigned ports
ii. Insert the port manually
b. For iSCSI, insert the IQN manually
7. Click Add.
The port is added. 

6.3.5 InfiniShell instructions

Use the following commands:

• host.add_port - to add ports to the host

• host.initiator_query - to query for available initiators

6.4 Mapping a volume to a host or a cluster

• About this task
• Prerequisites
• Before you begin
• Instructions
• Mapping a volume to a host or a cluster
• Mapping the host or cluster to a volume
• Querying for the mappings

6.4.1 About this task

This task maps a volume to a host in order to allow the host to access the volume.
INFINIDAT simplifies the mapping of volumes to hosts as much as possible. The LUN number assigned
for each mapping is insignificant. Its only role is to maintain the uniqueness of the volume - host
mapping, so as long as it works, InfiniBox does not bother the user with it and assigns the LUNs
automatically.
The InfiniBox RESTful management API for mapping a volume to host or cluster accepts a LUN number
as an optional parameter. If the parameter is not specified, the LUN is automatically assigned by the
system. (This is also the case with our VMware plugin.)
LUN numbers are automatically assigned as follows:

Hosts and clusters  –  87

 InfiniBox 4.0.10 User Documentation

• For host - next available number, starting from 1. 

• For cluster - next available number, starting from 11.

The reason we start assigning LUNs to a cluster starting with 11 is to avoid collision with LUNs mapped
to individual hosts within the cluster. As long as no more than 10 volumes are mapped to a host
individually, there is no collision between volumes mapped individually and volumes mapped to the
cluster. In the rare case that a host has more than 10 volumes mapped only to it, and it also belongs to
a cluster, the LUN number assignment has to be set manually.
The InfiniBox management GUI and CLI support setting the LUN number manually (in addition to the
default behavior that is explained above) by the user when mapping a volume to host or cluster.

6.4.2 Prerequisites
• A volume
• A host

6.4.3 Before you begin

• This task can be carried out from the host point of view or from the volume point of view.
• You may map the volume to a cluster (a cluster is a group of hosts that share the same storage
resources)

6.4.4 Instructions

Mapping a volume to a host or a cluster

1. From infinishell:
a. Run: vol.map name=v1 host=h1
or
Run: vol.map name=v1 cluster=c1 

2. From the GUI:

a. Go to the Volumes screen.
b. Right-click a volume and select Map Volume from the pop-up menu.
c. Select a host or a cluster from the list and click Map.

Mapping the host or cluster to a volume

1. From infinishell:

Hosts and clusters  –  88

 InfiniBox 4.0.10 User Documentation

a. Run: host.map name=h1 volume=v1

or
Run: cluster.map name=c1 volume=v1 

2. From the GUI:

a. Go to the Hosts and Clusters screen
b. Click either the Hosts tab or the Clusters tab
c. For host
i. Right-click a host and select Map Host from the pop-up menu
ii. Select a volume from the list and click Map
iii. Note: this GUI option allows to create a new volume
3. For cluster
a. Right-click a cluster and select Map Cluster from the pop-up menu
b. Select a volume from the list and click Map
c. Note: this GUI option allows to create a new volume

6.4.5 Querying for the mappings

Use either of the following options:

• From infinishell:
• vol.map_query
• host.query
• cluster.query
• map.query
• From the GUI:
• View the volume, host, or cluster screens and look at the Mapping column

6.5 Unmapping a volume from a host or a cluster

• About this task
• Prerequisites
• Before you begin
• Instructions
• Unmapping a volume from a host or a cluster
• Unmapping the host or cluster from a volume

Hosts and clusters  –  89

 InfiniBox 4.0.10 User Documentation

6.5.1 About this task

This task unmaps a volume from a host or a cluster.

6.5.2 Prerequisites
• A volume mapped to a host or a cluster

6.5.3 Before you begin

• This task can be carried out from the host point of view or from the volume point of view

6.5.4 Instructions

Unmapping a volume from a host or a cluster

1. From infinishell:
a. Run: vol.unmap name=v1 host=h1
or
Run: vol.unmap name=v1 cluster=c1 

2. From the GUI:

a. Go to the Volumes screen.
b. Right-click a volume and select Modify Mapping from the pop-up menu.
c. Select a host or a cluster from the list and click Apply.

Unmapping the host or cluster from a volume

1. From infinishell:
a. Run: host.unmap name=h1 volume=v1
or
Run: cluster.unmap name=c1 volume=v1 

2. From the GUI:

a. Go to the Hosts and Clusters screen
b. Click either the Hosts tab or the Clusters tab
c. For host
i. Click a host
ii. Select a volume from the Mapped Volume tab

Hosts and clusters  –  90

 InfiniBox 4.0.10 User Documentation

iii. Click Unmap
d. For cluster
i. Click a cluster
ii. Select a volume from the Mapped Volume tab

6.6 Setting the host security properties

• About this task
• Before you begin
• Prerequisites
• GUI instructions

6.6.1 About this task

This task sets the security properties of an iSCSI host.

6.6.2 Before you begin

Read here on the available types of CHAP security: Overview of iSCSI connectivity.

6.6.3 Prerequisites
• iSCSI host

6.6.4 GUI instructions

1. Click the Hosts & Clusters icon on the toolbar at the left of the InfiniBox Management Console
screen.
2. Select a Host.
3. Select iSCSI Security from the Actions menu.
4. Select between:
a. None - the property that is set to the host upon creation
i. Check whether to clear previously set users and passwords
b. CHAP - an Inbound only security setting
i. Set the User
ii. Set the Target Secret - optional: click Auto Generate to generate a 16-digits secret
c. Mutual - a both Inbound and Outbound security setting

Hosts and clusters  –  91

 InfiniBox 4.0.10 User Documentation

i. Set pairs of user and secret for the target and the initiator
5. Click Modify.
The host security properties are set. 

6.7 Deleting a host

• About this task
• Prerequisites
• GUI instructions
• InfiniShell instructions

6.7.1 About this task

This task deletes a host on the InfiniBox system.

6.7.2 Prerequisites
• A host

6.7.3 GUI instructions

1. Click the Hosts & Clusters icon on the toolbar at the left of the InfiniBox Management Console
screen.
2. Select a host
3. Remove all of the host mappings
4. Right-click the host and select Delete Host 
a. The host is deleted

6.7.4 InfiniShell instructions

Use the following command:
• host.delete name=h1

Hosts and clusters  –  92

 InfiniBox 4.0.10 User Documentation

6.8 Clusters

6.8.1 Cluster commands

• Creating a cluster
• Description - Create a cluster by setting its name.
• Example - cluster.create name=cluster1

• Adding a host to a cluster

• Description - Add a host to a cluster. You can add up to 32 hosts to a cluster.
• Example - cluster.add_host name=cluster1 host=host1

• Deleting a cluster
• Description - Delete a cluster
• Command - cluster.delete name=cluster1
• Querying for clusters
• Description - List existing clusters with their member hosts. This command displays the
number of hosts and LUNs for each cluster, along with the cluster creation date and the
last time it was modified.
• Example - cluster.query
• Mapping a dataset to a cluster
• Description - Map a volume, or a snapshot, to a cluster
• Command - cluster.map
• Querying for cluster mappings
• Description - list existing cluster mappings
• Command - cluster.map_query
• Removing a host from a cluster
• Description - remove a host from a cluster
• Command - cluster.remove_host
• Renaming a cluster
• Example - cluster.rename name=cluster1 new_name=cluster2
• Unmapping a host 
• Description - Unmap a dataset from the cluster
• Example - cluster.unmap name=cluster1 vol=vol1

Hosts and clusters  –  93

 InfiniBox 4.0.10 User Documentation

6.9 Querying for mappings and for unmapped initiators

• Querying for mapping
• Logged-in initiators

6.9.1 Querying for mapping

• map.query - you can view results for hosts only (without clusters) by using the
host.map_query command.

6.9.2 Logged-in initiators

• host.initiator_query - When an initiator is zoned to InfiniBox, it is automatically identified
and made available to be assigned to a host (the InfiniBox entity). Thus, adding a port to a host is
done via a list of available initiators, in addition to the manual option to insert the port WWPN.

6.10 Creating a cluster

• About this task
• Prerequisites
• GUI instructions
• InfiniShell instructions
• See also

6.10.1 About this task

This task creates a cluster on the InfiniBox system. The cluster can be populated with hosts later on.

6.10.2 Prerequisites
• FC ports (see here: Querying for FC ports)

6.10.3 GUI instructions

1. Click the Hosts & Clusters icon on the toolbar at the left of the InfiniBox Management Console
screen.
2. Select Cluster from the drop-down list at the right of the Create button.
The Create Cluster screen opens.

Hosts and clusters  –  94

 InfiniBox 4.0.10 User Documentation

• Alternately, click the Clusters tab and click Create. 

3. Provide a name to the cluster. 
4. Click Create.
The cluster is created. 

6.10.4 InfiniShell instructions

Use the following commands:

• cluster.create

6.10.5 See also

• Adding a host to a cluster

6.11 Adding a host to a cluster

• About this task
• Prerequisites
• GUI instructions
• InfiniShell instructions

6.11.1 About this task

This task adds a host to a cluster.

6.11.2 Prerequisites
• Cluster - see: Creating a cluster
• Host - see: Creating a host

6.11.3 GUI instructions

1. Click the Hosts & Clusters icon on the toolbar at the left of the InfiniBox Management Console
screen.
2. Select a cluster.
3. Click the Hosts tab.
4. Click Add Host.
a. Select a host from the list.

Hosts and clusters  –  95

 InfiniBox 4.0.10 User Documentation

b. Click Add.
The host is added to the cluster.

6.11.4 InfiniShell instructions

Use the following commands:

• cluster.add_host

6.12 Viewing the host connectivity status

• About this task
• GUI instructions
• InfiniShell instructions

6.12.1 About this task

This task displays the connectivity status of a host that is connected to InfiniBox. The may have either of
the following statuses:

• Connected - the host is 

• Degraded - the host is connected to some InfiniBox nodes, but not to all of them
• Non-resilient - connected to one node only
• Disconnected - the host is connected to none of the InfiniBox nodes

6.12.2 GUI instructions

1. Click the Hosts & Clusters icon on the toolbar at the left of the InfiniBox Management Console
screen.
An overview of the connectivity state of hosts is available on this screen:

 
2. Select a Host.
3. Click the Ports tab.

Hosts and clusters  –  96

 InfiniBox 4.0.10 User Documentation

a. A connected host looks like this:

 
Click the port to see further details:

 
b. A degraded host looks like this:
c. A disconnected host looks like this:

6.12.3 InfiniShell instructions

Use the following command:

• host.port_query

Hosts and clusters  –  97

 InfiniBox 4.0.10 User Documentation

6.13 Mapping more than 32 hosts to a volume

• About this task
• See also
• GUI instructions
• InfiniShell instructions

6.13.1 About this task

This task maps more than 32 hosts to the same volume.

This task overcomes the limit of 32 hosts for a single cluster by mapping more than a single cluster to
the same volume.

6.13.2 See also

• Adding a host to a cluster

6.13.3 GUI instructions

1. Create as many clusters as needed, as follows:
a. Cluster 1 - with up to 32 hosts
b. Cluster 2 - with up to 32 hosts
c. More clusters - as needed
2. Click one of the cluster. On the Mapped Volumes tab (that is displayed on the screen by default),
click Map Volume, select the volume and click Map. The cluster is mapped to the volume.

Hosts and clusters  –  98

 InfiniBox 4.0.10 User Documentation

3. On the same screen, click the volume and select Modify Mapping from the action menu.

4. On the Mapping screen that opens, click the action menu and check Enable Multiple Hosts
Selection. As a result, the other clusters are available on the screen.

Hosts and clusters  –  99

 InfiniBox 4.0.10 User Documentation

5. Select the other cluster (or clusters), click Map and then Apply.

6. Approve the dangerous operation warning.

The volume is now mapped to all of the selected clusters.

The mapping is visible from:

• Each of the mapped clusters

• The volume's screen

6.13.4 InfiniShell instructions

1. Use vol.map to map each of the clusters to the volume
2. Use vol.map_query to view the mapping 

vol.map_query vol=vol1
NAME MAPPING TYPE MAPPED TO LUN ID
vol1 CLUSTER cluster1 11
vol1 CLUSTER cluster2 12

Hosts and clusters  –  100

 InfiniBox 4.0.10 User Documentation

7 Network connectivity and configuration

7.1 Ethernet ports connectivity

• About this topic
• Setup
• Ports
• Chart

7.1.1 About this topic

The following chart displays the way the InfiniBox TCP/UDP ports are connected to the various services.

Setup
The ethernet network is set up by INFINIDAT.

Ports
• Replication
• In: 80, 443
• Out: 8067
• Management: 80, 443
• NAS  
• Portmapper: 
• TCP/111 
• UDP/111 
• Mount: 
• TCP/20048 
• UDP/20048
• NFS: 
• TCP/2049
• iSCSI
• iSCSI: 3260
• ISNS: 3205

Network connectivity and configuration  –  101

 InfiniBox 4.0.10 User Documentation

Chart

7.2 Network connectivity glossary

Node An InfiniBox server. Typically, there are 3 nodes in a system.

Ethernet Port Physical Ethernet port (copper / optical).  

(Port)

Network One or more Ethernet Ports that are grouped together for the purpose of redundancy
Interface and/or performance using LACP.

Ethernet
Interface Partial A failure of 1 or more Ethernet Ports which are part of an Ethernet
failure modes failure Interface where at least one port remains available.

Complete A failure of all Ethernet Ports in an Ethernet Interface.

failure

Service A network data service, e.g. NFS, Replication

IP address A group of floating IP addresses that will offer the service (Each IP offers 1 service). It’s
recommended that each Network Space will have 6 IP Addresses to ensure even
workload distribution.

Network connectivity and configuration  –  102

 InfiniBox 4.0.10 User Documentation

Network A set of definitions for a network address space, annotated using CIDR. Example:
10.0.0.0/24

7.2.1 Network space diagram

Network space name NFS_Space ASYNC_Space

Service type NFS Remote Replication

IP address 192.9.200.0/24 10.0.0.0/24

Default Gateway 192.9.200.1 10.0.0.1

IP addresses 192.9.200.10 10.0.0.0.21

192.9.200.11 10.0.0.22

192.9.200.12 10.0.0.23

192.9.200.13 10.0.0.24

192.9.200.14 10.0.0.25

192.9.200.15 10.0.0.26

7.3 FC and Ethernet network diagrams

• About this page
• Ethernet network diagram
• FC ports diagram

7.3.1 About this page

This page provides diagrams and images of the patch panel and ports connectivity for both FC and
Ethernet.

Network connectivity and configuration  –  103

 InfiniBox 4.0.10 User Documentation

7.3.2 Ethernet network diagram

This diagram displays the Ethernet ports to which InfiniBox attaches its logical entities.

7.3.3 FC ports diagram

The following image shows the 8 available FC ports on node 1. 

7.4 Creating a network interface

• About this task
• Before you begin
• Querying for Ethernet ports on the GUI
• Querying for Ethernet ports on InfiniShell
• GUI Instructions
• Following the completion of this task
• Related tasks

Network connectivity and configuration  –  104

 InfiniBox 4.0.10 User Documentation

7.4.1 About this task

This task creates a network interface using the InfiniBox Ethernet ports. The network interface is later
on used for creating a network space.

7.4.2 Before you begin

Query for the InfiniBox Ethernet ports. The following ports are required for creating a network
interface:

• Management (for a replication network space)

• 1 Management port
• 1 IP address
• Firewall ports: 80, 443
• Data (for all network spaces)
• 3 to 6 (3 * 1 to 2) ports
• 3 to 6 IP addresses
• Firewall port: 8067

Querying for Ethernet ports on the GUI

1. Click the GUI Settings icon and then go to the Network Interfaces tab.
The system's Ethernet ports are displayed on screen. 

Querying for Ethernet ports on InfiniShell

1. Run config.ethernet.port.query to view the system's ports. 

7.4.3 GUI Instructions

1. Click the GUI Settings icon and then go to the Network Interfaces tab. 
2. Click Create.
Create Ethernet Interface screen opens.
3. Provide a name to the interface, select the ports that will be associated with the interface and
determine whether to allow the interface to use ports from all three nodes.

Network connectivity and configuration  –  105

 InfiniBox 4.0.10 User Documentation

 
Click Create.
The interface is created and is visible on the screen.
4. Repeat these steps to create more interfaces. 

7.4.4 Following the completion of this task

You can now create network spaces.

7.4.5 Related tasks

This section describes commands that create, modify, enable and disable and delete an Ethernet
interface.

• config.ethernet.interface.create - This command creates interface

• config.ethernet.interface.enable - This command enables an interface
• config.ethernet.interface.disable - This command disables an interface
• config.ethernet.interface.query - This command lists the interfaces that are defined in the
system. It can be filtered by interface name, and by node
• config.ethernet.interface.rename - This command renames an interface
• config.ethernet.interface.add_port - This command adds a port to an interface

Network connectivity and configuration  –  106

 InfiniBox 4.0.10 User Documentation

• config.ethernet.nterface.remove_port - This command removes a port from an interface

• config.ethernet.interface.set_rate_limit - This command limits the available bandwidth
that the interface can utilize

The commands are displayed in length, including examples on the InfiniShell Reference Guide.

7.5 Enabling a network interface

• About this task
• Prerequisites
• GUI Instructions
• InfiniShell instructions

7.5.1 About this task

This task enables a network space. A network interface may be disabled as a prerequisite for deleting
the interface.

However, you can enable a disabled interface. 

7.5.2 Prerequisites
• A network interface

7.5.3 GUI Instructions

Assuming that the network interface is disabled:

1. Select a network interface and click Enable Interface on the Actions menu.
The interface is now enabled.

7.5.4 InfiniShell instructions

Use the following command:

• config.ethernet.interface.enable - to enable the interface

7.6 Disabling a network interface

• About this task
• Prerequisites

Network connectivity and configuration  –  107

 InfiniBox 4.0.10 User Documentation

• GUI Instructions
• InfiniShell instructions

7.6.1 About this task

This task disables a network space. Disabling a network space is a prerequisite for deleting the
interface.

You can enable a disabled interface. 

7.6.2 Prerequisites
• A network interface

7.6.3 GUI Instructions

1. Select a network interface and click Disable Interface on the Actions menu.
2. Approve the warning message.
The interface is now disabled.

7.6.4 InfiniShell instructions

Use the following command:

• config.ethernet.interface.disable - to disable the interface

7.7 Modifying a network interface

• About this task
• Before you begin
• GUI Instructions
• Related tasks

7.7.1 About this task

This task adds ports to the network space.

7.7.2 Before you begin

To access the network interfaces:

Network connectivity and configuration  –  108

 InfiniBox 4.0.10 User Documentation

• Click the GUI Settings icon and then go to the Network Interfaces tab.

7.7.3 GUI Instructions

1. Select a network interface and click Modify Interface on the Actions menu.

Modify Ethernet Interface screen opens.

2. Modify the interface as follows:
a. Rename the interface
b. Add ports to the interface. Note that if the interface uses only one node, you can add ports
that belong to this node, but you cannot add ports that belong to the other nodes.
3. Click Modify.
The screen closes, and the interface is modified.

7.7.4 Related tasks

To remove ports, or to set the interface rate limit, use InfiniShell:

• config.ethernet.interface.remove_port - This command removes a port from an interface

7.8 Deleting a network interface

• About this task
• Before you begin
• GUI Instructions
• InfiniShell instructions

7.8.1 About this task

This task deletes a network space. In order to better service your storage needs, you may want to
reorganize your network spaces. To do so, you may want to recreate network interfaces and to
reallocate Ethernet ports to them.  

Network connectivity and configuration  –  109

 InfiniBox 4.0.10 User Documentation

7.8.2 Before you begin

• The interface has to be disabled.
• To disable the interface, see the instructions below.
• To access the network interfaces:
• Click the GUI Settings icon and then go to the Network Interfaces tab.

7.8.3 GUI Instructions

1. Select a network interface and click Disable Interface on the Actions menu.
2. Approve the warning message.
The interface is now disabled.
3. Select the network interface again and click Delete Interface on the Actions menu.
The interface is deleted.

7.8.4 InfiniShell instructions

Use the following commands:

• config.ethernet.interface.query - to see what interfaces are defined on InfiniBox

• config.ethernet.interface.disable - to disable the interface
• config.ethernet.interface.delete - to delete the interface

7.9 What is a network space

InfiniBox provides storage administrators with means to configure the external connectivity of the
system. Network connectivity is a prerequisite for using features such as Asynchronous Remote
Replication and Network File System (NFS).

Storage administrators configure their storage system to integrate with network hardware (switches,
routers) and with network services (e.g. NFS and Replication). In order to simplify management and
provisioning processes, InfiniBox Network Spaces was created. It is a logical construct that groups the
required networking resources and the configuration required to create a Service, for example,
Replication.

7.10 Creating a network space

• About this task
• Prerequisites
• Before you begin

Network connectivity and configuration  –  110

 InfiniBox 4.0.10 User Documentation

• Instructions
• Related tasks

7.10.1 About this task

This task creates a network space using the GUI wizard. 

• Network Space - the definition of a specific data service (NAS / replication / iSCSI) within one
specific subnet. 

This task is a prerequisite for:

• Creating a link
• Creating a replication 
• Deploying a service (for example: NAS)

For multi-InfiniBox services (Replication), repeat this task for each InfiniBox.

7.10.2 Prerequisites
• Ports:
• Data
• 1 or more ports per node (preferably of the same configuration across all nodes)
• IP addresses
• iSCSI - 6
• NAS - 6
• Asynchronous replication - 4
• Synchronous replication - 7 
• Firewall port: 8067 (for asynchronous replication only)
• Management (required only for Replication network space)
• 1 Management port
• 1 IP address
• Firewall ports: 80, 443
• Network interface

7.10.3 Before you begin

• On the GUI, click the Settings icon on the toolbar on the left, and then click the Network Spaces
tab. 

Network connectivity and configuration  –  111

 InfiniBox 4.0.10 User Documentation

7.10.4 Instructions
1. Click Create on the Network Space pane. The Create Network Space screen opens.
2. The wizard is comprised on two panes: Ethernet Interfaces and IP Configuration. 
Fill in the Network Name and select a Service. 

3. The wizard takes into consideration previously created Interfaces. 

Network connectivity and configuration  –  112

 InfiniBox 4.0.10 User Documentation

a. If you need to create an Interface, click the Create Interfaces button. 

Network connectivity and configuration  –  113

 InfiniBox 4.0.10 User Documentation

b. Allocate Ethernet ports for the Interface, and click Create.

The Interface is created.

c. Click Next to move to the IP Configuration screen.

4. Type the IP addresses of the network, netmask, default gateway, management and data.
Type Enter or click Add for each management and data IP address.
Note that the IP addresses list contains 7 IPs (1 management, 6 data) but only 6 are visible on

Network connectivity and configuration  –  114

 InfiniBox 4.0.10 User Documentation

screen (scroll the IP addresses pane to view them all).

5. Click Finish. The Network Space is created and is visible on the screen.

7.10.5 Related tasks

• Creating a network interface.

7.11 Creating a network space on InfiniBox 2.0.x

• About this task
• Prerequisites

Network connectivity and configuration  –  115

 InfiniBox 4.0.10 User Documentation

• Before you begin

• Instructions
• Related tasks

7.11.1 About this task

This task creates a network space using the GUI wizard. In this task we define:

• Interface - One or more Ethernet Ports that are grouped together for the purpose of
redundancy and/or performance using LACP.
• Network Space - A set of definitions for a network address space, annotated using CIDR.

This task is a prerequisite for:

• Creating a link
• Creating a replication 

For multi-InfiniBox services (Replication), repeat this task for each InfiniBox.

7.11.2 Prerequisites
• Management
• 1 Management port
• 1 IP address
• Firewall ports: 80, 443
• Data
• 3 to 6 (3 * 1 to 2) ports
• 3 to 6 IP addresses
• Firewall port: 8067

7.11.3 Before you begin

On the GUI, click the Settings icon on the toolbar on the left, and then click the Network tab. The
Ethernet ports that are visible at the bottom-left of the screen are the physical ports of the InfiniBox

Network connectivity and configuration  –  116

 InfiniBox 4.0.10 User Documentation

system.

7.11.4 Instructions
1. Click Create on the Network Space pane. The Create Network Space screen opens.
2. The wizard is comprised on two panes, Ethernet Interfaces and IP Configuration. 
Fill in the Network Name and select a Service. 

3. The wizard takes into consideration previously created Interfaces. 

Network connectivity and configuration  –  117

 InfiniBox 4.0.10 User Documentation

a. If you need to create an Interface, click the Create Interfaces button. 

Network connectivity and configuration  –  118

 InfiniBox 4.0.10 User Documentation

b. Allocate Ethernet ports for the Interface and click Create.

The Interface is created.

c. Click Next to move to the IP Configuration screen.

Network connectivity and configuration  –  119

 InfiniBox 4.0.10 User Documentation

4. Type the IP addresses of the network, netmask, default gateway, management and data.
Type Enter or click Add for each management and data IP address.

5. Click Finish. The Network Space is created and is visible on screen.

7.11.5 Related tasks

For creating a network space using InfiniShell, see: Creating a network interface.

7.12 Adding an IP address to a network space

• About this task
• Prerequisites
• Before you begin
• InfiniShell instructions
• GUI instructions

Network connectivity and configuration  –  120

 InfiniBox 4.0.10 User Documentation

7.12.1 About this task

This task adds an IP address to a network space.

7.12.2 Prerequisites
This task assumes that the following entity is already defined:

• Network space

7.12.3 Before you begin

On the GUI, click the Settings icon on the toolbar on the left, and then click the Network Spaces tab. 

7.12.4 InfiniShell instructions

Use the following InfiniShell command:

• config.net_space.ip.create

7.12.5 GUI instructions

1. Click the Network Space entity to open it on screen.

Network connectivity and configuration  –  121

 InfiniBox 4.0.10 User Documentation

2. Click Create IP address. The Create IP address screen opens.

 
3.  Insert an IP address and click Create.
The IP address is added to the network space.

7.13 Modifying a network space

• About this task
• Prerequisites
• Before you begin
• InfiniShell instructions
• GUI instructions

7.13.1 About this task

This task modifies the following network space attributes:

• Network space name

• Service - the service that uses the network space
• IP addresses
• Network interfaces

7.13.2 Prerequisites
This task assumes that the following entity is already defined:

• Network space

Network connectivity and configuration  –  122

 InfiniBox 4.0.10 User Documentation

7.13.3 Before you begin

On the GUI, click the Settings icon on the toolbar on the left, and then click the Network Spaces tab. 

7.13.4 InfiniShell instructions

Use the following InfiniShell commands to modify the network space attributes:

• config.net_space.rename
• config.net_space.set_default_gateway   
• config.net_space.set_interfaces
• config.net_space.set_network
• config.net_space.set_rate_limit
• config.net_space.set_service

7.13.5 GUI instructions

1. Click the Network Space Actions menu and select Modify Network Space from the menu.
The Modify Network Space screen opens.

 
2. Modify any of the network space attributes and click Modify. 

7.14 Viewing the network space details

• About this task
• Prerequisites
• Viewing the network space on the GUI 
• Querying for network space details on InfiniShell
• Querying for the network space definitions
• Querying for the network space IP addresses
• Related tasks

7.14.1 About this task

This task lists the various views and queries that provide information on the network space.

Network connectivity and configuration  –  123

 InfiniBox 4.0.10 User Documentation

7.14.2 Prerequisites
This task assumes that the following entity is already defined:

• Network space

7.14.3 Viewing the network space on the GUI 

1. Click the GUI Settings icon and then go to the Network Spaces tab.
All network spaces are listed on the screen.
2. Click a network space to expand its view.

• On the left pane you can see the network space definitions:
• Services
• Interfaces
• Rate limit
• On the right pane you can see the network spaces IP addresses.

7.14.4 Querying for network space details on InfiniShell

Querying for the network space definitions

Run config.net_space.query. The output looks like this:

NAME SERVICE NETWORK RATE LIMIT GATEWAY IPS INTERFACES

default_nas_space NAS 172.16.32.0/19 - - 6 n1eth-data1, n2eth-data1, n3eth-data1

Querying for the network space IP addresses

Run config.net_space.ip.query. The output looks like this:

Network connectivity and configuration  –  124

 InfiniBox 4.0.10 User Documentation

NETWORK SPACE IP ADDRESS ENABLED NODE NETWORK INTERFACE REPLICATION MANAGEMENT IP

default_rmr_space 172.16.63.126 yes 1 n1eth-data1 yes
default_rmr_space 172.16.63.127 yes 1 n1eth-data1 no
default_rmr_space 172.16.63.128 yes 2 n2eth-data1 no
default_rmr_space 172.16.63.129 yes 3 n3eth-data1 no
default_rmr_space 172.16.63.130 yes 1 n1eth-data1 no
default_rmr_space 172.16.63.131 yes 2 n2eth-data1 no

The first IP address is for management and all other IP addresses are for data.

7.14.5 Related tasks

For creating a network space see: Creating a network space.

7.15 Network Spaces commands

This section describes commands that create, modify, enable and disable and delete a network space.

• config.net_space.create - This command creates a new network space on one or all of the
system’s nodes
• config.net_space.query - This command lists network spaces
• config.net_space.set_interfaces - This command assigns interfaces to the network space
• config.net_space.delete - This command deletes a network space
• config.net_space.set_service - This command assigns a network space for a service
• config.net_space.rename - This command renames a network space
• config.net_space.ip.query - This command lists the network space IP addresses
• config.net_space.ip.create - This command creates an IP address – or a range of consecutive
IP addresses – for a network space
• config.net_space.ip.delete - This command deletes an IP address from a network space
• config.net_space.ip.enable - This command enables an IP address of a network space
• config.net_space.ip.disable - This command disables an IP address of a network space

The commands are displayed in length, including examples on the InfiniShell Reference Guide.

7.16 Deleting a network space

• About this task
• Prerequisites
• Before you begin
• InfiniShell instructions

Network connectivity and configuration  –  125

 InfiniBox 4.0.10 User Documentation

• GUI instructions

7.16.1 About this task

This task deletes a network space.

7.16.2 Prerequisites
This task assumes that the following entity is already defined:

• Network space

7.16.3 Before you begin

On the GUI, click the Settings icon on the toolbar on the left, and then click the Network Spaces tab. 

7.16.4 InfiniShell instructions

Use the following InfiniShell commands to modify the network space attributes:

• config.net_space.delete

7.16.5 GUI instructions

1. Click the Network Space Actions menu and select Delete Network Space from the menu. 
2. Delete the Network Space ports in the following order:
a. Disable the data IP addresses
b. Delete the data IP addresses
c. Disable the Management IP address
d. Delete the Management IP address
3. Approve the warning message.
The Network Space is deleted.

7.17 Configuring Network Spaces using VLANs

• Introduction
• Limits
• Limitations
• Viewing the VLANs on the GUI

Network connectivity and configuration  –  126

 InfiniBox 4.0.10 User Documentation

7.17.1 Introduction
A VLAN (virtual LAN) is a logical interface that is configured on top of a physical interface, providing an
additional virtual layer, thus allowing to support more services. Services (NAS, iSCSI, replication) can
either use VLANs or other, non-VLAN interfaces. The data they send is VLAN-tagged - or not tagged -
accordingly.

7.17.2 Limits
• Maximum 200 VLANs per node
Since nodes are configured symmetrically, this also means 200 VLANs per system. 

7.17.3 Limitations
• VLANs are only supported on LACP interfaces that were created on InfiniBox 3.0
• VLANs are not supported on LACP interfaces that were created on InfiniBox versions prior
to 3.0
• VLANs are not supported on ports.
• To enable VLANs on pre-3.0 interfaces, contact support.
• As VLANs inherit their state from the underlying LACP interface, they cannot be disabled
• The name of the VLANs has to be identical across all nodes.

7.17.4 Viewing the VLANs on the GUI

To access the VLANs view:

1. Click Settings from the GUI menu and then click the Network Interfaces tab. Click one of the
Network Interface. The Network Interface screen opens.
2. The Network Interface screen displays the way the Network Interface uses VLANs for the
various InfiniBox services.

Network connectivity and configuration  –  127

 InfiniBox 4.0.10 User Documentation

7.18 Deploying replication services with redundancy

• About this task
• Before you begin
• Physical Deployment
• InfiniBox entities

7.18.1 About this task

Following the discussion on Deployment considerations for services, this article discusses the ability to
achieve redundancy for Replication services.

Before you begin

• You cannot delete an Interface while it is in use

Network connectivity and configuration  –  128

 InfiniBox 4.0.10 User Documentation

7.18.2 Physical Deployment

• There are two stacked switches available
• A port from each node connects to each switch
• Each of the ports of each node forms an LACP (Link Aggregation Control Protocol) (6 in total)

7.18.3 InfiniBox entities

• There are 1 LACP per switch per node, * 2 switches * 3 nodes = 6 LACPs
• There is an interface per LCAP group
• There are two network spaces, each utilizes all of the interfaces of one of the switches
• There are two services, one per network space
• Only one service serves the replication link
• The other service is in stand-by

7.19 Deployment considerations for services

The content of this page was significantly improved. It is now available as a chapter of the Setting Up
Services document:

Network connectivity and configuration  –  129

 InfiniBox 4.0.10 User Documentation

• InfiniBox 4.0 Documentation

7.20 Network routing table

7.20.1 About routing tables

Storage environments often enjoy a rather simple network architecture, and do not require complex
networking settings. For systems that require a more tailored configuration, InfiniBox supports static
routes, configurable per network space.
These could be used to:

• Separate replication traffic from production system A to DR systems B and C, so that each
replication stream will go through a different gateway (spreading the load between them)
• Routing specific IP subnets (E.g. DMZ hosts) through a specific gateway
• Etc.

InfiniBox routes are configured per network space. Since each service requires a network space on each
subnet it is connected to, different subnets can enjoy the flexibility of different routing rules.
However, contradicting rules in different networks are not supported. For example, routing network
space A to destination network X via gateway AX and routing another net space (B) to the same target
network X via gateway BX, is not supported. 

7.20.2 Limits
• Maximum of 16 routes per Network Space
• Maximum of 256 routes per service
• Each route needs to be between CIDR 0 and 29 (0.0.0.0 - 255.255.255.248), smaller subnets are
not supported in this version
• Contradicting routing rules are not supported

7.20.3 Creating a routing table

The routing table can be created either automatically or by the user.

Auto-created routing table

A routing table is automatically added to a network space that has a default gateway. The routing table
includes the following properties: destination network, network address, netmask and gateway IP
address. The routing table does not have an interface property, as it serves all of the properties of the
network space. When the network space gateway is modified or removed, the routing table is modified,
or removed, accordingly.

Network connectivity and configuration  –  130

 InfiniBox 4.0.10 User Documentation

Manually adding a routing table

The routing table can be manually added, modified and removed using the GUI, CLI or API.

7.21 How to change the Replication Network Space IP addresses

• About this task
• Prerequisites
• Before you begin
• GUI instructions

7.21.1 About this task

In order to accommodate the Replication settings to changes in the network, sometimes there is a need
to change the IP addresses the Network Space uses to a different network. 

7.21.2 Prerequisites
This task assumes that the following entity is already defined:

• Network space
• Link
• Replica

7.21.3 Before you begin

Prior to any modification to the Network Space IP addresses, there is a need to:

• Detach the Network Space from the Replication link. See instructions here: Detaching and
attaching a link. 

 Do not delete the link.

Network connectivity and configuration  –  131

 InfiniBox 4.0.10 User Documentation

• Approve the detachment. 

• The replicas will automatically suspend, keeping their initial copy.

7.21.4 GUI instructions

1. Go to Settings, Network Spaces and click the relevant Replication Network Space. Click the IP
Addresses tab. 

2. One by one, disable and delete the data IP addresses.

3. Disable and delete the Management IP address.

Network connectivity and configuration  –  132

 InfiniBox 4.0.10 User Documentation

4. Click the Network Space Actions menu and select Modify Network Space from the menu.
The Modify Network Space screen opens.

5. Modify the Network Space attributes if needed (see relevant instructions here: Modifying a
network space). Edit the Network IP address, Netmask, Default Gateway and MTU if needed.
6. Return to the IP Addresses tab. Add the new IP addresses.
7. Repeat the process, if necessary, on the remote site.
8. If IP addresses were added on the remote site (on step 7), go to the Links screen, select the
relevant link and click Change Remote IP.

Network connectivity and configuration  –  133

 InfiniBox 4.0.10 User Documentation

Enter the IP address of the remote site:

9. Attach the link.

Select the modified Network Space.

Network connectivity and configuration  –  134

 InfiniBox 4.0.10 User Documentation

Confirm the request to refresh the link.

The replica will resume automatically.

Network connectivity and configuration  –  135

 InfiniBox 4.0.10 User Documentation

8 Hardware monitoring

8.1 Overview of hardware monitoring

• InfiniBox hardware
• Hardware queries

8.1.1 InfiniBox hardware

InfiniBox hardware contains no single point of failure. The InfiniBox system reports daily on the state of
its hardware components, thus, maintenance operation are planned in advance.

The state of the InfiniBox hardware components is visible via:

• Events - see Overview of event notification

• Queries
• The Health screen on InfiniBox GUI - see Monitoring the system health

8.1.2 Hardware queries

The following queries are available from the CLI and API (for GUI, see further down this help page):

• CLI
• drive.query
• enclosure.query
• node.query
• node.drive.query
• API
• GET api/rest/components/
• GET api/rest/components/racks/{rackId}/enclosures/{enclosure_index}/drives
• GET api/rest/components/racks/{rackId}/enclosures

8.2 Handling faulty drives

• Introduction
• Flow
• On the customer premise
• Tracking the returned drives
• On INFINIDAT premise

Hardware monitoring  –  136

 InfiniBox 4.0.10 User Documentation

• Drives are always sent to the vendor

8.2.1 Introduction
During the InfiniBox warranty period, faulty drives can either be retained at the customer premise or
returned to INFINIDAT. The returned drives are totally erased, as described in the following process.

8.2.2 Flow

8.2.3 On the customer premise

• A drive is identified as Faulty
• As part of INFINIDAT warranty on InfiniBox parts, INFINIDAT replaces faulty drives
• The drive is ejected from the InfiniBox rack
• As decided during the InfiniBox purchase, faulty drives may either remain at the customer
premise or be returned to INFINIDAT

8.2.4 Tracking the returned drives

• The Drives are managed on our JIRA ticketing system

Hardware monitoring  –  137

 InfiniBox 4.0.10 User Documentation

• The returned drive is marked as Faulty and Returned to INFINIDAT

8.2.5 On INFINIDAT premise

• Any drive that responds (i. e., if it is not dead) goes through a failure analysis and erasure 
• Erasure details: 
• The drive is completely written with a new pattern, hence totally erased 
• One complete WRITE is done to the whole disk space
• RANDOM WRITEs cover an undetermined portion of the disk
• The drive is marked as Faulty and Erased
• A dead drive cannot be erased, as it cannot be written into

8.2.6 Drives are always sent to the vendor

• The drive, either dead or erased, is sent to the vendor

8.3 Monitoring the system health

• About this task
• Visual health indicators
• PDU failure
• BBU warning
• BBU is missing
• One PSU failure
• Both PSU failure
• Internal switch failure
• Enclosure us fed by only one PSU
• Enclosure that is not fed by either PSU
• Node - enclosure connectivity
• Enclosure - drive warning / error
• Internal disk failure
• The node view
• The node's patch panel
• Enclosure and disk health

Hardware monitoring  –  138

 InfiniBox 4.0.10 User Documentation

8.3.1 About this task

InfiniBox provides visibility to the status and health of the system hardware. This view includes the
status of the system rack, nodes, enclosures, and drives.

The System Health and the Hardware Components part of the screen provide clickable visual hints on
the system health. Going over an indicator allow focusing on a health issue.

8.3.2 Visual health indicators

PDU failure
• 1 failed PDU out of 2 - 8 enclosures turn yellow.
• 1 failed PDU out of 4 – 4 enclosures turn yellow.

BBU warning
Two nodes might turn yellow when the BBU stops providing power to one of their PSUs.

Hardware monitoring  –  139

 InfiniBox 4.0.10 User Documentation

BBU is missing
BBU turns red.

One PSU failure

The nodes that are fed by this PSU turn yellow.

Both PSU failure

• The node that is fed by both failed PSU turns red
• The nodes that are fed by only one failed PSU turn yellow

Internal switch failure

This issue is reported by an event, with no GUI visibility.

Hardware monitoring  –  140

 InfiniBox 4.0.10 User Documentation

Enclosure us fed by only one PSU

The enclosure turns yellow.

Enclosure that is not fed by either PSU

The enclosure turns red.

Node - enclosure connectivity

• No connectivity – the enclosure turns red
• Partial connectivity – the Node turns red

Enclosure - drive warning / error

The enclosure is marked with an icon:

• Yellow – for warning

• Red – for error

Hardware monitoring  –  141

 InfiniBox 4.0.10 User Documentation

Internal disk failure

The failed disk turns red.

8.3.3 The node view

The Node view displays the Node along with its local and SSD drives.

Hardware monitoring  –  142

 InfiniBox 4.0.10 User Documentation

The node's patch panel

Clicking on the Node’s Back button brings up the patch panel. The patch panel displays the active ports
of the Node.

Click the Node View button to return to the Node’s view.

Hardware monitoring  –  143

 InfiniBox 4.0.10 User Documentation

8.3.4 Enclosure and disk health

The Enclosure visual indicator displays the health status of both the Enclosure and its disk drives.

The following image provides an example of a health indicator that provides information on a Warning
state of a disk drive within an Enclosure. As indicated on the image, the health of the Enclosure is OK.

Clicking the Enclosure that is colored red (above) opens a detailed view of the Enclosure’s drives along
with their exact status (below).

8.4 Powering off the InfiniBox rack

• About this task
• Procedure
• Expected result
• Troubleshooting

8.4.1 About this task

• This task provides instructions for powering off the InfiniBox rack. 
• Involve a local facility personnel

8.4.2 Procedure
1. Gracefully shutdown the system using the following infinishell command:

Hardware monitoring  –  144

 InfiniBox 4.0.10 User Documentation

system.shutdown

As a result, the nodes will be automatically powered off. 

8.4.3 Expected result

The InfiniBox is Off.

8.4.4 Troubleshooting
• Contact support

8.5 Powering on the InfiniBox rack

• About this task
• Before you begin
• Procedure
• Expected result
• Troubleshooting

8.5.1 About this task

This task provides instructions for powering on the InfiniBox rack. 

8.5.2 Before you begin

Verify that:

• The machine is located and cabled

• Disks are inserted
• The machine is grounded

 • It is important to follow the steps as ordered

• The PDUs are connected to source A first, then power-on, then connected to source B
• Power on the enclosures before powering on the nodes

Hardware monitoring  –  145

 InfiniBox 4.0.10 User Documentation

8.5.3 Procedure
1. Verify that all of the Enclosures switches are OFF (two switches on each Enclosure).
2. Verifying the power sources:
a. The system has one power cable for each PDU (the system has either 2 or 4 PDUs). 
i. In 2 PDUs formation, each PDU is connected to a power source.
ii. In 4 PDUs formation, PDUs from each side of the system is connected to a power
source.
b. To verify that the machine is correctly fed from two separate power sources: 
i. Turn on power source A, that feeds the left-side PDUs
ii. Verify that the left-side PDU outlets are on
iii. Turn on power source B
iv. Verify that the right-side PDU outlets are on.
3. Turn on the machine
a. If the enclosures are not yet turned on, turn them on now (the two switches are located
one on each PSU). 

b. Turn on the 3 BBUs by pressing the On button for 3 seconds:

 
c. The nodes should turn on automatically.
i. Verify that the nodes are turned on.
ii. The system starts activation only the average BBU load is above 50%
d. In case that the nodes are not turned on, verify that the UPSs are at least 80% loaded.

Hardware monitoring  –  146

 InfiniBox 4.0.10 User Documentation

8.5.4 Expected result

The InfiniBox is On.

8.5.5 Troubleshooting
• Verify that the power cables are connected to the BBUs and to the 1U drawer (ATSs)
• Contact support

Hardware monitoring  –  147

 InfiniBox 4.0.10 User Documentation

9 Pools

9.1 Overview of pools

• What is a pool
• The pool functionality
• The pool capacity
• The system physical and virtual capacity
• Pools terminology

9.1.1 What is a pool

The pool is a logical unit that provisions capacity to datasets.

The pool maintains capacity properties that apply to all of the datasets (volumes, filesystems and their
snapshots) that belong to the pool. A dataset cannot be created outside a pool.

Pools control the way the INfiniBox capacity is used by imposing quotas of physical capacity and virtual
capacity, and a set of rules that determine wat happens when the capacity is depleted. The storage
admin creates pools with different capacity quotas and other properties and allocate them to the
applications that InfiniBoxs serves. These applications are mapped to volumes or file systems, where
any volume or filesystem in InfiniBox belongs only to one pool, thus, it conforms to a specific capacity
allocation. The volume, or fielsystem, can move between pools, with their snapshots and optionally
with their capacity (see further along this article).

Changes to the pool’s capacity properties consist of metadata transactions and do not entail copying
data. Thus, there are instantaneus and do not affect the system’s performance.

9.1.2 The pool functionality

The pool manages a group of volumes and filesystems along their snapshots, offering the following key
benefits:

• Improved capacity management – specific volumes and/or filesystems that serve a specific
application or a group of applications can be grouped within a pool and have their capacity
managed together
• Improved capacity regulation – the behavior in a case of capacity depletion takes place on the
pool level, thus limiting the available capacity and determining whether and how the volumes
keep serving read and write requestes, applies to all of the volumes and filesystems within the
pool

Pools  –  148

 InfiniBox 4.0.10 User Documentation

9.1.3 The pool capacity

The pool has two capacity counters: physical and virtual. Physical capacity is allocated prior to the
capacity consumption. Contrary to that, virtual capacity allocated capacity as-you-go, using physical
capacity only if it is needed and when it is needed.

The ability to allocated virtual capacity only when it is needed allows to define to pool’s virtual capacity
to be larger than the pool’s physical capacity. As the pool physical capacity and virtual capacity are
defined independently, the virtual capacity can also be smaller than the physical capacity, or identical to
the physical capacity.

InfiniBox specifies virtual capacity at the pool level. The pool datasets (volumes and filesystems) are
attributed with a thin provisioning toggle. Thus, they can be either thinly provisioned, or not. If they are
thinly provisioned, the virtual capacity that is available for them derives from the virtual capacity of the
pool.

The pool physical and virtual capacities can each be defined to be as large as the system capacity. 

The system physical and virtual capacity

The system capacity is determined as follows:

• Physical capacity – the net physical capacity of the system is preconfgured per InfiniBox model
• Virtual capacity –
• Upon system inception - is defined to be identical to the system physical capacity
• Can be increased – by INFINIDAT personnel (contact Support for that)
• On InfiniBox software release 3.0 and up – the vitual capacity is 2.5 times the physical
capacity

The maximum pool pysical and virtual capacity is limited by the system-level capacity.

9.1.4 Pools terminology

Provisioning capacity to pools uses the following definitions:

• Pool entities - Volumes, filesystems and their snapshots

• Pool physical capacity - The maximum usable net capacity that can be allocated to the pool
entities.
• Pool virtual capacity - The limit on the size of all volume (thin and thick provisioned), excluding
snapshots
• System physical capacity - The total physical capacity that can be allocated within a single system.
• System virtual capacity - The maximum system capacity that can be defined by pools. This
capacity can be larger than the system physical capacity.

Pools  –  149

 InfiniBox 4.0.10 User Documentation

• System free virtual capacity - The system virtual capacity minus the sum of the pools virtual
capacity.
• Pool allocated virtual space - The sum of all volumes (thick and thin, excluding snapshots). The
allocated space cannot exceed the pool virtual capacity.
• The sum of the virtual size (thick and thin, excluding thin snapshots) - the allocated space cannot
exceed the pool virtual capacity 
• Pool allocated physical space - The sum of all thick volumes, plus the physical space consumed
by all the thin volumes, and their snapshots. The used space cannot exceed the pool physical
capacity.
• Pool free physical space - The defined pool physical capacity minus the allocated physical space.
• Pool free virtual space - The defined pool virtual capacity minus the allocated virtual space.
• Thick volume - Reserved-space volume. Virtual and Physical space allocation is done immediately
as Thick Volume is created, at the amount of the volume size.
• Thin volume - Shared-space volume. Thin volumes consume physical space upon writes. The
virtual space is allocated upon volume creation to the amount of the volume size. The initially
allocated physical space is 0 (zero). In case of space depletion, no allocation is guaranteed for
thin volumes.
• Reserved space - Automatically defined spare physical space for the pool, to support pool
physical space depletion behavior.
• Emergency buffer - A policy for automatically extending of the pool physical capacity in the event
of pool physical space depletion.
• Limited state - Upon further accepting writes, the pool enters a Limited state where no new
volumes can be created, but the existing volumes can be written into.
• Locked state - If the pool can no longer expand, it enters a Locked state and its entities can no
longer accept writes.

9.2 Creating a pool

• About this task
• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

Pools  –  150

 InfiniBox 4.0.10 User Documentation

9.2.1 About this task

The pool is a logical space that contains volumes, file systems and snapshots. The pool allocates
physical and virtual storage space for these entities. The pool also determines a policy for automatic
extension in case of space depletion. The physical and virtual capacity counters are graphically
displayed on the InfiniBox Management Console.

This task creates a pool and provisions physical and virtual capacity. You can later on create datasets
(volumes and filesystems) in the pool. A broad explanation of provisioning terminology is available
here: Overview of pools.

9.2.2 Prerequisites
• None

9.2.3 Before you begin

On the GUI, click the Pools icon on the toolbar on the left. The Pools screen opens.

9.2.4 GUI instructions

1. Click Create. The Create Pool screen opens.
2. Provide the following parameters:
• Pool name
• Physical capacity and virtual capacity
• The default setting is identical physical and virtual capacities.
You can click the link to virtual capacity in order to set a capacity that is different
than the physical capacity.
3. Click Advanced to change the values of the following default settings:
• Emergency buffer - the default value is Unlimited. You can set the value to be
either Disabled or Limited. The emergency buffer works as follows:
• Unlimited - The pool is allowed to extend without limitation
• Limited - The pool will extend within a predefined limitation
• Disabled - The pool is not allowed to extend at all.
• Notification thresholds - the default values are 80% for Warning and 90% for critical.
• SSD cache - the default is to allow entities that belong to this pool to use the InfiniBox SSD
drives.
• Pool admins - select users that will be provided with PoolAdmin rights to this pool.

Pools  –  151

 InfiniBox 4.0.10 User Documentation

4. Click Create.
The pool is created and is visible on the screen.

9.2.5 InfiniShell instructions

Use the pool.create command.

9.3 Querying for pools and their properties

• About this task
• Capacity indicators
• Specifying the capacity units
• GUI instructions
• InfiniShell instructions

9.3.1 About this task

The GUI Pools screen displays the system pools along with their properties.

Pools  –  152

 InfiniBox 4.0.10 User Documentation

Capacity indicators
• State - indicates whether the pool is Normal or Locked

• Physical capacity – indicates the following:

a. Pool total capacity

b. Pool used capacity, in percentage
i. Green means Normal
ii. Red means Locked
c. Pool thresholds (orange for warning, red for critical)
d. Pool allocated capacity - capacity that is allocated to the pool entities
e. Pool Free capacity - the amount of pool capacity that is free for use
• Virtual Total - indicates the following:
a. Pool total virtual capacity
b. Pool used virtual capacity, in percentage
i. Green means Normal
ii. Red means Locked
c. Pool allocated virtual capacity - capacity that is allocated to the pool entities
d. Pool free virtual capacity - the amount of pool capacity that is free for use

Pools  –  153

 InfiniBox 4.0.10 User Documentation

Specifying the capacity units

Use the unit argument to specify the way the capacity fields of the output are displayed. Available units
are:

• B – Byte. For example, 1000000192512

• G – GB. For example, 1000.00 GB
• GB – Same as G.
• GiB – Gibibyte (210)3 bytes. For example, 931.32 GiB.
• T – TB. For example, 1.000 TB.
• TB – same as T.
• TiB – Tebibyte (210)3 bytes. For example, 0.909 TiB.
• Block –512 bytes. For example, 1907349.

9.3.2 GUI instructions

1. Click the Pools icon on the toolbar on the left.
The Pools screen opens, displaying the system's pools.

9.3.3 InfiniShell instructions

Run: pool.query. 

9.4 Assigning a PoolAdmin to a pool

• About this task
• Prerequisites
• GUI instructions
• PoolAdmin -related InfiniShell commands

9.4.1 About this task

The PoolAdmin user is the administrator of specific pools. This user has admin rights that are similar to
the InfiniBox admin, but limited to the pool (or pools) that the PoolAdmin is assigned to.

9.4.2 Prerequisites
To assign a pool to a user you need:

• A user with PoolAdmin role

Pools  –  154

 InfiniBox 4.0.10 User Documentation

• You can assign the user to a pool upon creation of the user, as well
• A pool

You may add more than one pool to the same PoolAdmin, and more than one PoolAdmin to a pool.

9.4.3 GUI instructions

1. Modify the pool (see instructions here: Deleting a pool
2. On the Modify Pool screen, click Advanced
3. Click on the Pool Admin field to see a list of available users
4. Click Modify
The PoolAdmin user is now associated to the pool. 

9.4.4 PoolAdmin -related InfiniShell commands

• user.create - In the following example, we create a user with the type PoolAdmin. 
• user.create name=pool-01-admin password=123456
[email protected] role=PoolAdmin

• user.add_pool - adding a pool to a user

• user.add_pool name=pool-01-admin pool=pool-01 

• user.query - query for the user and the pool

• user.query user=pool-01-admin

• pool.add_admin - adds a PoolAdmin to a pool.

• pool.remove_admin - removes a PoolAdmin from a pool.
• pool.user_query - lists pool administrators per pool

9.5 Disconnecting a PoolAdmin from a pool

• About this task
• Prerequisites
• GUI instructions
• PoolAdmin -related InfiniShell commands

9.5.1 About this task

The PoolAdmin user is the administrator of specific pools. This user has admin rights that are similar to
the InfiniBox admin, but limited to the pool (or pools) that the PoolAdmin is assigned to.

You can add and remove users to and from pool according to your evolving needs.

Pools  –  155

 InfiniBox 4.0.10 User Documentation

You may add more than one pool to the same PoolAdmin, and more than one PoolAdmin to a pool.

9.5.2 Prerequisites
To assign a pool to a user you need:

• A pool with a pool admin

9.5.3 GUI instructions

1. Modify the pool (see instructions here: Deleting a pool
2. On the Modify Pool screen, click Advanced
3. Remove the Pool Admin
4. Click Modify
The PoolAdmin user is now disconnected from the pool. 

9.5.4 PoolAdmin -related InfiniShell commands

• user.create - In the following example, we create a user with the type PoolAdmin. 
• user.create name=pool-01-admin password=123456
[email protected] role=PoolAdmin

• user.add_pool - adding a pool to a user

• user.add_pool name=pool-01-admin pool=pool-01 

• user.query - query for the user and the pool

• user.query user=pool-01-admin

• pool.add_admin - adds a PoolAdmin to a pool.

• pool.remove_admin - removes a PoolAdmin from a pool.
• pool.user_query - lists pool administrators per pool

9.6 Locking a pool

• About this task
• Auto-locking of a pool
• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

Pools  –  156

 InfiniBox 4.0.10 User Documentation

9.6.1 About this task

This task locks the pool, blocking the following operations:

• Writing into the pool's datasets

• Resizing the pool's datasets
• Restoring a dataset from a snapshot
• Creating datasets in the pool

A locked pool can be unlocked, thus restoring the capabilities mentioned above.

Auto-locking of a pool
When a pool exceeds its capacity settings it is locked.

9.6.2 Prerequisites
• Pool

9.6.3 Before you begin

On the GUI, click the Pools icon on the toolbar on the left. The Pools screen opens.

9.6.4 GUI instructions

1. The Pools screen displays the InfiniBox pools. Select a pool and then select Lock Pool from the
Actions menu. 
2. Approve the warning message.
The pool is locked. 
The pool's state is visible on screen. 

9.6.5 InfiniShell instructions

• Use the pool.lock command

9.7 Unlocking a pool

• About this task
• Analyzing the causes to the pool lock
• Prerequisites

Pools  –  157

 InfiniBox 4.0.10 User Documentation

• Before you begin

• GUI instructions
• InfiniShell instructions

9.7.1 About this task

This task unlocks the pool, restoring the ability to perform the following operations:

• Writing into the pool's datasets

• Resizing the pool's datasets
• Restoring a dataset from a snapshot
• Creating datasets in the pool

Analyzing the causes to the pool lock

The pool is usually automatically locked upon depleting its capacity settings. In such a case, prior to the
pool unlock - in order to prevent it from auto-locking again - it is advised to increase its capacity
(see Deleting a pool for resizing instructions).

9.7.2 Prerequisites
• Pool

9.7.3 Before you begin

On the GUI, click the Pools icon on the toolbar on the left. The Pools screen opens.

9.7.4 GUI instructions

1. The Pools screen displays the InfiniBox pools. Select a pool and then select Unock Pool from the
Actions menu. 
The pool is unlocked.

9.7.5 InfiniShell instructions

• Use the pool.unlock command

9.8 Deleting a pool

• About this task

Pools  –  158

 InfiniBox 4.0.10 User Documentation

• Prerequisites
• A note on a PoolAdmin that admins this pool
• Before you begin
• GUI instructions
• InfiniShell instructions

9.8.1 About this task

This task deletes a pool.

9.8.2 Prerequisites
• Pool with all of its datasets deleted or removed to other pools.

A note on a PoolAdmin that admins this pool

A PoolAdmin that has access only to this pool will have read-only rights to the InfiniBox, upon deletion
of this pool.

9.8.3 Before you begin

On the GUI, click the Pools icon on the toolbar on the left. The Pools screen opens.

9.8.4 GUI instructions

1. The Pools screen displays the InfiniBox pools. Select a pool and then select Delete Pool from the
Actions menu.
2. Approve the warning message.
3. The pool is deleted.

9.8.5 InfiniShell instructions

• Use the pool.add_admin command

Pools  –  159

 InfiniBox 4.0.10 User Documentation

10 Provisioning volumes

10.1 Overview of volumes

• Volumes and snapshots
• Volume operations
• Snapshot mechanism
• Redirect-on-write
• Writable snapshots
• Snapshot refresh
• Restoring a volume from a snapshot

10.1.1 Volumes and snapshots

The volume is a virtual disk. InfiniBox volumes span across all active drives, allowing for self-healing in
case of a drive failure.

The volume capacity is determined by the used and is affected by the settings of the pool that the
volume belongs to. Each volume must belong to a pool and can belong to only one pool. The volume
can move among pools.

The volume snapshot is a point-in-time representation of the volume’s data. The volume can have up to
999 snapshots. A snapshot can have snapshots, as well. Taking a snapshot of multiple volumes at the
same point in time is also supported. In order to do this, the volumes are grouped into consistency
groups.

Volume operations
The volume is the basic data container that is presented to the hosts as a logical disk. The term volume
is sometimes used for an entity that is either a volume or a snapshot, as hosts view volumes and
snapshots through the same protocol. 

To distinguish the volume from its snapshots, whenever required, the term master volume is used for a
volume to clearly

Each volume has the following basic configuration attributes: name, size, pool, provisioning type and
ssd-enabled. The volume name is an alphanumeric string that is not related to the SCSI protocol. The
volume size represents the number of blocks in the volume that the host sees. InfiniBox presents the
volume size in bytes and multiplications of bytes (KB, GB, TB).

Available volume operations are (all of the operations are available from the GUI, CLI and API):

Provisioning volumes  –  160

 InfiniBox 4.0.10 User Documentation

• Creating a volume – creates a volume by name, pool and size (mandatory parameters), attributes
such as provisioning type, usage of SSD drives and compression. The GUI and CLI also allow to
create series of volumes at once  
• Querying for volumes – displaying a filterable and sortable list of the system’s volumes
• Displaying the volume tree – displaying the parental relations between the volume and its
snapshots in a tree format
• Resizing a volume – increasing the volume size

• Shrinking the volume is not available

• Resizing the volume is not applicable for replicated volumes

• Setting the volume to use the system’s SSD drives – as InfiniBox provides several hardware layers
to choose from, it is possible to allow highly active volumes to utilize the system’s SSD drives. This
attribute is inherited from the pool, and can be changed individually for each volume
• Renaming the volume – the volume can be renamed (in accordance with InfiniBox naming
scheme)
• Write-protection – locking the volume to host writes
• Changing the thin provisioning setting – the volume can be set to be thin-provisioned or thick-
provisioned
• Thick-provisioned - the capacity that is allocated to the volume is determined upon the
volume creation. 
• Thin-provisoned - the capacity that is allocated to the volume is determined upon
incoming writes. 
• Mapping the volume to a host – InfiniBox exposes the volume to the host in order to allow for
reads and writes. To be mapped to a volume, he host has to be connected to InfiniBox.
• Creating a snapshot and restoring from a snapshot
• Deleting a volume
• Setting the volume to be compressed
• Moving the volume between pools

See a broader discussion below for each of the operations.

Snapshot mechanism
InfiniBox uses the redirect-on-write approach to snapshots taking, where creating a snapshot means
creating a pointer to the existing volume data, rather than copying the volume aside in a mechanism
known is copy-on-write. The InfiniBox natively enables the user to take any number of snapshots with
no performance penalty whatsoever.

Provisioning volumes  –  161

 InfiniBox 4.0.10 User Documentation

As taking the snapshot does not impact performance, and at the moment of creation the snapshot
contains only pointers to the volume, the system can store an unlimited number of snapshots. For
functional considerations, the number of snapshots for a volume is limited to 999. All of the snapshots
of a volume share a single copy of unmodified data.  

Redirect-on-write
The redirect-on-write mechanism creates a volume snapshot as follows:

1. All of the data of the volume is not impacted by the snapshot. It remains in place, not moved and
not copied.
2. Following the creation of the snapshot, as the volume accepts new writes, the new writes are
stored on a new storage space.
3. The snapshot keeps pointing to the old data (as it represents the volume data on a specific given
time) and the volumes points to the new data instead of the old data.

Writable snapshots
Taking a snapshot of a snapshot is done similarly to taking a snapshot of a volume.
The parent snapshot is not impacted by this action, and the child snapshot only points to data.

When a snapshot is created, it is write-protected. This means that the snapshot depicts the volume (or
parent snapshot) at a specific point-in-time such that when the volume is restored from the snapshot, it
reflects its own content at the point-in-time in which the snapshot was taken.

When a snapshot becomes writable and starts accepting writes, its child snapshots behave similarly to
the way a snapshot of a volume behaves. For every write that the snapshot receives, the child snapshot
keeps pointing at the old data, with no unecessary writes and no impact on performance.

Snapshot refresh
Refreshing the snapshot means that the snapshot points to the current volume data, but it maintains
its metadata (SCSI serial, creation date, location on the volume tree). The refresh mechanism involves
changing only the pointers of the snapshot. Storage space that is released by the refreshed snapshot
and is not pointed to by any other snapshot is freed, and this operation takes place outside of the
scope of the snapshot refresh.

Restoring a volume from a snapshot

When a volume is restored from its snapshot, it points to the blocks of data that the snapshot points to.
No actual data copy takes place, and the operation is instantaneous similarly to the operation of taking
a snapshot. The volume stops pointing to blocks of data that were not pointed to by the snapshot from
which it is restored. In order not to lose this data, it is recommended to create a snapshot prior to the
restore operation.

Provisioning volumes  –  162

 InfiniBox 4.0.10 User Documentation

10.2 Creating a volume

• About this task
• Volume capacity properties
• Forecasting the effect of the created volume(s) on the pool capacity
• Creating a series of volumes
• Prerequisites
• Before you begin
• Instructions

10.2.1 About this task

This task creates a volume on an InfiniBox system.

Volume capacity properties

The volume capacity properties are:

• Size - specified upon volume creation and seen by the host SCSI initiator.
• Used – the physical space that is allocated to the volume and was actually written into.
• Allocated – the physical space that was allocated to the volume upon creation.
• Tree allocated – the sum of the allocated properties of the volume’s descendants. The value of
this property is zero (0) if there are no descendants.

Forecasting the effect of the created volume(s) on the pool capacity

The Volume Create screen on the GUI displays physical and virtual capacity counters for the pool that
the volume belongs to. Prior to clicking on the Create button, you can click TAB, to see the effect of the
volume(s) you are about to create on the pool capacity. You can change the numbers on the series field
and click TAB each time, to see how the capacity changes, and to adjust the number of volumes
accordingly.

Creating a series of volumes

You can either create a single volume or a batch of volumes, through the Create series field. The
volumes names have a common prefix and they are indexed according to the following rule. 

• The provided series numbers can be zero-padded (up to 5 zeros)

Provisioning volumes  –  163

 InfiniBox 4.0.10 User Documentation

Volume Create series Number of Volume names

prefix created
volumes

v- 1-5 5 v-1 through v-5

vol- 1-10 10 vol-1 through vol-10

v-padded- 001-010 10 v-padded-001 through v-

padded-010

v- 20 20 v-1 through v-20

It is recommended to denote a new prefix for each batch. 

10.2.2 Prerequisites
• A pool with sufficient capacity
• Admin or Pool Admin privileges

10.2.3 Before you begin

There are two options to create a volume on the GUI. One option is from the Pools screen. The other is
from the Volumes & FS screen.

• Click the Pools icon and place the cursor on the pool in which you will create the volume
• Click the Volumes & FS icon on the toolbar on the left

10.2.4 Instructions
1. Create a volume either:
• On the Pools screen, right-click a pool and select Create Volume.
• On the Volumes screen, click Create. 
2. Set the volume’s name, size and whether it is thin provisioned.
3. Select a pool from the Pool drop-down list. Consult the physical and virtual capacity indicators.
4. Optionally, click Advanced.
a. (Optionally) Create a series of volumes
1. Set the number of volumes that will be created 
2. Specify their ordinal numbers. 
b. (Optionally) Check SSD Cache. 
By default, the volume will be created with SSD drives enabled.

Provisioning volumes  –  164

 InfiniBox 4.0.10 User Documentation

This option is available only for systems that have SSD drives. On a system with no SSD
drives, this checkbox is not displayed on the screen.
5. Click Create.

The volume is created and is visible on the Volumes & FS screen.

10.3 Modifying a volume

• About this task
• Before you begin
• Instructions
• Related GUI tasks 

10.3.1 About this task

This task modifies the following volume attributes:

• Name
• Size
• Thin provisioning
• SSD cache

10.3.2 Before you begin

Access the volume from the Volumes & FS screen. Then, either:

• Select a volume and select Modify Volume from the Actions menu.
• Click a volume to display it on screen, and select Modify Volume from the Actions menu.

10.3.3 Instructions
1. Upon selecting Modify Volume, the Modify Volume screen opens.
2. Modify one of the attributes that are available for modification.
a. When resizing the volume to more than 2TB, approve the warning.
3. Click Modify.
The volume is modified.

10.3.4 Related GUI tasks 

You can modify other volumes values using the following tasks:

Provisioning volumes  –  165

 InfiniBox 4.0.10 User Documentation

• Calculating the reclaimable space of a filesystem

• Write-protecting a filesystem
• Write-enabling a filesystem

10.4 Moving a volume

• About this task
• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

10.4.1 About this task

This task moves a volume from one pool to another.

• The volume moves with its descendants.

• You can select whether to move the volume with its capacity. In such a case, the destination pool
will increase in capacity while the source pool will decrease its capacity. 

10.4.2 Prerequisites
• A volume

10.4.3 Before you begin

Access the volume from the Volumes & FS screen. Then, either:

• Select a volume 
• Click a volume to display it on screen

10.4.4 GUI instructions

1. Click Move Volume Family from the Actions menu. The Move Volume Family screen opens.
2. Select a target pool from the list.
3. Select whether to move the volume with its capacity.
a. The default is to move the volume without its capacity

Provisioning volumes  –  166

 InfiniBox 4.0.10 User Documentation

4. Click Move.
The volume moves to the new pool. 

10.4.5 InfiniShell instructions

• vol.move - moving the volume between pools

10.5 Write-enabling a volume

• About this task
• Prerequisites
• Before you begin
• GUI instructions

10.5.1 About this task

This task sets a volume that is write-protected to be write-enabled again.

• This operation does not impact the write-enable state of the volume's snapshots

10.5.2 Prerequisites
• A write-protected volume

10.5.3 Before you begin

Access the filesystem from the Volumes & FS screen. Then, either:

• Select a volume 
• Click a volume to display it on screen

10.5.4 GUI instructions

1. Click Write Enable from the Actions menu. 
The volume is write-enabled. 

10.6 Write-protecting a volume

• About this task

Provisioning volumes  –  167

 InfiniBox 4.0.10 User Documentation

• Prerequisites
• Before you begin
• GUI instructions

10.6.1 About this task

This task sets the volume to stop accepting writes.

10.6.2 Prerequisites
• A volume

10.6.3 Before you begin

Access the volume from the Volumes & FS screen. Then, either:

• Select a volume 
• Click a volume to display it on screen

10.6.4 GUI instructions

1. Click Write Protect from the Actions menu. 
The volume is now write-protected. 

10.7 Mapping a volume

• About this task
• Prerequisites
• Instructions
• Related tasks

10.7.1 About this task

This task maps a volume to a host or a cluster.
INFINIDAT simplifies the mapping of volumes to hosts as much as possible. The LUN number assigned
for each mapping is insignificant. Its only role is to maintain the uniqueness of the volume - host
mapping, so as long as it works, InfiniBox does not bother the user with it and assigns the LUNs
automatically.
The InfiniBox RESTful management API for mapping a volume to host or cluster accepts a LUN number
as an optional parameter. If the parameter is not specified, the LUN is automatically assigned by the
system. (This is also the case with our VMware plugin.)

Provisioning volumes  –  168

 InfiniBox 4.0.10 User Documentation

LUN numbers are automatically assigned as follows:

• For host - next available number, starting from 1. 
• For cluster - next available number, starting from 11.

The reason we start assigning LUNs to a cluster starting with 11 is to avoid collision with LUNs mapped
to individual hosts within the cluster. As long as no more than 10 volumes are mapped to a host
individually, there is no collision between volumes mapped individually and volumes mapped to the
cluster. In the rare case that a host has more than 10 volumes mapped only to it, and it also belongs to
a cluster, the LUN number assignment has to be set manually.
The InfiniBox management GUI and CLI support setting the LUN number manually (in addition to the
default behavior that is explained above) by the user when mapping a volume to host or cluster.

10.7.2 Prerequisites
• A volume
• A host, or a cluster

10.7.3 Instructions
1. Right-click a volume from the Volume & FS screen and select Modify Mapping from the pop-up
menu.
2. Browse or search for a host and click Map.
3. Click Apply.
The volume is mapped to the host. 

10.7.4 Related tasks

Mapping a volume to a host or a cluster can be done by the following InfiniShell command:

• vol.map

The mapping is visible through the following queries:

• vol.query
• host.query
• cluster.query
• map.query 

10.8 Calculating the reclaimable space of a volume

• About this task
• Prerequisites

Provisioning volumes  –  169

 InfiniBox 4.0.10 User Documentation

• Before you begin

• GUI instructions

10.8.1 About this task

This task calculates the reclaimable space of a volume. Whenever you need to delete a volume in order
to free storage space, you can simulate the deletion and calculate the space that the deletion would
free.

10.8.2 Prerequisites
• A volume

10.8.3 Before you begin

Access the volume from the Volumes & FS screen. 

10.8.4 GUI instructions

1.  Either:
• Select a volume 
• Click a volume to display it on screen
2. Click Calculate reclaimable from the Actions menu
A screen pops, displaying the calculation result
3. Click Close to return to the volume 

10.9 Deleting a volume

• About this task
• Prerequisites
• Before you begin
• GUI instructions

10.9.1 About this task

This task deletes a volume with all of its snapshots.

This operation is irreversible.

Provisioning volumes  –  170

 InfiniBox 4.0.10 User Documentation

10.9.2 Prerequisites
• A volume

10.9.3 Before you begin

Access the filesystem from the Volumes & FS screen. Then, either:

• Select a volume
• Click a volume to display it on screen

10.9.4 GUI instructions

1. Click Delete Volume from the Actions menu. 
2. Approve the warning message.
The volume and its snapshots are deleted. 

10.10 Querying for volumes and their properties

• About this task
• The volumes table columns
• Specifying the capacity units
• The volume capacity counters
• InfiniShell instructions
• GUI instructions

10.10.1 About this task

The Volumes screen displays volumes along with their properties.

Provisioning volumes  –  171

 InfiniBox 4.0.10 User Documentation

The volumes table columns

Columns that are displayed by default:

• Name - the name of the volume

• Pool - the pool that the volume belongs to
• Consistency group - the consistency group that the volume belongs to 
• N/A - indicates that the volume does not belong to a consistency group
• Thin - indicates whether the volume can use the pool's virtual capacity
• Mapped - indicates whether the volume is mapped to a host or cluster
• Size - specified upon volume creation and seen by the host SCSI initiator
• Used - the physical capacity that is allocated to the volume and was actually written into
• Snapshots - the sum of the allocated properties of the volume’s descendants. 
• The value of this property is zero (0) if there are no descendants
• If the volume is THICK the system will always allocate the entire size. 

Provisioning volumes  –  172

 InfiniBox 4.0.10 User Documentation

• If the volume is THIN the system will allocate the amount of capacity that is actually in use
for user data. 

• Write protected - states whether the host can write into the volume
• Creation date - Date and time of the creation of the volume

Columns that are not displayed by default, but can be added to the table:

• Replication Role - for a replicated volume, this column indicated whether the volume is a source
or a target
• Serial  - the volume's serial number
• Data Reduction Ration - the ratio between the pre-compressed volume capacity to the
compressed volume capacity

Specifying the capacity units

Use the unit argument to specify the way the capacity fields of the output are displayed. Available units
are:

• B – Byte. For example, 1000000192512

• G – GB. For example, 1000.00 GB
• GB – Same as G.
• GiB – Gibibyte. For example, 931.32 GiB.
• T – TB. For example, 1.000 TB.
• TB – same as T.
• TiB – Tebibyte. See GiB above. For example, 0.909 TiB.
• Block –512 bytes. For example, 1907349.

Provisioning volumes  –  173

 InfiniBox 4.0.10 User Documentation

10.10.2 The volume capacity counters

Click a volume to open its detailed view. At the top left side of the view you can find the volume's
capacity counters.

1. Size - specified upon volume creation and seen by the host SCSI initiator
2. Used - the physical capacity that is allocated to the volume and was actually written into
3. Free - the amount of the volume size that is not in use 
4. Allocated - the capacity that is allocated for the volume
• If the volume is THICK the system will always allocate the entire size. 
• If the volume is THIN the system will allocate the amount of capacity that is actually in use
for user data. 
5. Tree Allocated - the sum of the allocated properties of the volume’s descendants. The value of
this property is zero (0) if there are no descendants

10.10.3 InfiniShell instructions

• Run: vol.query.

10.10.4 GUI instructions

• Click the Volumes & FS icon on the toolbar on the left.
The Volumes screen opens, displaying the system's volumes. 

10.11 Snapshot management

Provisioning volumes  –  174

 InfiniBox 4.0.10 User Documentation

• Introducing a new snapshots mechanism

• Transferring old snapshots to the new mechanism
• Restoring a volume from a snapshot
• Increased space efficiency

10.11.1 Introducing a new snapshots mechanism

InfiniBox release 3.0 introduces an improved snapshot mechanism. This mechanism replaces the
snapshot - clone dualism with a single entity that can become writable.

Transferring old snapshots to the new mechanism

InfiniBox systems with software versions lower than 3.0 are transparently upgraded to the new
snapshot mechanism.

• A snapshot leaf on the volume tree remains a snapshot

• A clone leaf on the volume tree turns into a snapshot

Restoring a volume from a snapshot

A volume can be restored from any of the snapshots down the volume tree.

Provisioning volumes  –  175

 InfiniBox 4.0.10 User Documentation

Increased space efficiency

The new snapshots mechanism requires less writes and saves more space. Consider the following
example.

1. Create a volume and create a snapshot of this volume

• The snapshot references to block A of the volume

 
2. Write to the volume
• Block A is replaced with block A' 

Provisioning volumes  –  176

 InfiniBox 4.0.10 User Documentation

• The snapshot writes the capacity of block A as it is no longer available on the volume

3. Write to the snapshot - to the same block

• The snapshot does not point at blocks A or A' as these blocks are replaced with block A''
• The space of block A - as it is no longer needed - is reclaimed

10.12 Creating a volume snapshot

• About this task
• Prerequisites
• GUI Instructions
• Following this task

10.12.1 About this task

The snapshot is an entity that allows the user to restore a volume to a specific point-in-time.

Provisioning volumes  –  177

 InfiniBox 4.0.10 User Documentation

• It is possible to restore a volume from a writable snapshot

• Read more about this ability here: Snapshot management
• Snapshot operations:  create a snapshot, delete it, rename it, enable SSD cache, view its relation
with the volume and other snapshots, map it to a host or a cluster, unmap it, query the mapping
and query all snapshots per volume or pool.

10.12.2 Prerequisites
• Volume - the volume that its snapshot is taken

10.12.3 GUI Instructions

1. Select a volume from the Volumes tab.
2. Right-click the volume and select Create Snapshot from the menu.

 
3. Name the snapshot and click Create.

Provisioning volumes  –  178

 InfiniBox 4.0.10 User Documentation

The snapshot is visible on screen. 

10.12.4 Following this task

View the snapshot using the vol.snap.query command.

Use the unit argument to specify the units of the capacity fields of the output. Available units are:

• B – Byte. For example, 1000000192512

• G – GB. For example, 1000.00 GB
• GB – Same as G.
• GiB – Gibibyte (see definition in Wikipedia). For example, 931.32 GiB.
• T – TB. For example, 1.000 TB.
• TB – same as T.
• TiB – Tebibyte. See GiB above. For example, 0.909 TiB.
•  Block –512 bytes. For example, 1907349.

10.13 Querying for snapshots and their properties

• About this task
• Capacity indicators
• Specifying the capacity units
• The snapshot detailed query
• GUI instructions

10.13.1 About this task

The snapshots table on the volume's screen displays snapshots along with their properties.

Provisioning volumes  –  179

 InfiniBox 4.0.10 User Documentation

Capacity indicators
• Size – The defined size of the parent volume at snapshot creation time. 
• If you change the parent volume size after the snapshot was taken, the snapshot size is
not impacted
• Allocated – The amount of capacity the system had to allocate to support the snapshot.  A
snapshot is always thin provisioned.
• Used – The amount of data the host wrote to the parent volume after the creation time of the
snapshot

Specifying the capacity units

Use the unit argument to specify the way the capacity fields of the output are displayed. Available units
are:

• B – Byte. For example, 1000000192512

• G – GB. For example, 1000.00 GB
• GB – Same as G.
• GiB – Gibibyte. For example, 931.32 GiB.
• T – TB. For example, 1.000 TB.
• TB – same as T.
• TiB – Tebibyte. See GiB above. For example, 0.909 TiB.
• Block –512 bytes. For example, 1907349.

The snapshot detailed query

Use the detailed argument to display more information for the queried pool:

• SSD Cache - determines whether the snapshot is set to use the system's cache

Provisioning volumes  –  180

 InfiniBox 4.0.10 User Documentation

• Replication Role - specifies the replication role of the parent volume 

• Created At - the time the snapshot was created

10.13.2 GUI instructions

1. Click the Volumes & FS icon on the toolbar on the left.
The Volumes screen opens, displaying the system's volumes.
2. Expand a volume in order to view its snapshots. 

10.14 Refreshing a volume snapshot

• About this task
• Prerequisites
• GUI Instructions

10.14.1 About this task

• The snapshot is an entity that allows the user to restore a volume to a specific point-in-time.
• The snapshot can be refreshed so that its content adheres to a new point-in-time but all other
snapshot attributes remain the same.

10.14.2 Prerequisites
• Volume 
• Snapshot

Provisioning volumes  –  181

 InfiniBox 4.0.10 User Documentation

10.14.3 GUI Instructions

1. Select a volume from the Volumes tab.
2. Select one of its snapshots.
3. Right-click the snapshot and select Refresh Snapshot from the menu.

 
4. Approve the dangerous operation.
The snapshot is refreshed.

10.15 Restoring a volume from its snapshot

• About this task
• Restoring relations among the volume and its descendants
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands

10.15.1 About this task

This task restores a volume from one of its snapshots.

Restoring relations among the volume and its descendants

• The volume and it snapshots form an ancestry tree. Within the tree, the volume can be restored
from each of its snapshots. 

Provisioning volumes  –  182

 InfiniBox 4.0.10 User Documentation

10.15.2 Prerequisites
• A volume
• A snapshot of this volume (located anywhere on the volume tree)

10.15.3 Before you begin

1. Access the volume from the Volumes & FS screen. 
2. Select a volume.
3. In the Snapshots tab, expand the view to see the volume snapshots.

10.15.4 Instructions
1. Select a snapshot and select Restore from this Snapshot from the Actions menu.
The Restore from Snapshot screen opens.
2. Select which ancestor of the snapshot to restore.
3. Click Restore.
The selected volume is restored from the snapshot.

10.15.5 Related InfiniShell commands

Modifying the volume attributes using InfiniShell is available via the following commands:

• vol.restore - restores a volume from one of its snapshots

Provisioning volumes  –  183

 InfiniBox 4.0.10 User Documentation

11 Provisioning filesystems

11.1 Creating a filesystem

• About this task
• Prerequisites
• Before you begin
• Acceptable filesystem size
• Determining the filesystem size
• Small file capacity accounting
• Symbolic links capacity accounting
• Instructions
• Related tasks
• Related InfiniShell commands

11.1.1 About this task

This task creates a filesystem on an InfiniBox system.

11.1.2 Prerequisites
• A pool with sufficient capacity, see: Creating a pool
• Admin or Pool Admin privileges. 
• Creating a network interface
• Creating a network space

11.1.3 Before you begin

There are two options to create a filesystem on the GUI. One option is from the Pools screen. The
other is from the Volumes & FS screen.

• Click the Pools icon and place the cursor on the pool in which you will create the filesystem
• Click the Volumes & FS icon on the toolbar on the left

Acceptable filesystem size

InfiniBox rounds up filesystem with a size that is not a multiplication of 64KiB.

• A filesystem with a size that is not a multiplication of 512B is rejected with an error. 

Provisioning filesystems  –  184

 InfiniBox 4.0.10 User Documentation

Determining the filesystem size

• When determining the physical size of the filesystem, it is recommended to leave some room for
the filesystem metadata (several megabytes per filesystem)

Small file capacity accounting

• InfiniBox optimizes data layout per file size:
• Very small files (below 128 Bytes) are stored inside the metadata structure (Btree) and do
not consume additional space at all
• Small files (below 64KB) 
• Allocated with a 64KB space, and the empty space (within the 64KB) is filled with
zeroes, which our compression removes
• The files are stored in 4KB increments, so, for example, a 7KB file will be either:
• 4KB if it was able to compress to <4KB
• 8KB if it was not able to compress at all, or compressed to >4KB
• Large files (above 64KB) are compressed and their compressed-size is rounded up to 4KB
increments
• Since the compression affects the "Allocated", and "Snapshots" capacity, but does not reduce the
"Used" capacity counters, small files are still reported back to the client as consuming 64KB
• Filesystems that are provisioned with a size that is a multiplication of 4KB will have their
"Used" counter identical to "Allocated"
• The best practice is to this provision the filesystem and increase its size to compensate for the
virtually inflated files

Symbolic links capacity accounting

As symbolic links are small enough to be stored within the filesystem metadata structure, they do not
require an actual file. The symbolic link capacity is accounted for as follows:

• 1 iNode
• 0 capacity

11.1.4 Instructions
1. On the GUI, either:

Provisioning filesystems  –  185

 InfiniBox 4.0.10 User Documentation

• On the Pools screen: Click the Actions button and select Create a filesystem from the
pop-up menu.

• On the Volumes & FS screen: Click Create and select Filesystem from the list.

The Create Filesystem screen opens.

2. Enter a name for the filesystem. You can provision several filesystems at once. In such a case, the
name will serve as a prefix for the created series of filesystems.
Enter a size for the filesystem, select whether the filesystem is thin provisioned, and select a pool.
Click Advanced to set the number of filesystems that will be created, and select whether to allow
the filesystem to use the system's SSD drives.
Watch the pool capacity gauges at the right pane of the screen. The capacity gauges provide an
idea of the impact of the created filesystem on the pool capacity.

Provisioning filesystems  –  186

 InfiniBox 4.0.10 User Documentation

Click Create.

3. The filesystem is created. In the example on the screenshot, 10 filesystems are created,
numbered 101 to 110.

11.1.5 Related tasks

This task, and all other filesystem tasks are available both on the GUI and InfiniShell.

Related InfiniShell commands

• fs.cache - enabling the filesystem to use InfiniBox flash cache
• fs.create - creating a new filesystem

Provisioning filesystems  –  187

 InfiniBox 4.0.10 User Documentation

• fs.delete - deleting a filesystem

• fs.move - moving the filesystem between pools
• fs.query - listing the filesystems and their properties
• fs.reclaimable - calculating the reclaimed capacity in case of filesystem deletion
• fs.rename - renaming the filesystem
• fs.resize - resizing the filesystem
• fs.restore - restore data state of a filesystem from a previous snapshot
• fs.set_thin - setting the filesystem provisioning type
• fs.tree - displaying the hierarchy of the filesystem along with its snapshots
• fs.write_enable - enabling write access to a filesystem
• fs.write_protect - protecting a filesystem from writes

11.2 Filesystems concepts

InfiniBox features support for filesystems in addition to the already existing support for volumes.
Filesystems and volumes share the backend storage and can co-exist within the same storage pool. This
release includes support for NFS protocol version 3 and a lot of the storage capabilities of volumes (e.g.
Snapshots, Flash Cache).

• InfiniBox filesystems main design points 

• InfiniBox NAS terminology
• How filesystems compare to volumes
• Filesystem limitations

11.2.1 InfiniBox filesystems main design points 

InfiniBox filesystems can be as large as useable capacity in InfiniBox, as fast as the network supports,
benefit from the system SSD cache, be thinly provisioned, use snapshots, use self-encrypted drives and
be mounted to any client.

InfiniBox implements NAS directly on top of the data layer allowing it to get similar data services to our
SAN implementation. It features the following:

• User mode – for enhances stability, InfiniBox NAS does not use kernel resources
• Shared cache - the filesystem shares the cache with the SAN volumes, thus reduces memory
consumption and increases the performance.
• Transaction protection for taking a snapshot during a transaction, without performance impact
• RAID, data placement, and provisioning operations are taken by the block device

Provisioning filesystems  –  188

 InfiniBox 4.0.10 User Documentation

• Built for scale:

• High capacity 
• High file count
• Large number of filesystems  
• High performance

11.2.2 InfiniBox NAS terminology

• NAS - Network-attached storage provides file-level storage that serves files. In NAS (as
opposed to SAN, the filesystem is managed by the storage array rather than by the hosts).
• NFS v3 - Network File System a distributed file system protocol that applications use to
communicate with the storage. 
• IFS - INFINIDAT File System a B-tree based filesystem. Btree allows IFS to translate a host's
request for a file to its location inside the storage with a single operation, allowing the filesystem
to scale to billions of files. 

11.2.3 How filesystems compare to volumes

Filesystems and SAN volumes have a lot of similarities, and some differences:

Property Volumes (SAN) Filesystems (NAS)

Provisioned inside pools

Can be moved between pools non-

disruptively

Thin / Thick provisioned

Capable of snapshots

Can be write-enabled or write-protected

(read-only)

Can benefit from SSD cache

Supports snapshots

Connectivity configuration Network Spaces Network Spaces

Provisioning filesystems  –  189

 InfiniBox 4.0.10 User Documentation

Property Volumes (SAN) Filesystems (NAS)

Access method Map a volume to a host / Export the filesystem to clients

cluster
(Many clients can access the
(Normally only 1 host / same filesystem)
cluster)

Protocol Fiber Channel (FC)  NFS v3 

 Protocol semantics Writing to blocks (LBAs) Writing to offsets inside files

inside a volume 

Protocol Transport Fiber Channel TCP/IP

Growing capacity Requires  host side actions No host side action required 

Link Redundancy Multipath - Host is in LACP provides redundancy on

charge of using all layer 2
available paths
When LACP fails, IP addresses
automatically failover to another
LACP.

Physical fabric Fibre Channel Ethernet

11.2.4 Filesystem limitations

• Maximum length of the Permissions field for a single export: 3,000 client entries.

11.3 Modifying a filesystem

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related GUI tasks
• Related InfiniShell commands

Provisioning filesystems  –  190

 InfiniBox 4.0.10 User Documentation

11.3.1 About this task

This task modifies the following filesystem attributes:

• Name
• Size
• Thin provisioning
• SSD cache

11.3.2 Prerequisites
• A filesystem

11.3.3 Before you begin

Access the filesystem from the Volumes & FS screen. Then, either:

• Select a filesystem and select Modify Filesystem from the Actions menu.
• Click a filesystem to display it on screen, and select Modify Filesystem from the Actions menu.

11.3.4 Instructions
1. Upon selecting Modify Filesystem, the Modify Filesystem screen opens.
2. Modify one of the attributes that are available for modification.
3. Click Modify.
The filesystem is modified.

11.3.5 Related GUI tasks

You can modify other filesystem values using the following tasks:

• Calculating the reclaimable space of a filesystem

• Write-protecting a filesystem
• Write-enabling a filesystem

11.3.6 Related InfiniShell commands

Modifying the filesystem attributes using InfiniShell is available via the following commands:

• fs.cache - enabling the filesystem to use InfiniBox flash cache

• fs.rename - renaming the filesystem

Provisioning filesystems  –  191

 InfiniBox 4.0.10 User Documentation

• fs.resize - resizing the filesystem

• fs.set_thin - setting the filesystem provisioning type

11.4 Moving a filesystem

• About this task
• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

11.4.1 About this task

This task moves a filesystem from one pool to another.

• The filesystem moves with its descendants.

• You can select whether to move the filesystem with its capacity. In such a case, the destination
pool will increase in capacity while the source pool will decrease its capacity. 

11.4.2 Prerequisites
• A filesystem

11.4.3 Before you begin

Access the filesystem from the Volumes & FS screen. Then, either:

• Select a filesystem 
• Click a filesystem to display it on screen

11.4.4 GUI instructions

1. Click Move Filesystem Family from the Actions menu. The Move Filesystem Family screen
opens.
2. Select a target pool.
3. Select whether to move the filesystem with its capacity.
4. Click Move.
The filesystem moves to the new pool. 

Provisioning filesystems  –  192

 InfiniBox 4.0.10 User Documentation

11.4.5 InfiniShell instructions

• fs.move - moving the filesystem between pools

11.5 Write-enabling a filesystem

• About this task
• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

11.5.1 About this task

This task sets a filesystem that is write-protected to be write-enabled again.

11.5.2 Prerequisites
• A write-protected filesystem

11.5.3 Before you begin

Access the filesystem from the Volumes & FS screen. Then, either:

• Select a filesystem 
• Click a filesystem to display it on screen

11.5.4 GUI instructions

1. Click Write Enable from the Actions menu. 
The filesystem is write-enabled. 

11.5.5 InfiniShell instructions

• fs.write_enable - moving the filesystem between pools

11.6 Write-protecting a filesystem

• About this task

Provisioning filesystems  –  193

 InfiniBox 4.0.10 User Documentation

• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

11.6.1 About this task

This task sets the filesystem to no longer accept writes.

11.6.2 Prerequisites
• A filesystem

11.6.3 Before you begin

Access the filesystem from the Volumes & FS screen. Then, either:

• Select a filesystem 
• Click a filesystem to display it on screen

11.6.4 GUI instructions

1. Click Write Protect from the Actions menu. 
2. Approve the warning message.
The filesystem is now write-protected. 

11.6.5 InfiniShell instructions

• fs.write_protect - disabling write-access on filesystem.

11.7 Calculating the reclaimable space of a filesystem

• About this task
• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

Provisioning filesystems  –  194

 InfiniBox 4.0.10 User Documentation

11.7.1 About this task

This task calculates the reclaimable space of a filesystem. Whenever you need to delete a filesystem in
order to free storage space, you can simulate the deletion and calculate the space that the deletion
would free.

11.7.2 Prerequisites
• A filesystem

11.7.3 Before you begin

Access the filesystem from the Volumes & FS screen. 

11.7.4 GUI instructions

1.  Either:
• Select a filesystem 
• Click a filesystem to display it on screen
2. Click Calculate reclaimable from the Actions menu
A screen pops, displaying the calculation result
3. Click Close to return to the filesystem 

11.7.5 InfiniShell instructions

• fs.reclaimable - calculating the reclaimed capacity in case of filesystem deletion

11.8 Querying for filesystems and their properties

• About this task
• The filesystems table columns
• Specifying the capacity units
• InfiniShell instructions
• GUI instructions

11.8.1 About this task

The Volumes screen displays filesystems along with their properties.

Provisioning filesystems  –  195

 InfiniBox 4.0.10 User Documentation

The filesystems table columns

• Name - the name of the filesystem
• Pool - the pool that the filesystem belongs to
• Thin - indicates whether the filesystem can use the pool's virtual capacity
• Mapped - indicates whether the filesystem is mapped to a host or cluster
• Size - specified upon filesystem creation and seen by the host SCSI initiator
• Used - the physical space that is allocated to the filesystem and was actually written into
• Tree allocated - the sum of the allocated properties of the filesystem’s descendants. The value of
this property is zero (0) if there are no descendants
• If the filesystem is THICK the system will always allocate the entire size. 
• If the filesystem is THIN the system will allocate per 64KB written that are not zero. 
• For example, 1GB thin filesystem might show host wrote 1GB in the Used column and
the Allocated column can be anywhere between zero – 1GB. In this example if the host
wrote just zeros the Allocated will be zero.
• Write protected - States whether the host can write into the filesystem
• Creation date - Date and time of the creation of the filesystem

Specifying the capacity units

Use the unit argument to specify the way the capacity fields of the output are displayed. Available units
are:

• B – Byte. For example, 1000000192512

• G – GB. For example, 1000.00 GB
• GB – Same as G.
• GiB – Gibibyte. For example, 931.32 GiB.
• T – TB. For example, 1.000 TB.
• TB – same as T.
• TiB – Tebibyte. See GiB above. For example, 0.909 TiB.
• Block –512 bytes. For example, 1907349.

11.8.2 InfiniShell instructions

• Run:fs.query.

11.8.3 GUI instructions

1. Click the Volumes & FS icon on the toolbar on the left.

Provisioning filesystems  –  196

 InfiniBox 4.0.10 User Documentation

2. Click the Filesystems tab.

The Filesystems screen opens. 

11.9 Deleting a filesystem

• About this task
• Prerequisites
• Before you begin
• GUI instructions
• InfiniShell instructions

11.9.1 About this task

This task deletes a filesystem with all of its snapshots.

This operation is irreversible.

11.9.2 Prerequisites
• A filesystem

11.9.3 Before you begin

Access the filesystem from the Volumes & FS screen. Then, either:

• Select a filesystem 
• Click a filesystem to display it on screen

11.9.4 GUI instructions

1. Click Delete Filesystem from the Actions menu. 
2. Approve the warning message.
The filesystem and its descendants are deleted. 

11.9.5 InfiniShell instructions

• fs.delete - deleting a filesystem

11.10 Creating a snapshot of a filesystem

Provisioning filesystems  –  197

 InfiniBox 4.0.10 User Documentation

• About this task

• Prerequisites
• Instructions
• Related tasks
• Related InfiniShell commands

11.10.1 About this task

This task creates a snapshot of a filesystem on an InfiniBox system.

11.10.2 Prerequisites
• A filesystem

11.10.3 Instructions
1. You can create the snapshot either from the Volumes & FS screen or from the Filsystem screen.
• On the Volumes & FS screen: right-click a filesystem and select Create Snapshot from
the pop-up menu.

• On the Volumes & FS screen: Click click a Filesystem to open the Filesystem view. Click
the Actions menu and select Create Snapshot. 
The Create Snapshot screen opens.

Provisioning filesystems  –  198

 InfiniBox 4.0.10 User Documentation

2. Provide a name for the snapshot and click Create.

The snapshot is visible on the filesystem view.

It is also visible on the Volumes & FS view. Click the expansion arrow that is placed to the left of
the filesystem to expand the filesystem tree and reveal the snapshot.

11.10.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related InfiniShell commands

• fs.snap.cache -  enabling the snapshot to use InfiniBox flash cache
• fs.snap.create - creating a snapshot of a filesystem
• fs.snap.delete - deleting a snapshot
• fs.snap.query - listing the snapshots and their properties
• fs.snap.rename - renaming the snapshot

Provisioning filesystems  –  199

 InfiniBox 4.0.10 User Documentation

• fs.snap.tree - displaying the hierarchy of the snapshot and its descendants

11.11 Restoring a filesystem from its snapshot

• About this task
• Restoring relations among the filesystem and its descendants
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands

11.11.1 About this task

This task restores a filesystem from one of its snapshots.

Restoring relations among the filesystem and its descendants

The filesystem and its snapshots form an ancestry tree. Within the tree, the filesystem can be restored
from each of its snapshots.

11.11.2 Prerequisites
• A filesystem
• A filesystem snapshot

11.11.3 Before you begin

1. Access the filesystem from the Volumes & FS screen. 
2. Select a filesystem.
3. In the Snapshot tab, expand the view to see the filesystem snapshots.

Provisioning filesystems  –  200

 InfiniBox 4.0.10 User Documentation

11.11.4 Instructions
1. Select a snapshot and select Restore from this Snapshot from the Actions menu.

The Restore from Snapshot screen opens.

2. Select which ancestor of the snapshot to restore.
3. Click Restore.
The selected filesystem is restored from the snapshot.

11.11.5 Related InfiniShell commands

Modifying the filesystem attributes using InfiniShell is available via the following commands:

• fs.restore - restores a filesystem from one of its snapshots

11.12 Modifying a filesystem snapshot

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands

11.12.1 About this task

This task modifies the filesystem snapshot name.

11.12.2 Prerequisites
• A filesystem snapshot

Provisioning filesystems  –  201

 InfiniBox 4.0.10 User Documentation

11.12.3 Before you begin

1. Access the filesystem from the Volumes & FS screen. 
2. Select a filesystem
3. Select a snapshot of a filesystem

11.12.4 Instructions
1. Select Modify Snapshot from the Actions menu. The Modify Snapshot screen opens.
2. Modify the filesystem snapshot name.
3. Click Modify.
The filesystem snapshot is modified.

11.12.5 Related InfiniShell commands

Modifying the filesystem attributes using InfiniShell is available via the following commands:

• fs.snap.rename - renaming the filesystem snapshot

11.13 Deleting a filesystem snapshot

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands

11.13.1 About this task

This task deletes a filesystem snapshot.

• The snapshot descendants will also be deleted

11.13.2 Prerequisites
• A filesystem snapshot

Provisioning filesystems  –  202

 InfiniBox 4.0.10 User Documentation

11.13.3 Before you begin

1. Access the filesystem from the Volumes & FS screen. 
2. Select a filesystem 
3. Select a snapshot of a filesystem

11.13.4 Instructions
1. Select Delete Snapshot from the Actions menu. 
2. Approve the warning message
The filesystem snapshot is deleted.

11.13.5 Related InfiniShell commands

• fs.snap.delete - delete the filesystem snapshot

11.14 Deleting an export of a filesystem or a snapshot

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands
• Related GUI tasks

11.14.1 About this task

This task deletes an export of a filesystem or a snapshot.

11.14.2 Prerequisites
• A filesystem, or a filesystem snapshot

11.14.3 Before you begin

Access the filesystem from the Volumes & FS screen.

• If needed, select a filesystem, expand its tree and select a snapshot.

Provisioning filesystems  –  203

 InfiniBox 4.0.10 User Documentation

11.14.4 Instructions
1. On the screen, click the Exports tab.
2. Select an export and then select Delete Export from the Actions menu
The export is deleted

11.14.5 Related InfiniShell commands

• fs.export.delete - deletes an export path of a filesystem

11.14.6 Related GUI tasks

• Exporting a filesystem
• Exporting a filesystem snapshot

11.15 Disabling an export

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands
• Related GUI tasks

11.15.1 About this task

This task disables an export of a filesystem, or a snapshot, so the host cannot use it in order to access
the filesystem.

• The export path is not deleted, and can be enabled.

• When the export is disabled, the user loses access to the data.

11.15.2 Prerequisites
• An export of a filesystem or a filesystem snapshot

Provisioning filesystems  –  204

 InfiniBox 4.0.10 User Documentation

11.15.3 Before you begin

• Access the filesystem from the Volumes & FS screen.
• If needed, select a filesystem, expand its tree and select a filesystem snapshot
• Click the Exports tab

11.15.4 Instructions
1. Select an export and then select Disable Export from the Actions menu
2. Approve the warning message
The export is disabled.

11.15.5 Related InfiniShell commands

• fs.export.disable - disables an export path of a filesystem, or a snapshot

11.15.6 Related GUI tasks

• Exporting a filesystem
• Exporting a filesystem snapshot

11.16 Exporting a filesystem

• About this task
• Prerequisites
• Before you begin
• Basic settings
• Advanced settings
• Instructions
• Related tasks

11.16.1 About this task

This task creates a filesystem export.

11.16.2 Prerequisites
• A filesystem 

Provisioning filesystems  –  205

 InfiniBox 4.0.10 User Documentation

11.16.3 Before you begin

The Export is created with basic setting (see below). You can change the settings upon the creation of
the Export, or later on (see here: Modifying the export permissions).

Read and write size fields accept values up to 9*1024*1024*1024 byte. 

Basic settings

Parameter Description Default value Accepted value

range

Export path The external path that is seen by clients "/ • Starts with a
as they mount. <filesystemNam
e>" "/"
• Cannot have
spaces
• Accepts: A-Z,
a-z, 0-9

Internal path The directory being exported. "/" • Starts with a

The directory has to exist prior to the "/"
export creation.
• Cannot have
spaces
• Accepts: A-Z,
a-z, 0-9

Advanced settings

Parameter Description Def Accepted value

ault range
valu
e

Privileged ports only When set to on, prevents users with root Off
permissions from mounting the export with
source port >= 1024

Provisioning filesystems  –  206

 InfiniBox 4.0.10 User Documentation

Parameter Description Def Accepted value

ault range
valu
e

Max read size The maximum allowed read size for this export 1MB Binary value: 512
bytes to 8MiB
(8,388,608 bytes)

Max write size The maximum allowedwritesize for this export 1MB

Preferred read size The read IO size preferred by this export 64KB

Preferred write size The write IO size preferred by this export 64KB

Preferred readdir siz Preferred output size for readdir command. 64KB

e Affects the performance of listing directories with
high file counts

Return 32-bit FileIDs Causes InfiniBox to zero out the top 32 bits Off
of iNode so it doesn't break some older OSes and
applications

Anonymous UID The UID given to users after squashing their • A numerical
original UID
value 0-65535

Anonymous GID The GID given to users after squashing their • A numerical
original GID
value 0-65535

Squash Users and When set to on, all UID and GID will be set to the Off
groups anonymous UID & GID

Provisioning filesystems  –  207

 InfiniBox 4.0.10 User Documentation

11.16.4 Instructions
1. Click the Exports tab on the filesystem screen, and click Create.

2. Enter the Export Path and Internal Path and click Create.

The export is displayed on the screen.

Provisioning filesystems  –  208

 InfiniBox 4.0.10 User Documentation

3. Click the export path to see its details.

4. Click Add Rule. A rule is added, with the following default settings:

• Client - * (all clients)
• Access - Read/Write
• Root access - no
5. It is possible to define rules for
a. Specific client using IP address, ex: 10.0.0.1
b. Network range, ex: 10.0.0.1-10.0.0.5
c. All clients, ex: *
6. In case of overlapping rules, the following order is defined:
a. First, host IPs rules are evaluated
b. Then, network ranges are evaluated. 
If there is an overlap between two ranges:
i. Between rw and ro - rw wins
ii. Root permission to be defined by the same rule which is selected to define Access
iii. Example:

Client Access Root Actual permission

10.0.0.1-32 RO Yes RO-Yes for all clients but 2 IPs below

10.0.0.4 RW No RW-No

10.0.0.5 RW Yes RW-Yes

c. Access rule for the remaining clients is defined by '*' 

Provisioning filesystems  –  209

 InfiniBox 4.0.10 User Documentation

11.16.5 Related tasks

• To edit the permissions, see: Modifying the export permissions
• This task and all other filesystem tasks are available both on the GUI and InfiniShell.

11.17 Exporting a filesystem snapshot

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands
• Related GUI tasks

11.17.1 About this task

This task exports a filesystem snapshot, making it visible to the clients.

11.17.2 Prerequisites
• A filesystem snapshot

11.17.3 Before you begin

1. Access the filesystem from the Volumes & FS screen. 
2. Select a filesystem
3. Click a snapshot of a filesystem

11.17.4 Instructions
1. On the filesystem snapshot screen, click the Exports tab.
2. Click Create. The Create Export screen opens.
3. Create an export path
4. You may also modify an internal export path
5. Approve, or overwrite the default export settings
6. Click Create.
The filesystem snapshot has an export path.

Provisioning filesystems  –  210

 InfiniBox 4.0.10 User Documentation

11.17.5 Related InfiniShell commands

The following command is applicable to both filesystems and snapshots:

• fs.export.create - creates an export path to a filesystem, or a snapshot

11.17.6 Related GUI tasks

• Exporting a filesystem

11.18 Modifying the export permissions

• About this task
• Available access rights
• Prerequisites
• Instructions
• Related tasks
• Related InfiniShell commands

11.18.1 About this task

This task provides a client with access to the filesystem.

The instructions below apply to:

• Filesystem
• Filesystem snapshot

11.18.2 Available access rights

The client is assigned with one of the following access rights:

• Access - either of the following: read-only, read-write, none

• Root access - either permitted or not permitted

These access rights are only for the export. Access rights to the filesystem itself are set by the InfiniBox
admin (See: Overview of user management). 

11.18.3 Prerequisites
• A host, an IP address or a range of IP addresses
• a Filesystem export 

Provisioning filesystems  –  211

 InfiniBox 4.0.10 User Documentation

11.18.4 Instructions
1. Click an Export on the Exports tab on the filesystem's screen. The Export Permissions tab
opens.
2. Click Add Rule to add a new permission setting.
3. A new row is added to the screen. Insert a client - either a wildcard, an IP address or a range of IP
addresses. Select an access level and then whether to allow a root access. Click Done.
Client examples:
• 172.16.66.172
• 172.16.66.172-172.16.66.180
• Wildcard (*)
The screen closes. To view, add, or remove the permissions settings, repeat step 1. 

11.18.5 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related InfiniShell commands

• fs.export.permission.add - adding a permission to an export
• fs.export.permission.modify - modifying attributes of a permission
• fs.export.permission.query - querying for the permissions of an export
• fs.export.permission.remove - removing a permission from an export

11.19 Modifying the export settings

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related InfiniShell commands
• Related GUI tasks

11.19.1 About this task

This task modifies the following export settings:

Provisioning filesystems  –  212

 InfiniBox 4.0.10 User Documentation

Basic settings

Parameter Description Default value Accepted value

range

Export path The external path that is seen by clients "/ • Starts with a
as they mount. <filesystemNam
e>" "/"
• Cannot have
spaces
• Accepts: A-Z,
a-z, 0-9

Internal path The directory being exported. "/" • Starts with a

The directory has to exist prior to the "/"
export creation.
• Cannot have
spaces
• Accepts: A-Z,
a-z, 0-9

Advanced settings

Parameter Description Def Accepted value

ault range
valu
e

Privileged ports only When set to on, prevents users with root Off
permissions from mounting the export with
source port >= 1024

Max read size The maximum allowed read size for this export 1MB Binary value: 512
bytes to 8MiB
(8,388,608 bytes)

Max write size The maximum allowedwritesize for this export 1MB

Preferred read size The read IO size preferred by this export 64KB

Provisioning filesystems  –  213

 InfiniBox 4.0.10 User Documentation

Parameter Description Def Accepted value

ault range
valu
e

Preferred write size The write IO size preferred by this export 64KB

Preferred readdir siz Preferred output size for readdir command. 64KB

e Affects the performance of listing directories with
high file counts

Return 32-bit FileIDs Causes InfiniBox to zero out the top 32 bits Off
of iNode so it doesn't break some older OSes and
applications

Anonymous UID The UID given to users after squashing their • A numerical
original UID
value 0-65535

Anonymous GID The GID given to users after squashing their • A numerical
original GID
value 0-65535

Squash Users and When set to on, all UID and GID will be set to the Off
groups anonymous UID & GID

11.19.2 Prerequisites
• An export of a filesystem, or a filesystem snapshot

11.19.3 Before you begin

• Access the filesystem, or snapshot, from the Volumes & FS screen.
• If needed, select a filesystem, expand its tree and select a filesystem snapshot 
• Click the Exports tab

11.19.4 Instructions
1. Select an export and then select Modify Export from the Actions menu of the Exports tab.
2. Modify the export settings as needed.
3. Click Modify.
The export settings are modified.

Provisioning filesystems  –  214

 InfiniBox 4.0.10 User Documentation

11.19.5 Related InfiniShell commands

• fs.export.modify - modifies the export and its permissions

11.19.6 Related GUI tasks

• Exporting a filesystem
• Exporting a filesystem snapshot
• Modifying the export permissions

11.20 Optimized reading and writing small files

• Handling files smaller than 128B
• Performance gain
• Further enhancements

11.20.1 Handling files smaller than 128B

In order to improve the efficiency of storing small files (smaller than 128 Bytes), the files are written into
the Btree.

Performance gain
As the files are written faster (into one place instead of two) they are consequentially read faster, when
required.

• Files that are larger than 128B occupy a section and affect the amount of free space. 

11.20.2 Further enhancements

Further optimization for small files are coming in the next NAS release

11.21 Querying for NFS clients and the filesystem they are using
• About this task

Provisioning filesystems  –  215

 InfiniBox 4.0.10 User Documentation

• Note
• How to query for NFS clients and the filesystem they are using
• From InfiniShell
• Filtered by a filesystem:
• Filtered by the source IP: 
• From the GUI

11.21.1 About this task

InfiniBox provides a way to query which clients are connected to a specific filesystem, similar to the
host-side showmount -a command.

Note
Only live IOs can be queried. The query does not display mounted filesystems that have no activity.

11.21.2 How to query for NFS clients and the filesystem they are using
The NFS IO can be filtered by either a filesystem or an export.

From InfiniShell

Filtered by a filesystem:

metrics.nas.fs.show fs=fs1

Filtered by the source IP: 

First, look up the incoming clients, using this command:

metrics.nas.source_ip.top fs=fs1

The output looks like this:

Provisioning filesystems  –  216

 InfiniBox 4.0.10 User Documentation

System ibox1150
Top NAS source IPs in system by OPS

Collecting data...
System ibox1150
Top NAS source IPs in system by OPS

SOURCE IP OPERATIONS THROUGHPUT LATENCY

172.16.88.152 210 13 MiB/sec 0.175 ms

Once we have a list of NFS Client IPs, we can show the detailed performance for that NFS client.

metrics.nas.source_ip.show source_ip=172.16.88.152

The output looks like this:

System ibox1150
Source IP '172.16.88.152' NAS performance in system

Collecting data...
System ibox1150
Source IP '172.16.88.152' NAS performance in system

--- OPERATIONS THROUGHPUT LATENCY

WRITE 0 - 0.000 ms
METADATA 0 - 0.000 ms
READ 75 4 MiB/sec 0.154 ms

Provisioning filesystems  –  217

 InfiniBox 4.0.10 User Documentation

From the GUI

1. Select Performance from the InfiniBox GUI menu.

2. On the Performance screen, add a chart for Top NAS Client IP.

Provisioning filesystems  –  218

 InfiniBox 4.0.10 User Documentation

3. Click the Filter button.

4. Select a filesystem or an export.

11.22 Overview of filesystem tree quota

• Limiting the filesystem capacity and number of iNodes
• Examples of quotas
• Snapshots tree quota
• Target filesystems
• Calculating the quota of symbolic links
• Limits
• Types of quotas
• Available quota types
• Implementation of both quota types
• Inapplicability of TreeQ for some filesystems
• Quota states

Provisioning filesystems  –  219

 InfiniBox 4.0.10 User Documentation

• Transition between states

• Hysteresis - decreasing the number of generated events
• Crossing the soft quota threshold
• Crossing the hard quota threshold
• Writing to a nearly full TreeQ
• Moving hard links between TreeQs
• Moving files across TreeQ boundaries

11.22.1 Limiting the filesystem capacity and number of iNodes

InfiniBox allows imposing capacity limits on the filesystem tree level. The limit can be either on the
capacity of the tree or on the number of iNodes, or on both. The TreeQ feature provides the storage
admin with means for controlling the usage of system resources (represented by trees).

InfiniBox multiple levels of capacity and operational provisioining:

• Pools - the InfiniBox system is divided into pools - storage units that can impose capacity limits
on the filesystems (and volumes) that are in them
• Filesystems - the filesystem stores many files and deploys protection schemes (snapshots and
replica) on the filesystem unit
• TreeQ - a filesystem subtree (or a folder) that can be assigned with a quota on the number of
files or the capacity. A single filesystem can contain thousands of TreeQs

For ease of operation, INFINIDAT recommends to create a small number of filesystems, keeping in mind
that the number of files within a single filesystem is practically unlimited and a high file count does not
incur a performance impact on InfiniBox.

Provisioning filesystems  –  220

 InfiniBox 4.0.10 User Documentation

Examples of quotas
The quota can be imposed on either the capacity or number of iNodes, or both.

• 2GB capacity
• 1,000 iNodes
• 1GB capacity and 1,200 iNodes (in such a case, the first hard quota to be reached, blocks the
subtree IO. See below for more details)

Snapshots tree quota

• Writable snapshots inherit their TreeQ definitions from the parent filesystem. These definitions
can be overriden
• Restoring a filesystem from a snapshot restores the TreeQ and their definitions to the time the
snapshot was created

Target filesystems
• A filesystem that is a target of a replica inherits the TreeQ definitions from the source filesystem.
These definitions cannot be modified on the target, as long as it is part of the replica.
• When a quota is reached an event is generated on the source side only 

Calculating the quota of symbolic links

As symbolic links are small enough to be stored within the filesystem metadata structure, they do not
require an actual file. A symbolic link quota is calculated as follows:

• 1 iNode
• 0 capacity

Limits
• The TreeQ can have a capacity of 500MB - 10,000 PB
• The TreeQ can have up to 100*108 iNodes
• The number of TreeQ per filesystem is 4,095 
• Size limits:
• Hard capacity >= 1GB (1*109, not GiB)
• Hard capacity - Soft capacity >= 100MB (100*106, not MiB)
• Soft inode > 1

Provisioning filesystems  –  221

 InfiniBox 4.0.10 User Documentation

• Soft capacity >= 500 MB (500*106, not MiB)

11.22.2 Types of quotas

Available quota types

The quota that is associated with a filesystem tree comes in two types:

• Soft quota - A threshold for alerting the storage administrator of high quota usage, without
blocking writes. This allows the administrator time to respond before writes will be rejected 
• When the soft quota is exceeded: 
• An event is generated within a 50MB window above the quota
• The tree keeps accepting IO
• Hard quota - a threshold that blocks IO when reached.
• When the hard quota is reached: 
• An event is generated when the a user tries to reach above the assigned quota
(iNodes, Capacity)
• Events are only generated once per hour per TreeQ hard quota
• The event may be triggered within a 1MB window below the quota
• For capacity quota: the tree ceases accepting writes. Read and delete operations are
accepted 
• For iNodes quota: the TreeQ ceases accepting creates (while writes to existing
iNodes are not ceased)

Implementation of both quota types

• When both soft quota and hard quota are implemented all of the above behavior is valid, with
the exception of a case when both quota types are defined. Both soft and hard quota can be
reached within an atomic user operation (for example, when a file is moved into a TreeQ). In such
a case, the event for crossing the soft quota is not generated.
• This behavior is applicable for either capacity or file count.
• As a convention, as the soft quota incurs softer measures than the hard quota (generating an
event vs. blocking I/O), the soft quota is always smaller than the hard quota.

Inapplicability of TreeQ for some filesystems

Note that the TreeQ feature is not applicable for:

Provisioning filesystems  –  222

 InfiniBox 4.0.10 User Documentation

• Write-protected filesystems
• Locked filesystems

To enable TreeQ for such filesystems, they need to get out of these states (i.e. write-enabled, or move
them to a non-locked pool).

Provisioning filesystems  –  223

 InfiniBox 4.0.10 User Documentation

11.22.3 Quota states

Each of the TreeQ's quotas (Capacity / iNodes) is has one of the following states:

• NO_QUOTA - when no quota was set by the admin (the system will only count usage but won't
send events / block the user from writing)
• BELOW_QUOTA - the soft limit was not exceeded yet, users can write and no event will be
triggered
• SOFT_EXCEEDED - the TreeQ has exceeded its soft quota but has not reached the hard limit (user
can still write data)

Transition between states

• The soft quota threshold is always crossed before the hard quota threshold
• The soft and hard quota can be crossed together, as discussed above
• The filesystem state returns to BELOW_QUOTA when the capacity (or number of iNodes)
decrease

Hysteresis - decreasing the number of generated events

As the capacity of the filesystem subtree may frequently change and repeatedly keep crossing (or
reaching) a threshold, InfiniBox implements a mechanism for decreasing the number of generated
events.

Crossing the soft quota threshold

The following image provides examples for crossing the soft quota:

1. Both soft quota and hysteresis thresholds were crossed, so an event is generated

Provisioning filesystems  –  224

 InfiniBox 4.0.10 User Documentation

2. The capacity is above the soft quota, so when new capacity was added, the soft quota threshold
was not crossed. No event is generated
3. The capacity moves from below the soft quota to above the soft quota, but the hysteresis
threshold was not crossed. No event is generated
4. Both soft quota and hysteresis limit were crossed so an event is generated

Crossing the hard quota threshold

The event generation in a case where the hard quota is reached depends on the time that has passed
since the last event. The default time interval is 60 minutes. 

Cas Time that has passed since the last Event is Writes / file creation
e event generated? blocked?

1 1st time yes yes

2 < configurable time no yes

3 > configurable time yes yes

Provisioning filesystems  –  225

 InfiniBox 4.0.10 User Documentation

Writing to a nearly full TreeQ

When creating a file and writing into it while the TreeQ is nearly full, the size of the file may exceed the
TreeQ size. In such a case, the file will be created but the write operation will fail.

Note that as the TreeQ is smaller than the filesystem, the file can be moved to another TreeQ and
remains writable.

11.22.4 Moving hard links between TreeQs

Hard links crossing TreeQs are not allowed. When moving a hard link to another TreeQ, the Linux "mv"
command will fail back to copy and remove operations instead.

Moving files across TreeQ boundaries

Moving files from one TreeQ to another undergoes as follows:

1. The directory structure is recreated as it cannot be copied

2. The files are moved to the new directory
3. The old directory - now empty from files - is deleted

11.23 TreeQ limitations

• Hard links cannot cross the TreeQ boundaries
• Converting a directory to TreeQ

Provisioning filesystems  –  226

 InfiniBox 4.0.10 User Documentation

• Regular directory cannot be converted into TreeQ. 

• TreeQ cannot be converted to regular directory
• When moving files across TreeQ boundaries, the directory structure is recreated and the
files are moved
• Replication
• Filesystem with TreeQ can be replicated only to an InfiniBox with a software version that
supports TreeQ
• A TreeQ of a target filesystem cannot be queried
• Moving a TreeQ
• Moving a TreeQ is rejected by InfiniBox 
• However, some implementations of the mv command (Linux, for example) will try to copy
the TreeQ to another location and then delete the source.
• The newly copied folder will not be a TreeQ (thus, the TreeQ will have to be created by the
storage admin).
• CLI auto-complete doesn't work on TreeQ name / path
• For improved performance, the soft capacity state change and event generation are slightly
decoupled
• The state changes immediately when you cross the soft capacity
• The event can be triggered at any point between the soft capacity and the addition of
50MB

Provisioning filesystems  –  227

 InfiniBox 4.0.10 User Documentation

12 Consistency groups

12.1 Overview of consistency groups

• What is a consistency group
• Pool association
• Creating a snapshot group
• Adding a member to a consistency group
• Removing a member from a consistency group

12.1.1 What is a consistency group

A consistency group is a group of volumes that maintain write-order fidelity.

The consistency group allows to take snapshots of all of its volumes at the same point-in-time, thus
ensuring a consistent image of the volumes at that time. The consistency group is typically used when a
single application spans multiple volumes and there is a need to create a point-in-time copy of these
volumes. From the application perspective, point-in-time backups of the entire application are crucial
for maintaining data integrity.

12.1.2 Pool association

• Upon creation, the consistency group is associated with a pool.
• All of the volumes of a consistency group reside in the same pool. 
• The consistency group can be moved between pools, but individual members cannot move
between pools independently. 
• When a consistency group moves, all of its members and their snapshots move with it.

12.1.3 Creating a snapshot group

Upon creating a snapshot group, a snapshot is created for each of the consistency group's datasets
(volumes). All of these snapshots are taken at the same point-in-time and are grouped in a snapshot
group. The snapshot group depicts the consistency group at a specific point in time. Restoring the
consistency group from one of its snapshot groups returns the members to this point in time. The
following image shows a consistency group along with its snapshot group.

Consistency groups  –  228

 InfiniBox 4.0.10 User Documentation

The consistency group can be restored from its snapshot group (subject to specific constraints - see
further down this page). Upon a restore, any of the consistency group's datasets is restored from its
matching snapshot.

Consistency groups  –  229

 InfiniBox 4.0.10 User Documentation

The consistency group and its members all belong to the same pool.

12.1.4 Adding a member to a consistency group

Following an addition of a member to a consistency group and prior to taking another snapshot group,
the consistency group is in a state where, if it needs to be restored from one of its snapshot group, the
snapshot group does not have a snapshot for each of the consistency group members. In the example

Consistency groups  –  230

 InfiniBox 4.0.10 User Documentation

that is provided in the following image, a consistency group (1) has two members. A snapshot group is
created, in which there are two snapshots, one for each of the consistency group's members (2). A new
member is added to the consistency group, creating a situation in which the snapshot group cannot
restore the entire consistency group, if asked to do so (3). Upon the creation of a new snapshot group, it
contains a snapshot for each of the consistency group members (4).

12.1.5 Removing a member from a consistency group

Usually, the snapshot group contains snapshots in a number that matches the number of members of
the consistency group (see 1 in the image below). When a member is removed from a consistency
group, the snapshot groups all have a superfluous member (2). The snapshot groups are adjusted - by
removing the no longer needed snapshot - so they again have a number of snapshots that matches the
number of members in the consistency group. 

Consistency groups  –  231

 InfiniBox 4.0.10 User Documentation

12.2 Creating a consistency group

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.2.1 About this task

This task creates a consistency group on an InfiniBox system.

12.2.2 Prerequisites
• A pool 

Consistency groups  –  232

 InfiniBox 4.0.10 User Documentation

12.2.3 Before you begin

The purpose of the consistency group is to group volumes in a way that will allow to take consistent
snapshots for all of them. A group of consistent snapshots can be later used for replicating all of the
grouped volumes together, maintaining a consistent replica for all of them.

12.2.4 Instructions
1. On the GUI: click the Volumes & FS icon. Then, click the Consistency Groups tab.
The Consistency Groups screen opens.
2. Click Create Group.

 
The Create a Consistency Group screen opens. 
3. Enter a name for the consistency group.
Select a pool from the list of available pools.
Click Create Group.

The consistency group is created and is displayed on screen.

Consistency groups  –  233

 InfiniBox 4.0.10 User Documentation

12.2.5 Related tasks

This task and all other consistency group tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Adding a member to a consistency group

Related InfiniShell commands

• cg.create - create a new consistency group
• cg.add_member - add a member to a consistency group
• cg.delete - delete a consistency group
• cg.move - move a consistency group to another pool
• cg.remove_member - remove a member from a consistency group
• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.3 Adding a member to a consistency group

• About this task
• Prerequisites
• Adding a member to a replicating consistency group
• Before you begin
• Pool association
• Instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.3.1 About this task

This task adds a member to a consistency group on an InfiniBox system.

Consistency groups  –  234

 InfiniBox 4.0.10 User Documentation

12.3.2 Prerequisites
• A consistency group
• Datasets (volumes) that are added as members to the consistency group

Adding a member to a replicating consistency group

When adding a member to a replicating consistency group, the consistency group has to fulfill the
following prerequisites:

• The link has to be up

• The replica state cannot be Initializing

For more information, see here: Adding a member to, or removing a member from, a replicated
consistency group whose replication type is SYNC

12.3.3 Before you begin

The purpose of the consistency group is to group datasets in a way that will allow to take consistent
snapshots for all of them.

• Note: Only volumes qualify as members of a consistency group.

Pool association
• Upon creation, the consistency group is associated with a pool.
• All of the volumes of a consistency group reside in the same pool. 
• The consistency group can be moved between pools, but individual members cannot move
between pools independently. 
• When a consistency group moves, all of its members and their snapshots move with it.

12.3.4 Instructions
1. Click on a consistency group on the Consistency Groups tab.
The Consistency Groups screen opens. If the consistency group has no members yet - for
example, if it was just created - a bubble pops, indicates the availability of the option to add

Consistency groups  –  235

 InfiniBox 4.0.10 User Documentation

members.

2. Click Add Members. Two options are made available on screen: 

Create Volume - create new volumes that will be members of the consistency group.
Add to Group - add volumes - from the list of volumes - to the consistency group

3. Creating / adding volumes:

• In case of adding volumes: the list displays volumes from the same pool of the consistency
group. Check the desired volumes and click Add to Group.
The volumes are added to the consistency group.

Consistency groups  –  236

 InfiniBox 4.0.10 User Documentation

• In case of creating a new volume, a Create Volume screen opens.

• You can create - and add to the consistency group up to 100 volumes, according to the
following guidelines:
• Setting index numbers to volumes. The created volumes will be indexed from 1 to
the set number. For example: 100.
• Setting a range of index numbers. This option accepts any range under a limitation
of 7 characters (hyphen included). For example: 101-200, 900-999. The largest index
number available is 999.
• Click Create to create the volumes. The volumes are displayed on the screen.
• Select the volumes and click Add to Group to add them to the consistency group.

12.3.5 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group

Related InfiniShell commands

• cg.create - create a new consistency group

Consistency groups  –  237

 InfiniBox 4.0.10 User Documentation

• cg.add_member - add a member to a consistency group

• cg.delete - delete a consistency group
• cg.move - move a consistency group to another pool
• cg.remove_member - remove a member from a consistency group
• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.4 Consistency group replication concepts

• Replicating a consistency group
• The replica RPO
• Adding a member to a replicating consistency group
• Impact on the consistency group state
• Removing a member from a replicating consistency group
• Capacity and throughput reporting

12.4.1 Replicating a consistency group

The consistency group is replicated as a whole. The replica is comprised of a source consistency group
and a target consistency group. All of the consistency group's members are replicated, and the replica
state of the consistency group abides all of its members.

12.4.2 The replica RPO

For instructions on how to set the RPO, see: How to set the replica RPO and interval.

The RPO (Recovery Point Objective) is the maximum time the replica is allowed to lag. The RPO is
configured with another parameter - the Sync Interval. The RPO has to be greater than the Sync Interval
and the sync job is expected to be completed prior to the completion of the Sync Interval. The best
practice is to set the RPO to be twice the time of the Sync Interval.

The RPO is measured in seconds.

RPO states: 

• RPO OK – The lag between the source and target < RPO
• RPO LAGGING - The lag between the source and target > RPO
• N/A – RPO not set

Consistency groups  –  238

 InfiniBox 4.0.10 User Documentation

The system creates events for changes to the RPO status.

12.4.3 Adding a member to a replicating consistency group

As the consistency group always has all of its members replicated (or all of them not replicated), adding
a member to a consistency is available according to the following guidelines:

• It is OK to add a non-replicating dataset to a non-replicating consistency group

• it is impossible to add a replicating member to a non-replicating consistency group.

Adding a member to a replicating consistency group is done by adding a source member to the source
consistency group and instructing the replica to either add a target member to the target consistency
group or to create a target member to a non-replicating member that is added to the source. The
instruction is provided to the source system, where the Adding a Member operation takes place.

The consistency group settings (interval and RPO) are imposed on the added member.

Impact on the consistency group state

The added member undergoes initialization upon the next sync job. This causes the entire consistency
group to undergo an initialization. A replicating consistency group is usually in either replicating or idle
state. Adding a member to the consistency group forces the entire consistency group to yield to an
initializing state. This state is imposed on the consistency group even if all of its members (but the new
addition) are synchronized with the target. 

A consistency group that is in initialization state cannot have its role changed in case of a disaster
recovery. In order to change the role of the consistency group, the initializing member has to be

Consistency groups  –  239

 InfiniBox 4.0.10 User Documentation

removed - losing any advancement in replicating to the target member - in order to move the
consistency group out of the initializing state and allow the role change.

It is advised to consider this implication prior to adding the new member to the consistency group. It
might prove useful to replicate this member separately - prior to adding it to the consistency group -
and only when it is synchronized with the target, to add it to the consistency group (as well as add its
target to the target consistency group).

12.4.4 Removing a member from a replicating consistency group

Removing a member from the consistency group removes the source member from the source
consistency group, removes the target member from the target consistency group, and - optionally -
retains the replica between the source and target member. If instructed by the user to be retained, the
replica of the removed dataset keeps the settings (interval and RPO) and the state (synchronized, not-
synchronized) of the consistency group. It is possible to remove a member out of a consistency group
regardless of the state of the consistency group.

12.4.5 Capacity and throughput reporting

• The system reports the capacity consumption of the replication snapshots
• The system reports the throughput of replica that is currently replicating

12.5 Consistency groups limitations

12.5.1 Limitations
• Maximum number of members in a group - 100
• Maximum number of consistency group in a system - 10,000
• Maximum number of snapshot groups per consistency group - 250

Consistency groups  –  240

 InfiniBox 4.0.10 User Documentation

12.6 Controlling the replica using the PoolAdmin user

The PoolAdmin has access rights to the pool and its entities. As such, the PoolAdmin is capable of
replicating any of the entities of the pool it administers. However, the Pool Admin cannot set up the link
between the source system and the target system. The link has to be created by the InfiniBox Admin as
a prerequisite. 

12.7 Creating a snapshot group

• About this task
• How to choose prefix and suffix
• Prerequisites
• GUI instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.7.1 About this task

This task creates a snapshot group for a consistency group. The snapshot group is comprised of a
snapshot group entity and of individual snapshots - a snapshot for each member of the consistency
group. The snapshot group name can be determined freely. The snapshot names are each comprised
of the volume name and a prefix or a suffix.

How to choose prefix and suffix

The table below provide examples for naming the snapshots.

Example 1

Members vol1,... vol5

Prefix s1-

Snapshots s1-vol1,...s1-vol5 

Example 2

Members vol1,... vol5

Consistency groups  –  241

 InfiniBox 4.0.10 User Documentation

Suffix -s1

Snapshots vol1-s1,...vol1-s5 

12.7.2 Prerequisites
• A consistency group

12.7.3 GUI instructions

1. Right-click a consistency group and select Take Snapshot from the pop-up menu.
The Take Snapshot screen opens.

2. Provide a name for the snapshot group and either a prefix or a suffix to the snapshots.
Click Take Snapshot.

Consistency groups  –  242

 InfiniBox 4.0.10 User Documentation

The snapshot group is created.

12.7.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group

Related InfiniShell commands

• cg.create - create a new consistency group
• cg.add_member - add a member to a consistency group
• cg.delete - delete a consistency group
• cg.move - move a consistency group to another pool

Consistency groups  –  243

 InfiniBox 4.0.10 User Documentation

• cg.remove_member - remove a member from a consistency group

• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.8 Deleting a snapshot group

• About this task
• Prerequisites
• Instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.8.1 About this task

This task deletes a snapshot group.

12.8.2 Prerequisites
• A snapshot group

Consistency groups  –  244

 InfiniBox 4.0.10 User Documentation

12.8.3 Instructions
1. Right-click a snapshot group and select Delete Group from the pop-up menu.

The Delete Group screen opens.

2. Select whether to delete the datasets snapshots.

Click Delete.
3. Approve the warning message.
The snapshot group is deleted.

12.8.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Consistency groups  –  245

 InfiniBox 4.0.10 User Documentation

Related GUI tasks

• Creating a consistency group
• Creating a snapshot group

Related InfiniShell commands

• cg.snap.create - create a snapshot group
• cg.snap.delete - delete a snapshot group
• cg.snap.rename - rename a snapshot group
• cg.snap.query - list all snapshot groups
• cg.snap.member_query - list all of the members of a snapshot group

12.9 Modifying a consistency group

• About this task
• Prerequisites
• Instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.9.1 About this task

This task renames a consistency group on an InfiniBox system.

12.9.2 Prerequisites
• A consistency group

Consistency groups  –  246

 InfiniBox 4.0.10 User Documentation

12.9.3 Instructions
1. Right-click a consistency group and select Modify Group from the pop-up menu.

2. Provide a new name for the consistency group and click Modify Group.

The consistency group is renamed.

12.9.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group

Related InfiniShell commands

• cg.create - create a new consistency group
• cg.add_member - add a member to a consistency group
• cg.delete - delete a consistency group

Consistency groups  –  247

 InfiniBox 4.0.10 User Documentation

• cg.move - move a consistency group to another pool

• cg.remove_member - remove a member from a consistency group
• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.10 Modifying a snapshot group

• About this task
• Prerequisites
• Instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.10.1 About this task

This task renames a snapshot group.

12.10.2 Prerequisites
• A snapshot group

Consistency groups  –  248

 InfiniBox 4.0.10 User Documentation

12.10.3 Instructions
1. Right-click a snapshot group and select Modify Group from the pop-up menu.
The Modify Group screen opens.

2. Rename the snapshot group and click Modify.

The snapshot group is renamed.

12.10.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group
• Creating a snapshot group

Related InfiniShell commands

• cg.snap.create - create a snapshot group
• cg.snap.delete - delete a snapshot group
• cg.snap.rename - rename a snapshot group
• cg.snap.query - list all snapshot groups
• cg.snap.member_query - list all of the members of a snapshot group

Consistency groups  –  249

 InfiniBox 4.0.10 User Documentation

12.11 Moving a consistency group between pools

• About this task
• Prerequisites
• Instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.11.1 About this task

This task moves a consistency group among pools.

12.11.2 Prerequisites
• A consistency group
• A second pool, in addition to the pool that the consistency group is stored in

12.11.3 Instructions
1. Right-click a consistency group and select Move to Pool from the pop-up menu.

The Move to Pool screen opens.

2. Select the pool that the consistency group will move to. Select whether to move the consistency
group along with its capacity.
Click Move.

Consistency groups  –  250

 InfiniBox 4.0.10 User Documentation

The consistency group is moved.

12.11.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group

Related InfiniShell commands

• cg.create - create a new consistency group
• cg.add_member - add a member to a consistency group
• cg.delete - delete a consistency group
• cg.move - move a consistency group to another pool
• cg.remove_member - remove a member from a consistency group
• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.12 Removing a member from a consistency group

• About this task
• Prerequisites
• Before you begin
• Instructions
• Related tasks

Consistency groups  –  251

 InfiniBox 4.0.10 User Documentation

• Related GUI tasks

• Related InfiniShell commands

12.12.1 About this task

This task removes a member from a consistency group on an InfiniBox system.

12.12.2 Prerequisites
• A consistency group
• A consistency group member that will be removed

12.12.3 Before you begin

The purpose of the consistency group is to group datasets in a way that will allow to take consistent
snapshots for all of them. Only root volumes qualify as members of a consistency group.

• When a member is removed from a consistency group, its snapshots are removed from the
respective snapshot groups. 
• When the last member is removed from a snapshot group, the snapshot group is deleted. There
are no empty snapshot groups.
• When the last member is removed from the consistency group, the consistency group remains.

12.12.4 Instructions
1. Click on a consistency group on the Consistency Groups tab.
The Consistency Groups screen opens.
2. Select a member (or several members).

 
The Remove Members button becomes available.

Consistency groups  –  252

 InfiniBox 4.0.10 User Documentation

3. Click Remove Members.

A warning message is displayed on screen.

4. Click Yes to approve the warning message.

The members are removed from the consistency group.

12.12.5 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group
• Adding a member to a consistency group

Related InfiniShell commands

• cg.create - create a new consistency group
• cg.add_member - add a member to a consistency group
• cg.delete - delete a consistency group
• cg.move - move a consistency group to another pool
• cg.remove_member - remove a member from a consistency group
• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.13 Restoring a consistency group from a snapshot group

• About this task
• Prerequisites
• Instructions

Consistency groups  –  253

 InfiniBox 4.0.10 User Documentation

• Related tasks
• Related GUI tasks
• Related InfiniShell commands

12.13.1 About this task

This task restores a consistency group from its snapshot group. This operation restores each of the
consistency group members from its snapshot, where all snapshots belong to the same snapshot
group. 

12.13.2 Prerequisites
• A consistency group
• A snapshot group

12.13.3 Instructions
1. Expand the consistency group to view its snapshot groups.

2. Right-click a snapshot group and select Restore from this Snapshot from the pop-up menu.

Consistency groups  –  254

 InfiniBox 4.0.10 User Documentation

3. Approve the warning message.

The consistency group is restored.

12.13.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group

Related InfiniShell commands

• cg.create - create a new consistency group
• cg.add_member - add a member to a consistency group
• cg.delete - delete a consistency group
• cg.move - move a consistency group to another pool
• cg.remove_member - remove a member from a consistency group
• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.14 Deleting a consistency group

• About this task
• Prerequisites
• Instructions
• Related tasks
• Related GUI tasks
• Related InfiniShell commands

Consistency groups  –  255

 InfiniBox 4.0.10 User Documentation

12.14.1 About this task

This task deletes a consistency group.

• All of the snapshot groups of this consistency group are deleted.

• The consistency group members are always retained.
• In a case that the consistency group has a snapshot group - meaning that the members have
snapshots - the snapshots also retain, but not as a part of a snapshot group.
• A dataset that is a member of a consistency group cannot be deleted individually.

12.14.2 Prerequisites
• A consistency group

12.14.3 Instructions
1. Right-click a consistency group and select Delete Group from the pop-up menu.
The Delete Group screen opens.

2. Click Delete.
The consistency group is deleted.

12.14.4 Related tasks

This task and all other filesystem tasks are available both on the GUI and InfiniShell.

Related GUI tasks

• Creating a consistency group

Consistency groups  –  256

 InfiniBox 4.0.10 User Documentation

Related InfiniShell commands

• cg.create - create a new consistency group
• cg.add_member - add a member to a consistency group
• cg.delete - delete a consistency group
• cg.move - move a consistency group to another pool
• cg.remove_member - remove a member from a consistency group
• cg.restore - restore a consistency group from one of its snapshot groups
• cg.rename - rename a consistency group
• cg.query - list all consistency groups
• cg.member_query - list all of the members of a consistency group

12.15 Adding a member to, or removing a member from, a replicated

consistency group whose replication type is SYNC
• About this task
• Prerequisites
• Instructions
• Related topics

12.15.1 About this task

Adding a member to a consistency group and removing a member from a consistency group
are allowed only for consistency groups whose replication type is ASYNC. Whenever we want to add or
remove a member, we need to assure that the replication type of the consistency group is ASYNC. In
addition, in case that the added member is already replicated, its replication type has to be ASYNC, as
well.

Prerequisites
• The link that the replica is using has to be up during the add / remove action

12.15.2 Instructions
1. Verify that the replication type of the consistency group is ASYNC
a. If the replication type is SYNC, change it to ASYNC
2. Verify that the replication type of the member is ASYNC

Consistency groups  –  257

 InfiniBox 4.0.10 User Documentation

a. If the replication type is SYNC, change it to ASYNC

b. If the member is not replicated, disregard this step
3. Add or remove the member
4. Change the replication type of the consistency group to SYNC

12.15.3 Related topics

• Adding a member to a replicated consistency group
• Removing a member from a replicated consistency group

12.16 Removing a member from a replicated consistency group

• About this task
• Prerequisites
• Following the removal
• Before you begin
• Instructions

12.16.1 About this task

This task removes a member from a replicated consistency group on an InfiniBox system.

Prerequisites
• The link has to be up

Following the removal

• A new snapshot group is created, depicting the member removal (that is, no longer containing a
snapshot of the removed member)
• When the last member is removed from a replicating consistency group, the consistency group
retain the replication properties

12.16.2 Before you begin

The purpose of the consistency group is to group datasets in a way that will allow taking consistent
snapshots for all of them. When a member is removed from a consistency group, it is no longer
consistent with the other members of the group. If needed, the member can be moved to a pool other
that the pool of the consistency group.

Consistency groups  –  258

 InfiniBox 4.0.10 User Documentation

12.16.3 Instructions
1. Click on a replicated consistency group on the Consistency Groups tab.
The Consistency Groups screen opens. 
2. Select a member (or several members) and click Remove Members. Three options are made
available on screen: 
a. Keep replicating the member individually - the member stays replicated and preserves its
synchronization status
b. Stop replicating the member - the member stops replicating and the staging area (the
replication-associated snapshots) is deleted
c. Stop replicating the member, but retain the staging area - the member stops replicating
but the staging area (the replication-associated snapshots) is preserved
3. Click Remove. The member is removed from the consistency group.

12.17 Adding a member to a replicated consistency group

The content of this topic was unified into: Adding a member to a consistency group.

Consistency groups  –  259

 InfiniBox 4.0.10 User Documentation

13 Link

13.1 Creating a link

• About this task

13.1.1 About this task

The link is the entity that connects the source InfiniBox system to the a target system using a
predefined service. The image below shows the entities hierarchy from the interfaces, through the
network space and service all the way up to the links.

 The instructions that were provided on this page are now available on the InfiniBox Best
Practices for Setting up Services documents, that are available here.

13.2 Authenticating a link

• About this task
• Automatic authentication
• Manual authentication
• Prerequisites:
• GUI instructions
• InfiniShell instructions
• Link-related InfiniShell commands

13.2.1 About this task

The link is an entity that connects the source InfiniBox system to the a target system using a predefined
service. The link has to be authenticated on the target system.

Automatic authentication
The link is automatically authenticated on the target system whenever the following conditions are met:

• You are logged into the source system with the default Admin user
• The default Admin user exists on the remote system

Link  –  260

 InfiniBox 4.0.10 User Documentation

Manual authentication
A manual authentication is required whenever:

• You are logged into the source system with a user other than Admin (for example, a PoolAdmin)

In such a case, follow the instructions below.

13.2.2 Prerequisites:
This task assumes that the following entities are defined:

• Link

13.2.3 GUI instructions

1. Upon creating the link, the source system checks whether an automatic authentication is viable.
If it is not, a screen pops up.
Fill in the user credentials for the target system.
The link is now authenticated.

13.2.4 InfiniShell instructions

Run link.authenticate providing the target system and user credentials.

Link-related InfiniShell commands

This section describes commands that create and authenticate links between InfiniBox systems.

• link.create - This command creates a link between a network space on the local system and a
target system. Such a link is required for services that span across systems, for example, 
Asynchronous Remote Replication.
• link.authenticate - This command verifies that the user of the local system can operate the
remote system as well
• link.delete - This command deletes the link
• link.query - This command lists the links that are defined on the system along with their state
and the last time they were in use

The commands are displayed in length, including examples on the InfiniShell Reference Guide.

Link  –  261

 InfiniBox 4.0.10 User Documentation

13.3 Detaching and attaching a link

• About this task

• Link states
• IP addresses redistribution
• Changing the network space configuration
• Prerequisites
• See also
• GUI instructions 
• Detaching a link 
• Attaching a link

13.3.1 About this task

The main purpose of the ability to detach a link from a network space and to attach it back, is to allow
for changing the network space configuration. 

The link can move from one network space to another, providing a flexibility in deploying and utilizing
the InfiniBox networking resources. 

13.3.2 Link states

The replica link can be in either of the following states:

• Connected - all of the IP addresses are available for the link 

• The replication services of both source and target systems are communicating 
• Degraded - at least one IP address is not available (is down)
• Disconnected - none of the IP addresses is available 
• The replication services are not available
• A detached link is in a Disconnected state on both source and target systems

13.3.3 IP addresses redistribution

When IP addresses are removed and added to a network space that serves a Replication link, the
removed IP address causes a shortage in either the Asynchronous or Synchronous IP addresses, and
the added IP address automatically fills this shortage.

Link  –  262

 InfiniBox 4.0.10 User Documentation

Changing the network space configuration

Changing the network space configuration (adding IP addresses, for example) is done in the following
way:

1. Detaching the link from the network space

• This step suspends all replicas 
2. Carrying out the changes to the network space. Available operations:
a. Adding IP addresses
b. Removing IP addresses
3. Attaching the link to a network space
• The link can be attached either to the same network space or to another network space
• At the end of this step, the link needs to be refreshed and the replicas need to be
resumed 

13.3.4 Prerequisites
• This task assumes that a link already exists.
• In order to be attached to a network space, the link has to be detached.

13.3.5 See also

• Creating a link
• Querying for a link

13.3.6 GUI instructions 

Detaching a link 
1. On the InfiniBox GUI, click the Replication icon on the let toolbar to open
the Replication screen.
2. Click the Links tab.
3. Right-click a link and select Detach from the pop-up menu.
4. Confirm the warning message.

Results:

• The link is detached from the network space

• The link state changes to Disconnected

Link  –  263

 InfiniBox 4.0.10 User Documentation

• Replicas that use this link are suspended locally

Attaching a link
Once the network space configuration has changed:

1. On the InfiniBox GUI, click the Replication icon on the let toolbar to open

the Replication screen.
2. Click the Links tab.
3. Right-click a link and select Attach from the pop-up menu.
4. Select a network space.
5. Click Attach.
Results:
a. The link is attached to the network space
b. The link state changes to connected
6. Click Refresh in order to refresh the link definitions with changes that were made to the local side
of the link.
7. Resume all replicas that were suspended upon link detachment.

13.4 Querying for a link

• About this task
• Prerequisites:
• GUI instructions
• InfiniShell instructions
• Querying for the network space and IP addresses that serve the link
• Link-related InfiniShell commands

13.4.1 About this task

The system links are viewable from both GUI and InfiniShell.

In the following example, a source InfiniBox system is connected to 4 target systems.

Link  –  264

 InfiniBox 4.0.10 User Documentation

13.4.2 Prerequisites:
This task assumes that the following entities are defined:

• Link

13.4.3 GUI instructions

1. On the InfiniBox GUI, click the Replication icon on the let toolbar to open
the Replication screen.
The links are visible on the screen. 

13.4.4 InfiniShell instructions

Run link.query.

Link  –  265

 InfiniBox 4.0.10 User Documentation

Querying for the network space and IP addresses that serve the link
See here: Viewing the network space details.

Link-related InfiniShell commands

This section describes commands that create and authenticate links between InfiniBox systems.

• link.create - This command creates a link between a network space on the local system and a
target system. Such a link is required for services that span across systems, for example, 
Asynchronous Remote Replication.
• link.authenticate - This command verifies that the user of the local system can operate the
remote system as well
• link.delete - This command deletes the link
• link.query - This command lists the links that are defined on the system along with their state
and the last time they were in use

The commands are displayed in length, including examples on the InfiniShell Reference Guide.

13.5 Deleting a link

• About this task
• Prerequisites
• GUI instructions
• InfiniShell instructions
• Link-related InfiniShell commands

13.5.1 About this task

This task deletes a link between two systems.

13.5.2 Prerequisites
This task assumes that the following entities are defined:

• Link

Link  –  266

 InfiniBox 4.0.10 User Documentation

13.5.3 GUI instructions

1. On the InfiniBox GUI, click the Replication icon on the let toolbar to open
the Replication screen.
2. Click the Links tab.
3. Click the actions menu and select Delete Link.
4. Approved the warning message.
The link is deleted.

13.5.4 InfiniShell instructions

Run link.delete providing the link name.

Link-related InfiniShell commands

This section describes commands that create and authenticate links between InfiniBox systems.

• link.create - This command creates a link between a network space on the local system and a
target system. Such a link is required for services that span across systems, for example, 
Asynchronous Remote Replication.
• link.authenticate - This command verifies that the user of the local system can operate the
remote system as well
• link.delete - This command deletes the link
• link.query - This command lists the links that are defined on the system along with their state
and the last time they were in use

The commands are displayed in length, including examples on the InfiniShell Reference Guide.

Link  –  267

 InfiniBox 4.0.10 User Documentation

14 Replication

14.1 Overview of replication

 • skip-sync until 4.0.10 GA

14.1.1 Replication Overview

InfiniBox features both synchronous (sync) and asynchronous (async) replication types.

Overview of asynchronous replication

The InfiniBox asynchronous (async) replication feature is a snapshot-based solution that allows the
users to protect their data by replicating it to a remote site without adding latency to the host I/Os.

The async replication allows overcoming large geographic distances by:

• Sending the I/O to the remote site after it was already acknowledged by the host
• Allowing the user to define the interval between the snapshots that are sent to the remote site
• Supporting a minimal 4 secs RPO (Recovery Point Objective) if the link quality requirements
between the sites are fulfilled.

Async replication topology diagram

With InfiniBox async replication, when the host sends a write request to a dataset on the source system,
the write is acknowledged to the host immediately and then replicated to the target system.

The following diagram shows the async replication data flow:

1. The host sends a write I/O request to the source InfiniBox

2. The source InfiniBox acknowledges the write I/O to the host
3. The source InfiniBox replicates the data to the target InfiniBox
4. The target InfiniBox acknowledges the replication to the source InfiniBox

Replication  –  268

 InfiniBox 4.0.10 User Documentation

Overview for Synchronous Replication

The InfiniBox synchronous replication feature allows the user to protect the data synchronously,
sending the I/O to the remote site before acknowledging the host. The synchronous replication will
have an impact on the latency of the write I/O to the host since the acknowledge to the host will be sent
only after the data was written in both sites. Read I/O will be served locally from the source system so
no latency will be added. The InfiniBox synchronous replication solution depends on the quality of the
link between the sites. If the link requirements are not fulfilled, the InfiniBox will move the replica to an
internal asynchronous mode until the link requirements are met.

Synchronous replication topology diagram

With InfiniBox synchronous replication, when the host sends a write request to a dataset on the source
system, the write is replicated to a dataset on the target system prior to any acknowledge to the host.
Only after the target system has acknowledged the write to the source system, the source system
acknowledges the write to the host.

The following diagram shows the synchronous replication data flow:

1. The host sends a write I/O request to the source InfiniBox

2. The source InfiniBox synchronously replicates the data to the target InfiniBox
3. The target InfiniBox acknowledges the replication to the source InfiniBox
4. The source InfiniBox acknowledges the write I/O to the host

Replication  –  269

 InfiniBox 4.0.10 User Documentation

Glossary

Source A volume, file system or a consistency group that is replicated.

dataset

Target A volume, file system or a consistency group on the target system that is the destination of the
dataset replicated data. A target dataset cannot accept host writes.

Replica An entity that matches a pair of source and target datasets and other settings essential for the
replication.

Network A set of definitions for a network address space, annotated using CIDR.
Space

Link A bi-directional connection definition between local and remote systems.

Sync job The process of creating a snapshot of the source and delivering it to the target.

Sync The amount of time between two planned sync jobs.

interval

RPO Recovery Point Objective. The amount of data, measured in time units, that is at risk and might be
lost, in the case of a disaster.

Staging The storage space that is used by snapshots that are in use for the replication process, both on
area the source and the target InfiniBox systems.

Replication  –  270

 InfiniBox 4.0.10 User Documentation

System Connectivity - Replication Links

Overview
In order to replicate data from one InfiniBox system to another, the user needs to connect the two
systems by defining Replication Network Spaces on each of the InfiniBox systems. On top of Network
Spaces, the user will create a bi-directional Replication Link that will define the connection between the
two Network Spaces. If the systems are using sync and async replication between them, the same link
will be used for both replication types, using different protocols.

Available deployment schemes

Each InfiniBox system can serve as a source, or a target, to several replicated datasets. The system can
be linked to several other systems, but each two systems can be linked together with only one link.

Access control for connecting between the systems

All of the user operations relevant for Replication Network Spaces and Replication Links require Admin
user role. The Admin permissions are required on both source and target systems. The source system
tries to log into the target system with the same credentials and if the access is denied, it asks the user
to provide credentials to the target system.

Access to the target system is required for all two-sided operations. It is not required for operations
that are carried out only on the source system.

Replicating data from the source system to the target system can be carried out by both Admin and
Pool Admin user roles.

• The Admin user can replicate any of the system's datasets that are available for replication.
• The Pool Admin can replicate only datasets from the relevant pool.

Defining a Replication Network Spaces

The Network Space groups ethernet interfaces from all three InfiniBox nodes to assure a reliable and
redundant replication service between InfiniBox systems.

A Network Space for Replication service can be defined as "Async Only" or "Async and Sync".

The Network Space definition requires the user to define minimum 4/7 IP's depending on the
replication type:

• Minimum 4 IPs for Async Only Network Space - The first IP will be used as the Management IP,
and the rest will be used as data IPs for async replication
• Minimum 7 IPs for Sync and Async - The first IP will be used as the Management IP, and the rest
will be used as data IPs for sync and async replication

Replication  –  271

 InfiniBox 4.0.10 User Documentation

The Management IP transfers the replica configuration and Management commands between the
source and the target. The Management IP does not send the data.

Firewall ports that need to be open:

• Management - TCP 80, TCP 443

• Data (both Sync and Async) - TCP 8067

Defining a Replication Link

The link is the entity that connects the local InfiniBox system to the remote system using predefined
Network Spaces on both systems.

For link creation, the user must have an admin permission on both local and remote systems.

The link is bi-directional. It can be created on any of the systems by identifying the second system via an
IP address.

The same link can be used for both target and source replicas between the 2 systems.

Link states 
The replica link can be in either of the following states:

• Connected - All of the IP addresses are reachable

• Degraded - Not all of the IP addresses are unreachable 
• Disconnected - None of the data IP addresses is available
• Unknown - The remote management could not be reached or local async service is not available

Attach/Detach links
In case the user wants to change IPs on a linked replication network space, the link will have to be
detached from the network space and re-attached after the network space is updated, or attach it to a
new network space.

• Detaching a link - When the link is detached, the relationship between the link and the network
space is disconnected. All of the replicas that use the link are automatically suspended.
• Attaching the link - When the link is attached, all of the replicas that were suspended as a result
of the detach operation will be automatically resumed.
• Replicas that were suspended prior to the detach, will not be automatically resumed.

Replication  –  272

 InfiniBox 4.0.10 User Documentation

14.1.2 Infinibox Replica

Overview
A replica is an entity that matches a pair of source and target datasets and other settings essential
for the replication.

• A replica can be of type async (asynchronous) or sync (synchronous) replication.

• Both replication types share most of the commands and attributes described below.
• For each local replica entity, there is a paired replica entity on the remote system. Actions on the
replica can be local or cross-system, depending on the action and the connection state between
the systems.
• One side of the replica will be defined as the source, this is the side that holds the replicated
dataset. The linked side will be defined as the target and holds the dataset we replicated to.
• The replica will be created by the user on the source system, and the target system will
automatically create the linked replica on the target system.

Supported datasets for replication

InfiniBox version 4.0 supports the creation of async replica on:

• Volumes
• File Systems
• Consistency Groups

Infinibox version 4.0 supports the creation of sync replica on:

• Volumes

Replication  –  273

 InfiniBox 4.0.10 User Documentation

• Consistency Groups

General replica attributes

Managing a replica is mainly based on the following attributes:

• Replica type - Sync or Async

• Replica role - Source or Target
• Replicated dataset - volume, filesystem or consistency group on the local system
• Remote dataset - Volume, filesystem or consistency group on the remote system
• RPO (for Async only) - The amount of data, measured in time units, that is at risk and might be
lost, in the case of a disaster.
• Interval (for Async only) - The amount of time between two planned sync jobs.

Replica roles
The dataset that is replicated is called source and the dataset that receives the replicated data is called
target.
The replica role can be changed via user commands.

Every change in roles will cause the replica to reach Auto-Suspended state due to configuration
mismatch.

A change of the replica role will cause a change in the functionality of the changed replica:

• Change source to target -

• May cause a loss of updated data that was not replicated to the target yet.
• The source dataset will be changed to target dataset and will not accept host writes.
• Changing source to target can be done on suspended replicas only.
• Change target to source -
• The target dataset will be changed to source dataset and will accept user writes.
• The replica will block writes from the other system.

In Sync replication there is a command for "switch role" that switches the source and target roles
synchronously.

Replica states
InfiniBox uses replica states in order to manage a prompt functionality of the replication, the replica
state definitions are the same for sync and async replicas:

• Active - The replica is active and data is transferred from the source to target as needes

Replication  –  274

 InfiniBox 4.0.10 User Documentation

• Suspended - The replica is suspended by the user. No data is transferred from the source to the
target. The manual suspension of the replica pauses the current sync job.
• Auto-suspended - The replica is automatically suspended by InfiniBox due to a permanent error
or a timeout. The user will have to manually resume the replica once the cause of the suspension
is fixed.

Initial Sync
When the replica is defined, and the source is not yet replicated to the target, the replica is in an
Initialization state. In this state, all of the source data has to be replicated.

This state may take a long time depending on the amount of data that needs to be replicated.

Once the replica is synchronized (the source is fully replicated to the target), the InfiniBox system will
replicate only the new I/Os or differences between the source dataset and the target.

When the replica is in an Initialization state:

• The replica role cannot be changed

• It is impossible to take a snapshot on the target dataset (as the target is not yet consistent with
the source)

Snapshot-based Initial Sync (For Async replicas only)

For async replicas, the user can retain the internal snapshots when the replica is deleted and see them
as regular snapshots.

These retained snapshots are uniquely identified by the system and can be used as base for re-creating
the replica between the source and target datasets, as long as they were not changed by the user.

If the user creates the replica using the retained snapshots as base, the initial sync job is skipped, a new
snapshot of the source dataset is taken and only the difference between the retained snapshot and the
new snapshot is transferred to the target.

Operations on replicated datasets

Most of the user operations on a dataset will work as they normally do when the dataset is replicated.

The following user operations that are usually available on datasets are blocked on replicated datasets:

• Delete dataset
• Restore dataset
• Resize dataset
• Disable Write-protect dataset on a target dataset
• Export a filesystem on a target dataset

Once the replica object will be deleted, the operations on the dataset will be allowed as usual.

Replication  –  275

 InfiniBox 4.0.10 User Documentation

Replica Operations

Replica Create
Creating the replica will be done on the source system since this is the system holding the dataset data
and connected to the application active host.

When creating a replica, the user provides the following:

• The replication type

• The source dataset - the volume, filesystem or consistency group that will be replicated
• The target system
• As a prerequisite, the target system has to be connected to the source system via a link
• The link has to be defined prior to the replica creation
• The link has to be in a Connected state
• The target pool or target dataset:
• If the target pool is provided - the dataset will be created as a part of the replica creation in
the specified pool
• If the target dataset is provided (for volumes/CG only), it has to be created in advance,
empty and the same size as the source dataset (In case of a consistency group, please see
CG section below)
• Interval and RPO have to be supplied during the creation of an Async replica only

Replica Delete
A replica entity should be deleted from the source replica if possible. The system will automatically
delete the target replica as well.

When the replica is deleted, the pairing between the replicated datasets is deleted and both source and
target datasets return to regular usage. Deleting the replica will not delete the replicated datasets on
either side.

In case there is no connectivity between the systems or if there is a configuration mismatch between
the replicas, there is an option to delete the replica locally using a force flag. This will require the user to
do the cleanup on the other system.

When deleting an async replica, there is an option to retain the staging area (the last snapshot
replicated) and expose it to the user for future use (see snapshot-base init above). If the replica is in
Initializing state, there are no snapshots to retain.

Replica Suspend and Resume

The user can suspend and resume the replica at any time from the source replica only.

Replication  –  276

 InfiniBox 4.0.10 User Documentation

Suspend Replica command will cause the source replica to stop transferring data to the target replica,
and Resume Replica will resume the data transfer between the datasets if possible.

When there is no connectivity between the source and target or in case of a configuration mismatch
between them, the resume command will fail.

Replica Change Role

The user can change the replica role at any time except when the replica is not yet initialized.

Change role can be done on the source or the target replica, the source replica has to be suspended
prior to changing the role.

When changing source to target, the source dataset will be changed to target dataset and will not
accept user writes. This may cause a loss of updated source data that was not replicated to the target
yet.

When changing target to source the target dataset will be changed to source dataset and will accept
host writes. I/O from the another system will be blocked.

After a change role commands, the replica will have to be manually resumed in order to continue
replicating.

Replica Switch Role (For Sync replicas only)

In Sync Replication there is an option to switch the replica direction by synchronously change both sides
roles.

The switch role command can be used only from the source replica and only if the link between the
systems is connected and the replica is in synchronized state.

Consistency Group Replica

Consistency Group is an entity that groups several volumes together and allows the user to take a
consistent snapshot of these volumes.

A replicated CG (consistency group) keeps the volumes in the CG on the target side consistent with each
other.

The consistency group is replicated as a whole.

Replication  –  277

 InfiniBox 4.0.10 User Documentation

Create a CG Replica
Creating a replicated CG is done just like a volume or filesystem replica create.

In CG replica there are several options for the local dataset at the time of creation:

• An empty CG - The replica entity will be created between the 2 empty CGs and once the user will
add volumes to the CG they will automatically be replicated.
• A CG with volumes - All the volumes in the source CG will be paired with target volumes and start
a replication process.
• The members on the target side can be created automatically or previously created by the
user and paired specifically.

Add a member to CG Replica

Adding a member to a replicated CG can be done on the source replica only.

The added member can be a volume or an async replica. In both cases, the new member will get the
async replica definitions from the CG (RPO and interval).

Adding a member to a replicated CG will change the CG replica sync state to initializing until all the
volumes are replicated and the targets are consistent.

Remove a member from a CG Replica

Removing a member to a replicated CG can be done on the source replica only.

In order to remove a member, the replica link must be connected and the replica has to be from async
type. To remove a member to a sync CG, the replica type has to be changed to async beforehand.

When removing a member, the user can choose to keep the member replicated, and a new replica
entity will be created for the removed member.

The user can also choose to retain the staging area for the removed member, as done in delete replica.

Delete a CG replica
Deleting a CG replica is similar to the deleting a volume.

CG replica delete should be done from the source replica if possible and will delete the replica entity on
both sided.

Deleting the CG replica will not delete the CG on either side or change it in any way.

In case there is no connectivity between the systems or if there is a configuration mismatch between
the replicas, there is an option to delete the replica locally using a force flag. This will require the user to
do the cleanup on the other system.

When deleting an async CG replica, there is an option to retain the staging area, this will expose a snap-
group of all the last replicated snapshots of the CG.

Replication  –  278

 InfiniBox 4.0.10 User Documentation

14.1.3 Async replication Specifics

Async replication mechanism

The InfiniBox Async replication feature is snapshot-based replication, based on Sync jobs that are
scheduled automatically by the system.

The sync job creates a snapshot on the source dataset and delivers it to the target. The next sync job
takes a new snapshot of the source, calculates the diff from the previous snapshot and sends only the
data that was changed since the previous sync job. 

The amount of time between two scheduled sync jobs is called sync interval. The sync interval can be
changed by the user. Changes to the replica sync interval take effect on the next sync job. 

The async snapshots are internal snapshots and are not visible to the user. The capacity will be
presented to the user in the replica information as staging area capacity.

Sync Now Command

There is an option for the user to define a None interval for a replica. In this case, the system will not
initiate any sync jobs for the replica and the user will have to manually trigger a sync job using a "Sync
now" command.

The user can use the "Sync now" operation on any async replica, regardless of the interval defined, and
a sync job will be initiated (if there is no sync job currently replicating).

Sync Job states

When the replica is in Active state, it initiates a Sync Job as needed. InfiniBox manages the Sync Job
through the following states:

• Pending - This sync job is planned but not yet executed

• Initializing - This is an initial sync job that replicates all of the source data to the target
• Replicating - The sync job is now running
• Done - The sync job has finished
• Paused - The sync job is paused because the replica is suspended by the user
• Stalled - The sync job is stalled due to link problems

RPO State
In addition to the replica state, Async replicas there isalsoaRPOstate.

The RPO state can be:

• RPO OK - The replica recovery point is within the RPO defined

Replication  –  279

 InfiniBox 4.0.10 User Documentation

• RPO Lagging - The replica passed the RPO defined and potential data loss in case of a disaster
might be larger than planned. This state might be reached when there are connectivity issues
preventing a proper data flow or an incorrect RPO definition.

Possible Async replica states matrix

Replica States Sync Job States RPO states

Active Pending RPO OK / RPO Lagging

Initializing

Replicating

Done

Stalled

Suspended Paused

Done

Auto Suspend Paused

Done

Best practices for setting the Sync interval and RPO

Since the recovery point is based on the sync interval defined, the best practice is to set the RPO to be
at least twice as large as the sync interval.

14.1.4 Sync Replication Specifics

Sync replication mechanism

InfiniBox synchronous replication uses the same network entities as asynchronous replication to
achieve zero RPO.

Replication  –  280

 InfiniBox 4.0.10 User Documentation

In synchronous replication, as each host write is replicated to the target system prior to acknowledging
the host, the replica depends on the link quality. InfiniBox takes measures to handle the synchronous
replica in case that the link between the source and the target cannot support the replica, including
safely return to synchronous replication when the connectivity conditions are back to normal.

Internal fail-over to async replication

In case of a problem that prevents the replica from being synchronized, the replica fails back to an
internal async replication mode. The replica operates as if it is async until a Synchronization state is
reached. The replica state will be Out Of Sync, or Sync In Progress until InfinBox will return the replica
to a Synchronized state.

The replication type will stay sync at all time and the user will not be able to perform async replication
operations, such as sync now, nor configure the replica RPO and Interval.

The Failback to sync replication does not require an Initialization and will be done automatically by the
system.

Replica Sync states

In addition to the replica states that are shared for all replication types (mentioned above), sync replicas
have a sync state similar to the sync job states in async:

Possible states in sync state:

• Synchronized - The source and target datasets data is identical

• Sync in progress - The replica is in the process of returning to synchronized state (using the async
internal mode)
• Initializing - The replica initial process is running, copying the entire source data to the target
• Initializing pending - Initialization process will start once other initialization processes will end
• Out of sync - The replica source cannot send data to the target and the replication process is
paused.

The state of the rpelica is visible only on the Source. On the Target, the state is N/A. To view the replica
state, query for it on the Source system.

Possible Sync replica states matrix

Replica States Sync States

Active Synchronized

Initializing

Replication  –  281

 InfiniBox 4.0.10 User Documentation

Replica States Sync States

Sync in progress

Out of sync

Suspended Out of sync

Auto Suspend Out of sync

14.1.5  Changing the replication type

The replication type of a replica is determined by the user when the replica is created and can be
changed anytime later on (except when the replica undergoes Initialization).

• When the replication type is changed from Synchronous to Asynchronous the user has to

specify the Interval and RPO
• When the replication type is changed from Asynchronous to Synchronous:
• The Sync state is set to Sync In Progress until the replica is Synchronized

14.1.6 Disaster Recovery scenarios with replication

Disaster Recovery using target snapshots

Since the target dataset is always write-protected, for DR scenarios, InfiniBox supports taking a
snapshot of the target.

In async replicas, the snapshot that is taken on the target is always consistent with the last replication
cycle.

This ability to take a snapshot on the target is ideal for Disaster Recovery tests that aim to verify the
integrity of the data on the target without affecting the replication process.

It allows the user to map the snapshot to the remote host without stopping the replication of the
source dataset to the target.

Note that it is also possible to map the target dataset itself to a remote host, taking into account that
the dataset will be consistent and write-enabled only after the replica was changed to source or
deleted.

Replication  –  282

 InfiniBox 4.0.10 User Documentation

Testing the Disaster Recovery site (Firedrill)

Failover and failback are operations that handle a situation in which the connectivity between the local
and the remote systems is down. These operations switch the roles of the source and the target.

Failover

• The link between the source and the target is down and the target is connected to a host and can
serve it.
• The target has to have its role changed to a source and will now accept host writes.
• The replica on the original source side gets into an auto-suspended state.
• During this phase, the target and the source are no longer consistent.

Failback

The user returns both source and target to their original roles, and the replica returns to the state it was
prior to the failover.

• The original target should be changed back to target (was changed to source in the failover).
• The replica should be resumed from the original source side.
• A sync job will start in order to return the replica to synchronize state.

Switching the replica in case of a real disaster

In some disaster scenarios, the workload will have to be moved to the target system and the
applications will continue working on the target system.

In this case, the user might want to replicate the data changed on the target datasets back to the source
datasets.

To do so, the user will need to do the following:

Failover

• The source system is down and the target is connected to a host and can serve it.
• The target has to have its role changed to a source and will now accept host writes.
• During this phase, the source datasets are unavailable to the user due to the disaster.

Failback

The user switches the original roles of the replica and synchronizes the data from the new source to the
old source.

• The original source should be changed to target.

• The replica should be resumed from the new source side.
• A sync job will start in order to return the replica to synchronize state.

Replication  –  283

 InfiniBox 4.0.10 User Documentation

After this procedure, the user can decide whether to keep the replicas roles reversed or change them
back to the original sites (using switch role for sync if possible, or change role on both sides when there
is no new data on the source, to prevent data loss in the process)

14.2 Remote replication prerequisites

• Required entities
• Required ports
• Creating ports
• Create a network interface
• Create a network space
• Creating a link
• Creating a replica

14.2.1 Required entities

The replication uses the following entities, defined one on top of the other:

• Ethernet ports - every InfiniBox system includes 3 controller nodes. Every node has access to all
drives. Depending on the available InfiniBox model, one may have either 2 (F61xx/F21xx) or
4 (F62xx/F22xx) 10Gbps Ethernet ports per node.
• Network space - different InfiniBox services require a network space of their own. 
• The same network space serves both Asynchronous and Synchronous replicas
• Separate network spaces are required for Replication and for NAS.
• Service - an InfiniBox feature that requires network connectivity. For example: Replication, NAS.

Required ports
Make sure you have accessibility (admin privileges) to both local and remote systems.

The first IP will be used to Management the next 3 IP addresses will be used for Async replication and
the last 3 IP addresses will be used for Sync replication.

On each of the systems (source and target):

Functionality Connected ports IP addresses Firewall ports

Management 1 1 80,443

Replication  –  284

 InfiniBox 4.0.10 User Documentation

Data 3 or 6 3 or 6 8067

• For Asynchronous replication - 3

• For Synchronous replication - 3

Creating ports
Create a port on each of the Nodes:

On Node 1: 

config.ethernet.interface.create_single_port port=N1ETH1

On Node 2: 

config.ethernet.interface.create_single_port port=N2ETH1

On Node 3: 

config.ethernet.interface.create_single_port port=N3ETH1

Create a network interface

• See detailed instructions here: Creating a network interface

Create a network space

• See detailed instructions here: Creating a network space

Creating a link
• The link verifies that there are enough IP addresses on both systems (1 for Management, 3 for
Synchronous replication, 3 for Asynchronous replication, as explained above)
• For a description of Link States, see: Detaching and attaching a link
• See detailed instructions here: Creating a link

Replication  –  285

 InfiniBox 4.0.10 User Documentation

14.2.2 Creating a replica

• For detailed instructions and volume prerequisites, see here: 
Creating a replica

14.3 Creating a replica

• About this task
• Replicating a consistency group
• Naming restrictions
• Renaming the target 
• Prerequisites
• Before you begin
• GUI Instructions
• InfiniShell instructions

14.3.1 About this task

Remote replication is the process of copying a dataset from a local InfiniBox to a remote InfiniBox for
the purpose of data protection or disaster recovery. The replication ensures uninterrupted business
operation in case of a site failure. The INFINIDAT Remote Replication technology can asynchronously or
synchronously replicate data from a source InfiniBox to a target InfiniBox.

This task replicates a volume, a consistency group or a filesystem, from a source InfiniBox to a target
InfiniBox.

Replicating a consistency group

InfiniBox allows replicating consistency group members while maintaining a consistent replica on the
target system.

Unable to render include or excerpt-include. Could not retrieve page.

Naming restrictions
Two replicated entities (volume, consistency group or a filesystem) cannot have the same name. 

Renaming the target 

Renaming the target entity (either a volume, consistency group or filesystem) is reflected on the source
side of the replica. In order to allow the renaming operation to take effect, the link has to be up. 

Replication  –  286

 InfiniBox 4.0.10 User Documentation

• In case that the link is not up (either degraded or down), the new name will not be reflected on
the source side, even when the link is up again. In any case that the link is not up, it is advised to
wait until it is up and only then rename the target.

14.3.2 Prerequisites
Creating a replica from a source InfiniBox to a target InfiniBox requires a link between the two systems.
See the following help topics for instructions on how to fulfill the prerequisites:

• Remote replication prerequisites

• Creating a link
• Creating a network space
• Target entity - can be either:
• Created as part of the replica creation
•  Previously existing, and an empty dataset

14.3.3 Before you begin

Prepare the following information prior to creating the replica:

• The source system

• The source entity (volume, consistency group or filesystem)
• The target system (Link)
• The target pool - if the target entity does not exist it will be created by the system
• Target entity name - a name for the target dataset (optional)
• Target suffixes - suffixes for names of the target consistency group's volumes (optional)
• Target entity - if the target entity exists
• Replica attributes
• Sync interval - the interval between two consecutive sync jobs
• RPO (Recovery Point Objective) - the lag between the source and target sites

14.3.4 GUI Instructions

1. On the InfiniBox GUI, click the Replication icon on the left toolbar to open the Replication
screen.
2. Click Create. The Create Replica screen opens.
3. Fill in the following information:
a. Replica type - select between Volume and Consistency Group.

Replication  –  287

 InfiniBox 4.0.10 User Documentation

b. Source - Select the source dataset

c. Remote system - select a remote system from a list of linked system.
d. Remote target - select whether to create a new target dataset or to replicate into an
already existing dataset
e. Remote pool - select the remote pool
f. Interval and RPO - see a broader discussion with recommended values here: How to set
the replica RPO and interval
4. Examine the graphical display of the replica definition on the right pane on the screen. Change
the replica definition as needed.
5. Click Create. The replica is created and is visible on the Replication screen. 
6. The replica automatically initializes and its replication progress is visible on the screen.

14.3.5 InfiniShell instructions

Look up the following commands on the InfiniShell Command Reference:

• replica.create
• replica.advanced.create - for replicating volume a from the source and target snapshots
• replica.advanced.create_cg - for replicating a consistency group from the source and volume
snapshot groups

14.4 Failover and failback

• Failover
• Failback

Failover and failback are operations that handle a situation in which the link between the local and the
remote systems is down. These operations switch the roles of the source and the target. The Replica
Switch Roles starts with a source and a target that are both live and linked between them. 

14.4.1 Failover
On a Failover scenario, the link between the source and the target is down and the target is connected
to a host can serve it. To do so, the target has to have its role changed to a source. As the target serves
the host, the replica gets into an Auto Suspended state. During this phase, the target and the source are
no longer consistent.

In this phase, we change the role of the target to source. The volume becomes write-enable, as it will
now accept host writes.

Replication  –  288

 InfiniBox 4.0.10 User Documentation

14.4.2 Failback
Once the link is back, there is a need to synchronize the target – that has accepted host writes during
the Failover – and the source. To do so, the target remains a source, and its volume is replicated to the
other side of the replica, to the old source. To allow the replication in this direction, we change the role
of the source to target. When changing source to target, the volume becomes write-disable and the
replica will be suspended.

The user resumes the replication and the target and source become synchronized.

Replication  –  289

 InfiniBox 4.0.10 User Documentation

Now, we return both source and target to their original roles, and the replica returns to the state it was
prior to the failover.

14.5 How to set the replica RPO and interval

• About this task

Replication  –  290

 InfiniBox 4.0.10 User Documentation

• Recommended RPO and Interval values

• Default values
• Recommended values
• Instructions for modifying the Interval and RPO from the GUI
• Creating a replica with no RPO
• Creating a replica with no Interval

14.5.1 About this task

The RPO (Recovery Point Objective) is the maximum lag time between the source and target sites of the
replica. This lag is measured as the time that has passed from the last time the target was consistent
with the source. This time difference is, in other words, the time that has passed since the last sync job
was successfully completed.The RPO has the following possible states:

• RPO OK - the time that has passed since the last sync job was successfully completed is smaller
than the RPO.
• RPO lagging - the time that has passed since the last sync job was successfully completed is
greater than the RPO.
• N/A - no RPO was set for the replica. See more on this below.

14.5.2 Recommended RPO and Interval values

Default values
• When creating a replica from the GUI, the default values of the Interval and RPO are:
• Interval - 60 seconds
• RPO - 5 minutes
• When creating a replica from InfiniShell of InfiniAPI, there are no default Interval and RPO values

Recommended values
• It is recommended to set the values of Interval and RPO so the RPO value is at least twice the
value of the Interval
• The default values described above meet this recommendation

Replication  –  291

 InfiniBox 4.0.10 User Documentation

14.5.3 Instructions for modifying the Interval and RPO from the GUI
1. On the InfiniBox GUI, right-click the Replication icon on the let toolbar to open
the Replication screen.
2. Right-click a replica and select Modify Replica from the pop-up menu.
The Modify Replica screen opens. 
3. Modify the RPO and Interval and click Modify. 

14.5.4 Creating a replica with no RPO

When a replica is created with no RPO (InfiniShell syntax: rpo=NONE), the InfiniBox system does not
manage the RPO states, nor create events for the transition from RPO OK to RPO lagging and back.

14.5.5 Creating a replica with no Interval

When a replica is created with no Interval, a sync job can be created only manually (see: Sync now).

14.6 Mapping the target to the host

• About this task
• Related topics

14.6.1 About this task

In a case of a disaster, one of the failover steps is to map the target dataset to the host, so the dataset
will be able to serve IO. It is possible to prepare the remote site beforehand, so in the case of a failover
the target dataset will already be mapped to the host. Throughout the normal operation of the replica,
the target dataset is connected to the host in a read-only mode. It is serving host IO only upon a
failover.

Replication  –  292

 InfiniBox 4.0.10 User Documentation

14.6.2 Related topics

• Creating a replica
• Failover and failback

14.7 Replication glossary

Source The InfiniBox system whose volume is replicated.

system

Target The InfiniBox system that the volume is replicated to.

system

Source A volume on the source system that is replicated (or planned to be replicated).
dataset

Replication  –  293

 InfiniBox 4.0.10 User Documentation

Target A volume on the target system that is the destination of the replicated data. A target volume
dataset that is part of a replica cannot accept host writes.

Replica An entity that matches a pair of source and target volumes and other settings essential for the
replication.

Local The InfiniBox system through which the user manages the replica.

Remote The InfiniBox system at the other end of the replica.

Both local and remote system may contain source and target datasets.

Link A bi-directional means of communication between local and remote systems.

The link is defined via Management.

Sync job The process of creating a snapshot of the source and delivering it to the target.

Sync The amount of time between two planned sync jobs.

interval

Staging area The storage space that is reserved to, or used by, snapshots that are in use for the replication
process, both on the source and the target InfiniBox systems.

14.8 Replication limits and limitations

14.8.1 Replicated entity limitations

Asynchronous replication supports replicating:

• volumes
• consistency groups
• filesystems

Synchronous replication supports replicating:

• volumes

Replication  –  294

 InfiniBox 4.0.10 User Documentation

14.8.2 Replicated entities limitations

• Only root volumes/filesystems can be replicated
• An entity can be replicated only to one target
• A replicated entity cannot be resized, restored or deleted
• A target entity cannot be write-enabled
• No more than 6-15 replicas can be initialized at the same time (depending on the Infinibox
model)
• The maximum number of replicas in InfiniBox is (the lower between the two):
• 250 replica definitions (Consistency groups, volumes, and filesystems)
• 2048 datasets (volumes or filesystems)

14.8.3 Network space limitations

• IP addresses cannot be added or removed from a replication network space when a link is
attached

14.9 Sync now

• About this task
• Prerequisites
• GUI Instructions
• InfiniShell instructions
• Related tasks

14.9.1 About this task

InfiniBox allows for an extra-curricular synchronization of a replica. Such synchronization is usually in
use in order to achieve application-consistent replica. The sync now task synchronizes a replicated
dataset (volume, or a consistency group) on demand, regardless of the scheduled replica cycle. The
value of the replica interval is immaterial for the sync job. Specifically, the replica interval does not have
to be changed to NONE in order to issue the sync now command.

The sync now command creates a new snapshot on the source and initiates a sync job. Future, interval-
initiated, sync jobs are not canceled. Multiple sync now commands - one per replica - can be run on the
same source system concurrently.

Replication  –  295

 InfiniBox 4.0.10 User Documentation

14.9.2 Prerequisites
This command requires an up and running replica definition, that is:

• Past the Initialization state

• Not in Suspended state

The command fails in case there is a sync job that is currently running.

14.9.3 GUI Instructions

1. On the InfiniBox GUI, click the Replication icon on the let toolbar to open the Replication
screen.
2. Right-click a replica and select Synchronize Now from the pop-up menu.

A sync job is initiated. 

14.9.4 InfiniShell instructions

Look up the following command on the InfiniShell Command Reference:

• replica.sync_now

14.9.5 Related tasks

See all related Replication help topics here.

14.10 Tracking the replica on InfiniShell

Replication  –  296

 InfiniBox 4.0.10 User Documentation

• About this task

• Prerequisites
• Instructions
• Querying for the replica from InfiniShell
• Available fields
• Tracking the replication from the GUI
• Related tasks

14.10.1 About this task

This task tracks the progress of the replication.

14.10.2 Prerequisites
A replica definition.

14.10.3 Instructions
Query for the replicating volumes on both source and target in order to see the progress of the
replication. See the following examples for the way this query is used, and the data that can be queried
for.

Querying for the replica from InfiniShell

The replica.query command displays the status of the replication. In the following example, the sync
job has ended, so the State of the replication is IDLE and the Progress of the replication is 100%.

LOCAL DATASET TYPE ROLE REMOTE SYSTEM REMOTE DATASET STATE PROGRESS THROUGHPUT RPO STATE
RESTORE POINT
cg-1 CG SOURCE ibox3000 ibox1140_cg-1 IDLE 100% - OK
2015-09-06 11:05:04
cg-2 CG SOURCE ibox3000 ibox1140_cg-2 REPLICATING 1% - LAGGING
2015-09-06 07:24:34

 
Whenever the replication was queries amidst the sync job, the State is INITIAL_REPLICATION and the
Progress is a number between 0% and 100%.

(Some fields were removed from this example, in order to allow for a clear display on this document.)

LOCAL DATASET TYPE ROLE REMOTE SYSTEM REMOTE DATASET STATE PROGRESS THROUGHPUT
RPO STATE RESTORE POINT

Replication  –  297

 InfiniBox 4.0.10 User Documentation

cg-3 CG SOURCE ibox3000 ibox1140_cg-1 INITIAL_REPLICATION 10% -

- -

Available fields
You can set the query to display the following fields (some of them are not included in the default
output that is shown above).

To select which fields to display, run replica.query with the columns operator. Available columns are:
 

local_pool_name    The name of the pool that the local dataset belongs to.

remote_system      The name of the remote system.

state_description              A plain English description of the state of the replication.

           

updated_at The last time the replication was updated. Usually, this is the timestamp of the last
sync job.

created_at The time the replica was created.

progress The completion of a currently running sync job, in percents.

restore_point The time of the last replicated snapshot on the source, that is known for sure to
be replicated to the target.

This is valid only on the source.

state_reason A sub code of the replica state.

last_synchronized  The last time data was replicated from the source to the target.

remote_dataset The name of the dataset on the remote system.

role The role of the local dataset. Either Source or Target.

sync_duration The duration of the last sync job.

Replication  –  298

 InfiniBox 4.0.10 User Documentation

local_dataset The name of the dataset on the local system.

remote_pool_name The name of the pool that the remote dataset belongs to.

state The state of the replication. Available values:

• Unknown
• Idle
• Replicating

sync_interval The preset interval between sync jobs.

staging_area_size The capacity that is dedicated for replica snapshots. This field is available only on
the target system.

For more information on how to select which columns to display, see: Controlling the columns of the
displayed query output.

Tracking the replication from the GUI

The Replication screen displays the state of each replica. 

14.10.4 Related tasks

See all related Replication help topics here.

14.11 Tracking the replica on the GUI

• About this task
• Prerequisites
• Instructions
• Viewing the replica details 

Replication  –  299

 InfiniBox 4.0.10 User Documentation

14.11.1 About this task

This task tracks the progress of the replication.

14.11.2 Prerequisites
A replica definition.

14.11.3 Instructions
1. On the GUI, click the Replication icon on the menu on the left. The Replication screen opens.
(This screen has two tabs, the Replicas is the default tab.)
2. Click a replica to view its details on the expanded view.

14.11.4 Viewing the replica details 

The replica details pane displays the following information:

• The name of the local replicated entity

• The role of the local entity (either source or target)
• The name of the remote system
• The data and time the replica was created
• The RPO of the replica
• The Interval of the replica
• The restore point
• The size of the staging area 
• The replica state
• The names of the source and the target entities. For a replicated consistency group - a list of the
replicated datasets on both source and target

Replication  –  300

 InfiniBox 4.0.10 User Documentation

14.12 Deleting a replica

• About this task
• Prerequisites
• Before you begin

Replication  –  301

 InfiniBox 4.0.10 User Documentation

• GUI Instructions
• InfiniShell instructions
• Retaining the staging area on the target
• Best practice
• Related tasks

14.12.1 About this task

This task deletes a replica. The replica should be deleted from the source. However, it can be deleted
locally from the target using a force flag is needed. 

14.12.2 Prerequisites
• A replica

14.12.3 Before you begin

Deleting a replica from the source:

• The replica definition is deleted

• The link is not deleted
• The source dataset is not deleted
• The remote dataset is not deleted
• Optionally - the staging area can be retained

Deleting a replica from the target:

• The replica definition is deleted

• The link is not deleted
• The target dataset is not deleted
• The remote dataset is not deleted
• Optionally - the staging area can be retained

14.12.4 GUI Instructions

1. On the InfiniBox GUI, right-click the Replication icon on the let toolbar to open the Replication
screen.
2. Right-click a replica and select Delete Replica from the pop-up menu.

Replication  –  302

 InfiniBox 4.0.10 User Documentation

• An example of deletion from the source

 
• An example of deletion from the target 

3. Approve the warning message.

The replica is deleted. 

14.12.5 InfiniShell instructions

Use the replica.delete command.

Retaining the staging area on the target

Deleting the replica with retain_staging_area creates a snapshot or a snap-group of the source and
target entities. These snapshots are uniquely identified by the systems and can, later on, be used as a
base for re-creating a replica on these entities.

However, this operation may fail in the following cases:

• The replica is in Initialization phase, so the target is not consistent with the source

Replication  –  303

 InfiniBox 4.0.10 User Documentation

• A dataset was added to the consistency group and the consistency group replica hasn't reached
consistent state on the target yet

Best practice
It is advised to wait for the Initialization phase to complete prior to deleting the replica if you would like
to retain the staging area.

14.12.6 Related tasks

See all related Replication help topics here.

14.13 Restoring a thick dataset from its snapshot

• About this topic
• Applicability to replication change role
• Instructions for restoring a dataset fro its snapshpot
• Step 1: Creating a thick dataset and taking a snapshot
• Step 2: Accepting writes after the snapshot was created
• Step 3: Restoring the dataset from the snapshot

14.13.1 About this topic

When restoring a thick dataset from one of its snapshots, we need to consider the pool physical
capacity. The dataset accepts writes as long as the pool physical capacity is not depleted.

Applicability to replication change role

When the source and target of the replication change roles, the target dataset is restored from its
snapshot. The mechanism that is discussed below also applies to target dataset.

If there is not enough room for restoring the target, the replication change role operation will not take
place. 
In this case, you may either:

• Increase the target pool physical capacity

• Change the dataset provisioning type to thin

Replication  –  304

 InfiniBox 4.0.10 User Documentation

14.13.2 Instructions for restoring a dataset fro its snapshpot

Step 1: Creating a thick dataset and taking a snapshot

1. A thick dataset is created. 
2. The dataset accepts writes. 
3. A snapshot is taken. 

Step 2: Accepting writes after the snapshot was created

As the dataset keeps accepting writes, that snapshot consumes capacity. 

Step 3: Restoring the dataset from the snapshot

At the time of restore, for a brief moment, the snapshot is restored to a copy of the dataset, while the
original dataset is not deleted yet.

At this point, the dataset and snapshot both consume capacity. This capacity may exceed the pool
physical capacity.

In this case, you may either:

• Increase the pool physical capacity

• Change the dataset provisioning type to thin

These changes can be reverted after the successful completion of the restore operation.

Replication  –  305

 InfiniBox 4.0.10 User Documentation

15 Encryption at-rest

15.1 How does encryption-at-rest work

INFINIDAT utilizes disk encryption-at-rest for regulatory compliance, security audit readiness and other
purposes.

Disk encryption-at-rest allows for data protection across all scenarios in which data that is stored in the
disks is compromised due to disks removal from the site. With data encryption using AES256 and the
ability to securely erase a disk, the risk of data exposure is eliminated. 

The InfiniBox storage system can be set to run either with data-at-rest protection or without it. InfiniBox
encryption-at-rest uses the standard method of encrypting data, so there is no performance penalty.
To benefit from this feature, your InfiniBox has to be equipped with self-encrypting disks.

• Encryption-at-rest terminology
• Encryption-at-rest mechanism
• InfiniBox infrastructure
• Feature activation
• Hot upgrade

15.1.1 Encryption-at-rest terminology

• SED - Self-encrypting drive. A drive that encrypts all its data all the time using an internal
Encryption key. 
• Password - The string that unlocks a drive and allow IO. Often referred to as Authentication
Keys or AK. Passwords are stored in the InfiniBox OS and are sent to the drives when needed. All
passwords are saved in an encrypted form for security, and in 3 locations to avoid password
corruption.
• Encryption key - The binary key used to encrypt / decrypt the actual data. Often referred to
as Data Encryption Key or DEK. The encryption key is stored inside the drive and never leaves
it. InfiniBox uses 256 bit AES encrypted drives.

15.1.2 Encryption-at-rest mechanism

InfiniBox infrastructure
• InfiniBox generates unique passwords per-drive and per-system so that different drives will
always have different passwords.
This means that even in the theoretical case of a drive compromised, all other drives in the
system remain secure.

Encryption at-rest  –  306

 InfiniBox 4.0.10 User Documentation

• Drives are activated and authenticated upon any power loss.

Feature activation
• Activation and locking:
• The activation is performed by INFINIDAT Support
• InfiniBox creates passwords for all drives
• InfiniBox locks the drives
• InfiniBox unlocks the drives using the passwords (self-test)
• This task takes several minutes to complete and is done for one drive at a time
• Drive activation
• If the drive is already locked, InfiniBox replaces its password

Hot upgrade
• Hot upgrade is not affected by the encryption-at-rest feature

Encryption at-rest  –  307

 InfiniBox 4.0.10 User Documentation

16 InfiniSpares

16.1 How does InfiniSpares work

• General
• Upon install
• User data is written to the system
• Upon a disk failure
• As more and more disks fail
• As the unallocated capacity depletes

16.1.1 General
The InfiniSpares feature allows the InfiniBox system to keep working even when a preconfigured
number of disks were failed. InfiniBox handles the increased number of failed drives by using
unallocated capacity as spare capacity. Replacing the failed drives increases back the unallocated
Capacity.

InfiniSpares improves the system ability to sustain mass disk failures at the expense of usable capacity.

16.1.2 Upon install

Initially, the system allocates disks capacity according to the following:

• Reserved spares - up to 5% of the total capacity (this number varies across models, see below)
• The rest is divided between allocated and unallocated capacity, according to the actual usage

16.1.3 User data is written to the system

As new IO arrive at the system, the user capacity grows on the expense of the unallocated capacity. The
amount of Reserved Spares is not affected. 
In the example below (these numbers are only an example that illustrates the mechanism): 

• User capacity - 40% of the total capacity

• Unallocated capacity - 57.5% of the total capacity (in this example, on F6000)
• Reserved Spares: 2.5% of the total capacity

InfiniSpares  –  308

 InfiniBox 4.0.10 User Documentation

16.1.4 Upon a disk failure

When a disk fails, the system uses some of the Reserved Spares for the rebuild process. When the failed
disk is replaced, this capacity returns to the Reserved Spares.

16.1.5 As more and more disks fail

If the number of failed disks exceeds the number of Reserved Spares, some of the Unallocated Capacity
is used for rebuild process. When the failed disks are replaced, this space returns to the Unallocated
Capacity and then to the Reserved Spares. 

16.1.6 As the unallocated capacity depletes

When the system Unallocated Capacity is exhausted, the system performs the following steps,
regardless of the capacity distribution of User Capacity and Reserved Spares:

• The system no longer accepts writes.

• Upon the next drive failure, the system performs an emergency shutdown

InfiniSpares  –  309

 InfiniBox 4.0.10 User Documentation

16.2 What is InfiniSpares

The Infinispares feature enables the InfiniBox system to keep running even in a situation where dozens
of disks have failed across a long period of time. In such a situation, where all of the system spare disks
space is consumed, an additional disk failure will cause the system to activate the InfiniSpares feature. 

When the feature is engaged, InfiniBox allocates free system capacity as spare disks. At any given time,
InfiniBox ensures that the system has an enough spare capacity to hold 2 additional disks failures.

16.2.1 Preconfigured number of spare drives

The maximum number of InfiniSpares disks is set to:

InfiniBox model Max spare capacity Number of spare drives

F6xxx 100 12

F4xxx 100 8

F2xxx 50 6

F1230 20 6

16.2.2 Limitations 
• When engaged, InfiniSpares consumes system usable physical capacity to recover data on failed
disk drives,  this capacity will be reclaimed back as usable capacity when failed drives are
replaced 
• When engaged, InfiniSpares preserved ahead physical usable capacity equals to 2 disk drives,
this capacity will be reclaimed back as usable capacity when failed drives are replaced
• If more than the above number of disks have failed, the system performs an emergency
shutdown 

InfiniSpares  –  310

 InfiniBox 4.0.10 User Documentation

16.2.3 Events 
INFINISPARES_ENGAGED event with error severity is emitted on any disk failure when system has no
"native" spare disks left and the InfiniSpares feature has to allocate spare capacity for the rebuild. 

InfiniSpares  –  311

 InfiniBox 4.0.10 User Documentation

17 Event notification

17.1 Overview of event notification

InfiniBox monitors and issues events on its health, performance and capacity, as well as on the user
activity. The events are displayed on a central event log that is accessible via the InfiniBox GUI,
InfiniShell and API. The events can also be sent outside the InfiniBox for further troubleshooting.

• Events
• Event severity
• Event reporter
• Event retention
• Event behavior
• Custom events
• Heartbeat event
• Event notification
• Event notification rule

17.1.1 Events

Event severity
InfiniBox assigns each of the events it generates with a severity level. The severity level is the quickest
way to filter events that are displayed in the event log and when setting a notification rule. The severity
level is preconfigured into the event definition and cannot be changed. The available severity levels are:

• Info - informational - the system reports on an operation that does not have a negative
implication on the system operation. For example, a creation of an entity, user login.
• Warning - example, a change in the state of an hardware component, pool space low, etc.
• Error - a failure in a hardware component that requires user action, a user action that may limit
the system operation. For example, pool space very low. 
• Critical - a failure that requires an immediate user action

Event reporter
InfiniBox events are grouped by reporters, which are the InfiniBox component that generates them. 

Event notification  –  312

 InfiniBox 4.0.10 User Documentation

Event retention
• InfiniBox keeps 5M events
• Events are deleted in 100,000 increments 

Event behavior
• The event is exposed to the event log 10 seconds after it is generated

Custom events
• Available templates
• CUSTOME_INFO_EVENT
• CUSTOM_WARNING_EVET
• CUSTOM_ERROR_EVENT
• CUSTOM_CRITICAL_EVENT
• Maximum event size: 100kb

Heartbeat event
• Sent to INFINIDAT via the preconfigured SMTP server once a day
• The content of the event informs INFINIDAT support on the state of the INfiniBox system
• The event does not contain user information

17.1.2 Event notification

InfiniBox sends event notification to the following destination types:

• SMTP
• SNMP
• SysLog

You can send the notifications to up to 10 destinations.

Active polling for events (SNMPWALK, etc.) is not supported 

Event notification rule

The event notification rule includes the following information:

Event notification  –  313

 InfiniBox 4.0.10 User Documentation

• Rule name
• Destination
• Target - a destination to where the notification is sent
• Target parameters - for example, an email address for SMTP rule
• Filter
• Event level - all of the events of this level will be sent
• Additional events - events that belong to a level that is not marked by this rule, and will be
sent
• Excluded events - events that belong to a level that is marked by this rule, and will not be
sent

17.2 Creating an event rule

• About this task
• Before you begin
• SMTP server
• Predefined event rules
• Creating an event rule

17.2.1 About this task

Event rules control the way events are sent to the user.

Event notifications are sent according to the following rules:

• Event severity level

• Event code that is included in the rule
• Event code that is excluded from the rule
• Email, SNMP and Syslog targets

17.2.2 Before you begin

SMTP server
Out-of-the-box, InfiniBox has a predefined INFINIDAT SMTP server.

• To define your local SMTP server, go to the Settings > SMTP  screen.

Event notification  –  314

 InfiniBox 4.0.10 User Documentation

Predefined event rules

• InfiniBox sends heartbeat events to the INFINIDAT SMTP server
• To add more rules, see the instructions below

17.2.3 Creating an event rule

1. User either of the following:
a. Go to the Settings > SMTP  screen and click on View Event Rules.
b. Go to the Events screen and click on Event Rules. 
2. Click on the Create button.
The Create Event Rule screen opens.
3. Fill in: 
a. Name
b. Target
c. Recipients - email addresses for the event notifications
d. Event level
e. Additional events - insert individual events that this rule will notify on
f. Excluded events - insert individual events that this rule will not notify on
4. Click Create.
The event rule is created.

17.3 Notifying on events

• SMTP
• Creating an SMTP notification target
• Listing SMTP notification targets
• Testing an SMTP notification target
• SNMP
• Creating a new SNMP notification target
• An example for setting a notification target for SNMP versions 1 and 2c:
• Examples for setting a notification target for SNMP version 3
• The MIB file
• Syslog
• Creating a Syslog notification target

Event notification  –  315

 InfiniBox 4.0.10 User Documentation

17.3.1 SMTP
You can send the InfiniBox event notifications via Simple Mail Transfer Protocol (SMTP).

Creating an SMTP notification target

Set an SMTP target for InfiniBox notifications. An SMTP server address is required. Optionally, you can
define a new server by setting the host, port, user and password. You can also determine whether to
use the secured TLS protocol.

config.notifications.smtp.modify host=smtp port=25 tls=tls [email protected]

Listing SMTP notification targets

NAME HOST PORT TLS USERNAME FROM ADDRESS SUBJECT PREFIX

customer-smtp smtp.infinidat.com 25 yes - [email protected] -

Testing an SMTP notification target

This command sends a test notification to an email recipient.

SMTP notification target "customer-smtp" tested successfully

17.3.2 SNMP

Creating a new SNMP notification target

Define an SNMP trap to which the InfiniBox sends events. State the name and host and then select the
version of the SNMP protocol.

For SNMP versions 1 and 2c you need to define a community. For SNMP version 3 you need to define
an SNMP Engine ID and a user name.

You also need to select the user-based security model among authentication and privacy,
authentication without privacy and no authentication and no privacy. Optionally, you may set the
authentication type, and whether to encrypt the events and a private key.

Event notification  –  316

 InfiniBox 4.0.10 User Documentation

An example for setting a notification target for SNMP versions 1 and 2c:

config.notifications.snmp.define name=snmpv1-dest host=snmp version=1 community=infinidat

Examples for setting a notification target for SNMP version 3

Where the messages are unauthenticated and unencrypted:

config.notifications.snmp.define name=snmpv3-dest host=snmp version=3 engine_id=1 username=user1

security=NoAuthNoPriv

Where the messages are authenticated and not encrypted:

config.notifications.snmp.define name=snmpv3-dest host=snmp version=3 engine_id=1 username=user1

security=AuthPriv password=123456 auth_type=SHA priv_key=privatekey encryption=DES

Where the messages are authenticated and encrypted:

config.notifications.snmp.define name=snmpv3-authpriv host=snmp version=3 engine_id=1 username=user1

security=AuthPriv password=123456 auth_type=SHA priv_key=privatekey encryption=DES

The MIB file

• Click here to view the MIB file: INFINIDAT-INFINIBOX-MIB.txt

17.3.3 Syslog

Creating a Syslog notification target

Define a syslog notification target by providing a host and a port. You can override the default UDP
protocol with TCP. You can override the default Facility local7 with any other value between local0 and
local7. In the following example, a target is defined with the default protocol and facility:

config.notifications.syslog.define name=target0 host=syslog

17.4 Viewing events on InfiniShell

Event notification  –  317

 InfiniBox 4.0.10 User Documentation

• Viewing event codes

• Querying for events
• Viewing the event details
• Viewing event levels
• Viewing event reporters
• Watching events
• Creating an event

Event commands control the way InfiniBox events are displayed.

17.4.1 Viewing event codes

List all of the InfiniBox event codes.

Filter the list by code, level or reporter. State the code to list only the event codes without their details.
State the level and select among critical, error, warning and info. State the reporter and select among
block, custom, file, management, platform and REPLICATION.

• Run the command without parameters to get a full list of all of the event codes: event.codes
• Run the command specifying a code to see its details: eventcodes code=VOLUME_CREATED

CODE VISIBILITY LEVEL REPORTER DESCRIPTION

VOLUME_CREATED CUSTOMER INFO MGMT Created {prov_type} volume
'{volume_name}' in pool {pool_name} of size: {size_in}{size_format}

17.4.2 Querying for events

List the events that the InfiniBox emitted.

Filter the query by time stamp, event ID, code, level and reporter.

• From a specific point-in-time forward: event.query from='2015-06-10 10:00:00'

• Between two points in time: event.query from='2015-06-10 10:00:00' to='2015-06-10
20:00:00'

• Between two event IDs: event.query from_seq=201 to_seq=220

The events are listed on screen with the following information:

• Timestamp
• Level
• Code
• Description

You can run the event in detailed mode, adding the following fields:

Event notification  –  318

 InfiniBox 4.0.10 User Documentation

• Seq – the sequential number of the event

• Reporter
• User name

17.4.3 Viewing the event details

Display a detailed view of a single event: event.details id=100

Affected Entity Id 416

Code HOST_ADD_PORT
Description Added port '000000000000003c'
to 'host-04'
Seq 100
Level INFO
Reporter MGMT
Seq Num 85
Source Node Id 1
System Version 1.5.0.11
Timestamp 2014-06-09 14:36:57
Username admin
Data: Host Id 416
Data: Host Name host-04
Data: Initiator Address 000000000000003c

17.4.4 Viewing event levels

• Run this command to see the available event levels: event.levels

17.4.5 Viewing event reporters

• List InfiniBox areas that report events: event.reporters 

17.4.6 Watching events

• See a live display of events on the InfiniShell screen. Click Ctrl+C to get back to the command
prompt: event.watch

17.4.7 Creating an event

Create a new custom event. Set the level and text of the event you create. Then, run
the event.query command to see the event. 

• Example: event.create level=INFO description='custom event'

• Output:

Event notification  –  319

 InfiniBox 4.0.10 User Documentation

TIMESTAMP LEVEL CODE DESCRIPTION

2015-06-01 10:00:00 INFO CUSTOM_INFO_EVENT custom event

17.5 Viewing events on the GUI

Viewing events on the GUI
1. Click the Events icon on the InfiniBox GUI main screen.

 
2. The Events screen opens.

Event notification  –  320

 InfiniBox 4.0.10 User Documentation

18 GUI and InfiniShell software upgrade

18.1 Upgrading the GUI and InfiniShell software

• About this task
• Prerequisites
• GUI instructions
• InfiniShell instructions

18.1.1 About this task

The GUI and InfiniShell software packages are deployed both as part of the InfiniBox code and
independently through the web. Whenever the GUI, or InfiniShell is updated, it can be downloaded and
installed on the InfiniBox Management Console.

18.1.2 Prerequisites
Download the software update package from http://repo.infinidat.com/#infinishell

18.1.3 GUI instructions

1. From the GUI
a. Open the InfiniBox Management console.
Watch the GUI and CLI Update from the menu at the top right of the screen.

GUI and InfiniShell software upgrade  –  321

 InfiniBox 4.0.10 User Documentation

In case that there is an available update, the will be an indicator on the screen.

b. Click the GUI and CLI Update menu item.

The GUI and InfiniShell Update screen opens.
c. Browse to the package that you downloaded on step 1.
i. The screen displays all of the available packages.
ii. Click Update.
2. Following the upgrade you will be prompted to clear the browser's cache

18.1.4 InfiniShell instructions

1. From InfiniShell:
a. Run system.upgrade file=installationfile

GUI and InfiniShell software upgrade  –  322

 InfiniBox 4.0.10 User Documentation

19 Metadata

19.1 Setting and viewing InfiniBox metadata

• Setting a metadata object
• Viewing the metadata
• Clrearing and removing metadata

19.1.1 Setting a metadata object

You can set one key at a time. Setting a value to an already existing key overrides the previous string.

metadata.set objtype=vol object=vol-1 key=mappings value=keep-unmapped

The result:

Volume "vol-1" metadata key "mappings" set to "keep-unmapped"

19.1.2 Viewing the metadata

Get the metadata by object type and object:

metadata.get objtype=vol object=vol-1

The result:

KEY VALUE
mappings keep-unmapped

Clrearing and removing metadata

• metadata.clear - clears all of the metadata from an object
• metadata.remove - removes a single metadata key from an object

Metadata  –  323

 InfiniBox 4.0.10 User Documentation

20 Performance monitoring

20.1 Monitoring the system performance

• Viewing system performance on the GUI
• Viewing system performance using InfiniMetrics
• On measuring latency of SAN entities

InfiniBox provides visibility to key performance indicators. Each graph can monitor the system, a host, a
volume or a filesystem. The available indicators are: throughput, IOPS/FOPS, latency, read IOPS/FOPS,
write IOPS/FOPS and open requests (for SAN only).

Performance monitoring is done from the InfiniBox GUI, or through InfiniMetrics, an INFINIDAT VMware
plugin. 

20.1.1 Viewing system performance on the GUI

The GUI Performance screen monitors allows for adding, editing and removing monitors.

20.1.2 Viewing system performance using InfiniMetrics

INFINIDAT InfiniMetricsTM is a VMware plugin for collecting and displaying performance metrics from
one or more InfiniBoxes.

For InfiniMetrics documentation, refer to: InfiniMetrics Release Notes and User Guide.

Performance monitoring  –  324

 InfiniBox 4.0.10 User Documentation

20.1.3 On measuring latency of SAN entities

InfiniBox measures IO latency of SAN entities for the round-trip time (until the moment the host
acknowledges). Round-trip latency measures end-to-end traverse of I/O, including storage latency,
fabric latency and initiator acknowledgment.

20.2 Adding a chart

• About this task
• Prerequisites
• Before you begin
• Instructions for adding a chart
• Breaking-down the chart
• Filtering the chart

20.2.1 About this task

Add a chart that will gather performance counters for either of the following entities:

• All of the entities (either SAN or NAS on the same chart) in the InfiniBox system
• All of the entities (either SAN or NAS on the same chart) within a specific pool
• A specific host
• A specific filesystem or a volume

20.2.2 Prerequisites
• None

20.2.3 Before you begin

On the GUI, click the Performance icon on the toolbar on the left. 
The Performance screen opens. 

20.2.4 Instructions for adding a chart

1. Click Add Chart and select a Counter from the list. The Add Chart screen opens, displaying, as a
default,  information for SAN and NAS.
2. Add an entity from the list.
3. Select whether to display a line graph or a stacked graph. 

Performance monitoring  –  325

 InfiniBox 4.0.10 User Documentation

20.2.5 Breaking-down the chart

You may break-down any of the charts to preset components. For example, SAN Latency can be broken-
down by protocol, operation category, SCSI status, IO of Replication and operation size. 

The break-down option is available from the Action menu to the right of the Counter, on the chart
legend.

20.2.6 Filtering the chart

You may further filter the chart by parameters that are not available on the Break Down pop-up menu.

The filter option is available from the Action menu to the right of the Counter, on the chart legend.

20.3 Performance monitoring limits and limitations

Monitoring the InfiniBox performance is subject to the following limits: 

• The maximum accepted sampling interval is:

• SAN entities: 5 seconds
• NAS entities: 1 seconds
• The maximum number of samples stored on the server (if not collected by the GUI): 128
• When collecting data, the maximum number of collectors (per type) in a single collect request: 
• 40 collectors
• 2 TOP collectors
• 10 Histogram collectors
• 2 aggregator collectors

20.4 Creating and monitoring Live Counters using InfiniAPI (4.0)

• How to collect performance data
• Creating a filter 
• Refining the filter
• Creating a COUNTER collector
• Collecting the metrics
• Creating a TOP collector
• Collecting the metrics
• Creating a HISTOGRAM collector
• Collecting the metrics

Performance monitoring  –  326

 InfiniBox 4.0.10 User Documentation

• Creating an AGGREGATOR collector

• Setting the metrics collection interval
• Collecting the metrics
• Field Reference
• NFS
• Collector fields
• FC
• iSCSI
• RMR
• SAN
• SAN_QOS

20.4.1 How to collect performance data

The Live Counters Metrics resource reports on InfiniBox performance counters. This is the same API
that is used by the InfiniBox GUI to display performance charts. It is recommended to explore the
Performance section in the InfiniBox GUI in order to see what can be achieved using this API.

Each performance counter is a combination of a filter and a collector. The filter is used to slice-and-
dice the stream of I/O operations passing through the system, in order to define the subset of
operations that should be counted. For example, a filter that is limited to the SAN protocol type
matches all SAN I/O operations. A filter that is limited to the SAN protocol type and to a specific pool
matches only SAN I/O operations on that pool. The collector defines how to summarize the operations
matched by the filter, and which metrics to collect (e.g. IOPS, throughput per second, or average
latency). There are four types of collectors, suitable for different use cases:

• COUNTER - this is the simplest type of collector. It simply counts the operations that match the
filter.
• TOP - used to find out which entities in the system have the highest value in some parameter. For
example, which hosts are doing the most read IOPS, or which filesystems have the highest write
latencies.
• HISTOGRAM - counts operations that match the filter, separating them into "buckets" according
to some parameter. A typical use case would be to count the number of I/O operations per block
size (e.g. 4Kb, 8Kb, 16Kb etc.) or per type (read, write, xcopy etc.).
• AGGREGATOR - this collector counts operations across all entities of a specific type. For example,
it could collect write operations per volume in the system, or read operations per pool.

 All collectors return performance counters in 1 second intervals, except for aggregators which use
10 second intervals by default.

The following steps are needed to create and use a performance counter:

Performance monitoring  –  327

 InfiniBox 4.0.10 User Documentation

1. Create a filter.
2. Add one or more filtering criteria to the filter.
3. Create a collector (attaching it to the filter).
4. Poll the collector periodically to retrieve the performance metrics.
5. Delete the collector when it is no longer needed.

 A collector that isn't polled gets automatically garbage-collected after 500 seconds of inactivity (this
value is different in case of AGGREGATOR collectors - see below for details).

The following sections describe the relevant API calls in detail.

20.4.2 Creating a filter 

When creating a filter we must first specify the protocol or protocol type this filter will be applied to. The
API supports discovery of all available filter and collector fields, so we can use this request to get a list
of available protocols and types:

Available Protocols Request

GET /api/rest/metrics/available_fields

Available Protocols Response

{
"result": {
"available_filter_fields": [{
"name": "protocol",
"type": "enum",
"values": ["NFS", "NFS_QOS", "FC", "iSCSI"]
}, {
"name": "protocol_type",
"type": "enum",
"values": ["RMR", "SAN", "SAN_QOS"]
}]
},
"error": null,
"metadata": {
"ready": true
}
}

To create the filter we'll send a POST request to the filters endpoint, and specify the wanted protocol or
protocol type. For example to create filter which will apply to SAN I/O operations: 

Performance monitoring  –  328

 InfiniBox 4.0.10 User Documentation

Create Filter Request

POST /api/rest/metrics/filters
{
"protocol_type": "SAN"
}

Create Filter Response

{
"result": {
"id": 35184372088996
},
"error": null,
"metadata": {
"ready": true
}
}

Refining the filter

We now have a filter that matches all I/O operations under a specific protocol or protocol type. We can
refine it further, so that it will match only a subset of the operations.

The response of the POST request above contains the id of the filter that was created. Using this id we
can discover the available filter fields that can be used for refining the filter:

Available Fields Request

GET /api/rest/metrics/filters/<filter_id>/available_fields

Available Fields Response

{
"result": {
"id": 35184372088996,
"available_filter_fields": [{
"name": "protocol",
"type": "enum",
"values": [
"FC", "iSCSI"
]
}, {
"name": "operation_category",
"type": "enum",
"values": [
"login", "other", "read", "unmap", "write", "writesame", "xcopy"
]
}, {

Performance monitoring  –  329

 InfiniBox 4.0.10 User Documentation

"name": "operation_type",
"type": "enum",
"values": [
"caw", "format_unit", "inquiry", "log_sense", "login", "mode_select",
"mode_sense", "persistent_reserve_in", "persistent_reserve_out", "read",
"read_buf", "read_capacity", "receive_copy_result", "release", "report_luns",
"request_sense", "reserve", "rtpg", "send_diagnostic", "start_stop_unit",
"sync_cache", "task_management_functions", "test_unit_ready",
"unmap", "verify", "write", "write_buf", "write_same", "xcopy"
]
}, {
"name": "scsi_status",
"type": "enum",
"values": [
"busy", "error", "good"
]
}, {
"name": "pool_id",
"type": "uint"
}, {
"name": "vol_id",
"type": "uint"
}, {
"name": "host_id",
"type": "uint"
},
.... SNIP ...
],
"available_collector_fields": [{
"name": "operation_size",
"unit": "B",
"description": "Total size of all SCSI operations (successful and failed)"
}, {
"name": "throughput",
"unit": "B/Sec",
"description": "Total size of successful SCSI operations"
}, {
"name": "external_latency",
"unit": "ms",
"description": "The total latency to complete the SCSI command, from the initial request was received
until the storage sent the final status.
External latency may increase due to host / fabric delays"
}
.... SNIP ...
}]
},
"error": null,
"metadata": {
"ready": true
}
}

Performance monitoring  –  330

 InfiniBox 4.0.10 User Documentation

 The request above lists only the basic fields. To see the full list, add ?level=ADVANCED to the URL.
This document includes below a reference of all filter fields and collector fields per protocol / protocol
type.

We can now use the filter id and any of the available filter fields to further limit the I/O operations that
will be matched by our filter. For example if we want to count only read operations:

Modify Filter Request

PUT /api/rest/metrics/filters/<filter_id>
{
"operation_category": "read"
}

Modify Filter Response

{
"result": {
"id": 35184372088996
},
"error": null,
"metadata": {
"ready": true
}
}

 It is not possible to send more than one filtering field per request. Send a separate PUT request for
each field.

20.4.3 Creating a COUNTER collector

Three parameters are required to create a COUNTER collector:

• filter_id - the id of the filter that was previously created

• type - "COUNTER"
• collected_fields - a list of field names to collect, out of the available collector fields for the filter's
protocol or protocol type.

Create COUNTER Collector Request

POST /api/rest/metrics/collectors
{
"filter_id": 35184372088996,
"type": "COUNTER",
"collected_fields": ["ops", "throughput"]
}

Performance monitoring  –  331

 InfiniBox 4.0.10 User Documentation

Create COUNTER Collector Response

{
"result": {
"id": 35184372089045,
"filter_id": 35184372088996,
"collected_fields": ["ops", "throughput"],
"type": "COUNTER"
},
"error": null,
"metadata": {
"ready": true
}
}

It is also possible to create a filter and a COUNTER collector using a single API request, without creating
the filter in advance:

Create Filter and COUNTER Collector Request

POST /api/rest/metrics/collectors
{
"type": "COUNTER",
"filters": {
"protocol_type": "SAN",
"operation_category": "read"
},
"collected_fields": ["ops", "throughput"]
}

Create Filter and COUNTER Collector Response

{
"result": {
"id": 35184372089047,
"filters": {
"protocol_type": "SAN",
"operation_category": "read"
},
"filter_id": 35184372089046,
"collected_fields": [
"ops",
"throughput"
],
"type": "COUNTER"
},
"error": null,
"metadata": {
"ready": true
}

Performance monitoring  –  332

 InfiniBox 4.0.10 User Documentation

Collecting the metrics

Once the collector is created, it starts accumulating data. We need to periodically poll the collector to
retrieve the data which was collected so far. The data is requested using the collector id; up to 40
COUNTER collector ids can be specified in a single request if they belong to the same protocol.

Get Collector Data Request

GET /api/rest/metrics/collectors/data?collector_id=in:[<id>,<id>,...]

Get Collector Data Response

{
"result": {
"collectors": [{
"id": 35184372089043,
"fields": ["ops", "throughput"],
"data": [
[14451, 467937729],
[12736, 413005671]
],
"collector_type": "COUNTER",
"interval_milliseconds": 1000,
"end_timestamp_milliseconds": 1496922832312
}]
},
"error": null,
"metadata": {
"ready": true
}
}

The response contains a list of collectors, with one item for each collector id that was requested. Each
item includes the following fields:

• id - the collector id
• fields - names of the collected data fields
• data - a list of collected entries, one entry per second. Each entry is a list of values ordered
according to the field names that appear under fields.
• interval_milliseconds - the number of milliseconds covered by each entry in the data list.
• end_timestamp_milliseconds - the timestamp of the last entry in the data list, in milliseconds
since the epoch. Using the timestamp and the interval, it is possible to calculate the timestamp of
all entries in the list.

So the response above contains the following metrics:

Performance monitoring  –  333

 InfiniBox 4.0.10 User Documentation

Timestamp Ops Throughput

1496922831312 (2017-06-08 11:53:51.312 UTC) 14451 467937729

1496922832312 (2017-06-08 11:53:52.312 UTC) 12736 413005671

20.4.4 Creating a TOP collector

Several parameters are required to create a TOP collector:

• filter_id - the id of the filter that was previously created

• type - "TOP"
• collected_fields - a list of field names to collect, out of the available collector fields for the filter's
protocol or protocol type.
• grouping_field - a filtering field that defines the type of entities we're interested in. For example
vol_id, pool_id, source_ip etc.
• sorting_field - a collector field that defines which parameter to sort the entities by. For example
ops, throughput, errors, etc.
• max_results - the number of entities to return (up to 1000).
• sort_ascending - when true, sorts the sorting_field from low to high. Otherwise, sorts it from
high to low (this is the default).

You can understand how these parameters work together by putting them into a sentence: Return the
top <max_results> <grouping_field> ordered by <sorting_field>.

For example, the top <10> <vol_id> ordered by <ops> :

Create TOP Collector Request

POST /api/rest/metrics/collectors
{
"type": "TOP",
"filter_id": 35184372089056,
"collected_fields": ["ops", "throughput"],
"grouping_field": "vol_id",
"sorting_field": "ops",
"max_results": 10
}

Create TOP Collector Response

{
"result": {

Performance monitoring  –  334

 InfiniBox 4.0.10 User Documentation

"id": 35184372089057,
"filter_id": 35184372089056,
"collected_fields": [
"ops",
"throughput"
],
"grouping_field": "vol_id",
"sorting_field": "ops",
"max_results": 10,
"sort_ascending": false,
"type": "TOP"
},
"error": null,
"metadata": {
"ready": true
}
}

Collecting the metrics

Once the collector is created, it starts accumulating data. We need to periodically poll the collector to
retrieve the data which was collected so far. The data is requested using the collector id; up to 2 TOP
collector ids can be specified in a single request if they belong to the same protocol.

Get Collector Data Request

GET /api/rest/metrics/collectors/data?collector_id=in:[<id>,<id>,...]

Get Collector Data Response

{
"result": {
"collectors": [{
"id": 35184372089057,
"fields": ["vol_id", "ops", "throughput"],
"data": [
[
["61321", 2794, 67253941],
["61317", 2352, 101169969],
["61319", 2324, 77666658]
],
[
["61321", 4636, 111570820],
["61319", 3479, 116310076],
["61317", 3471, 149286622]
]
],
"collector_type": "TOP",
"interval_milliseconds": 1000,
"end_timestamp_milliseconds": 1497258041312,

Performance monitoring  –  335

 InfiniBox 4.0.10 User Documentation

"grouping_field": "vol_id",
"sorting_field": "ops"
}]
},
"error": null,
"metadata": {
"ready": true
}
}

The response contains a list of collectors, with one item for each collector id that was requested. Each
item includes the following fields:

• id - the collector id

• fields - names of the collected data fields
• data - a list of collected entries, one entry per second. Each entry is a list of up to
max_results top entities. The values in each entity are ordered according to the field names that
appear under fields.
• interval_milliseconds - the number of milliseconds covered by each entry in the data list.
• end_timestamp_milliseconds - the timestamp of the last entry in the data list, in milliseconds
since the epoch. Using the timestamp and the interval, it is possible to calculate the timestamp of
all entries in the list.

So the response above contains the following metrics:

Timestamp Top volume ids Ops Throughput

1497258040312 (2017-06-12 09:00:40.312 61321 2794 67253941

UTC)
  61317 2352 101169969
 

61319 2324 77666658

1497258041312 (2017-06-12 09:00:41.312 61321 4636 111570820

UTC)
  
61319 3479 116310076

61317 3471 149286622

Performance monitoring  –  336

 InfiniBox 4.0.10 User Documentation

20.4.5 Creating a HISTOGRAM collector

Four parameters are required to create a HISTOGRAM collector:

• filter_id - the id of the filter that was previously created.

• type - "HISTOGRAM"
• collected_fields - a list of field names to collect, out of the available collector fields for the filter's
protocol or protocol type.
• histogram_field - a filter field of type enum or histogram. The possible values of this field define
the "buckets" of the histogram.

For example if we want to collect the number of I/O operations and the total throughput per SAN
operation category (read, write, xcopy, etc.):

Create HISTOGRAM Collector Request

POST /api/rest/metrics/collectors
{
"type": "HISTOGRAM",
"filter_id": 35184372089058,
"collected_fields": ["ops", "throughput"],
"histogram_field": "operation_category"
}

Create HISTOGRAM Collector Response

{
"result": {
"id": 35184372089061,
"filter_id": 35184372089058,
"collected_fields": ["ops", "throughput"],
"histogram_field": "operation_category",
"type": "HISTOGRAM"
},
"error": null,
"metadata": {
"ready": true
}
}

Collecting the metrics

Once the collector is created, it starts accumulating data. We need to periodically poll the collector to
retrieve the data which was collected so far. The data is requested using the collector id; up to 10
HISTOGRAM collector ids can be specified in a single request if they belong to the same protocol.

Performance monitoring  –  337

 InfiniBox 4.0.10 User Documentation

Get Collector Data Request

GET /api/rest/metrics/collectors/data?collector_id=in:[<id>,<id>,...]

Get Collector Data Response

{
"result": {
"collectors": [{
"id": 35184372089061,
"fields": ["ops", "throughput"],
"data": [
[
[641, 30521856], [3751, 111668224], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]
],
[
[1837, 87529598], [10901, 323443980], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]
]
],
"collector_type": "HISTOGRAM",
"interval_milliseconds": 1000,
"end_timestamp_milliseconds": 1497259383017,
"histogram_field": "operation_category",
"ranges": ["read", "write", "xcopy", "unmap", "writesame", "login", "other"]
}]
},
"error": null,
"metadata": {
"ready": true
}
}

The response contains a list of collectors, with one item for each collector id that was requested. Each
item includes the following fields:

• id - the collector id.

• fields - names of the collected data fields.
• data - a list of collected entries, one entry per second. Each entry contains buckets matching the
buckets listed under ranges. Inside each bucket is a list of values ordered according to the field
names that appear under fields.
• interval_milliseconds - the number of milliseconds covered by each entry in the data list.
• end_timestamp_milliseconds - the timestamp of the last entry in the data list, in milliseconds
since the epoch. Using the timestamp and the interval, it is possible to calculate the timestamp of
all entries in the list.
• ranges - a list of names, one for each bucket in the histogram.

So the response above contains the following metrics:

Performance monitoring  –  338

 InfiniBox 4.0.10 User Documentation

Timestamp Histogram bucket Ops Throughput

1497259382017 (2017-06-12 09:23:02.017 read 641 30521856

UTC)

write 3751 111668224

xcopy 0 0

unmap 0 0

writesame 0 0 

login 0 0 

other 0  0 

1497259383017 (2017-06-12 09:23:03.017 read 1837 87529598

UTC)

write 10901 323443980

xcopy 0 0

unmap 0 0

writesame 0 0

login 0 0

other 0 0

20.4.6 Creating an AGGREGATOR collector

Four parameters are required to create a AGGREGATOR collector:

• filter_id - the id of the filter that was previously created

• type - "AGGREGATOR"

Performance monitoring  –  339

 InfiniBox 4.0.10 User Documentation

• collected_fields - a list of field names to collect, out of the available collector fields for the filter's
protocol or protocol type.
• grouping_field - a filtering field that defines the type of entities we're interested in. For example
vol_id, pool_id, source_ip etc.

It is also possible to specify the interval_milliseconds to be used by the aggregator - see below.

For example if we want to collect the number of I/O operations and the total throughput for all
volumes:

Create AGGREGATOR Collector Request

POST /api/rest/metrics/collectors
{
"type": "AGGREGATOR",
"filter_id": 35184372089062,
"collected_fields": ["ops", "throughput"],
"grouping_field": "vol_id"
}

Create AGGREGATOR Collector Response

{
"result": {
"id": 35184372089063,
"filter_id": 35184372089062,
"collected_fields": ["ops", "throughput"],
"grouping_field": "vol_id",
"type": "AGGREGATOR"
},
"error": null,
"metadata": {
"ready": true
}
}

Setting the metrics collection interval

Aggregators collect data in 10 second intervals by default. It is possible to select another interval from
the table below by passing it as the value of interval_milliseconds when creating
the AGGREGATOR collector. The chosen interval also controls how often the collector must be polled for
data. Failing to do so will lead to the collector's deletion.

Interval (ms) In words Maximum polling time

1000 1 second 80 seconds

Performance monitoring  –  340

 InfiniBox 4.0.10 User Documentation

Interval (ms) In words Maximum polling time

10000 10 seconds 80 seconds

30000 30 seconds 3 minutes

60000 1 minute 6 minutes

120000 2 minutes 12 minutes

300000 5 minutes 30 minutes

Collecting the metrics

Once the collector is created, it starts accumulating data. We need to periodically poll the collector to
retrieve the data which was collected so far. The data is requested using the collector id; up to 2
AGGREGATOR collector ids can be specified in a single request if they belong to the same protocol.

Get Collector Data Request

GET /api/rest/metrics/collectors/data?collector_id=in:[<id>,<id>,...]

Get Collector Data Response

{
"result": {
"collectors": [{
"id": 35184372089064,
"fields": ["vol_id", "ops", "throughput"],
"data": [
[
[61321, 42269, 1017175338],
[61319, 36030, 1207902580],
[61317, 36014, 1548901690]
],
[
[61319, 34874, 1169198774],
[61321, 41330, 994592385],
[61317, 34928, 1502204618]
]
],
"collector_type": "AGGREGATOR",
"interval_milliseconds": 10000,
"end_timestamp_milliseconds": 1497258041312,
"grouping_field": "vol_id"
}]
},
"error": null,

Performance monitoring  –  341

 InfiniBox 4.0.10 User Documentation

"metadata": {
"ready": true
}
}

The response contains a list of collectors, with one item for each collector id that was requested. Each
item includes the following fields:

• id - the collector id.

• fields - names of the collected data fields.
• data - a list of collected entries, one entry per 10 seconds. Each entry contains all the entities
that performed I/O (entities that did not do any I/O are omitted). For each entity there is a list of
values ordered according to the field names that appear under fields.
• interval_milliseconds - the number of milliseconds covered by each entry in the data list.
• end_timestamp_milliseconds - the timestamp of the last entry in the data list, in milliseconds
since the epoch. Using the timestamp and the interval, it is possible to calculate the timestamp of
all entries in the list.

The response above contains the following metrics, for all volumes that did any I/O:

Timestamp Volume id Ops Throughput

1497258031312 (2017-06-12 09:00:31.312 UTC) 61321 42269 1017175338

 
  61319 36030 1207902580

61317 36014 1548901690

1497258041312 (2017-06-12 09:00:41.312 UTC) 61319 34874 1169198774

  
61321 41330 994592385

61317 34928 1502204618

 Note that even though AGGREGATORS can collect data in multi-second intervals, the collected fields
contain values averaged over the entire interval. So In the table above, 42269 is the average number of
operations per second.

Performance monitoring  –  342

 InfiniBox 4.0.10 User Documentation

20.4.7 Field Reference

NFS

Filter fields

Name Type Allowed values

Performance monitoring  –  343

 InfiniBox 4.0.10 User Documentation

operation_type enum • access

• commit
• create
• dump
• export
• fs_information
• fs_statistics
• get_attributes
• link
• lookup
• make_directory
• make_node
• mount
• null_mnt
• null_nfs
• path_configuration
• read
• read_directory
• read_directory_plus
• read_link
• remove
• remove_directory
• rename
• set_attributes
• symbolic_link
• unmount
• unmount_all
• write

user_id uint

group_id uint

fs_entity_id uint

Performance monitoring  –  344

 InfiniBox 4.0.10 User Documentation

export_id uint

offset uint

source_ip uint

destination_ip uint

operation_category enum • meta_data

• read
• write

file_path uint[] The file_path parameter is a combination of the fs_id and the
file_path

treeq uint[] The treeq parameter is a combination of the fs_id and the treeq

pool_id uint

operation_size_histogram histogra • ge_32k_lt_64k

m
• ge_4k_lt_8k
• ge_512k
• ge_64k_lt_512k
• ge_8k_lt_32k
• lt_4k

Collector fields

Name Unit Description

ops n/a The number of Operations per second

operation_size bytes Breakdown of operations by size

average_operation_size bytes Average operations size

Performance monitoring  –  345

 InfiniBox 4.0.10 User Documentation

throughput bytes/sec Total throughput per second

latency ms Average latency for all operations

worker_time ms Average latency for all operations excluding waiting time in the
queue

sections_read n/a The total number of sections which have been read

sections_read_from_cache n/a The number of sections which have been read from cache

sections_read_from_ssd n/a The number of sections which have been read from ssd

sections_read_from_disk n/a The number of sections which have been read from disk

Collector fields

Name Unit Description

qos_latency ms The total time it took QOS to answer

approved_ios n/a The number of approved operations per second

approved_throughput n/a Total size of approved operations per second

FC

Filter fields

Name Type Allowed values

protocol_type enum • SAN

• SAN_QOS

Performance monitoring  –  346

 InfiniBox 4.0.10 User Documentation

operation_category enum • login

• other
• read
• unmap
• write
• writesame
• xcopy

Performance monitoring  –  347

 InfiniBox 4.0.10 User Documentation

operation_type enum • caw

• format_unit
• inquiry
• log_sense
• login
• mode_select
• mode_sense
• persistent_reserve_in
• persistent_reserve_out
• read
• read_buf
• read_capacity
• receive_copy_result
• release
• report_luns
• request_sense
• reserve
• rtpg
• send_diagnostic
• start_stop_unit
• sync_cache
• task_management_functions
• test_unit_ready
• unmap
• verify
• write
• write_buf
• write_same
• xcopy

scsi_status enum • busy

• error
• good

Performance monitoring  –  348

 InfiniBox 4.0.10 User Documentation

pool_id uint

vol_id uint

lba uint

host_id uint

initiator_port_key uint

target_WWPN uint

target_port uint

source_ip uint

destination_ip uint

network_space_id uint

initiator_WWPN uint

initiator_IQN uint

operation_size_histogram histogram • ge_32k_lt_64k

• ge_4k_lt_8k
• ge_512k
• ge_64k_lt_512k
• ge_8k_lt_32k
• lt_4k

Collector fields

Name Unit Description

Performance monitoring  –  349

 InfiniBox 4.0.10 User Documentation

operation_si bytes Total size of all SCSI operations (successful and failed)
ze

throughput bytes Total size of successful SCSI operations

/sec

external_late ms The total latency to complete the SCSI command, from the initial request was
ncy received until the storage sent the final status. External latency may increase due to
host / fabric delays

internal_late ms The total latency incured waiting for the InfiniBox backend to complete the
ncy operation. Internal latency is not affected by host / fabric delays.

external_late ms Same as extenal latency, not counting errors

ncy_wout_er
r

internal_late ms Same as intenal latency, not counting errors

ncy_wout_er
r

sections_rea n/a The total number of sections which have been read
d

sections_rea n/a The number of sections which have been read from cache
d_from_cach
e

sections_rea n/a The number of sections which have been read from ssd
d_from_ssd

sections_rea n/a The number of sections which have been read from disk
d_from_disk

ops n/a The number of Operations per second

aborted_task n/a Number of tasks aborted by the InfiniBox backend

errors n/a Number of errors detected in SCSI operations

Performance monitoring  –  350

 InfiniBox 4.0.10 User Documentation

average_ope bytes Average size of all SCSI operation (successful and failed)
ration_size

iSCSI

Filter fields

Name Type Allowed values

protocol_type enum • SAN

• SAN_QOS

operation_category enum • login

• other
• read
• unmap
• write
• writesame
• xcopy

Performance monitoring  –  351

 InfiniBox 4.0.10 User Documentation

operation_type enum • caw

• format_unit
• inquiry
• log_sense
• login
• mode_select
• mode_sense
• persistent_reserve_in
• persistent_reserve_out
• read
• read_buf
• read_capacity
• receive_copy_result
• release
• report_luns
• request_sense
• reserve
• rtpg
• send_diagnostic
• start_stop_unit
• sync_cache
• task_management_functions
• test_unit_ready
• unmap
• verify
• write
• write_buf
• write_same
• xcopy

scsi_status enum • busy

• error
• good

Performance monitoring  –  352

 InfiniBox 4.0.10 User Documentation

pool_id uint

vol_id uint

lba uint

host_id uint

initiator_port_key uint

target_WWPN uint

target_port uint

source_ip uint

destination_ip uint

network_space_id uint

initiator_WWPN uint

initiator_IQN uint

operation_size_histogram histogram • ge_32k_lt_64k

• ge_4k_lt_8k
• ge_512k
• ge_64k_lt_512k
• ge_8k_lt_32k
• lt_4k

Collector fields

Name Unit Description

Performance monitoring  –  353

 InfiniBox 4.0.10 User Documentation

operation_si bytes Total size of all SCSI operations (successful and failed)
ze

throughput bytes Total size of successful SCSI operations

/sec

external_late ms The total latency to complete the SCSI command, from the initial request was
ncy received until the storage sent the final status. External latency may increase due to
host / fabric delays

internal_late ms The total latency incured waiting for the InfiniBox backend to complete the
ncy operation. Internal latency is not affected by host / fabric delays.

external_late ms Same as extenal latency, not counting errors

ncy_wout_er
r

internal_late ms Same as intenal latency, not counting errors

ncy_wout_er
r

sections_rea n/a The total number of sections which have been read
d

sections_rea n/a The number of sections which have been read from cache
d_from_cach
e

sections_rea n/a The number of sections which have been read from ssd
d_from_ssd

sections_rea n/a The number of sections which have been read from disk
d_from_disk

ops n/a The number of Operations per second

aborted_task n/a Number of tasks aborted by the InfiniBox backend

errors n/a Number of errors detected in SCSI operations

Performance monitoring  –  354

 InfiniBox 4.0.10 User Documentation

average_ope bytes Average size of all SCSI operation (successful and failed)
ration_size

RMR

Filter fields

Name Type Allowed values

operation_type enum • read

• write

io_type enum • delta

• delta_with_unmap
• offline_unmap
• unmap

replica_id uint

replication_type enum • async

• sync

dataset_id uint

pair_dataset_id uint

node_id uint

Collector fields

Name Unit Description

ops n/a The number of Operations per second

Performance monitoring  –  355

 InfiniBox 4.0.10 User Documentation

average_operation_size bytes Average operations size

throughput bytes/sec Total throughput per second

latency_histogram ms Breakdown of operations by latency

latency ms Average latency for all operations

sections_read_from_cache n/a The number of sections which have been read from cache

sections_read_from_disk n/a The number of sections which have been read from disk

sections_read_from_ssd n/a The number of sections which have been read from ssd

sections_read n/a The total number of sections which have been read

SAN

Filter fields

Name Type Allowed values

protocol enum • FC
• iSCSI

operation_category enum • login

• other
• read
• unmap
• write
• writesame
• xcopy

Performance monitoring  –  356

 InfiniBox 4.0.10 User Documentation

operation_type enum • caw

• format_unit
• inquiry
• log_sense
• login
• mode_select
• mode_sense
• persistent_reserve_in
• persistent_reserve_out
• read
• read_buf
• read_capacity
• receive_copy_result
• release
• report_devid
• report_id
• report_luns
• request_sense
• reserve
• rtpg
• send_diagnostic
• start_stop_unit
• sync_cache
• task_management_functions
• test_unit_ready
• unmap
• verify
• write
• write_buf
• write_same
• xcopy

Performance monitoring  –  357

 InfiniBox 4.0.10 User Documentation

scsi_status enum • busy

• error
• good

pool_id uint

vol_id uint

lba uint

host_id uint

initiator_port_key uint

target_WWPN uint

target_port uint

source_ip uint

destination_ip uint

network_space_id uint

initiator_WWPN uint

initiator_IQN uint

operation_size_histogram histogram • ge_32k_lt_64k

• ge_4k_lt_8k
• ge_512k
• ge_64k_lt_512k
• ge_8k_lt_32k
• lt_4k

Performance monitoring  –  358

 InfiniBox 4.0.10 User Documentation

Collector fields

Name Unit Description

operation_si bytes Total size of all SCSI operations (successful and failed)
ze

throughput bytes Total size of successful SCSI operations

/sec

external_late ms The total latency to complete the SCSI command, from the initial request was
ncy received until the storage sent the final status. External latency may increase due to
host / fabric delays

internal_late ms The total latency incured waiting for the InfiniBox backend to complete the
ncy operation. Internal latency is not affected by host / fabric delays.

external_late ms Same as extenal latency, not counting errors

ncy_wout_er
r

internal_late ms Same as intenal latency, not counting errors

ncy_wout_er
r

sections_rea n/a The total number of sections which have been read
d

sections_rea n/a The number of sections which have been read from cache
d_from_cach
e

sections_rea n/a The number of sections which have been read from ssd
d_from_ssd

sections_rea n/a The number of sections which have been read from disk
d_from_disk

ops n/a The number of Operations per second

Performance monitoring  –  359

 InfiniBox 4.0.10 User Documentation

aborted_task n/a Number of tasks aborted by the InfiniBox backend

errors n/a Number of errors detected in SCSI operations

average_ope bytes Average size of all SCSI operation (successful and failed)
ration_size

SAN_QOS

Filter fields

Name Type Allowed values

qos_status enum • lag

• reject
• success

qos_pool_id uint

qos_entity_id uint

resource_type enum • entity

• pool

resource_unit enum • bw
• io

Collector fields

Name Unit Description

qos_latency ms The total time it took QOS to answer

Performance monitoring  –  360

 InfiniBox 4.0.10 User Documentation

approved_ios n/a The number of approved operations per second

approved_throughput n/a Total size of approved operations per second

interval_milliseconds

Performance monitoring  –  361

 InfiniBox 4.0.10 User Documentation

21 Quality of service

21.1 Quality of Service overview

• Introduction
• The QoS policy
• QoS configuration limits
• What happens when the limit is reached
• Burst
• Burst allowance
• Amount of non-consumed IOPS (or throughput)
• Burst factor
• QoS policy and special SCSI commands
• QoS policy and replicated volumes
• Assigning a QoS policy to a pool

21.1.1 Introduction
InfiniBox system features a virtualized storage service that is shared by all hosts and applications.
InfiniBox accommodates high-performance requirements for multiple applications with varying
performance objectives allowing for fair resource allocation. QoS is available either at the single volume
level, or all of the volumes within a single pool. The QoS feature aligns the system performance to the
varying business needs, by enabling the user to restrict IOPS and throughput, or both to the relevant
entity. In addition, QoS solves the noisy neighbor problem by limiting the resources the entity can use.
The Quality of Service (QoS) provides a facility to limit the performance of volumes (either individually
or as part of a pool) on a per IOPS and/or throughput basis. 

The QoS feature allows to establish performance tiers and to sustain high-performance needs, allowing
the user to differentiate performance levels to their applications.

21.1.2 The QoS policy

The QoS policy is a logical entity that sets performance limits levels that can later be assigned to
datasets. The QoS policy levels are comprised of the following:

• Max IOPS - the maximum number of IOPS that are allowed for a volume
• Max throughput - the maximum throughput that is allowed for a volume
• Burst factor - the factor of increased performance during the burst. The burst cannot exceed this
factor. See also: Burst

Any volume, or pool, in the InfiniBox system can be assigned to, or removed from a QoS policy. 

Quality of service  –  362

 InfiniBox 4.0.10 User Documentation

The QoS policy applies restrictions only on FC-originated IOPS and throughput (not iSCSI).

QoS configuration limits

See: QoS limits.

21.1.3 What happens when the limit is reached

In case that the actual performance exceeds the performance limit (as the host keeps sending
requests), InfiniBox takes measures to slow down the pace of the requests. InfiniBox applies artificial
latency that moderately returns the pace of requests to the amount that is allowed according to the
policy. The added latency is proportional to the amount of IOPS or throughput that the entity tries to
consume above the limit. The effect of the added latency on the system resources is minimal.

If the host still does not slow down the pace of the requests, InfiniBox will reject the extraneous IO
requests (using the DEVICE_BUSY response). 

21.1.4 Burst
The Burst option allows the entity with an assigned QoS policy to consume more IOPS (or throughput)
than the Max IOPS (and Max Throughput) for a short period of time. An entity that has not consumed
the entire allowed performance gains allowance that can be used later to exceed the QoS max limits.
 The burst factor determines the amount of performance that the entity (either a volume or a pool) is
allowed to consume above its max limits. The actual IOPS (or Throughput) during the burst cannot
exceed the multiplication of the burst factor by the max IOPS (or Throughput). 

Burst allowance
The burst allowance is determined by the lowest of the following two factors:

• Amount of non-consumed IOPS (or throughput)

• The burst factor that is set in the policy

Amount of non-consumed IOPS (or throughput)

The following picture shows the IOPS consumed by an entity along a specified period of time. On the
graph, two time frames are marked, denoting the following:

1. The area marked with 1, represents the number of IOPS (or throughput) that the entity did not
consume although the policy allowed that
2. The area marked with 2, represents the number of IOPS (or throughput) that is available for the
entity, for above-the-maximum consumption. This number is proportional to the amount of not-
consumed IOPS (or throughput)

Quality of service  –  363

 InfiniBox 4.0.10 User Documentation

Burst factor
The burst is also limited by the burst factor:

21.1.5 QoS policy and special SCSI commands

Some SCSI commands are not limited by the QoS policy due to their special nature, or the way they are
utilized. 

SCSI commands that are not accounted for credits and are not limited:

• SCSI-INQUIRY
• Test-unit-ready

SCSI commands that are not accounted for throughput (they are accounted for IOPS only):

• WRITE_SAME
• UNMAP

Quality of service  –  364

 InfiniBox 4.0.10 User Documentation

• XCOPY - accounted only for the source volume IOPS

21.1.6 QoS policy and replicated volumes

The IOPS (and throughput) that the source volume receives from the host are accounted for the QoS.
The replication-associated operations (reads from the source and writes to the target) that occur during
the replication are not accounted for. This is the case for both asynchronous and synchronous
replication.

Note: The replication throughput rate can be limited from the Network Space. To set the network space
rate limit, see: Modifying a network space.

21.1.7 Assigning a QoS policy to a pool

When the pool is assigned to a policy, all of its volumes cumulatively share the same policy limits. The
performance limits imposed by the pool QoS policy apply to all of the pool's volumes. IOPS (and
throughput) that are consumed by a specific volume, are deducted from the IOPS (and throughput) that
are available to all other volumes of the pool. When the pool reaches the limit of IOPS (or throughput),
all of the pool volumes are affected. IOPS (or throughput) to all of the pool volumes is slowed-down (or
even rejected, as explained above), regardless of their previous IOPS (or throughput) consumption (first-
come, first-served).

In the following example, a pool with a QoS policy with max IOPS has 3 volumes. Each of the 3 volumes
accepts 1,000 IOPS. The graph below displays a stacked view of the volumes IOPS, where cumulatively
the 3 volumes accept 3,000 IOPS. At this point (time t0)The cumulative IOPS of all of the volumes is
below the policy's max IOPS limit of 5,000. Next, at time t1, each of the volumes accepts an increasing
amount of IOPS. If the volumes did not have the constraint of the pool QoS policy, they could accept
more IOPS. However, as the volumes are subject to the pool QoS policy, they fairly share the allowed
IOPS (5,000 IOPS) between them.

Quality of service  –  365

 InfiniBox 4.0.10 User Documentation

If the pool-volumes QoS policy allows burst, the burst is allowed to all of the pool's volumes.

The volume can be policed by a single volume policy and a single pool-volumes policy (that is imposed
on the pool that the volume belongs to) at the same time. A volume that is both individually policed and
belongs to a policed pool, is affected by both policies. Whenever one of the limits is reached, the
volume performance is affected.

Note: only the pool's volumes are affected by the pool's QoS policy. The pool's filesystems are not
affected. Their OPS and throughput are not accounted. 

21.2 QoS limits

• Policy
• The maximum number of QoS policies on the system: 5000
• Maximum entities in a single policy: 500
• Capping
• The lowest value for Volume Performance capping (IO/s, MB/s) is: 1,000 IO/s and/or 50MB/
s
• The lowest value for pool Performance capping (IO/s, MB/s) is: 10,000 IO/s and/or 250MB/s
• Burst
• Burst factor minimum value: x1.1
• Burst factor maximum value: x10
• QoS effect on IO requests when crossing performance thresholds 

Quality of service  –  366

 InfiniBox 4.0.10 User Documentation

• The added latency for delayed IO requests is up to 200 ms

• The maximum outstanding IO requests per FC port due to QoS added latency is 800 

21.3 QoS policy operations

• Creating a QoS policy
• GUI instructions
• Assign an entity to a policy
• GUI Instructions
• Modify a QoS policy
• GUI Instructions
• Delete a QoS policy
• GUI Instructions
• Assign a volume to a policy
• From the Volumes screen
• From the QoS Policy screen
• Unassign a volume to a policy
• From the Volumes screen
• From the QoS Policy screen
• Assign a pool to a policy
• From the Pools screen
• From the QoS Policy screen
• Unassign a pool to a policy
• From the Volumes screen
• From the QoS Policy screen

21.3.1 Creating a QoS policy

The QoS policy groups the following performance capping levels: Max IOPS, Max MBPS, Burst,
Burst factor and Burst duration. The QoS policy entity is assigned to an entity type (either a volume, or a
pool).

Note: burst duration is available from the API only, and is hidden by CLI/GUI

GUI instructions
1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.

Quality of service  –  367

 InfiniBox 4.0.10 User Documentation

2. Click Create Policy. The Create Policy screen opens.

3. Set the QoS attributes:
a. Policy name
b. Type - either a volume or pool-volumes
c. Set the Max IOPS
d. Set the Max Throughput
e. Check whether to enable burst
f. Set the burst factor
4. Click Create. The QoS policy is created and you are moved to the Policy screen.

21.3.2 Assign an entity to a policy

You can assign volumes to a Volume policy and pools to a Pool-Volume policy.

GUI Instructions
1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.
2. Click a policy. The policy screen opens.
3. Click Assign Volume or Assign Pool.
4. Select an entity from the list and click Assign. The entity is assigned.

21.3.3 Modify a QoS policy

You can modify the attributes of the QoS policy in order to change the way the policy affects the entities
performance.

GUI Instructions
1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.
2. Click a policy. The policy screen opens.
3. Select Modify Policy from the Actions menu.
4.  Change any of the policy's attributes, but the Type.
5. Click Modify. The policy is modified.

21.3.4 Delete a QoS policy

You can delete a policy while there are entities assigned to it.

Quality of service  –  368

 InfiniBox 4.0.10 User Documentation

GUI Instructions
1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.
2. Click a policy. The policy screen opens.
3. Select Delete Policy from the Actions menu.
4. Approve the warning message.
5. The policy is deleted.

21.3.5 Assign a volume to a policy

This task can be performed either from the Volumes screen or from the QoS policy screen. The QoS
column of the Volumes screen shows whether the volume is assigned to a policy, and to which policy it
is assigned.

From the Volumes screen

1. To access the Volumes screen: on the InfiniBox GUI, click the Volumes & FS icon on the menu to
the left. Then, find the relevant volume.
2. Right-click the volume and select Assign QoS Policy from the menu. The Assign QoS Policy
menu opens.
3. Select a policy and click Submit. The screen closes and the volume is assigned to the policy.

From the QoS Policy screen

1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.
2. Click a policy. The policy screen opens.
3. Click Assign Volume. A list of the system's volumes opens on the screen.
4. Select a volume and click Assign. The volume is assigned to the policy.

21.3.6 Unassign a volume to a policy

This task can be performed either from the Volumes screen or from the QoS policy screen.
The QoS column of the Volumes screen shows whether the volume is assigned to a policy, and to which
policy it is assigned.

Quality of service  –  369

 InfiniBox 4.0.10 User Documentation

From the Volumes screen

1. To access the Volumes screen: on the InfiniBox GUI, click the Volumes & FS icon on the menu to
the left. Then, find the relevant volume.
2. Right-click the volume and select Unassign QoS Policy from the menu. The volume is
unassigned from the policy.

From the QoS Policy screen

1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.
2. Click a policy. The policy screen opens.
3. Select a volume and click Unassign Volume. The volume is unassigned from the policy.

21.3.7 Assign a pool to a policy

This task can be performed either from the Pools screen or from the QoS policy screen.
The QoS column of the Pools screen shows whether the pool is assigned to a policy, and to which policy
it is assigned.

From the Pools screen

1. To access the Pools screen: on the InfiniBox GUI, click the Pools icon on the menu to the left.
Then, find the relevant pool.
2. Right-click the pool and select Assign QoS Policy from the menu. The Assign QoS Policy
menu opens.
3. Select a policy and click Submit. The screen closes and the pool is assigned to the policy.

From the QoS Policy screen

1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.
2. Click a policy. The policy screen opens.
3. Click Assign Pool. A list of the system's pools opens on the screen.
4. Select a pool and click Assign. The pool is assigned to the policy.

Quality of service  –  370

 InfiniBox 4.0.10 User Documentation

21.3.8 Unassign a pool to a policy

This task can be performed either from the Pools screen or from the QoS policy screen.
The QoS column of the Pools screen shows whether the pool is assigned to a policy, and to which policy
it is assigned.

From the Volumes screen

1. To access the Pools screen: on the InfiniBox GUI, click the Pools icon on the menu to the left.
Then, find the relevant pool.
2. Right-click the pool and select Unassign QoS Policy from the menu. The pool is unassigned from
the policy.

From the QoS Policy screen

1. To access the QoS screen: on the InfiniBox GUI, click the Settings icon on the menu to the left.
Then, click the QoS tab.
2. Click a policy. The policy screen opens.
3. Select a pool and click Unassign Pool. The pool is unassigned from the policy.

21.4 Examples of assigning QoS policies

• Examples of assigning policies
• Preparations
• Assigning policies to volumes
• Unassigning policies from volumes
• Assigning policies to a pool and a volume
• Unassigning all of the volumes
• Unassigning the pool

21.4.1 Examples of assigning policies

The examples that a discussed in this article use the following entities:

• Pool - a pool that contains some volumes, named p1

• Volumes - 3 volumes that are named v1, v2, v3
• Host - the host is mapped to all 3 volumes and is capable of writing nearly 30,000 IOPS

Quality of service  –  371

 InfiniBox 4.0.10 User Documentation

• Policies - several policies with various values of the max_iops parameter, named 9000iops,
5000iops, 3000iops and 1000iops

In order to prepare this article we used:

• InfiniShell - to create the pool, volumes and policies and to assign and unassign policies to the
pool and its entities
• InfiniMetrics - to view the impact of the policies assignment on the host writes

21.4.2 Preparations
Prior to the policies assignment, we query for volumes to verify that none of them is assigned to a
policy.

vol.query pool=p1 --columns=name,policy

NAME POLICY
v1 -
v2 -
v3 -

We also query for the policies.

qos_policy.query

NAME TYPE MAX OPS MAX THROUGHPUT BURST FACTOR

9000iops POOL_VOLUME 9000 - -

5000iops VOLUME 5000 - -
3000iops VOLUME 3000 - -
1000iops VOLUME 1000 - -

All of the volumes accept host writes. The cumulative 30,000 IOPS are evenly divided among all
volumes. The 3 volumes are visible on InfiniMetrics.

Quality of service  –  372

 InfiniBox 4.0.10 User Documentation

21.4.3 Assigning policies to volumes

We assign policies to the volumes:

• v1 is assigned to a policy with max 5,000 IOPS

• v2 is assigned to a policy with max 3,000 IOPS
• v3 is assigned to a policy with max 1,000 IOPS

The following numbered list refer to the numbers on the next image:

2a. The 3 volumes have exactly the same IOPS so the graph lines overlap (only v3 (color purple) is
visible)  
2b. When v1 is assigned to a policy that limits the IOPS to 5,000 (from the nearly 10,000 IOPS it
accepted to the host prior to the policy assignment), v2 and v3 accept more host writes
2c. When v2 is assigned to a policy that limits the IOPS to 3,000, v3 accepts more writes
2d. v3 is assigned to a policy that limits the IOPS to 1,000 

Shifting the InfiniMetrics to an unstacked (side-by-side) view, we can pin-point at the moment when v1  

Next, the total IOPS that are accepted by the 3 volumes are aligned to 5,000 + 3,000 + 1,000 = 9,000
IOPS.

21.4.4 Unassigning policies from volumes

Next, v1 and v2 are unassigned from their policies. v3 is still abiding the limitation that is imposed by
the policy is assign to. The two other volumes share all of the IOPS the host write. 

Quality of service  –  373

 InfiniBox 4.0.10 User Documentation

21.4.5 Assigning policies to a pool and a volume

We assign two policies, one for a pool and the other to a volume:

• The pool that the 3 volumes belong to is assigned to a policy with max 9,000 IOPS
• Volume v2, that was unassigned in the previous step, is assigned to a policy with max 1,000 IOPS
(that same policy that v3 is assigned to)

As a result, the total IOPS that are imposed by the policy the pool is assigned to is 9,000. This number is
divided between volumes v2 and v3 that are assigned to a policy, and volume 1 that is not assigned to a
policy.

21.4.6 Unassigning all of the volumes

When the volumes are unassigned from their policies, the pool is still assigned to its policy. Thus, the
volumes accept 9,000 IOPS that are equally divided between them (3,000 IOPs to each volume).

Quality of service  –  374

 InfiniBox 4.0.10 User Documentation

21.4.7 Unassigning the pool

When the pool is unassigned from its policy, the volumes are capable of accepting all of the available
host writes. 

Quality of service  –  375