COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Hyperscale XTM – Deployment

course
Module - Technology
Sept 2023

commvault.com | 888.746.3849
© 2023 Commvault. See here for information about our trademarks and patents.
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Notices and Disclaimers

Commvault, Commvault and logo, the "C hexagon” logo, Commvault Systems, Metallic, Metallic and logo,
the “Wave” logo, Commvault HyperScale X, HyperScale X, Recovery Reserve, and ThreatWise are
trademarks or registered trademarks of Commvault Systems, Inc. (“Commvault) The unauthorized use of
any Commvault trademark is strictly prohibited.

Other company and product names mentioned herein may be trademarks of their respective owners.
References to any third-party products, services, or websites should not be considered an endorsement
by Commvault. Some examples are for illustration only and are fictitious.

All right, title, and interest, including all intellectual property rights in and to this document and to any
related subject matter (collectively “Ownership Rights”) are owned and expressly reserved by Commvault.
No Ownership Rights are granted to you.

This document is intended for distribution to and personal reference use solely by Commvault customers;
all use of Commvault Solutions, including this document, is governed by Commvault’s Master Terms &
Conditions (currently available at https://www.commvault.com/legal/master-terms-and-conditions) which
are incorporated herein in their entirety.

This document is provided “as is.” Information in this document, including any specifications, URLs, or
other references, is subject to change without notice.
See www.commvault.com/IP for more information about our trademarks, patents, and other IP rights.

Confidentiality

This document contains information that is confidential and proprietary to Commvault. Without limiting
rights under copyright or otherwise, this information is provided with the express understanding that it will
be held in strict confidence and that no part of this document will be disclosed, used, reproduced, stored,
or transmitted, in whole or in part, for any purpose other than as expressly approved or provided by
Commvault in writing.

©1999-2023 Commvault

2
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Table of Contents
HyperScale XTM ................................................................................................................................................... 4

Learning Objectives .......................................................................................................................................... 5

HyperScale XTM Technology stack .............................................................................................................. 6

Commvault File System (CVFS) ................................................................................................................... 7

CVFS Storage Architecture Overview ........................................................................................................ 8

Commvault File System Proxy Overview .................................................................................................. 9

Commvault File System Server Architecture Overview ................................................................... 10

Resiliency .............................................................................................................................................................11

Using 20TB drives as 18TB .............................................................................................................................12

Quiz ....................................................................................................................................................................... 14

Standard Resiliency Mode ........................................................................................................................... 15

CVFS Healing ..................................................................................................................................................... 16

Improved Performance, Scalability and Resiliency ............................................................................. 17

HyperScale XTM High Availability ................................................................................................................ 18

Wrap Up............................................................................................................................................................... 19

3
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

HyperScale XTM

HyperScale XTM
Deployment Course

Module: HyperScale XTM Technology

© Commvault 2023
Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

Hello, and welcome to the Commvault HyperScale XTM Technology Module.

4
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Learning Objectives

Learning Objectives

• Understand key components of • Know how to speak in further

HyperScale XTM Technology depth to each individual layer
of the HyperScale XTM
• Understand what our file system
technology stack and how they
provides
work in concert with one
After completing this • Understand how the components another
work together
• Identify fundamental skills
module, you will: • Understand HyperScale X’s required to interact with
resiliency HyperScale XTM
• Understand CommServe® server • Know how to speak to the
options scalability and resiliency
available with HyperScale XTM

© Commvault 2023

Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

In this module, we’re going to discuss the individual components of the HyperScale XTM technology stack.
We’ll go over in further depth on how the Commvault® file system works in tandem with individual
components within the hardware. We’ll wrap up by focusing on HyperScale XTM resiliency and the
CommServe® options that are available with the HSX solution.

This module is intended to help guide more technical conversations around the HyperScale XTM solution for
scenarios that may occur in the field while performing or even just discussing HyperScale XTM.

5
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

HyperScale XTM Technology stack

HyperScale XTM Technology stack

Overview

RHEL
01 Red Hat Enterprise Linux powers the
Commvault HyperScale XTM platform.

Commvault® File System

02 Commvault’s own resilient and horizontally
scalable File System. CVFS

KVM
03 KVM package, qemu-kvm, is used to serve
CommServe® VMs for HyperScale XTM Appliances.

Commvault® Backup and Recovery

04 Comprehensive and modern Data Protection Software
capabilities with latest updates.

© Commvault 2023
Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

• HyperScale XTM hardware for both Appliances and Reference Architecture feature identical
technology stacks with one differentiating factor between them. The technology stack works in
concert with one another to deliver a smooth and streamlined experience, quickly, securely, and
efficiently. So let’s get into it starting with.

• The operating system. Commvault® uses Red Hat Enterprise Linux as it’s choice of operating
system. In this role, RHEL will allow Commvault® applications to engage the network, memory, and
computer power harnessed in the hardware. RHEL also features built in security minded tools, like
SELinux, to name one. As RHEL is a command line-based operating system, knowing how to get
around Linux via the command line and understanding syntax use for commands, is highly
recommended.

• Next, we have the underlying file system for the storage within, our own, Commvault® File System.
The Commvault® File System is how we administer and deliver the resiliency and scalable solution
that is offered with HyperScale XTM, but more on that later.

• A hypervisor is loaded onto each of the HyperScale XTM nodes. While this is loaded on all
HyperScale XTM servers, only HyperScale XTM Appliances can support a CommServe® server
running on their hardware. Currently, that is the differentiating factor between HyperScale XTM
Appliance and Reference Architecture. This is a barebones way of managing the CommServe®
server VMs that can be deployed with Appliances similar to any other hypervisor. Interactions with
KVM directly, are generally minimal but all necessary virsh commands to interact with KVM, are
available on our documentation website.

Naturally and lastly, we have our own data protection software loaded onto each HyperScale XTM node --
as each node perform roles of a Media Agent. They have the required software loaded onto each of them to
be used in this way.

6
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Commvault File System (CVFS)

Commvault ®
File System
(CVFS)
The File system
01 02 03
Commvault® File System:
Physical disks in the logical disk • Distinguishes how
HyperScale XTM are exposed to data is organized in
layered with high HyperScale XTM nodes HyperScale XTM for
performance, XFS is formatted with the efficiency and
Commvault® File resiliency with
CVFS erasure coding.
System

© Commvault 2023
Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

• Now let’s talk about file systems. File systems help arrange files in disk storage and manage
attributes like file name, file size, creation date information.

• All the physical disks in HyperScale XTM are layered with high performance, X F S.

• On top of that, the logical disk exposed to HyperScale XTM nodes is formatted with the
Commvault® File System.

• The Commvault® File System is our own distributed storage file system which provides massive
scalability and resiliency options.

• It distinguishes how data is organized in HyperScale XTM for efficiency and resiliency with erasure
coding.

• This also means that Commvault® Support can directly troubleshoot, diagnose, and accelerate
solutions for related incidents.

7
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

CVFS Storage Architecture Overview

CVFS Storage
Architecture
Overview
HyperScale XTM with CVFS Storage Layer

• Storage Proxy and • Storage proxy • Single Virtual disk in

Storage services communicates to the the cluster uses
live on the same backend storage services Erasure Coded 4+2 or
node and the MediaAgent 8+4 for a highly resilient
filesystem across the
nodes
CVFS Storage CVFS Storage Commvault
Proxy Node MediaAgent

© Commvault 2023
Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

• HyperScale X’s storage architecture is comprised of three moving pieces, two of which are part of
the Commvault® File System and the other being Commvault’s MediaAgent software. All three
work in concert with one another to provide a single, all-in-one package for simplicity as well as
performance.

• The storage proxy on each node communicates to its own MediaAgent, and all storage nodes in the
cluster, to ensure the best writes and reads in the cluster with low latency using round robin
algorithms.

• The single virtual Disk in the cluster is protected with Erasure Coding 4+2 or 8+4, dispersing data
to different disks per each chunk of data to ensure protection across the nodes.

8
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Commvault File System Proxy Overview

Commvault® File System Proxy Overview

Requests
(read, write, delete)
The Storage Proxy runs on each HyperScale
node

The Proxy is a lightweight software component

that provides intelligent NFS access to the
HyperScale nodes, directing I/O requests based
on Round Robin to every node in the cluster

CVFS
The Storage Proxy acts as a gatekeeper for all
I/O requests to the HyperScale Storage Nodes

The Storage Proxy component load balances I/O

requests to the backend storage nodes and Node 1 Node 2 Node 3
helps optimize storage performance
HyperScale XTM Logical Storage Layer (Storage Proxy)

© Commvault 2023
Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

The CVFS Proxy is the acting gatekeeper for all I/O requests to all HyperScale X TM Storage nodes in a
cluster.

• This service is what provides NFS access to the virtual disk to each individual node and controls the
load balancing processes native to our scale-out file system.

• It is a crucial piece to ensure that data is distributed optimally on all disks that are part of the cluster.

9
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Commvault File System Server Architecture Overview

Commvault® File System Server Architecture

overview
Core concepts and components (Storage Node)

Pages metadata processes

• Fault tolerance across all processes
• Ensures sharded, replicated persistence of all
metadata
• Tracks the state of all other HBlock and Pages
processes in the CVFS Cluster
• Responsible for optimal replica assignment and
tracks all writes in the cluster

Hblock write transactions

• Receives incoming write blocks
• Erasure codes and forwards to appropriate replicas
• Writes incoming blocks to memory and high-speed commit log

© Commvault 2023
Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

CVFS Storage Node Processes

The Commvault® File System is composed of several services to ensure fault tolerance. Each component
plays a critical role to the write and read operations. We're focusing here on two of the main components to
help explain the storage node process.

The pages process on each node, talk to each other to form a global distributed view of the cluster.

The Hblock process is responsible for the replication and persistence of user data and optimal assignment of
containers to each of the disks.

10
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Resiliency

Resiliency

Primary driver when Avoids interruption to Data corruption scenarios Gated by several
choosing scalable the end user or are reduced with configurations on
storage business erasure coding HyperScale XTM

© Commvault 2023

Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

Resiliency is a concept that most should be familiar with prior to consuming this curriculum, but for
those that may be newer to the concept, one might ask what is the importance of resiliency? When
customers are selecting scalable storage solution, resiliency is the primary driver of what makes
them attractive. In the event of any hardware failure, whether that’s disks or something else on the
nodes themselves, scalable storage solutions are able to still run and provide their functionality with
little to no interruption to the end user or business. Data corruption scenarios are reduced just due
to the inherent nature of erasure coded resiliency offered with scalable storage solutions. There are
naturally, limits to how resilient a solution can be which in the case of HyperScale XTM, is gated by
several configurations available based on the number of nodes that are available at the time of
deployment.

11
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Using 20TB drives as 18TB

4+2 VS 8+4

Resiliency through Erasure Coding

Standard
01 Medium High
Requires 3+ Nodes Requires 6+ Nodes Requires 12+ Nodes
• All resiliency models
here, are landed on
during initial cluster
deployment
EC4:2 EC4:2 EC8:4
Resiliency Resiliency Resiliency

Any 2 Disks Any 2 Disks Any 4 Disks

02
OR OR OR
• discuss this with your
Any 1 Node
Commvault SE contact Any 2 Nodes Any 2 Nodes

©©Commvault
Commvault2023
2023
CommvaultProprietary
Commvault Proprietaryand
andConfidential
ConfidentialInformation
InformationInternal
Internaland
andPartner
PartnerUnder
UnderNDA
NDAUse
UseOnly
Only- -Do
DoNot
NotDistribute.
Distribute.

Notes:

• At the time of the creation of this module, the HyperScale X platform offers three different resiliency
modes that are dependent on the number of nodes at the time of the deployment.

• On the smallest and most common configuration, we have standard resiliency. This mode only
requires a minimum, three to five nodes to use and offers a common 4+2 erasure code that allows
the loss tolerance of up to any 2 disks on any single node or 1 entire node. When a node or pair of
disks fail, the Commvault® File System does not wait for the failed drive to be replaced to begin
healing. Instead, what happens is a proactive recovery process where the data that remains
available on the disks, is redistributed, using the free capacity within the cluster. This can help
reduce the recovery time that occurs after the hardware issue is addressed, by up to three days,
with no user intervention required beyond addressing the hardware issue. This configuration also
has a specific means on continuing to write data safely, during this state, but more on that later.
Data will remain readable in this state as well, but anything more than the any 2 disks on a single
node or any 1 node failing, not only stops writes from occurring, but also reads, and ultimately,
imposes a risk for data loss.

• A newer resiliency model that we now offer, is our medium spec, which requires at minimum, 6
nodes to be deployed simultaneously. In its current iteration, this mode still provides the same any 2
disk failure, but improves by supporting up to any 2 nodes going offline. In this medium spec, read
and write operations will work just as normal and in the backend, data will be redistributed to fulfill
the resiliency requirements when resources go offline, just the same as before. While this mode
does offer availability of data for up to two nodes failing, when this happens, the cluster will no
longer receive writes and will go exclusively into a read-only mode in order to avoid compromising
the integrity of any new incoming data.

• Lastly, we have our high resiliency mode which will require 12 nodes to be deployed initially and
simultaneously, in order to be able to use. This changes the erasure coding schema from the
standard 4+2, to 8+4 instead, meaning instead of having to write four data blocks with 2 parity
pieces, 8 are written instead with 4 parity pieces. This mode can tolerate up to any four disks or any

12
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

2 nodes going offline and still render the cluster secure with data intact, available for both read and
write.

• All customers that are interested in pursuing either the Medium or High resiliency models, should
understand two things.

o 1) All resiliency models here, are landed on during initial cluster deployment. So for
example, if a cluster starts off using a standard resiliency model, it cannot, later, convert to a
medium or high resiliency model as again, it can only be defined at the initial deployment of
the HyperScale XTM cluster.

o If a customer is interested in either of these two models, it is required to discuss this in

email at [email protected] to file additional information regarding the
configuration.

13
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Quiz

Quiz

The answer is standard resiliency as again,

this is a configuration that is the default for all
If a customer is interested in deployments that are below six nodes. If the
purchasing 5 nodes for an
initial deployment, which
customer purchases a sixth node, that is when
resiliency mode would fit they can qualify for a medium resiliency spec,
best for the user? instead during deployment.

© Commvault 2023

Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

• Pop quiz time. If a customer is interested in purchasing 5 nodes for an initial deployment, which
resiliency mode would fit best for the user?

• The answer is standard resiliency as again, this is a configuration that is the default for all
deployments that are below six nodes. If the customer purchases a sixth node, that is when they
can qualify for a medium resiliency spec, instead, during deployment.

14
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Standard Resiliency Mode

Notes:

• HyperScale XTM users that deploy in the standard resiliency model have an exclusive way of
ensuring that data redundancy is able to be maintained during disk or node failures. (CLICK) On
each of the HyperScale X nodes, a hidden space is available when the file system is unable to
complete any form of erasure coded writes. This hidden space is an intended stop gap that allows
writes to come in and continue to be secured by writing that data as RF3 data instead. When the
resources are then brought back online, the data that was written there temporarily, is sharded
through the standard erasure coded methods, in order to flush the temporary space out.

• It is important to consider the fact that this is a limited space as well, so even though it may provide
a safety net, it is intended to run for a short enough time to address a hardware issue.

• So what happens in a six node cluster with 4+2 erasure coding?

15
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

CVFS Healing

CVFS Healing
Maintain High Availability in the event of
concurrent hardware failures Node 1

Node 2
On a highly available system, data is most vulnerable during
01 recovery after a failure
Node 3

Disks are under heavy load, which increases the chances of an

02 additional failure Node 4

Node 5
HyperScale XTM maintains data redundancy even when cluster is
03 in a degraded state
Node 6

Ensures data is available should another failure

04 occur
Data fragment 1

Data fragment 2
© Commvault 2023
Data fragment 3 Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

• We mentioned earlier how CVFS does not wait for the hardware to be replaced to start healing, but
what does that look like in motion?

• Let’s say we have our six-node cluster here, that has received three different pieces of data.
Erasure coding in this 4+2 configuration dictates that the data will have to be split into four data
blocks and two parity pieces. If in this example, node 6 were to fail in this configuration, the healing
process begins and ensures that the data fragments that were originally on node 6, are
reconstructed into a brand-new set of those same three data fragments to be dispersed on the
remaining hardware. This process is further optimized through an automated load balancing
mechanism that is native to CVFS.

16
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Improved Performance, Scalability and Resiliency

Improved Performance,
Scalability, and Resiliency Node 1
Storage Expansion and Load Balancing Overview

Node 2

Scale cost-effectively in single node increments

01 after initial 3-node deployment Node 3

Node 4

During expansion, data is automatically redistributed

02 across all nodes to optimize redundancy and
Node 5
performance

Node 6
During normal operation, data is automatically
03 load balanced among nodes to balance
Data fragment 1
performance across disks
Data fragment 2
© Commvault 2023
Data fragment 3 Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

Commvault® File system periodically, will need to perform data load balancing. This automated load
balancing can occur when either nodes are added to the existing cluster or if a node happens to become
unbalanced, like say in the scenario we just covered. Load balancing streams the uneven data to a new
destination and once it’s successfully copied over to its new location, the source location is cleaned up and
then deleted. As more nodes are added, the data continues to be distributed across the cluster evenly, but
this is a low-performance process that has next to no impact to any users of the HyperScale XTM solution.

17
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

HyperScale XTM High Availability

HyperScale XTM High Availability

CommServe® Manual Site 01 - Primary Site 02 – Stand By

Placement CS
CSDB Transaction
Log Backup CS
CSDB Transaction
Log Restore

Strategies Automated Every 15 minutes

by default
Every 15 minutes
by default

© Commvault 2023

Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

• High availability for the CommServe® server should be in all design and deployment scenarios,
including those with HyperScale XTM. CommServe® Live Sync is one of the recommended methods
to ensure high availability. It offers both a manual and automated failover solution for a Primary
CommServe® server, to another standby CommServe® that is available within the LiveSync group.

• The location and placement strategy of the primary and standby CommServe® servers can vary by
environment. In smaller environments, they can reside on a HyperScale XTM appliance. However,
Customers may want to consider using Metallic® or placing their CommServe® servers off-host --
outside of the HyperScale XTM servers. This includes potentially deploying multiple standby
CommServe® servers in different geos to mitigate localized, disastrous events.

• We’ll cover more on this though in the next module.

18
COMMVAULT PROPRIETARY AND CONFIDENTIAL INFORMATION - INTERNAL AND PARTNER UNDER NDA USE ONLY- DO NOT DISTRIBUTE

Wrap Up

Wrap Up

1 2 3
Identified Technology Commvault® File System Scenario that illustrates the
components in the resiliency mechanism
HyperScale XTM solution

CVFS

© Commvault 2023
Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

Notes:

• In this module, we’ve gone ahead and identified each of the individual technology components in the
TM
HyperScale X solution. This included a more detailed focus on the Commvault ® File System
along with its individual moving components. We wrapped up with a hypothetical scenario that
illustrates the resiliency mechanism available to HyperScale X TM before setting the stage for our
next module, CommServe® Servers and HyperScale XTM.

19
COMMVAULT PROPRIETARY/CONFIDENTIAL – FOR COMMVAULT PARTNERS UNDER NDA USE – NOT TO BE FURTHER DISTRIBUTED

Thank You

© Commvault 2023

Commvault Proprietary and Confidential Information Internal and Partner Under NDA Use Only - Do Not Distribute.

commvault.com | 888.746.3849
© 2023 Commvault. See here for information about our trademarks and patents.