POWERFLEX RACK

ADMINISTRATION -
INSTRUCTOR GUIDE

INSTRUCTOR GUIDE

INSTRUCTOR GUIDE
Table of Contents

PowerFlex rack Administration......................................................................................... 1

PowerFlex rack Overview ................................................................................................ 2
PowerFlex rack ............................................................................................................... 3
The PowerFlex Family ..................................................................................................... 4
PowerFlex Family Comparison Table ............................................................................... 5
PowerFlex Appliance Sample Network Topology .............................................................. 7
PowerFlex rack Sample Network Topology ...................................................................... 8
PowerFlex rack Benefits .................................................................................................. 9
PowerFlex rack Use-Cases ........................................................................................... 11
PowerFlex rack Architecture .......................................................................................... 12
PowerFlex Rack Hardware Components ........................................................................ 13
PowerFlex Node ........................................................................................................... 16
Access Switch............................................................................................................... 17
Aggregation Switches .................................................................................................... 19
PowerFlex rack Management Controller ......................................................................... 20
Management Switch ...................................................................................................... 21
Spine-Leaf Architecture ................................................................................................. 22
PowerFlex rack Network Connectivity ............................................................................ 24
PowerFlex Rack Sample Network Topology ................................................................... 28
PowerFlex rack Configurations ...................................................................................... 29
PowerFlex Architecture Basics....................................................................................... 30
Storage Data Server (SDS) ........................................................................................... 31
Storage Data Client (SDC)............................................................................................. 32
Metadata Manager (MDM) ............................................................................................. 33
MDM Cluster ................................................................................................................. 34
Nodes ........................................................................................................................... 37
Component Traffic Types .............................................................................................. 39
PowerFlex Data Workflow.............................................................................................. 42
Write I/O ....................................................................................................................... 44
Reads ........................................................................................................................... 45
PowerFlex rack Deployment Options and Management Tools ......................................... 46

PowerFlex rack Administration - Instructor Guide

Page ii © Copyright 2020 Dell Inc.

 PowerFlex Family Deployment Options .......................................................................... 47
Management Interfaces Overview .................................................................................. 49
PowerFlex Manager ...................................................................................................... 51
Resource ...................................................................................................................... 52
Template ...................................................................................................................... 53
Service ......................................................................................................................... 54
Compliance................................................................................................................... 55
Repository .................................................................................................................... 56
Integrated Dell Remote Access Controller (iDRAC)......................................................... 57
PowerFlex Manager User Interface Orientation .............................................................. 58
Networking and Switch Management ............................................................................. 59
Remote Connectors ...................................................................................................... 60
Storage Resource Management .................................................................................... 62
PowerFlex rack Storage Constructs ............................................................................... 63
Protection Domain Benefits ........................................................................................... 66
Storage Pool Benefits .................................................................................................... 67
Zero-Padding ................................................................................................................ 68
Read Flash Cache Acceleration Pool .......................................................................... 69
NVDIMM Acceleration Pool ........................................................................................... 70
Service Deployment Workflow Using PowerFlex Manager .............................................. 71
Presentation Server Deployment ................................................................................... 73
Service Deployment Procedure...................................................................................... 74
Adding Storage Constructs Using PowerFlex Manager ................................................... 78
Creating a Protection Domain in PowerFlex Web UI - Hands On ..................................... 80
Creating a Storage Pool in PowerFlex Web UI - Hands On ............................................. 81
Storage Pool Granularity ............................................................................................... 82
Fine Granularity Storage Pool ........................................................................................ 86
Creating an Acceleration Pool in PowerFlex Web UI - Hands On..................................... 91
Administrating Volumes ................................................................................................. 92
Compute Resource Management................................................................................... 95
Compute Resources in PowerFlex rack .......................................................................... 96
VMware vCenter for Compute Resource Management ................................................... 97
VMware vSphere Environments for PowerFlex rack........................................................ 98

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page iii

 PowerFlex Management Controller Cluster .................................................................... 99
PowerFlex Node Cluster ...............................................................................................101
ESXi Boot Device .........................................................................................................103
Provisioning Storage for Production Virtual Machines ....................................................104
Create Datastore on PowerFlex Volume........................................................................105
Virtual Machine Deployment Options ............................................................................106
Create VM Example .....................................................................................................108
Add Storage to VM .......................................................................................................109
Virtual Machine Management .......................................................................................110
VM Migration Using vSphere VMotion ...........................................................................112
Migration Wizard ..........................................................................................................114
Red Hat Virtualization Manager for Compute Resource Management.............................115
Network Resource Management ...................................................................................116
Network Types .............................................................................................................117
Verifying Connectivity ...................................................................................................119
Checking the Maximum Transmission Unit (MTU) .........................................................121
Adding a Network to a Service ......................................................................................122
Adding a VLAN to Access Switches ..............................................................................123
Access Switch Port Configurations................................................................................124
Logical Network V3 Support .........................................................................................126
NS-OX Query Commands ............................................................................................127
Logical Networking .......................................................................................................128
vSphere Network Topology ...........................................................................................129
ESXi Host Network Interface.........................................................................................130
Adding a Production Network........................................................................................131
Security and User Management ....................................................................................135
User Roles and Authentication......................................................................................136
Managing System Parameters ......................................................................................143
Data Protection Options................................................................................................153
Rebuild and Rebalance ................................................................................................154
Snapshot and Snapshot Policy .....................................................................................160
Native Asynchronous Replication ..................................................................................164
Maintenance Modes .....................................................................................................172
Protected Maintenance Mode Using PowerFlex Manager ..............................................175

PowerFlex rack Administration - Instructor Guide

Page iv © Copyright 2020 Dell Inc.

 CloudLink Integration ...................................................................................................177
VMware Data Protection Options ..................................................................................180
PowerFlex rack-Integrated Data Protection ...................................................................184
Monitoring and Troubleshooting ....................................................................................187
Monitoring with PowerFlex Manager .............................................................................188
Virtual Environment Monitoring .....................................................................................201
PowerFlex Monitoring ...................................................................................................208
Hardware Monitoring ....................................................................................................215
Troubleshooting PowerFlex rack ...................................................................................221
You Have Completed This eLearning. ...........................................................................232

Glossary ................................................................................................ 233

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page v

 Error! No text of specified style in document.

PowerFlex rack Administration

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 1

Error! No text of specified style in document.

PowerFlex rack Overview

PowerFlex rack Administration - Instructor Guide

Page 2 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex rack

PowerFlex rack is a rack-scale engineered system with integrated

networking that provides linear scalability and enterprise-grade
availability. PowerFlex rack is engineered, manufactured,
managed, supported, and sustained as one system for single end-
to-end life-cycle support.

PowerFlex rack solution provides:

• Fully integrated rack-scale fabric
• Complete life cycle management and orchestration
• Simplifies deployment and upgrade procedures
• Keeps system stabilized and optimized

Watch the PowerFlex overview video below and learn how the PowerFlex rack
solution delivers agility for the modern data center.

If you have trouble watching the video below, click here.

Movie:

The web version of this content contains a movie.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 3

Error! No text of specified style in document.

The PowerFlex Family

The PowerFlex family includes VxFlex Ready Node, PowerFlex appliance, and
PowerFlex rack. The fundamental building block is PowerFlex, a software-defined
storage service. PowerFlex enables customers to create scale-out infrastructure
based on Hyper Converged architecture.

Click here to review the PowerFlex family comparison at a glance.

Scale-out block storage service that enables customers to create a scale-out Server SAN or HCI infrastructure.

Simplified, smaller scale engineered

Scalable, reliable and highly solution with economic form factor
A fully
configurable HCI building integrated and
most flexible
block
hyperconverg
ed rack-scale
system
PowerFlex Web UI, iDRAC, manage PowerFlex Web UI, iDRAC, and PowerFlex PowerFlex Web UI, iDRAC, and PowerFlex Manager
Manager (required and included)
and monitor node clusters Comprehensive ITOM software

Comprehensive ITOM software

The Dell EMC HCI portfolio offers a wide range of solutions to meet
customer needs. These HCI solutions are preengineered and tested to
provide a turnkey experience. To learn more about the Dell EMC HCI
portfolio offerings, see https://www.dellemc.com/hi-in/converged-
infrastructure/hyper-converged-infrastructure.htm

PowerFlex rack Administration - Instructor Guide

Page 4 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Family Comparison Table

VxFlex Ready Node PowerFlex PowerFlex rack

appliance

Experience Build Turnkey Fully engineered

Networking Bring your own Bring your own Integrated

Sample network Sample network
topology topology

Bare Metal Yes Yes Yes

Support

Management PowerFlex Web UI, PowerFlex PowerFlex

iDRAC Manager: Manager:
Monitoring, Monitoring,
Alerting, Alerting, Lifecycle
Lifecycle Management,
Management, IC RCM Compliance
Compliance

Lifecycle Element Managers, PowerFlex PowerFlex

Management API, CLI Manager, IC Manager, RCM

Initial Services optional Services Services required:

Deployment required: Pre-built, pre-
ProDeploy Plus integrated at
factory, white-
glove service with
onsite services

Upgrades Services optional Services Recommended to

Optional use PS led
upgrade support

Transaction 5-10 business days 10-15 business 45-60 business

Time days days

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 5

Error! No text of specified style in document.

Expansion Add drives and memory Add per node Add expansion
nodes/rack

Expansion Customer ProDeploy ProDeploy

Service deployable/optional services (to services
services come)

PowerFlex rack Administration - Instructor Guide

Page 6 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Appliance Sample Network Topology

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 7

Error! No text of specified style in document.

PowerFlex rack Sample Network Topology

PowerFlex rack Administration - Instructor Guide

Page 8 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex rack Benefits

Review how PowerFlex rack benefits the customers.

2
3

1: Simple and effective operations:

PowerFlex offers a robust toolset for simplifying IT operations for the entire
infrastructure. PowerFlex also supports standards-based open APIs, making it a
breeze to integrate with third-party tools and custom workflows. PowerFlex
Manager offers tools for IT operations and life cycle management that automate
infrastructure workflows, boosting IT efficiency and agility while simplifying
compliance. PowerFlex Manager provides self-healing, self-balancing architecture,
which automatically rebalances resources while mitigating the impact of failures,
and eliminating common storage management overhead. PowerFlex rack offers

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 9

Error! No text of specified style in document.

integrated networking with professional deployment, simplifying deployment

operations.

2: Unmatched flexibility and scale:

PowerFlex allows you to deploy and scale on your terms. It offers the flexibility to
choose your deployment architecture, whether Two Layer (Server SAN), HCI,
storage only or mixed, in a single scalable deployment. It supports a broad range of
operating environments – bare metal operating system, hypervisors, and container
management tools - on a single platform. This feature enables you to choose the
environment that best meets the application requirements and consolidate mixed-
application workloads on shared infrastructure. PowerFlex can scale from a few
nodes to hundreds, on-demand and non disruptively, with performance and
throughput scaling linearly. You can scale by adding storage, HCI, or compute
nodes, flexibly, to meet your exacting requirements. PowerFlex also offers flexible
consumption options, with rack, appliance, and Ready Nodes.

3: Enterprise-class performance:
PowerFlex delivers massive performance with an architecture that pools large sets
of resources and eliminates bottlenecks in the system, delivering millions of IOPs at
sub millisecond latency. Dell Technologies lab testing demonstrates that PowerFlex
delivers leading-edge performance for enterprise applications, including high-
performance databases, big-data analytics, and AI or ML workloads. PowerFlex
also delivers mission-critical Six 9’s (99.9999%) availability for your high-value
workloads with multiple protection groups and fast rebuilds.

PowerFlex rack Administration - Instructor Guide

Page 10 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex rack Use-Cases

Here are some key PowerFlex rack use cases for different scenarios.

Click below to read real-time customer stories of PowerFlex rack and

learn how software-defined storage saves millions and better serves
communities.
• CUSTOMER STORY: FINANCIAL SERVICES

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 11

Error! No text of specified style in document.

PowerFlex rack Architecture

PowerFlex rack Administration - Instructor Guide

Page 12 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Rack Hardware Components

A PowerFlex rack consists of PowerFlex nodes, controller nodes, access switches,

out-of-band management switch, and aggregation switches. The switches then
connect through the customer core uplink to customer network.

Explore each rack components and links within the description to understand its
functionality.

6
5

7
2

3
8

1:

PowerFlex nodes use Dell PowerEdge servers. These nodes are clustered
together to provide the computing power for production virtual machines and bare
metal workloads.
Click here to learn more about PowerFlex node types.

2:

The nodes communicate to each other using a pair of access switches. In a multi
rack configuration, each rack requires a pair of access switches.
Click here to learn more about the access switch function and configurations that
are used in PowerFlex rack.

3:

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 13

Error! No text of specified style in document.

A pair of aggregation switches is used for inter-rack and customer core

communications.
Click here to learn more about how aggregation switches provide scalability by
aggregating the leaf access switches.

4:

A minimum of three nodes are clustered to provide the PowerFlex Management

Controller. They are clustered separately from the production PowerFlex nodes,
and run various management and service VMs.
Click here to learn more about PowerFlex Controller nodes.

5:

A single management switch provides connections for out-of-band management

traffic, such as iDRAC. In a multirack configuration, more management switches
can be added based on port count requirements.

6:

A single management switch provides connections for out-of-band management

traffic, such as iDRAC. In a multirack configuration, more management switches
can be added based on port count requirements.

7:

The nodes communicate to each other using a pair of access switches. In a

multirack configuration, each rack requires a pair of access switches.
Click here to learn more about the access switch function and configurations that
are used in PowerFlex rack.

8:

A pair of aggregation switches is used for interrack and customer core

communications.
Click here to learn more about how aggregation switches provide scalability by
aggregating the leaf access switches.

9:

PowerFlex nodes use Dell PowerEdge servers. These nodes are clustered
together to provide the computing power for production virtual machines and bare-

PowerFlex rack Administration - Instructor Guide

Page 14 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

metal workloads.
Click here to learn more about PowerFlex node types.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 15

Error! No text of specified style in document.

PowerFlex Node

All types of nodes are available in various configurations that are based on
PowerEdge R640, R740xd, and R840. Refer to Dell EMC website for detail
specifications.

PowerFlex rack offers flexible node types:

• Hyperconverged nodes run a CentOS kernel Embedded OS hypervisor and
provide both compute and storage resources for production virtual machines.
• Storage-only nodes are used to provide storage capacity. Storage-only nodes
run a lightweight CentOS kernel embedded operating system. Storage capacity
from these nodes is added to the overall storage pool.
• Compute-only nodes come with a large amount of CPU and memory but less
storage, typically minimum two drives. They provide computing power, but do
not contribute any storage to the PowerFlex storage pool.

Dell PowerEdge R640 Server

Dell PowerEdge R740xd Server

Dell PowerEdge R840 Server

PowerFlex rack Administration - Instructor Guide

Page 16 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Access Switch

Dell EMC Networking S5248F-ON

Cisco Nexus 93180YC-EX

Cisco Nexus 93240YC-FX2

Access Switch provide communication between the PowerFlex nodes in intra-rack

and between PowerFlex and controller nodes in inter-rack system. PowerFlex rack
uses a pair of access switches to carry traffic in PowerFlex rack. Access switches
carry Production traffic, which includes communication between the PowerFlex
nodes and data traffic to the PowerFlex storage and to systems or networks
outside PowerFlex rack. Besides production traffic, access switches also carry
VMware vSphere and management traffic between controller nodes and PowerFlex
nodes.

PowerFlex rack offers Dell EMC PowerSwitch S5248F-ON or Cisco Nexus

93180YC-EX as the access switch options. Both switches are one rack unit, and
provides 48, 10/25GbE SFP28 ports for PowerFlex node connectivity. They also
provide six 100 Gbps QSFP. But the PowerFlex rack supports only the 25GbE
downlink switch port.

Note: Cisco Nexus N93240YC-FX2 is an additional access switch option available.

Dell EMC S5248F-ON provides four 100GbE QSFP28, and two 100GbE QSFP-DD
(quad small form-factor pluggable double density) ports for uplink and inter-switch
connectivity. But the PowerFlex rack offers only the 25GbE switch port.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 17

Error! No text of specified style in document.

Another option for the access switch is Cisco Nexus 93240YC-FX2. It is a 1.2-RU
switch designed for spine-leaf deployment in data centers. This switch has 48 x
1/10/25 Gbps SFP28 ports and 12 x 40/100 Gbps QSFP28 ports.

Note: Older models of PowerFlex rack use a pair of 1-RU Cisco Nexus 3132Q-X or
2U 3164Q access switches. These switches provide 32 and 64, 40 Gbps QSFP+
ports respectively. They use 4x10 Gbps breakout cables to connect to the
PowerFlex nodes and Controller nodes.

PowerFlex rack Administration - Instructor Guide

Page 18 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Aggregation Switches

PowerFlex rack system requires a pair of Dell EMC PowerSwitch S5232F-ON or

Cisco Nexus 9236C switches to aggregate the access switches from each rack.
Both switches provide 36 QSFP28 ports of 40/100-Gbps connectivity in a one rack
unit form factor. Only aggregation switches connect to customer core network.

Another option for the aggregation switch is Cisco Nexus 9336C-FX2.

As the number of nodes and cabinets grow, more access switches can be added to
the aggregation switch.

• Dual uplink connection from access to aggregation switches can connect up to

7 pairs of access switches to aggregation switch pair. This is the default setting:
− Maximum nodes allowed - 7 x24 =168 Nodes (3 Controller + 165 FLEX
Nodes)
• Access Switches with Single Uplink – Connects upto 14 pair of aggregation
Switches (This is a less preferred and less recommended method of access to
aggregation switch connection).

− Maximum nodes allowed - 14 x24 =336 Nodes (3 Controller + 333 FLEX

Nodes)

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 19

Error! No text of specified style in document.

PowerFlex rack Management Controller

PowerFlex controller nodes use the PowerEdge R640 purpose-built for

management control plane. The controller nodes run VMware ESXi.

There are two type of management controller nodes: small and large. The small
controller nodes come with single socket, 20 cores, 192GB RAM and 9.6 TB
storage capacity. The large controller node comes with dual socket, 40 cores,
384GB RAM and 15.36TB storage capacity.

The drive count for small controller node is 5x1.92T SSD and for large node is
8x1.92TB SSD.

The small controller node runs all management and orchestration software needed
for a large deployment up to 200 nodes. A management control plane with large
controller node is recommended for deployment of more than 200 nodes in the
cluster or when additional management components such as VMware vROPs
needs to be added to the cluster.

Similar to the PowerFlex nodes, the controller node drives are used as storage for
the virtual machines running on them. However, this storage pool is separate from
the production storage and uses a VMware vSAN software defined storage
solution. VMware vSAN provides high availability cluster for the management
controller.

PowerEdge R640 8-drive Servers

PowerFlex rack Administration - Instructor Guide

Page 20 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Management Switch

The PowerFlex rack uses Dell EMC PowerSwitch S4148T-ON or Cisco Nexus
31108TC-V management switch.

Each node and switch have a copper management network interface which
connects to the management switch. It is a one rack unit (1U) switch that provides
48 x 1/10 Gbps RJ45 ports and 6 x 40/100 Gbps QSFP+ ports.

Note: Older systems use Cisco Nexus 3172TQ as the management switch.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 21

Error! No text of specified style in document.

Spine-Leaf Architecture

For greater scalability, multipath redundancy, and high throughput, PowerFlex rack
cluster are configured with spine-leaf architecture. In this architecture, every leaf
switch (Cisco Nexus 93240YC-FX2) is replaced for an access switch and is 1.2U
in size and have 12 more SPF ports than the access switches. The spine
switches (Cisco Nexus 9336C-FX2) are replaced for the aggregation switches in
the network. The leaf switches all connect to spine switches in a full-mesh
topology.

The spine switches then connect to two to four border leaf switches (Cisco Nexus
9336C-FX2). Uplink to the customer core network is provided through border leaf
switches.

PowerFlex rack Administration - Instructor Guide

Page 22 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

With the spine-leaf architecture, Layer 3 gateways are available at leaf switches.
This distributed gateway enables VM migration seamlessly between the racks.
PowerFlex supports three or six spine switches. With six spine switches, the
maximum nodes that are allowed are 348 (three controller nodes + 381 FLEX
nodes). With six spine switches, there is no oversubscription on the switches and
no performance degradation, even when every node is using its network with full
capacity.

Leaf-Spine architecture switches - Connects up to 16 pairs of Cisco Nexus

93240C-FX2 leaf switches

• Maximum nodes allowed- 16 x24 =384 Nodes (three Controller + 381 FLEX
Nodes)

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 23

Error! No text of specified style in document.

PowerFlex rack Network Connectivity

PowerFlex Node Connectivity

The access switches connect to the PowerFlex nodes with 25 Gbps SFP+ cables.

In a hyperconverged deployment, each node has four connections to the access

switches (two for each switch). Two of these connections (one from each switch)
are aggregated and they carry hypervisor management and customer production
traffic. The remaining two connections (one from each switch) are dedicated for
PowerFlex data traffic. If the cables from each switch connect to different network
cards on the PowerFlex rack node to provide redundancy, a switch or network card
fails.

In two layer deployments, each storage-only node has four connections to each
access switch. One connection is dedicated to the back-end storage traffic.

Multiple VLANs are used to separate the different types of traffic on the access
switches. VLANs keep the production, vSphere, vMotion, PowerFlex, and vSAN
traffic segregated. Dell EMC PowerEdge R640/R740xd/R840 servers are used as
the PowerFlex nodes.

Management and
Production Data

Management and
Production Data

PowerFlex rack Administration - Instructor Guide

Page 24 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Controller Node Connectivity

The access switches also connect to each Controller node with 10 Gbps
connection. Each switch has two connections to each Controller node. One
connection is for management traffic and the other is for storage services such as
vSAN and vMotion. As with the PowerFlex nodes, each switch connects to two
different NICs on the Controller node.

A pair of Cisco Nexus 93180YC-EX or Dell EMC Networking S5248F-ON are used
as the access switches in the rack.

Dell EMC PowerEdge R640 servers are used as the controller nodes.

Out-of-Band Management Connectivity

The management switch provides connectivity for out-of-band management traffic.

It has 1 Gbps connections to the iDRAC ports of every PowerFlex node and
controller node. It also connects to the management ports of all switches.

In addition to the iDRAC connection, each controller node has a second 1 Gbps
connection into the management switch. These secondary connections allow the
jump server and PowerFlex Manager (management interface) to have access to
the out-of-band network to perform management of the components.

The management switch also connects to the customer network. This allows the
customer to have out-of-band network access to the components of the PowerFlex
rack.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 25

Error! No text of specified style in document.

One Cisco Nexus 31108TC-V switch or a Dell EMC Networking S4148T-ON is

used as the management switch.

Interrack Connectivity

Multiple racks are connected using the aggregation switches. Even if a PowerFlex
rack cluster has only one rack, it often will have a pair of aggregation switches to
allow for easier expansion deployments. A pair of aggregation switches connect to
pairs of access switches in each rack.

Each access switch has a 40 Gbps fiber optic connection to each aggregation
switch. There are also two peer connections between the aggregation switches.

The aggregation switches also connect into the customer network. This uplink
carries all the production traffic to and from the PowerFlex rack.

A pair of Dell EMC Networking S5232F-ON or Cisco Nexus 9336C-FX2 are used
as the aggregation switches.

PowerFlex rack Administration - Instructor Guide

Page 26 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 27

Error! No text of specified style in document.

PowerFlex Rack Sample Network Topology

PowerFlex rack Administration - Instructor Guide

Page 28 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex rack Configurations

PowerFlex rack configurations are categorized into single and multiple rack
configurations.

Play the video below to learn more.

If you have trouble watching the video below, click here.

Note: PowerFlex rack is shipping with IPI Gen 3.0.

Movie:

The web version of this content contains a movie.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 29

Error! No text of specified style in document.

PowerFlex Architecture Basics

PowerFlex uses existing local storage devices (Direct Attached Storage) and turns
them into shared block storage. The shared block storage is available as software-
defined storage to the applications.

Metadata Metadata

Storage Media
Logical Volumes

Applications

Software-defined Storage

Virtualized Storage Mapped Storage

The major components in a PowerFlex Cluster are:

• Three base pieces of software:

− Storage Data Server (SDS)

− Storage Data Client (SDC)
− Meta Data Manager (MDM)
• Meta Data Manager Clusters Overview
• Hardware:

− Nodes
• Component Traffic Types

PowerFlex rack Administration - Instructor Guide

Page 30 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Storage Data Server (SDS)

The Storage Data Server (SDS) is a software daemon that enables a server in the
cluster to contribute its local storage to the aggregated storage pool. It owns the
contributing devices and together with the other SDS forms a protected mesh from
which storage pools are created.

An instance of the SDS runs on every server that contributes some or all its local
storage space (SSDs, or NVMe devices). The SDS manages the capacity of a
single server. The SDS performs the requested SDC back-end I/O operations, and
MDM rebuild and rebalance operations.

Storage-only Nodes
TCP/IP

Storage Media
Server Logical Volumes
Block Device Drivers

TCP/IP

Server

Block Device Drivers

Software-defined Storage

Virtualized Storage Read Operation

Mapped Storage

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 31

Error! No text of specified style in document.

Storage Data Client (SDC)

The Storage Data Client (SDC) is a lightweight block device driver that exposes
PowerFlex shared block volumes to the operating system. The SDC runs on the
same server as the application. The SDC enables the application to issue an I/O
request. The SDC fulfills the I/O request regardless of where the particular blocks
physically reside.

The SDC communicates with other nodes (beyond its own local server) over
TCP/IP-based protocol. The only I/O in the stack that the SDC intercepts are the
I/O that are directed at the volumes that belong to PowerFlex.

Storage Media
Logical
Volumes
Compute-only Node
TCP/I
P

Client

Volume Manager

File Manager

Software-defined Storage

Applications

Vitualized Storage Write Operation

Mapped Storage

PowerFlex rack Administration - Instructor Guide

Page 32 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Metadata Manager (MDM)

Metadata Metadata

Storage-only Nodes

Compute-only Node
Server
Read Write
Block Device Drivers Operatio Operatio

Client
Software-defined Storage Volume Manager

File Manager
Mapped Mapped
Storage Storage
Server
Block Device Drivers

Applications

The MDM configures and monitors the PowerFlex system. It contains all the
metadata that is required for system operation. Although the MDM is responsible
for data migration, rebuilds, and all system-related functions, the user data never
passes through MDM.

The MDM hands out instructions to each SDC and SDS about its role and how to
play it. The MDM gives each component all the information it needs but nothing
more.

The number of MDM entities can grow with the number of nodes. Three or five
instances of the MDM run on different servers to support high availability.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 33

Error! No text of specified style in document.

MDM Cluster

The MDM cluster consists of a combination of Master MDM, Slave MDMs, and Tie-
breaker MDMs. There is also the Standby MDM which is not a part of the cluster.

Note: For a 5 or more PowerFlex node system, 5 MDM cluster is the default for
PowerFlex rack.

MDM

MDM is a daemon service that runs on any PowerFlex node. An MDM is assigned
a Master, Slave or a Tie-breaker role, during installation. At minimum 3 MDMs; 1
Master, 1 Slave, and 1 Tie-breaker MDMs are needed to make an MDM cluster.

MDM has a unique MDM ID and can be given unique names. Before the MDM can
be part of the cluster, it must be promoted to a Standby MDM first. A standby MDM
must be manually activated in order to be part of an active cluster when Master or
Slave MDM goes down. There is always an odd number of MDMs in a cluster such
as 3 or 5.

Node Node Node Node Node Node

5-Node MDM Cluster

Master MDM

The Master MDM functions like a brain and controls all the SDCs, SDSs, and SDRs
in the cluster. In an MDM cluster, only one MDM can be a primary at a given time.
The Master MDM contains and updates the MDM repository, the database that
stores SDS configurations, and how data is distributed between the SDSs.

PowerFlex rack Administration - Instructor Guide

Page 34 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Nod Nod Nod Nod Nod Nod

Master

5-Node MDM Cluster

Slave MDM

The Slave MDM is an MDM in the cluster that is ready to take over as the Master
MDM. In a 3-node cluster, there is one Slave MDM, thus allowing a single point of
failure. If you have 5 nodes or more the default is a 5 node MDM cluster.

Node Node Node Node Node Node

Master Slave Slave

5-Node MDM Cluster

Tie-breaker MDM

A Tie-breaker MDM is the MDM that determines which MDM becomes the Master
MDM. It helps in maintaining quorum in the cluster. In a 5-node MDM cluster, two
MDMs will have a tie-breaker role.

Node Node Node Node Node Node

Master Slave Slave Tie-breaker 1 Tie-breaker 2

5-Node MDM Cluster

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 35

Error! No text of specified style in document.

Standby MDM

Once a MDM is added as a Standby MDM, it will be added or locked for the
specific system. When promoted to a cluster member, it can be considered as a
Slave or Tie-breaker MDM.

Node Node Node Node Node Node

Tie-breaker 1 Stand-by Up to 8
Master Slave Slave Tie-breaker 2
stand-by
5-Node MDM Cluster MDMs

Manager MDM

Metadata managers (MDMs) control the behavior of the PowerFlex system. They
determine and publish the mapping between clients and their data, keep track of
the state of the system, and issue reconstruct directives to SDS components. A
Manager MDM acts as a Master or Slave in the cluster. Manager MDMs have a
unique system ID and can be given unique names. A manager can be a standby or
a member of the cluster.

Node Node Node Node Node Node

Slave Tie-breaker 1 Up to 8
Master Slave Tie-breaker 2 Stand-by
stand-by
MDMs
Manager 5-Node MDM Cluster

PowerFlex rack Administration - Instructor Guide

Page 36 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Nodes

PowerEdge R640

PowerEdge R740xd

PowerEdge R840

PowerFlex Nodes are the basic hardware units, which are used to install and run
the hypervisor and the PowerFlex system.

Dell EMC VxFlex Ready Nodes, PowerFlex appliance, and PowerFlex rack bring
together Dell EMC PowerEdge servers and Dell EMC PowerFlex software. The
combination is done in a reliable, quick, and easy to deploy building block. These
building blocks are ideal for server SAN, heterogeneous virtualized environments,
and high-performance databases.

PowerFlex rack nodes offers the following three types:

• Hyperconvered (HC) Node - When both PowerFlex SDS and SDC run in the
same PowerFlex node chassis, it is called HC node.
• Compute Only (CO) Node - When only PowerFlex SDC runs in a PowerFlex
node chassis it is called CO node. CO nodes are compute-heavy and have little
to no storage capacity.
• Storage Only (SO) Node- A SO PowerFlex node runs only SDS and it is
storage-heavy with small computing power (CPU and memory)

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 37

Error! No text of specified style in document.

The variants of PowerFlex nodes that customers use are:

• PowerEdge R640 is a node that has a single rack unit.
− Supports 8 - 10 drives per node (minimum of 5 disks is required)
• PowerEdge R740xd is a node that has two rack units.
− Fully populated 24 drives, other partially populated nodes come with 8, 10,
16 drives
• PowerFlex nodes based on PowerEdge R840 are dual-socket and compute-
compute heavy. The majority of these nodes are either CO or HC. There is no
SO node configuration in the R840 chassis.

− Supports 24 drives per node (partially populated configuration comes with 8,

10, or 16 drives).

PowerFlex rack Administration - Instructor Guide

Page 38 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Component Traffic Types

The software components that makeup PowerFlex (the SDCs, SDSs, and MDMs)
converse with each other in predictable ways. For designing a PowerFlex
deployment, you should be aware of these traffic patterns in order to make choices
about the network layout.

Note: The front-end and back-end storage traffic is a logical distinction and does
not require physically distinct networks.

Storage Data Client (SDC) to Storage Data Server (SDS)

Traffic between the SDCs and the SDSs forms the bulk of front-end storage traffic.
Front-end storage traffic includes all read and write traffic arriving at or originating
from a client. This network has a high throughput requirement.

Storage Data Server (SDS) to Storage Data Server (SDS)

Traffic between SDSs forms the bulk of back-end storage traffic. Back-end storage
traffic includes writes that are mirrored between SDSs, rebalance traffic, rebuild
traffic, and volume migration traffic. This network has a high throughput
requirement.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 39

Error! No text of specified style in document.

Meta Data Manager (MDM) to Meta Data Manager (MDM)

MDMs are used to coordinate operations inside the cluster. Directs PowerFlex to
manage rebalance, rebuild, and redirect traffic. They also coordinate Replication
Consistency Groups, determine replication journal interval closures, and maintain
metadata synchronization with PowerFlex replica-peer systems. MDMs are
redundant and must continuously communicate with each other to establish
quorum and maintain a shared understanding of data layout.

MDMs do not carry or directly interfere with I/O traffic. The data exchanged among
them is relatively lightweight, and MDMs do not require the same level of
throughput required for SDS or SDC traffic.

MDM to MDM traffic requires a stable, reliable, low latency network. MDM to MDM
traffic is considered back-end storage traffic. PowerFlex supports the use of one or
more networks dedicated to traffic between MDMs.

Meta Data Manager (MDM) to Storage Data Client (SDC)

The primary (also known as the master) MDM must communicate with SDCs in the
event that data layout changes. This can occur because the SDSs that host an
SDC’s volume(s) storage for the SDCs are added, removed, placed in maintenance
mode, or go offline. It may also happen if a volume is placed into a Replication
Consistency Group.

Communication between the Master MDM and the SDCs is lazy and asynchronous
but still requires a reliable, low latency network. MDM to SDC traffic is considered
front-end storage traffic.

Meta Data Manager (MDM) to Storage Data Server (SDS)

The primary (or master) MDM must communicate with SDSs to monitor SDS and
device health and to issue rebalance and rebuild directives. MDM to SDS traffic
requires a reliable, low latency network. MDM to SDS traffic is considered back-end
storage traffic.

Other Traffic

There are many other types of low-volume traffic in a PowerFlex cluster. The other
traffic includes infrequent management, installation, and reporting. This also

PowerFlex rack Administration - Instructor Guide

Page 40 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

includes traffic to the PowerFlex Gateway (REST API Gateway, Installation

Manager, and SNMP trap sender), the vSphere Plugin, PowerFlex Manager, traffic
to and from the Light Installation Agent (LIA), and reporting or management traffic
to the MDMs (such as syslog for reporting and LDAP for administrator
authentication). It also includes CHAP authentication traffic among the MDMs the
SDSs and SDCs.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 41

Error! No text of specified style in document.

PowerFlex Data Workflow

The data workflow in PowerFlex is categorized as Write I/O and Read I/O. Each
workflow has their own functional operation.

Write Operation

Write I/O from the application is sent to the SDC service running on the server
node where the application is installed.

The Write operational steps:

1. The SDC sends the I/O to the SDS where the primary copy of the storage block
is located.

2. The SDS on the node writes the data to the primary block and send the data to
the other SDS that is holding the secondary block for the primary block.

3. The SDS that holds the secondary block acknowledges the data packet.

4. Once the SDS that holds the primary data block receives an acknowledgment
from the SDS that holds the secondary data block, it sends an acknowledgment
back to the SDC. The SDC forwards the acknowledgment to the application and
the write operation is marked completed.

Movie:
The web version of this content contains a movie.

Read Operation

Reads consist of Read Hits and Read Misses. A Read Hit is a read from the
PowerFlex system (SDS) where it finds the requested data already in the system
Read Cache space. Therefore, read hits are run at memory speeds and not at disk
speeds. A Read Miss is a read to the PowerFlex system when requested data is
not in cache and must be retrieved from physical disks.

Read Operation (Hit)

1. The application triggers the SDC to issue the I/O to the SDS

PowerFlex rack Administration - Instructor Guide

Page 42 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

2. If there is a read hit, then the request I/O is acknowledged

3. The data is sent back to the application

Read Operation (Miss)

1. The application triggers the SDC to issue the I/O to the SDS

2. If there is a read miss, then data is retrieved from the physical disk

3. A copy is placed in the read cache to service I/O request

4. The request I/O is acknowledged and the data is sent back to the application

Movie:
The web version of this content contains a movie.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 43

Error! No text of specified style in document.

Write I/O

The SDC fulfills the write I/O request regardless of where any particular storage
block physically resides. Writes are buffered in the host memory for Read after
Write caching. One way to achieve Write buffering is to use RAID controllers (for
example LSI, PMC) that have battery backup for write buffering. The DRAM buffer
is protected against sudden power outages to avoid any data loss.

PowerFlex rack Administration - Instructor Guide

Page 44 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Reads

Reads that the RAID controllers cache services are still considered a Read Miss
from the PowerFlex management point of view. Consider that sequential reads are
not counted separately. If any I/O is serviced from the host read cache, then the I/O
is counted as Read Hits. Any other I/O is counted as Read Misses. In terms of
resources consumed, the host Write I/O generates two I/O over both the network
and back-end drives. A read I/O generates one network I/O and one back-end I/O
to the drives.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 45

Error! No text of specified style in document.

PowerFlex rack Deployment Options and Management Tools

PowerFlex rack Administration - Instructor Guide

Page 46 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Family Deployment Options

PowerFlex supports traditional two-layer and hyperconverged deployments.

PowerFlex also supports the mix of hyperconverged and traditional architecture
deployment.

Two-layer Deployment

In a two-layer deployment, the SDS is installed on a separate node from the SDC.
The front-end (client) is separated from the back-end (storage) data traffic.

Two-layer deployments allow compute and storage resources to grow

independently. A two-layer deployment segregates the rebuild and rebalance traffic
from the application workloads. In this deployment, client (SDC to SDS) and
storage (SDS to SDS) traffic travel on separate dedicated pair of VLANs.

The two-layer configuration simplifies day-2 operations by separating the compute

from storage. Simplified upgrade and life-cycle management are some of the
added benefits of this configuration.

Hyperconverged Deployment

In the hyperconverged (HC) configuration, both the SDC and the SDS are installed
on the same node. HC enables the applications and storage to reside on the same
node.

HC deployment maximizes hardware utilization and reduces infrastructure

requirements. In a HCI deployment, both client (SDC to SDS) and storage (SDS to
SDS) traffic travel on the same VLANs.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 47

Error! No text of specified style in document.

Mixed Deployment

PowerFlex also supports the mix of hyperconverged and two-layer deployments.

The mixed configurations are supported within a single rack. This flexibility allows
organizations to adopt both approaches and enable business units to choose the
model that best aligns to their processes.

PowerFlex rack Administration - Instructor Guide

Page 48 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Management Interfaces Overview

Users can configure, provision, maintain, and monitor PowerFlex rack systems with
the help of various management tools.

Note: PowerFlex Manager is the main management tool that is used for PowerFlex
rack. The other tools should be used only for tasks PFxM cannot perform.

5 3

4 1

1: The PowerFlex Command Line Interface (CLI) is used to perform the entire set
of configuration, maintenance, and monitoring activities in PowerFlex rack.

2: The PowerFlex Web UI enables you to perform standard configuration and

maintenance activities, as well as monitoring the storage system health and
performance. The PowerFlex Web UI also helps you to view the entire system and
examine different elements.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 49

Error! No text of specified style in document.

3: For the VMware hypervisor environments, vSphere Web Client and vSphere
Client enable the management of all virtual components. These components
include the ESXi hosts, virtual machines, distributed switches, datastores, and
more. vSphere Web Client and vSphere Client connects to the VMware vCenter
Server Appliance (VCSA) which is running as virtual machine cluster on the
management controller. PowerFlex rack has two separate vSphere environments,
one for the production cluster and another for the management controller.

Note: The Dell EMC PowerFlex rack plug-in is supported for vCenter 6.5 and 6.7
versions with the old branding. There is no plug-in UI support available for vSphere
7.0 yet.

To learn more about VMware vCenter, and Web Client functionality, see the
VMware website.

4:

Click Here for more information about PowerFlex Manager.

5:

Click Here for more information about iDRAC.

Click the links to explore each management interface in more detail.

• PowerFlex Manager
• PowerFlex Web UI
• Command Line Interface
• iDRAC
• vSphere Plugin
• vSphere Web Client

PowerFlex rack Administration - Instructor Guide

Page 50 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Manager

PowerFlex Manager provides the centralized IT operations management for

PowerFlex rack. It increases efficiency by reducing time-consuming manual tasks
that are required to manage system operations. IT administrators use PowerFlex
Manager to deploy and manage new and existing data center environments.

Key Terminology

Resource Template Service Compliance Repository

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 51

Error! No text of specified style in document.

Resource

The PowerFlex nodes, network switches, element managers, and

hypervisor managers in the system. When PowerFlex Manager
discovers resources, it identifies each component and how to talk to
its management interface.

PowerFlex rack Administration - Instructor Guide

Page 52 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Template

Contains the configuration requirements that must be applied to a

resource or group of resources during deployment. These
requirements include firmware and software, the operating system
and or hypervisor, along with the PowerFlex configuration. A
template represents the wanted state of your deployed
configuration.

PowerFlex Manager provides sample templates that represent some common

configuration options. You can clone a sample and use it for your implementation.
You can also define a custom template if necessary.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 53

Error! No text of specified style in document.

Service

A group of resources that are managed as a cluster. It is the state of

your deployed configuration. PowerFlex Manager performs health
monitoring, RCM/IC compliance monitoring and remediation, and
the ability to add or remove resources.

A service is the finished outcome.

PowerFlex rack Administration - Instructor Guide

Page 54 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Compliance

The comparison between current firmware and software levels and

target compliance versions, which contain the baseline firmware and
software versions. PowerFlex Manager shows any deviation from the
baseline in the compliance status of the resources. You can use
PowerFlex Manager to initiate updates to bring the resources to a
compliant state.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 55

Error! No text of specified style in document.

Repository

The location of firmware and software packages from which

updates can be performed. You can upload more than one
repository, selecting one as the default. This default repository
is used to calculate compliance for all undeployed resources.
PowerFlex Manager lets you specify other repositories for
deployed resources in the services page. The operating
system is included with the compliance repository.

PowerFlex rack Administration - Instructor Guide

Page 56 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Integrated Dell Remote Access Controller (iDRAC)

The Integrated Dell Remote Access Controller (iDRAC) allows out of band remote
access to a Dell PowerEdge server. iDRAC alerts administrator of server issues,
helps them perform remote server management and reduces the need for physical
access to the server.

Key features include:

• A powerful, easy-to-use, remote management and configuration option
• Embedded in every Dell PowerEdge server—no operating system or hypervisor
required
• Deploys, updates, monitors, and maintains PowerEdge servers
• Enables management even if the server is not operational

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 57

Error! No text of specified style in document.

PowerFlex Manager User Interface Orientation

PowerFlex Manager is purpose built software for the PowerFlex product family. The
comprehensive IT operations management software automates and simplifies
implementation, expansion, and lifecycle management. Best of all, PowerFlex
Manager is wizard-driven, making it easy to navigate, consume, and manage your
PowerFlex system resources.

The web version of this content contains an interactive activity.

PowerFlex rack Administration - Instructor Guide

Page 58 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Networking and Switch Management

Dell EMC Networking OS10 Enterprise Edition

OS10EE bundles an industry-hardened networking stack that features standard L2

and L3 protocols over a standard, and well accepted, CLI interface. For more
information about OS10 CLI, see OS10 Enterprise Edition User Guide for a specific
switch type available on support.dellemc.com

Cisco NX-OS CLI for Nexus Switches

NX-OS CLI screenshot

Cisco Nexus switches run NX-OS, an embedded operating system to control the
switch functions. NX-OS has a command-line interface to manage and monitor the
switch environment. NX-OS supports single device management for authentication,
configuration, and updates. NX-OS CLI is used to manage services, health,
performance, and troubleshooting of Cisco Nexus Switches.

For more information on NX-OS CLI, see the Cisco website.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 59

Error! No text of specified style in document.

Remote Connectors

PowerFlex provides two options to remotely connect and manage the PowerFlex
system.

Secure Remote Services

Secure Remote Services support enables secure, high-speed, 24x7, remote

connection between Dell EMC and customer installations, including:

• Remote diagnosis and repair

• Daily sending of system events (rsyslog output), alerts, and PowerFlex topology

Remote Syslog

The MDM syslog service can send events, via TCP/IP, to RFC 6587-compliant
remote (or local) Syslog servers. Messages are sent with facility local0, by default.
Once the syslog service is started, all events will be sent until the service is
stopped.

PowerFlex rack Administration - Instructor Guide

Page 60 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Simple Network Management Protocol

• Collects information from SNMP enabled devices

− Cisco Nexus and Dell switches

− VMware vCenter
− Dell servers and PowerFlex
• Helps proactive alerting for critical and warning
level events

− Including hardware failure and software faults

• All the server hardware and PowerFlex alerts are
sent to OpenManage Enterprise, which works with
PowerFlex Manager to forward them to Secure
Remote Services

− Customer can have SNMP agents that are configured to send information
directly to SNMP server.
Simple Network Management Protocol (SNMP) is a network management protocol,
which is used for collecting status information from network devices, such as
servers and switches. The SNMP enabled device runs SNMP agent, and
communicates with the SNMP management server to share information about
device status. In PowerFlex rack, all SNMP traps should be directed towards
customer’s active SNMP monitoring system. This will provide proactive alerting for
critical and warning level events. These events include, but are not limited to,
hardware failures requiring field replacement and software faults that could
negatively impact the stability of the system.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 61

Error! No text of specified style in document.

Storage Resource Management

PowerFlex rack Administration - Instructor Guide

Page 62 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex rack Storage Constructs

Storage in a PowerFlex cluster is organized into pools which reside in protection

domains.

Use forward and back arrows to review each storage construct.

Protection Domain

Protection Domain is a group of nodes (servers) or SDSs that provide data

isolation, security, and performance benefits. A node (with SDS) can only
participate in one Protection Domain at a time. When data is written to a PowerFlex
cluster, it is mirrored on two different SDSs within a Protection Domain.

For example, if two SDSs that are in different Protection Domains fail
simultaneously, data is still available.

Storage Pool

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 63

Error! No text of specified style in document.

A Storage Pool is a subset of physical storage devices in a Protection Domain.

Each storage device belongs to only one Storage Pool. When a PowerFlex volume
is configured, the volume contents are distributed over all the devices residing in
the same Storage Pool.

Storage Pools enable creating different storage tiers in the PowerFlex system. The
best practice is to have the same type of storage devices within a Storage Pool.
This ensures that each volume is distributed over the same performance type of
storage device. PowerFlex provides two types of Storage Pools "Medium
Granularity (MG)" and "Fine Granularity (FG)" Pools. MG and FG Storage Pools
work in either zero-padding enabled or disabled.

Fault Set

A Fault Set is a logical entity that contains a group of SDSs within a Protection
Domain, that have a higher chance of going down together when they are all
powered within the same rack. By grouping them into a Fault Set, PowerFlex will
mirror the data for a Fault Set on the SDSs that are outside the same Fault Set.
Availability is assured even if all the PowerFlex nodes are within one Fault Set fail
simultaneously.

A Fault Unit can be a Fault Set or a SDS which is not assigned to a Fault Set. A
minimum of three fault sets are required per protection domain.

Once the Fault Sets are created, SDSs can be distributed across them equally. A
SDS can only be added to a Fault Set during the initial configuration of the SDS. An
improper or unbalanced configuration can cause volume creation to be
unsuccessful.

PowerFlex rack Administration - Instructor Guide

Page 64 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Acceleration Pool

Acceleration Pools are a group of acceleration devices that are added to a

Protection Domain to accelerate Storage Pool performance. They provide caching
and prepare data for physical write operations. Acceleration Pools help minimize
physical I/O activities and therefore provide better endurance of the SSD media.

Acceleration Pools enable RFCache of SSD or NVMe devices to accelerate an

HDD storage pool. An Acceleration Pool created from NVDIMM devices is a
requirement for Fine Granularity (FG) storage pools.

Note: Hard Disk Drive (HDD) is not supported in PowerFlex rack.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 65

Error! No text of specified style in document.

Protection Domain Benefits

Protection Against Simultaneous Failures

More than one Protection Domain helps in improving system resilience. This
solution keeps the production I/O unaffected even if there is a failure to the server
or media device.

Performance Isolation

It helps to establish SLA tiers by separating volumes for performance planning. For
example, assigning highly accessed volumes in "less busy" domains or dedicating
a particular domain to an application, which is server-based tiering.

Data Location Control

Tenants are segregated efficiently and securely for data location and partitioning in
multitenancy deployments.

Network Constraint

Protection domains help to spread out network workloads evenly on vLANS.

PowerFlex rack Administration - Instructor Guide

Page 66 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Storage Pool Benefits

Protection against failure

Each volume block has two copies that are on two different SDSs. The copies
enable the system to maintain data availability following a device, network, or
server failure. The data is available following multiple failures, when each single
failure took place in a different storage pool.

Storage Pool tiers

Creation of storage pools helps in data isolation in multitenant environments and

performance isolation within devices of the same type. If all SDS's in a Protection
Domain has two hard drives that are associated with them, then define two Storage
Pools as follows:

• Define a capacity Storage Pool consisting of all HDDs in the Protection Domain.
• Define a performance Storage Pool consisting of all SSDs in the Protection
Domain.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 67

Error! No text of specified style in document.

Zero-Padding

Enabled

• Enabled zero-padding ensures that every read from an area that is previously
not written to returns zeros. Some applications might depend on this behavior.
Furthermore, zero-padding ensures that reading from a volume does not return
information that was previously deleted from the volume. This behavior incurs
some performance overhead on the first write to every area of the volume.

Disabled

• Disabled zero-padding is a read from an area that is previously not written to

that returns unknown content. This content might change on subsequent reads.

PowerFlex rack Administration - Instructor Guide

Page 68 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Read Flash Cache Acceleration Pool

RFCache devices are grouped in acceleration pools. The RFCache feature

consumes entire devices and cannot be shared with other features. Enabling and
disabling RFCache can be done per Protection Domain, Storage Pool, and SDS. A
storage device benefits from RFCache acceleration, when enabled RFCache at all
three levels (PD, SP, and SDS), and the SDS contains RFCache devices.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 69

Error! No text of specified style in document.

NVDIMM Acceleration Pool

In an Acceleration Pool, the acceleration data resides on NVDIMM devices. These

devices are mapped to a logical DAX device which is created with a new linux tool
called ndctl. Also the additional libraries are assigned to the acceleration pool. The
acceleration pools are in turn assigned to an FG storage pool. Writes are
assembled and buffered in acceleration pools.

The benefit of the NVDIMM device is a faster allowance of quicker operation

performance such as compression with a minimal addition of latency. The
acknowledge is sent for an I/O when the I/O is written to the battery protected
NVDIMM. When the compression operation is finished the blocks of I/O is written to
the SSD device.

PowerFlex rack Administration - Instructor Guide

Page 70 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Service Deployment Workflow Using PowerFlex Manager

The image describes the PowerFlex Manager high-level workflow which is typically
done by the Dell Technologies Professional Services team. View each step details
highlighted in red.

3
4

1 5

1: A template is an object in PowerFlex Manager that represents resource types

and their required configuration and topology. A template specifies requirements for
the deployment of a set of infrastructure resources through PowerFlex Manager
automation workflows. These requirements include firmware and software, the
operating system, and or hypervisor, along with the PowerFlex configuration. A
template represents the wanted state of your deployed configuration.
PowerFlex Manager provides sample templates that represent some common
configuration options. You can clone a sample and use it for your implementation.
You can also define a custom template if necessary.

2: A service is a PowerFlex Manager object representing the infrastructure that is

configured during a deployment that persists in PowerFlex Manager if the
infrastructure is required. A service is viewed and interacted with to perform
compliance, health, and life cycle management on the deployed infrastructure.
PowerFlex Manager performs health monitoring, IC compliance monitoring and
remediation, and the ability to add or remove resources. A service is the finished
outcome.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 71

Error! No text of specified style in document.

3: A resource must be discovered in PowerFlex Manager in order for PowerFlex

Manager to manage it. A resource in PowerFlex Manager is categorized into one of
the following groups: element manager, node, switch, VM manager, and PowerFlex
Gateway.
Compliance includes comparison between current firmware and software levels
and target compliance versions, which contain the baseline firmware and software
versions. PowerFlex Manager shows any deviation from the baseline in the
compliance status of the resources. You can use PowerFlex Manager to initiate
updates to bring the resources to a compliant state.

4: Deployment is the automated process of selecting and configuring specific

resource requirements that are outlined in a template using integrated automation
workflows. The template provides the parameters to deploy the service once it is
configured.

5: PowerFlex Manager monitors services for compliance. When the service is out
of compliance, PowerFlex Manager displays the service status on the Dashboard
compliance tab and on the Services page status column. Administrators perform
the update on the service from the Services page.

6: When you first log in to PowerFlex Manager, you are prompted with an Initial
Setup wizard for initial rack configuration. This wizard enables you to configure the
basic settings that are required to start using PowerFlex Manager.

7: The PowerFlex Manager Getting Started page guides you through the common
configuration task that are required to prepare a new PowerFlex Manager
environment.

PowerFlex rack Administration - Instructor Guide

Page 72 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Presentation Server Deployment

Click here for Service Deployment Procedure.

Wait for the video to load.

Movie:

The web version of this content contains a movie.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 73

Error! No text of specified style in document.

Service Deployment Procedure

Service Deployment Procedure

Let us look at the steps to deploy a new service - a 4-node hyperconverged
vSphere cluster:

1. From the PowerFlex Manager Dashboard, select Deploy New Service

2. In this example, we’ll deploy the 4-node VMware HCI Service. In the Deploy
Service popup wizard,

a. Select a Published Template: use Hyperconverged Nodes – ESXi.

b. Enter a Service Name and a Service Description. Try to enter unique

and meaningful information.

c. Select RCM 3.5.2.0, or leave it as the default catalog.

d. Click Next.

3. In the Deployment Settings page, since we are using an HCI template, the
information is pre-populated. The information given is based on the values
already configured in the template, but PowerFlex Manager allows overriding
some of the values if needed. Examine and accept the default values for the
cluster. Click Next.

PowerFlex rack Administration - Instructor Guide

Page 74 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

4. Finally, you can select whether to Deploy Now, or Deploy Later. In this case,
we’ll deploy it now.
Select Deploy Now and then click Next in the lower right corner.

5. In the summary page, the Service to be deployed, including node and network
configurations, the vCenter and PowerFlex configuration, CloudLink setup, and
the Storage Pools details are presented.

a. Scroll to the bottom and select Finish. Select Yes in the confirmation
window that pops up.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 75

Error! No text of specified style in document.

6. We will be presented with a graphical representation of the deployed Service.

Below it, we find details of the Service: the vCenter instance, the PowerFlex
Gateway, and each of the servers that PowerFlex Manager pulled into the
cluster.

7. We must create volumes and datastores. The deployment process created a

single Medium-Granularity Storage Pool (SP-1), in which we will create the
volumes.
Note that we can configure and create more than one volume and datastore at
once in this step.

a. We must add two volumes for the deployment to be complete.

PowerFlex rack Administration - Instructor Guide

Page 76 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

8. After both volumes are added, the health status of the Service switches to
Green/Healthy, and the volumes appear in the Service Details map and under
the Storage resources listed below.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 77

Error! No text of specified style in document.

Adding Storage Constructs Using PowerFlex Manager

The PowerFlex Manager (PFxM) is the primary interface for the PowerFlex rack
management and orchestration. PowerFlex Manager is used to monitor and
manage the PowerFlex Cluster.

PFxM uses Template, Resources, and Service constructs concept to deploy

resources.

When defining a template for a new PowerFlex rack deployment, PowerFlex

Manager provides the option to specify the number of Storage pools, Acceleration
Pools, and Protection Domains needed for the PowerFlex Cluster.

A service deployment results in the creation of a separate storage pool for each
type of disk found in the nodes. The deployment process adds the disks from the
nodes to the appropriate storage pools based on the expected types for each pool.

After the Service is deployed, from the Service page in PowerFlex Manager, you
can view the state of a service at the component level for PowerFlex rack
deployments.

Once the Presentation Server is deployed using the PowerFlex Manager, the
service can be used to launch the PowerFlex UI.

Click here to watch a demonstration on PowerFlex Manager interface and

deployment workflow.

PowerFlex rack Administration - Instructor Guide

Page 78 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Clicking the Management IP address launches the PowerFlex Web UI (Presentation Server).

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 79

Error! No text of specified style in document.

Creating a Protection Domain in PowerFlex Web UI - Hands

On

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

The Online Course Contains an Interaction Here.

This is a hands-on simulation that is part of the lab exercises.

PowerFlex rack Administration - Instructor Guide

Page 80 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Creating a Storage Pool in PowerFlex Web UI - Hands On

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

The Online Course Contains an Interaction Here.

This is a hands-on simulation that is part of the lab exercises.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 81

Error! No text of specified style in document.

Storage Pool Granularity

Medium vs Fine Granularity Storage Pool

Depending on the allocation unit size, a storage pool layout can either be of
"Medium Granularity (MG)" or "Fine Granularity (FG)" type.

• In MG storage pools, volumes are divided into 1 MB allocation units, which are
distributed and replicated across all disks contributing to a pool. The MG
storage pool works great for a performance-driven workload.
• FG storage pools are more space efficient, with an allocation unit of just 4 KB
and a physical data placement scheme based on Log Structure Array (LSA)
architecture built on NVDIMMs. If you need to enable compression or replication
options, you must have an FG storage pool.

Medium Granularity

Medium Granularity Pools

PowerFlex rack Administration - Instructor Guide

Page 82 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Volumes Supports thick or thin-provisioned volumes and also it supports

Zero padded and Non Zero padded Storage pools

Space 1 MB units
Allocation

Device Type SSD Media

Compression Not supported

Usage Recommended for workloads with high-performance

requirements

Fine Granularity

Fine Granularity Pools

Volumes Supports only thin-provisioned, "zero-padded" volumes

Space Allocation 4 KB units

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 83

Error! No text of specified style in document.

Device Type Require SSD/NVMe Media with NVDIMM-N persistent

memory modules.

Compression Supported

Usage Ideal for workloads where space efficiency is more valuable

than raw I/O performance

Storage Pool Considerations

PowerFlex offers a distinctive, competitive advantage with the ability to enable

compression per-volume compared to the entire storage or array. It also provides
the ability to choose the best layout for each individual workload.

Each storage platform has its own advantages and use-cases, and administrators
are free to choose between both layouts. A system can support both FG and MG
pools on the same SDS nodes, and volumes can be nondisruptively migrated
between the two pools.

Why FG Pools?

MG Pool FG Pool

Snapshots causes increased overhead because new writes and FG pools drastically reduce snapshot overhead.
updates to the volume’s data each require a 1MB read/copy Enabling compression or making heavy use of
action. This might have an impact on performance in some snapshots has almost zero impact on the performance of
cases. the volumes.

Cannot enable compression. Uncompressed blocks of data Solution Offers space-saving services and additional data
consume a predictable size on disk that data size that is integrity. FG pools have the same elasticity and
written is equal to data size stored. Even when compression is scalability properties of MG pools. They are a great
enabled at application level, it creates irregular block sizes and choice for most cases where data is compressible and
empty regions. where space efficiency is more important than raw I/O.

PowerFlex rack Administration - Instructor Guide

Page 84 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Why MG Pools?

FG Pool MG Pool
When compression is enabled, reads are slower than in cases where The MG layout would be a better choice for workloads with
compression is disabled, therefore in some cases FG will be slower high performance requirements.
than MG.
MG layout would also be a good option for cases where the
Larger footprint - In FG pools, 256x more metadata is written due to
data is incompressible (if data is already compressed or if it
the 4 KB allocation unit compared to 1 MB allocation unit. Byte
Solution is encrypted by the application).
alignment further increases the amount of metadata. Compression
results in more data to be stored, which further adds to more
metadata. The metadata of FG pools cannot be saved in memory like
in MG pools. So, FG reserves some space on each disk to save the
metadata.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 85

Error! No text of specified style in document.

Fine Granularity Storage Pool

Fine Granularity

The Fine Granularity (FG) layout requires both Flash media (SSD or NVMe) and
NVDIMM to create an FG storage pool. FG layout is thin-provisioned and zero-
padded by nature and enables PowerFlex to support inline compression, more
efficient snapshots, and persistent checksums.

Layout

The fine granularity (FG) layout improves space utilization because of a smaller
data footprint. FG layout requires an NVDIMM-based acceleration pool. The
acceleration pool along with SSD or NVMe media is assigned to the FG pool at the
time of creation. If you only have one NVDIMM on each SDS, you can only have
one FG storage pool in the protection domain. If there are two NVDIMMs per SDS,
you can either have a larger acceleration pool for the single FG pool or have two
separate FG pools.

PowerFlex rack Administration - Instructor Guide

Page 86 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Data Compression

Fine granularity (FG) enables data compression. Within an

FG pool, compression may be enabled or disabled on a per-
volume basis. Data compression is not supported for medium
granularity (MG) pools.

LSA Architecture Volume

Fine granularity (FG) pools use log structure array (LSA) to

store data in fixed size (256 KB) containers called logs. This
architecture mitigates fragmentation issues and minimizes
empty regions. It also enables inline defragmentation and
performs a garbage collection of full logs during rewrites.

Defragmentation

Log Structure Architecture (LSA)

Fine Granularity (FG) layout supports only thin-provisioned, "zero-padded" volumes

and provided space allocation at 4 KB units. When possible, FG layout reduces the
actual size of user-data that are stored on the disk. FG layout includes Persistent
Check-summing for data integrity. This occurs because the FG layout uses the Log
Structure Array (LSA) architecture.

What problem is being solved with FG Layout?

Placing or fitting data on disk becomes complex with compression, so a new way to
lay out the data on a disk is needed. Also, how snapshots are written can cause
write amplification problems.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 87

Error! No text of specified style in document.

What solution does LSA provide?

LSA provides the FG layout with a smaller allocation unit to handle problems with
snapshots and provides a different way to place data on the disk and handle gaps
left by compression.

Click here to learn more about the LSA solution.

LSA architecture provides a viable solution in the FG layout. Click here to learn
more about why we need the FG layout with the LSA architecture in storage pools.

How does LSA work?

Click here to view a simple illustration of how LSA works.

The Solution: Log Structure Array (LSA)

The web version of this content contains an interactive activity.

Why FG Layout with LSA?

The web version of this content contains an interactive activity.

How LSA Works

The web version of this content contains an interactive activity.

Inline Data Compression

How does inline compression work?

Inline compression reduces the volume of data that is stored on the disk and
improves space utilization. This storage efficiency feature is enabled using
NVDIMM devices that are used for Fine Granularity storage pools.

What FG Pool modes use inline compression?

Inline data Compression can be used in FG compressed and FG non-compressed

modes.

PowerFlex rack Administration - Instructor Guide

Page 88 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

• FG compressed pools can be chosen for situations where space efficiency is

more valuable than I/O performance, and the data is compressible. If PowerFlex
determines that the user data is not compressible, it will override the
compression attribute.
• FG non-compressed pools are used where it does not make sense to enable
compression, but you still need read-intensive performance.

How does inline compression handle noncompressible data?

In case a volume has compression that is enabled, and non-compressible data is

being written, the system can determine that it should not compress the data, thus
avoiding the additional CPU utilization required by compression.

When is inline compression enabled?

Compression can be enabled at the volume level. However, an administrator can

also set the compression at the pool level. Setting the compression at the pool level
makes it a default setting when the volume is created on the storage pool.

Inline Compression Data Flow

Write I/O Flow (Compressed Data):

1. SDC sends uncompressed data to the SDS.

Write I/O Flow.

2. SDS gets uncompressed data.

3. SDS compresses the data (in RAM).
4. SDS allocates the right amount of space for the data in the LSA logs.
5. SDS writes the data in one of the LSA lines on the NVDIMM. Parts of the LSA
reside in the NVDIMM. The SDS moves LSA data between the NVDIMM and

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 89

Error! No text of specified style in document.

the SSD. The NVDIMM is used to enable compression and as performance tier
for the SSD and for SSD endurance. However, some data is always present on
the NVDIMM only.

Read I/O Flow (Compressed Data):

1. SDC asks for 4 K data (regular application I/O to storage).

Read I/O Flow

2. SDS looks-up the LSA logs containing the data. If the data is not on the
NVDIMM, it is fetched from the SSD.
3. SDS gets the compressed data (copied into the NVDIMM).
4. SDS decompresses the data - now it is exactly 4 K.
5. SDS sends the uncompressed data to the SDC.
6. SDC gets an uncompressed 4 K block of data.

PowerFlex rack Administration - Instructor Guide

Page 90 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Creating an Acceleration Pool in PowerFlex Web UI - Hands

On

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

The Online Course Contains an Interaction Here.

This is a hands-on simulation that is part of the lab exercises.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 91

Error! No text of specified style in document.

Administrating Volumes

Administrating Volumes

Adding a Volume Using PowerFlex Manager

Storage volumes for a PowerFlex Cluster can be created after a Service

deployment. Initially, there are no volumes for a new service. The service is in an
incomplete state until volumes are added. At least one volume must be added to a
service to get it out of the incomplete state.

To add a new volume to the cluster, from the Services page, choose Add Volume
under Add resources option. From the Add Volume wizard, parameters such as
volume size, compression can be defined. Volumes can be assigned to existing
datastores, or new datastores can also be created.

Once the storage volumes are added to the service, PowerFlex Manager lets you
view details about the storage volumes from the Services Page.

After you create the volumes in a storage-only service, they are added to
PowerFlex, but not mapped. When you add the volumes to a compute-only service,
PowerFlex maps the volumes and creates the datastores.

For a hyperconverged service, the added volumes are mapped to the datastore.
PowerFlex Manager requires you to enter the datastore name for each new volume
that must be added. This is because PowerFlex Manager also creates an ESXi
cluster and vCenter datastore for a hyperconverged service. PowerFlex Manager

PowerFlex rack Administration - Instructor Guide

Page 92 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

allows you to enable compression when adding volumes to a storage-only or

hyperconverged service.

Creating and Mapping a Volume in PowerFlex Web UI - Hands On

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

Volume Migration

Migrating PowerFlex volumes from one storage pool to another, migrate the
volume and all its snapshots together (known as vTree granularity). Migration is
nondisruptive to ongoing I/O and is supported across storage pools within the same
protection domain or across many protection domains. Zero-padding must be
enabled for those storage pools.

Volume Migration is performed using PowerFlex Web UI.

Play the video to view a volume migration demonstration.

Movie:

The web version of this content contains a movie.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 93

Error! No text of specified style in document.

Migrating a Volume in PowerFlex Web UI - Hands On

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

The Online Course Contains an Interaction Here.

This is a hands-on simulation that is part of the lab exercises.

PowerFlex rack Administration - Instructor Guide

Page 94 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Compute Resource Management

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 95

Error! No text of specified style in document.

Compute Resources in PowerFlex rack

The PowerFlex rack computes only, and hyperconverged nodes provide the
required computing resources. These resources are pooled together and
configured to host virtual machines. On these VMs, you can run your application
workloads and other services. You have a choice of hypervisors to use on these
nodes: VMware vSphere, Red Hat Virtualization (Red Hat Virtualization), or Hyper-
V (two-layer deployment only).

Virtual machines play a few different roles in a PowerFlex rack system.

• Production VMs: You deploy your production virtual machines onto the
PowerFlex rack system to perform your computing needs. These VMs could be
database servers, application servers, or any other types of virtual machine.
• PowerFlex rack services VMs: Many internal functions and PowerFlex rack
management tools that are run on virtual machines. This includes vCenter, Red
Hat Virtualization Manager, PowerFlex Manager, and support services. These
VMs are created during installation only.
• Storage Services VMs: Storage virtual machines must use the physical
storage on a hyper-converged node. These VMs give PowerFlex access to
storage.

PowerFlex rack Administration - Instructor Guide

Page 96 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

VMware vCenter for Compute Resource Management

Once you have deployed infrastructure using PowerFlex

Manager, you can use the VMware vCenter as the centralized
tool. vCenter is used to manage the ESXi hosts and VMs that
are running in the PowerFlex Node cluster. PowerFlex rack also
supports bare metal deployments.

VMware vCenter enables you to standardize and simplify

configuration and management of VMware ESXi hosts. With
VMware vCenter, you can perform the following activities:
• Capture a template of a known, validated VM
configuration—including compute, networking, storage, and
security settings—and deploy it to many hosts.
• Allocate processor and memory resources to virtual
machines. If RAM hot-add and CPU hot-plug are enabled,
modify allocations of the resources while virtual machines
are running.
• Enable applications to dynamically acquire more resources to adapt to periods
of peak demands.
• Automatically allocate available resources among virtual machines according to
predefined rules that reflect business needs and changing priorities.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 97

Error! No text of specified style in document.

VMware vSphere Environments for PowerFlex rack

PowerFlex rack has two separate vSphere environments - one for the PowerFlex
Management Controller cluster, and the other for the PowerFlex node cluster.

1: The Controller cluster hosts VMs that provide services for the PowerFlex rack
system itself. It is a VMware vSphere cluster where all the nodes run the ESXi
hypervisor, and they are managed by VMware vCenter. For storage, the Controller
cluster uses VMware vSAN. Similar to PowerFlex, vSAN aggregates the locally
attached disks of the PowerFlex Controller nodes to create a pool of distributed
shared storage.

2: The PowerFlex node clusters primarily host the production virtual machines.
Nodes in the PowerFlex cluster run either ESXi or Red Hat Virtualization
hypervisors and are managed by VMware vCenter or by Red Hat Virtualization
Manager. Unlike the Controller cluster, the PowerFlex node cluster that uses
PowerFlex for all the customer production data. PowerFlex provides massive
scalability and flexibility in terms of hypervisor/OS and bare-metal deployments. If
using ESXi nodes, Storage Virtual Machines must provide storage to PowerFlex.

PowerFlex rack Administration - Instructor Guide

Page 98 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Management Controller Cluster

• Consists of at least three PowerFlex rack Controller nodes - running ESXi

− Local storage allocated to vSAN
− Boots off BOSS card (Boot Optimized Storage Solution).
• VMs commonly available on a PowerFlex rack Controller are:

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 99

Error! No text of specified style in document.

− Virtualization Management
o PowerFlex Controller VCSA with vCenter Server High Availability
o PowerFlex node cluster VCSA
o PowerFlex node Red Hat Virtualization Manager or RHV-M (Used as an
optional component only when Red Hat virtualization is deployed)
− PowerFlex Gateway
− PowerFlex Manager and OpenManage Enterprise
− Secure Remote Services
− Windows jump servers
The PowerFlex Controller cluster maintains the environment for the PowerFlex rack
management. The virtual machines running on the Controller Cluster include
vCenter Server Virtual Appliance (vCSA), PowerFlex Gateway VM, PowerFlex
Manager, and OpenManage Enterprise VMs, Secure Remote Services VMs, and
Windows jump servers for support access. This cluster uses vSAN for storage, so
you do not need PowerFlex in the controller cluster.

Note: An administrator may not interact with most of these VMs for day-to-
day administration activities.

PowerFlex rack Administration - Instructor Guide

Page 100 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Node Cluster

• Consists of PowerFlex rack nodes (forms the

production clusters)
− Local storage is allocated to PowerFlex system.
− Hyper-Converged or Compute-only node runs
ESXi or Red Hat Enterprise Linux or Hyper-V
hypervisor
− Storage-only nodes run embedded operating
system that is based on CentOS kernel as an
Operating system for PowerFlex software
− Boots off BOSS card (Boot Optimized Storage
Solution).
• Every ESXi host has a Storage Virtual Machine or
SVM.
− Runs Storage Data Server
− May also run MetaData Manager.
− Immune to normal VMware HA and DRS
− Cannot be VMotioned to another host.
− Cannot be backed up using snapshots.
• RHV nodes run PowerFlex software directly on Red Hat Enterprise Linux
operating system
• Customer application VMs - for example Linux, Windows (not shown in the
graphics)

The PowerFlex node clusters (or production clusters) provide compute and storage
to customer applications. PowerFlex pools together the local storage of the node.

To provide storage to PowerFlex system, ESXi hosts need a Storage Virtual

Machine (SVM). The SVM has the storage controller of the node, who is mapped to
it using DirectPath I/O. The DirectPath I/O gives the SVM direct access to the
storage controller. Rebooting or powering off the SVM causes PowerFlex to believe
that the node failed.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 101

Error! No text of specified style in document.

The Storage VMs cannot be migrated to other hosts because they need direct
access to the local storage of the node. However, other VMs consuming the
PowerFlex storage can be migrated from one ESXi host to another or from one
RHV host to another.

Nodes running RHV do not need storage VMs. Instead, PowerFlex SDC and SDS
software run directly on the Red Hat Enterprise Linux operating system of the node.

The Storage-only nodes run embedded operating system that is based on CentOS
kernel and contributes storage to the PowerFlex cluster. No customer application
runs on Storage-only nodes. Compute only nodes provide computing power, but do
not contribute any storage to the PowerFlex storage pool

The VXSA hosted on the controller cluster manages the ESXi nodes in the
PowerFlex cluster. Similarly, Red Hat Virtualization Manager (RHV-M) virtual
machine on the controller cluster manages RHV.

PowerFlex rack Administration - Instructor Guide

Page 102 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

ESXi Boot Device

Each ESXi node requires a Storage Virtual Machine (SVM), to access PowerFlex
storage. Because the SVM provides access to PowerFlex storage, it cannot be
stored on PowerFlex storage. Instead, its files are stored on a small datastore that
uses the internal BOSS of the node (PowerEdge 14G) or SATADOM (13G)
storage. These datastores are labeled DASXX or something similar.

This is the boot drive of the PowerFlex node. And this device could have other
operating system running on depending on the nodes use in the PowerFlex rack
cluster (Linux, Windows, and ESXi).

DASXX datastores should only store the SVMs and system files. Production virtual
machines should be stored on datastores that are backed by PowerFlex storage.

DASxx datastores are for system use

Used by SVMs only

Shown is the storage view in the vSphere Web Client for the PowerFlex cluster.
Notice the five DASXX datastores in this example. There is one datastore for each
node. The device backing of the datastores is labeled as a local SATADOM device.
Along with the datastore, these devices also host the ESXi operating system.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 103

Error! No text of specified style in document.

Provisioning Storage for Production Virtual Machines

Any virtual machine that is used for production must be stored on the PowerFlex
storage. The first thing that you must do is to create a volume in PowerFlex. Thick
provisioning for the PowerFlex volume is recommended. Thick provisioning is
recommended because the hypervisor is not aware of whether the volume is over
provisioned. Thin provisioning can be then used on VMWare when creating the
virtual machine disks (if needed).

The volume must be mapped to all the SDCs so that they have access to the
volume. Then make a note of the PowerFlex Volume ID number of the new volume.
This number is used to locate the volume in vSphere.

In vSphere, you can see the details of the Storage Devices available to a
PowerFlex node cluster. The Fibre Channel Disks that you see here are the
PowerFlex volumes that are mapped or available to a specific host. The ends of
their identifiers match the PoweFlex Volume ID that was assigned during the
creation of the volume.

PowerFlex Volumes

Matches ID from
PowerFlex

PowerFlex rack Administration - Instructor Guide

Page 104 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Create Datastore on PowerFlex Volume

After you have created the PowerFlex volumes, you can create a datastore on the
PowerFlex Volume Fibre Channel Disk. When creating a datastore, be sure to
select a device that uses a PowerFlex volume. Here, you see the wizard screen to
select the device. The selected device has an identification number that matches
the volume that is created in PowerFlex. After completing the wizard, select Finish,
and the datastore is created.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 105

Error! No text of specified style in document.

Virtual Machine Deployment Options

PowerFlex rack provides the same methods for building virtual machines in
vSphere as in any vSphere environment. Some of these methods, such as cloning
or deploying VMs from a template, require vCenter. Others are universal to
whatever management platform is being used. Using the New Virtual Machine
wizard makes it easy. One key difference when allocating storage for a VM is that
you should choose a datastore that uses PowerFlex volumes.

To review, some of the most common methods of VM deployment include.

Building a virtual machine

Building a fresh new VM involves, installing a Guest operating system into the VM,
and installing VMware Tools. Installing an operating system into a VM takes about
the same amount of time it would take on a physical system. Most other
deployment methods involve imaging a base VM template to avoid installing the
operating system repeatedly for each VM.

Cloning a VM

Cloning VMs involve taking a copy of a base VM (image) to create a new virtual
machine. Cloning takes a copy of the configuration and storage files for a VM and
uses it as the basis for a new virtual machine.

Deploying a VM from Template

Deploying a VM from template involves building a template off a base image VM

and then using it to provision the new VMs. Templates consist of the same files that

PowerFlex rack Administration - Instructor Guide

Page 106 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

support a virtual machine. Deploying a VM from a template essentially involves

copying these files to create a VM.

Importing a virtual
appliance

For a VM to be imported as a virtual appliance it must be prepared in Open

Virtualization Format (OVF) before an import. After it is imported, the virtual
appliance can be started in the ESXi or vCenter inventory.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 107

Error! No text of specified style in document.

Create VM Example

When creating VMs in the PowerFlex rack environment, ensure that you select the
PowerFlex storage and not the individual datastore on each host. Allocating
storage is part of the process when using the New Virtual Machine wizard in the
vSphere Web Client.

Choose PowerFlex
Datastore

In the vSphere Web Client app, you can choose any datastore. You must be careful and ensure that
you select the PowerFlex datastore and not the local datastore that is shown as DASXX.

PowerFlex rack Administration - Instructor Guide

Page 108 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Add Storage to VM

You can expand the storage capacity of a virtual machine by adding a virtual disk,
and this is referred to as Raw Device Mapping (RDM). To add a disk to a virtual
machine, select the New Hard disk under New Device in the Edit Settings
screen. Specify the size of the new disk, expand the New Hard disk, and then
expand Location. Select either Store with the virtual machine or Browse.
Browse shows you a list of devices (as shown in the image). If you set up your
VMFS datastores with meaningful names, it can help you to choose the correct
device.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 109

Error! No text of specified style in document.

Virtual Machine Management

There are many options to control a VM and its environment. Some of these
options include monitoring, creating a snapshot, cloning, creating a template, and
adding/removing devices. To see all the options, select the VM in the left navigation
pane and click Actions.

Management of VM
• VM power on/off
• VM Clone/Template
• VM Snapshot
• Edit Settings

− Add/remove virtual hardware

− Set virtual machine options.
− Control a virtual machine’s CPU and memory resources.
You can modify virtual machine settings with the Edit Settings option. With
supported guest operating systems, you can also add CPU and memory while the
virtual machine is powered on.

PowerFlex rack Administration - Instructor Guide

Page 110 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 111

Error! No text of specified style in document.

VM Migration Using vSphere VMotion

A vSphere VMotion migration moves a powered-on running virtual machine from

one host to another in a cluster. Concurrent migrations are possible.

Click here to look at the migration wizard UI.

Migration wizard

VMotion Migration

• Improves overall hardware use

• Continuous VM operation for scheduled downtime
• Supports vSphere DRS to balance workload across hosts

VM requirements for Migration

• It must not have a connection to an internal standard switch: Virtual switch with
zero uplink adapters.
• It must not have CPU affinity configured.
• vSphere VMotion must be able to create a swap file accessible to the
destination host before migration can begin.

Types of Migration

• Cold: Migrate a virtual machine that is powered off

• Suspended: Migrate a virtual machine that is suspended
• vSphere VMotion: Migrate a virtual machine that is powered on

PowerFlex rack Administration - Instructor Guide

Page 112 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

See VMware website for the latest information about maximum concurrent migration to a single
vSphere VMFS datastore.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 113

Error! No text of specified style in document.

Migration Wizard

Note: Storage virtual machines (SVM) are not candidates for migration as they use
local storage.

PowerFlex rack Administration - Instructor Guide

Page 114 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Red Hat Virtualization Manager for Compute Resource

Management

You use Red Hat Virtualization Manager (RHV-M) to manage the Red Hat
Virtualization environment. RHV-M runs on a virtual machine that is hosted on the
controller cluster.

Note: RHV-M is an alternative interface for VMWare Vsphere. It is used when the
nodes are running Red Hat Linux instead of ESXi as hypervisor.

RHV-M enables you to perform many common virtualization management tasks

including:
• Creating Virtual Machines from scratch or from an OVF template.
• Allocate processor and memory resources to virtual machines.
• Migrate virtual machines from one node to another.

To create a VM from an OVF template, go to Compute, select Virtual Machines,

and select Import from the upper right corner drop-down.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 115

Error! No text of specified style in document.

Network Resource Management

PowerFlex rack Administration - Instructor Guide

Page 116 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Network Types

Click here to learn more about:

Network Configuration Details

Network resources are managed using PowerFlex Manager. From the PowerFlex
Manager UI, go to Dashboard>Settings>Networks.

View the various network types below.

1
2

9
10
8

1: Used for management of Hypervisors in the system. These network VLANs are
defined at system level.

2: Used for provisioning of operation system or hypervisor on the PowerFlex

Nodes.

3: Allows static or DHCP network for operating system imaging on nodes.

4: Used for Storage Data Client traffic only

5: Used for Storage Data Server traffic only.

6: Used for data traffic between Storage Data Servers (SDS) and Storage Data
Clients (SDC).

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 117

Error! No text of specified style in document.

7: Used for PowerFlex System Management.

8: Used to manage the network that you want to use for live migration. Live
migration enables you to move running virtual machines from one node of the
failover cluster to a different node in the same cluster.

9: Used to support PowerFlex replication.

10: Used for out-of-band management of hardware infrastructure.

PowerFlex rack Administration - Instructor Guide

Page 118 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Verifying Connectivity

The connectivity between Storage Data Server (SDS) and Storage Data Client
(SDC) and PowerFlex Gateway can be verified using an SSH session.

Between SDS and SDC

Use this procedure to ping the Storage Data Server (SDS) from Storage Data
Client (SDC).
1. Open an SSH session with a VMware ESXi host using PuTTy or a similar SSH
client.
2. Log in to the host using root.
3. Type vmkping to ping each SDC and SDS.
4. Repeat from each VMware ESXi host.

− In the vmkping command example:

{root@node7:~] vmkping -d -s 8972 -I vmk3 192.168.176.6
8980 bytes from 192.168.176.6: icmp_seq=0 ttl=64
time=0.191 ms
-s specifies the packet size, and the number of data bytes sent. (The packet
size is 8972 because the IP header is 20 bytes and the ICMP header is 8
bytes.)
-d prohibits fragmentation.
-I specifies the outgoing VMkernel interface.

Between SDS and Gateway

Verifying connectivity between Storage Data Server (SDS) and PowerFlex

Gateway from Storage Data Server (SDS).

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 119

Error! No text of specified style in document.

Use this procedure to ping the SDS and PowerFlex Gateway from SDS.

1. Open an SSH session with an SDS host using PuTTy or a similar SSH client.
2. Log in to the host using root.
3. Ping each SDS and the PowerFlex Gateway using a 9000 byte packet without
fragmentation on the SDS to SDS data networks.
4. Repeat for each SDS host.
5. Repeat for the PowerFlex Gateway.

− In the ping command example:

[root@node4 ~]# ping -s 8972 -M do 192.168.152.102
8980 bytes from 192.168.152.102: icmp_seq=1 ttl=64
time=0.299 ms
-s Specifies the packet size and the number of data bytes to be sent. (The
packet size is 8972 because the IP header is 20bytes and the ICMP header
is 8 bytes.)
-M do Prohibits fragmentation

PowerFlex rack Administration - Instructor Guide

Page 120 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Checking the Maximum Transmission Unit (MTU)

Maximum transmission unit (MTU) is the largest physical packet size, which is
measured in bytes, that a network can transmit. Any messages larger than the
MTU are divided into smaller packets before transmission. MTU on Access
switches can be checked using PuTTy or an SSH client. We can also check MTU
on VM Kernel ports and all other port groups using the vSphere web client.

The procedure can be used to check the maximum transmission unit on access
switches.

1. Open an SSH session with the switch using PuTTy or a similar SSH client. You
can also connect to the serial console.
2. Check each interface for their MTU configuration.

a. Dell EMC Networking:

i. For the Dell EMC S5048F Networking switch, type: S5048F-45#show
interfaces port-channel 100 |grep MTU
MTU 9216 bytes, IP MTU 9198 bytes
ii. For the Dell EMC S5224F Networking switch, type: S5224F#show
interfaces port-channel 100 | grep MTU
MTU 9216 bytes, IP MTU 9198 bytes
iii. For the Dell EMC S4148F Networking switch, type: S4148F#show
interfaces port-channel 100 | grep MTU
MTU 9216 bytes, IP MTU 9198 bytes
b. Cisco Nexus
i. Cisco_Access-A# show interface port-channel100 | grep
MTU
MTU 9216 bytes, BW 100000 Kbit, DLY 10 usec

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 121

Error! No text of specified style in document.

Adding a Network to a Service

You can add an available network to a service or choose to define a new network
that was initially deployed outside of PowerFlex Manager. You cannot remove an
added network using PowerFlex Manager.

PowerFlex Manager supports static route configurations for both replication and
nonreplication use cases. A static route allows communication between compute-
only and storage-only nodes in different network environments. When you define a
network, PowerFlex Manager enables you to specify the IP address for the subnet.
You can add a static route to a template before deployment, or add it later as a
resource on the deployed service.

Services > View Details > Resource Action > Add Resources > Add Network

PowerFlex rack Administration - Instructor Guide

Page 122 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Adding a VLAN to Access Switches

To configure an access switch that is connected to a PowerFlex appliance cluster,

use the commands provided. This must be performed on both access switches.

Dell EMC Networking switch

Dell#configure

Dell(conf)#interface vlan<vlan id>

Dell(config-vlan)#no shutdown

Dell(config-vlan)# exit

Dell#copy running-config startup-config

Cisco Nexus switch

These commands are an example of how to add VLAN 10 to the uplink port-
channel 100.

Cisco_Access-A# configure

Cisco_Access-A(config)# vlan 10

Cisco_Access-A(config-vlan)# exit

Cisco_Access-A(config)# interface port-channel 100

Cisco_Access-A(config-if)# switchport trunk allowed vlan add

10

Cisco_Access-A(config-if)# end

Cisco_Access-A# copy running-config startup-config

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 123

Error! No text of specified style in document.

Access Switch Port Configurations

The access switches provide networking for both the controller nodes and
PowerFlex nodes. The traffic coming from these nodes use various VLANs to allow
the traffic to remain separated, even if it is traveling over the same physical cable.
The switches are configured with virtual PortChannel (vPC) to allow multiple
physical connections to act as one, even if they are on separate switches.
PowerFlex rack uses virtual PortChannel for all the management and production
traffic.

PowerFlex Node Connectivity

VLAN Tagged

Access Switch

VLAN Trunked

VLAN Tagged

Access Switch

VLAN Trunked
Mgmt
Data

PowerFlex Data

vP

PowerFlex Node

• Management Data - These ports are configured in switchport trunk mode,

because different types of traffic from different VLANs are traveling on these
ports.

PowerFlex rack Administration - Instructor Guide

Page 124 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

• PowerFlex Data - These ports tag all incoming traffic with a VLAN ID for that
data network. The two PowerFlex data ports on the two switches use different
VLAN IDs, and they are not part of a vPC. Instead PowerFlex handles the load
balancing of traffic on these two connections.
• Virtual Port Channels (vPC) - vPCs are connected to different network devices,
but act as a single port channel to a third device. The benefits of vPC include
high availability, fast convergence, and increased bandwidth.

Controller Node Connectivity

VLAN Trunk

Access Switch

VLAN Trunk

VLAN Trunk

Access Switch

VLAN Trunk
vPC
Mgmt
Data

Services Data

vPC

Controller Node

Each access switch has two ports that connect to each controller node. Since each
port carries traffic that is segregated on different VLANs, they are configured in
switchport trunk mode. This allows them to accept traffic tagged with multiple
VLANs.

To provide higher bandwidth, these connections are configured as virtual port

channels (vPCs). The vPCs are already set up on the management network
connecting to the uplink switches. Do not change these settings.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 125

Error! No text of specified style in document.

Logical Network V3 Support

PowerFlex Manager supports logical network v3 configurations. This support

covers new deployments and upgrades and expansions on previous configurations.
The sample templates have also been updated to include appropriate fields for
logical network v3 settings. Most sample templates contain the latest best practice
node networking configurations (logical v3 networking).

Older network configurations at a customer site require editing the template node
settings to match the customer node settings. The Node Switch Port Configuration
setting in the operating system Settings section of the template specifies whether a
Cisco virtual PortChannel (vPC) or Dell Virtual Link Trunking (VLT) is enabled or
disabled. The Port Channel options of the Switch Port Configuration turns on or off
the vPC or VLT. The sample template default Port Channel option provides link
aggregation or bundle of multiple port channel connections through the Link
Aggregation Control Protocol (LACP). This default setting is the logical v3 network
configuration.

PowerFlex rack Administration - Instructor Guide

Page 126 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

NS-OX Query Commands

To view the switch configuration, connect the switch and use Cisco NX-OS. Here,
you are logged into the Top of Rack switch.

The web version of this content contains an interactive activity.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 127

Error! No text of specified style in document.

Logical Networking

In PowerFlex rack, logical networking is provided at the switch level and at the
node level. Distributed virtual switches or DVswitches are configured to manage
virtual networking. Both the Management node and the PowerFlex node consists of
three DVswitches, each with multiple port groups.

The logical network topology can be viewed in VMware vSphere. Click here to see
how to go to the logical network topology.

Hover over the components in the image below to see the networks between the
physical components and the DVswitches.

The web version of this content contains an interactive activity.

PowerFlex rack Administration - Instructor Guide

Page 128 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

vSphere Network Topology

This view shows the logical networking in VMware vSphere. This view can be
accessed by selecting the Network icon, and then the distributed virtual switch you
want to see the networking of. Then, select the Configure tab > Topology from the
menu on the left.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 129

Error! No text of specified style in document.

ESXi Host Network Interface

Using the vSphere Web Client, you can view the physical adapters on an ESXi host
by selecting it and going to the Configure > Physical adapters section. This
shows a list of network interface cards and their speeds, MAC addresses, and so
on. In this image, we are looking at physical network interface cards, which show
us four 10Gb ports that are used for VxFlex integrated rack networking. It also
shows which distributed switch is using each interface as an uplink.

View details under Configure > Physical adapters

PowerFlex rack Administration - Instructor Guide

Page 130 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Adding a Production Network

Production virtual machines need their own port groups and VLANs to have
networking. They should not use the existing port groups or VLANs, which are for
PowerFlex rack system use only. Production traffic should also be separate from
the PowerFlex data traffic. Put Production traffic on DVSwitch0, and configure
separate port groups and VLANS.

Discover Switch Configuration

To configure the physical switches, find a VLAN number that is not in use. This is
your new VLAN. Display the port channel interfaces with the show interface
description command. Make a note of the interface names, that are used for
the uplink, peer-link, and connections to all ESXi hosts. These are the port
channels that need the new VLAN added.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 131

Error! No text of specified style in document.

On the access switches:

• Discover current VLANs.
show vlan

− Find an unused VLAN ID.

• Discover current port channels.
show interface description

− Use description to find Uplink, Peer-link, ESXi connections.

− Note Po numbers.

Configure Access Switches

Configure the access switches to accept traffic tagged with the new VLAN ID. Add
the VLAN to virtual port channels for the peer link, uplink, and ESXi hosts. Use the
vPC numbers identified earlier. Confirm that the VLANs have been added with the
show vpc command. This shows your new VLAN listed under each of the vPCs.

Create VLAN

Add VLAN to:

Peer link

Uplink

Hosts

On both access switches:

• Add new VLAN
vlan <vlan-id>
name <vlan-name>

PowerFlex rack Administration - Instructor Guide

Page 132 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

• Add VLAN to vPC peer link port channel

interface port-channel <peer link vPC number>
switchport trunk allowed vlan add <vlan-id>
• Add VLAN to uplink port channel
interface port-channel <uplink vPC number>
switchport trunk allowed vlan add <vlan-id>
• Add VLAN to each ESXi host port channel
interface port-channel <host's vPC number>
switchport trunk allowed vlan add <vlan-id>

Create Port Groups

You also must create a port group that will use the new VLAN ID. Traffic from any
virtual machine using that port group will be tagged with the correct VLAN ID, and
be allowed to travel across the access switches.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 133

Error! No text of specified style in document.

Enables all settings in

wizard

• Add new Port Group to DVswitch0

• Configure VLAN ID - Set the VLAN ID to match the VLAN you had configured
on the access switches
• Teaming and Failover, Load Balancing - Select Route Based on IP hash
− Required for virtual peer channels
• Configure VMs to use this port group for networking

PowerFlex rack Administration - Instructor Guide

Page 134 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Security and User Management

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 135

Error! No text of specified style in document.

User Roles and Authentication

User Roles

Every PowerFlex Manager user account can be assigned to a user role.

User permissions

2 3

1 4

1: Users with the administrator role can view all pages and perform all operations in
PowerFlex Manager and grant permission to standard users to perform certain
operations. The administrator can perform functions like deploying a service,
exporting it to a file, and performing upgrades.

2: Users with the standard role can view pages and perform operations that are
based on the permission that is granted by an administrator. Also, standard users
can grant permission to other users to view and perform operations that they own.
A standard user with owner privileges can edit service information, and perform
compliance upgrades.

PowerFlex rack Administration - Instructor Guide

Page 136 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

3: Users with the operator role can view pages and perform operations that are
based on the permission that is granted by an administrator. Drive replacement is
the primary operation that is performed by operator users.

4: Users with the read-only role can view all operations but are not allowed to
perform any operations. When a user logs in as a read-only user, PowerFlex
Manager does not allow the user to perform any operations.

User Permissions

The table provided shows a concise list of permissions for the user roles.

To get a detailed view of the permissions for each user role, navigate to the Online
Help section in the PowerFlex Manager interface.

Feature Permission Administrator Standard Operator Read-

only

Dashboard View Yes Owner and Yes Yes

or
participant

Read Yes Owner and Yes Yes

or
participant

Services View Yes Owner and Yes Yes

or
participant

Deploy a Yes Owner and No No

service or
participant

Export to file Yes Owner and Participant No

or
participant

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 137

Error! No text of specified style in document.

Add an Yes Owner and No No

existing or
service participant

Templates View Yes Participant Yes

Read Yes Participant N/a Yes

template

Create new Yes No N/a No

template

Edit template Yes No N/a No

Delete Yes No N/a No

template

View Yes Participant N/a Yes

template
details

Clone Yes No N/a No

template

Resources View Yes Participant Participant Yes

View all Yes Participant Participant Yes

resources

Run Yes No No No
discovery

Remove Yes No No No
resources

Perform Yes Owner No No

compliance
upgrades

PowerFlex rack Administration - Instructor Guide

Page 138 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

View Yes Owner Participant Yes

compliance
report

Creating, Editing, and Deleting User

Users can be added, modified, and deleted using the PowerFlex Manager
interface.

Create User

Settings > Users > Create

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 139

Error! No text of specified style in document.

Edit User

Settings > Users > Edit

Delete User

Settings > Users > Delete

Securing Connections Using SSL Encryption

For security reasons, all cluster management connections are recommended to be

secured. In a PowerFlex system, all management interfaces are encrypted with
Secure Socket Layer (SSL) to the cluster. It provides a secure connection between

PowerFlex rack Administration - Instructor Guide

Page 140 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

external components (CLI, UI, PowerFlex Gateway) and the MDM. A PowerFlex
system supports SSL login for external components. The MDM cluster has an SSL
certificate that is used externally with all components. There is a certificate for each
of the active MDMs (in any MDM cluster mode).

In the PowerFlex Installation Manager UI, the certificate is generated upon the
installation of the MDM components. This certificate is a self-signed certificate. The
customer signs the certificate using a Certificate Signing Request (CSR).

You can also generate or upload an SSL certificate from the PowerFlex Manager
UI post installation. See the below images.

PowerFlex Installation Manager

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 141

Error! No text of specified style in document.

PowerFlex Manager

PowerFlex rack Administration - Instructor Guide

Page 142 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Managing System Parameters

MDM Cluster Modes

Cluster Mode Member Description

Three-node • Master MDM Two copies of the repository - can

(default) withstand one MDM failure.
• Slave MDM
• TieBreaker

Five-node • Master MDM Three copies of the repository - can

withstand two MDM failures.
• Two Slave MDM
• Two TieBreaker

Master MDM: The MDM in the cluster controls the SDSs and SDCs. The Master
MDM contains and updates the MDM repository, which is the database that stores
the SDS configuration. The repository also specifies how data is distributed
between the SDSs in the system. This repository is constantly replicated to the
Slave MDMs, so they can take over during a system component failure, such as a
network link. Every MDM cluster has one Master MDM.

Slave MDM: An MDM in the cluster that is ready to assume the role of the Master
MDM if necessary. In a three-node cluster, there is one Slave MDM, thus enabling
a single point of failure. In a five-node cluster, there are two Slave MDMs, thus
enabling for two points of failure. This increased resiliency is a major benefit to
enabling the five-node cluster. If you have five nodes or more, it is the
recommended best practice to deploy a five node cluster.

Monitoring MDM Cluster

The Administrators can view and monitor the PowerFlex MDM cluster from the
PowerFlex Manger or PowerFlex Web UI page.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 143

Error! No text of specified style in document.

PowerFlex Manager

From PowerFlex Manager UI, go to Services>Health and select the cluster you
want to check the health.

PowerFlex Web UI

More details on each MDM are available in the Configuration: MDM Cluster
Settings page.

PowerFlex rack Administration - Instructor Guide

Page 144 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Switching Primary MDM Using CLI - Hands On

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

Reconfiguring MDM Roles Using PowerFlex Manager

PowerFlex Manager enables you to change the MDM role for a node in a
PowerFlex cluster. For example, if you add a node to the cluster, you might want to
switch the MDM role from another node to the new node.

You can launch the wizard for reconfiguring MDM roles from the Services page or
from the Resources page.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 145

Error! No text of specified style in document.

Procedure

• To access the wizard from the Resources page:

i. On the menu bar, click Services.

ii. Select a service that has the PowerFlex Gateway for which you want to
reconfigure MDM roles.

iii. In the right pane, click View Details. The Service Details page is displayed.

iv. On the Service Details page, click Reconfigure MDM Roles. You can also
click Reconfigure MDM Roles on the Node Actions menu on the Services
Details page. The MDM Reconfiguration page is displayed.

• To access the wizard from the Services page:

i. On the menu bar, click Resources.

ii. Select the PowerFlex Gateway for which you want to reconfigure MDM
roles.
iii. In the right pane, click View Details. The Details page is displayed for the
selected PowerFlex Gateway.

iv. On the Details page, click Reconfigure MDM Roles. The MDM
Reconfiguration page is displayed.

• Review the current MDM configuration for the cluster.

• Find each MDM role that you want to reassign and choose the new Host Name
or IP address for the role in the Select New Node for MDM Role drop-down.
You can reassign multiple roles at one time.
• Click Next. The Summary page is displayed.
• Type CHANGE MDM ROLES to confirm your changes.
• Click Finish

PowerFlex rack Administration - Instructor Guide

Page 146 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Configuring Virtual IP Address

Up to four virtual IP addresses can be defined for the MDM

Installation of Virtual IP Address
cluster. SDCs are then mapped to the virtual IP addresses of
the MDM cluster, instead of the static MDM IP addresses.
Installation PowerFlex VMware
Manager CSV file Installation Wizard

Benefits of using Virtual IP include:

• Virtual IP address is always mapped to the active MDM.
• SDCs mapped to Virtual IP address of the MDM cluster
• Maintenance procedures are made simpler.
• If the MDM hosted server is replaced, the SDCs need not be reconfigured

Using Virtual IP is the recommended PowerFlex best practice. Click the link for the
process to configure virtual IP address.

Installation of Virtual IP Address can be done through:

Linux

In new installations on Linux environments, the virtual IP address of the MDM

cluster can be added and mapped using the Installation Manager .csv file.

VMware

In VMware environments, virtual IP addresses are mandatory, and configuration is

performed using the PowerFlex VMware Installation Wizard, in the Configure SVM
stage.

REST API

The REST API can also be used to add virtual IP addresses to the cluster. Always,
a virtual IP NIC placeholder must be mapped to each virtual IP address. Ensure
that there are NICs available for this purpose.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 147

Error! No text of specified style in document.

PowerFlex Manager

Virtual IP addresses are created as a default in PowerFlex manager.

Configuring Virtual IP Address

To configure virtual IP addresses in the vSphere Web plug-in:

1. From the VxFlex OS Systems page, click Actions and select Configure
virtual IP address.
2. In the Configure virtual IP's dialog box, select the network, and enter a virtual IP
address.

PowerFlex rack Administration - Instructor Guide

Page 148 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

License Management

PowerFlex Manager licensing is based on the total number of managed resources.

The standard license is the valid license type, which is a full-access license type.
After uploading an initial license, you can upload subsequent licenses that replace
the existing license.

You can add multiple standard licenses. In that scenario, details of all the licenses
are displayed together in the License Management section on the Virtual Appliance
Management page.

If you try to upload the same standard license second time, you get an error
message stating that License has already been used.

Background Device Scanner

The Background Device Scanner scans the system for errors and fixes the errors
before they can affect the system. The scanner provides SNMP reporting about
errors that are found, and keeps statistics about its operation. Information about
errors is provided in event reports.

The Background Device Scanner has the followings settings available:

• Enable Background Device Scanner: When the scanner is enabled, it uses
the device's internal checksum mechanism to validate the primary and

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 149

Error! No text of specified style in document.

secondary data. If a read succeeds in both devices, no action is taken. If a

faulty area is read, an error is generated.
• Fix Local Device Errors: When this setting is enabled, the system will
automatically clear device errors.
• Compare Data: This is only available if zero padding is enabled. The scanner
calculates and compares the checksums of primary and secondary data. If this
comparison fails, the compare errors counter is increased, and the scanner
attempts to overwrite the secondary device with the data from the primary
device. If this attempt fails, a device error is issued.
• Bandwidth Limit: This setting limits the bandwidth that is allowed for scanning.
This setting should be used carefully. High bandwidth can negatively impact
performance.

The scanning function is enabled and disabled (default) at the Storage Pool level.

Checksum Protection

The checksum feature identifies errors that change the payload during the transit
through the PowerFlex system. PowerFlex protects data in-flight by calculating and
validating the checksum value for the payload at both ends.

There are two types of checksum operations:

PowerFlex rack Administration - Instructor Guide

Page 150 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Read Operation

Application

Validate checksum Send data Read data from SDS Calculate

to application checksum

During read operations, the checksum is calculated when the data is read from the
SDS device. It is validated by the SDC before the data returns to the application. If
the validating end detects a discrepancy, it initiates a retry.

Write Operation

Validate checksum Write

Receive write request Calculate
to storage
checksum
Application

Send write request

During write operations, the checksum is calculated when the SDC receives the
write request from the application. This checksum is validated just before each
SDS writes the data on the storage device. If the validating end detects a
discrepancy, it initiates a retry.

The checksum feature can be enabled through Storage Pool -> Settings ->
General. Pools with Fine Granularity, with or without compression, have persistent
checksum by default.

Important: The checksum feature may have a major impact on

performance and availability. The recommended practice is to keep it
disabled.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 151

Error! No text of specified style in document.

Set Capacity Threshold on Storage Pools

To maintain system performance, some of the system's capacity must remain

available. Using the capacity settings enables administrators to receive alerts prior
to experiencing performance issues. The alerts are important to pay attention to. If
the capacity goes over 95%, the entire array is set to read-only mode.

The Capacity Settings dialog box has the following options:

• Spare Percentage Policy - Should be set at n + 1 (n is the space of one node).
• Set Capacity Alert Thresholds: High - Recommended setting is 80%.
• Set Capacity Alert Thresholds: Critical - Recommended setting is 85%.

PowerFlex rack Administration - Instructor Guide

Page 152 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Data Protection Options

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 153

Error! No text of specified style in document.

Rebuild and Rebalance

PowerFlex Rebuild

PowerFlex maintains user data in a RAID 1 mesh mirrored layout. A RAID 1

approach is used to write the data and ensure that there is always a second copy
of the primary data.

When a failure occurs on a server, device, or network, PowerFlex immediately

initiates a process to protect the data. This process is called Rebuild.

There are two types of rebuild operations.

Forward Rebuild

Forward Rebuild creates another copy of the data on a new server. In this process,
all the devices in the Storage Pool work together, in a many-to-many fashion to
create copies of all the failed storage blocks. Creating copies of all failed storage
blocks ensures a faster rebuild.

Movie:
The web version of this content contains a movie.

Backward Rebuild

Backward Rebuild re-synchronizes the copies created during the Forward Rebuild
process. The re-synchronization is done by passing back the changed data while

PowerFlex rack Administration - Instructor Guide

Page 154 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

the copy was inaccessible during a device or node failure. This process minimizes
the amount of data that is transferred over the network during recovery.

Movie:
The web version of this content contains a movie.

PowerFlex Rebalance

Rebalance is the process of moving data copies across different servers in the
system. It occurs when PowerFlex detects that the user data is not evenly balanced
across the devices in a Storage Pool. In this process, data is moved from the most
used devices to the least used ones.

The rebalance process simplifies PowerFlex management by eliminating long

refresh cycles and making the environment more dynamic and flexible.

The rebalance process is initiated as a result of several conditions.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 155

Error! No text of specified style in document.

Addition

Rebalancing occurs when new devices are added to the pool.

The software automatically accommodates the addition of
new devices by migrating the storage and rebalancing within
the pool.

Removal

When storage or a device is

removed from the pool, the software is intelligent
enough to rebalance the existing data. The software
avoids any data loss within the storage pool.

Combination

PowerFlex can also handle a combination of both addition and

removal of storage and devices by dynamically adapting to the
changing requirements resulting in increased flexibility.

Managing Rebuild and Rebalance

Rebuild and rebalance processes are fully automated, but are

configurable. When managing the performance of the PowerFlex
system, rebuild and rebalance must be considered. There are four
settings that can affect the resources that are used to perform these actions which
may help system performance and affect recovery times after a failure.

Rate Limits

Rate limits can be configured for each Protection Domain:

• Maximum rebuild rate (MB/sec)
• Maximum rebalance rate (MB/sec)
• Total network bandwidth used by PowerFlex (application I/O +
rebuild/rebalance)

PowerFlex rack Administration - Instructor Guide

Page 156 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

For example, if the rebuild rate is limited to 60MB/sec, and the rebalance rate to 30
MB, then concurrently running rebuild jobs and rebalance jobs is limited to
60MB/sec.

Note: When both rebuild and rebalance occur simultaneously, the aggregate
bandwidth that is consumed by both will not exceed the individual maximum for
each type.

Disable or Enable

Rebuild and rebalance can be enabled or disabled for a specific Storage Pool. A
practical example of usage is the application I/O workload is high when new
servers are added to the cluster during the work week. To avoid network
congestion from rebuild and rebalance operations, they can be deferred to the
weekend. The decision to defer rebuilds should be carefully considered since a
rebuild mitigates a protection issue (single copy of data). Deferring rebalance, but
not rebuild, makes more sense in a production environment.

Priority setting

Priorities can be defined for application I/O, rebuild, rebalance and migration
workloads. The administrator can choose one of these prioritization schemes for
each Storage Pool.

NOTE: For a given pool, a different prioritization scheme can be set for rebuild and
rebalance.

• Unlimited

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 157

Error! No text of specified style in document.

• Limit concurrent IO - The number of simultaneously active rebuild and

rebalance jobs can be set for specific storage pools. This depends on the
capabilities of the specific device type. For example, SSD devices can typically
handle more I/O threads in parallel than magnetic disk drives. Once the limit is
reached, the next incoming rebuild or rebalance I/O waits until the completion of
a currently run rebuild or rebalance I/O. This completes the rebuild quickly for
the best reliability, however, there is a risk of host application impact.Limit the
number of concurrent rebuild I/Os per SDS device (default). The rebuild I/Os
are limited to a predefined number of concurrent I/Os. Once the limit is reached,
the next incoming rebuild I/O waits until the completion of a currently executed
rebuild I/O. This will complete the Rebuild quickly for best reliability, however,
there is a risk of host application impact.
• Favor application I/O - Limit the number of IOPs due to rebuild and rebalance
so that the application I/O can be favoured.

Improving Rebuild and Rebalance Performance

Below are more details on network throttling and how to improve rebuild
performance.

Network throttling

Both Rebuild, and Rebalance will compete with the application I/O for the system
resources. This includes network, CPU, and storage media. PowerFlex provides a
rich set of parameters that can control this resource consumption. While the system
is factory-tuned for balancing between expedient rebuild/rebalance and
minimization of the effect on the application I/O, the user has fine-grained control

PowerFlex rack Administration - Instructor Guide

Page 158 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

over the rebuild and rebalance impact. These limits can be modified using the
Network Throttling option for Protection Domains.

Improving Rebuild performance

During Rebuild, it is advisable to enable more bandwidth. High throughput reduces

the time taken for a failed node or failed disk to rebuild.
• Number of Concurrent I/Os per device during rebuild/rebalance: It is
recommended to leave this value at a default of “1” as raising this value can
cause contention during the rebuild and rebalance jobs. By managing various
system-related jobs, client performance can be preserved. However, during
rebuild, it may be advisable to enable more bandwidth, to increase the speed of
the reprotect process, and provide for multiple data copy protections.
• Network throughput is a critical component when designing your PowerFlex
implementation. Throughput factor is important to reduce the amount of time it
takes for a failed node to rebuild. It also reduces the amount of time it takes to
redistribute data in the event of uneven data distribution, and optimize the
amount of I/O a node is capable of delivering. While PowerFlex can be
deployed on a 1-gigabit network, storage performance will likely be bottle-
necked by network capacity.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 159

Error! No text of specified style in document.

Snapshot and Snapshot Policy

Snapshots in PowerFlex

Data protection in PowerFlex is enabled using PowerFlex snapshot feature.

A Snapshot is an instantaneous, point in time copy of the system files of a volume.

It does not include a copy of the storage data. A snapshot gives you a roll-back
configuration in case you must revert to a specific configuration.

With PowerFlex, you can create up to 126 snapshots for MG and FG pools. Out of
these snapshots, 60 snapshots can be policy-managed.

Some of the key features of PowerFlex snapshots include:

• They are thinly provisioned, regardless of the original volumes that could be
either thin or thick.
• When a volume or snapshot is created, snapshot capacity and volume capacity
can be viewed in the Capacity view of the dashboard.
• Write and read operations can be performed on all snapshots, once mapped to
an SDC.
• The newly generated snapshots can be mapped to any SDC in the system.
• Snapshots can either be secured or nonsecured

PowerFlex Web UI > Volumes > volume name > More > Create Snapshot

PowerFlex rack Administration - Instructor Guide

Page 160 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Snapshot Policy

Snapshot policies contain a few attributes and elements, offering the ability to
implement automatically run snapshots for specified volumes based on specified
retention schedules. Up to 60 policy-managed snapshots can be retained per root
volume.

Important: Policies can be created and managed only through

PowerFlex CLI.

Syntax to create a snapshot policy:

scli --add_snapshot_policy [--snapshot_policy_name <NAME>] --

snapshot_creation_cadence <CADENCE> --
number_of_snapshots_per_retention_level <LEVELS> [--paused]
[--read_only_snapshots]

• --snapshot_policy_name <NAME> Snapshot policy name

• --snapshot_creation_cadence <CADENCE> The interval in minutes
between two snapshots in the policy
• --number_of_snapshots_per_retention_level <LEVELS> A comma-
separated list which represents the number of snapshots per retention level.
There are one to six levels, and the first level has the most frequent snapshots.
There can be up to 60 snapshots in all levels totaled (note that two consecutive
levels overlap by one).
• --paused Indicates that the snapshot policy should paused on creation.
• --read_only_snapshots Specify all mappings of the created auto
snapshots to be limited to read-only access.

Example:

scli --add_snapshot_policy --snapshot_policy_name S_Pol_1 --

snapshot_creation_cadence 4 --

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 161

Error! No text of specified style in document.

number_of_snapshots_per_retention_level 6,4,3,2,2,1 --
read_only_snapshots

Secure Snapshot

A secure snapshot sees a snapshot that cannot be deleted for a predefined period.
This feature was in PowerFlex 3.5 to secure data for financial compliance
regulations.

A secure snapshot is distinguished by two key properties.

• Secured flag
• Expiration time

A secure snapshot with a set secured flag and expiration time cannot be manually
removed by a user or automatically deleted by a snapshot policy.

• Secure snapshot is read-only.

• Secure snapshot cannot be overwritten.
• Secure snapshot cannot be deleted as long as the expiration time did not pass.
• A secure snapshot can only be removed by support after following a strict dual-
signature policy.

Creating Secure Snapshot - Hands On

The Online Course Contains an Interaction Here.

This is a hands-on simulation that is part of the lab exercises.

PowerFlex rack Administration - Instructor Guide

Page 162 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

Creating Consistency Group - Hands On

The Online Course Contains an Interaction Here.

This is a hands-on simulation that is part of the lab exercises.

Instructor Note: This is a hands-on simulation that is part of the

lab exercises. The instructor can choose to run this in class as
part of lecture material as well.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 163

Error! No text of specified style in document.

Native Asynchronous Replication

Native Asynchronous Replication

The main component of Native Asynchronous Replication is:

• Storage Data Replicator

With native asynchronous replication, the data is replicated from one cluster to
another cluster. The cluster can be both a replication Source or a Target. The
Storage Data Replicator (SDR) mediates the flow of replicated data traffic between
the source and target cluster.

PowerFlex asynchronous replication uses a journal-based architecture. Journals

reside as volumes in a storage pool. A replication journal is copied from the source
journal buffer by the SDR to the target journal buffer. Once the journals are secured
at the destination, they are deleted from the source, making space for new journals.
Journal-based architecture is not constrained by the maximum number of available
snapshots in the system, or on a given volume.

Replication Consistency Groups (RCG) are elements of PowerFlex storage.

Replication Consistency Groups ensures the consistency of the replicated data and
track replication activity; from determining what gets replicated, to monitoring the
state of replication in general. They can be enabled and disabled as needed when
there are application, compute system, and storage system maintenance and
related downtime.

PowerFlex rack Administration - Instructor Guide

Page 164 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Storage Data Replicator

The Storage Data Replicator (SDR) is a logical storage component that manages
the I/O of replicated logical volumes. It is located alongside SDS on the same node.
From the point-of-view of the SDS, the SDR appears to be an SDC. Conversely to
the SDC, the SDR appears to be an SDS. The SDR mediates the flow of replicated
traffic, and the MDM instructs each of these logical elements where to read and
write data. The SDC uses its metadata cache to determine what data gets passed
to the SDS. It routes only the IO for replicated volumes through the SDR. Non-
replicated IO will not travel through the SDR.

Replication Consistency Group

Replication Consistency Groups (RCG), establish the attribute and behavior of the
replication of one or more volume pairs. One such attribute is the target replication
storage cluster. While a given RCG can replicate to only one target cluster, other
RCGs may replicate to other clusters provided they have exchanged certificates

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 165

Error! No text of specified style in document.

and have been peered. RCGs are flexible. For some use cases, all volumes that
are associated with an application can be assigned to a single RCG. For large
applications, multiple RCGs can be created based on data retention, datatype, or
related application quiescing procedures, enabling read-consistent snapshots when
needed. In general, RCGs are crash-consistent, but related snapshots can be read-
consistent if application quiescing rules have been followed. Concerning Recovery
Point Objectives specified in the RCG configuration, you see that they can be
based on seconds, minutes, hours, or even days.

Note: Before creating RCGs, our replication volumes must exist on both the source
and target systems, and they must be of the same size. They are not required to
reside in the same storage pool type (MG, FG.). As a best practice in PowerFlex
rack volume pair has the same attributes (including zero padding and granularity)
In case there is a need to resize a volume, the target volume should be expanded
first to prevent disruptions in replication. It is not possible to migrate replicated
volumes from one Protection Domain to another, as the replication journals do not
span across Protection Domains.

Click here to learn about the replication attributes defined in the RCG.

Replication Attributes

The following replication attributes are defined in the RCG.

Consistency Mode

The RCG consistency mode defines the way the data is applied at the destination.
When the RCG is set to consistent (which is the default mode), the data is applied
at the destination only when the destination has a consistent image in the journal.

PowerFlex rack Administration - Instructor Guide

Page 166 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

When the mode is set to inconsistent, the data is applied at the destination on
arrival without waiting for a full consistent image.

Inconsistent mode is considered a risky state since a failure while in inconsistent

mode may leave the target volumes with an inconsistent image. Inconsistent mode
is used during a restore or reverse process after failing over. Inconsistent mode is
useful when there has been a long network outage and it is likely that there will be
insufficient room in the destination journal for the I/Os processed during the outage.

Pause Mode

When the RCG is paused, all application I/Os are stored in the source journal.
When replication of the RCG is resumed, the source SDR sends the journal
contents to the target SDR to be applied to the target volumes. You may want to
pause an RCG to handle a network issue between the peer systems or when fixing
a hardware issue.

Freeze Mode

When the RCG is frozen, replication remains active. The application I/Os are
replicated from the source SDR to the target journal but are not applied to the
target volumes. When the RCG is unfrozen, the target SDR starts applying the data
in the target journal to the target volumes. You may want to freeze an RCG in order
to create a snapshot of the replicated volume.

Replication I/O Flow

Native asynchronous replication enables disaster recovery and data protection of

database or application data. It can be used to facilitate having warm standby
servers available for applications to connect to if there is a disaster. It can also be
used for data migration. Once the data is replicated the application can be switched
from the local site to the remote site. Another important use case for replication is
the distribution of workloads to secondary environments including Operational BI,
Analytics, Data Warehousing, Demand Planning, What if analysis. Replication
enables availability of multiple copies of the data to be used in parallel system
tests.

In the source system, for replicated volumes, the SDCs communicate with the
SDR. For nonreplicated volumes, the SDCs communicate directly with the SDSs.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 167

Error! No text of specified style in document.

Physical devices are connected to the SDS. The SDRs serve as a pipeline for I/Os.
Application I/Os (both reads and writes) intended for replication volumes are sent
from an SDC to an SDR. The source SDR packages the data into a consistent
journal barrier and distills them so that only the most recent writes are included.
The source SDR sends the journal barrier over the WAN to the target SDR. At the
target system, the SDR processes the journal barrier and applies it to the volumes.

Asynchronous replication defines a point in time, and ensures that all writes carried
out before that point, and no writes carried out after that point, are copied to the
destination copy. Once all the data is transmitted to the destination, the destination
copy is consistent. PowerFlex 3.5 supports only one to one replication topologies.

Watch the video to understand the replication I/O workflow.

Click here to view a graphical representation of the replication I/O flow.

Movie:

The web version of this content contains a movie.

Replication I/O Flow

Replication Steps

Below are the replication steps.

PowerFlex rack Administration - Instructor Guide

Page 168 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Click here to view a demonstration on native asynchronous replication.

Prepare

In order to prepare the system for replication:

• Configure authentication certificates on source and target systems. This
ensures data security.
• Configure IP addresses for replication at source and destination domains.
• Create a pairing relationship between the MDMs in both domains.
• Assign capacities for replication on both source and target. Guidelines on sizing
journal capacity will be provided.
• Add SDRs to the cluster.

Configure

To configure a system for replication:

• Create an RCG
− Defined by remote protection domain, replication direction, and policies
− Remote protection domain
− Replication direction: assignment of source and target roles
− Replication policies: Recovery Point Objective (RPO)
• Create volume pairs and add them to the RCG

− Volume pair: A volume on the source domain and its copy on the target
domain.
− The volumes must be pre-created and can span different storage pools
− Consistency is kept across all volume pairs in an RCG

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 169

Error! No text of specified style in document.

Initialize

First time initialization is the process where the source volumes data is copied to
the target volumes. At the end of the initialization the target volume image is
consistent. During the process the application's writes continue without interruption.

Steady State

When the system arrives at the steady state:

• On all the SDRs in the source domain
− Writes are accumulated into an interval Journal
− At the end of the collection interval the interval journal is closed (and a new
interval is opened)
− The interval journal data is transmitted to the target
• On all the SDRs in the target domain

− Transmitted data is received to a journal

− Once all the data for the interval journal is received, the data is applied to
the application volume
− Consistency is maintained by the application process

Enable Replication Using PowerFlex Manager

The storage node replication template in PowerFlex Manager can be used to

deploy a storage only service with replication enabled (SDR install). Replication is
inherently a multisystem feature. It should be enabled in both sites and paired.

Steps to enable replication while creating a template

PowerFlex rack Administration - Instructor Guide

Page 170 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Steps to Enable Replication while Creating a Template

1. When you select the replication template, the Enable Replication option is
enabled by default.

2. In the template, under Node > Network settings, select the required VLANs to
enable replication on interface 1 port 1 and port2 . Repeat the same on
interface 2.
3. Under Node > Static routes , select Enabled. Click Add New Static Route.

4. Choose the source and destination VLANs from the menu, and manually enter
the gateway IP address of the source replication VLAN. Repeat for the second
replication VLAN.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 171

Error! No text of specified style in document.

Maintenance Modes

Introduction to Maintenance Modes

The maintenance of PowerFlex is primarily limited to configuration changes of the
physical and virtual layers. When maintenance or planned restart of an SDS is
required, the Maintenance Mode feature can be used to streamline system
operation. Sometimes a node must be taken offline for maintenance in order to
repair or replace or upgrade hardware components or due to rolling PowerFlex
upgrades. Maintenance Mode enables you to restart a server that hosts an SDS,
with minimal impact on the VxFlex Ready Node system. By avoiding the disruption
and effort that is caused by disorderly shutdown, Protection Domain shutdown, and
orderly shutdown. You have several ways to accomplish this and 3.5 introduces a
new maintenance mode that is called Protected Maintenance mode (PMM).

A few conditions must be met before the entering into maintenance modes. The
drop-down has more details.

9. One fault unit

Only one Fault Unit (or stand-alone SDS) can be in Maintenance Mode at any
given time.

10. Adequate Space on SDSs

The server or fault unit in maintenance mode enables writes during

maintenance mode, so there must be adequate space on the remaining SDSs
for the writes that occur.

PowerFlex rack Administration - Instructor Guide

Page 172 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

11. No other SDS in failed state

No other SDS can be in degraded or failed state (force override can be used).

Maintenance Mode Options

Instant Maintenance Mode

In Instant Maintenance Mode (IMM), the node is immediately and temporarily

removed from active participation without building a new copy of the data on other
nodes. Changes are tracked and resynced when the node is available again. The
SDS entering IMM can have degraded capacity.

Movie:
The web version of this content contains a movie.

• Entering IMM
− The node is immediately and temporarily removed from active participation
without building a new copy of the data on other nodes.
− Existing data is, in effect, temporarily frozen.
− A rebuild is not triggered when the node goes offline.
• During IMM
− The system mirrors new writes.
− Changes are tracked for writes that would have affected the node under
maintenance.
− If the node undergoing maintenance fails, IMM assures no data
unavailability (DU). If it did fail, a rebuild of the old data must return the
cluster to normal health.
• Exiting IMM

− The system applies the new writes to the original node.

− It unfreezes and reuses the original data after syncing back the changes.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 173

Error! No text of specified style in document.

Protected Maintenance Mode

Protected Maintenance Mode (PMM) combines the entrance phase of a full

remove/re-add and the exit phase of IMM. Data is mirrored on two nodes. PMM
and IMM cannot be mixed in the same PD at the same time. Within a given PD, all
SDSs concurrently in, or concurrently entering PMM must belong to the same Fault
Set. There are no cross PD dependencies for PMM. The system must be generally
be healthy without any failed capacity or other critical issue. The SDS entering
PMM can have degraded capacity (similar to IMM). Other SDS in the same fault set
may have degraded capacity.

Movie:
The web version of this content contains a movie.

• Entering PMM
− Creates a new, temporary copy of the data by leveraging the many-to-many
rebalance.
− The data on the node being maintained is unchanged. This makes for three
copies, but only two are available.
• During PMM
− Like IMM, PMM does not need to rehydrate the node with data, only
resyncing the deltas back that occurred during maintenance. Recycles the
same code as used by IMM today.
− The system mirrors new writes. Changes are tracked for writes that would
have affected the node under maintenance.
• Exiting PMM

− Changes are tracked and re-synched when the node is available again.

PowerFlex rack Administration - Instructor Guide

Page 174 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Protected Maintenance Mode Using PowerFlex Manager

PowerFlex Manager enables you to put a node in service mode when you must
perform maintenance operations on the node. When you put a node in service
mode, you can specify whether you are performing short-term maintenance or
long-term maintenance work.

To enter the maintenance mode through the PowerFlex Manager, follow these
steps.

1. From the PowerFlex Manager UI, go to the Services page.

2. Click Enter Service Mode under Service Actions.

a. NOTE: The service should have at least three nodes to enter into PMM
using PowerFlex Manager.
3. Select the node that you want to place in Service mode and click Next.
a. NOTE: For an environment with Fault set, PowerFlex Manager can put a
single node or full fault set in protected maintenance mode. For an
environment without Fault sets, PowerFlex Manager can put a four node
minimum in protected maintenance mode.
4. Select Protected Maintenance Mode and click Finish.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 175

Error! No text of specified style in document.

PowerFlex rack Administration - Instructor Guide

Page 176 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

CloudLink Integration

CloudLink Deployment Workflow Using PowerFlex Manager

For PowerFlex 3.5, CloudLink can be deployed using the PowerFlex Manager.
View the CloudLink deployment steps below.

1. Deployment of two CloudLink Center VMs.

• Requires Two VMs for CloudLink HA configuration.

• VM-VM affinity rule set for the CloudLink controller VMs as they must run on
separate ESXi hosts. The affinity rule is set using the vSphere Client.

2. The CloudLink controller VMs are discovered in the PowerFlex Manager

Resources page.

• Once a resource has been deployed, it will be auto-discovered by the

PowerFlex Manager.
• The image shows the VMs discovered and compliant.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 177

Error! No text of specified style in document.

3. Deployment of Presentation Server and PowerFlex Gateway.

• Reuse the available sample templates and configure the settings for the
presentation and gateway VM.

4. Discovery of the Presentation Server and Gateway in the Resources page.

• The image displays the PowerFlex Gateway discovered in the Resources page.

PowerFlex rack Administration - Instructor Guide

Page 178 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

5. Deployment of CloudLink Encryption for the appropriate CloudLink Center.

• Ensure the correct target CloudLink Center is selected in the CloudLink Center
Settings.

Post CloudLink Deployment

Postdeployment of CloudLink Encryption, the PowerFlex Manager

1. Downloads and installs the CloudLink Agent.

2. Ensures that the SDS is connected to the
CloudLink center.
3. Encrypts each drive.
4. Adds the encrypted drives to the SDS.
5. Waits for PowerFlex to complete its rebalance
operation before continuing on to the next SDS.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 179

Error! No text of specified style in document.

VMware Data Protection Options

VMware Snapshots

VMware vSphere enables taking snapshots of individual virtual machines. A

snapshot is a point-in-time copy of a virtual machine. They can be taken while the
virtual machine is either powered on or off, so they are useful for capturing the
state of a virtual machine. Then, if needed, a user can roll back to a snapshot to
revert to the state it was in. This is often used before making large configuration
changes to enable you to undo any changes that were made.

Although a snapshot acts as a copy of the entire virtual machine, only the changes
to the virtual machine are stored. This means that the initial size of a snapshot is
small. The longer a snapshot is retained, the more capacity it uses, since the
number of changes to a VM grows.

PowerFlex rack Administration - Instructor Guide

Page 180 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

To create a snapshot, right-click the virtual machine and select Snapshots > Take Snapshot.

VMware vSphere High Availability (HA)

VMware High Availability (HA) automatically restarts virtual machines on another

host in the cluster.

It is triggered on.
• ESXi host failure or isolation

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 181

Error! No text of specified style in document.

• Storage connectivity loss

• VM not sending heartbeats to VMware Tools

During normal operation, the virtual machine only uses one ESXi host as its
compute source. When there is a problem, the virtual machines go down and then
restart on another host. When virtual machines are restarted on another host, they
reboot.

High Availability requires that the virtual machines use shared storage and that the
hosts are placed in a cluster with a shared management network. PowerFlex rack
already has all hosts in a cluster with a shared management network, and
PowerFlex provides shared storage between all hosts. PowerFlex rack
environment, therefore, can enable the use of VMware vSphere High Availability
feature.

Configuring High Availability

HA can be configured when the cluster is created. In a PowerFlex rack

environment, the cluster is created during the deployment and implementation. To
configure HA, edit the cluster settings and turn on vSphere HA. Select which
components are to be monitored. This action determines what triggers a VM to
restart on another host.

Select Cluster > Configure > vSphere Availability > Edit cluster settings > Turn on vSphere HA

VMware Distributed Resource Scheduler (DRS)

VMware Distributed Resource Scheduler (DRS) automatically optimizes resources

in a cluster. Virtual machines automatically migrate to underutilized hosts or
storage. This way, if one ESXi server is running many virtual machines while others

PowerFlex rack Administration - Instructor Guide

Page 182 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

are not, the VMs can automatically migrate to the other hosts. It is also possible to
configure affinity rules as certain VMs may prefer certain resources, or so the group
of VMs run on the same resource.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 183

Error! No text of specified style in document.

PowerFlex rack-Integrated Data Protection

PowerFlex rack also supports Data Protection options that are provided by other
products.

VMware data protection

VMware provides its own methods of data protection.

• VMware Snapshots
• VMware High Availability (HA)
• VMware Fault Tolerance (FT)

PowerFlex rack Administration - Instructor Guide

Page 184 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Back up with Avamar and Data Domain

Avamar is a backup and recovery software with

integrated data deduplication technology. It
solves the challenges that are associated with
traditional backup, enabling fast, reliable
backup and recovery for remote offices,
VMware environments and datacenter LANs.
Unlike traditional solutions, Avamar reduces
the size of backup data at the source before it
transfers across the network and stores to a
disk. Avamar delivers fast, efficient daily full
backups despite slow or congested
infrastructure, and data encryption for added
security. Avamar uses patented RAIN
technology for high availability, and a scalable
grid architecture enables you to upgrade
capacity and performance when necessary.

This Integrated Data Protection solution connects seamlessly to the PowerFlex

rack network. The pair of access switches of Integrated Data Protection connect
directly to PowerFlex rack aggregation switches. Remote Management Console
connections are direct from the Avamar node or Data Domain to the PowerFlex
rack management switches.

Click here to learn more about the Integrated Data Protection options for
PowerFlex Rack system. You will be redirected to a different course, you can
bookmark it to take the course later.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 185

Error! No text of specified style in document.

RecoverPoint for Virtual Machines

RecoverPoint for Virtual Machines is a software-only

product that protects virtual machines that reside in a
VMware vSphere environment. It provides local and
remote replication capabilities with virtual machine-level
granularity.

RecoverPoint for Virtual Machines targets the VMware

administrator. The RecoverPoint for Virtual Machines
virtual RecoverPoint Appliances (vRPA) are installed in
the VMware vSphere environment. The appliances
provide the RecoverPoint for Virtual Machines plug-in for
the VMware vSphere Web Client.

RecoverPoint for Virtual Machines fully supports all standard RecoverPoint

operations:
• Test copy
• Recover production
• Fail over

PowerFlex rack Administration - Instructor Guide

Page 186 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Monitoring and Troubleshooting

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 187

Error! No text of specified style in document.

Monitoring with PowerFlex Manager

Monitoring Tools

You can provision, maintain, and monitor PowerFlex using the following
management tools:

PowerFlex PowerFlex Manager is a unified management and

Manager automation solution. It discovery and monitors the
PowerFlex Gateway.

PowerFlex Use the PowerFlex Web UI to perform standard

Web UI configuration and maintenance activities, and monitor the
health and performance of the storage system. You can use
the UI to view the entire system and then examine different
elements.

vSphere Plug- The Plug-in enables the VMware admin to perform all the
in monitoring and management operation for PowerFlex within
the VMware environment.

CLI The PowerFlex CLI can be used to perform the entire set of
configure, maintain, and monitor activities in a PowerFlex
system.

REST Gateway A REST API can be used to expose monitoring and provisioning
through the REST interface. The REST gateway is installed as
part of the PowerFlex Installation Manager (IM).

PowerFlex rack Administration - Instructor Guide

Page 188 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Monitoring with PowerFlex Manager

PowerFlex Manager is used to monitor system health monitoring, performance

metrics, storage details for PowerFlex Gateway, Alerting, Lifecycle Management,
RCM Compliance, and Secure Remote Services. PowerFlex Manager stores up to
15 GB of PowerFlex Gateway metrics. Once this threshold is exceeded, the
PowerFlex Manager automatically purges the oldest data to free up space.

When upgrading a service containing a PowerFlex Gateway, PowerFlex Manager

upgrades the PowerFlex Gateway version and the SDSs of all services that are tied
to the PowerFlex Gateway.

The image displays the performance metrics for PowerFlex.

Current IOPS and bandwidth (KB/S) display in the top-left corner of the
Performance tab. The data is automatically refreshed every thirty seconds.
Performance data is available when PowerFlex is deployed to a service.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 189

Error! No text of specified style in document.

PowerFlex Manager Dashboard

The PowerFlex Manager Dashboard provides a quick overview of your PowerFlex

rack system, all in one page. It is a quick and easy way to see if there have been
any errors or warnings throughout the system, without having to log in to multiple
interfaces. PowerFlex Manager gathers information from the switches, nodes,
virtualization components, and PowerFlex, and presents it on the dashboard.

Further down in the Dashboard, you can see the utilization of nodes. In the
example, 42% of the nodes, or 6 out of 14, are in use by a service. The remaining
nodes can be added later if more capacity is needed. They can be configured to
provide compute and storage to the existing service, or to a new service.

The Dashboard also shows PowerFlex storage usage. In the example shown,
there is only one PowerFlex cluster that only has 512 GB of storage provisioned.

Click here to view a video on Monitoring with PowerFlex Manager.

PowerFlex rack Administration - Instructor Guide

Page 190 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Manager Services and Resource Monitoring

PowerFlex Manager allows easy monitoring of Services and Resources through the
PowerFlex Manager UI.

Service Monitoring

From the Services section of PowerFlex Manager, you can view your services.
Selecting a service shows a diagram of all the resources in the service. You can
quickly see each resource along with their statuses. You can see details on each
resource by clicking it. You can then view logs from that resource. Some
maintenance tasks are also available. For example, you can place a node into
Service Mode which places the PowerFlex and VMware services into maintenance
mode.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 191

Error! No text of specified style in document.

vCenter PowerFlex

Nodes

PowerFlex Volumes

Resource Monitoring

A PowerFlex Manager resource is any physical or virtual element in a PowerFlex

appliance. This includes switches, nodes, vCenter servers, and PowerFlex clusters.
You can view all of the resources in the Resources section. This includes both
resources that are currently used by a service and ones that are not in use.

In this view, you can quickly see information about each resource. This includes
whether they are healthy and if they are compliant with the IC level. It also provides
links to the management interfaces of each component, or in the case of a switch,
its IP that you can connect to.

Click the hyperlinks for information on Resource Details and Node Details.

PowerFlex rack Administration - Instructor Guide

Page 192 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Links or IPs to management UI

Health and Compliance status Details

Resource Details

You can view details of a resource by selecting it in the Resources view and
clicking View Details. Here, you can see detailed information about the resource
including performance statistics.

Shown is the details page for a PowerFlex system. It shows its capacity and
historical IOPS data.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 193

Error! No text of specified style in document.

Node Details

The Resources page displays detailed information about all the resources and
node pools that PowerFlex Manager has discovered and inventoried. You can
perform various operations from the All Resources and Node Pools tabs.

Here, you can see that the Resource Details page displays detailed information
about the resource and associated components. Performance details, including
system usage, CPU usage, memory usage, and I/O usage are displayed.
Performance usage values are updated every five minutes.

Compliance Scan

PowerFlex Manager monitors current firmware and software levels and compares
them to the active RCM occurrences, which contains the baseline firmware and
software versions. It shows any deviation from the baseline in the compliance
status of the resources. You can use the PowerFlex Manager to update the servers
to a compliant state. Using PowerFlex Manager, you can choose a default RCM, or
add new RCM. You can update the firmware and software of shared resources
from the Resources page. A firmware or software update on a node that is part of a
cluster is successful only if the node is set to maintenance mode. PowerFlex
Manager sets nodes in a cluster to maintenance mode before performing an
update. To ensure that the node remains in maintenance mode, ensure that there
are other nodes available in the cluster to host the virtual machines of the node
being updated.

You can view RCM compliance by clicking a service in the Services window and
clicking the View Compliance Report button.

PowerFlex rack Administration - Instructor Guide

Page 194 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Update Resources

Compliance Scan Using PowerFlex Manager - Hands On

The online course contains an interaction here.

his is a hands-on simulation that is part of the lab exercises.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 195

Error! No text of specified style in document.

Instructor Note: This is a hands-on simulation that is part of the lab

exercises. It is available in the "Supporting Simulations" folder in the
VLP. The instructor can choose to run this in class as part of lecture
material as well.

Secure Remote Services

Secure Remote Services for Call Home

PowerFlex Manager and PowerFlex connect with Secure Remote Services(SRS) to

transmit encrypted RCM compliance assessment. It also transmits server hardware
and PowerFlex alerts to Dell EMC Customer Support.

If there is an issue or potential issue that requires attention, Customer Support

engineers can connect back in to troubleshoot or repair.

Secure Remote Services with PowerFlex Manager

You can configure PowerFlex Manager to receive and display alerts from
discovered PowerFlex appliance components. The alert connector is available
through PowerFlex Manager. It sends alerts on the health of PowerFlex nodes and
PowerFlex software securely through Secure Remote Services. Secure Remote
Services routes alerts to the Dell EMC support queue for diagnosis and dispatch.

When using the alert connector with Secure Remote Services, critical alerts can
automatically generate service requests. Dell Technologies Services continuously
evaluates and updates which alerts automatically generate service requests.

During node discovery, you can configure iDRAC nodes to automatically send
alerts to PowerFlex Manager. PowerFlex Manager receives SNMP alerts directly
from iDRAC and forwards them to Secure Remote Services.

PowerFlex rack Administration - Instructor Guide

Page 196 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Dell EMC Dell EMC

Services Customer
CUSTOMER
Infrastructure Services

Decryption,
IC and Alerts Analysis
Diagnostic and
PowerFlex appliance Parsing

SERVICE ACTIVITY

Recommendations and Remote Connect-in to Resolve Issues

SNMP Monitoring

SNMP communication can be configured to enable PowerFlex Manager to receive

and forward SNMP traps. PowerFlex Manager can receive SNMP traps from
system devices and forward them to one or more remote network management
systems.

Monitoring PowerFlex appliance

Simple Network Management Protocol (SNMP) is a network management protocol,

which is used for collecting status information from network devices, such as
servers and switches. The SNMP enabled device runs SNMP agent, and
communicates with the SNMP management server to share information about
device status.

All SNMP traps should be directed towards the active SNMP monitoring system of
the customer. This provides proactive alerting for critical and warning level events.
These events include, but are not limited to, hardware failures requiring field
replacement and software faults that could negatively impact the stability of the
system.

SNMP and Syslog Forwarding with PowerFlex Manager

PowerFlex Manager enables users to forward SNMP traps or syslog to local

servers of their selection

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 197

Error! No text of specified style in document.

PowerFlex Manager can be configured to forward syslogs to up to five destination

remote servers.

To configure SNMP, specify the access credentials for the SNMP version you are
using and then add the remote server as a trap destination. PowerFlex Manager
and the network management system use access credentials with different security
levels to establish two-way communication. For SNMPv2 traps to be sent from a
device to PowerFlex Manager, you need to provide PowerFlex Manager with the
community strings on which the devices are sending the traps.

SNMP v2 and v3 support in PowerFlex Manager for SNMP trap forwarding:

• Devices supported using SNMP v2: All resources types
• Devices supported using SNMP v3: iDRACs only at this time

Configuring vCenter to Send SNMP Alerts

Configure vCenter to send SNMP alerts to perform following steps:

1. From a selected host in a vCenter, add the new alarm.
2. Under the General page:
− Add the Alarm name vcesys-supportalarms.
− Select Host as the inventory object to Monitor.
3. Under the Triggers page:

PowerFlex rack Administration - Instructor Guide

Page 198 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

− Choose ANY for conditions that generate trigger.

− Click ‘+” and select specific trigger conditions, for example, CPU usage.
− Click the settings for Operator, Warning, and Critical Condition
4. Under the Actions page:

− Select Send a notification trap as the action.

Alert Connector - Hands On

The web version of this content contains an interactive activity.

Instructor Note: This is a hands-on simulation that is not part of the

lab exercises. The instructor needs to run this in class as part of
lecture material.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 199

Error! No text of specified style in document.

Monitoring the System Using PowerFlex Manager - Hands On

The online course contains an interaction here.

This is a hands-on simulation that is part of the lab exercises.

Instructor Note: This is a hands-on simulation that is part of the lab

exercises. It is available in the "Supporting Simulations" folder in the
VLP. The instructor can choose to run this in class as part of lecture
material as well.

PowerFlex rack Administration - Instructor Guide

Page 200 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Virtual Environment Monitoring

Monitoring Virtual Compute and Network

PowerFlex node cluster vCenter is used to monitor the health of the ESXi hosts
and the clusters to which they belong. All production VMs run in this vCenter and
must be monitored for resource usage, and performance. You can also monitor and
manage virtual networks, such as distributed virtual switches and port groups
settings. The PowerFlex node cluster uses datastores which are also monitored for
capacity usage and performance. These datastores are created on the PowerFlex
storage. vCenter provides some PowerFlex monitoring capability through the
PowerFlex plug in.

vSphere Monitoring

Monitoring Resources

Resource usage is reported on the Summary tab of the vSphere interface.

Resources can be monitored at cluster level, ESXi hosts, or VM level. Check
capacity usage trends, and add capacity as required to avoid capacity problems.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 201

Error! No text of specified style in document.

Monitor Resources

Monitoring Inventory Objects

The vSphere statistics subsystem collects data on the resource usage of inventory
objects. Data on a wide range of metrics are collected at frequent intervals. The
data are processed and archived in the vCenter Server database. Statistical
information can be accessed through command line monitoring utilities or by
viewing performance charts in the vSphere Web Client.

Provides resource utilization and performance chart

Monitoring VMs

The vSphere Web Client lets you look at a virtual machine at a high level with the
Summary tab. It also enables you to monitor a specific aspect of a VM. The
Monitor tab gives you options to look at Issues, Performance, Tasks and

PowerFlex rack Administration - Instructor Guide

Page 202 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Events, Policies, and Utilization. The screenshot displays the recent events that
have occurred on the VM.

Monitor CPU, Memory, and Storage and view VM events

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 203

Error! No text of specified style in document.

VM Performance Monitoring

Continuous high utilization indicates that the VM is resource that is constrained.

Information about CPU utilization on virtual machines is available in the vCenter

Server.

Temporary spikes in CPU usage indicate that you are making the best use of CPU
resources. Consistently high CPU usage might indicate a problem. You can use the
vSphere Web Client CPU performance charts to monitor CPU usage for hosts,
clusters, resource pools, virtual machines, and vApps.

Host machine memory must be at least larger than the combined active memory of
the virtual machines on the host. Memory size of a virtual machine must be larger
than the average guest memory usage. Increasing the virtual machine memory size
results in more overhead memory usage.

Network performance depends on application workload and network configuration.

Dropped network packets indicate a bottleneck in the network. Slow network
performance can be a sign of load-balancing problems. If you suspect that a virtual
machine is network that is constrained, confirm that VMware Tools are installed
and measure the effective bandwidth between the virtual machine and its node
system.

PowerFlex rack Administration - Instructor Guide

Page 204 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

vSphere Distributed Switch

Distributed Switch Health

You can view the status of each host, or node, and its VDS from the vSphere Web
Client. To view VDS health, go to Network, select the VDS in the left pane, select
the Monitor tab, and then Health.

Distributed Switch - Port Groups and Uplinks

Under Networks, Topology provides a high-level view of the distributed switch

layout, its VMkernel port, and uplink status. Here, you can see all of the VMkernel
ports and Virtual Machine ports and their connections. If you click a port, it shows
which uplinks it uses.

The green color of the port plugs show that the port is active. One of the ports is
down in the example shown.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 205

Error! No text of specified style in document.

Port is down

Highlight shows path from

selected vmk to uplinks

Monitor VSAN Health

Monitoring vSAN health is critical for the proper functioning of the PowerFlex
Controller cluster. To validate the health of the PowerFlex Controller cluster,
perform the vSAN health test periodically. Select the cluster, and under the
Monitor tab, select vSAN > Health and then click the Retest button. In a healthy
system, all tests should pass successfully.

Retest Periodically

PowerFlex rack Administration - Instructor Guide

Page 206 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Monitoring vSphere Events and Alerts

vSphere includes a user-configurable events and alarms subsystem. This

subsystem tracks events happening throughout vSphere and stores the data in log
files and the vCenter Server database. This subsystem also enables you to specify
the conditions under which alarms are triggered. This functionality is useful when
you want to be informed, or take immediate action, that is based on events or
conditions.

Events are records of user actions or system actions that occur on objects in
vCenter Server or on a host. Examples of events include license key expiry, VM
power on, or lost host connection. Event data includes details about the event such
as who generated it, when it occurred, and what type of event it is.

Alarms are notifications that are activated in response to an event, a set of

conditions, or the state of an inventory object. Triggered alarms are visible in
several locations throughout the vSphere Web Client. To view all triggered alarms,
click All in the Alarms sidebar panel.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 207

Error! No text of specified style in document.

PowerFlex Monitoring

Monitoring with PowerFlex Web UI

The PowerFlex Web UI is a new HTML based Web UI introduced with PowerFlex
3.5. PowerFlex Web UI enables you to perform many standard maintenance
activities, and monitor the health and performance of the storage system.

PowerFlex Web UI enables you to perform standard configurations and maintain

activities, as well as to monitor the health and performance of the storage system.
You can use the PowerFlex UI to view the entire system, and then examine to
different elements.

The tabs display different views and data that are beneficial to a storage
administrator. You can review the overall status of the system, examine to the
object level, and monitor these objects.

Dashboard

The interface Dashboard displays three sections- Health, Performance, and

Capacity- to monitor the PowerFlex System.

• The Health section provides the system overview and a summary of any major
alerts. As displayed in the UI, the current system is healthy.
• The Performance section provides details of IOPS, Performance graph, used
Bandwidth details, Rebuild, and Rebalance status.
• The Capacity section describes the Usable Capacity, and Data Savings
information.

PowerFlex rack Administration - Instructor Guide

Page 208 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Configuration

The Configuration option provides configuring options for the components of

PowerFlex.

Protection Domains, Fault Sets, SDSs, Storage Pools, Acceleration Pools,

Devices, Volumes, and SDCs are configured here. The Details pane provides
performance and capacity information for the respective components.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 209

Error! No text of specified style in document.

Replication

The Replication option helps the administrator with monitoring the replication
process between systems. As displayed in the image, RCG status, Journal
Capacity, and Bandwidth can be monitored here.

Alerts

The Alerts view provides a list of the alert messages currently active in the system,
in a table format. They can be filtered according to alert severity, and object types
in the system.

MDM

The MDM view in the interface displays the cluster settings, its health state, and its
MDM details.

PowerFlex rack Administration - Instructor Guide

Page 210 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

PowerFlex Events Overview

PowerFlex generates and records events in specific response to changes in

system during normal operation.

As a response to changes that have occurred, an event message is generated

while the system is running. Each event message is associated with a severity
level, which indicates the risk to the system.

The PowerFlex system generates and records events in specific response to

changing conditions within the cluster. Every PowerFlex event has a severity level
that is associated with it that ranges from INFO to CRITICAL, depending on the
condition causing the event to be raised is logged.

Each event has severity level, which can be one of:

• INFO: Events that you should be aware of, but that do not put the system at risk
(No Urgency).
• WARNING: A failure from an acceptable condition (for example user error), but
can also indicate a possible failure.
• ERROR: An error alarm is raised by the system. This error requires your
attention. The system is stable, but could be degraded.
• CRITICAL: A major error alarm is raised by system. The system requires
immediate attention.

Critical events require immediate user intervention. They indicate a data

unavailability condition that the PoweFlex cluster cannot recover from without
explicit administrative actions for recovery.

Warning and Error events may be transient conditions, or they may require admin
intervention.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 211

Error! No text of specified style in document.

Event Record Structure

Shown here is the structure of PowerFlex events as recorded in the system. Every
PowerFlex event has six distinct fields: ID, Date, Name, Severity, Message, and
Extended. These fields are selected for a particular event as displayed by the
showevents.py command. These events are decoded in a similar manner within the
Web UI.

Every PowerFlex event has six distinct fields:

• Identifying number for the event
• Date of the event
• Name of the event, searchable in the KnowledgeBase
• Severity level for the event
• Message reported by the event
• Extended data, when additional information is available

The following is a breakdown of the event according to the fields in the event
record (as described above):

View Events Locally on Primary MDM

Events and Alerts are available in the Web UI. Events may also be viewed when
logged in on the primary MDM using the showevents.py script that is provided as
part of the PowerFlex installation. Shown here is the Web UI with an example of
the scli command syntax. The MDM stores the events in a persistent, private
database file, and also periodically archives them. You can use the GREP
command to search for specific errors or content within the event logs.

PowerFlex rack Administration - Instructor Guide

Page 212 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Alert Analysis

This example shows the recommended action that is documented in the Monitoring
Guide for the alert “CAPACITY_UTILIZATION_ABOVE_CRITICAL_THRESHOLD."
The Monitor Dell EMC PowerFlex v3.5 Guide documents every possible alert by its
uniquely identifying Name field. For each type of alert, it indicates the
recommended action.

PARAMETER DESCRIPTION

Alert Message Capacity utilization above critical threshold

in PowerFlex
Web UI

Alert Message CAPACITY_UTILIZATION_ABOVE_CRITICAL_THRESHOLD

in REST

Alert Message MDM.Storage_Pool.Capacity_Utilization_Above_Critical_Threshol

in SNMP Trap d

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 213

Error! No text of specified style in document.

Alert Code (for SIO02.04.0000003

Secure
Remote
Services)

Severity 5 (Critical)

Recommende The capacity utilization of the Storage Pool is reaching a critical

d Action threshold. Remove unneeded volumes and snapshots, if possible,
or add physical storage.

Forward PowerFlex Events to Syslog

PowerFlex events may optionally be forwarded to a remote syslog server. This

enables for convenient monitoring in data centers that have standardized on syslog
as a mechanism to aggregate logs from all applications. Events are forwarded to
local or remote syslog server with the scli --start_remote_syslog
command.

PowerFlex rack Administration - Instructor Guide

Page 214 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Hardware Monitoring

Server Hardware Monitoring Using iDRAC

The iDRAC interface provides high-level health status of the various system
components.

iDRAC Dashboard Monitoring

The Dashboard indicates that there are no health issues with the server using
green checkmarks. You can click each component to find more details.

The Dashboard also provides basic information about the system including model,
service tag, iDRAC MAC address, BIOS, and firmware version.

You can also launch a console session from the Dashboard. Power controls are
available here in the blue button under the Dashboard title. The tabs on the top of
the home page take you to specific details based on which action you would like to
perform.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 215

Error! No text of specified style in document.

System Information

From the System view, iDRAC allows you to monitor different hardware
components such as Batteries, CPU, and Power Supplies. You can drill down
various components to find more information.

For example, information about fans and system temperature can be found under
Cooling. If a fan has an issue, its status goes to a warning or critical state. There
are similar details for the memory, network devices, and other components.

PowerFlex rack Administration - Instructor Guide

Page 216 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Storage Monitoring

Storage is examined under its own tab where you can see the high-level status of
the physical disks and drill down into detail on each device. If a device has an
issue, the status will change color.

Configuring iDRAC Alerts

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 217

Error! No text of specified style in document.

You can configure how iDRAC handles different alerts on the Configuration,
System Settings page.

For example, you may want some alerts to generate an email or SNMP trap. Some
events can even be configured to perform an action when they occur, such as an
automatic reboot of the server.

To use email and SNMP, you must configure their settings, such as the SMTP
server and email address information, in the SMTP (Email) Configuration selection.

Configuring SNMP for iDRAC and vCenter - Hands On

The online course contains an interaction here.

This is a hands-on simulation that is part of the lab exercises.

Instructor Note: This is a hands-on simulation that is part of the lab

exercises. It is available in the "Supporting Simulations" folder in the
VLP. The instructor can choose to run this in class as part of lecture
material as well.

Switch Monitoring

Check Interfaces on Physical Switch

Here is an example from a switch showing brief information about the interfaces on
this switch. show interface brief is a good first command to run and see
attributes, such as Status and Speed. You can see the assigned VLAN, its access
mode – whether its access or trunk mode and port speed. Notice that the 40G trunk
is used to connect to other switches. You can also see the reason that a port is
down. When you see Administratively down, it means that the admin set the port to
down or shutdown. The only way this port can be active again is if the admin
purposely activates it with the no shutdown attribute on the port.

PowerFlex rack Administration - Instructor Guide

Page 218 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Check VLAN Status on Switch

Showing the VLAN can provide a high-level view of VLAN definition. This example
is from an access switch. Notice there are port channels in use here, indicated as
Po in the Ports column. To get more details on the port channels, you can run a
show port-channel command.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 219

Error! No text of specified style in document.

Switch Monitoring - Hands On

Instructor Note: This is a hands-on simulation that is not part of the

lab exercises. The instructor needs to run this in class as part of
lecture material.

SSH to the IP address of the access switch and run the below mentioned
commands.

The web version of this content contains an interactive activity.

The online course contains an interaction here.

This is a hands-on CLI Walkthrough experience that is not part of the lab
exercises.

Examining the Health of the PowerFlex rack System - Hands On

The online course contains an interaction here.

This is a hands-on simulation that is part of the lab exercises.

Instructor Note: This is a hands-on simulation that is part of the lab

exercises. It is available in the "Supporting Simulations" folder in the
VLP. The instructor can choose to run this in class as part of lecture
material as well.

PowerFlex rack Administration - Instructor Guide

Page 220 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Troubleshooting PowerFlex rack

Log Collection

Log Collection Using PowerFlex Manager

Application Logs

PowerFlex Manager provides an activity log of user and system-generated actions

to use for troubleshooting activities. By default, log entries display in order of
occurrence. The logs contain information for PowerFlex Manager application
activities. Check component-level logs when a troubleshooting bundle is generated.

Settings > Logs

The application log entries can be exported to a comma-delimited CSV file for
troubleshooting using the Export All option. The Purge option allows deleting log
entries which are based on date and severity.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 221

Error! No text of specified style in document.

Troubleshooting Bundle

Settings > Virtual Appliance Management > Generate Troubleshooting Bundle

A troubleshooting bundle is a compressed file that contains logging information for

PowerFlex Manager managed components. If necessary, download the bundle and
send it to Dell EMC support for issue debugging.

The troubleshooting bundle includes the following logs:

• PowerFlex Manager application
• Alert connector
• PowerFlex Gateway
• iDRAC life cycle
• Dell EMC Networking switch
• Cisco Nexus switch
• VMware ESXi

If using Secure Remote Services, Send to Configured Secure Remote Services

is selected by default. If not using Secure Remote Services, select Download
Locally to download the troubleshooting bundle to a local file.

PowerFlex rack Administration - Instructor Guide

Page 222 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

VMware Logs

VMware vCenter Logs

To troubleshoot the virtual environment, you need to access the VMware support
log bundle.

To view vCenter Server logs, select the vCenter Server and navigate to Monitor >
System Logs. Also, you can right-click the ESXi host or a VM and select Export
System Logs to start the Export Logs wizard.

Generally, you do not need performance data. There are more options under some
of the selection choices. Under the Storage selection, you have different elements
that are related to vSAN. If you are working in the PowerFlex cluster, you do not
need to select vSAN.

VMware Logs Using CLI

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 223

Error! No text of specified style in document.

On the VCSA virtual machine, the vc-support.sh script can be run to collect the
vCenter log bundle. It records all logs and the information from the VCSA until the
time of the collection. This script creates a vcsupport.zip file in the /root directory of
the vCSA. Either use SCP to export the generated support bundle to another
location or download from https://<VCSAIP>:443/appliance/<support-bundle>.tgz
using root credentials.

The vm-support command can be run on the ESXi nodes to generate the
vSphere log bundle. The bundle is displayed in /var/tmp, /var/log, or the current
working directory. The vSphere bundle is the standard vmsupport bundle that is
collected in typical ESXi troubleshooting. The bundle contains all log files for a
specific node. This command creates a .tgz file that contains many log files related
to the ESXi node. You can also download the logs remotely with the URL:
https://<esxihost>/cgi-bin/vm-support.cgi.

Another important script for debugging or re-creating the system with the collected
data is the reconstruct.sh script file. This file is created in the root directory of
the support bundle. Certain commands in vm-support generate a large file. This file
consumes more resources and is likely to result in a timeout error or takes a
considerable amount of time to execute. To control the creation of larger files, the
reconstruct.sh file breaks down the larger file into fragments when added into
the vm-support bundle. Upon completion of the support tool, users can re-create
the larger file by running the reconstruct.sh file above the extracted bundle
directory.

PowerFlex rack Administration - Instructor Guide

Page 224 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Access Logs for Storage-Only Nodes

Linux log files can be collected using the emcgrab procedure. This procedure is a
comprehensive collection of key elements from the Linux storage only node. Log in
to the Linux console with the root user id and retrieve logs using the scp command.
Collect the Linux operating system logs from the /var/log directory. Alternatively,
EMC Grab Utility can also be used to collect the log files. For more information,
search for EMC Grab at https://www.dell.com/support/home.

Log Collection with PowerFlex Installer

For a PowerFlex in VMware deployment, the vSphere plug-in automatically creates

a separate PowerFlex VM to host the Installation Manager(IM) and PowerFlex
gateway. The Installation Manager can be launched from the vSphere Web Client
or using the IP address of the installation manager.

• Log in to Installation Manger, click Maintain tab.

• Enter the MDM and LIA credentials.
• Select the Retrieve system topology button.
• Click System Logs Collection, and select Collect Logs.
• In the open, reenter the MDM password and click Collect Logs.
• When the process completes, click Download logs to download the log files.
• Click Mark operation completed.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 225

Error! No text of specified style in document.

Click here to watch a video demonstrating the procedure to download logs

from PowerFlex Installation Manager.

Log Collection Through vSphere PowerFlex Plug-In

PowerFlex installation logs can be collected using the Show server log option in the
vSphere web client. Use copy and paste to save the information to a file.
Troubleshoot any PowerFlex installation issues with the server log in the
PowerFlex plug-in.

PowerFlex rack Administration - Instructor Guide

Page 226 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

1:

Collect Server Logs Using iDRAC

System Events can be downloaded from the Maintenance section after logging into
iDRAC. Lifecycle Log files can be viewed online.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 227

Error! No text of specified style in document.

Troubleshooting PowerFlex Manager

Back Up and Restore PowerFlex Manager

PowerFlex rack Administration - Instructor Guide

Page 228 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Performing a backup saves all user-created data to a remote share from which it
can be restored. Perform frequent backups to guard against data loss and
corruption.

Best practices around backups and restores are as follows:

1. Perform a backup and snapshot before performing PowerFlex Manager

appliance upgrades.
2. Anytime changes are made in the configuration of PowerFlex Manager
(resources, services, templates, or settings) you should take a backup of the
PowerFlex Manager appliance.
3. If you perform a restore of a PowerFlex Manager backup to a new appliance,
you should perform a new backup of the appliance so you have a new recover
point. (not a snapshot.)
4. Backups can vary in size that are based the environment and should be sent to
a location with plenty of space.

If you back up a PowerFlex Manager virtual appliance with a working alert

connector configuration and restore that backup onto a different IP address, the
alert connector comes up in an error state. The Secure Remote Services gateway
allows communication on only the original IP address. In this case, you must
deregister the alert connector after restoring the backup, and then re-register it.

The Backup and Restore page displays information about the last backup
operation that was performed on the PowerFlex Manager virtual appliance.
Information in the Settings and Details section applies to both manual and
automatically scheduled backups.

Important: To restart the PowerFlex Manager appliance, run the

sudo shutdown -r now command.

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 229

Error! No text of specified style in document.

General Service Deployment Troubleshooting

Administrators can access the Help section in the PowerFlex Manager

interface for any feature details.

Listed are steps to locate the deployment logs and log examples:
• To locate the deployment log directory:
− You can find the service/deployment ID from the URL in the address bar.

− Service logs are located in /opt/Dell/ASM/deployments/[DEPLOYMENT_ID].

• The deployment.json file is the template input to the service.
• The deployment.log is a general log file of service progress.
• The exception.log file includes the final exception if the service fails. It can
create specific exceptions for specific devices. Here are some failures that you
might see in the exception log:

− "The switch connectivity for rackserver-7nn9mr2 cannot be determined."

This error means that the switch was not discovered in PowerFlex Manager
resources.
− "The node switch connectivity cannot be determined because LLDP is not
enabled on any of the discovered switches." This error means that LLDP
was not enabled when the switches were configured.
− "The sudo Puppet asm process_node has failed." This error means that the
Puppet configuration has failed. Check the puppet.out file for further
troubleshooting.

PowerFlex rack Administration - Instructor Guide

Page 230 © Copyright 2020 Dell Inc.

 Error! No text of specified style in document.

Online Help

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 231

Error! No text of specified style in document.

You Have Completed This eLearning.

Click the Save Progress and Exit button below to record this
eLearning as complete.
Go to the next eLearning or assessment, if applicable.

PowerFlex rack Administration - Instructor Guide

Page 232 © Copyright 2020 Dell Inc.

Glossary
Acceleration Pools
Acceleration Pools are a group of acceleration devices that are added to a
Protection Domain to accelerate Storage Pool performance.

Network configuration details

In PowerFlex Manager, you can configure a PowerFlex cluster to use two data
networks (for a Hyperconverged node with a shared configuration) or four data
networks (for a Storage Only node with a two-layer dedicated configuration). In a
shared two network configuration, both SDC, and SDS communications share the
same SIODATA1 (152) and SIODATA2 (160) VLANS. For a two network
configuration, you need two PowerFlex Data networks that are defined within
PowerFlex Manager. In a dedicated four network configuration, the SDC to SDC
communications go through the SIODATA1 (152) and SIODATA2 (160) VLANS,
whereas the SDS to SDS communications go through the SIODATA3 (168) and
SIODATA4 (176) networks. For a four network configuration, you need two
PowerFlex Data (SDC Traffic Only) and two PowerFlex operating system Data
(SDS Traffic Only) networks that are defined within PowerFlex Manager.

Protection Domain
A Protection Domain is a group of nodes (servers) or SDSs that provide data
isolation, security, and performance benefits. A node (with SDS) can only
participate in one Protection Domain at a time.

RPO
RPO defines the maximal data loss, in time units, that the client is willing to lose.
The length of the data collection and data transmission intervals is bounded by the
RPO. In PowerFlex release 3.5, the smallest RPO offered is 30 seconds. Over
time, it is possible this might reduce in subsequent releases.

Storage Pool
A storage pool is a subset of physical storage devices in a Protection Domain.
Each storage device belongs to only one Storage Pool. When a PowerFlex volume
is configured, the volume contents are distributed over all the devices residing in
the same Storage Pool.

VMware Snapshots

PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 233

VM snapshots are a point-in-time copy of a virtual machine. They can be taken
while the virtual machine is either powered on or off, so they are useful for
capturing the state of a virtual machine.

VMware Fault Tolerance (FT)

VMware Fault Tolerance can be enabled for individual virtual machines to provide
zero (or near-zero) VM downtime upon a host failure. It works by creating a
secondary copy of the virtual machine on another ESXi host of the cluster. This
secondary copy has its own set of virtual machine files and memory, which is kept
synchronized with the primary virtual machine.

VMware High Availability (HA)

VMware High Availability (HA) automatically restarts virtual machines on another
host in the cluster if a failure occurs. For example, if an ESXi host goes down or
becomes isolated, another ESXi host accesses the virtual machine files on shared
storage and restarts the virtual machine.

PowerFlex rack Administration - Instructor Guide

Page 234 © Copyright 2020 Dell Inc.

 PowerFlex rack Administration - Instructor Guide

© Copyright 2020 Dell Inc. Page 235