0% found this document useful (0 votes)

1K views238 pages

ONTAP Cluster Fundamentals

Ontap cluster fundamentals -9.0

Uploaded by

saravananthangaraj

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

1K views238 pages

ONTAP Cluster Fundamentals

Ontap cluster fundamentals -9.0

Uploaded by

saravananthangaraj

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 238

ONTAP Cluster Fundamentals

© 2018 NetApp, Inc. All rights reserved. Legal Notices

Welcome to ONTAP Cluster Fundamentals.

1
The ONTAP Cluster Fundamentals course:
▪ Is for cluster administrators of any experience level
▪ Is divided into five modules:
▪ Clusters
▪ Management
Welcome ▪ Networking
▪ Storage Virtual Machines
▪ Maintenance

▪ Is followed by a final assessment

The ONTAP Cluster Fundamentals course is written for cluster administrators of any
experience level. The course is divided into five modules, with each module based on
a specific topic. The course is followed by a final assessment.

2
ONTAP Compliance
Solutions Administration
ONTAP Data Protection
Fundamentals
ONTAP Data Protection
Administration

ONTAP SAN ONTAP SAN

Welcome Fundamentals Administration

ONTAP SMB
Administration
ONTAP NAS
Fundamentals
ONTAP NFS
Administration

ONTAP Cluster ONTAP Cluster

Fundamentals Administration

Foundational Intermediate

Each course of the ONTAP 9 Data Management Software training focuses on a

particular topic. You build your knowledge as you progress up the foundational
column, so you should take the fundamentals courses in the order shown. Likewise,
you build your knowledge as you progress up the intermediate column. The
foundational courses are prerequisites for the intermediate courses. The courses are
color coded to enable you to identify the relationships. For example, the ONTAP NAS
Fundamentals, ONTAP NFS Administration, and ONTAP SMB Administration focus
on NAS.

The location marker indicates the course that you are attending. You should complete
this course before you attend the ONTAP Cluster Administration course.

3
How to Complete This Course
ONTAP Cluster Fundamentals Pre-Assessment
▪ If you achieved 80% or greater:
▪ Review any of the ONTAP Cluster Fundamentals modules (optional)
▪ Take the final assessment
Instructions ▪ If you received a list of recommended course modules:
▪ Study the recommended course modules, or study all course modules
▪ Take the final assessment

When you completed the ONTAP Cluster Fundamentals Pre-Assessment, if you

achieved 80% or greater on all the modules, you are welcome to review any of the
ONTAP Cluster Fundamentals modules, or you can go directly to the final
assessment.
If you did not achieve 80% or greater on all the modules, you received a list of
recommended course modules. At a minimum, you should study the recommended
course modules, but you are encouraged to study all five. Then take the final
assessment to complete the course.

4
ONTAP Cluster Fundamentals:
Clusters

© 2018 NetApp, Inc. All rights reserved. Legal Notices

Welcome ONTAP Cluster Fundamentals: Clusters.

5
1. Clusters
2. Management
3. Networking
4. Storage Virtual Machines
Course
5. Maintenance
Modules

The ONTAP Cluster Fundamentals course has been divided into five modules, each
module based on a specific topic. You can take the modules in any order. However,
NetApp recommends that you take Clusters first, Management second, Networking
third, Storage Virtual Machines fourth, and Maintenance fifth.

This module was written for cluster administrators and provides an introduction to the
concept of a cluster.

6
This module focuses on enabling you to do the following:
▪ Identify the components that make up a cluster
▪ Describe the cluster configurations that are supported
▪ Create and configure a cluster
About This
Module ▪ Describe the physical storage components
▪ Describe the Write Anywhere File Layout (WAFL) file system

This module identifies and describes the components that make up a cluster. The
module also describes the supported cluster configurations and details the steps that
are required to create and configure a cluster. Then the module discusses the
physical storage components and the Write Anywhere File Layout file system, also
known as the WAFL file system.

7
NetApp ONTAP Is the Foundation for Your Data Fabric

Departments or
Remote Offices
Data Mobility

Data Fabric Off-Premises

Clouds
Seamless Data Management
On-Premises
Data Center

Data Fabric powered by NetApp weaves hybrid cloud mobility with uniform data
management.

Data Fabric seamlessly connects multiple data-management environments across

disparate clouds into a cohesive, integrated whole. Organizations maintain control
over managing, securing, protecting, and accessing data across the hybrid cloud, no
matter where the data is located. IT has the flexibility to choose the right set of
resources and the freedom to change the resources whenever necessary. NetApp
works with new and existing partners to continually add to the fabric.

For more information about Data Fabric, see the Welcome to Data Fabric video. A link
to this video is available in the Resources section.

8
Lesson 1
Cluster Components

Lesson 1, Cluster Components.

9
Harness the Power of the Hybrid Cloud

▪ Simplify data management for

any application, anywhere
▪ Accelerate and protect data across
the hybrid cloud
▪ Future-proof your data infrastructure

This lesson introduces NetApp ONTAP 9 data management software and the
components that make up a cluster.

A basic knowledge of the components helps you to understand how ONTAP can
simplify the transition to the modern data center.

10
Clusters

Cluster
interconnect
All Flash
FAS
FAS

For product specifications, see the

Hardware Universe:
hwu.netapp.com

You might be wondering, “What exactly is a cluster?” To answer that question, this
lesson examines the components individually, but begins with a high-level view.

A cluster is one or more FAS controllers or All Flash FAS controllers that run ONTAP.
A controller running ONTAP is called a “node.” In clusters with more than one node, a
cluster interconnect is required so that the nodes appear as one cluster.

A cluster can be a mix of various FAS and All Flash FAS models, depending on the
workload requirements. Also, nodes can be added to or removed from a cluster as
workload requirements change. For more information about the number and types of
nodes, see the Hardware Universe at hwu.netapp.com. A link is provided in the
module resources.

11
Nodes
What a node consists of:
▪ A FAS or All Flash FAS controller running
ONTAP software:
▪ Network ports
▪ Expansion slots
Controller ▪ Nonvolatile memory (NVRAM or NVMEM)

▪ Disks

Disk Shelf For product specifications, see the

Hardware Universe.
Node

A node consists of a FAS controller or an All Flash FAS controller that is running
ONTAP software. The controller contains network ports, expansion slots, and
NVRAM or NVMEM. Disks are also required. The disks can be internal to the
controller or in a disk shelf.

For information about specific controller models, see the product documentation on
the NetApp Support site, or see the Hardware Universe.

12
High-Availability Pairs
FAS8060 with an internal
interconnect ▪ Characteristics of high-availability (HA) pairs:
▪ Two connected nodes that form a partnership
▪ Connections to the same disk shelves
▪ Ability of surviving node to take control of failed
partner’s disks

Nodes 1 and 2 ▪ Components of HA pair connections:

▪ HA interconnect
▪ Multipath HA shelf connectivity
▪ Cluster interconnect connectivity
Disk Shelf 1

Disk Shelf 2

In multinode clusters, high-availability (HA) pairs are used. An HA pair consists of two
nodes that are connected to form a partnership. The nodes of the pair are connected to
the same shelves. Each node owns its disks. However, if either of the nodes fails, the
partner node can control all the disks, its own and its partners.

The controllers in the nodes of an HA pair are connected either through an HA

interconnect that consists of adapters and cables or through an internal interconnect. In
this example, the FAS8060 model uses an internal interconnect. The nodes must be
connected to the same shelves using redundant paths. The nodes also need to be
connected to a cluster interconnect, even if the cluster is composed of only one HA pair.

13
Networks
▪ Cluster interconnect:
▪ Connection of nodes
▪ Private network

▪ Management network:
▪ For cluster administration
▪ Management and data may be on a shared
Ethernet network

▪ Data network:
Management Network ▪ One or more networks that are used for data
access from clients or hosts
▪ Ethernet, FC, or converged network
Data Network

Clusters require one or more networks, depending on the environment.

In multinode clusters, nodes need to communicate with each other over a cluster
interconnect. In a two-node cluster, the interconnect can be switchless. When more
than two nodes are added to a cluster, a private cluster interconnect using switches is
required.

The management network is used for cluster administration. Redundant connections

to the management ports on each node and management ports on each cluster
switch should be provided to the management network. In smaller environments, the
management and data networks might be on a shared Ethernet network.

For clients and host to access data, a data network is also required. The data network
can be composed of one or more networks that are primarily used for data access by
clients or hosts. Depending on the environment, there might be an Ethernet, FC, or
converged network. These networks can consist of one or more switches, or even
redundant networks.

14
Ports and Logical Interfaces

Logical Logical interface (LIF) smv1-mgmt smv1-data1

Virtual LAN (VLAN) a0a-50 a0a-80

Virtual
Interface group a0a

Physical Port
e2a e3a

Nodes have various physical ports that are available for cluster traffic, management
traffic, and data traffic. These ports need to be configured appropriately for the
environment.

Ethernet ports can be used directly or combined by using interface groups. Also,
physical Ethernet ports and interface groups can be segmented by using virtual
LANs, or VLANs. Interface groups and VLANs are called virtual ports, and virtual
ports are treated similarly to physical ports.

A logical interface, or LIF, represents a network access point to a node in the cluster.
A LIF can be associated with a physical port, an interface group, or a VLAN to
interface with the management network or data network.

15
ONTAP Storage Architecture

Dynamic Virtualization Engine

Files and LUNs

Logical Layer FlexVol Volumes

Aggregate

Physical Layer
RAID Groups of Disks

The ONTAP storage architecture uses a dynamic virtualization engine, where data
volumes are dynamically mapped to physical space.

Disks are grouped into RAID groups. An aggregate is a collection of physical disk
space that contains one or more RAID groups. Each aggregate has a RAID
configuration and a set of assigned disks. The disks, RAID groups, and aggregates
make up the physical storage layer.

Within each aggregate, you can create one or more FlexVol volumes. A FlexVol
volume is an allocation of disk space that is a portion of the available space in the
aggregate. A FlexVol volume can contain files or LUNs. The FlexVol volumes, files,
and LUNs make up the logical storage layer.

16
Physical Storage
▪ Disk:
▪ Disk ownership can be assigned to one controller.
▪ A disk can be used as a spare or added to a
RAID group.

▪ RAID group:
▪ A RAID group is a collection of disks.
▪ Data is striped across the disks.

▪ Aggregate:
▪ One or more RAID groups can be used to form
an aggregate.
▪ An aggregate is owned by a one controller.

There are three parts that make up the physical storage on a node.

When a disk enters the system, the disk is unowned. Ownership is automatically or
manually assigned to a single controller. After ownership is assigned, a disk will be
marked as spare until the disk is used to create an aggregate or added to an existing
aggregate.

A RAID group is a collection of disks across which client data is striped and stored.

To support the differing performance and data sharing needs, you can group the
physical data storage resources into one or more aggregates. Aggregates can contain
one or more RAID groups, depending on the desired level of performance and
redundancy. Although aggregates can be owned by only one controller, aggregates
can be relocated to the HA partner for service or performance reasons.

17
Revised Slide 15

Logical Storage
▪ Storage virtual machine (SVM):
▪ Container for data volumes
Data
LIF ▪ Client data is accessed through a LIF

Client Access ▪ Volume:

SVM with FlexVol ▪ Logical data container for files or LUNs
Volumes
▪ ONTAP provides three types of volumes:
FlexVol volumes, FlexGroup volumes, and
Infinite volumes

▪ LIF:
▪ Representation of the network address that
Cluster is associated with a port
▪ Access to client data

A storage virtual machine, or SVM, contains data volumes and logical interfaces, or
LIFs. The data volumes store client data which is accessed through a LIF.

A volume is a logical data container that might contain files or LUNs. ONTAP software
provides three types of volumes: FlexVol volumes, FlexGroup volumes, and Infinite
volumes. Volumes contain file systems in a NAS environment and LUNs in a SAN
environment.

A LIF represents the IP address or worldwide port name (WWPN) that is associated
with a port. Data LIFs are used to access client data.

18
SVM with FlexVol Volumes
▪ FlexVol volume:
Qtree
Q3
Data ▪ Representation of the file system in a
LIF
Q2
Q1
NAS environment
Client Access ▪ Container for LUNs in a SAN environment
LUN Data
LIF ▪ Qtree:
SVM Host Access ▪ Partitioning of FlexVol volumes into
smaller segments
▪ Management of quotas, security style, and
CIFS opportunistic lock (oplock) settings

▪ LUN: Logical unit that represents a

Cluster SCSI disk

An SVM can contain one or more FlexVol volumes. In a NAS environment, volumes
represent the file system where clients store data. In a SAN environment, a LUN is
created in the volumes for a host to access.

Qtrees can be created to partition a FlexVol volume into smaller segments, much like
directories. Qtrees can also be used to manage quotas, security styles, and CIFS
opportunistic lock settings, or oplock settings.

A LUN is a logical unit that represents a SCSI disk. In a SAN environment, the host
operating system controls the reads and writes for the file system.

19
New Slide 17

FlexGroup Volumes
▪ A scale-out NAS container constructed from a group of FlexVol volumes,
which are called “constituents.”
▪ Constituents are placed evenly across the cluster to automatically and
transparently share a traffic load.

FlexGroup volumes provide

the following benefits:
▪ High scalability
Essentially unlimited
▪ Performance
Consistently low latency
▪ Manageability
Visually the same as FlexVol volumes /FlexGroup

In addition to containing FlexVol volumes, an SVM can contain one or more

FlexGroup volumes. A FlexGroup volume is a scale-out NAS container that leverages
the cluster resources to provide performance and scale. A FlexGroup volume
contains a number of constituents that automatically and transparently share a traffic
load.

FlexGroup volumes provide several benefits:

• High scalability: The maximum size for a FlexGroup volume in ONTAP 9.1 and
later is 20 PB, with 400 billion files on a 10-node cluster.
• Performance: FlexGroup volumes can leverage the resources of an entire cluster
to serve high-throughput and low-latency workloads.
• Manageability: A FlexGroup volume is a single namespace container that enables
simplified management that is similar to the management capability provided by
FlexVol volumes.

For more information about FlexGroup volumes, see the Scalability and Performance
Using FlexGroup Volumes Power Guide.

20
SVM with Infinite Volume
▪ Infinite Volume:
▪ One scalable volume that can store up to 2
Infinite billion files and tens of petabytes of data
Data
LIF ▪ Several constituents
Volume
Client Access
▪ Constituent roles:
SVM ▪ The data constituents store data.
▪ The namespace constituent tracks file
names, directories, and the file's physical
data location.
D NS D D D D D ▪ The namespace mirror constituent is a
D D D D D M D
data protection mirror copy of the
Cluster namespace constituent.

An SVM can contain one infinite volume. An infinite volume appears to a NAS client
as a single, scalable volume that can store up to 2 billion files and tens of petabytes of
data. Each infinite volume consists of several, typically dozens, of separate
components called constituents.

Constituents play one of various roles.

The data constituents, shown on the slide in blue, store the file’s physical data.
Clients are not aware of the data constituents and do not interact directly with them.
When a client requests a file from an infinite volume, the node retrieves the file's data
from a data constituent and returns the file to the client.

Each infinite volume has a one namespace constituent, shown on the slide in green.
The namespace constituent tracks file names, directories, and the file's physical data
location. Clients are also not aware of the namespace constituent and do not interact
directly with the namespace constituent.

A namespace mirror constituent, shown on the slide in red, is a data protection mirror
copy of the namespace constituent. It provides data protection of the namespace
constituent and support for incremental tape backup of infinite volumes.

For more information about infinite volumes, see the Infinite Volumes Management
Guide.

21
Knowledge Check
▪ Match each term with the term’s function.

Cluster Provides seamless scalability

Node Controls its physical storage and network resources

Provides availability of partner’s physical resources during

HA pair
a node failover

Aggregate A collection of RAID groups

SVM Owns its logical storage and network resources

FlexVol Volume Represents a filesystem

LIF Provides a network access point to an SVM

Match each term with the term’s function.

22
Knowledge Check
▪ Which three are network types? (Choose three.)
▪ Cluster interconnect
▪ Management network
▪ Data network
▪ HA network

Which three are network types?

23
Lesson 2
Cluster Configurations

Lesson 2, Cluster Configurations.

24
Consolidate Across Environments with ONTAP 9
Simplify data management for any application, anywhere

ONTAP 9
Storage Array Converged Heterogeneous SDS Near Cloud Cloud

Common Data Management

SDS = software-defined storage

ONTAP is mostly known as the data management software that runs on FAS and All
Flash FAS controllers. ONTAP 9 has many deployment options to choose from.

ONTAP can be deployed on engineered systems, which includes FAS and All Flash
FAS; converged systems, which includes FAS and All Flash FAS as part of a FlexPod
solution; third-party or E-Series storage arrays that use FlexArray virtualization
software; or near the cloud with NetApp Private Storage (NPS), which uses FAS or All
Flash FAS systems.

ONTAP can also be deployed on commodity hardware as software-defined storage

using ONTAP Select, or in the cloud using ONTAP Cloud.

Whichever deployment type you choose, you manage ONTAP in much the same
way, for a variety of applications. Although the ONTAP Cluster Fundamentals course
focuses on ONTAP clusters using FAS or All Flash FAS, the knowledge is also
applicable to all the deployment options.

25
Supported Cluster Configurations

Single-Node

Two-Node Switchless Multinode Switched

MetroCluster

NetApp supports single-node configurations, two-node switchless configurations,

multinode switched configurations, and MetroCluster configurations.

26
Revised Slide 24

Single-Node Cluster
▪ Single-node cluster:
▪ Special implementation of a cluster that runs on a
standalone node
▪ Appropriate when your workload requires only one
node and does not need nondisruptive operations
▪ Use case: Data protection for a remote office

▪ Features and operations that are

not supported:
▪ Storage failover and cluster high availability
▪ Multinode operations

A single-node cluster is a special implementation of a cluster running on a standalone

node. You can deploy a single-node cluster if your workload requires only one node
and does not need nondisruptive operations. For example, you could deploy a single-
node cluster to provide data protection for a remote office.

Some features and operations are not supported for single-node clusters. Because
single-node clusters operate in a standalone mode, storage failover and cluster high
availability are not available. If the node goes offline, clients cannot access data
stored in the cluster. Also, any operation that requires more than one node cannot be
performed. For example, you cannot move volumes, perform most copy operations,
or backup cluster configurations to other nodes.

27
Understanding HA Pairs
▪ HA pairs provide hardware redundancy to
do the following:
▪ Perform nondisruptive operations and upgrades
▪ Provide fault tolerance
▪ Enable a node to take over its partner’s storage and
later give back the storage
▪ Eliminate most hardware components and cables as
single points of failure
▪ Improve data availability

HA pairs provide hardware redundancy that is required for nondisruptive operations

and fault tolerance. The hardware redundancy gives each node in the pair the
software functionality to take over its partner's storage and later give back the
storage. These features also provide the fault tolerance required to perform
nondisruptive operations during hardware and software upgrades or maintenance.

A storage system has various single points of failure, such as certain cables or
hardware components. An HA pair greatly reduces the number of single points of
failure. If a failure occurs, the partner can take over and continue serving data until
the failure is fixed. The controller failover function provides continuous data
availability and preserves data integrity for client applications and users.

28
HA Interconnect
HA
Interconnect

Node 1 Node 2

Node 2 Storage
Node 1 Storage

Primary connection
Standby connection
Note: Multipath HA redundant
storage connections are not shown

Each node in an HA pair requires an HA interconnect between the controllers and

connections to both its own disk shelves and its partner node's shelves.

This example uses a standard FAS8080 EX HA pair with native DS4246 disk shelves.
The controllers in the HA pair are connected through an HA interconnect that consists
of adapters and cables. When the two controllers are in the same chassis, adapters
and cabling are not required because connections are made through an internal
interconnection. To validate an HA configuration, use the Hardware Universe.

For multipath HA support, redundant primary and secondary connections are also
required. For simplicity, these connections are not shown on the slide. Multipath HA is
required on all HA pairs except for some FAS2500 series system configurations,
which use single-path HA and lack the redundant standby connections.

29
Two-Node Cluster Interconnect

In a two-node switchless
cluster, ports are
connected between nodes. Onboard
10-GbE
4 x Ports
Cluster interconnect ports
on a FAS8060

In clusters with more than one node, a cluster interconnect is required. This example
shows a FAS8060 system that has two controllers installed in the chassis. Each
controller has a set of four onboard 10-GbE ports that can be used to connect to the
cluster interconnect.

In a two-node switchless cluster, a redundant pair of these ports is cabled together as

shown on this slide.

30
Switched Clusters

Inter-Switch
Cluster Interconnect Links (ISLs)

Cluster Switch Cluster Switch

If your workload requires more than two nodes, the cluster interconnect requires
switches. The cluster interconnect requires two dedicated switches for redundancy
and load balancing. Inter-Switch Links (ISLs) are required between the two switches.
There should always be at least two cluster connections, one to each switch, from
each node. The required connections vary, depending on the controller model.

After the cluster interconnect is established, you can add more nodes as your
workload requires.

For more information about the maximum number and models of controllers
supported, see the Hardware Universe.

For more information about the cluster interconnect and connections, see the Network
Management Guide.

31
MetroCluster
Benefits of MetroCluster software:
▪ Zero data loss
▪ Failover protection
▪ Nondisruptive upgrades

MetroCluster uses mirroring to protect the data in a cluster. The MetroCluster

continuous-availability and disaster recovery software delivers zero data loss, failover
protection, and nondisruptive upgrades.

MetroCluster provides disaster recovery through one MetroCluster command. The

command activates the mirrored data on the survivor site.

32
MetroCluster Configurations

Two-Node Configuration Four-Node Configuration Eight-Node Configuration

▪ Single-node cluster at each ▪ Two-node cluster at each site ▪ Four-node cluster at each
site site
▪ Protects data on a local level
▪ Protects data on a cluster and a cluster level ▪ Protects data on a local level
level and a cluster level

Cluster A Cluster B Cluster A Cluster B Cluster A Cluster B

Data Center A Data Center B Data Center A Data Center B Data Center A Data Center B

There are various two-node, four-node and eight-node MetroCluster configurations.

In a two-node configuration, each site or data center contains a cluster that consists
of a single node. The nodes in a two-node MetroCluster configuration are not
configured as an HA pair. However, because all storage is mirrored, a switchover
operation can be used to provide nondisruptive resiliency similar to that found in a
storage failover in an HA pair.

In a four-node configuration, each site or data center contains a cluster that consists
of an HA pair. A four-node MetroCluster configuration protects data on a local level
and on a cluster level.

In an eight-node configuration, each site contains a four-node cluster that consists of

two HA pairs. Like a four-node MetroCluster, an eight-node MetroCluster
configuration protects data on both a local level and a cluster level.

For more information about the MetroCluster configurations, see the MetroCluster
Management and Disaster Recovery Guide.

33
Knowledge Check
▪ Which cluster configuration provides a cost-effective,
nondisruptively scalable solution?
▪ Single-node
▪ Two-node switchless
▪ Multi-node switched
▪ MetroCluster

Which cluster configuration provides a cost-effective, nondisruptively scalable

solution?

34
Knowledge Check
▪ What is the maximum number of cluster switches that can be used in a
multinode switched cluster configuration?
▪ One
▪ Two
▪ Three
▪ Four

What is the maximum number of cluster switches that can be used in a multinode
switched cluster configuration?

35
Lesson 3
Create and Configure a Cluster

Lesson 3, Create and Configure a Cluster.

36
Revised Slide 34

Creating a Cluster
▪ Cluster creation methods:
▪ Cluster setup wizard, using the CLI
▪ Guided Cluster Setup, using OnCommand
System Manager

▪ The CLI method:

▪ Create the cluster on the first node.
▪ Join remaining nodes to the cluster.
▪ Configure the cluster time and AutoSupport.

▪ The Guided Cluster Setup method:

▪ Use your web browser.
▪ Use this link: https://<node-management-IP-address>

After installing the hardware, you can set up the cluster by using the cluster setup wizard (via
the CLI) or, in ONTAP 9.1 and later, by using the Guided Cluster Setup (via OnCommand
System Manager).

Before you set up a cluster, you should use a cluster setup worksheet to record the values that
you will need during the setup process. Worksheets are available on the NetApp Support
website.

Whichever method you choose, you begin by using the CLI to enter the cluster setup wizard
from a single node in the cluster. The cluster setup wizard prompts you to configure the node
management interface. Next, the cluster setup wizard asks whether you want to complete the
setup wizard by using the CLI.

If you press Enter, the wizard continues using the CLI to guide you through the configuration.
When you are prompted, enter the information that you collected on the worksheet. After
creating the cluster, you use the node setup wizard to join nodes to the cluster one at a time.
The node setup wizard helps you to configure each node's node-management interface.

It is recommended that, after you complete the cluster setup and add all the nodes, you
configure additional settings, such as the cluster time and AutoSupport.

If you choose to use the Guided Cluster Setup, instead of the CLI, use your web browser to
connect to the node management IP that you configured on the first node. When prompted,
enter the information that you collected on the worksheet. The Guided Cluster Setup discovers
all the nodes in the cluster and configures them at the same time.

For more information about setting up a cluster, see the Software Setup Guide.

37
Cluster Administration
▪ Cluster administrators administer
the entire cluster:
▪ All cluster resources
▪ SVM creation and management
▪ Access control and roles
▪ Resource delegation

▪ Login credentials:
▪ The default user name is “admin.”
▪ Use the password that was created
during cluster setup.

You access OnCommand System Manager through a web browser by entering the
cluster administration interface IP address that was created during cluster setup. You
log in as cluster administrator to manage the entire cluster. You manage all cluster
resources, the creation and management of SVMs, access control and roles, and
resource delegation.

To log in to the cluster, you use the default user name “admin” and the password that
you configured during cluster creation.

38
Managing Resources in a Cluster
OnCommand System Manager: The CLI:
▪ Visual representation of the ▪ Manual or scripted commands
available resources ▪ Manual resource creation that might require
▪ Wizard-based resource creation many steps
▪ Best-practice configurations ▪ Ability to focus and switch between specific
▪ Limited advanced operations objects quickly

There are many tools that can be used to create and manage cluster resources, each
with their own advantages and disadvantages. This slide focuses on two tools.

OnCommand System Manager is a web-based UI that provides a visual

representation of the available resources. Resource creation is wizard-based and
adheres to best practices. However, not all operations are available. Some advanced
operations might need to be performed by using commands in the CLI. Also, the
interface may change between ONTAP versions as new features are added.

The CLI can also be used to create and configure resources. Commands are entered
manually or through scripts. Instead of the wizards that are used in System Manager,
the CLI might require many manual commands to create and configure a resource.
Although manual commands give the administrator more control, manual commands
are also more prone to mistakes that can cause issues. One advantage of using the
CLI is that the administrator can quickly switch focus without having to move through
System Manager pages to find different objects.

39
Knowledge Check
▪ In OnCommand System Manager, which user name do you use to
manage a cluster?
▪ admin
▪ administrator
▪ root
▪ vsadmin

In OnCommand System Manager, which user name do you use to manage a cluster?

40
Knowledge Check
▪ In the CLI, which user name do you use to manage a cluster?
▪ admin
▪ administrator
▪ root
▪ vsadmin

In the CLI, which user name do you use to manage a cluster?

41
Lesson 4
Physical Storage

Lesson 4, Physical Storage.

42
ONTAP Storage Architecture

Files and LUNs

Logical Layer
FlexVol Volumes

Aggregate

Physical Layer
RAID Groups of Disks

This lesson focuses on the physical storage layer. The physical storage layer consists
of disks, RAID groups, and the aggregate.

43
Disks Types
ONTAP Industry-Standard
Disk Class Description
Disk Type Disk Type
BSAS Capacity SATA Bridged SAS-SATA disks
FSAS Capacity NL-SAS Near-line SAS
mSATA Capacity SATA SATA disk in multidisk carrier storage shelf
SAS Performance SAS Serial-attached SCSI
SSD Ultra-performance SSD Solid-state drive
ATA Capacity SATA FC-connected Serial ATA
FC-AL Performance FC Fibre Channel
LUN Not applicable LUN Array LUN
Virtual Machine Disks that VMware ESX
VMDISK Not applicable VMDK
formats and manages

At the lowest level, data is stored on disks. The disks that are most commonly used
are SATA disks for capacity, SAS disks for performance, and solid-state drives, or
SSDs, for ultra-performance.

The Virtual Machine Disk, or VMDISK, is used in software-only versions of ONTAP,

for example, ONTAP Select.

The LUN disk type is not the same as a LUN that is created in a FlexVol volume. The
LUN disk type appears when the FlexArray storage virtualization software presents
an array LUN to ONTAP.

44
Identifying Disks

Shelf ID
DS4246

SAS Disk Name = <stack_id>.<shelf_id>.<bay>

Example: 1.0.22

In all storage systems, disks are named to enable the quick location of a disk. The
example identifies disk 1.0.22 located in a DS4246 shelf.

ONTAP assigns the stack ID, which is unique across the cluster. The shelf ID is set
on the storage shelf when the shelf is added to the stack or loop. The bay is the
position of the disk within its shelf.

45
Array LUNs
▪ Array LUNs are presented to ONTAP
using FlexArray storage virtualization
E-Series
software:
▪ An array LUN is created on the enterprise
or
storage array and presented to ONTAP.
Enterprise Array LUNs
▪ Array LUNs can function as hot spares or be
Storage Array assigned to aggregates.

▪ Array LUNs in an aggregate:

▪ Aggregates use RAID 0.
▪ Aggregates can contain only array LUNs.
Aggregate

Like disks, array LUNs can be used to create an aggregate. With the FlexArray
storage virtualization software licenses, you enable an enterprise storage array to
present an array LUN to ONTAP. An array LUN uses an FC connection type.

The way that ONTAP treats an array LUN is similar to the way it treats a typical disk.
When array LUNs are in use, the aggregates are configured with RAID 0. RAID
protection for the array LUN is provided by the enterprise storage array, not ONTAP.
Also, the aggregate can contain only other array LUNs. The aggregate cannot contain
hard disks or SSDs.

For more information about array LUNs, see the FlexArray Virtualization
Implementation Guides.

46
Disks and Aggregates
▪ What happens when a disk is
Unowned inserted into a system:
Disks ▪ The disk is initially “unowned.”
▪ By default, disk ownership is
assigned automatically.
▪ Disk ownership can be changed.

▪ What happens after ownership

is assigned:
Spare ▪ The disk functions as a hot spare.
Disks ▪ The disk can be assigned to an aggregate.

Aggregate

When a disk is inserted into a storage system’s disk shelf or a new shelf is added, the
disk is initially unowned. By default, the controller takes ownership of the disk. In an
HA pair, only one of the controllers can own a particular disk, but ownership can be
manually assigned to either controller.

After disk ownership is assigned, the disk functions as a spare disk.

When an aggregate is created or disks are added to an aggregate, the spare disks
are used.

47
RAID Groups
▪ Disks are added to RAID groups
within an aggregate.
▪ Disk must be same type:
▪ SAS, SATA, or SSD
▪ Array LUNs

▪ Disks should be the same speed

Data Disks Parity Double- and size:
▪ SAS speeds: 15K or 10K
Disk Parity
▪ SATA speed: 7.5K
Disk
Hot Spares ▪ You should always provide enough
hot spares.

When an aggregate is created or disks are added to an aggregate, the disks are
grouped into one or more RAID groups. Disks within a RAID group protect each other
in the event of a disk failure. Disk failure is discussed on the next slide.

Disks within a RAID group or aggregate must be the same type and usually the same
speed.

You should always provide enough hot spares for each disk type. That way, if a disk
in the group fails, the data can be reconstructed on a spare disk.

48
RAID Types
▪ RAID 4:
▪ RAID 4 provides a parity disk to protect the data in
the event of a single-disk failure.
▪ RAID 4 data aggregates require a minimum of
three disks.

▪ RAID-DP:
▪ RAID-DP provides two parity disks to protect the
data in the event of a double-disk failure.
Data Disks Parity Double Triple ▪ RAID-DP data aggregates require a minimum of
Disk Parity Parity five disks.
Disk Disk
▪ RAID-TEC:
▪ RAID-TEC provides three parity disks to protect
the data in the event of a triple-disk failure.
▪ RAID-TEC data aggregates require a minimum of
seven disks.

Three primary RAID types are used in ONTAP: RAID 4, RAID-DP, and RAID-TEC.

RAID 4 provides a parity disk to protect data in the event of a single-disk failure. If a
data disk fails, the system uses the parity information to reconstruct the data on a
spare disk. When you create a RAID 4 data aggregate, a minimum of three disks are
required.

RAID-DP technology provides two parity disks to protect data in the event of a
double-disk failure. If a second disk fails or becomes unreadable during
reconstruction when RAID 4 is in use, the data might not be recoverable. With RAID-
DP technology, a second parity disk can also be used to recover the data. When you
create a RAID-DP data aggregate, a minimum of five disks are required. RAID-DP is
the default for most disk types.

RAID-TEC technology provides three parity disks to protect data in the event of a
triple-disk failure. As disks become increasingly larger, RAID-TEC can be used to
reduce exposure to data loss during long rebuild times. When you create a RAID-TEC
data aggregate, a minimum of seven disks are required. RAID-TEC is the default for
SATA and near-line SAS hard disks that are 6 TB or larger.

49
Aggregates
Storage System ▪ Aggregates are composed RAID
Aggregate groups that contain disks or array
LUNs:
Plex0 (Pool 0) ▪ All RAID groups must be the same RAID
type.
rg0
▪ Aggregates contain the same disk type.

rg1 ▪ Aggregates have a single copy of

data, which is called a plex:
▪ A plex contains all the RAID groups that
belong to the aggregate.
Pool 0 Hot Spares ▪ Mirrored aggregates have two plexes.
▪ A pool of hot spare disks is assigned to
each plex.

To support the differing security, backup, performance, and data sharing needs of
your users, you can group the physical data storage resources on your storage
system into one or more aggregates. You can then design and configure these
aggregates to provide the appropriate level of performance and redundancy.

Each aggregate has its own RAID configuration, plex structure, and set of assigned
disks or array LUNs. Aggregates can contain multiple RAID groups, but the RAID
type and disk type must be the same.

Aggregates contain a single copy of data, which is called a plex. A plex contains all
the RAID groups that belong to the aggregate. Plexes can be mirrored by using the
SyncMirror software, which is most commonly used in MetroCluster configurations.
Each plex is also assigned a pool of hot spare disks.

50
Aggregate Types

Root Aggregate ▪ Root aggregate (aggr0):

▪ Creation is automatic during system initialization.
▪ Container is only for the node’s root volume with
log files and configuration information.

ONTAP prevents you from creating other

volumes in the root aggregate.
Data Aggregate
▪ Data aggregate:
▪ Default of RAID-DP with a five-disk minimum for
most disk types
▪ Container for SAS, SATA, SSD, or array LUNs

Each node of an HA pair requires three disks to be used for a RAID-DP root
aggregate, which is created when the system is first initialized. The root aggregate
contains the node’s root volume, named vol0, which contains configuration
information and log files. ONTAP prevents you from creating other volumes in the root
aggregate.

Aggregates for user data are called non-root aggregates or data aggregates. Data
aggregates must be created before any data SVMs or FlexVol volumes. When you
are creating data aggregates, the default is RAID-DP with a minimum of five disks for
most disk types. The aggregate can contain hard disks, SSDs, or array LUNs.

51
Advanced Disk Partitioning
▪ Advanced Disk Partitioning
(ADP):
▪ Shared disks for more efficient

<- N1 Parity
<- N1 Parity

<- N2 Parity
N2 Parity
resource use

<- N1 Spare

N2 Spare
<- N1 Data

Data Spare<- N2 Data

N1 Data
N1 Data

<- N2 Data

Data Spare<- N2 Data

▪ Lower disk

Parity
Parity
Parity

Parity
consumption requirements

User Aggr Parity

User Aggr Parity
Node1 Root

Node2 Root
User Aggr

Node2 Root<-
Node2 Root<-
Node1 Root<-
Node1 Root<-

User Aggr
▪ Partitioning types: Root Partition
▪ Root-data
▪ Root-data-data (not shown)

Spare
Parity
Parity
Data
Data

Data

Data
Data
Data

Data
Data Partition
▪ Default configuration for:
▪ Entry-level FAS2xxx systems
▪ All Flash FAS systems 1 21 23 34 45 56 6 7 7 8 8 99 1010 11
11 12
12

All nodes require a dedicated root aggregate of three disks, and a spare disk should
be provided for each node. Therefore, a 12-disk, entry-level system, as shown here,
would require at least eight disks before a data aggregate could even be created.
This configuration creates a challenge for administrators because the four remaining
disks do not meet the five-disk minimum for a RAID DP data aggregate.

Advanced Disk Partitioning, or ADP, overcomes this challenge by enabling the

controllers to share disks. This configuration lowers the disk consumption
requirements.

ADP reserves a small slice from each disk to create the root partition that can be used
for the root aggregates and hot spares. The remaining larger slices are configured as
data partitions that can be used for data aggregates and hot spares. The partitioning
type that is shown is called root-data partitioning. A second type of partitioning that is
called [8] root-data-data partitioning creates one small partition as the root partition
and two larger, equally sized partitions for data.

ADP is the default configuration for entry-level systems and for All Flash FAS
systems. Different ADP configurations and partitioning types are available, depending
on the controller model, disk type, disk size, or RAID type.

For more information about ADP configurations, see the Hardware Universe.

52
Hybrid Aggregates
Flash Pool aggregate

▪ What Flash Pool aggregates contain:

▪ SAS or SATA disks for user data
▪ SSDs for high-performance caching

▪ How Flash Pool improves performance:

▪ Offloads random read operations
▪ Offloads repetitive random write operations

▪ Use case: Online transactional processing

(OLTP) workloads

A Flash Pool aggregate is a special type of hybrid data aggregate.

A Flash Pool aggregate combines SAS or SATA disks and SSDs to provide a high-
performance aggregate that is more economical than an SSD aggregate. The SSDs
provide a high-performance cache for the active dataset of the data volumes that are
provisioned on the Flash Pool aggregate. The cache offloads random read operations
and repetitive random write operations to improve response times and overall
throughput for disk I/O-bound data access operations.

Flash Pool can improve workloads that use online transactional processing, or OLTP,
for example a database application’s data. Flash Pool does not improve performance
of predominantly sequential workloads.

53
Hybrid Aggregates
FabricPool aggregate

▪ What FabricPool aggregates contain:

▪ A performance tier for frequently accessed (“hot”) data, Hot
which is located on an all-SSD aggregate
▪ A capacity tier for infrequently accessed (“cold”) data, On-premises
which is located on an object store

▪ How FabricPool can enhance the efficiency of

your storage system:
▪ Automatically tier data based on frequency of use Public Private
▪ Move inactive data to lower-cost cloud storage Cloud Cloud
▪ Make more space available on primary storage for
Cold
active workloads

A FabricPool aggregate is a type of hybrid data aggregate that was introduced in

ONTAP 9.2.

A FabricPool aggregate contains a performance tier for frequently accessed (“hot”)

data, which is located on an all-SSD aggregate, and a capacity tier for infrequently
accessed (“cold”) data, which is located on an object store. FabricPool supports
object store types that are in the public cloud using Amazon Simple Storage Service
(Amazon S3) and private cloud using StorageGRID Webscale software.

Storing data in tiers can enhance the efficiency of your storage system. FabricPool
stores data in a tier based on whether the data is frequently accessed. ONTAP
automatically moves inactive data to lower-cost cloud storage, which makes more
space available on primary storage for active workloads.

For more information about FabricPool aggregates, see the Disks and Aggregates
Power Guide.

54
Knowledge Check
▪ What is the minimum number of disks that are required to create a
RAID-DP data aggregate (excluding hot spares)?
▪ Two
▪ Three
▪ Four
▪ Five
▪ Six

What is the minimum number of disks that are required to create a RAID-DP data
aggregate (excluding hot spares)?

55
Knowledge Check
▪ What does a Flash Pool aggregate contain?
▪ Hard disks only
▪ Solid state drives (SSDs) only
▪ Hard disks for data storage and SSDs for caching
▪ Hard disks and SSDs that are used for data storage

What does a Flash Pool aggregate contain?

56
Lesson 5
WAFL

Lesson 5, WAFL.

57
Write Anywhere File Layout
Write Anywhere File Layout (WAFL) file system:
▪ Organizes blocks of data on disk into files
▪ FlexVol volumes represent the file system

FlexVol Volume
Inode file

Inode Inode

A B C D E

The Write Anywhere File Layout, or WAFL, file system organizes blocks of data on
disks into files. The logical container, which is a FlexVol volume, represents the file
system.

The WAFL file system stores metadata in inodes. The term “inode” refers to index
nodes. Inodes are pointers to the blocks on disk that hold the actual data. Every file
has an inode, and each volume has a hidden inode file, which is a collection of the
inodes in the volume.

58
NVRAM and Write Operations
▪ What happens when a host or client
Client Access
writes to the storage system:
▪ The system simultaneously writes to
system memory and logs the data in System
NVRAM. memory
▪ If the system is part of an HA pair, the
system also mirrors the log to the partner.
▪ The write can safely be acknowledged
because the NVRAM is battery-backed
memory. CP

▪ Write operations are sent to disk

NVRAM
from system memory at a
consistency point (CP).

When a host or client writes to the storage system, the system simultaneously writes
to system memory and logs the data in NVRAM. If the system is part of an HA pair,
the system also simultaneously mirrors the logs to the partner.

After the write is logged in battery-backed NVRAM and mirrored to the HA pair, the
system can safely acknowledge the write to the host or client.

The system does not write the data to disk immediately. The WAFL file system
caches the writes in system memory. Write operations are sent to disk, with other
write operations in system memory, at a consistency point, or CP. The system only
uses the data that is logged in NVRAM during a system failure, so after the data is
safely on disk, the logs are flushed from NVRAM.

59
Consistency Points
Certain circumstances trigger a CP:
▪ A ten-second timer runs out.
▪ An NVRAM buffer fills up and it is time to
flush the writes to disk.
▪ A Snapshot copy is created.

Inode New Snapshot

CP
Block A B C D E D’

WAFL optimizes all the incoming write requests in system memory before committing
the write requests to disk. The point at which the system commits the data in memory
to disk is called a consistency point because the data in system memory and disks is
consistent then.

A CP occurs at least once every 10 seconds or when the NVRAM buffer is full,
whichever comes first. CPs can also occur at other times, for example when a
Snapshot copy is created.

60
Direct Write Operation
Network interface card (NIC) or
Client Access host-bus adapter (HBA)