◦ If your system does not have onboard keymanager, NSE or NVE configured, complete the steps in this

section.
6. From the LOADER prompt, enter the boot_ontap command.

If you see… Then…

The login prompt Go to the next Step.

Waiting for giveback… a. Log into the partner controller.

b. Confirm the target controller is ready for giveback with the
storage failover show command.

7. Connect the console cable to the partner controller.

8. Give back the controller using the storage failover giveback -fromnode local command.
9. At the cluster prompt, check the logical interfaces with the net int -is-home false command.

If any interfaces are listed as "false", revert those interfaces back to their home port using the net int
revert command.

10. Move the console cable to the repaired controller and run the version -v command to check the ONTAP
versions.
11. Restore automatic giveback if you disabled it by using the storage failover modify -node local
-auto-giveback true command.

Restore OKM, NSE, and NVE as needed - AFF C190

Once environment variables are checked, you must complete steps specific to systems
that have Onboard Key Manager (OKM), NetApp Storage Encryption (NSE) or NetApp
Volume Encryption (NVE) enabled.
1. Determine which section you should use to restore your OKM, NSE, or NVE configurations: If NSE or NVE
are enabled along with Onboard Key Manager you must restore settings you captured at the beginning of
this procedure.
◦ If NSE or NVE are enabled and Onboard Key Manager is enabled, go to Restore NVE or NSE when
Onboard Key Manager is enabled.
◦ If NSE or NVE are enabled for ONTAP 9.6, go to Restore NSE/NVE on systems running ONTAP 9.6
and later.

Restore NVE or NSE when Onboard Key Manager is enabled

Steps
1. Connect the console cable to the target controller.
2. Use the boot_ontap command at the LOADER prompt to boot the controller.
3. Check the console output:

28
 If the console displays… Then…
The LOADER prompt Boot the controller to the boot menu: boot_ontap menu

Waiting for giveback…. a. Enter Ctrl-C at the prompt

b. At the message: Do you wish to halt this node rather than wait
[y/n]? , enter: y
c. At the LOADER prompt, enter the boot_ontap menu command.

4. At the Boot Menu, enter the hidden command, recover_onboard_keymanager and reply y at the
prompt
5. Enter the passphrase for the onboard key manager you obtained from the customer at the beginning of this
procedure.
6. When prompted to enter the backup data, paste the backup data you captured at the beginning of this
procedure, when asked. Paste the output of security key-manager backup show OR security
key-manager onboard show-backup command

The data is output from either security key-manager backup show or security
key-manager onboard show-backup command.

Example of backup data:

--------------------------BEGIN BACKUP--------------------------
TmV0QXBwIEtleSBCbG9iAAEAAAAEAAAAcAEAAAAAAADuD+byAAAAACEAAAAAAAAA
QAAAAAAAAABvOlH0AAAAAMh7qDLRyH1DBz12piVdy9ATSFMT0C0TlYFss4PDjTaV
dzRYkLd1PhQLxAWJwOIyqSr8qY1SEBgm1IWgE5DLRqkiAAAAAAAAACgAAAAAAAAA
3WTh7gAAAAAAAAAAAAAAAAIAAAAAAAgAZJEIWvdeHr5RCAvHGclo+wAAAAAAAAAA
IgAAAAAAAAAoAAAAAAAAAEOTcR0AAAAAAAAAAAAAAAACAAAAAAAJAGr3tJA/
LRzUQRHwv+1aWvAAAAAAAAAAACQAAAAAAAAAgAAAAAAAAACdhTcvAAAAAJ1PXeBf
ml4NBsSyV1B4jc4A7cvWEFY6lLG6hc6tbKLAHZuvfQ4rIbYAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
.
.
.
.
H4nPQM0nrDRYRa9SCv8AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAA

---------------------------END BACKUP---------------------------

7. At the Boot Menu select the option for Normal Boot.

The system boots to Waiting for giveback… prompt.

8. Move the console cable to the partner controller and login as "admin".

29
 9. Confirm the target controller is ready for giveback with the storage failover show command.
10. Giveback only the CFO aggregates with the storage failover giveback -fromnode local
-only-cfo-aggregates true command.
◦ If the command fails because of a failed disk, physically disengage the failed disk, but leave the disk in
the slot until a replacement is received.
◦ If the command fails because of an open CIFS sessions, check with customer how to close out CIFS
sessions.

Terminating CIFS can cause loss of data.

◦ If the command fails because the partner "not ready", wait 5 minutes for the NVMEMs to synchronize.
◦ If the command fails because of an NDMP, SnapMirror, or SnapVault process, disable the process. See
the appropriate Documentation Center for more information.
11. Once the giveback completes, check the failover and giveback status with the storage failover show
and `storage failover show-giveback` commands.

Only the CFO aggregates (root aggregate and CFO style data aggregates) will be shown.

12. Move the console cable to the target controller.

a. If you are running ONTAP 9.6 or later, run the security key-manager onboard sync:
b. Run the security key-manager onboard sync command and then enter the passphrase when
prompted.
c. Enter the security key-manager key query command to see a detailed view of all keys stored
in the onboard key manager and verify that the Restored column = yes/true for all authentication
keys.

If the Restored column = anything other than yes/true, contact Customer Support.

d. Wait 10 minutes for the key to synchronize across the cluster.

13. Move the console cable to the partner controller.
14. Give back the target controller using the storage failover giveback -fromnode local
command.
15. Check the giveback status, 3 minutes after it reports complete, using the storage failover show
command.

If giveback is not complete after 20 minutes, contact Customer Support.

16. At the clustershell prompt, enter the net int show -is-home false command to list the logical
interfaces that are not on their home controller and port.

If any interfaces are listed as false, revert those interfaces back to their home port using the net int
revert command.

17. Move the console cable to the target controller and run the version -v command to check the ONTAP
versions.
18. Restore automatic giveback if you disabled it by using the storage failover modify -node local
-auto-giveback true command.

30
Restore NSE/NVE on systems running ONTAP 9.6 and later

Steps
1. Connect the console cable to the target controller.
2. Use the boot_ontap command at the LOADER prompt to boot the controller.
3. Check the console output:

If the console displays… Then…

The login prompt Go to Step 7.

Waiting for giveback… a. Log into the partner controller.

b. Confirm the target controller is ready for
giveback with the storage failover show
command.

4. Move the console cable to the partner controller and give back the target controller storage using the
storage failover giveback -fromnode local -only-cfo-aggregates true local
command.
◦ If the command fails because of a failed disk, physically disengage the failed disk, but leave the disk in
the slot until a replacement is received.
◦ If the command fails because of an open CIFS sessions, check with customer how to close out CIFS
sessions.

Terminating CIFS can cause loss of data.

◦ If the command fails because the partner "not ready", wait 5 minutes for the NVMEMs to synchronize.
◦ If the command fails because of an NDMP, SnapMirror, or SnapVault process, disable the process. See
the appropriate Documentation Center for more information.
5. Wait 3 minutes and check the failover status with the storage failover show command.
6. At the clustershell prompt, enter the net int show -is-home false command to list the logical
interfaces that are not on their home controller and port.

If any interfaces are listed as false, revert those interfaces back to their home port using the net int
revert command.

7. Move the console cable to the target controller and run the version -v command to check the ONTAP
versions.
8. Restore automatic giveback if you disabled it by using the storage failover modify -node local
-auto-giveback true command.
9. Use the storage encryption disk show at the clustershell prompt, to review the output.
10. Use the security key-manager key query command to display the key IDs of the authentication
keys that are stored on the key management servers.
◦ If the Restored column = yes/true, you are done and can proceed to complete the replacement
process.

31
 ◦ If the Key Manager type = external and the Restored column = anything other than yes/true,
use the security key-manager external restore command to restore the key IDs of the
authentication keys.

If the command fails, contact Customer Support.

◦ If the Key Manager type = onboard and the Restored column = anything other than yes/true,
use the security key-manager onboard sync command to re-sync the Key Manager type.

Use the security key-manager key query command to verify that the Restored column =
yes/true for all authentication keys.

11. Connect the console cable to the partner controller.

12. Give back the controller using the storage failover giveback -fromnode local command.
13. Restore automatic giveback if you disabled it by using the storage failover modify -node local
-auto-giveback true command.

Return the failed part to NetApp - AFF C190

Return the failed part to NetApp, as described in the RMA instructions shipped with the
kit. See the Part Return & Replacements page for further information.

Chassis

Overview of chassis replacement - AFF C190

To replace the chassis, you must move the power supplies, hard drives, and controller
module or modules from the impaired chassis to the new chassis, and swap out the
impaired chassis from the equipment rack or system cabinet with the new chassis of the
same model as the impaired chassis.
All other components in the system must be functioning properly; if not, you must contact technical support.

• You can use this procedure with all versions of ONTAP supported by your system.
• This procedure is written with the assumption that you are moving all drives and controller module or
modules to the new chassis, and that the chassis is a new component from NetApp.
• This procedure is disruptive. For a two-node cluster, you will have a complete service outage and a partial
outage in a multi-node cluster.

Shut down the controllers - AFF C190

You must shut down the controller or controller in the chassis prior to moving them to the
new chassis.
About this task
• If you have a cluster with more than two controllers, it must be in quorum. If the cluster is not in quorum or
a healthy controller shows false for eligibility and health, you must correct the issue before shutting down
the impaired controller; see Synchronize a node with the cluster.
• If AutoSupport is enabled, suppress automatic case creation by invoking an AutoSupport message:

32
 system node autosupport invoke -node * -type all -message
MAINT=number_of_hours_downh

The following AutoSupport message suppresses automatic case creation for two hours: cluster1:*>
system node autosupport invoke -node * -type all -message MAINT=2h

Steps
1. If your system has two controller modules, disable the HA pair.

If your system is running Then…

clustered ONTAP with…
Two controllers in the cluster cluster ha modify -configured false

storage failover modify -node node0 -enabled false

More than two controllers in the storage failover modify -node node0 -enabled false
cluster

2. Halt the controller, pressing y when you are prompted to confirm the halt: system node halt -node
node_name

The confirmation message looks like the following:

Warning: This operation will cause controller "node-name" to be marked

as unhealthy. Unhealthy nodes do not participate in quorum voting. If
the controller goes out of service and one more controller goes out of
service there will be a data serving failure for the entire cluster.
This will cause a client disruption. Use "cluster show" to verify
cluster state. If possible bring other nodes online to improve the
resiliency of this cluster.

Do you want to continue? {y|n}:

You must perform a clean system shutdown before replacing the chassis to avoid losing
unwritten data in the nonvolatile memory (NVMEM/NVRAM). Depending on your system, if
the NVMEM/NVRAM LED is flashing, there is content in the NVMEM/NVRAM that has not
been saved to disk. You need to reboot the controller and start from the beginning of this
procedure. If repeated attempts to cleanly shut down the controller fail, be aware that you
might lose any data that was not saved to disk.

3. Where applicable, halt the second controller to avoid a possible quorum error message in an HA pair
configuration: system node halt -node second_node_name -ignore-quorum-warnings true
-skip-lif-migration-before-shutdown true

Answer y when prompted.

33
Move and replace hardware - AFF C190

Move the power supplies, hard drives, and controller module or modules from the
impaired chassis to the new chassis, and swap out the impaired chassis from the
equipment rack or system cabinet with the new chassis of the same model as the
impaired chassis.

Step 1: Move the power supply

Moving out a power supply when replacing a chassis involves turning off, disconnecting, and removing the
power supply from the old chassis and installing and connecting it on the replacement chassis.

1. If you are not already grounded, properly ground yourself.

2. Turn off the power supply and disconnect the power cables:
a. Turn off the power switch on the power supply.
b. Open the power cable retainer, and then unplug the power cable from the power supply.
c. Unplug the power cable from the power source.
3. Squeeze the latch on the power supply cam handle, and then open the cam handle to fully release the
power supply from the mid plane.
4. Use the cam handle to slide the power supply out of the system.

When removing a power supply, always use two hands to support its weight.

5. Repeat the preceding steps for any remaining power supplies.

6. Using both hands, support and align the edges of the power supply with the opening in the system chassis,
and then gently push the power supply into the chassis using the cam handle.

The power supplies are keyed and can only be installed one way.

Do not use excessive force when sliding the power supply into the system. You can damage
the connector.

7. Close the cam handle so that the latch clicks into the locked position and the power supply is fully seated.
8. Reconnect the power cable and secure it to the power supply using the power cable locking mechanism.

Step 2: Remove the controller module

To replace the chassis, you must remove the controller module or modules from the old chassis.

1. If you are not already grounded, properly ground yourself.

2. Loosen the hook and loop strap binding the cables to the cable management device, and then unplug the
system cables and SFPs (if needed) from the controller module, keeping track of where the cables were
connected.

Leave the cables in the cable management device so that when you reinstall the cable management
device, the cables are organized.

3. Remove and set aside the cable management devices from the left and right sides of the controller module.

34
 4. Squeeze the latch on the cam handle until it releases, open the cam handle fully to release the controller
module from the midplane, and then, using two hands, pull the controller module out of the chassis.

5. Set the controller module aside in a safe place, and repeat these steps if you have another controller
module in the chassis.

Step 3: Move drives to the new chassis

You need to move the drives from each bay opening in the old chassis to the same bay opening in the new
chassis.

1. Gently remove the bezel from the front of the system.

2. Remove the drives:
a. Press the release button at the top of the carrier face below the LEDs.
b. Pull the cam handle to its fully open position to unseat the drive from the midplane, and then gently
slide the drive out of the chassis.

The drive should disengage from the chassis, allowing it to slide free of the chassis.

When removing a drive, always use two hands to support its weight.

Drives are fragile. Handle them as little as possible to prevent damage to them.

3. Align the drive from the old chassis with the same bay opening in the new chassis.

35
 4. Gently push the drive into the chassis as far as it will go.

The cam handle engages and begins to rotate upward.

5. Firmly push the drive the rest of the way into the chassis, and then lock the cam handle by pushing it up
and against the drive holder.

Be sure to close the cam handle slowly so that it aligns correctly with the front of the drive carrier. It clicks
when it is secure.

6. Repeat the process for the remaining drives in the system.

Step 4: Replace a chassis from within the equipment rack or system cabinet

You must remove the existing chassis from the equipment rack or system cabinet before you can install the
replacement chassis.

1. Remove the screws from the chassis mount points.

2. With the help of two or three people, slide the old chassis off the rack rails in a system cabinet or L
brackets in an equipment rack, and then set it aside.
3. If you are not already grounded, properly ground yourself.
4. Using two or three people, install the replacement chassis into the equipment rack or system cabinet by
guiding the chassis onto the rack rails in a system cabinet or L brackets in an equipment rack.
5. Slide the chassis all the way into the equipment rack or system cabinet.
6. Secure the front of the chassis to the equipment rack or system cabinet, using the screws you removed
from the old chassis.
7. If you have not already done so, install the bezel.

Step 5: Install the controller module

After you install the controller module and any other components into the new chassis, you need to boot it to a
state where you can run the interconnect diagnostic test.

For HA pairs with two controller modules in the same chassis, the sequence in which you install the controller
module is especially important because it attempts to reboot as soon as you completely seat it in the chassis.

1. Align the end of the controller module with the opening in the chassis, and then gently push the controller
module halfway into the system.

Do not completely insert the controller module in the chassis until instructed to do so.

2. Recable the console to the controller module, and then reconnect the management port.
3. Repeat the preceding steps if there is a second controller to install in the new chassis.
4. Complete the installation of the controller module
a. With the cam handle in the open position, firmly push the controller module in until it meets the
midplane and is fully seated, and then close the cam handle to the locked position.

Do not use excessive force when sliding the controller module into the chassis to avoid
damaging the connectors.

36
 b. If you have not already done so, reinstall the cable management device.
c. Bind the cables to the cable management device with the hook and loop strap.
d. Repeat the preceding steps for the second controller module in the new chassis.
5. Connect the power supplies to different power sources, and then turn them on.
6. Boot each controller to Maintenance mode:
a. As each controller starts the booting, press Ctrl-C to interrupt the boot process when you see the
message Press Ctrl-C for Boot Menu.

If you miss the prompt and the controller modules boot to ONTAP, enter halt, and then
at the LOADER prompt enter boot_ontap, press Ctrl-C when prompted, and then
repeat this step.

b. From the boot menu, select the option for Maintenance mode.

Restore and verify the configuration - AFF C190

You must verify the HA state of the chassis and run System-Level diagnostics.

Step 1: Verify and setting the HA state of the chassis

You must verify the HA state of the chassis, and, if necessary, update the state to match your system
configuration.

1. In Maintenance mode, from either controller module, display the HA state of the local controller module and
chassis:

ha-config show

The HA state should be the same for all components.

2. If the displayed system state for the chassis does not match your system configuration:
a. Set the HA state for the chassis:

ha-config modify chassis HA-state

The value for HA-state can be one of the following:

▪ ha
▪ non-ha
b. Confirm that the setting has changed:

ha-config show

3. If you have not already done so, recable the rest of your system.
4. The next step depends on your system configuration.

37