Vnxe Service Commands
Vnxe Service Commands
Technical Notes
January 2013
Copyright © 2013 EMC Corporation. All rights reserved. Published in the USA.
EMC2, EMC, and the EMC logo are registered trademarks or trademarks of EMC
Corporation in the United States and other countries. All other trademarks used
herein are the property of their respective owners.
For the most up-to-date regulatory document for your product line, go to the
technical documentation and advisories section on the EMC online support
website.
Introduction ........................................................................................................................ 10
Purpose ......................................................................................................................................... 10
Scope ............................................................................................................................................ 10
Audience ....................................................................................................................................... 11
Terminology ................................................................................................................................... 11
Business case The VNXe series is designed to be serviced by the user. Common system problems
can be solved from within Unisphere on the system’s Service Page. However, a
problem may occur that is not diagnosable or solvable by the options found on the
Service Page.
Solution overview A set of problem diagnostic, system configuration, and system recovery commands
are installed on the system’s operating environment. These commands provide an in-
depth level of information and a lower level of system control than is available
through Unisphere. This document describes these commands and their common use
cases.
Key results / The Service (svc) Commands listed in this document are a subset of the operating
recommendations environment’s software tools for servicing a VNXe system. Users may use the UEMCLI
scriptable system configuration for additional capability. UEM is not discussed in this
document.
Purpose This document describes the commands available for diagnosing and solving system
problems that may not be correctable through Unisphere. It also discusses common
uses for the Service Commands.
Scope This document provides a list of software tools available within the VNXe’s SSH that,
when combined with the proper methodology, can troubleshoot VNXe system
problems. Troubleshooting is the process of finding a problem, diagnosing its cause,
and applying its solution.
The Service Commands listed are available on VNXe systems running OE revisions
2.0.0.X through 2.3.X.X. Some changes have occurred between revisions. Enter the
“svc_help” command to review commands available in a specific OE version.
Additional Service Commands (not found in this document) may have been installed
on the system by authorized technical-support personnel for troubleshooting
purposes. Do not run additional commands without the approval of your authorized
Service Representative.
Commands are run after logging into a VNXe Storage Processor (SP) as the Service
user account. The login is through the console’s Secure Shell (SSH) or a terminal
attached to the platform’s serial connector. Before using these commands, apply for
the Service Account password, enable SSH, get any terminal hardware ready, and
install supporting software applications.
VNXe SPs can run in one of two operational modes: Normal Mode or Service Mode.
Some Service Commands run in both modes. Different Service Commands run in
different SP modes:
The Service Commands run on the VNXe’s Linux operating environment. You need to
be familiar with the Linux shell, your VNXe’s installed hardware, and the VNXe’s
operating environment to use these commands.
Additional information that may be helpful when using these commands can be
found in www.emc.com/vnxesupport.
Table 1. Terminology
Term Definition
Admin user The “admin” account has the ability to manage and
configure servers as well as provision the storage system.
This account is the only default user that may log in to
Unisphere in Normal Mode.
Backend Repository A portion of the first four drives in the DPE are dedicated
to VNXe system space. A section of this space is the
backend repository, which is dedicated to maintaining
known good images of VNXe software.
EMC Secure Remote Support ESRS IP Solution (ESRS IP) provides a secure, IP-based,
(ESRS) distributed support solution for command, control, and
visibility into a system by an authorized remote support
representative.
Normal Mode Standard operational mode for a VNXe system A user can
manage and configure servers as well as provision
storage. User data is accessible while the system is in
Normal Mode.
Service User Account The Service Account has the rights to perform
maintenance and troubleshooting on the VNXe system.
This account is the only one that can log into the VNXe CLI
(via SSH or serial connection) and the Service Page within
Unisphere.
Storage Processor (SP) A discrete, highly-available server that hosts both file and
block level virtualized storage as well as management for
these services. SPs are physically located within the DPE
and hold the CPU, memory, onboard SSD, and Battery
Backup Unit (BBU).
Initial Configuration
(svc_initial_config)
Service User Password Configuration
(svc_service_password)
Help (svc_help)
Interface with Encrypted Drives
(svc_key_restore)
ConnectEMC Configuration
(svc_connectemc)
Upload SSL Certificates
(svc_custom_cert)
ESRS Setup (svc_esrs)
Help (svc_help)
Shutdown (svc_shutdown)
Note that many Service Commands support a help option. This option is run with
either the “--help”, “- h” or “- ?” switch (no quotes). This will list usage syntax, usage
examples, and other information about the command's use. For example, to see the
help option for svc_ssh, run: svc_ssh --help
Backend Image Overwrites the OE stored on the backend drive image repository.
Restore Function: Recovery
(svc_backend_ima
ge_restore) Mode: Service
Description
This allows the Service user to overwrite the copy of the OE image stored in the image
repository. When the SP’s image repository gets corrupted, this service command is
used to update the image repository with a known-good image. After a successful
update, the svc_reimage service command writes this updated image to the SP’s
system partition.
All SPs have to be placed in Service Mode to run this command.
Use Cases
Usage: svc_backend_image_restore [<path>]
Related Commands
Restore VNXe OE (svc_reimage), Reinitialize VNXe to Factory Settings (svc_reinit)
Apply Custom Log- Configures the login banner type for Linux and Unisphere.
on Banners
(svc_banner) Function: Configuration
Mode: Both
Description
This allows the Normal user to specify what type of banner is displayed when logging
into Linux via SSH or serial terminal connection. The svc_banner operations only need
be run on one SP and the changes will synchronize between all SPs.
o system hostname
o Unisphere IP Address
Setting any of the banner types destroys all current banner information present on the
system. For example, setting a “complex” banner will destroy any custom banner files
present.
Custom or localized banners can only be set in Service Mode. As the Service user,
create plain-text files in /home/service whose names follow the convention of:
• Underscore ( _ )
• “.txt”
After creating these banner files, run svc_banner --put <list of files> to apply them to
the system.
Example Usage
To activate banner files in US English, Belarusian and Chinese, firstly create files with
localized content named:
• en_US.txt
• be_BY.txt
• zh_CN.txt
Then, to apply these files, run the following command as the Service user:
service@spa spa:~> svc_banner --put en_US.txt be_BY.txt
zh_CN.txt
To revert back to the default style banner, run the following command as the Service
user:
service@spa spa:~> svc_banner --set-simple
Related Commands
None.
Mode: Both
Description
Sets, clears, or lists boot control block tallies and breakpoints before the VNXe
specific software is loaded. It sets up for booting directly into the Linux operating
environment for troubleshooting.
Use Cases
Usage: svc_boot_control [<qualifiers>] <param>
Where the qualifiers are:
--help [-?]: Display this message
--set [-s]: Request stop before c4-boot
Parameter:
[net]
--clear [-c]: Clear request to stop before c4-boot
Parameter:
[net]
--list [-l]: Lists boot_control current settings.
Related Commands
None.
Single SP Cache Sets up and manages the cache card on a single SP system.
Card Control Function: System Operations
(svc_cachecard)
Mode: Both
Description
Manages, configures, and upgrades the cache card on a single SP VNXe 3100. By
default, the single SP VNXe 3100 comes with its cache card configured. This
command gets information about the cache card and runs card functions such as
rebooting and updating its firmware.
Use Cases
Usage: svc_cachecard [options]<param>
Options:
--help [-h]: display this help message
--info [-i]: display cache card information
--reboot [-r]: reboot cache card
--upgrade [-u]: upgrade cache card with firmware
Parameter:
[firmware]
--init [-init]: cache mirror init (any vaulted cache will be
removed)
Related Commands
None.
Anti-Virus Sets up and manages the CIFS file system’s anti-virus protection via the EMC®
Configuration Celerra® AntiVirus Agent (CAVA).
(svc_cava)
Function: Diagnostic
Mode: Normal
Description
This command allows a knowledgeable user to perform anti-virus activities from the
CLI. CAVA is an antivirus solution for clients using industry-standard CIFS (Common
Internet File System) protocols in a Microsoft Windows server. CAVA uses third-party
antivirus software to identify and eliminate known viruses before they infect files on
the storage system.
Use Cases
Usage: svc_cava { <movername> | ALL }
[ -audit | -update ]
| [ -set accesstime={ now | none |
[[[[yy]mm]dd]hh]mm[.ss] }]
| [ -fsscan [ <fs_name> { -list | -create [offline]
| -delete } ] ]
Example Usage
root@VNXe-spa spa:/nas/bin> svc_cava server_2
server_2 :
10 threads started.
2 Checker IP Address(es):
10.x.x.216 OFFLINE at Tue Feb 21 18:45:41 2012 (GMT-00:00)
MS-RPC over SMB, CAVA version: 4.6.8, ntStatus:
CONNECTION_DISCONNECTED
AV Engine: Network Associates
Server Name: win2008-srv-01.pedomain.local.116.245.10.in-
addr.arpa
Last time signature updated: Mon Feb 20 22:19:42 2012 (GMT-
00:00)
10.x.x.171 ONLINE at Tue Feb 21 18:45:41 2012 (GMT
00:00)
MS-RPC over SMB, CAVA version: 4.6.8,ntStatus: SUCCESS
AV Engine: Network Associates
Server Name: win-95j4fp0i27r.pedomain.local
Related Commands
None.
Mode: Normal
Description
This command may be used by the Service User to backup a system’s configuration at
any time. The system automatically performs a daily backup. The backup archives are
available at /EMC/backend/service/CBR.
To restore a system, contact your Service Provider. Only EMC Escalation Engineers are
authorized to use this recovery option.
Use Cases
usage: svc_cbr [<options>]
options:
--help[-h]: display this help message
Related Commands
None.
Clear Dirty Cache Identifies the LUNs with dirty cache and clears them.
(svc_cdca)
Function: Recovery
Mode: Normal
If a system boots into this dirty cache state, authorized support representatives must
first evaluate the system and then run this command to restore access to user data.
Use Cases
Usage: svc_cdca [<qualifiers>]
Qualifiers:
--help[-?] - displays this text
--list-cdca [-l] - list all cache dirty luns of current SP.
--clear[-c] - clear all the cache dirty luns for
current SP, prepare any system luns for auto fsck.
--fsck-list[-f] - re-do the auto fsck pre-processing, based
on last ‘clear’ processing.
--user-fs-list[-u] - generate the list of user
filesystems that should be checked, run this where nasdb is
mounted.
--perform-fsck[-p] - actually perform the fsck. Does the
same procesing as --user-fs-list, but also executes required
fscks.
--done[-d] - clean up all crumbs related to CDCA
script. Will not be able to use --fsck-list or --user-fs-
list until the next clear.
--show-all[-s] - show all LUN information for both SPs.
--boot-control-stop - set system to halt boot when we
can clear CDCA
--boot-control-continue - continue a c4 boot that is
currently halted
Related Commands
None.
Mode: Both
Use Cases
Usage: svc_change_config [<qualifiers>]
Qualifiers ::= -h | -c | -a
Where <qualifiers> are:
--help[-h]: print this message
--commit[-c]: valid in service mode to commit an upgrade
--abort[-a]: valid in service mode to abort an upgrade
--cleanup_network : valid in normal mode to remove network
interfaces on non-existent ports
Example Usage
service@VNXe-spa:~> svc_change_config
The current system configuration is: Single SP, 4GB.
These are your options:
1) Convert to Dual SP, 4GB.
2) Convert to Dual SP, 8GB.
3) Quit this program.
Enter selection: 1
The following upgrade is selected: Dual SP, 4GB.
Please verify that all of your upgrade components are
available before you continue. Your array will reboot if you
continue, and all of your data will be unavailable until the
upgrade is completed. The upgrade is a 'one way' change to
your array.
Enter 'continue' to proceed: continue
Checking system state, this may take several minutes, please
wait...In 30 seconds your array will reboot into Service
Mode. Then you can power down your array, and perform the
upgrade.
turning OFF write cache..
trying to obtain write-cache...acquired write-cache done
Related Commands
None.
Clean Solid State Checks and reclaims capacity on the system partition’s storage device.
Disk
(svc_clean_ssd) Function: System Operations
Mode: Both
Description
Checks if the local or peer SP system partition’s available space is above the upgrade
threshold. The command will also reclaim space to increase available capacity on the
storage device by removing unnecessary system files.
Use Cases
usage : svc_clean_ssd [option] [<mount_point>]
Options:
-s: Force Single SP operation.
-h: Displays this message
-f: Find all large files on the system or for the mount
point indicated device and generate a file system usage
report.
-c: Clear capacity on the root file system or on mount point
indicated device.
Example Usage
service@VNXe-spb:~> svc_clean_ssd -f
Now searching for files that could be deleted to prepare for
upgrade on LOCAL SP...Done!
…
[content removed]
…
Related Commands
None.
Description
The ConnectEMC configuration allows you to email notice of critical errors to EMC
(referred to as “home”) that occur on the storage system. Information transmitted
includes:
• Sending address
• Destination address
• Processing SMTP server
If ConnectEMC is enabled, EMC requests that users disable it during a maintenance
window before making changes. This will prevent planned downtime from being
reported as a ‘Data Unavailable’ failure to EMC.
Use Cases
Usage: svc_connectemc [<qualifiers>] <key> <value>
where <qualifiers> are:
--help[-?]: Display this message
--get[-g]: Get the value of the key
key:
Host
Sender
Destination
Subject
--set[-s]: Set the value of the key
key:
Host
Sender
Destination
Subject
value
--purge[-p]: Purge the phone home events
--start: Start connectEMC
--stop: Stop connectEMC
Example Usage
A user wishes to stop ConnectEMC during a maintenance window.
service@VNXe-spa spa:~> svc_connectemc --stop
connectemc (stop) running, process 26783
A user needs to change the source email address of the VNXe to meet security
requirements.
service@VNXe-spa:~> svc_connectemc -g sender
[email protected]
service@VNXe-spa:~> svc_connectemc -s sender [email protected]
init_complete set
ok
ok
ok
service@VNXe-spa:~> svc_connectemc -g sender
Related Commands
None.
Mode: Normal
Description
This command installs a specified SSL certificate for use with the Unisphere
management interface. To use the certificate, you must already have a private key
and certificate file. The private key strength needs to be at least 2048 bits and in the
PEM format.
The key file must be fully pathed and named according to the following convention:
<certfilename>.pk
The certificate file must be fully pathed and named according to the following
convention: <certfilename>.crt
Use Cases
Usage: svc_custom_cert [<qualifiers>] <cert file base path>
where <qualifiers> are:
--help[-h]: Display this message
Example Usage
To install with private key file: /home/service/mykey.pk and cert
/home/service/mykey.crt:
svc_custom_cert /home/service/mykey
Related Commands
None.
Mode: Both
Description
Data Collection is the main source of content for remote analysis of the VNXe system.
There are a number of additional use cases for the svc_dc. These options are outlined
below.
Use Cases
Usage: svc_dc [<options>] [<Params>]
Example Usage
service@VNXe-spb:~> svc_dc
[DC spb]: invoked from 26555 26554 26555 -bash
[DC spb]: Arguments:
[DC spb]: DCPID 26612
[DC spb]: Spawned group 26687
[DC spb]: Destination folder is:
/EMC/backend/service/data_collection
Related Commands
None.
Description
General purpose to diagnose for system troubleshooting. The command’s output may
include a basic set of general diagnostics, or be targeted to a specific subsystem.
• basic—General diagnostic
• boot control—System (re)boot status
• cru extended—Current state of the hardware
• network—Frontend port configuration information including MTU size.
• resume—Provides Midplane Part and Serial Numbers.
• sas—Advanced diagnosis of the backend SAS ports.
• software—State of the system software stack
• spinfo—Detailed output of SP hardware and software information
Executing the command without arguments results in the basic running diagnostic.
Use Cases
Usage: svc_diag [--state=<target>[,<target>]...]
target::= basic | bootcontrol |cru extended |network |resume
|sas |software |spinfo
Related Commands
Service Mode Information (svc_rescue_state)
Description
Enables the VNXe system's internal network between SPs in Service Mode. If the PCIe
interface did not automatically become available, this command manually sets up a
connection by loading the kernel modules needed for enabling the Ethernet interface.
Use Cases
Usage: svc_enable_internal_network
Related Commands
None.
ESRS Setup Outputs the system’s EMC Secure Remote Server (ESRS) configuration.
(svc_esrs)
Function: Configuration
Mode: Both
Description
Enables and disables EMC Secure Remote Support (ESRS) from the command line in
Service Mode.
Enabling or disabling ESRS is a local change to the SP. The change does not persist
after a reboot.
If no DEVICE_IP is present, then the VNXe system is not configured for ESRS access.
Except for the gqext information option (--g/--gwext_info ), options can only
run in Service Mode.
Use Cases
Usage: svc_esrs [<option>] <param>
Options:
--help [-h]: Display help this text
--enable [-e]: Enable and start ESRS on the local SP -
Automatically downloads configuration, if required.
Parameter:
[-f]: Forces a reprovision
--disable [-d]: Disable ESRS
Example Usage
service@VNXe-spa:~> svc_esrs -g
PRODUCT_ID=VNXe3100
DEVICE_TLA=FCNCH082100134
DEVICE_IP=
Related Commands
ConnectEMC Configuration (svc_connectemc)
Mode: Both
Description
Allows the user to specify whether the system’s web server in Service Mode should
operate in FIPS mode or not.
Operating in FIPS mode restricts the cryptographic functions of the web server to a
limited, FIPS 140-2 approved. The client's web browser must be FIPS-compatible in
order to connect to Unisphere on a VNXe when operating in FIPS mode. Consult your
web browser's help for configuration information.
“set” operations to set FIPS mode are only allowed when all system SPs are in Normal
Mode.
This tool should only be used if FIPS mode is needed for a particular reason (i.e.
Government regulation compliance).
Use Cases
Usage:
svc_fips_mode [-s (ON|OFF) | -g | -h]
Parameters:
-s|--set (ON|OFF) Sets FIPS mode ON or OFF
-g|--get Gets FIPS mode (ON or OFF)
-h|--help Prints this message
Related Commands
None.
Mode: Both
Description
Lists the Service Commands available to the service user found on the VNXe system.
Use Cases
N/A
Related Commands
Inject Troubleshooting Software Tool (svc_inject)
Mode: Normal
Description
Creates an XML-formatted data structure describing all details of the VNXe system's
hardware.
This command can also be used to generate a data structure containing only a subset
of the entire system's hardware tree by using the include or exclude parameters.
This script is primarily used by other system utilities to create an XML-formatted data
structure, which is easily parable by machine and, describes the system’s installed
hardware components.
It is not recommended to run this command continuously inside of a shell script. This
rate of execution may have an adverse affect on system performance.
Use Cases
Usage: [ papi | svc ]_hw_tree [ list of arguments ], where:
Example Usage
service@VNXe-spa:~/> svc_hw_tree –include_service_info
<?xml version="1.0" encoding="UTF-8"?>
<node type="Storage system" id="1">
Related Commands
Hardware Component Tree (svc_hw_tree), System Diagnostics
(svc_diag)
Mode: Normal
Description
Provides a CLI-based method to perform the steps of initial configuration through the
Connection Utility GUI. Note that the Connection Utility is recommended whenever
possible and is available at www.emc.com/vnxesupport. You need to set up:
• friendly hostname
• management IP address
• subnet mask
• default gateway
To complete this command successfully, the system must not have been configured
with a management IP. This command is only used with a serial connection to the SP
when the VNXe Connection Utility fails to automatically discover a system.
Use Cases
Usage: svc_initial_config [Options]
Options:
--add [-a]: Attempts to configure the VNXe with the given
friendly_name and network parameters. Requires options
described below.
--force Force configuration even if previous
configuration detected.
Parameters:
--network [-n]: IP address, netmask and default gateway for
management interface. Must be enclosed by quotes (“”) and
given in that exact order.”IP NMASK DEF_GW" Use "disable"
instead of "IP NMASK DEF_GW" to disable ipv4 configuration
Requires -f to be set.
Related Commands
None.
Description
This script allows additional troubleshooting tools to be injected onto a VNXe system.
The VNXe system hotfix procedure also uses this functionality to inject changes to the
system OS as deemed necessary by Engineering.
Injectable tools are packaged in one of the following ways:
Since software version 2.1.x, when performing a service tool injection a Dual-SP,
VNXe will attempt to inject the service tool on both SPs, regardless of their
operational mode.
Note: Once a service tool package file is used for injection, it is removed from the
VNXe system. Injected tools also have an expiration date. After that date, they may
not be run. The tool must be re-injected to re-enable it when it has passed its
expiration date.
Use Cases
Usage: svc_inject [<options>] [<params>]
[<AdditionalParams>]
options:
--help [-?]: Displays this text
--key [-k]: Perform key-based injection- see more info below
Parameters:
[serial]: Serial key
--hotfix [-h]: Install or erase a hotfix
Parameters:
-- Install [-I [package] ]: Install
-- Erase [-e [package]]: Erase
--tool [-t]: Install or erase a service tool
Parameters:
-- Install [-I [package] ]: Install
-- Erase [-e [package]]: Erase
--list [-l]: List currently-installed hotfixes, service
tools or both hotfixes and service tools.
Parameters:
-- hotfixes[-h ]: list hotfixes
-- servicetools [-t]: list service tools
-- all [-a]: list all
--repository [-r]: Add, delete, list repositories, or dump
package list
Parameters:
-- add [-a ]: add repository
Example Usage
Gaining root privileges with the svc_service_shell –k key-
based option:
Related Commands
None.
Description
svc_key_restore provides an interface for performing “restore” and “revert”
operations on encrypted data disks.
A restore operation will attempt to restore the Authentication Key specified by the
user on the command line. It will also prompt for the password that was used to
encrypt the Authentication key when the Key was backed up. Furthermore, a restore
operation is only available when all system SPs are in Service Mode. The command
only needs to be issued on one SP. This command requires that you have a backup
key file and provide its path on the command line.
A revert operation will revert an encrypted drive back to its factory defaults. Revert
operations are only allowed with both SPs in Normal Mode. The data on the disk will
be securely and permanently erased and authentication on the drive will be disabled.
This command requires that the CRU identifier and PSID of the disk are given as
arguments. The CRU identifier is the drive's CRU ID made up of SAS port bus,
enclosure and slot. PSID is an alphanumeric string that is printed on the label of an
encrypted drive. This value can be manually read and entered or could be read using
a bar code scanner.
Example usage syntax for revert and restore operations is provided below.
Use Cases
Usage: svc_key_restore [<qualifiers>]
Restore:
This option will attempt to restore the Authentication Key
specified by the "--restore" qualifier. It prompts for
password.
Example:
> svc_key_restore --sed --restore /home/service/backup.key
Revert:
This procedure will revert a drive to factory defaults.
The data on the drive will be crypto erased and
authentication on the drive will be disabled.
Example:
> svc_key_restore --sed -revert <bus_encl_slot> <PSID>
> svc_key_restore --sed -revert 0_0_10
ABCD1E23FG4H5IJKL7MN9OPQR1STUV23
Mode: Normal
Description
Allows the Service User the ability to view information about NFS locks which are
currently held for provisioned VNXe storage. It will only run in Normal Mode when the
system storage software is active.
The user must specify which storage server to work with on the command line. The
tool can list all active locks using the “list” command. Statistics can be listed using
the “stat” command; in addition, a stat reset can be issued by specifying “reset” after
the “stat” command on the command line. To view more detailed information about a
particular lock, the “info” command can be used. To remove a known stale NFS lock,
use the “remove” command. “remove” and “info” require a lock be specified on the
command line.
Use Cases
Usage : svc_lockd server_[2|3] [command]
Commands:
h, help Prints this message
list lists current locks info
stat [reset] shows statistics about current locks
info shows detailed information about specified
lock
remove removes specified lock
Example Usage
Example: Get locks info for server_2
> svc_lockd server_2 info
Related Commands
None.
Mode: Service
Description
Mount system partitions when the VNXe is in Service Mode. This command is
automatically run by the SP when entering Service Mode. It can also be run manually
by the service user to mount the system partitions. If the system startup has a failure
in the management software, partitions used in system troubleshooting may fail to
mount. This command may be used to attempt a manual force mount of those
partitions.
The mountable system partitions include: SSD, Backend, and Cores. The SSD
partition contains the VNXe’s Linux installation and other storage/ management
software required for system operation. The Backend partition stores persistent
configuration information and service data. The Cores partition is where system core
dumps are stored for triage and diagnostic purposes.
This command will attempt to mount the SSD, backend mirror and Cores partitions at
/mnt/ssdroot, /mnt/backend, and /mnt/cores respectively with read-only
privileges. If svc_mount is invoked manually, the "-w" qualifier will attempt to
mount partitions with write privileges if absolutely necessary for troubleshooting or
recovery operations.
Use Cases
Usage: svc_mount [<qualifiers>]
where the qualifiers are:
--help [-?]: Display this message
--write-mode [-w]:Mount with read/write permissions
--ssd-only [-s]: Only mount SSD.
--cores-only [-c]: Only mount cores
--backend-only [-b]: Only mount the backend
--unmount [-u]: Unmount ssd, cores, and backend partitions
Related Commands
Boot Control (svc_boot_control), Create Management Interface (svc_network)
End-to-End Storage Maps internal storage from file system to disk as well as from disk to file system.
Mapping Function: Diagnostic
(svc_neo_map)
Mode: Normal
Description
Lists all pools, WWNs, status of the storage and disks.
Use Cases
Usage: svc_neo_map [<qualifiers>] <param>
where <qualifiers> are:
--help[-?]: Display this text.
--lun [-l]: Display mappings of a specific LUN
Example Usage
service@VNXe-spa:~> svc_neo_map --lun=1
#### Neo Map Utility. Get map based on FLARE LUN 1.
Create Configures the remote access management interface for troubleshooting or recovery.
Management Function: Diagnostic
Interface
(svc_network) Mode: Service (Revision 2.2.x and higher: Both)
Description
Allows the creation of a non-persistent management configuration for SSH access
when no management IP has been configured or if the system software is unable to
automatically configure the interface.
Use Cases
Usage: svc_network <-? | -d> [parameters] <qual>
Optional operators are:
--help [-?]: Display this message
--debug [-d]: Turns on extra debugging messages
Related Commands
Boot Control (svc_boot_control), Mount Storage (svc_mount)
Network Collects network information and performs diagnostics using Storage Server and
Configuration Linux network interfaces.
Information Function: Diagnostic
(svc_networkcheck)
Mode: Both
Description
Collects network information about one or all SPs. It allows the service user to run
common network troubleshooting utilities. These utilities include netstat, tracert,
ping, and ethtool. It provides information relating to the management IP connectivity.
Besides, it performs specialized network checks using the storage server network
interfaces, and the Linux management interface.
With the exception of the "-m, --management" command which can be run from
Service Mode, this command can only be run in Normal Mode on the primary SP. The
output of the command is logged to
/home/service/svc_networkcheck.log.
Use Cases
Usage: svc_networkcheck [command] <parameters>
where the required commands are:
--help [-h]: Display this message
--info [-i]: Perform network checks such as TCP stats, NAS
routing tables, and the underlying network configuration
--replication [-r]: Perform replication checks
--netstat [-n]: Perform standard netstat
--tracert [-t]: Perform tracert to IP
Example Usage
service@VNXe-spa:~> svc_networkcheck -e all
======================= [spa][Thu May 19 22:50:38 UTC 2011]
Beginning Run =======================
[output continues]
Related Commands
Collect Performance Information (svc_perfcheck), Data Collection (svc_dc)
Mode: Normal
Description
Allows the Service User to synchronize the VNXe's time with the NTP server. It will
only run in Normal Mode. It can be only used on the primary SP with a management IP
configured. In addition, this tool can list information about the configured NTP servers
and their status.
Use Cases
Usage: svc_ntp [OPTION]
Example Usage
Example: Synchronize NTP time
> svc_ntp -s
Example: List the status about NTP servers
> svc_ntp -i
Related Commands
None.
Description
Outputs frontend and backend setup and performance metrics. This script is
designed to extract information that may be useful for performance troubleshooting.
Similar content is available in Data Collects bundles – see svc_dc.
Some performance statistics that users can monitor for are visible through the
options on svc_perfcheck.
Use --sar option to produce output similar to the Linux sar command for the SP. This
output includes CPU usage distribution.
The --ktrace option provides low-level information of every IO operation. Its use is
intended for storage experts only.
Use Cases
Usage: svc_perfcheck <qualifiers> [<value>]
Where <qualifiers> are:
--help [-h|-?]: Display this message
Example Usage
service@VNXe-spb:~> svc_perfcheck -s -i 10 -n 15
Running sadc on SPA 15 times at 10 seconds interval.....
Peer is up, running sadc on SPB 15 times at 10 seconds
interval.....
Waiting for 155 seconds for sadc to finish....
sadc finished on SPA
sar finished on SPA
Finished sadc on SPB
sar finished on SPB
sar output is located at /EMC/backend/perf_stats/sar/
Description
Lists all commands and output from the Pre-Upgrade Health Check. It also allows for
running the Linux dig utility. This utility returns records from a specified Domain
Name Server (DNS).
Use Cases
Usage: svc_puhccheck [command] <parameters>
where the required commands are:
--help [-h]: Display this message
--check [-c]: Perform various health checks
--dig [-d]:: Perform a dig of named server.
Parameter
[server_name]: Server to run on.
Example Usage
service@VNXe-spa:~> svc_puhccheck -c
Delete Error Logs Deletes error logs from the system log database.
(svc_purge_c4logs)
Function: System Operations
Mode: Normal
Description
This command purges the system logs from log database. These system logs are
needed for root cause analysis of failure events. This script is intended for use only by
authorized technical representatives when requested by Engineering.
Use Cases
Usage: svc_purge_c4logs [<qualifiers>] <param>
Related Commands
None.
Restore VNXe OE Overwrites the SP’s System Partition with a known-good software image while
(svc_reimage) retaining the system’s configuration and all customer data.
Function: Recovery
Mode: Service
Description
Overwrites the SP’s system partition with a known good image stored on the backend
system device while maintaining the persistent configuration information, such as
hostname, host registration, and user data.
Using this command is a valid troubleshooting step when an SP is in Service Mode,
because the SYSTEM Boot Counter has reached its threshold. It is also used
automatically by the system software after a SP has the onboard System Drive
replaced or if the SP detects that the system partition is corrupt or has too little space
left for normal operation.
Use Cases
Usage: svc_reimage [<qualifiers>]
where the qualifiers are:
--help [-h]: Prints this text
--reboot [-r]: Reboot after reimage
--powerdown [-p]: Powerdown after reimage
--force [-f]: Skip all prompts
The script does no shutdown/reboot actions by default. Note that the SP must be
rebooted before a reimage operation will occur. In this case, the Boot Counters must
be cleared using the svc_rescue_state service tool prior to rebooting the system else
it will reboot back into Service Mode and the reimage operation will not occur.
Related Commands
Service Mode Information (svc_rescue_state)
Mode: Service
Description
Reinitializes a VNXe system to remedy configuration problems or to prepare a system
for decommission.
The onboard SSD that contains the VNXe system’s OE is overwritten with the image
currently contained in the backend image repository. All persistent configurations on
the system, such as hostname, storage configuration, and host registration, are
permanently overwritten. All storage provisioning information and user data is lost.
This command can only be used when all installed SP(s) have been placed into the
Service Mode.
This command should be considered a last resort troubleshooting solution. It is
recommended to try a SP reimage (svc_reimage) to try to correct the problem before
using this command. It’s also recommended that users obtain a Data Collection and
consult with their support provider before making the decision to run this command.
Use Cases
Usage: svc_reinit [<qualifiers>]
where the qualifiers are:
--help [-h]: Display this message
--force [-f]: Skip all prompts
Related Commands
Backend Image Restore (svc_backend_image_restore), Restore VNXe OE
(svc_reimage)
Service Mode Hint Specifies the reason why the SP has entered Service Mode
(svc_rescue_reason) Function: Diagnostic
Mode: Service
Description
The rescue reason signifies the specific reason an SP is in Service Mode. The rescue
reason is a hexadecimal-formatted code. It is used mainly by the system’s utilities.
The command will return a value of “0x0000” if the system has no rescue reason set,
or the rescue reason has it been cleared before it was run.
In most cases, svc_diag should be used to view the system's rescue reason. That
command contains the rescue reason code, the rescue reason’s definition, and
additional information about the current system state.
Use Cases
Usage: svc_rescue_reason [<qualifiers> | none]
Where the qualifiers are:
--help [-h]: Display this message
Service Mode Outputs or changes the boot counters controlling the system’s boot into Service
Information Mode.
(svc_rescue_state)
Function: Diagnostic
Mode: Both
Description
Views, sets, or clears the boot counters. This command is used in the recommended
remote shutdown procedures. It can also be used to determine which system
component is responsible for the boot into Service Mode.
The List option details why the logged-on SP has booted into Service Mode. See
svc_diag for further content.
The Set option manually forces the SP to boot into Service Mode the next time it is
rebooted.
The Clear option initializes all the Boot Counters, and instructs the SP to attempt to
boot into Normal Mode on the next reboot. Note that errors or faults may still cause
the system to boot back into Service Mode. Address the reason the system re-booted
into Service Mode before proceeding with clearing the counters.
Use Cases
Usage: svc_rescue_state [<qualifiers>]
where <qualifiers are:
--help[-?]: display this message
--list[-l]: list all boot counters
--set[-s]: set the Service-Mode boot counter
--clear[-c]: clear all the boot counters
Example Usage
service@VNXe-spa:~> svc_rescue_state -s
1
service@VNXe-spa:~> svc_rescue_state -l | egrep
"action|USER"
comp code count max action
USER 0x09 1 1 0x05 (RESCUE
MODE REQUESTED)
service@VNXe-spa:~> svc_rescue_state -c
Resetting boot control data to initial defaults...
service@VNXe-spa:~> svc_rescue_state -l | egrep
"action|USER"
comp code count max action
Related Commands
System Diagnostics (svc_diag), Shutdown (svc_shutdown)
Description
This command replaces svc_restart_mgmt. The new command has extended
features.
A user can run these commands to initialize the system software for troubleshooting
purposes, by exiting the process and then restarting it on the local SP.
Restarting portions of the system software interrupts their function.
The features which may be restarted are:
• MGMT—Launches Unisphere when an SP is in Normal Mode. MGMT is also
responsible for snapshot schedules and expansion of backend storage. For
example, when auto-file system extension runs out of local pooled resources.
Restarting of the management software can disrupt management activities
such as provisioning storage, configuration of networking, and other
activities. Use svc_restart_service MGMT only after confirming that no other
users are managing the system. The service user can run the Linux command
last to see if other users are logged into the system).
• NAS_A—Controls SPA file server virtualization software. Restarting NAS_A will
result in ‘Date Unavailable’ for all servers and shares accessed through this
service.
• NAS_B—Controls SPB file server virtualization software. Restarting NAS_B
results in ‘Date Unavailable’ for all servers and shares accessed through this
service.
• NASDB—Controls the services used for NAS management activities from
failover to configuration. Data is still accessible during this reboot activity.
Use Cases
Usage: svc_restart_service <qualifiers>
where the qualifiers are:
-MGMT:Restart the MGMT stack
NAS_A: Restart the NAS_A stack
NAS_B: Restart the NAS_B stack
NASDB: Restart the NASDB stack
Related Commands
Restart Management Stack (svc_restart_mgmt)
Description
ECOM controls the ability to use the Unisphere GUI when an SP is in Normal Mode. It
also is responsible for snapshot schedules and expansion of backend storage.
Expansions of backend storage occur when auto-file system extension runs out of
local pooled resources.
Restarting ECOM can disrupt management activities such as provisioning storage,
configuration of networking, and other activities. It is recommended to only use this
command after confirming no other users are managing the system. The Service user
can run the Linux command last to see if other users are logged to the system.
Note: This command was replaced by svc_restart_service MGMT.
Use Cases
Usage: svc_restart_mgmt
Example Usage
service@VNXe-spa:/tmp> svc_restart_mgmt
Found ECOM on spa
Related Commands
System Diagnostics (svc_diag)
Mode: Normal
Description
Produces a file that contains details of the system configuration. The file can be
created in either a text or XML format. By default, the data captured in the file does
not include private data like IP addresses, user comments and user defined names.
This private data can be included by specifying the "-showPrivateData" option. The
data includes:
• System identification
• Hardware inventory
• Software inventory
• License and service contract data
Use Cases
Usage : svc_save_cem_config[-xml [-showPrivateData]]
Options:
No options Capture configuration in text format.
Example Usage
Running the command for reviewing all parts and part numbers:
root@egreen-1131-SPA spa:~> svc_save_cem_config -xml
/EMC/backend/service/configuration_history/20120508_112204_F
NM00120600554_EMC-UEM-Telemetry.xml
cat 20120508_112204_FNM00120600554_EMC-UEM-Telemetry.xml
----
CEMSYSTEMS
-----
----CEMSYSTEM---
ExternalModelNumber:VNXe3150
Status:ok
CaptureDurationS:2012-05-06 06:22:19
SystemName:egreen-1131
ManagementSubnetMask:255.255.255.0
Model:NEO B JR 25 DUAL DPE W 4 CORE 8G
ManagementIPv6ConfigType:disabled
[Output continues]
Related Commands
System Diagnostics (svc_diag), Output VNXe System Status (svc_sysstatus)
Enable serial port Enables serial port for small file transfer operations.
transfers Function: Recovery
(svc_serial_transfer)
Mode: Both
Description
Allows the user to send or receive files over the SP’s serial connection using a
software terminal application that can make connections using a modem, or a null
modem cable which can emulate modem communication.
Terminal applications that support the Zmodem protocol can be used. Some of those
include:
• Microsoft HyperTerminal™
• Linux sz
• PuTTY
• Telnet (with Zmodem support)
Use Cases
Note: For a step-by-step procedure of how to use this functionality, use the -i option.
Usage of the command: svc_serial_transfer <option>
Where options are:
--help [-h]: Displays this text
--instructions [-i]: Prints instructions on sending a file
using Zmodem protocol.
--receive [-r]: Receives a file over serial
--send [-s]: Sends a file over serial.
Description
Allows the user to check if the Service user password is set to its default, is valid, or
change it.
The default Service user password is “service” (no quotes). When this command
shows the default is in use, it is recommended that a new password be set. The
isdefault mode switch returns a zero (0), if the default is in use. The reset
option of this command sets the service password back to the default value.
In order for a password to be considered compliant with VNXe security policies and
accepted as valid it must adhere to the following guidelines:
• 8-40 characters in length
• At least 1 digit
o !,@#$%^*_~?
• Password must be unique from the previous three Service Mode passwords.
Changes made to the service password using this tool while in Service Mode are
considered non-persistent - the service user will have the same password it had
before using the command to change it when the system returns to Normal Mode.
Use Cases
Usage: svc_service_password <qualifiers> [<Password>]
where <qualifiers> are:
--help[-?]: Display this message
--set[-s]: Set password with either supplied on command line
or prompt.
Password:New password.
--reset[-r]: Reset the default password
--validate[-v]: Validate the supplied password
Password:New password.
Example Usage
service@VNXe-spa:~> svc_service_password -d
no
service@VNXe-spa:~> svc_service_password -v
Password: <User entered correct current service password>
ok
service@VNXe-spa:~> svc_service_password -r
service@VNXe-spa:~> svc_service_password -d
yes
service@VNXe-spa:~> svc_service_password -s
New Password: <User entered one of last 3 passwords used>
Error: password is one of the last 3 previously-used
passwords.
service@VNXe-spa:~> svc_service_password -s
New Password:
service@VNXe-spa:~>
Related Commands
None.
Mode: Both
Description
Allows an authorized support entity to gain super user access to the VNXe’s operating
system in both Normal and Service modes.
This tool must firstly be enabled by the support entity before it can be used. Once
enabled, the tool will remain functional for 3 days. After 3 days it will automatically
disable itself.
Use Cases
None.
Related Commands
None.
Shutdown Performs a safe reboot or shutdown of a SP, and shutdown of the system.
(svc_shutdown)
Function: System Operations
Description
This service tool is used to perform a safe reboot or power down (halt) of a VNXe SP,
and shutdown of the system.
SP reboot and shutdown can be performed in both service and Normal Mode. System
shutdown can be performed only when at least one of the SPs is in Normal Mode.
When initiated, the reboot operation attempts to gracefully deactivate the active
storage software and all running Linux processes before performing a warm reboot of
the SP. The SP will then automatically run through its boot processes and – assuming
it is in a healthy state and no boot counters are set – come back online.The halt
operation is similar to the reboot in that it performs the same graceful bring down of
the system software but it does not reboot the SP. Instead, the SP is left in a powered-
on state with no software running. A physical power -down of the SP is required to
bring it back online. The halt option is available on VNXe system software revisions
2.2.x and later.
An optional Reboot Reason code may be added to the shutdown command for
logging the need for the voluntary reboot. Reason codes are user definable. Reason
codes are logged to the /EMC/C4Core/log/start_c4.log file.
Note: Systems rebooted when in Service Mode will return to Service Mode if Boot
Counters are not cleared. See svc_rescue_state for more information on Boot Counter
management.
The system-halt operation halts all SPs in the storage system. The system is left in a
powered-on state with no software running. A physical power-cycle of the system is
required to bring the system back online. The system-halt operation is available on
VNXe system software revisions 2.2.0 and later.
Use Cases
Usage: svc_shutdown [<qualifier>] [<param>]
Where <qualifiers> are:
--help [-h]: Display this message
--halt: Halt the SP for reason code
Parameter
[<reason>]: A <reason code> is self defined by
the user.
--reboot [-r]: Reboot the system
Parameter
[<reason>]: A <reason code> is self defined by
the user.
--system-halt [--force]: Halt the complete system. force
omits confirmation.
Example Usage
Example 1:User wants to reboot SPA
Related Commands
Service Mode Information (svc_rescue_state)
Output System Outputs the health status of the system partition’s storage device.
Partition's SMART Function: System Operations
Information
(svc_smartdata) Mode: Both
Description
This command collects and outputs information about the SP hosting the system
partition.
When run with no options, this command assesses the health of the onboard storage
device. The command also estimates the percentage of service hours remaining on
the device, based on current usage. The SMART attribute values of the device are
recorded at the time of the run into a database. The location option ("--db-
location”) outputs the logical path to this database.
The dumpdata option outputs all the attributes. Format of the output is:
RUN#|UTC Timestamp of RUN|Erasable|AttrID#|P/F of attr|Value
of ATTR ID
The command returns zero (0) for a successful run and a healthy disk. A one (1) is
returned if an error occurs or the storage device is not healthy. A two (2) is returned if
the device has shown an unexpected amount of wear since the last command run.
Use Cases
Usage: svc_smartdata <[options]> <param>
Options:
--help [-h]: Displays this message
--read [-r]: Read data
--dump-data [-d]: Dumps database
--db-location: Displays location of database
--parseable: Displays output in parseable format
Example Usage
service@VNXe-spa:~> svc_smartdata
INFO: Backend found, attempting to use backend database ...
SMART overall-health self-assessment test result: PASSED
INFO: 94% of life remaining for device /dev/ssd.
Related Commands
None.
Mode: Both
Description
Enables or disables the ability for the service user to connect to the VNXe’s Linux CLI
via SSH connection over the system’s Management interface. If SSH connectivity is
disabled, or did not turn on when requested from within Unisphere a Service user
may manually enable SSH while logged into Linux through a serial connection to the
SP with this command.
This command may also be used to disable SSH connections to the VNXe system. If
SSH is disabled while a user is connected to the SP through SSH, that user’s session
will end.
Use Cases
Usage: svc_ssh [options]
Options:
--help [-h]: Displays this text.
--enable [-e]: Enables SSHD on the system
--disable [-d]: Disables SSHD on the system
Example Usage
service@VNXe-spa:~> svc_ssh -d
Disabling sshd ...
Connection to 10.x.x.43 closed by remote host.
Related Commands
None.
Run FSCK on Runs File System Check (FSCK) on storage file systems and returns them to a
Storage mountable state.
(svc_storage_integ
ritycheck) Function: Recovery
Mode: Normal
Description
Automatically unmounts file systems that are in use, runs FSCK on them, and then
remounts the file systems when it is safe to do so.
This command can be used to get a file system remounted as soon as possible.
A file system will only be listed as an output if it is corrupt and unmounted. The
command gives an option to fix and remount corrupted and unmounted file systems.
Use Cases
Usage: svc_storage_integritycheck [<qualifiers>]
Where the qualifiers are:
--help [-h]: Display this message
Example Usage
service@sentry-564-spa spa:~> svc_storage_integritycheck
=======================================
List of corrupt storage elements found:
=======================================
1 -> tst
=======================================
Please enter the item you would like to fix from the list,
or enter "q" to quit: 1
############################################################
###############
WARNING: Attempting to fix this corruption may take a long
time to complete
Please wait for status to say "Done..Done" before proceeding
############################################################
###############
Enter "yes" if want to proceed with this action: yes
Running check on: 1 -> tst
id = 23
name = tst
Related Commands
None.
Mode: Normal
Description
This script allows you to investigate specifics about all virtual storage content on the
system. Information about particular kinds of shares (CIFS or NFS), iSCSI storage or
lower layers in the configuration stack can be queried.
This script can only be run on the Primary SP. Output is logged to the
/home/service/svc_storagecheck.log file. The output of this command is
lengthy if running all available checks. It is recommended to view the output log at
the location specified above using the Linux less utility.
Use Cases
Usage: svc_storagecheck [command] <parameters>
where the commands are:
--help [-h]: Output this text.
--dedupe [-d]
--backend [-b]
--sizes [-s]
--list [-l]
--nascheckpoint [-p
Parameter:
[<fs_name>]
--iscsi [-i]: iSCSI port.
Parameter:
[<-target>]
[<-lun>]
[<-snaps>]
[<-stats>]
[<-all>]: All iSCSI metrics
--nfs [-n]
--cifs [-c]
--ndmp [-d]
--replication [-r]
Parameter:
[<-sessions>]
[<-remotesys>]
[<-interconnects>]
--all [-a]: Output all metrics.
Output VNXe Outputs high-level status information about the VNXe system.
System Status
(svc_sysstatus) Function: Diagnostic
Mode: Both
Description
Outputs an XML formatted data structure containing information on the system’s
current health.
Use Cases
Usage: svc_sysstatus [<qualifiers>]
The qualifiers are:
--help [-?]: Display this message.
--quick [-q]: Quick mode (displays the local subset of
system state).
Related Commands
System Diagnostics (svc_diag), Hardware Component Tree (svc_hw_tree).
Redirect Output Attempts to run tcpdump network diagnostics on a given system interface in a safe,
(svc_ tcpdump) controlled manner.
Function: Diagnostic
Mode: Both
Output files from this command may be read using the tcpdump -r command.
Many options are analogous to their tcpdump counterparts so usage is similar to the
standard tool.
Use Cases
Usage: svc_tcpdump [<options>]
-?|--help
dump line.
-y|--dlink [type] : Set the data link type to use while capturing packets to
datalinktype.
Rotations (-W) and size (-C) must be >= 1. The default file size is 50MB and the default
number of files is 5. The default interface is mgmt_vdev. By default, the output files
are named "dump.out[0-4]" & stored in /home/service
Example Usage:
EXAMPLE
Capture mgmt_vdev, save 2 100MB files called tcpdump.out[0-
1] to /home/service:
svc_tcpdump -i mgmt_vdev -p /home/service -w tcpdump.out -W
2 -C 100
Related Commands
None.