ECS Management and

Monitoring
Lab Guide
November 2016

EMC Education Services

 Copyright
Copyright ©2016 EMC Corporation. All Rights Reserved. Published in the USA. EMC believes the information in this publication is
accurate as of its publication date. The information is subject to change without notice.

THE INFORMATION IN THIS PUBLICATION IS PROVIDED “AS IS.” EMC CORPORATION MAKES NO REPRESENTATIONS OR
WARRANTIES OF ANY KIND WITH RESPECT TO THE INFORMATION IN THIS PUBLICATION, AND SPECIFICALLY DISCLAIMS
IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Use, copying, and distribution of any EMC software described in this publication requires an applicable software license. The
trademarks, logos, and service marks (collectively "Trademarks") appearing in this publication are the property of EMC Corporation
and other parties. Nothing contained in this publication should be construed as granting any license or right to use any Trademark
without the prior written permission of the party that owns the Trademark.

EMC, EMC², the EMC logo, AccessAnywhere Access Logix, AdvantEdge, AlphaStor, AppSync ApplicationXtender, ArchiveXtender,
Atmos, Authentica, Authentic Problems, Automated Resource Manager, AutoStart, AutoSwap, AVALONidm, Avamar, Aveksa, Bus-
Tech, Captiva, Catalog Solution, C-Clip, Celerra, Celerra Replicator, Centera, CenterStage, CentraStar, EMC CertTracker. CIO
Connect, ClaimPack, ClaimsEditor, Claralert ,CLARiiON, ClientPak, CloudArray, Codebook Correlation Technology, Common
Information Model, Compuset, Compute Anywhere, Configuration Intelligence, Configuresoft, Connectrix, Constellation Computing,
CoprHD, EMC ControlCenter, CopyCross, CopyPoint, CX, DataBridge , Data Protection Suite. Data Protection Advisor, DBClassify,
DD Boost, Dantz, DatabaseXtender, Data Domain, Direct Matrix Architecture, DiskXtender, DiskXtender 2000, DLS ECO, Document
Sciences, Documentum, DR Anywhere, DSSD, ECS, elnput, E-Lab, Elastic Cloud Storage, EmailXaminer, EmailXtender , EMC
Centera, EMC ControlCenter, EMC LifeLine, EMCTV, Enginuity, EPFM. eRoom, Event Explorer, FAST, FarPoint, FirstPass, FLARE,
FormWare, Geosynchrony, Global File Virtualization, Graphic Visualization, Greenplum, HighRoad, HomeBase, Illuminator ,
InfoArchive, InfoMover, Infoscape, Infra, InputAccel, InputAccel Express, Invista, Ionix, Isilon, ISIS,Kazeon, EMC LifeLine,
Mainframe Appliance for Storage, Mainframe Data Library, Max Retriever, MCx, MediaStor , Metro, MetroPoint, MirrorView, Mozy,
Multi-Band Deduplication,Navisphere, Netstorage, NetWitness, NetWorker, EMC OnCourse, OnRack, OpenScale, Petrocloud,
PixTools, Powerlink, PowerPath, PowerSnap, ProSphere, ProtectEverywhere, ProtectPoint, EMC Proven, EMC Proven Professional,
QuickScan, RAPIDPath, EMC RecoverPoint, Rainfinity, RepliCare, RepliStor, ResourcePak, Retrospect, RSA, the RSA logo, SafeLine,
SAN Advisor, SAN Copy, SAN Manager, ScaleIO Smarts, Silver Trail, EMC Snap, SnapImage, SnapSure, SnapView, SourceOne,
SRDF, EMC Storage Administrator, StorageScope, SupportMate, SymmAPI, SymmEnabler, Symmetrix, Symmetrix DMX,
Symmetrix VMAX, TimeFinder, TwinStrata, UltraFlex, UltraPoint, UltraScale, Unisphere, Universal Data Consistency, Vblock, VCE.
Velocity, Viewlets, ViPR, Virtual Matrix, Virtual Matrix Architecture, Virtual Provisioning, Virtualize Everything, Compromise
Nothing, Virtuent, VMAX, VMAXe, VNX, VNXe, Voyence, VPLEX, VSAM-Assist, VSAM I/O PLUS, VSET, VSPEX, Watch4net,
WebXtender, xPression, xPresso, Xtrem, XtremCache, XtremSF, XtremSW, XtremIO, YottaYotta, Zero-Friction Enterprise Storage.

Revision Date: April 2017

Course Number: MR-1CN-ECSMGTMON.2.0.4

EMC Education Services i

 Table of Contents
LAB EXERCISE 1: CONNECTING TO YOUR CLASS LAB ENVIRONMENT ................................................................................ 1

LAB 1: PART 1 - CONNECT TO YOUR LAB POD................................................................................................................................2

LAB 1: PART 2 – LOGIN TO THE ECS PORTAL .................................................................................................................................4

LAB EXERCISE 2: CONFIGURE ECS STORAGE INFRASTRUCTURE ......................................................................................... 6

LAB 2: PART 1 - CREATE STORAGE POOLS .....................................................................................................................................7

LAB 2: PART 2 - CREATE A VDC ................................................................................................................................................11
LAB 2: PART 3 - CREATE REPLICATION GROUPS............................................................................................................................15
LAB 2: PART 4 - CREATE VDC FEDERATION AND A REPLICATION GROUP (GLOBAL) .............................................................................17

LAB EXERCISE 3: BASIC TESTS OF I/O ACCESS FROM VARIOUS DATA CLIENTS................................................................. 24

LAB 3: PART 1 – CREATE ECS NAMESPACES, LOCAL USERS AND BUCKETS ........................................................................................25
LAB 3: PART 2 – PERFORMING ECS METADATA SEARCH ...............................................................................................................34
LAB 3: PART 3 - TEST I/O ACCESS TO ECS FROM THE AWS S3 BROWSER ........................................................................................38
LAB 3: PART 4 - TEST I/O ACCESS TO ECS FROM CYBERDUCK (OPENSTACK SWIFT OBJECTS) ................................................................42
LAB 3: PART 5 – PUT AND GET CENTERA C-CLIPS FROM ECS USING CAS TOOLS................................................................................56
LAB 3: PART 6 – TEST “DATA-IN-PLACE” ACCESS TO S3 DATA WITHIN ECS FROM HADOOP ................................................................69

LAB EXERCISE 4: ECS CUSTOMIZATIONS: ACLS, QUOTAS AND RETENTION ..................................................................... 84

LAB 4: PART 1 – TEST ACLS WITH LOCAL OBJECT USERS IN ECS .....................................................................................................85
LAB 4: PART 2 – DEFINE ECS RETENTION POLICIES AND STUDY THEIR EFFECT ................................................................................. 101
LAB 4: PART 3 – ADVANCED RETENTION MANAGEMENT ............................................................................................................ 114
LAB 4: PART 4 – CONFIGURE AND VERIFY ENFORCEMENT OF ECS QUOTAS ..................................................................................... 117

LAB EXERCISE 5: MULTI-TENANCY WITH ACTIVE DIRECTORY/LDAP INTEGRATION ....................................................... 128

LAB 5: PART 1 – REVIEW THE CONFIGURATION OF AN ACTIVE DIRECTORY SERVER ........................................................................... 129
LAB 5: PART 2 – ADD ACTIVE DIRECTORY SERVER AS AN ECS AUTHENTICATION PROVIDER ................................................................ 131
LAB 5: PART 3 – CONFIGURE ECS NAMESPACES WITH DOMAIN GROUPS FOR MULTI-TENANCY ........................................................ 133
LAB 5: PART 4 – VERIFY I/O ACCESS TO ECS FROM TENANT USERS.............................................................................................. 137

LAB EXERCISE 6: ECS METERING AND MONITORING..................................................................................................... 142

LAB 6: PART 1 – VIEW ECS MONITORING DATA AND PERFORM BASIC HEALTH CHECKS ................................................................... 143

APPENDIX – A – RACK COLOR AND NODE NAMES ........................................................................................................ 153

EMC Education Services ii

 Lab Exercise 1: Connecting to Your Class Lab Environment

Purpose: Review lab guide for this class, and establish a Remote Desktop
session to your management station

 Connect to VDC to get access to the lab equipment.

Tasks:
 Test Remote Desktop access to the management station
within your assigned lab pod.

EMC Education Services i

 Lab 1: Part 1 - Connect to Your Lab Pod
The environment for this lab includes the following:

• 1 x Windows Management Station. This server is where you are going to perform most of
the lab exercises. It provides access into the other components of this lab. This server also
provides DNS and AD services to the environment.

• 1 x Linux server running a Hadoop node. This node runs HortonWorks HDP 2.3. This
environment provides the Hadoop File System that will be used with the ECS ViPRFS
Client.

• Two sites with one ECS node each. Each node is a VM running ECS 3.0 software (ECS
Community Edition – Single node). Real world ECS installs require a minimum four-node
setup; this one-node install is for demonstration purposes only with limited functionality.
It’s worth mentioning that although this is a virtual environment, all lab exercises perform
as a real world ECS install.

Step Action

1 Your instructor should have assigned you an ECS lab pod number, and given you the lab
configuration sheet showing the IP addresses of various components within that pod. If
you don’t have either of these, contact your instructor.

From your lab configuration sheet, write down the information below for your pod. You
will need it for lab access throughout this class:

My ECS pod number: _____________________________________________

Management station (Windows Host) : ______________________________

2 VDC Login Credentials (VDC is the platform used to access the lab equipment)

You should receive instructions from your instructor on how to login to the EMC
Education Services VDC (Virtual Data Center). Write down the following information
from your instructor:

VDC Website Address: ___________________________________________

Your assigned VDC User name: ____________________________________

VDC Password: _________________________________________________

Note: If you are using a personal laptop, Citrix Receiver and XenApp (www.citrix.com)
applications must be installed in order to access VDC

EMC Education Services 2

 Step Action

3 At this point, you are logged in on your management station, from which you have
convenient access to all needed tools, and every other host in your pod.

You may disconnect now from your Remote Desktop by closing the RDP window. This
leaves the session up and running and you can connect back in to the same session at
any time using the Administrator/P@ssw0rd credentials.

Continued on next page

EMC Education Services 3

 Lab 1: Part 2 – Login to the ECS Portal

Step Action

1 Open Google Chrome browser in your management station, and type the IP address of
your ECS node in the address bar (192.168.73.54) of the site1 vECS-1 node.

2 If there is a security certificate error, click Advanced and then click Proceed (unsafe).

EMC Education Services 4

 Step Action

3 Provide the authentication below to log into the ECS Portal:

User Name: root

Password: P@ssw0rd

4 Once authenticated, take a moment, expand and explore the following options:
Dashboard, Monitor, Manage, and Settings. These options are located on the left side
of the screen.

End of Lab Exercise

EMC Education Services 5

 Lab Exercise 2: Configure ECS Storage Infrastructure

Purpose: Using the ECS web portal, configure the core ECS storage
infrastructure elements for your system: Storage Pool(s), VDC(s) and
Replication Group(s)

 Login to the ECS web portal for management access to your

Tasks:
system.

 Create Storage Pool(s)

 Create VDC(s)

 Create Replication Group(s)

 Create VDC Federation and a Global Replication Group for it

References: Student Resource Guide

EMC Education Services 6

 Lab 2: Part 1 - Create Storage Pools
As you prepare the ECS Appliance to write data on it, there are specific abstracts that must be created in
order to guarantee a successful configuration. Let’s begin creating the storage pool.

Step Action

1 Bring up a browser and provide the IP address (192.168.73.54) of the site1 vECS-1
node. This will bring to the ECS Portal login screen. Provide the authentication below to
log into the remote ECS Portal:

User Name: root

Password: P@ssw0rd

2 When you login to the ECS portal for the first time, the GETTING STARTED checklist is
invoked. Since you will configure the system following the lab guide, click:

No thanks, I'll get started on my own

EMC Education Services 7

 Step Action

3 Go to the Storage Pool Management page by navigating to

Manage > Storage Pools > New Storage Pool

EMC Education Services 8

 Step Action

4 Enter the following information:

Name: pod#site1sp1 (where "#" is your Pod number, and the "1" at the end
indicates that this is the first storage pool you are creating. Example: pod1site1sp1)

From the Available Nodes field, select all nodes available (a minimum of 1 node is
required) and click the Add icon “+” to add them to the Selected Nodes area.

Notice the host name of your ECS node. Each node has a unique default name, and each
rack would have a unique color. These values make up the name that cannot be
changed. See the appendix at the end of this lab guide for more information.

5 When the nodes are selected click Save to create the storage pool.

EMC Education Services 9

 Step Action

6 Warning! The creation of the storage pool is a time sensitive step. You must allow a
minimum of 15 minutes for this to complete. The storage pool may show ready as its
status but you must not proceed to the next lab exercise until at least 15 minutes has
elapsed since the Save button was clicked. The status may show partially ready when 1
node is selected.

Continued on next page

EMC Education Services 10

 Lab 2: Part 2 - Create a VDC
Now that the storage pool has been created, it is time to configure the Virtual Data Centers, VDC. In this
lab exercise, you will create a single VDC

Step Action

1 Go to the Virtual Data Center Management page by navigating to Manage > Virtual Data Center.
Before creating the VDC, an Access Key must be generated. Click Get VDC Access Key.

2 When the key is generated, copy it since it will be required in the next step. Open a new Notepad an
paste the Access Key by using the <Ctrl>+<V> keys together.

EMC Education Services 11

 Step Action

3 Let's proceed to create the VDC by selecting Virtual Data Center under Manage and click New Virtu
Data Center.

EMC Education Services 12

 Step Action

4 On the New Virtual Data Center page, enter the following information to successfully create a VDC
within your assigned ECS Appliance:

Name: pod#site1vdc1 (where "#" is your Pod number, and the "1" at the end indicates that this i
the first VDC you are creating. Example: pod1site1vdc1)

Key: <Paste the Access Key generated from step 2>

Replication Endpoints: Enter the public IP address of each node in the VDC's storage pools
(192.168.73.54). Supply them as a comma-separated list.

Management Endpoints: Enter the public IP address of each node in the VDC's storage pools
(192.168.73.54). Supply them as a comma-separated list.

5 When the information is completed, click Save to create the VDC.

Warning! Allow at least five minutes for the VDC to become available before proceeding onto the
next lab exercise.

EMC Education Services 13

 Continued on next page

EMC Education Services 14

 Lab 2: Part 3 - Create Replication Groups
Now that the storage pool and VDC have been created, let’s configure the replication group. In this lab
exercise, you will create a local replication group.

Step Action

1 Navigate to Manage > Replication Group to open the Replication Group Management
page. Click New Replication Group to create a replication group for your pod.

EMC Education Services 15

 Step Action

2 On the New Replication Group page, enter the name of your Replication Group

Name: pod#site1rg1 (where "#" is your Pod number, and the "1" at the end indicates
that this is the first replication group you are creating. Example: pod1site1rg1)

Click Add VDC and select the VDC (Created in Lab1-Part2) and Storage Pool (in Lab1-Part1) from
the drop-down list.

3 Click Save to create the replication group.

Click the down arrow to the left of the name.

Once the replication group has been created, its status should be Online. Contact your
instructor if it is not.

Continued on next page

EMC Education Services 16

 Lab 2: Part 4 - Create VDC Federation and a Replication Group (Global)

Step Action

1 Bring up a browser and provide the IP address (192.168.73.56) of the site2 vECS-2 node. This will br
to the ECS Portal login screen. Provide the authentication below to log into the remote ECS Portal:

User Name: root

Password: P@ssw0rd

2 To create a storage pool which will be a part of the VDC that we will create, go to the Storage Pool
Management page by navigating to Manage > Storage Pools and click New Storage Pool.

EMC Education Services 17

 Step Action

3 Enter the following information:

Name: pod#site2sp1 (where "#" is your Pod number, Example: pod1site2sp1)

In the Available Nodes field, select the remote node and click the Add icon “+” to add them to th
Selected Nodes area.

Warning! The creation of the storage pool is a time sensitive step. You must allow a minimum of 15
minutes for this to complete. The storage pool may show ready as its status but you must not procee
to the next lab exercise until at least 15 minutes has elapsed since the Save button was clicked. The
status may show partially ready when 1 node is selected.

EMC Education Services 18

 Step Action

4 Go to the Virtual Data Center Management page by navigating to Manage > Virtual Data Center.
Before creating the VDC, an Access key must be generated. Click Get VDC Access Key. When the key
generated, copy it to the Notepad.

5 Once you have copied the site 2 key to Notepad you can close the vECS-2 Portal to avoid any confus
in the future. We will not need it the vECS-Site2 portal again.

6 MAKE SURE TO RETURN TO YOUR PRIMARY (SITE 1 vECS-1) NOW

Log in to site1 vECS-1 at 192.168.73.54 (in case you logged out).

EMC Education Services 19

 Step Action

7 On site1 vECS-1 go to the Virtual Data Center Management page by navigating to Manage > Virtual
Data Center. Click New Virtual Data Center to create a virtual data center.

EMC Education Services 20

 Step Action

8 On the New Virtual Data Center page, enter the following information to create a VDC within your
assigned ECS Appliance:

Name: pod#site2vdc1 (where "#" is your Pod number, Example: pod1site2vdc1)

Key: <Paste the Access Key generated for site2 from step 6>

Replication Endpoints: Enter the IP address of Site2 vECS-2 192.168.73.56

Management Endpoints: Enter the IP address of Site2 vECS-2 192.168.73.56

Click Save.

9 The VDC Federation is successfully created which is shown by two VDCs with two different endpoint

EMC Education Services 21

 Step Action

10 To create a global replication group for the VDC Federation, go to the Replication group managemen
page by navigating to Manage > Replication Group. Click New Replication Group to create a
replication group.

11 On the New Replication Group page, enter the name of your Replication Group

Name: pod#site2rg2 (where "#" is your Pod number, and the "2" at the end indicates that this is the
second replication group you are creating. Example: pod1site2rg2)

Click Add VDC and select both the VDCs (in the primary instance and remote instance) and corresponding
Storage Pools (in the primary instance and remote instance) from the drop-down list.

Click Save.

Note: When you go to select the second site VDC it may indicate that it is Temporary Unavailable. Wait for it
available so you can select the Storage Pool.

EMC Education Services 22

 Step Action

12 The Replication Group is successfully created.

End of Lab Exercise

EMC Education Services 23

 Lab Exercise 3: Basic Tests of I/O Access from Various Data Clients

Purpose: Using readily available data clients, test basic I/O access by
performing "CRUD" operations on ECS data repositories (commonly
referred to as "buckets")

 Create namespaces, local object users, and buckets for initial

Tasks:
testing of I/O access to your ECS system

 Perform ECS Metadata search

 Validate AWS S3 access to ECS using the S3 object browser

 Validate OpenStack Swift access to ECS using the CyberDuck

GUI tool

 Write and read Centera C-Clips to CAS-Enabled ECS Buckets

 Verify "data-in-place" access from a Hadoop client to S3

objects in an ECS bucket

References: Student Resource Guide

EMC Education Services 24

 Lab 3: Part 1 – Create ECS Namespaces, Local Users and Buckets
In this lab, you will perform the following tasks

• Create an ECS Namespace in ECS Portal as root user.

• Create an object user. Then, generate and retrieve S3 Access Key for that user.

• Create a bucket, and assign the object user as the bucket owner.

Step Action

1 Using http://vdc.emc.com Login to the VM that you have been assigned using the
username and password that has been shared with you.

You will perform all the lab exercises from this management station.

2 Using Chrome browser login to the site1 vECS-1 portal at 192.168.73.54 using the
credentials below

User Name: root

Password: P@ssw0rd

EMC Education Services 25

 Step Action

3 Navigate to Manage > Namespaces and on the Namespace Management page, click
New Namespace.

EMC Education Services 26

 Step Action

4 Enter the following details for the new namespace

Name: pod#ns1 (where "#" is your Pod number, and the "1" at the end indicates
that this is the first namespace you are creating)

User Admin: root

Note: A namespace can have more than one admin user. If there are
multiple admin users, enter comma separated user names in the User
Admin field. In this lab, we will keep things simple and make the root
user the namespace admin

Replication group: pod#site2rg2

Leave the remaining namespace options and configuration to the default value for this
lab.

Click Save.

EMC Education Services 27

 Step Action

5 After successful creation of a namespace, notice that it gets listed on the Namespace
Management page, as shown below.

You can at any time, use the Edit action to modify the Namespace properties. But note
that the Namespace name once created cannot be modified. You must delete the
namespace using Delete action, and recreate a new Namespace with the desired name.

6 Now, we need to create a user who can own a bucket and perform read and write
operations in it.

ECS has two types of user roles: Management users, who can perform ECS
administrative operations, and Object users, who can access ECS object storage for
CRUD operations (create, read, update and delete).

So let’s create a new object user for the namespace that we created in the previous
step. We will then use the object user to perform I/O operations on the bucket that we
will be creating soon.

Navigate to Manage > Users.

Click on Object Users, and then click New Object User.

EMC Education Services 28

 Step Action

7 Enter the following details for the new object user

Name: pod#ouser1 (“#”is your pod number, the "1" indicates that this is the first object
user you are creating)

Namespace: <Select your namespace>

An object user is mapped to a namespace, confining the user’s access only to the
buckets in the namespace the user is mapped to.

Click Next to Add Passwords to generate the S3 access key.

8 Generate & Add Password for the S3 client.

The Object Access section has options to generate password for various clients (S3,
Swift and CAS) that are supported for ECS object store access.

EMC Education Services 29

 Step Action

9 Click the S3 secret access key field (screenshot shown above). Press <Ctrl>+<A> to select
and <Ctrl>+<C> to copy the key to an editor such as Notepad.

You will need this key later to create an S3 account, and access the ECS object store
using S3 Browser application.

Click Close at the bottom of window.

10 Now that we have an object user created, let’s create a bucket with this object user as
the bucket owner.

11 Navigate to Manage > Buckets

Click New Bucket.

EMC Education Services 30

 Step Action

12 Enter the following details for the new bucket

Name: pod#bucket1 (# is your Pod number and the "1" is just our naming
convention, implying this is the first bucket that is created)

Namespace: <Select your Namespace>

Replication Group: pod#site2rg2

Bucket Owner: pod#ouser1 (the object user name you created. The bucket
owner will have the ability to modify bucket ACL and thus provide/remove
bucket access to other object users in the namespace)

EMC Education Services 31

 Step Action

13 Below are the other bucket configuration options. For now, leave all of these at default
values. You will experiment with some of these options in a later lab.

Quota: Set storage limit/quota on the bucket

File System: Enable/Disable file system access on the bucket

CAS: Enable/Disable CAS access for bucket

Access During Outage: Enable/Disable bucket access during site outage

Click Save.

EMC Education Services 32

 Step Action

14 Upon successful creation of bucket, you can see the bucket listed on the Bucket
Management page as shown below.

Note that you can filter and view the buckets in a particular namespace by selecting the
namespace from the Namespace drop-down list.

You cannot modify the bucket name, replication group and namespace attributes of a
bucket. But the Edit bucket option, under the Actions list, will allow you to change other
bucket properties like bucket owner, quota, ACLs, etc. which you will explore in
subsequent lab exercises.

Continued on next page

EMC Education Services 33

 Lab 3: Part 2 – Performing ECS Metadata Search

Step Action

1 Navigate to Manage > Buckets.

Click New Bucket.

2 Enter the following details for the new bucket (# indicates your Pod number).

Name: pod#bucket2
Namespace: pod#ns1
Replication Group: pod#site2rg2
Bucket Owner: pod#ouser1

EMC Education Services 34

 Step Action

3 Scroll down to Metadata Search. Click Enabled

4 To configure Metadata Search keys, the namespace admin must know the metadata
attributes that are required to be searchable. While system metadata attributes are
available to be selected, user metadata keys need to be manually entered.

The following is an example of attributes that can be added to a bucket. Attributes

change depending on customer needs:

image-width (integer)
image-height (integer)
image-viewcount (integer)
gps-latitude (decimal)
gps-longitude (decimal)

EMC Education Services 35

 Step Action

5 To configure these as search keys:

Click Add.

From the Metadata Key Type drop-down list, select User.

In the Key Name field, type x-amz-meta-image-width. The name is already prefixed,
complete the rest.

From the Data Type drop-down list, select integer.

Click Add.

EMC Education Services 36

 Step Action

6 To configure Additional Search keys, repeat the previous step for the remaining four
metadata search keys.

image-height integer

image-viewcount integer

gps-latitude decimal

gps-longitude decimal

When the five keys are complete, scroll down, and then click Save.

7 Verify that you have created an object user and provisioned 2 new buckets for the
teams in the media unit. They will now use the object user to ingest and access data

Continued on next page

EMC Education Services 37

 Lab 3: Part 3 - Test I/O Access to ECS from the AWS S3 Browser
In this lab, you will perform the following activities:

• Install and configure S3 Browser client

• Access the ECS storage using S3 Browser

• Perform CRUD operations on ECS buckets as an object user who you created in the
previous lab

Step Action

1 Install the S3Browser client (s3browser-5-5-3.exe) in your management station from

the C:\Lab path.

When installing, accept all the defaults, choose the Create shortcut to Desktop option.
That way, it will be easy for you to launch the application when required.

2 After installing S3 Browser, open it from the shortcut in your Desktop:

EMC Education Services 38

 Step Action

3 You will see the Add New Account screen.

Fill in the fields with the following details:

Account Name pod#ouser1 (Your object user name. # is your pod number)
Storage Type S3 Compatible Storage
192.168.73.54:9021
Endpoint can be the IP address of any one of the nodes you have
configured in the storage pool of your VDC.
REST Endpoint
ECS has specific port number designated for each client interface.
ECS S3 interface uses port 9020 for http, and port 9021 for https
connection
Access Key ID pod#ouser1 (# is your pod number)
<S3 secret access key>
S3 Secret access key of the object user that you generated is Lab3
Secret Access Key
Part1 Step8 and copied to notepad, from user management screen
in the ECS Portal
Note: # is your Pod number. See below for an example of how to fill in each field.

Click Add new account.

EMC Education Services 39

 Step Action

4 Soon after you add the account, the S3 Browser shows the 2 buckets pod#bucket1 and
pod#bucket2 that were created in the previous lab. You can see that in the left pane,
below.

This is because the object user was set as the bucket owner when the bucket was
created. Other object users in the same namespace cannot view this bucket until the
bucket owner modifies the ACL to allow a new object user to view and operate on a
bucket.

You may see a task that has failed, in the Tasks pane at the bottom of the S3 Browser,
as shown below. It is related to S3 Browser and does not concern ECS. So you may
ignore this error and proceed.

EMC Education Services 40

 Step Action

5 If you click the Permissions tab in the bottom pane, you can see that the object user
has Full Control permission set on both buckets, since the bucket owner by default,
would have full access over the bucket.

You will experiment with the bucket permissions also known as ACL (Access Control
List) for different object users in lab 3.

6 Now, click the Upload button and choose Upload files(s) to upload a file to the bucket.
Note: Use any of the files in the C:\lab\Files folder for testing uploads and downloads.

7 Now, upload more files into the bucket and try to download it using Download button.
You can also delete a file(s) using the Delete button.

8 Close your S3 Browser now.

Continued on next page

EMC Education Services 41

 Lab 3: Part 4 - Test I/O Access to ECS from CyberDuck (OpenStack Swift
objects)
In this lab, you will perform the following activities:

• Install and configure Cyberduck client

• Access the ECS storage using Cyberduck Browser

• Create an ECS OpenStack Swift bucket

• Perform CRUD operations on Swift bucket as an object user

Step Action

1 In your primary ECS (Site1 vECS-1) navigate to Manage Users.

Click Manage > Users > Object Users, and then click New Object User.

Create an object user named swiftuser1 to connect to the ECS using swift.

EMC Education Services 42

 Step Action

2 Leave Namespace at default pod1ns1

Click Next to Add Passwords.

3 Groups: admin

Swift password: swift.

Click the Set Password & Groups button.

Click Close when complete and the settings will be saved.

EMC Education Services 43

 4 Create a second object user named swiftuser2

Groups: admin

Swift password: swift

Click the Set Password & Groups button.

Click Close when complete and the settings will be saved.

EMC Education Services 44

 Step Action

5 Install Cyberduck from the C:\Lab directory on your lab machine’s virtual desktop.
During the installation do not install Bonjour. Accept other defaults.

Open Cyberduck.

Click the Add icon “+” on the bottom left.

EMC Education Services 45

 Step Action

6 In the New Connection dialog box, enter the following information:

Connection type: Swift (OpenStack Object Storage)

Nickname: 192.168.73.54-Swift
Server: 192.168.73.54 Port: 9025
Tenant ID:Access Key: pod#ns1:swiftuser1
(notice you are identifying the namespace and the user separated by a colon)

Close the dialog box with the X when done and settings will be saved.

7 Double-click the bookmark you just created.

EMC Education Services 46

 Step Action

8 Enter swift as the Secret Key and click Login.

EMC Education Services 47

 Step Action

9 Select the Always Trust check-box and then click Continue if there is a warning about
an invalid certificate. Click yes if prompted to install a security certificate.

EMC Education Services 48

 Step Action

10 Once the connection is open, go to File and click New Folder.

11 Name the folder container1. Click Create.

EMC Education Services 49

 Step Action

12 The container will be created and available for file upload, download, and delete. It will
appear in the ECS Portal as a bucket. Be sure to select the namespace the bucket was
created in and verify in your ECS instance that the new bucket was created.

ECS Portal:

EMC Education Services 50

 Step Action

13 In Windows Explorer navigate to C:\Lab\Files\ copy Test.txt file

Drag and drop Test.txt onto the container1 in CyberDuck.

If prompted about an invalid certificate, click Continue. This will copy the file to the
container, as shown below.

14 Click in the Bookmark icon in the CyberDuck application:

EMC Education Services 51

 Step Action

15 Configure Cyberduck for swiftuser2 by repeating the steps 5 to 10 of this lab exercise
using the information below. Choose Swift (OpenStack Object Storage) as the
connection type. For Tenant ID:Access Key you will use your pod # in pod#ns1.

In the New Connection dialog box, enter the following information:

Connection type: Swift (OpenStack Object Storage)

Nickname: 192.168.73.54-Swift2
Server: 192.168.73.54
Port: 9025
Tenant ID:Access Key: pod#ns1:swiftuser2

16 Double click the new bookmark icon for swiftuser2.

Enter in the password: swift
If prompted about an invalid certificate, click Continue.

EMC Education Services 52

 Step Action

17 You will see the container created by swiftuser1. This is because any ECS user with a
configured Swift password is placed by default in the admin group, and has full
permissions to all Swift containers.

EMC Education Services 53

 Step Action

18 (OPTIONAL STEP)
This step is optional this step is optional and may be performed or simply reviewed.

If you wish to limit container1 access, you will need to run some curl commands. You
can run curl by opening an SSH session (with credentials root/P@ssw0rd) to your
primary ECS node, using PuTTY in your virtual desktop.

The following commands assign object user swiftuser1 to group1, and configure the
bucket container1 with group1 permissions. In this example, any users in this group will
have read-only access to container1 after all the commands are run.

Note: Substitute values in <> as described. The # in <pod#ns1> should be substituted

with your pod #.

#Set variables for ECS login

export MANAGEMENT_ENDPOINT=https://<your-ecs-ip>:4443
export MANAGEMENT_USER=root
export MANAGEMENT_PASSWORD=P@ssw0rd

#Get authentication token

curl -I -s --location-trusted -k $MANAGEMENT_ENDPOINT/login -u
"$MANAGEMENT_USER:$MANAGEMENT_PASSWORD"

#Set variable for management token

export MANAGEMENT_TOKEN=<token-returned-by-last-command>

#Check management group of swift user

curl -s $MANAGEMENT_ENDPOINT/object/user-password/<swift-username> -k -
H "X-SDS-AUTH-TOKEN:$MANAGEMENT_TOKEN" -H "Accept: application/json"

#Set swift login variables

export SWIFT_USER=<swift-username>
export SWIFT_PASSWORD=<swift-password>
export SWIFT_ENDPOINT=https://<your-ecs-ip>:9025

#Authenticate using swauth as object user

curl -I -s -k -H "X-Auth-User:$SWIFT_USER" -H "X-Auth-
Key:$SWIFT_PASSWORD" $SWIFT_ENDPOINT/auth/v1.0

#Set variable for Swift token

export SWIFT_TOKEN=<X-AUTH-TOKEN response header value>

#Set group1 ACL for container

curl -I -s -k -X POST -H "X-Auth-Token:$SWIFT_TOKEN" -H "X-Container-
Read:group1" $SWIFT_ENDPOINT/v1/<pod#ns1>/<swift-container>

EMC Education Services 54

 Step Action

18 #Assign Swift user to group1

curl -s -X POST $MANAGEMENT_ENDPOINT/object/user-password/<swift-
username> -k -H "X-SDS-AUTH-TOKEN:$MANAGEMENT_TOKEN" -H "Accept:
application/json" -H "Content-Type:application/json" --data-binary
"{\"password\":\"<swift-
password>\",\"groups_list\":[\"group1\"],\"namespace\":\"<pod#ns1>\"}"

#Verify swift user can access container

curl -I -s -k -H "X-Auth-Token:$SWIFT_TOKEN" -H
"Accept:application/json" $SWIFT_ENDPOINT/v1/<pod#ns1>/<swift-
container>

Continued on next page

EMC Education Services 55

 Lab 3: Part 5 – Put and Get Centera C-Clips from ECS using CAS Tools
In this lab, you will perform the following activities:

• Create a CAS bucket and user

• Access the ECS storage using JCASScript

• Perform CRUD operations with JCASScript

Step Action

1 In the ECS Portal Manage > Bucket create a new bucket:

Name: pod#casbucket (where # is your pod number)

Namespace: pod#ns1

Replication Group: pod#site2rg2

Bucket Owner: root

Enable CAS and click save.

EMC Education Services 56

 Step Action

2 In ECS Portal Manage > Users create a new object user named pod#casuser (where # is
your pod number) using the existing namespace and click Next to Add Passwords.

EMC Education Services 57

 Step Action

3 Set the CAS password to caspassword.

Click the Set Password button.

Click Generate PEA File.

EMC Education Services 58

 Step Action

4 Copy the content of PEA File generated to the clipboard (Select the text and press
<CTRL> + <C>).

In Windows Explorer open Notepad and save the contents in a file named pea.p to your
Desktop. Click Close

5 From the ECS Portal, navigate to Manage > Buckets. On the Bucket Management page,
select your namespace so your buckets are listed. Once selected, open the
corresponding Actions drop-down list and choose Edit ACL for pod#casbucket (where #
is your pod number).

EMC Education Services 59

 Step Action

6 Click Add to add an ACL.

EMC Education Services 60

 Step Action

7 Fill in the User Name field with the CAS user name you’ve created in step 2 of this lab
exercise.
Be sure pod#casuser (where # is your pod number) has Full Control checked on the
bucket and click Save.

EMC Education Services 61

 Step Action

8 Go back to the Management > Users page and edit the user pod#casuser

Scroll down and set the Default Bucket to pod#casbucket.

Click the Set Bucket button.

Click Close.

EMC Education Services 62

 Step Action

9 Navigate to the folder C:\Lab

Extract the JCASScript-win32-3.2.35 .zip file to the C:\JCASScript-win32-3.2.35 folder

Move the pea.p file on your desktop to the C:\JCASScript-win32-3.2.35 folder.

10 Right Click the Start menu icon and open the Run box.
Type cmd and press Enter.
Right Click on the upper right corner on the window and select Properties
In Options Tab > Edit Options > Quick Edit Mode (allowing copy and paste)

EMC Education Services 63

 Step Action

11 Change Directory using cd C:\JCASScript-win32-3.2.35

Run the command java -jar JCASScript.jar to start the program.

12 Run the following command to connect to pod#casbucket :

poolOpen 192.168.73.54?pea.p

The IP address of any node of your ECS instance

Note: The command shown is using the relative path to the PEA file. The absolute path
can be specified alternatively using the following command:

“CASScript> poolOpen <ip_of_ECS>?C:\JCASScript-win32-

3.2.35\pea.p”

EMC Education Services 64

 Step Action

13 In your Windows VM, copy the file C:\Lab\Test.txt to the C:\ JCASScript-win32-3.2.35
directory.

14 Transfer the file and save it on the ECS in a clip in CAS bucket, run the command:

fileToClip Test.txt

A New Clip ID (Content Address) will be generated.

EMC Education Services 65

 Step Action

15 Run the command: clipOpen <contentAddress>

Using your mouse you can highlight and copy the new clip ID returned by the
“fileToClip” command as the “<contentAddress>”.

16 To view clip properties run the command: clipRawView.

EMC Education Services 66

 Step Action

17 To close the clip run the command: clipClose.

18 Run the command: clipTofile <contentaddress> savedclip.txt

This saves the clip to a file named “savedclip.txt” in your local C:\ JCASScript-win32-
3.2.35 directory.

EMC Education Services 67

 Step Action

19 To delete the clip from CAS bucket

run the command: clipDel <contentAddress>

20 Type exit and press Enter to quit JCASScript. Close the command window.

The CAS lab is now complete.

Continued on next page

EMC Education Services 68

 Lab 3: Part 6 – Test “Data-in-Place” Access to S3 Data within ECS
from Hadoop
In this portion of the lab, you will integrate Hadoop and ECS. Let’s first review the system architecture,
storage requirements, HDFS access protocols, and user credentials.

For simplicity, our system is predicated on a single Hadoop node, running HortonWorks HDP 2.3. For
management of Hadoop, you will access the Ambari Management portal via HTTP. In order to access ECS
storage, your Hadoop node will need to access the ECS appliance nodes via IP and be configured with
access to a S3 bucket with file system enabled.

Hadoop has two authentication modes: Simple and Kerberos. In our lab, we will implement Simple
authentication. With Simple authentication, Unix users can be users to ECS and they will appear as
‘anonymous’ to ECS and will have full control of the data space in the ECS bucket configured for HDFS.
Kerberos uses kinit and users must be specifically given permission to areas within a data space.

Please refer to your pod’s IP configuration document for the IP address of your Hadoop node.

NOTE: HDFS commands are case-sensitive!

EMC Education Services 69

 Step Action

1 Create/configure your HDFS enabled storage bucket.

From your ECS management portal, access the Manage > Buckets page and create a
new bucket for both S3 and HDFS access with the following parameters:

Name: pod#hdfsbucket (Where # is your pod number)

Namespace: pod#ns1
Replication Group: pod#site2rg2
Bucket Owner: pod#ouser1
File System: Enabled
Click Save when finished.

EMC Education Services 70

 Step Action

2 From the Manage > Buckets main page, on your created bucket, click Edit ACL from
the Actions drop-down list.

EMC Education Services 71

 Step Action

3 From the Bucket ACLs Management page, select Group ACLs. Add group name public
and provide all permissions

Now that we’ve set these parameters on our bucket, let’s configure Hadoop to access
ECS.

4 The Hadoop configuration consists of performing the following actions:

a) Installing the ViPRFS jar file on the Hadoop node

b) Stopping HDFS and MapReduce
c) Editing the core-site.xml file through the Ambari interface
d) Restarting services

Navigate to C:\lab\ folder and locate the hdfsclient-2.2.1.0.<version>.zip file. In this

lab exercise, the hdfsclient-2.2.1.0.<version>.zip file is located at C:\Lab in your
management station. This file contains the ECS HDFS JAR and support tools. This file is
also available on support.emc.com.

EMC Education Services 72

 Step Action

5 From the Windows command prompt copy the zip file to your Hadoop instance using the
“pscp” command as shown below:

cd \Lab
pscp hdfsclient-2.2.1.0.77331.4f57cc6.zip [email protected]:/var/tmp
if asked to “Store key in cache?” press: Y
if asked for Password type: hadoop

6 Using Putty ssh to the Hadoop instance at 192.168.73.51

User: root
Password: hadoop

EMC Education Services 73

 Step Action

7 Unzip the hdfsclient you pscp’d to /var/tmp and locate the latest jar file

cd /var/tmp

ls -lia

unzip hdfsclient-2.2.1.0.77331.4f57cc6.zip

cd viprfs-client-2.2.1.0.77331.4f57cc6

cd client

ls -li

8 Copy the jar file to a directory in Hadoop’s classpath. You need to copy it to the library
directory in Hadoop’s classpath. First, determine what the classpath is:

hadoop classpath

/usr/hdp/2.3.0.0-2557/hadoop/conf:/usr/hdp/2.3.0.0-
2557/hadoop/lib/*:/usr/hdp/2.3.0.0-
2557/hadoop/.//*:/usr/hdp/2.3.0.0-2557/hadoop-
hdfs/./:/usr/hdp/2.3.0.0-2557/hadoop-
hdfs/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-
hdfs/.//*:/usr/hdp/2.3.0.0-2557/hadoop-
yarn/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-
yarn/.//*:/usr/hdp/2.3.0.0-2557/hadoop-
mapreduce/lib/*:/usr/hdp/2.3.0.0-2557/hadoop-
mapreduce/.//*:::/usr/share/java/mysql-connector-java-
5.1.17.jar:/usr/share/java/mysql-connector-java-5.1.31-
bin.jar:/usr/share/java/mysql-connector-
java.jar:/usr/hdp/2.3.0.0-2557/tez/*:/usr/hdp/2.3.0.0-
2557/tez/lib/*:/usr/hdp/2.3.0.0-2557/tez/conf

EMC Education Services 74

 Step Action

9 Copy the *.jar file to /usr/hdp/2.3.0.0-2557/hadoop/lib

Note: In a multi-node Hadoop cluster, you will need to copy this jar file to every node,
and to the same lib directory.

cp -p viprfs-client-2.2.1.0-hadoop-2.7.jar /usr/hdp/2.3.0.0-2557/hadoop/lib/

10 Backup the core-site.xml file. The core-site.xml file contains properties specific to
components of Hadoop, i.e., MapReduce, HDFS, etc. In HortonWorks Hadoop, the
core-site.xml file is located in /etc/hadoop/conf

cp -p /etc/hadoop/conf/core-site.xml /etc/hadoop/conf/core-site.xml.orig

11 Login to the Ambari Management Portal to update/modify core-site.xml. Login by

using a web browser to: http://192.168.73.51:8080 and the following login
credentials:

Username: admin
Password: admin

EMC Education Services 75

 Step Action

12 Next we will be stopping both HDFS and MapReduce2 Services.

When you first login to Ambari, you’ll be presented with the dashboard. On the left-
hand side of the screen you will see a list of services. From this menu, select HDFS.

13 Next, open the Service Actions drop-down list and select Stop.

EMC Education Services 76

 Step Action

14 Confirm you want to stop the service, HDFS

Monitor the task and click OK when finished.

15 Perform the same procedure for the MapReduce2 Service.

16 Once both services have been stopped, it’s time to edit the core-site.xml file via the
Ambari interface. From the left-hand menu, select HDFS, and then select the Configs
tab. Next, select the Advanced tab.

Scroll down to the bottom of the page and select/open the Custom core-site menu.

EMC Education Services 77

 Step Action

17 Add ViPR/ECS specific values to Custom core-site. The core-site XML syntax is a basic
markup where you define key/value pairs. So, for instance, <thisisakey>and this would
be the value</thisisakey> would be the syntax you would find in a core-site.xml file.
However, with Ambari, all you need to define is the name of the key, and the
corresponding value.

At the bottom of the Custom core-site window, click Add Property.

EMC Education Services 78

 Step Action

18 Add the following properties to Custom core-site, via the Ambari interface. You can cut
and paste from here, modifying for your environment where needed.

Ensure that you add all 9 Key Value pairs!

Parameters are case sensitive.

Key Value

fs.AbstractFileSystem.viprfs.impl com.emc.hadoop.fs.vipr.ViPRAbstractFileSystem

fs.permissions.umask-mode 022

fs.vipr.installation.<Pod#>.hosts Enter the IP address of each node of your ECS.

Supply them as a comma-separated list.

Example:
fs.vipr.installation.Pod1.hosts
10.126.67.13,10.126.67.14,10.126.67.15,10.126.67.
16

fs.vipr.installation.<Pod#>.resolution dynamic
Example:
fs.vipr.installation.Pod1.resolution

fs.vipr.installation.<Pod#>.resolution. 900000
dynamic.time_to_live_ms
Example:
fs.vipr.installation.Pod1.resolution.dynamic.time_to
_live_ms 900000

fs.vipr.installations Pod# (Where # is your pod number)

Example:
Pod1 (This is case-sensitive)

fs.viprfs.auth.anonymous_translation CURRENT_USER

fs.viprfs.auth.identity_translation NONE

fs.viprfs.impl com.emc.hadoop.fs.vipr.ViPRFileSystem

EMC Education Services 79

 Step Action

19 Save your updated core-site configurations by clicking the Save button.

In the pop-up window, leave the Notes field blank and click Save again.

If the Configuration screen appears, click the Proceed Anyway button.

Once completed, the parameters should look like this:

Click OK on the Save Configuration Changes screen.

EMC Education Services 80

 Step Action

20 First Start the HDFS services which you stopped prior to adding your core-site
key/values.

Next Start the MapReduce2 service.

Click the service name, then from Service Actions, select Start.

21 First, test the integration from the S3 Browser.

Open your S3 browser using the pod#ouser1 account. Select the pod#hdfsbucket and
click the Upload button to upload a file to the bucket.

Note: Use any of the files in the C:\Lab\Files folder for this test.

EMC Education Services 81

 Step Action

23 Return to the Putty session (SSH connection) in your Hadoop instance. Run the
following commands and verify the object created:

hdfs dfs -ls

viprfs://yourhdfsbucket.yournameserver.yoursite/

Example:pod1ns1
hdfs dfs -ls viprfs://pod1hdfsbucket.pod1ns1.Pod1/

24 Now test the integration from your Hadoop node with CLI.

From the Putty session in your Hadoop node, test the following commands:

a) Verify whether local connectivity to HDFS is still operational.

hadoop fs -ls /

Example:
hdfs dfs -copyFromLocal /var/tmp/hdfsclient-
2.2.1.0.77331.4f57cc6.zip
viprfs://pod1hdfsbucket.pod1ns1.Pod1/tmp1

25 b) Verify connectivity to ECS storage. First, create a tmp directory and then verify that
the directory has been created.

hdfs dfs -mkdir

viprfs://yourhdfsbucket.yournameserver.yoursite/tmp1

hdfs dfs -ls

viprfs://yourhdfsbucket.yournameserver.yoursite/tmp1

Example:
hdfs dfs -mkdir viprfs://pod1hdfsbucket.pod1ns1.Pod1/tmp1

hdfs dfs -ls viprfs://pod1hdfsbucket.pod1ns1.Pod1/tmp1

EMC Education Services 82

 Step Action

26 c) Write data to your ECS bucket. For simplicity, write the zipped jar file for ViPR.

hdfs dfs -copyFromLocal file1

viprfs://yourhdfsbucket.yournameserver.yoursite/file2

Example:
hdfs dfs -copyFromLocal /var/tmp/hdfsclient-
2.2.1.0.77331.4f57cc6.zip
viprfs://pod1hdfsbucket.pod1ns1.Pod1/tmp1/

27 d) Verify the data.

hdfs dfs -ls

viprfs://yourhdfsbucket.yournameserver.yoursite/tmp1

Example:
hdfs dfs -ls viprfs://pod1hdfsbucket.pod1ns1.Pod1/tmp1

28 Return to the S3 Browser and verify that the data was written.

29 Close the Putty session and the Ambari Portal.

End of Lab Exercise

EMC Education Services 83

 Lab Exercise 4: ECS customizations: ACLs, Quotas and Retention

Purpose: Experiment with ECS features for access control (ACLs),

quotas and retention for object data

 Test ACLs with local object users in ECS

Tasks:
 Configure and verify the enforcement of quotas
within ECS

 Define retention policies and understand their effect

EMC Education Services 84

 Lab 4: Part 1 – Test ACLs with Local Object Users in ECS
This lab includes the usage of ACLs to control the access permissions on buckets for various object users.
You will perform the following tasks:

 Create a second, new object user in the existing namespace you created in the previous lab.
 Modify the bucket ACL to provide access to the new object user.
 Using the S3 Browser, verify that the ACL defined is regulating read/write access as you expected.

Experiment with the Group ACL option for a bucket

Step Action

1 Login to the Primary vECS-1 Portal at 192.168.73.54 using the following credentials:

User Name: root

Password: P@ssw0rd

2 From the ECS Portal, create a new object user:

Navigate to Manage > Users.

Click on Object Users, then click New Object User.

User name: pod#ouser2 (where # is your pod number)

Namespace: pod#ns1 (where # is your pod number)

Click Next to Add Passwords to generate the S3 access key.

Generate & Add Password for the S3 client.

<Ctrl>+<A> to select and <Ctrl>+<C> to copy the key to Notepad.

Click Close at bottom of window.

EMC Education Services 85

 Step Action

3 Next, create an account for this object user in S3 Browser. Because we are using the
free version of the S3 browser, we are only allowed to have two accounts.

In S3 Browser, under Accounts menu, select Add New Account.

Fill in the fields with the following details:

Account Name pod#ouser2 (Your object user name. # is your pod number)
Storage Type S3 Compatible Storage
192.168.73.54:9021
Endpoint can be the IP address of any one of the nodes you have
configured in the storage pool of your VDC.
REST Endpoint
ECS has specific port number designated for each client interface.
ECS S3 interface uses port 9020 for http, and port 9021 for https
connection.
Access Key ID pod#ouser2 (# is your pod number)
<S3 secret access key>
Secret Access Key S3 Secret access key of the object user that you copied from user
management screen in the ECS Portal.

Note: # is your pod number. See below for an example of how to fill-in each field:

Click Add new account.

EMC Education Services 86

 Step Action

4 After completing the previous step, you will now be using the new S3 account that was
just created for the pod#ouser2 user.

Change to the pod#ouser2 account:

Accounts > pod#ouser2

5 In the Bucket Explorer pane, S3 Browser will automatically list only the bucket(s) owned
by this pod#ouser2 user. To view other buckets that the same user has access to (via
ACLs), you must use the Add External Bucket under the Buckets menu of the S3
Browser.

In S3 Browser, navigate to Buckets Add External Bucket option.

EMC Education Services 87

 Step Action

6 Enter name of the bucket you created in the previous lab (pod#bucket1) and click Add
External bucket.

7 The bucket is listed in the left pane as in the example below:

EMC Education Services 88

 Step Action

8 Select the bucket to view the contents. You will see the following popup message.

Click Yes.

9 What do you see? You receive an error stating “Acces Denied.” Click OK.

This error occurs because the pod#ouser2 does not have read access on the bucket:

10 Now we will check what the bucket ACL looks like in the ECS Portal.

Login to the vECS-1 Portal as root with password: P@ssw0rd.

Navigate to Manage > Buckets.

EMC Education Services 89

 Step Action

11 Select your namespace from the Namespace drop-down list.

12 Choose the Edit ACL option from the Actions drop-down list of pod#bucket1.

13 You can see that there are two types of bucket ACLs shown:

 User ACL - enables admin user to provide read and write privileges on a bucket
for an object user.
 Group ACL - allows you set permissions for a set of pre-defined group.

We will first test User ACLs and then move on to Group ACLs.

As in the example below, you can see that by default, the User ACL has an entry for the
bucket owner with Full Control permission.

EMC Education Services 90

 Step Action

14 We want the pod#ouser2 user to read bucket contents, so we will add a new rule for
this user.

Click Add in the User ACLs.

Enter the object user name pod#ouser2 (# is your pod number).

You can see a list of available permissions. Unselect all the permissions except for Read.
We will assign only read privilege to the user.

Click Save.

15 Upon successful creation of the rule, you can see that the object user was added to the
User ACL list as seen below:

EMC Education Services 91

 Step Action

16 Return to the S3 Browser where pod#ouser2 is logged in and click Refresh.

Now you can see the files that you had uploaded to pod#bucket1 as pod#ouser1 user in
the previous lab.

17 Now, try performing an Upload operation.

Did you succeed? No, because the pod#ouser2 does not have write permission on the
bucket. You can see the “Access Denied” error in the Tasks pane at the bottom of S3
Browser as shown below:

What would you do to enable pod#ouser2 to perform write operations?

Experiment with various ACL permissions and test how they affect the operations you
can perform from the S3 Browser.

18 You tested how you could use ACLs to give permission to a user for bucket access.

Now you will see how Group ACLs can be used to provide permissions on a large set of
pre-defined user groups.

Below are the groups available in Group ACLs.

Public: All users, both authenticated and anonymous

All users: All authenticated users
Other: All authenticated users, except the bucket owner

EMC Education Services 92

 Step Action

19 Let’s first try the All users Group ACL. For this, you need to create a new object user in
the ECS Portal. From the ECS Portal, create a new object user:

Navigate to Manage > Users.

Click Object Users, and then click New Object User.

User name : pod#ouser3 (where # is your pod number)

Namespace : pod#ns1 (where # is your pod number)

Click Next to Add Passwords to generate the S3 access key

Generate & Add Password for the S3 client.

<Ctrl>+<A> to select and <Ctrl>+<C> to copy the key to Notepad.

Click Close at bottom of window.

20 Now, add a new Group ACL rule to allow all users to perform read operation. In the ECS
Portal, navigate to Manage > Buckets.

21 Select your namespace (pod#ns1) from the Namespace drop-down list.

Then, select Edit ACL from the Actions drop-down list for the pod#bucket1 bucket.

Select the Group ACLs tab.

You can see that the Group ACL does not have any rules. Click Add.

EMC Education Services 93

 Step Action

22 Select all users from the Group Name drop-down list.

Unselect all permissions except the Read permission, and click Save.

This rule will provide read permission on the bucket to all authenticated users.

23 Now, your Group ACL will appear as it does in the example below:

EMC Education Services 94

 Step Action

24 Now that you have Read permission set on the bucket for all authenticated users in the
same namespace, try to read this bucket as pod#ouser3 using the S3 Browser.

Note: S3 Browser free edition will allow a maximum of two accounts. Therefore, you will
receive a warning when you try to add a new account for pod#ouser3. Click No when
the pop-up message appears.

Delete the pod#ouser2 account by selecting Manage Accounts.

EMC Education Services 95

 Step Action

25 Add a new account for pod#ouser3.

26 Change the account by selecting Accounts > pod#ouser3.

EMC Education Services 96

 Step Action

27 Select Add external bucket to have the pod#bucket1 listed on the bucket explorer pane.

Now you can see that the pod#ouser3 is able to read the bucket. Note that there is no
ACL that specifically adds access to this particular user; our all users Group ACL enabled
the user to read buckets.

28 Let’s also experiment with the public Group ACL. Adding permission to this group
enables even anonymous, or unauthenticated, users to access the bucket.

S3 Browser will not allow us to create an account without any credentials. So we'll use
the curl command-line utility to test public access.

29 Connect to vECS-1 your node using putty to 192.168.73.54. (Putty executable is located
at C:\lab\putty in your management station.

Login: root
Password: P@ssw0rd

EMC Education Services 97

 Step Action

30 Issue the curl command below, which is an anonymous request to read the
pod#bucket1 bucket.
curl https://10.126.67.23:9021/pod#bucket1/ -H "x-emc-
namespace:pod#ns1" -k

Replace # with your pod number. In the command above, make sure to use Test.txt (or
some other small text file) that can be viewed with the Linux "cat" command. Note that
the file should have been already uploaded into the bucket by pod#ouser1.

As you can see below, you will receive the Access Denied error. This is expected,
because the bucket ACL does not permit anonymous user access.

EMC Education Services 98

 Step Action

31 Next in the ECS Portal, create a Group ACL which gives read permission to the public
group. This will allow both authenticated and anonymous users to perform read access
on the bucket.

Navigate to Manage > Buckets

Select your namespace (pod#ns1) from the Namespace drop-down list.

Then, select Edit ACL from the Actions drop-down list for the pod#bucket1 bucket.
Select the Group ACLs tab.

You can see that the Group ACL does not have any rules. Click Add.

Group Name: public

Permission: Read

Upon successful creation, the Group ACL of the bucket will appear as below:

EMC Education Services 99

 Step Action

32 Now from the putty session, re-run the curl command:

curl https://10.126.67.23:9021/pod#bucket1/ -H "x-emc-

namespace:pod#ns1" -k

Verify that the command succeeded.

Continued on next page

EMC Education Services 100

 Lab 4: Part 2 – Define ECS retention policies and Study Their Effect
This lab includes the usage of retention policy and retention period on objects and bucket. You will
perform the following tasks:

• Create retention policies in namespace.

• Set retention period on bucket.

• Use s3curl to create objects with retention policies and retention period.

• Experiment with bucket and object retention and determine which take precedence.

Step Action

1 Let us first experiment with retention period option on buckets.

Login to the vECS-1 Portal using the below credentials:

User Name: root

Password: P@ssw0rd

2 Navigate to Manage > Buckets

On the Bucket Management page, select your namespace pod#ns1 from the drop-
down list.

For the pod1bucket1 bucket, open the corresponding Actions drop-down list and select Edit
Bucket.

EMC Education Services 101

 Step Action

3 On the Edit Bucket page, you can see the Bucket Retention section.

The bucket retention period is set at the bucket or object level. It prevents the objects
to be modified or deleted until the retention period elapses, after the original object
creation time.

The bucket retention period can be set in units ranging from seconds to years.

There is also an Infinite option which when checked, prevents any modification of the
object indefinitely.

For our experiment: set the bucket retention to 1 month.

Click Save.

4 Now, from the S3 Browser, select the pod#ouser1 account.

Click on a file in pod#bucket1 to select it, and then click Delete.

EMC Education Services 102

 Step Action

5 Click Yes in the Confirm File Delete message.

6 The delete operation will fail because the object/file creation time is not more than the
1 month retention period that you had set on the bucket. You can see the error
message by clicking on the Failed task in the Tasks pane at the bottom of the S3
Browser.

As you can see below, the status message states that the object cannot be deleted
because it is subject to retention:

7 Modify the retention period of bucket to a smaller duration (duration less than the
current age of your test object, based on its creation time). Try again to delete the
object in the bucket. You can see that the Delete operation succeeds without any
problem.

8 Next, let us experiment with retention at the object level using a retention policy.

Retention policies can be configured for the Namespace. You can create multiple
retention policies in a Namespace, and set them to appropriate objects using S3 curl
commands.

Navigate to Manage > Namespace and click Edit on your pod#ns1 Namespace.

EMC Education Services 103

 Step Action

9 In the Create Retention Policy section, click Add.

Enter the following values for the new retention policy.

Name: pod#rpolicy1 (# is your pod number)

Value: 10 minutes

Click Add.

10 Create another retention policy using the values below:

Name: pod#rpolicy2 (# is your pod number)

Value: 20 minutes

Click Save.

You will use these two retention polices pod#rpolicy1 and pod#rpolicy2 on two
different objects in pod#bucket1 and test how retention works.

EMC Education Services 104

 Step Action

11 ECS Portal does not offer the ability to set retention policy on objects. You will need to
use s3curl utility to set this option.

S3curl is the Amazon S3 authentication tool for curl. Since ECS uses custom header
with x-emc string prefixed, s3curl script should be modified to include the x-emc in the
header attribute.

You can find the pre-modified s3curl.pl file at C:\lab\s3curl path in your management
station. You can find more details on modifications to be made on s3curl.pl file at
http://www.emc.com/techpubs/ecs/ecs_create_bucket-1.htm#GUID-2E37CDB4-12FB-
4BA7-9379-7D45044331E2

The C:\lab\s3curl path has these two files

 s3curl.pl – The modified s3curl file to include x-emc in header.

 dot_s3curl.txt – The sample configuration file containing the
authentication details.

Copy these files to any one of your ECS nodes using the commands below.
On your Windows Management Host open a command prompt.

cd c:\lab\s3curl

C:\lab\s3curl> pscp –pw P@ssw0rd s3curl.pl

[email protected]:s3curl.pl

C:\lab\s3curl> pscp –pw P@ssw0rd dot_s3curl.txt

[email protected]:.s3curl

Note: The dot_s3curl.txt should be named as .s3curl on the ECS node, and reside in
the home directory of the root user.

EMC Education Services 105

 Step Action

12 Log in to the ECS node as root using PuTTY, located in C:\lab\putty path in your
management station.

Edit the .s3curl file that you have copied to the root directory.

vi .s3curl

%awsSecretAccessKeys = (
my_profile => {
id => 'pod#ouser1',
key => ‘<S3 Secret Access Key copied from ECS Portal>'
},
root_profile => {
id => 'root',
key => 'P@ssw0rd'
},
);
push @endpoints,(’192.168.73.54',’logangreen.emc.edu’,
);

You need to update the my_profile with your object user’s credentials. Update the
endpoints with the IP address of the ECS node that you are currently logged into and
its hostname. (Run the “hostname” command to get the FQDN of your ECS node).

Perform the above changes, and save the .s3curl file.

EMC Education Services 106

 Step Action

13 Change the permission on the s3curl files by running:

chmod 600 /root/.s3curl
chmod 755 /root/s3curl.pl

14 In the putty session, run the below command to test if s3curl is functional.

./s3curl.pl
If everything is properly configured, it should display the s3curl help.

15 Now let us try to upload a file to the pod#bucket1 bucket as an object, and set
retention policy on that object.
You will need new files in your ECS node to test the retention policy feature.

Copy a few small files from C:\lab\files location in your management station to the ECS
node using pscp– back on your Windows host in the Command Prompt.

cd c:\lab\files

C:\lab\files pscp –pw P@ssw0rd test.txt

[email protected]:test.txt

EMC Education Services 107

 Step Action

16 Now, on the ECS Node in Putty run the s3curl command as below:

./s3curl.pl --debug --id=my_profile --put <local file> -- -H 'x-

emc-retention-
policy:<pod#rpolicy1>'https://192.168.73.54:9021/pod#bucket2/tes
t.txt -k

You can see that the command has executed successfully.

EMC Education Services 108

 Step Action

17 Now, go to the S3 Browser and click Refresh.

Click on the file that you uploaded using s3curl.

Then, select the Http Headers tab in the bottom pane as in the example below.

You can see that there is a new header x-emc-retention-policy set with the retention
policy as value. You will not find this header for other files that you uploaded directly
from S3 Browser.

18 Click on other files uploaded through S3 Browser and check their headers.

Using a retention policy with objects instead of hardcoding a retention period value
provides more manageability. Any change to the retention policy automatically applies
to every object configured with that particular retention policy.

EMC Education Services 109

 Step Action

19 Similar to the above, you can upload other objects and set a different retention policy
on them. Upload another sample file with the pod#rpolicy2 retention policy using
s3curl, and check its http header.

20 Now, try to delete the file before the retention policy expires.

Remember that the pod#bucket1 bucket already has a retention period on it. In order
to avoid conflicts, you may want to disable the bucket-level retention period (set it 0
seconds) before you try the retention policy use case.

Similar to the retention period set on a bucket, the retention policy will not let you
delete the object until the object life time exceeds the time period specified via the
retention policy.

EMC Education Services 110

 Step Action

21 You can also set a specific retention time period on objects using s3curl commands.
Return to the ECS node session in putty.

Create a new file for upload using the following command:

echo “retention period test” >> retentionperiod.txt

Now, run the s3curl upload command shown below:

./s3curl.pl --debug --id=my_profile --put retentionperiod.txt --

-H 'x-emc-retention-period:600'
https://192.168.73.54:9021/pod#bucket2/retentionperiod.txt -k

Note: The unit of retention period in the above command is in seconds. In the
command, you are setting object retention of 10 minutes on the retentionperiod.txt
file.

You can see above that the command has executed successfully.

We will now verify this in the S3 Browser.

EMC Education Services 111

 Step Action

22 Return to your S3 Browser and click Refresh.

Do you see the new file you uploaded in the previous step? Click the file to select it.

Select the Http Headers tab and view the headers.

In this case, there is a new header x-emc-retention-period.

23 Repeat the delete file operation with its retention period set.

EMC Education Services 112

 Step Action

24 At this point, you understand what retention period and policies are, and how they
work on object and bucket level.

Next, experiment with which takes precedence, the retention set at bucket level or the
object level. You can do this by using the following scenario:

Set retention period on pod#bucket1 to 10 minutes.

Set retention period on object to 5 minutes.

Now, try to delete the object after 5 minutes. What happens, are you able to delete
the object?

Next, you can try the reverse: set the retention period on the bucket to be less than
the retention period of the object. Then try deleting the object and observe the
behavior.

Continued on next page

EMC Education Services 113

 Lab 4: Part 3 – Advanced Retention Management
This lab includes applying advanced retention settings to a CAS Bucket. You will be applying the following
settings using the Min/Max Governor:

• Enforce Retention Information in Object

• Bucket Retention Period

• Minimum Fixed Retention Period

• Maximum Fixed Retention Period

• Minimum Variable Retention Period

• Maximum Variable Retention Period

Step Action

1 Login to the vECS-1 portal at 192.168.73.54 using the credentials below:

User name: root

Password: P@ssw0rd

2 Navigate to Manage > Buckets. On the Bucket Management page, select your
namespace pod#ns1 from the Namespace drop-down list.

EMC Education Services 114

 Step Action

3 For the pod1casbucket bucket created previously in Lab 3 Part 5, open the
corresponding Actions drop-down list and select Edit Bucket.

4 On the Edit Bucket page, scroll down to the Bucket Retention section and click Show
Options.

Upon clicking the button, the options for advanced retention settings are displayed.
Below is the detailed description of the options displayed.

Enforce Retention Information in Object: If this control is enabled, no CAS object can
be created without retention information (period or policy). An attempt to save such an
object will return an error.

Bucket Retention Period: The bucket retention period is set at the bucket or object
level. It prevents the objects to be modified or deleted until the retention period
elapses, after the original object creation time. If both a bucket-level and an object-level
retention period are set, the longer period will be enforced on the bucket. In a
Compliance-enabled environment, Bucket Retention Period is mandatory unless
retention information in the object is enforced.

Minimum/Maximum Fixed Retention Period: This feature governs the retention

periods specified in objects. If an object's retention period is outside of the bounds
specified here, an attempt to write the object fails.
Min/max retention constrains are applied to any C-Clip written to a bucket. If a clip is
migrated by any SDK-based third-party tool, the retention should be within bounds.
Else, an error is thrown.

Minimum/Maximum Variable Retention Period: This feature governs variable

retention periods specified in objects using Event-Based Retention (EBR). If an object's
new retention period is outside of the bounds specified here, an attempt to write the
object in response to the trigger fails.

EMC Education Services 115

 Step Action

5 The retention period can be set in units ranging from seconds to years. There is also an
Infinite option which when checked prevents any modification of the object indefinitely.

For this exercise, set the following values:

Enforce Retention Information in Object: Enabled

Bucket Retention Period: 1 Years

Minimum Fixed Retention Period: 1 Years

Maximum Fixed Retention Period: Infinite

Minimum Variable Retention Period: 3 Years

Maximum Variable Retention Period: Infinite

Click Save.

Continued on next page

EMC Education Services 116

 Lab 4: Part 4 – Configure and verify enforcement of ECS quotas
In this lab, you will experiment on implications of setting hard quota at the namespace and at the bucket
level. You will perform the following tasks

• Create a management user (Namespace Administrator)

• Create a new namespace with the Namespace Administrator as the owner

• Enable hard quota on the namespace

• Create two buckets in the namespace, with one of the buckets enabled with hard quota

• Test the quota behavior

Step Action

1 Login to the vECS-1 portal at 192.168.73.54 using the credentials below:

User name: root

Password: P@ssw0rd

EMC Education Services 117

 Step Action

2 First, we will create a new Namespace Administrator.

Namespace Administrator is a management user without system administrative

privileges. The Namespace Administrator has permission to manage buckets and users in
the Namespace this user owns.

Navigate to Manage > Users.

Select the Management Users tab.

Then, click New Management User.

EMC Education Services 118

 Step Action

3 On the New Management User page, enter the information below.

Select Local User option (a Namespace Administrator can be a local ECS user or a user in
Active Directory).

Name: pod#nsadmin (# is your pod number)

Password: P@ssw0rd
System Administrator: No (Leave the default value)
System Monitor: No (Leave the default value)

Click Save.

Click OK in the warning.

Note: As mentioned on the New Management User page, a management user without
the System Administrator rights will be able to login to the ECS portal only if the user is
mapped as a Namespace Administrator for a namespace.

4 After successful creation of management user, you can see the user listed on the
Management User page.

EMC Education Services 119

 Step Action

5 The next step is to create a new namespace, mapping the management user created in
previous step, as the Namespace Admin. You will also enable hard quota setting on this
namespace.

Navigate to Manage > Namespace

Create a new namespace with the following details:

Name: pod#ns2 ( # is you pod number)

Admin: pod#nsadmin ( # is you pod number)
Replication Group: pod#site2rg2
Quota: Enabled with 'Block access at' set to 2 GB

There are two options available to choose from, related to quota:

 Notification Only at-Known as soft quota, this option will trigger a notification
when the capacity used reaches the specified limit
 Block Access at-Known as hard quota, this option will block any further upload
operation when the quota limit is reached. This also sends a notification when a
specified percentage of the quota is reached.

Note: 1 GB is the minimum value that can be set for the quota.

Click Save.

EMC Education Services 120

 Step Action

6 Now that you have a namespace created, the next step is to login to the ECS Portal as
Namespace Administrator and create buckets in the namespace.

Log out of the portal and log in as Namespace Administrator using the credentials below:

User Name: pod#nsadmin (# is your pod number)

Password: P@ssw0rd

7 As a Namespace Administrator, you will now create an object user. This object user will
be used to perform read and write operations on the buckets created in the pod#ns2
namespace.

 Manage > Users

 Create new local object user pod#ns2ouser1
 Map the user to the pod#ns2 namespace
 Generate S3 secret access key and copy it to Notepad

8 Now, create a bucket in the namespace with the pod#ns2ouser1 created in the previous
step as the owner. You will also enable quota on this bucket.

Navigate to Manage > Buckets

On the Bucket Management page, select your namespace pod#ns2 from the drop-down
list.

Click New Bucket and create a bucket with the following details:

Name: pod#bucket1
Replication Group: <Your replication group>
Namespace: pod#ns2
Bucket Owner: pod#ns2ouser1 (object user you created earlier in this lab)
Quota: Enabled with 'block access at' set to 1 GB

Note: Similar to the namespace quota, a hard quota is set on this bucket to prevent
upload operations when the bucket’s quota limit is reached.

Click Save.

EMC Education Services 121

 Step Action

9 Now we will create another bucket in the same namespace pod#ns2, but this bucket will
not be quota enabled.

Use the following details to create new bucket:

Name: pod#bucket2
Replication Group: <Your replication group>
Namespace: pod#ns2
Bucket Owner: pod#ns2ouser1 (object user you created earlier in this lab)
Quota: Disabled

10 Upon successful creation of the buckets, the Bucket Management page will appear as in
the example below. You can see that pod2bucket1 has 1 GB of hard quota enabled and
pod2bucket2 does not have any quota set.

EMC Education Services 122

 Step Action

11 Now try to perform upload operation on these buckets.

Start the S3 Browser Accounts > Manage Accounts > delete pod#ouser3.

Create new account for pod#ns2ouser1. Fill in the fields with the following details:

Account Name pod#ns2ouser1 (Your object user name. # is your pod number)
Storage Type S3 Compatible Storage
192.168.73.54:9021
Endpoint can be the IP address of any one of the nodes you have
configured in the storage pool of your VDC.
REST Endpoint
ECS has specific port number designated for each client interface.
ECS S3 interface uses port 9020 for http, and port 9021 for https
connection
Access Key ID pod#ns2ouser1 (# is your pod number)
<S3 secret access key>
Secret Access Key S3 Secret access key of the object user that you copied from user
management screen in the ECS Portal

Note: # is your Pod number. See below for an example of how to fill in each field.

Note that you can switch between user accounts any time: Select the Accounts tab and
then select the required account name

12 In the S3 Browser Bucket Explorer pane on the left, you can see the buckets
pod#bucket1 and pod#bucket2 are listed by default. This is because the pod#ns2ouser1
is the owner of both the buckets.

Now upload three files into the pod#bucket1 from C:\lab\files path in your management
station.

Choose files with size around 350 MB for the upload operation.

EMC Education Services 123

 Step Action

13 You can see below that pod1bucket1 has approximately 1 GB of files in it.

14 Similarly, upload two files to pod#bucket2 with total size not more than 1 GB.

You can also check the number files in a bucket and the total object size in it from the
Properties tab in the bottom of S3 Browser.

Select the bucket name and then select the Properties tab to view the corresponding
information.

EMC Education Services 124

 Step Action

15 To test the quota option, it is very important to check the ECS Metering and ensure that
the number of objects in the buckets (pod#bucket1 and pod#bucket2) listed on the
Metering page match the actual number of files/objects in the bucket.

To verify the object count in ECS, Navigate to Monitor > Metering.

Select the namespace from the list using the Add icon “+”and select the bucket from the
list using the Add icon “+”.

Click Apply.

This will list the number of objects in the bucket.

Wait for approximatelyu 30 – 40 minutes for the Object Count to reflect the actual
number of files in the bucket.

16 As you can see below, the Object Count should display the actual number of objects
uploaded in the buckets.

EMC Education Services 125

 Step Action

17 In S3 Browser, log in using the pod#ns2ouser1 account.

Upload a small file, of a few KB in size to pod#bucket1 from C:\lab\files location in your
management station.

Note that the pod#bucket1 already has files of around 1 GB size in it. So when you try to
upload additional file, the upload operation will fail based on the Block Access at setting
that you had defined.

Select the Tasks tab and then click Failed.

You can see that the status shows “Failed – Forbidden: Check if quota has been
exceeded” error.

18 Now try to upload a file to pod#bucket2 bucket. This also fails.

You did not enable quota on this bucket, so why did the upload operation fail?

EMC Education Services 126

 Step Action

19 As root user, navigate to Monitor Events.

Highlighted below are the quota exceeded notifications for the namespace as well as the
bucket.

End of Lab Exercise

EMC Education Services 127

 Lab Exercise 5: Multi-tenancy with Active Directory/LDAP Integration

Purpose: Using readily available data clients, test basic I/O access by
performing "CRUD" operations on ECS data repositories (commonly
referred to as "buckets")

 Review the pre-configured domain, groups and domain users

Tasks:
in your Active Directory server

 Configure the Active Directory server as an Authentication

Provider to ECS

 Create and customize two namespaces to service a dual-

tenant environment (Finance and Sales groups)

 Test I/O client access to ECS from Active Directory domain

users in both tenants

EMC Education Services 128

 Lab 5: Part 1 – Review the Configuration of an Active Directory Server
This lab includes the Active Directory setup used for the multi-tenancy.

Step Action

1 To demonstrate the multi- tenancy feature of ECS, the following structure is created in
Active Directory.

Two user groups named Finance and Sales are created in AD. These groups will be
considered as individual tenants and they will have their own namespace created in
ECS.

Note that this structure is used for simple proof-of-concept purposes only. We have a
single Active Directory server, which would be a realistic representation of an Enterprise
customer of ECS, with multiple business units within the enterprise representing ECS
tenants. All business units are sharing a single Active Directory setup.

EMC Education Services 129

 Step Action

2 In this experiment, each user group within Active Directory (i.e. each tenant) will have
two types of users: Admin user and Object users. All users will have same AD privilege
and they will be part of two AD groups: Domain users user group, and the user group
named by their tenant.

Shown below are the properties of fadmin and fuser for the Finance tenant. Similarly,
Sales group will have sadmin and suser users who are members of Domain users and
Sales group.

From the ECS perspective, the Admin users (fadmin and sadmin) will be considered as
management users - specifically, namespace admins. They will have access to ECS
Portal with limited capabilities - each can manage their own namespace, e.g. add or
remove users in their namespace.

fuser1, fuser2, suser1 and suser2 are ECS Object users who will have access only to the
ECS object store, to perform CRUD operations.

In Active Directory, all users have been configured with ChangeMe1 as their password.

The above Active Directory structure is pre-created, and made available for you in this
lab. You will use these Active Directory details to add your authentication provider from
the ECS Portal.

Continued on next page

EMC Education Services 130

 Lab 5: Part 2 – Add Active Directory server as an ECS authentication
provider
In this lab, you will add an Active Directory server as the authentication provider to your ECS instance

Step Action

1 Login to the ECS Portal using the credentials below:

User Name: root

Password: P@ssw0rd

2 Navigate to Manage > Authentication

3 On the Authentication Provider Management page, click New Authentication Provider.

EMC Education Services 131

 Step Action

4 On the New Authentication Provider page, enter the following values:

Name ECS AD authentication provider

Description ECS AD authentication provider
Type Active Directory
Domains corp.emc.edu
Server URLs ldap://192.168.73.48
Manager DN CN=administrator,CN=Users,DC=corp,DC=emc,DC=edu
Manager password P@ssw0rd
Providers Enabled
Group attributes CN
*Finance*
Group whitelist *Sales*
*Domain Users*
Search Scope Subtree
Search Base CN=Users,DC=corp,DC=emc,DC=edu
Search filter userPrincipalName=%u

The Group whitelist above, will list the Active Directory groups which will be allowed to
access the ECS storage.

Click Save.

5 Upon successful addition of the ECS authentication provider, your authentication

provider management screen will appear as below.

You can use the Edit option from the Actions, if you need to modify the authentication
provider.

6 You will use this authentication provider in the next lab to create namespaces with
domain configuration.

Continued on next page

EMC Education Services 132

 Lab 5: Part 3 – Configure ECS Namespaces with Domain Groups
for Multi-tenancy
In this lab, you will create namespaces with domain configuration.

Step Action

1 Login to the ECS Portal using the following credentials:

User Name: root

Password: P@ssw0rd

2 Next, we need to create namespaces for the tenants (Finance and Sales) with the
domain details.

Navigate to Manage > Namespace

Click New Namespace.

3 Enter the following values for the new namespace:

Name: pod#financens
User Admin: [email protected]
Domain Group Admin: [email protected]
Replication Group: pod1site2rg2

EMC Education Services 133

 Step Action

4 Click Domain to add the domain.

Enter the following values for the domain configuration:

Domain: corp.emc.edu
Groups: Finance (This namespace will be assigned for Finance tenant users)
Attribute: objectCategory
Values: CN=Person,CN=Schema,CN=Configuration,DC=corp,DC=emc,DC=edu

Click Save.

5 Upon successful creation of a namespace, it is listed on the Namespace Management

page as shown below.

6 Now, try to login to ECS Portal as the Namespace Administrator using these credentials:

User Name: [email protected]

Password: ChangeMe1

How are these credentials being checked? Is it done by ECS, or by some other
component in your environment?

EMC Education Services 134

 Step Action

7 Navigate to Manage Namespace.

Notice that the Namespace Management page has only one namespace listed, which is
owned by [email protected].

When you log in as this Namespace Admin, you can only view the namespace that this
Admin owns.

8 Navigate to other ECS management views like Storage pools, VDC etc. Are you able to
view the details?

You cannot see those details because the Namespace Administrator’s access is limited
to bucket and object user management of a namespace. The user will not be authorized
to view other ECS system administrative attributes.

9 Now navigate to the User Management page and add a new domain object user using
the following details:

Name: [email protected]
Namespace: pod#financens (# is your pod number)

Now, log off of the portal and log in as [email protected] using AD password. You
can see that the authentication succeeds against LDAP but the user will not be able to
view or perform any operation in the ECS Portal because the user is not authorized.

Log off of the portal.

EMC Education Services 135

 Step Action

10 Log in to the ECS Portal as root user with P@ssw0rd as the password.

Navigate to the Namespace Management page and create another namespace for the
Sales tenant using the below details.

Name: pod#salesns
User Admin: [email protected]
Domain Group Admin: [email protected]
Replication Group: pod#site2rg2
Domain: corp.emc.edu
Groups: Sales
Attribute: objectCategory
Values: CN=Person,CN=Schema,CN=Configuration,DC=corp,DC=emc,DC=edu

11 Now, log off from the portal and login as Sales namespace administrative user using
these credentials:

User Name: [email protected]

Password: ChangeMe1

12 Navigate through different pages and observe what this user is able to view and the
actions the user is able to perform.

Were you able to see other namespaces and their object users?

Continued on next page

EMC Education Services 136

 Lab 5: Part 4 – Verify I/O Access to ECS from Tenant Users
In this lab, you will verify data isolation in the ECS multi-tenant setup by performing I/O operations as
different tenant object users.

You will also explore the self-service REST API feature available for domain users to create an object user
account for themselves, and claim their S3 secret access key.

Step Action

1 We will first explore the self-service ECS REST API to authenticate as a domain user and
then create a S3 secret access key.

Log in to one of your ECS nodes using putty using the following credentials:

Login: root
Password: P@ssw0rd

Note: Putty executable is available in C:\lab\putty path in your management station.

2 First, you need to authenticate as a domain user and get a cookie file for subsequent
REST calls.

Run the command below from the root path to authenticate as domain user, fuser1.

If you had created an object user for fuser1 in the previous lab, delete it before you try
the below command.

curl -L --location-trusted -k
https://192.168.73.54:4443/login?using-cookies=true -u
"[email protected]:ChangeMe1" -c cookiefile –v

Example

EMC Education Services 137

 Step Action

3 Run ls –li command to verify whether the cookie file is generated. You should see a
file named cookiefile.

4 Then, issue the REST API call to retrieve the S3 secret access key for the user. Note that
the cookiefile is being passed as one of the arguments for authentication.

curl -k https://192.168.73.54>:4443/object/secret-keys -b
cookiefile -v -H "Content-Type: application/json" -X POST -
d "{}"

EMC Education Services 138

 Step Action

5 Successful completion of the REST call generates the S3 secret key for the
[email protected] domain user.

The above REST call not only creates the S3 secret key but also creates an object user in
ECS.

Return to the ECS Portal and verify whether a new object user has been created on the
User Management page.

EMC Education Services 139

 Step Action

6 Now that you have secret access key and object user created for the domain user
[email protected], follow the steps below to perform read/write operations in the
S3 Browser.

The trial version of the S3 Browser only allows up to two accounts, you will need to
delete one: S3 Browser Accounts > Manage Accounts > Delete pod#ns2ouser.

Create a new account for [email protected] using the secret access key from the
ECS Portal.

EMC Education Services 140

 Step Action

7 Create a new bucket for [email protected].

Click New bucket.

Enter a bucket name as pod#fuser1bucket and click Create new bucket.

8 Upload few files from C:\lab\Files path in your management station.

9 (OPTIONAL STEP): this step is optional and may be performed or simply reviewed.

Use the self-service REST API call to create an object user, and generate S3 secret key
for suser1 who belongs to the Sales tenant group.

Then, create a bucket for this user in S3 Browser. You can then test the multi-tenancy
data isolation by trying to read the buckets created by Finance tenant users. Follow the
instructions in Lab 3: Part 1 “Test ACLs with local object users in ECS” to create ACLs and
add external bucket.

End of Lab Exercise

EMC Education Services 141

 Lab Exercise 6: ECS Metering and Monitoring

Purpose: Test the metering and monitoring capabilities that are provided in
the ECS web portal.

 Examine tenant-specific and user-specific metering data

Tasks:
available from the web portal

 View the available monitoring data from the portal for single-
site and multi-site environments

References: EMC Elastic Cloud Storage (ECS) Version 2.0 ECS Documentation 302-
001-980 01

EMC Education Services 142

 Lab 6: Part 1 – View ECS Monitoring Data and Perform Basic Health Checks
In this lab, you will perform the following tasks:

• Explore the metering capabilities of ECS using the ECS Portal

• View resource usage using ECS monitoring features

• Check hardware health and monitoring history

Step Action

1 Log in to the ECS Portal as Username: root password: P@ssw0rd

On the ECS Portal, expand the Monitor menu and select Metering.

In the Date Time Range drop-down list, select Custom. In the From field, enter
yesterday’s date. Similarly, in the To field, enter today’s date.

From the Select Namespaace list box, highlight pod#ns1. From the Select Buckets list
box select pod#bucket1 by using the Add icon “+”. Click Apply. This will show object
metrics and traffic that have occured within the past day in pod#bucket1.

EMC Education Services 143

 Step Action

2 From the Monitor menu, select Events and observe the recent events that have
occoured during the course of your lab exercises.

3 From the Monitor menu, select Capacity Utilization to view the storage pool capacity.
Click the History button to view the capacity history. You can hover your mouse over
points in the graph to view metrics at a specific time. Metrics are updated every hour.

EMC Education Services 144

 Step Action

4 Navigate to Monitor > Traffic Metrics to view the traffic metrics for the VDC. Click History
for a graphical representation. Clicking the VDC will show metrics on a per node basis.

What is the highest Avg. Latency Write for node 3? ______________________________

EMC Education Services 145

 Step Action

5 Select your pod number. This will bring up further traffic metrics data for each ECS node
in your cluster. Click History for a graphical representation. This will display resource
usage history.

EMC Education Services 146

 Step Action

6 Click Hardware Health and then choose the storage pool pod#site1sp1 (where # is your
POD #). This will show node and disk health. You can click your storage pool to view
further details per node.

EMC Education Services 147

 Step Action

7 Click Node & Process Health. Click the VDC named pod#site1vdc1 (where # is your pod
#). Here you can monitor the current resources usage for that VDC on a per node basis.
Clicking the History button displays a graphical representation of resource usage history.

EMC Education Services 148

 Step Action

8 Click Chunk Summary. Click the drop-down arrow to view further details.

9 Click Erasure Coding to view erasure coding activity.

EMC Education Services 149

 Step Action

10 Click Recovery Status. The progress of recovery of a storage pool can be tracked here.

11 Click Disk Bandwidth to view disk performance for the VDCs listed.

What is the peak read speed of your ECS for pod#site1vdc1 (where # is your pod #)?
_____________________________

EMC Education Services 150

 Step Action

12 Click Geo Replication. There are several buttons available to view further details on the
geo-configuration. Click through these buttons to view those attributes. If your ECS is
not configured for Geo Replication the fields will be blank.

EMC Education Services 151

 Step Action

13 (OPTIONAL STEP): this step is optional and may be performed or simply reviewed.

It is possible to retrieve monitoring data using the REST API. You will need to run some
curl commands. You can run curl by opening an SSH session (with credentials
root/P@ssw0rd) to any of your ECS nodes, using PuTTY in your virtual desktop.

The following commands use the REST API to pull monitoring data from ECS:

Note: Substitute values in <> as described. The # in <pod#ns1> should be substituted

with your POD #.

#Set variables for ECS login

export MANAGEMENT_ENDPOINT=<your-ecs-ip>:4443
export MANAGEMENT_USER=root
export MANAGEMENT_PASSWORD=P@ssw0rd

#Get authentication token

curl -I -s --location-trusted -k https://$MANAGEMENT_ENDPOINT/login -u
"$MANAGEMENT_USER:$MANAGEMENT_PASSWORD"

#Set variable for management token

export MANAGEMENT_TOKEN=<token-returned-by-last-command>

#Stores events information in /tmp/events.xml

curl -ku $MANAGEMENT_USER:$MANAGEMENT_PASSWORD
https://$MANAGEMENT_ENDPOINT/vdc/events -k -H "X-SDS-AUTH-
TOKEN:$MANAGEMENT_TOKEN" -H "Accept: application/xml" | xmllint –-format
-

#Stores node information in /tmp/nodes.json

curl -ku $MANAGEMENT_USER:$MANAGEMENT_PASSWORD
https://$MANAGEMENT_ENDPOINT/dashboard/zones/localzone/nodes -k -H "X-
SDS-AUTH-TOKEN:$MANAGEMENT_TOKEN" -H "Accept: application/json" | python
–m json.tool > /tmp/nodes.json

#View ECS capacity

curl -ku $MANAGEMENT_USER:$MANAGEMENT_PASSWORD
https://$MANAGEMENT_ENDPOINT/object/capacity -k -H "X-SDS-AUTH-
TOKEN:$MANAGEMENT_TOKEN" -H "Accept: application/xml" | xmllint --format
–

NOTE: The python -m json.tool creates pretty output for the json information. The
xmllint --format - creates pretty output for the xml information

End of Lab Exercise

EMC Education Services 152

 Appendix – A – Rack Color and Node Names

EMC Education Services 153

EMC Education Services 154
 END OF LAB

EMC Education Services 155

EMC Education Services 156