Chapter 12.

S3 API

12.1. Introduction

12.1.1. HyperStore Support for the AWS S3 API

The Cloudian HyperStore system supports the great majority of the Amazon Web Services S3 REST API,
including advanced features.

This documentation provides the details of the HyperStore system’s compliance with the S3 REST API. The
organization of this documentation parallels that of the AWS S3 API Reference. Links are provided to specific
parts of the AWS S3 API Reference so you can easily view additional information about individual API oper-
ations.

This documentation takes the approach of specifying in detail the things that the HyperStore system does sup-
port from the AWS S3 REST API — from operations down to the level of particular request parameters, request
headers, request elements, response headers, and response elements. If it’s not listed in this HyperStore S3
API Support documentation, the HyperStore system does not currently support it.

This documentation also describes ways in which the HyperStore system extends the AWS S3 API, to support
additional functionality. Most of these extensions are in the form of additional request headers that add
enhanced functionality to standard AWS S3 operations on buckets. These extensions are described within the
sections that document HyperStore compliance with standard AWS S3 operations. The extensions are always
identified by a sub-heading that says HyperStore Extension to the S3 API. (For a summary of the extensions
see "HyperStore Extensions to the S3 API" (page 1008).)

12.1.1.1. Caution About Mass Deletes

Do not attempt to delete more than 100,000 objects from a single bucket in less than an hour (using the S3 API
method DELETE Multiple Objects). Doing so will result in TombstoneOverwhelmingException errors in the
Cassandra logs and an inability to successfully execute an S3 GET Bucket (List Objects) Version 1 or GET
Bucket (List Objects) Version 2 operation on the bucket. If the system is in this error condition, you can trigger
a tombstone purge as described in "Dealing with Excessive Tombstone Build-Up" (page 537).

12.1.1.2. Using TLS/SSL

For information about setting up HTTPS for the S3 Service see "HTTPS" (page 145) . If HTTPS is enabled for
your S3 Service it will listen for HTTPS connections on port 443, as well as listening for regular HTTP con-
nections on port 80.

12.1.2. S3 Client Application Options

Broadly you have three options for using HyperStore's implementation of the AWS S3 API to create storage
buckets in HyperStore, upload objects, retrieve objects, and so on:

l "Using the CMC as Your S3 Client" (page 1002)

l "Using Third Party S3 Applications" (page 1003)
l "Developing Custom S3 Applications for HyperStore" (page 1003)

1001
Chapter 12. S3 API

IMPORTANT ! Some atypical ways of organizing data within a bucket can result in sub-optimal per-
formance for certain S3 operations on that bucket. For detail see "Object Metadata Structure in the
Metadata DB" (page 201).

Note When the CMC or other S3 client applications delete S3 objects, the HyperStore system deletes
the object metadata immediately but does not delete the actual objects immediately. Instead the
objects are batched for deletion by a cron job. When S3 clients overwrite S3 objects, the HyperStore
system writes the new version of the object immediately, and updates the object metadata immediately,
but does not delete the outdated version of the object immediately. Instead the outdated object ver-
sions are batched for deletion by the same cron job. (Note that in a bucket that has versioning
enabled, the old object versions would be retained rather than deleted.)

12.1.2.1. Using the CMC as Your S3 Client

The CMC’s Buckets & Objects section serves as a graphical S3 client for interacting with the HyperStore
object store. With the CMC, users can do the following:

l "Add a Bucket" (page 253)

l Set bucket properties
o "Set Custom S3 Permissions for a Bucket" (page 257)
o "Set "Canned" S3 Permissions for a Bucket" (page 260)
o "View a Bucket's Storage Policy Information" (page 262)
o "Configure a Bucket Lifecycle Policy for Object Auto-Tiering or Expiration" (page 262)
o "Configure a Bucket as a Static Website" (page 272)
o "Configure Cross-Region Replication for a Bucket" (page 275)
o "Set Versioning for a Bucket" (page 277)
o "Set Logging for a Bucket" (page 278)
l "Delete a Bucket" (page 282)

l "Create or Delete a "Folder"" (page 284)

l "Upload an Object" (page 284)
l Set file properties
o "Set Custom S3 Permissions on an Object" (page 288)
o "Set "Canned" S3 Permissions on an Object" (page 290)
o "Set Public URL Permissions on an Object" (page 292)
l "List or Search for Objects" (page 296)
l "Download an Object" (page 297)
l "Delete an Object" (page 300)
l "Restore an Auto-Tiered Object" (page 297)

Note The CMC system administrator role does not and cannot have its own S3 storage user account.
However you can create a regular user account for yourself, and use that to access the data store.

1002
 12.1. Introduction

You can also manage other regular users' data on their behalf, if that capability is enabled in your sys-
tem by configuration.

12.1.2.2. Using Third Party S3 Applications

Because of HyperStore’s comprehensive compliance with the AWS S3 API, you can use most off-the-shelf third
party S3 client applications with HyperStore. For feedback on particular S3 applications that you are con-
sidering using with HyperStore, consult with Cloudian Sales Engineering or Cloudian Support.

To check to see what is your HyperStore S3 Service endpoint -- the URI to which you will submit S3 requests
with your third party application -- go to the CMC's Security Credentials page or Cluster Information page.

12.1.2.3. Developing Custom S3 Applications for HyperStore

In nearly every way, developing a client application for the Cloudian HyperStore storage service is the same as
developing a client application for AWS S3. Consequently, when building S3 applications for the HyperStore
service you can leverage the wealth of resources available to AWS S3 developers.

Good online resources for S3 application developers include:

l Amazon Simple Storage Service Developer Guide

l Amazon S3 resources

12.1.2.3.1. What’s Distinct About Developing for the HyperStore S3 Service

In practice, the main differences between developing for the HyperStore S3 service and developing for
Amazon S3 are:

l HyperStore S3 client applications must use the HyperStore S3 service endpoint rather than the Amazon
S3 service endpoint. To check to see what is your HyperStore S3 Service endpoint -- the URI to which
you will submit S3 requests with your custom application -- go to the CMC's Security Credentials page
or Cluster Information page.
l As detailed in the "Supported S3 Operations" section of this documentation, the HyperStore S3 service
supports the great majority of but not the entire Amazon S3 API.
l Also as detailed in the "Supported S3 Operations" section of this documentation, the HyperStore S3 ser-
vice supports a small number of extensions to the Amazon S3 API. (For an overview of the extensions
see "HyperStore Extensions to the S3 API" (page 1008)).

12.1.3. Authenticating Requests (AWS Signature Version 4)

HyperStore supports AWS Signature Version 4 for authenticating inbound API requests. The HyperStore imple-
mentation of this feature is compliant with Amazon’s specification of the feature. For example, you can express
authentication information in the HTTP Authorization header or in query string parameters; and you can com-
pute a checksum of the entire payload prior to transmission, or for large uploads, you can use chunked upload.

For more information on this Amazon S3 feature, refer to the "Authenticating Requests (AWS Signature Ver-
sion 4)" section of the Amazon S3 REST API.

HyperStore continues to support AWS Signature Version 2 as well.

1003
Chapter 12. S3 API

Note For HyperStore, the region name validation aspect of Signature Version 4 is disabled by default.
You can enable it with the "cloudian.s3.authorizationV4.singleregioncheck" (page 614) and/or
"cloudian.s3.authorizationV4.multiregioncheck" (page 614) settings in mts.properties.erb. Even if
you do enable region name validation, the HyperStore S3 Service employs a fall-back device where if
the region name specified in the request’s authorization header does not match against the local
region name, the system checks whether the specified region name matches against the S3 service
domain. If both checks fail then the request is rejected. This is to accommodate legacy HyperStore sys-
tems where the S3 service endpoint may not necessarily include the region name.

12.1.4. Access Control List (ACL) Support

For the AWS S3 "Access Control List (ACL)" functionality, the HyperStore system supports the items listed
below. If a grantee group, permission type, or canned ACL type from the AWS S3 documentation is not listed
below, the HyperStore system does not support it.

For ACL usage information and for descriptions of ACL items, see Access Control List (ACL) Overview in the
AWS S3 documentation.

12.1.4.1. AWS S3 Predefined Groups

l Authenticated users group
l All users group
l Log delivery group

12.1.4.2. Permission Types

l READ
l WRITE
l READ_ACP
l WRITE_ACP
l FULL_CONTROL

12.1.4.3. Canned ACL

l private
l public-read
l public-read-write
l authenticated-read
l bucket-owner-read
l bucket-owner-full-control
l log-delivery-write

HyperStore Extension to the S3 API

The HyperStore system supports these additional canned ACLs:

1004
 12.1. Introduction

Canned ACL Applies to Permissions added to ACL

group-read Bucket and object Owner gets FULL_CONTROL. All other members of the
owner’s HyperStore service user group get READ access.

group-read-write Bucket and object Owner gets FULL_CONTROL. All other members of the
owner’s HyperStore service user group get READ and WRITE
access.

Note To grant access to groups other than the requester’s own group, you cannot use canned ACLs.
Instead, when using standard Amazon S3 methods for assigning privileges to a grantee (via request
headers or request body), specify "<groupID>|" as the grantee. The "<groupID>|" format (with vertical
bar) indicates that the grantee is a group — for example, "Group5|".

Note When access privileges have through separate requests been granted to a group and to a spe-
cific member of the group, the user gets the broader of the privilege grants. For example, if Group5 is
granted read-write privileges and a specific user within Group5 is separately granted read privileges,
the user gets read-write privileges.

12.1.5. S3 Common Request and Response Headers

12.1.5.1. Common Request Headers

From the "Common Request Headers" section of the AWS S3 REST API specification, HyperStore supports
the headers listed below. If a header from that specification section is not listed below, HyperStore does not
support it.

l Authorization
l Content-Length
l Content-Type
l Content-MD5
l Date
l Expect
l Host
l x-amz-content-sha256
l x-amz-date

l x-amz-expected-bucket-owner

Note
* Unlike the AWS documentation which lists x-amz-expected-bucket-owner as a supported
request header for nearly every individual S3 API call but omits the header from the Common
header list, this HyperStore documentation instead lists the x-amz-expected-bucket-owner
header here among the Common headers. For the HyperStore S3 Service, the x-amz-expected-
bucket-owner request header is supported for all S3 API calls except CreateBucket and
ListBuckets.
* If you use the optional x-amz-expected-bucket-owner request header in making S3 calls to the

1005
Chapter 12. S3 API

HyperStore S3 Service, identify the expected bucket owner by the bucket owner's canonical
user ID. A user's canonical user ID can be obtained by retrieving the user's profile in the CMC
or via the Admin API call GET /user.
* As with AWS, if the destination bucket in an API request is owned by an account other than the
expected bucket owner account, the request will fail with an HTTP 403 (Access Denied) error.

12.1.5.2. Common Response Headers

From the "Common Response Headers" section of the AWS S3 REST API specification, HyperStore sup-
ports the headers listed below. If a header from that specification section is not listed below, HyperStore does
not support it.

l Content-Length
l Content-Type
l Connection
l Date
l ETag
l Server
l x-amz-delete-marker
l x-amz-request-id
l x-amz-version-id

12.1.6. S3 Error Responses

From the "Error Responses" section of the AWS S3 API specification, HyperStore supports the error codes lis-
ted below, in the same format as indicated in the specification. If an error code from that specification section is
not listed below, HyperStore does not support it.

l AccessDenied
l AccountProblem
l AmbiguousGrantByEmailAddress
l BadDigest
l BucketAlreadyExists
l BucketAlreadyOwnedByYou
l BucketNotEmpty
l CrossLocationLoggingProhibited
l EntityTooLarge
l EntityTooSmall
l IllegalVersioningConfigurationException
l IncorrectNumberOfFilesInPostRequest
l InternalError
l InvalidAccessKeyId
l InvalidArgument

1006
 12.1. Introduction

l InvalidBucketName
l InvalidBucketState
l InvalidDigest
l InvalidEncryptionAlgorithmError
l InvalidLocationConstraint
l InvalidObjectState
l InvalidPart
l InvalidPartOrder
l InvalidPolicyDocument
l InvalidRange
l InvalidRequest
l InvalidSecurity
l InvalidTargetBucketForLogging
l InvalidURI
l KeyTooLong
l MalformedACLError
l MalformedPOSTRequest
l MalformedXML
l MaxMessageLengthExceeded
l MaxPostPreDataLengthExceededError
l MetadataTooLarge
l MethodNotAllowed
l MissingContentLength
l MissingSecurityHeader
l NoSuchBucket
l NoSuchBucketPolicy
l NoSuchKey
l NoSuchLifecycleConfiguration
l NoSuchReplicationConfiguration
l NoSuchUpload
l NoSuchVersion
l NotImplemented
l PermanentRedirect
l PreconditionFailed
l Redirect
l RestoreAlreadyInProgress
l RequestIsNotMultiPartContent
l RequestTimeout
l RequestTimeTooSkewed

1007
Chapter 12. S3 API

l SignatureDoesNotMatch
l ServiceUnavailable
l SlowDown
l TemporaryRedirect
l TooManyBuckets
l UnexpectedContent
l UnresolvableGrantByEmailAddress
l UserKeyMustBeSpecified

12.1.7. HyperStore Extensions to the S3 API

The HyperStore S3 Service supports the following extensions to the AWS S3 REST API. In each case the exten-
sions take the form of additional supported headers for standard AWS S3 API methods.

Extension Purpose Detail

l "CreateBucket" (page
Specify the Hyper-
x-gmt-policyid as optional request header for 1011)
Store storage policy
"CreateBucket" and response header for "ListOb- l "Storage Policies
to use for a new
jects", "ListObjectsV2", and "HeadBucket" Feature Overview"
bucket
(page 104)

x-gmt-tieringinfo and x-gmt-compare and x-gmt- l "PutBucketLifecycle"

post-tier-copy as optional request headers for Set up auto-tiering (page 1046)
"PutBucketLifecycle" and response headers for for a bucket l "Auto-Tiering Feature
"GetBucketLifecycle" Overview" (page 206)

x-gmt-error-code and x-gmt-message as supported Provide additional l "GetObject" (page 1026)

response headers for "GetObject" and "HeadOb- information about l "HeadObject" (page
ject" HTTP 4xx errors 1031)

x-gmt-crr-endpoint and x-gmt-crr-credentials as l "PutBucketReplication"

optional request headers for "PutBuck- Use for cross-sys- (page 1059)
etReplication" and response headers for "GetBuck- tem replication l "Cross-System Rep-
etReplication". lication" (page 220)

12.2. Supported S3 Operations

12.2.1. AbortMultipartUpload
This operation aborts a multipart upload.

Along with the common headers, HyperStore supports the operation-specific parameters listed below.

For operation details and examples see the AWS documentation: AbortMultipartUpload

Former operation name: Abort Multipart Upload

1008
 12.2. Supported S3 Operations

12.2.1.1. Query Parameters

l uploadId

12.2.2. CompleteMultipartUpload
Completes a multipart upload by assembling previously uploaded parts.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: CompleteMultipartUpload

Former operation name: Complete Multipart Upload

12.2.2.1. Request Elements

l CompleteMultipartUpload
o Part
n ETag
n PartNumber

12.2.2.2. Response Headers

l x-amz-expiration
l x-amz-server-side-encryption
l x-amz-version-id

12.2.2.3. Response Elements

l Bucket
l CompleteMultipartUploadResult
l ETag
l Key
l Location

12.2.3. CopyObject
Creates a copy of an object that is already stored in HyperStore.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: CopyObject

Former operation name: PUT Object - Copy

1009
Chapter 12. S3 API

12.2.3.1. Request Headers

l x-amz-acl
l x-amz-copy-source
l x-amz-copy-source-if-match
l x-amz-copy-source-if-modified-since
l x-amz-copy-source-if-none-match
l x-amz-copy-source-if-unmodified-since
l x-amz-copy-source-server-side-encryption-customer-algorithm
l x-amz-copy-source-server-side-encryption-customer-key
l x-amz-copy-source-server-side-encryption-customer-key-MD5
l x-amz-grant-full-control
l x-amz-grant-read
l x-amz-grant-read-acp
l x-amz-grant-write
l x-amz-grant-write-acp
l x-amz-metadata-directive
l x-amz-object-lock-legal-hold
l x-amz-object-lock-mode

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see
"Object Lock" (page 128).

l x-amz-object-lock-retain-until-date
l x-amz-server-side-encryption

Note For information about HyperStore's support of the x-amz-server-side-encryption and x-

amz-server-side-encryption-customer-* request headers, and set-up steps that you must per-
form in order to use HyperStore's server-side encryption features, see "Server-Side Encryp-
tion" (page 137).

l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key
l x-amz-server-side-encryption-customer-key-MD5
l x-amz-storage-class

Note HyperStore ignores the value of the x-amz-storage-class header and treats all requests as
being for storage class STANDARD.

l x-amz-source-expected-bucket-owner
l x-amz-tagging

1010
 12.2. Supported S3 Operations

l x-amz-tagging-directive
l x-amz-website-redirect-location

12.2.3.2. Response Headers

l x-amz-copy-source-version-id
l x-amz-expiration
l x-amz-server-side-encryption
l x-amz-version-id

12.2.3.3. Response Elements

l CopyObjectResult
o ETag
o LastModified

12.2.4. CreateBucket
Creates a new bucket.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: CreateBucket

Former operation name: PUT Bucket

IMPORTANT ! Some atypical ways of organizing data within a bucket can result in sub-optimal per-
formance for certain S3 operations on that bucket. For detail see "Object Metadata Structure in the
Metadata DB" (page 201).

Note By default each user is allowed a maximum of 100 buckets. You can change this setting in the
CMC's Configuration Settings page.

12.2.4.1. Request Headers

l x-amz-acl
l x-amz-grant-full-control
l x-amz-grant-read
l x-amz-grant-read-acp
l x-amz-grant-write
l x-amz-grant-write-acp
l x-amz-object-lock-enabled

1011
Chapter 12. S3 API

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see
"Object Lock" (page 128).

HyperStore Extension to the S3 API

The HyperStore system supports the following Request Header as an extension to the "PUT Bucket" operation:

Name Description Required

x-gmt-policyid This header specifies the unique ID of the storage policy to assign to the No
newly created bucket. The storage policy determines how data in the
bucket will be distributed and protected through either replication or eras-
ure coding. System administrators can create multiple storage policies
through the CMC and the system automatically assigns each a unique
policy ID that becomes part of the policy definition. (To obtain a list of stor-
age policies for your system and their policy IDs, you can use the Admin
API’s GET /bppolicy/listpolicy method).

With the "x-gmt-policyid" request header for "PUT Bucket", you specify the
ID of the desired storage policy when you create a new bucket. Note how-
ever that some policies may not be available to all user groups — a policy’s
availability is specified by system administrators at the time of policy cre-
ation, and this information becomes part of the policy definition. When you
specify an "x-gmt-policyid" value with a "PUT Bucket" request, the policy ID
must be for a policy that is available to the group to which the bucket owner
belongs.

Also the policy ID must be for a storage policy from the service region that is
specified in the "PUT Bucket" request’s LocationConstraint element.

If the "PUT Bucket" request does not include the "x-gmt-policyid" request
header, then the system will automatically assign the system default stor-
age policy to the bucket during bucket creation.

Note After a bucket is created, it cannot be assigned a different stor-

age policy. The storage policy assigned to the bucket at bucket cre-
ation time will continue to be bucket’s storage policy for the life of
the bucket.

Note A 403 error response is returned if you specify a policy ID that

does not exist, has been disabled, is not available to the region in
which the bucket is being created, or is not available to the group to
which the bucket owner belongs. A 403 is also returned if you do
not specify an "x-gmt-policyid" header and the system does not yet
have an established default storage policy.

Example header:

x-gmt-policyid: 1bc90238f9f11cb32f5e4e901675d50b

1012
 12.2. Supported S3 Operations

Name Description Required

For more information on storage policies, see "Storage Policies Feature
Overview" (page 104).

12.2.4.2. Request Elements

l CreateBucketConfiguration
o LocationConstraint

Note The HyperStore system enforces the same bucket naming restrictions as does Amazon S3.
Also, if you use an underscore in a bucket name you will not be able to enable auto-tiering for the
bucket (for transitioning objects to Amazon or other remote destinations on a configurable schedule).
It's best not to use underscores when naming new buckets, in case you may want to enable auto-tiering
on the bucket immediately or in the future.

12.2.5. CreateMultipartUpload
This operation initiates a multipart upload and returns an upload ID.

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: CreateMultipartUpload

Former operation name: Initiate Multipart Upload

12.2.5.1. Request Headers

l Cache-Control
l Content-Disposition
l Content-Encoding
l Content-Type
l Expires
l x-amz-acl
l x-amz-grant-full-control
l x-amz-grant-read
l x-amz-grant-read-acp
l x-amz-grant-write
l x-amz-grant-write-acp
l x-amz-meta-
l x-amz-object-lock-legal-hold
l x-amz-object-lock-mode

1013
Chapter 12. S3 API

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see
"Object Lock" (page 128).

l x-amz-object-lock-retain-until-date
l x-amz-server-side-encryption

Note For information about HyperStore's support of the x-amz-server-side-encryption and x-

amz-server-side-encryption-customer-* request headers, and set-up steps that you must per-
form in order to use HyperStore's server-side encryption features, see "Server-Side Encryp-
tion" (page 137).

l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key
l x-amz-server-side-encryption-customer-key-MD5
l x-amz-storage-class

Note HyperStore ignores the value of the x-amz-storage-class header and treats all requests as
being for storage class STANDARD.

l x-amz-website-redirect-location

12.2.5.2. Response Headers

l x-amz-abort-date
l x-amz-abort​-rule-id
l x-amz-server-side-encryption
l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key-MD5

12.2.5.3. Response Elements

l InitiateMultipartUploadResult
o Bucket
o Key
o UploadId

12.2.6. DeleteBucket
Deletes the bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucket

Former operation name: DELETE Bucket

1014
 12.2. Supported S3 Operations

12.2.7. DeleteBucketCors
Deletes the cors configuration information set for the bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucketCors

Former operation name: DELETE Bucket cors

12.2.8. DeleteBucketEncryption
This implementation of the DELETE operation removes default encryption from the bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucketEncryption

Former operation name: DELETE Bucket encryption

12.2.9. DeleteBucketInventoryConfiguration
Deletes an inventory configuration (identified by the inventory ID) from the bucket.

Along with the common headers, HyperStore supports the operation-specific parameter listed below.

For operation details and examples see the AWS documentation: DeleteBucketInventoryConfiguration

12.2.9.1. Query Parameter

l id

12.2.10. DeleteBucketLifecycle
Deletes the lifecycle configuration from the specified bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucketLifecycle

Former operation name: DELETE Bucket lifecycle

12.2.11. DeleteBucketOwnershipControls
Removes OwnershipControls for a bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucketOwnershipControls

12.2.12. DeleteBucketPolicy
This implementation of the DELETE operation uses the policy subresource to delete the policy of a specified
bucket.

For this operation HyperStore supports the S3 common headers.

1015
Chapter 12. S3 API

For operation details and examples see the AWS documentation: DeleteBucketPolicy

Former operation name: DELETE Bucket policy

12.2.13. DeleteBucketReplication
Deletes the replication configuration from the bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucketReplication

Former operation name: DELETE Bucket replication

12.2.14. DeleteBucketTagging
Deletes the tags from the bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucketTagging

Former operation name: DELETE Bucket tagging

12.2.15. DeleteBucketWebsite
This operation removes the website configuration for a bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteBucketWebsite

Former operation name: DELETE Bucket website

12.2.16. DeleteObject
Removes the null version (if there is one) of an object and inserts a delete marker, which becomes the latest
version of the object.

Along with the common headers, HyperStore supports the operation-specific headers listed below.

For operation details and examples see the AWS documentation: DeleteObject

Former operation name: DELETE Object

Note Successful completion of a DeleteObject request results in the system marking the object as hav-
ing been deleted. However the actual deletion of object data from disk will not occur until the next auto-
matic running of the object deletion batch processing job. By default this batch processing of object
data deletes runs hourly on each node. The frequency with which the batch processing job runs is con-
figurable by the "cloudian.delete.queue.poll.interval" (page 620) property in mts.properties.erb.

IMPORTANT ! Do not attempt to delete more than 100,000 objects from a single bucket in less than an
hour. Doing so will result in TombstoneOverwhelmingException errors in the Cassandra logs and an
inability to successfully execute a "ListObjects" (page 1036) operation on the bucket. If the system is

1016
 12.2. Supported S3 Operations

in this error condition, you can trigger a tombstone purge as described in "Dealing with Excessive
Tombstone Build-Up" (page 537).

12.2.16.1. Request Headers

l x-amz-bypass-governance-retention

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see
"Object Lock" (page 128).

12.2.16.2. Response Headers

l x-amz-delete-marker
l x-amz-version-id

12.2.17. DeleteObjects
This operation enables you to delete multiple objects from a bucket using a single HTTP request.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: DeleteObjects

Former operation name: Delete Multiple Objects

Note The HyperStore S3 Service allows a maximum of 1000 object deletes per DeleteObjects request.

Note Successful completion of a DeleteObjects request results in the system marking the objects as
having been deleted. However the actual deletion of object data from disk will not occur until the next
automatic running of the object deletion batch processing job. By default this batch processing of object
data deletes runs hourly on each node. The frequency with which the batch processing job runs is con-
figurable by the "cloudian.delete.queue.poll.interval" (page 620) property in mts.properties.erb.

IMPORTANT ! Do not attempt to delete more than 100,000 objects from a single bucket in less than an
hour. Doing so will result in TombstoneOverwhelmingException errors in the Cassandra logs and an
inability to successfully execute an S3 "ListObjects" (page 1036) or "ListObjectsV2" (page 1037)
operation on the bucket. If the system is in this error condition, you can trigger a tombstone purge as
described in "Dealing with Excessive Tombstone Build-Up" (page 537).

12.2.17.1. Request Headers

l x-amz-bypass-governance-retention

1017
Chapter 12. S3 API

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see
"Object Lock" (page 128).

12.2.17.2. Request Elements

l Delete
o Object
n Key
n VersionId
o Quiet

12.2.17.3. Response Elements

l DeleteResult
o Deleted
n DeleteMarker
n DeleteMarkerVersionId
n Key
n VersionId
o Error
n Code
n Key
n Message
n VersionId

12.2.18. DeleteObjectTagging
Removes the entire tag set from the specified object.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeleteObjectTagging

Former operation name: DELETE Object tagging

12.2.19. DeletePublicAccessBlock
Removes the PublicAccessBlock configuration for a bucket.

For this operation HyperStore supports the S3 common headers.

For operation details and examples see the AWS documentation: DeletePublicAccessBlock

1018
 12.2. Supported S3 Operations

12.2.20. GetBucketAcl
This implementation of the GET operation uses the acl subresource to return the access control list (ACL) of a
bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketAcl

Former operation name: GET Bucket acl

12.2.20.1. Response Elements

l AccessControlPolicy
o Owner
n DisplayName
n ID
o AccessControlList
n Grant
o Grantee
n DisplayName
n ID
n Permission

12.2.21. GetBucketCors
Returns the cors configuration information set for the bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketCors

Former operation name: GET Bucket cors

12.2.21.1. Response Elements

l CORSConfiguration
o CORSRule
n AllowedHeader
n AllowedMethod
n AllowedOrigin
n ExposeHeader
n ID
n MaxAgeSeconds

12.2.22. GetBucketEncryption
Returns the default encryption configuration for the bucket.

1019
Chapter 12. S3 API

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketEncryption

Former operation name: GET Bucket encryption

12.2.22.1. Response Elements

l ServerSideEncryptionConfiguration
o Rule
n ApplyServerSideEncryptionByDefault
o SSEAlgorithm

12.2.23. GetBucketInventoryConfiguration
Returns an inventory configuration (identified by the inventory configuration ID) from the bucket.

Along with the common headers, HyperStore supports the operation-specific parameter and elements listed
below.

For operation details and examples see the AWS documentation: GetBucketInventoryConfiguration

12.2.23.1. Query Parameter

l id

12.2.23.2. Response Elements

l InventoryConfiguration
o Destination
n S3BucketDestination
o AccountId
o Bucket
o Encryption
n SSE-KMS
o KeyId
n SSE-S3
o Format
o Prefix
o Filter
n Prefix
o Id
o IncludedObjectVersions
o IsEnabled
o OptionalFields
n Field

1020
 12.2. Supported S3 Operations

o Schedule
n Frequency

12.2.24. GetBucketLifecycle
Returns the lifecycle configuration information set on the bucket.

For operation details and examples see the AWS documentation: GetBucketLifecycle

Note Though HyperStore supports this API operation for backward compatibility, AWS has deprecated
this operation in favor of a newer version called GetBucketLifecycleConfiguration which HyperStore
also supports. If you used PutBucketLifecycleConfiguration to create a lifecycle use GetBuck-
etLifecycleConfiguration to retrieve the configuration.

12.2.25. GetBucketLifecycleConfiguration
Returns the lifecycle configuration information set on the bucket.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: GetBucketLifecycleConfiguration

Former operation name: GET Bucket lifecycle

12.2.25.1. Response Headers

HyperStore Extension to the S3 API

The HyperStore system supports the following Response Headers as extensions to the "GetBuck-
etLifecycleConfiguration" operation:

Name Description Required

x-gmt-tieringinfo

x-gmt-compare See "PutBucketLifecycleConfiguration" (page 1046). No

x-gmt-post-tier-copy

12.2.25.2. Response Elements

l LifecycleConfiguration
o Rule
n AbortIncompleteMultipartUpload
o DaysAfterInitiation
n Expiration
o Date
o Days

1021
Chapter 12. S3 API

o ExpiredObjectDeleteMarker
n Filter
o And
n Prefix
n Tag
o Key
o Value
o Prefix
o Tag
n Key
n Value
n ID
n NoncurrentVersionExpiration
o NoncurrentDays
n NoncurrentVersionTransition
o NoncurrentDays
o StorageClass
n Prefix
n Status
n Transition
o Date
o Days
o StorageClass

12.2.26. GetBucketLocation
Returns the Region the bucket resides in.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketLocation

Former operation name: GET Bucket location

12.2.26.1. Response Elements

l LocationConstraint

12.2.26.2. GetBucketLocation Response for Buckets in the Default Service Region

The GetBucketLocation operation behaves as follows:

l If the bucket specified in the GetBucketLocation request resides in a non-default service region, the
response indicates the name of the service region.

1022
 12.2. Supported S3 Operations

l If the bucket specified in the GetBucketLocation request resides in the default service region, the
response returns a null/empty value.

HyperStore's behavior of returning a null/empty value if the bucket is in the default region is the same as
Amazon Web Services' implementation of the GetBucketLocation operation. Some S3 client applications --
such as Veeam -- are unable to handle the return of a null/empty region value, and may display an error if the
actual default region name is set within the client application. The work-around is to not set the region in the cli-
ent application, or else set it to the AWS default region name: us-east-1.

12.2.27. GetBucketLogging
Returns the logging status of a bucket and the permissions users have to view and modify that status.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketLogging

Former operation name: GET Bucket logging

12.2.27.1. Response Elements

l BucketLoggingStatus
o LoggingEnabled
n TargetBucket
n TargetGrants
o Grant
n Grantee
n Permission
n TargetPrefix

12.2.28. GetBucketNotificationConfiguration
Returns the notification configuration of a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketNotificationConfiguration

12.2.28.1. Response Elements

l NotificationConfiguration
o QueueConfiguration
o Event
o Filter
n S3Key
o FilterRule
n Name
n Value

1023
Chapter 12. S3 API

o Id
o Queue

For Event types, HyperStore supports only the following:

l s3:ObjectCreated:*
l s3:ObjectCreated:Put
l s3:ObjectCreated:Post
l s3:ObjectCreated:Copy
l s3:ObjectCreated:CompleteMultipartUpload
l s3:ObjectRemoved:*
l s3:ObjectRemoved:Delete
l s3:ObjectRemoved:DeleteMarkerCreated

12.2.29. GetBucketOwnershipControls
Retrieves OwnershipControls for a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketOwnershipControls

12.2.29.1. Response Elements

l OwnershipControls
o Rule
n ObjectOwnership

12.2.30. GetBucketPolicy
Returns the policy of a specified bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketPolicy

Former operation name: GET Bucket policy

12.2.30.1. Response Elements

The response contains the (JSON) policy of the specified bucket.

12.2.31. GetBucketPolicyStatus
Retrieves the policy status for a bucket, indicating whether the bucket is public.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketPolicyStatus

1024
 12.2. Supported S3 Operations

12.2.31.1. Response Elements

l PolicyStatus
o IsPublic

Note HyperStore considers a bucket policy to be "public" if any statement in the policy is public. A state-
ment is considered public if the Effect is Allow and the Principal has a wildcard -- unless there is an
IpAddress:{aws:SourceIp condition associated with the statement that restricts the requesting source IP
to one or more specified IP addresses.

12.2.32. GetBucketReplication
Returns the replication configuration of a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketReplication

Former operation name: GET Bucket replication

12.2.32.1. Response Elements

l ReplicationConfiguration
o Role
o Rule

12.2.33. GetBucketTagging
Returns the tag set associated with the bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketTagging

Former operation name: GET Bucket tagging

Note The HyperStore Admin API supports a method for retrieving all the bucket tags for all users in a
specified group. Because it is implemented through the Admin API, that method does not require the
users' S3 access credentials. For more information see GET /bucketops/gettags.

12.2.33.1. Response Elements

l Tagging
o TagSet
n Tag
o Key
o Value

1025
Chapter 12. S3 API

12.2.34. GetBucketVersioning
Returns the versioning state of a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketVersioning

Former operation name: GET Bucket versioning

12.2.34.1. Response Elements

l VersioningConfiguration
o Status

12.2.35. GetBucketWebsite
Returns the website configuration for a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetBucketWebsite

Former operation name: GET Bucket website

12.2.35.1. Response Elements

l WebsiteConfiguration
o IndexDocument
o ErrorDocument

12.2.36. GetObject
Retrieves objects from the S3 storage system.

Along with the common headers, HyperStore supports the operation-specific parameters, headers, and ele-
ments listed below.

For operation details and examples see the AWS documentation: GetObject

Former operation name: GET Object

12.2.36.1. Query Parameters

l partNumber

Note Using the partNumber parameter may not work as expected if the object has been auto-
tiered, or if the object has been auto-tiered and restored. This is because an object's number of
parts when uploaded to HyperStore may be different than its number of parts when it is auto-
tiered to a remote destination system.

1026
 12.2. Supported S3 Operations

l response-cache-control
l response-content-disposition
l response-content-encoding
l response-content-language
l response-content-type
l response-expires
l versionId

12.2.36.2. Request Headers

l If-Match
l If-Modified-Since
l If-None-Match
l If-Unmodified-Since
l Range
l x-amz-server-side-encryption-customer-algorithm

Note For information about HyperStore's support of the x-amz-server-side-encryption-customer-

* request headers, and set-up steps that you must perform in order to use HyperStore's server-
side encryption features, see "Server-Side Encryption" (page 137).

l x-amz-server-side-encryption-customer-key
l x-amz-server-side-encryption-customer-key-MD5

12.2.36.3. Response Headers

l Last-Modified
l x-amz-delete-marker
l x-amz-expiration
l x-amz-meta-*
l x-amz-object-lock-legal-hold
l x-amz-object-lock-mode
l x-amz-object-lock-retain-until-date
l x-amz-replication-status
l x-amz-restore
l x-amz-server-side-encryption
l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key-MD5
l x-amz-tagging-count
l x-amz-version-id
l x-amz-website-redirect-location

HyperStore Extension to the S3 API

1027
Chapter 12. S3 API

The HyperStore system supports the following Response Headers as extensions to the "GET Object" oper-
ation. These headers are returned only in the event of an HTTP 4xx response. They are not returned with
HTTP 2xx, 3xx, or 5xx responses.

Name Description
x-gmt-error-code In the event of an HTTP 4xx response, these two response headers provide
additional information about the nature of the error. The x-gmt-error-code
x-gmt-message header values will be from among the list in "S3 Error Responses" (page
1006).

12.2.37. GetObjectAcl
Returns the access control list (ACL) of an object.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: GetObjectAcl

Former operation name: GET Object acl

12.2.37.1. Response Elements

l AccessControlPolicy
o Owner
n DisplayName
n ID
o AccessControlList
n Grant
o Grantee
n DisplayName
n ID
o Permission

12.2.38. GetObjectLegalHold
Gets an object's current Legal Hold status.

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: GetObjectLegalHold

Former operation name: GET Object legal hold

12.2.38.1. Query Parameters

l versionId

1028
 12.2. Supported S3 Operations

12.2.38.2. Response Elements

l LegalHold
o Status

12.2.39. GetObjectLockConfiguration
Gets the Object Lock configuration for a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetObjectLockConfiguration

Former operation name: GET Bucket object lock configuration

12.2.39.1. Response Elements

l ObjectLockConfiguration
o ObjectLockEnabled
o Rule
n DefaultRetention
o Days
o Mode
o Years

12.2.40. GetObjectRetention
Retrieves an object's retention settings.

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: GetObjectRetention

Former operation name: GET Object retention

12.2.40.1. Query Parameters

l versionId

12.2.40.2. Response Elements

l Retention
o Mode
o RetainUntilDate

12.2.41. GetObjectTagging
Returns the tag-set of an object.

1029
Chapter 12. S3 API

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetObjectTagging

Former operation name: GET Object tagging

12.2.41.1. Response Elements

l Tagging
o TagSet
n Tag
o Key
o Value

12.2.42. GetObjectTorrent
Return torrent files from a bucket.

For operation details and examples see the AWS documentation: GetObjectTorrent

Former operation name: GET Object torrent

12.2.42.1. Implementation Notes

l HyperStore does not provide a BitTorrent "tracker". You must either provide your own tracker or use one
of the many publicly available trackers. You must edit the "cloudian.s3.torrent.tracker" (page 632)
property in mts.properties.erb to specify the URL of the tracker that you are using.
l HyperStore implements BitTorrent HTTP seeding for in accordance with the BEP19 specification
(http://www.bittorrent.org/beps/bep_0019.html). Therefore torrent files returned by HyperStore in
response to GetObjectTorrent requests will include a "url-list" key and the value of that key will be the
URL of the object in HyperStore.
l HyperStore objects that have been auto-tiered to a destination S3 system cannot be retrieved via BitTor-
rent, unless the objects are first restored to local HyperStore storage (via the S3 "RestoreObject"
(page 1067) method). Restored objects can be retrieved from HyperStore via BitTorrent.
l Like with Amazon S3, with HyperStore only publicly readable objects are eligible for BitTorrent retrieval.
And like with Amazon S3, the following types of objects are not retrievable via BitTorrent:
o Objects larger than 5GB
o Non-current versions of versioned objects
o Objects encrypted via SSE-C (SSE with Customer-managed key; by contrast, BitTorrent retrieval
is supported for objects encrypted with regular SSE)

12.2.43. GetPublicAccessBlock
Retrieves the PublicAccessBlock configuration for a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: GetPublicAccessBlock

1030
 12.2. Supported S3 Operations

12.2.43.1. Response Elements

l PublicAccessBlockConfiguration
o BlockPublicAcls
o IgnorePublicAcls
o BlockPublicPolicy
o RestrictPublicBuckets

12.2.44. HeadBucket
This operation is useful to determine if a bucket exists and you have permission to access it.

Along with the common headers, HyperStore supports the operation-specific headers listed below.

For operation details and examples see the AWS documentation: HeadBucket

Former operation name: HEAD Bucket

12.2.44.1. Response Headers

l x-amz-bucket-region

HyperStore Extension to the S3 API

The HyperStore system supports the following Response Header as an extension to the "HeadBucket" oper-
ation:

Parameter Description
This header specifies the unique ID of the storage policy assigned to the
x-gmt-policyid
bucket. For more information see "CreateBucket" (page 1011).

12.2.45. HeadObject
The HEAD operation retrieves metadata from an object without returning the object itself.

Along with the common headers, HyperStore supports the operation-specific parameters and headers listed
below.

For operation details and examples see the AWS documentation: HeadObject

Former operation name: HEAD Object

12.2.45.1. Query Parameters

l partNumber

Note Using the partNumber parameter may not work as expected if the object has been auto-
tiered, or if the object has been auto-tiered and restored. This is because an object's number of

1031
Chapter 12. S3 API

parts when uploaded to HyperStore may be different than its number of parts when it is auto-
tiered to a remote destination system.

l versionId

12.2.45.2. Request Headers

l If-Match
l If-Modified-Since
l If-None-Match
l If-Unmodified-Since
l Range
l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key
l x-amz-server-side-encryption-customer-key-MD5

12.2.45.3. Response Headers

l Last-Modified
l x-amz-expiration
l x-amz-meta-*
l x-amz-object-lock-legal-hold
l x-amz-object-lock-mode
l x-amz-object-lock-retain-until-date
l x-amz-replication-status
l x-amz-restore
l x-amz-server-side-encryption
l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key-MD5
l x-amz-tagging-count
l x-amz-version-id

HyperStore Extension to the S3 API

The HyperStore system supports the following Response Headers as extensions to the "HeadObject" oper-
ation. These headers are returned only in the event of an HTTP 4xx response. They are not returned with
HTTP 2xx, 3xx, or 5xx responses.

Name Description
x-gmt-error-code In the event of an HTTP 4xx response, these two response headers provide
additional information about the nature of the error. The x-gmt-error-code
x-gmt-message header values will be from among the list in "S3 Error Responses" (page
1006).

1032
 12.2. Supported S3 Operations

12.2.46. ListBucketInventoryConfigurations
Returns a list of inventory configurations for the bucket.

Along with the common headers, HyperStore supports the operation-specific parameter and elements listed
below.

For operation details and examples see the AWS documentation: ListBucketInventoryConfigurations

12.2.46.1. Query Parameter

l continuation-token

12.2.46.2. Response Elements

l ListInventoryConfigurationsResult
l ContinuationToken
l InventoryConfiguration
o Destination
n S3BucketDestination
o AccountId
o Bucket
o Encryption
n SSE-KMS
o KeyId
n SSE-S3
o Format
o Prefix
o Filter
n Prefix
o Id
o IncludedObjectVersions
o IsEnabled
o OptionalFields
n Field
o Schedule
n Frequency
l IsTruncated
l NextContinuationToken

12.2.47. ListBuckets
Returns a list of all buckets owned by the authenticated sender of the request.

Along with the common headers, HyperStore supports the operation-specific parameter listed below.

1033
Chapter 12. S3 API

For operation details and examples see the AWS documentation: ListBuckets

Former operation name: GET Service

HyperStore Extension to the S3 API

The HyperStore system supports the following Query Parameter as an extension to the "ListBuckets" operation:

Note Support for this extension is disabled by default. To enable support for this extension, in "mts.-
properties.erb" (page 608) set cloudian.s3.enablesharedbucket to true, then do a Puppet push and
then restart the S3 Service.

Name Description Required

shared If the shared parameter is included in the request, the ListBuckets operation No
returns a list of buckets that other users have shared with the requesting
user. This will be buckets that have been shared specifically with the
requesting user, plus buckets that have been shared with the group to
which the requesting user belongs, plus buckets that have been shared
with everyone.

Example:

GET /?shared HTTP/1.1.

Host: s3-region1.enterprise4.mobi-cloud.com.
Accept-Encoding: identity.
Date: Fri, 05 Apr 2019 15:34:01 GMT.
Content-Length: 0.
Authorization: AWS akey2:jTcwd1Ta+5sZftVHGtEEyweojdk=.
User-Agent: Boto/2.42.0 Python/2.7.5 Linux/3.10.0-693.el7.x86_64.

HTTP/1.1 200 OK.

Date: Fri, 05 Apr 2019 15:34:01 GMT.
x-amz-request-id: 1721b414-267b-1341-93e6-d4ae52ce5402.
Content-Type: application/xml;charset=UTF-8.
Content-Length: 432.
Server: CloudianS3.

<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult

xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Owner><ID>8ce1c49e532edc91b0a43e0c7e7d5975</ID>
<DisplayName>robot1</DisplayName></Owner>
<Buckets><Bucket><Name>sharedbucket1</Name>
<CreationDate>2019-04-05T15:30:03.897Z</CreationDate></Bucket>
<Bucket><Name>sharedbucket2</Name>
<CreationDate>2019-04-05T15:27:26.300Z</CreationDate>
</Bucket></Buckets></ListAllMyBucketsResult>

Note When the 'shared' parameter is used, the ListBuckets call

returns only buckets that have been shared with the requesting user
-- not buckets owned by the requesting user. So to retrieve all
buckets that a user has access to, an S3 client application must sub-

1034
 12.2. Supported S3 Operations

Name Description Required

mit two ListBuckets calls -- one without the 'shared' parameter (to
retrieve the user's own buckets) and one with the 'shared' para-
meter (to retrieve buckets that have been shared with the user).

Note When the 'shared' parameter is used, in the ListBuckets

response body the "Owner" is the requesting user, not the actual
owner(s) of the shared bucket(s).

12.2.48. ListMultipartUploads
This operation lists in-progress multipart uploads.

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: ListMultipartUploads

Former operation name: List Multipart Uploads

12.2.48.1. Query Parameters

l delimiter
l encoding-type
l key-marker
l max-uploads
l prefix
l upload-id-marker

12.2.48.2. Response Elements

l ListMultipartUploadsResult
o Bucket
o KeyMarker
o UploadIdMarker
o NextKeyMarker
o Prefix
o Delimiter
o NextUploadIdMarker
o MaxUploads
o IsTruncated

1035
Chapter 12. S3 API

o Upload
n Initiated
n Initiator
o DisplayName
o ID
n Key
n Owner
o DisplayName
o ID
n StorageClass
n UploadId
o CommonPrefixes
n Prefix
o EncodingType

12.2.49. ListObjects
Returns some or all of the objects in a bucket.

Along with the common headers, HyperStore supports the operation-specific parameters, headers, and ele-
ments listed below.

For operation details and examples see the AWS documentation: ListObjects

Former operation name: GET Bucket (List Objects) Version 1

Note HyperStore also supports the newer version of this API operation, ListObjectsV2.

Note When using ListObjects, use the marker request parameter to improve performance in listing the
content of buckets that contain many objects. For detail see the AWS documentation for this API oper-
ation.

12.2.49.1. Query Parameters

l delimiter

Note The HyperStore system does not support %c2%85(U+0085) as a delimiter value

l encoding-type
l marker
l max-keys
l prefix

1036
 12.2. Supported S3 Operations

Note The HyperStore S3 extension request parameter meta=true is no longer supported.

12.2.49.2. Response Headers

HyperStore Extension to the S3 API

The HyperStore system supports the following Response Header as an extension to the "ListObjects" oper-
ation:

Name Description Required

This header specifies the unique ID of the storage policy assigned to
x-gmt-policyid No
the bucket. For more information see "CreateBucket" (page 1011).

12.2.49.3. Response Elements

l ListBucketResult
o IsTruncated
o Marker
o NextMarker
o Contents
n ETag
n Key
n LastModified
n Owner
o DisplayName
o ID
n Size
n StorageClass (values STANDARD and GLACIER only)
o Name
o Prefix
o Delimiter
o MaxKeys
o CommonPrefixes
n Prefix
o Encoding-Type

12.2.50. ListObjectsV2
Returns some or all of the objects in a bucket.

Along with the common headers, HyperStore supports the operation-specific parameters, headers, and ele-
ments listed below.

For operation details and examples see the AWS documentation: ListObjectsV2

1037
Chapter 12. S3 API

Former operation name: GET Bucket (List Objects) Version 2

Note For backward-compatibility HyperStore continues to also support the older version of this API
operation, ListObjects.

Note When using ListObjectsV2, use the continuation-token request parameter to improve per-
formance in listing the content of buckets that contain many objects. For detail see the Amazon doc-
umentation for ListObjectsV2.

12.2.50.1. Query Parameters

l continuation-token
l delimiter

Note The HyperStore system does not support %c2%85(U+0085) as a delimiter value

l encoding-type
l fetch-owner
l list-type
l max-keys
l prefix
l start-after

Note The HyperStore S3 extension request parameter meta=true is no longer supported.

12.2.50.2. Response Headers

HyperStore Extension to the S3 API

The HyperStore system supports the following Response Header as an extension to the "ListObjectsV2" oper-
ation:

Name Description Required

This header specifies the unique ID of the storage policy assigned to
x-gmt-policyid No
the bucket. For more information see "CreateBucket" (page 1011).

12.2.50.3. Response Elements

l ListBucketResult
o IsTruncated
o Contents
n ETag
n Key

1038
 12.2. Supported S3 Operations

n LastModified
n Owner
o DisplayName
o ID
n Size
n StorageClass (values STANDARD and GLACIER only)
o Name
o Prefix
o Delimiter
o MaxKeys
o CommonPrefixes
n Prefix
o Encoding-Type
o KeyCount
o ContinuationToken
o NextContinuationToken
o StartAfter

12.2.51. ListObjectVersions
Returns metadata about all of the versions of objects in a bucket.

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: ListObjectVersions

Former operation name: GET Bucket Object versions

12.2.51.1. Query Parameters

l delimiter
l encoding-type
l key-marker
l max-keys
l prefix
l version-id-marker

12.2.51.2. Response Elements

l ListVersionsResult
o IsTruncated
o KeyMarker
o VersionIdMarker

1039
Chapter 12. S3 API

o NextKeyMarker
o NextVersionIdMarker
o Version
n ETag
n IsLatest
n Key
n LastModified
n Owner
o DisplayName
o ID
n Size
n StorageClass
n VersionId
o DeleteMarker
n IsLatest
n Key
n LastModified
n Owner
o DisplayName
o ID
n VersionId
o Name
o Prefix
o Delimiter
o MaxKeys
o Encoding-Type

12.2.52. ListParts
Lists the parts that have been uploaded for a specific multipart upload.

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: ListParts

Former operation name: List Parts

12.2.52.1. Query Parameters

l key
l max-parts
l part-number-marker
l uploadId

1040
 12.2. Supported S3 Operations

12.2.52.2. Response Headers

l x-amz-abort-date
l x-amz-abort-rule-id

12.2.52.3. Response Elements

l ListPartsResult
o Bucket
o Key
o UploadId
o PartNumberMarker
o NextPartNumberMarker
o MaxParts
o IsTruncated
o Part
n ETag
n LastModified
n PartNumber
n Size
o Initiator
n DisplayName
n ID
o Owner
n DisplayName
n ID
o StorageClass

12.2.53. OPTIONS Object

A browser can send this preflight request to HyperStore to determine if it can send an actual request with the
specific origin, HTTP method, and headers.

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: OPTIONS Object

Former operation name: OPTIONS Object (no change)

12.2.53.1. Request Headers

l Access-Control-Request-Headers
l Access-Control-Request-Method
l Origin

1041
Chapter 12. S3 API

12.2.53.2. Response Headers

l Access-Control-Allow-Headers
l Access-Control-Allow-Methods
l Access-Control-Allow-Origin
l Access-Control-Expose-Headers
l Access-Control-Max-Age

12.2.54. POST Object

The POST operation adds an object to a specified bucket using HTML forms.

Along with the common headers, HyperStore supports the operation-specific form fields listed below.

For operation details and examples see the AWS documentation: POST Object

Former operation name: POST Object (no change)

12.2.54.1. Form Fields

l AWSAccessKeyId
l acl
l Cache-Control, Content-Type, Content-Disposition, Content-Encoding, Expires
l file
l key
l policy
l success_action_redirect, redirect
l success_action_status
l tagging
l x-amz-storage-class

Note HyperStore ignores the value of the x-amz-storage-class field and treats all requests as
being for storage class STANDARD.

l x-amz-meta-*

Note The metadata values must be UTF-8 and must not contain control characters less than
0x20 except for \r, \n, and \t. Also, normal XML escaping is required where appropriate.

l x-amz-website-redirect-location
l x-amz-object-lock-mode

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see
"Object Lock" (page 128).

1042
 12.2. Supported S3 Operations

l x-amz-object-lock-retain-until-date
l x-amz-object-lock-legal-hold
l x-amz-server-side-encryption

Note For information about HyperStore's support of the x-amz-server-side-encryption and x-

amz-server-side-encryption-customer-* request headers, and set-up steps that you must per-
form in order to use HyperStore's server-side encryption features, see "Server-Side Encryp-
tion" (page 137).

l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key
l x-amz-server-side-encryption-customer-key-MD5

12.2.54.2. Response Headers

l success_action_redirect, redirect
l x-amz-expiration
l x-amz-server-side-encryption
l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key-MD5
l x-amz-version-id

12.2.54.3. Response Elements

l Bucket
l ETag
l Key
l Location

12.2.55. PutBucketAcl
Sets the permissions on an existing bucket using access control lists (ACL).

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: PutBucketAcl

Former operation name: PUT Bucket acl

12.2.55.1. Request Headers

l x-amz-acl
l x-amz-grant-full-control
l x-amz-grant-read
l x-amz-grant-read-acp

1043
Chapter 12. S3 API

l x-amz-grant-write
l x-amz-grant-write-acp

12.2.55.2. Request Elements

l AccessControlPolicy
o AccessControlList
n Grant
l Grantee
o DisplayName
o ID
n Permission
o Owner
l DisplayName
l ID

12.2.56. PutBucketCors
Sets the cors configuration for your bucket.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: PutBucketCors

Former operation name: PUT Bucket cors

12.2.56.1. Request Headers

l Content-MD5

12.2.56.2. Request Elements

l CORSConfiguration
o CORSRule
n AllowedHeader
n AllowedMethod
n AllowedOrigin
n ExposeHeader
n ID
n MaxAgeSeconds

12.2.57. PutBucketEncryption
This implementation of the PUT operation uses the encryption subresource to set the default encryption state of
an existing bucket.

1044
 12.2. Supported S3 Operations

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutBucketEncryption

Former operation name: PUT Bucket encryption

Note For information about HyperStore's support for server-side encryption -- including the interaction
of object level, bucket level, and storage policy level encryption settings -- see "Server-Side Encryp-
tion" (page 137).

Note In the current HyperStore release, only the bucket owner is allowed to perform operations
relating to bucket encryption. HyperStore does not currently support the use of bucket policies to
extend bucket encryption permissions to users other than the bucket owner. Specifically, with regard to
"PutBucketPolicy" (page 1054), HyperStore does not currently support the "s3:PutEn-
cryptionConfiguration" or "s3:GetEncryptionConfiguration" actions.

12.2.57.1. Request Elements

l ServerSideEncryptionConfiguration
l Rule
l ApplyServerSideEncryptionByDefault
l KMSMasterKeyID
l SSEAlgorithm

12.2.58. PutBucketInventoryConfiguration
This implementation of the PUT action adds an inventory configuration (identified by the inventory ID) to the
bucket.

Along with the common headers, HyperStore supports the operation-specific parameter and elements listed
below.

For operation details and examples see the AWS documentation: PutBucketInventoryConfiguration

Note Unlike AWS, HyperStore does not use a system account to write inventory reports to the des-
tination bucket. Instead, reports are written by source bucket owner's account. In the current version of
HyperStore, the report destination bucket must be a bucket that is owned by the source bucket
owner.

12.2.58.1. Query Parameter

l id

1045
Chapter 12. S3 API

12.2.58.2. Request Elements

l InventoryConfiguration
o Destination
n S3BucketDestination
o AccountId
o Bucket
o Encryption

Note HyperStore currently does not support encryption of bucket invent-

ory reports. If you include the optional "Encryption" element in the request
body HyperStore will ignore it.

o Format

Note HyperStore currently only supports CSV format.

o Prefix
o Filter
n Prefix
o Id
o IncludedObjectVersions
o IsEnabled
o OptionalFields
n Field
o Schedule
n Frequency

12.2.59. PutBucketLifecycle
Creates a new lifecycle configuration for the bucket or replaces an existing lifecycle configuration.

For operation details and examples see the AWS documentation: PutBucketLifecycle

Note Though HyperStore supports this API operation for backward compatibility, AWS has deprecated
this operation in favor of a newer version called PutBucketLifecycleConfiguration which HyperStore
also supports. For new lifecycle configurations use the new version.

12.2.60. PutBucketLifecycleConfiguration
Creates a new lifecycle configuration for the bucket or replaces an existing lifecycle configuration.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

1046
 12.2. Supported S3 Operations

For operation details and examples see the AWS documentation: PutBucketLifecycleConfiguration

Former operation name: PUT Bucket lifecycle

Note With the HyperStore system, only the bucket owner can create bucket lifecycle rules.

12.2.60.1. Request Headers

HyperStore Extension to the S3 API

The HyperStore system supports the following Request Headers as extensions to the "PutBuck-
etLifecycleConfiguration" operation:

Note Do not set an auto-tiering lifecycle rule and a cross-region replication configuration on the
same source bucket.

Name Description Require-

d
x-gmt- The x-gmt-tieringinfo request header enables you to configure a bucket for schedule- No
tieringinf- based automatic transitioning of objects from local HyperStore storage to a remote stor-
o age system. For background information on the HyperStore auto-tiering feature, see
"Auto-Tiering Feature Overview" (page 206).

The x-gmt-tieringinfo header is formatted as follows:

x-gmt-tieringinfo: PROTOCOL|EndPoint:Endpoint,Action:Action
[,Mode:proxy][,Region:Region][,TieringBucket:TieringBucket]

l PROTOCOL (mandatory) — Specify one of these values, in all caps:

o S3 -- Transition the objects to Amazon S3 storage.

o S3GLACIER -- Transition the objects to Amazon Glacier.
o GCS -- Transition the objects to Google Cloud Storage.
o AZURE -- Transition the objects to Microsoft Azure.
o SPECTRA -- Transition the objects to a Spectra Logic BlackPearl des-
tination.

Note If you are tiering to an S3-compliant system other than

Amazon S3, Glacier, or Google Cloud Storage, use "S3" as the
protocol. This would include, for instance, tiering to a remote
HyperStore region or system.

Note Auto-tiering restrictions based on destination type:

* Tiering to Azure, Google Cloud, or Spectra BlackPearl is not
supported for source buckets that have versioning enabled or
that have had versioning enabled in the past.
* When auto-tiering to Spectra BlackPearl is used for a bucket,
objects in the bucket will not be auto-tiered unless they are lar-

1047
Chapter 12. S3 API

Name Description Require-

d

ger than 5MB. Objects 5MB or smaller will remain in HyperStore.

l EndPoint:EndPoint (mandatory) — The service endpoint URL to use as your

auto-tiering destination. For example with Amazon S3, choose the region end-
point that’s most suitable for your location (such as s3-us-west-1.amazon-
aws.com if your organization is in northern California). Or in the case of
Spectra BlackPearl, specify the URL for your Spectra BlackPearl destination.

If your ultimate tiering destination is Glacier, you must specify an Amazon S3

endpoint here, not a Glacier endpoint. The HyperStore system will first trans-
ition the objects to your specified Amazon S3 endpoint and then from there they
will be immediately transitioned to the corresponding Glacier location.

If you want to auto-tier to an external HyperStore system -- not a different region

within the same HyperStore system but rather a different HyperStore system
altogether -- see "Specify a Different HyperStore System as Tiering Destin-
ation, If Applicable" (page 213), in regard to the format requirements for the
tiering endpoint.

Note You must use nested URL encoding. First URL encode the End-
point value (the endpoint itself), and then URL encode the whole x-gmt-
tieringinfo value.

Note Once you've configured an auto-tiering lifecycle on a source

bucket you cannot subsequently change the tiering endpoint for that
source bucket.

l Action:Action (mandatory) — This parameter specifies how the HyperStore sys-

tem will handle S3 GET Object requests for objects that have been transitioned
to the tiering destination. The choices are:

o stream — If the client submits a GET Object request to HyperStore, the

HyperStore system retrieves the object from the destination and streams
it through to the client. This method is supported only if the Protocol is
S3, GCS, or AZURE.
o nostream — If the client submits a GET Object request to HyperStore,
the HyperStore system rejects the GET request. Instead, clients must
submit a POST Object restore request in order to temporarily restore a
copy of the object to local HyperStore storage.
o cache -- The local system GETs the object from the tiering destination
system and immediately streams it through to the client, and sim-
ultaneous saves a copy of the object to local storage. The local copy is
kept in local storage -- cached -- for a period that depends on how the
auto-tiering policy is configured:
n If the tiering policy uses a "Days After..." configuration, the cach-

1048
 12.2. Supported S3 Operations

Name Description Require-

d
ing retention period is the same as the number of days that trig-
gers the tiering of objects in the first place. For example, if the
auto-tiering policy is configured to tier objects 30 days after
object creation, and "Cache (Stream & Restore)" mode is selec-
ted for handling GETs of tiered objects, then when there's a GET
request for a tiered object the object is streamed to the client and
also cached locally for 30 days.
n If the tiering policy uses a "After Date..." configuration, the
cached local copy is retained only until the next running of the
auto-tiering / auto-expiration cron job (which runs once a day).
The same caching behavior applies to GETs of objects that were
tiered by "Bridge Mode" (proxy mode).

During the period while the object is cached locally, subsequent GETs
of the object can be served from local storage. After the cache period
expires, the local copy is automatically deleted by the next run of the
daily auto-tiering / auto-expiration cron job. Following deletion of the
cached copy, the next GET of the object will be served from the tiering
destination site (and a copy of the object will be once again be cached).

If the Protocol is S3, GCS, or AZURE you can use either "stream" or "nostream"
or "cache". If the Protocol is S3GLACIER or SPECTRA you must use "nostream"
(the "stream" and "cache" options are not supported for those destinations).

l Mode:proxy (optional) — If you specify this option, then:

o All objects uploaded to the bucket from this time forward (all objects
uploaded after you successfully submit the PUT Bucket lifecycle
request) will be immediately transitioned to the destination system.
o Any objects already in the bucket at the time that you submit the PUT
Bucket lifecycle request will be subject to the transition schedule that
you define in the request body.

Proxy mode is supported only if the Protocol is S3, GCS, or AZURE (proxy
mode is not supported for S3GLACIER or SPECTRA tiering). For more inform-
ation on proxy mode -- also known as "bridge mode" -- see "Auto-Tiering
Feature Overview" (page 206).

l Region: Region (optional) — If you are tiering to an S3 protocol endpoint other

than Amazon (for example a remote HyperStore destination) and you are hav-
ing the system create a destination bucket for you to tier to -- meaning you are
leaving the "TieringBucket" field empty or you are specifying the name of a
bucket that does not yet exist -- use the "Region" field to indicate the des-
tination service region in which you want the bucket created.
l TieringBucket:TieringBucket (optional) — The name of the bucket to transition
objects into, in the tiering destination system. This can be either:
o The name of a bucket that already exists in the destination system,
and for which you are the bucket owner. In this case HyperStore will
use this existing bucket as the tiering destination.

1049
Chapter 12. S3 API

Name Description Require-

d
o The name of a bucket that you want HyperStore to create in the des-
tination system, to use as the tiering destination. Be sure to choose a
bucket name that is very likely to be unique in the destination system. If
your supplied bucket name is not unique in the destination system,
HyperStore will be unable to create the bucket and the PUT Bucket life-
cycle request will fail.

If you omit the tiering bucket parameter, then in the destination system Hyper-
Store will create a tiering bucket named as follows:

<origin-bucket-name-truncated-to-34-characters>-<28-character-ran-
dom-string>

Example x-gmt-tieringinfo request headers:

# Example 1 (before URL encoding) Tiering to Amazon S3, into target bucket
# named 'bucket12'. Streaming for local GETs will be supported.

x-gmt-tieringinfo: S3|EndPoint:http://s3.amazonaws.com,Action:stream,
TieringBucket:bucket12

# Example 1 after nested URL encoding (endpoint value first, then whole
# header value)

x-gmt-tieringinfo: S3%7CEndPoint%3Ahttp%253A%252F%252Fs3.amazonaws.com
%2CAction%3Astream%2CTieringBucket%3Abucket12

# Example 2 (before URL encoding) Tiering to Azure. HyperStore will derive

target
# bucket name from source bucket name. Streamed local GETs will not be
supported,
# clients must use Restore.

x-gmt-tieringinfo:
AZURE|EndPoint:https://blob.core.windows.net,Action:nostream

# Example 2 after nested URL encoding (endpoint value first, then whole
# header value)

x-gmt-tieringinfo:
AZURE%7CEndPoint%3Ahttps%253A%252F%252Fblob.core.windows.net
%2CAction%3Anostream

x-gmt- If you include this header in your "PUT Bucket lifecycle" request and set the header No
com- value to "LAT", then in lifecycle rules that you configure with the "Days" comparator the
pare rule will be implemented as number of days since the object’s Last Access Time.

If you do not use this extension header, or if you include the header but assign it no
value or any value other than "LAT", then "Days" based lifecycle rules will be imple-
mented as number of days since the object’s Creation Time (the default Amazon S3
behavior).

1050
 12.2. Supported S3 Operations

Name Description Require-

d
You can use this header to create:

l Last Access Time based auto-tiering rules (use this header and also the x-gmt-
tierinfo header).
l Last Access Time based expiration rules (use this header but not the x-gmt-tier-
info header).

Note An object’s Last Access Time is updated if the object is accessed either
for retrieval (GET or HEAD) or modification (PUT/POST/Copy). If an object is
created and then never accessed, its Last Access Time will be its Creation
Time.

Note If you use the x-gmt-compare header and set it to "LAT", it does not apply
to any in NoncurrentVersionTransition or NoncurrentVersionExpiration rules
within the lifecycle policy (for non-current versions of versioned objects).
These types of rules are always based on the time elapsed since an object ver-
sion became non-current (was replaced by a new version of the object).

x-gmt- If you use the x-gmt-tieringinfo request header to configure auto-tiering for a bucket, No
post- you can optionally also use the x-gmt-post-tier-copy request header to specify a num-
tier- ber of days for which a local copy of auto-tiered objects should be retained. For
copy example if you set x-gmt-post-tier-copy: 7 then after each object is auto-tiered to the
tiering destination, a copy of the object will be kept in the HyperStore source bucket for
7 days. After that the local copy will be deleted and only object metadata will be
retained locally.

There is no upper limit on this value. So if you want the local copy retention period to
be practically limitless, you could for example set this header to 36500 to indicate a
local copy retention period of 100 years.

If you omit the x-gmt-post-tier-copy request header, then by default local objects are
deleted after they are successfully auto-tiered to the tiering destination system, and
only object metadata is retained locally.

12.2.60.2. Request Elements

l LifecycleConfiguration
o Rule
n AbortIncompleteMultipartUpload
l DaysAfterInitiation
n Expiration
l Date
l Days
l ExpiredObjectDeleteMarker

1051
Chapter 12. S3 API

n Filter
l And
o Prefix
o Tag
n Key
n Value
l Prefix
l Tag
o Key
o Value
n ID
n NoncurrentVersionExpiration
l NoncurrentDays
n NoncurrentVersionTransition
l NoncurrentDays
l StorageClass
n Prefix
n Status
n Transition
l Date
l Days
l StorageClass

Note If you are using "Bridge Mode" transition (whereby objects are auto-tiered immediately after
being uploaded to HyperStore), leave the "Prefix" attribute empty. Bridge Mode does not support fil-
tering by prefix. Also, Bridge Mode does not support filtering by tag(s).

12.2.61. PutBucketLogging
Set the logging parameters for a bucket and to specify permissions for who can view and modify the logging
parameters.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutBucketLogging

Former operation name: PUT Bucket logging

Note For a bucket that has bucket logging enabled, bucket logs (server access logs) are generated
every 10 minutes by a HyperStore system cron job, if there was activity for that bucket during that inter-
val.

1052
 12.2. Supported S3 Operations

Note If you are using bucket logging in your service, and if you use a load balancer in front of your S3
Service nodes, you should configure your S3 Service to support the HTTP X-Forwarded-For header.
This will enable bucket logs to record the true originating IP address of S3 requests, rather than the
load balancer IP address. By default the S3 Service does not support the X-Forwarded-For header.
You can enable support for this header using the system configuration file s3.xml.erb.

12.2.61.1. Request Elements

l BucketLoggingStatus
o LoggingEnabled
n TargetBucket
n TargetGrants
l Grant
o Grantee
n DisplayName
n EmailAddress
n ID
o Permission
n TargetPrefix

12.2.62. PutBucketNotificationConfiguration
Enables notifications of specified events for a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutBucketNotificationConfiguration

Note In the current HyperStore release, only the bucket owner is allowed to submit this request and
the bucket owner must also be the owner of the destination Queue.

Note HyperStore's bucket notification feature and its SQS Service (for notification message queueing
and delivery) are disabled by default. For information on how to enable this feature set see "Hyper-
Store Support for the AWS SQS API" (page 1115).

1053
Chapter 12. S3 API

12.2.62.1. Request Elements

l NotificationConfiguration
o QueueConfiguration
n Event
n Filter
l S3Key
o FilterRule
n Name
n Value
n Id
n Queue

For Event types, HyperStore supports only the following:

l s3:ObjectCreated:*
l s3:ObjectCreated:Put
l s3:ObjectCreated:Post
l s3:ObjectCreated:Copy
l s3:ObjectCreated:CompleteMultipartUpload
l s3:ObjectRemoved:*
l s3:ObjectRemoved:Delete
l s3:ObjectRemoved:DeleteMarkerCreated

12.2.63. PutBucketOwnershipControls
Creates or modifies OwnershipControls for a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutBucketOwnershipControls

12.2.63.1. Request Elements

l OwnershipControls
o Rule
n ObjectOwnership

IMPORTANT ! In the current version of HyperStore, only the BucketOwnerPreferred and

ObjectWriter ownership types are supported. HyperStore does not yet support the Buck-
etOwnerEnforced ownership type. If you use the BucketOwnerEnforced ownership type
in a PutBucketOwnershipControls API call, the call will fail with a Malformed XML error.

12.2.64. PutBucketPolicy
Applies an S3 bucket policy to an S3 bucket.

1054
 12.2. Supported S3 Operations

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutBucketPolicy

Former operation name: PUT Bucket policy

12.2.64.1. Request Elements

The request body is a JSON-formatted bucket policy containing one or more policy statements. Within a
policy's Statement block(s), HyperStore support for policy statement elements and their values is as follows:

l Sid -- Same as Amazon: Custom string identifying the statement, for example "Statement1" or "Only
allow access from partner source IPs"
l Effect -- Same as Amazon: "Allow" or "Deny"
l Principal -- The following formats are supported:
l "*" -- Statement applies to all users (also known as "anonymous access").
l {"CanonicalUser": "<canonicalUserId>"} -- Statement applies to the specified HyperStore
account root user.
l {"CanonicalUser": ["<canonicalUserId>", "<canonicalUserId>",...]} -- Statement applies to the spe-
cified HyperStore account root users.
l {"AWS":"arn:aws:iam::<canonicalUserId>:root"} -- Statement applies to the specified HyperStore
account root user.
l {"AWS":"arn:aws:iam::<canonicalUserId>:user/<iamUserName>"} -- Statement applies to the
specified IAM user. In this format the <canonicalUserId> is that of the parent account root user.

Note You can obtain a HyperStore user's canonical ID by retrieving the user through the
CMC's "Manage Users" (page 301) page or by using the Admin API method GET /user.

Note In formats of the "AWS":"arn:aws:iam::...." type, AWS uses "Account Id" to identify the
account root user. In HyperStore the canonical user ID is used for this purpose, since in
HyperStore there is not a separate account ID that's different than the canonical user ID.

l Action -- See details below.

l Resource -- Same as Amazon; must be one of:
l "arn:aws:s3:::<bucketName>" -- For bucket actions (such as "s3:ListBucket") and bucket subre-
source actions (such as "s3:GetBucketAcl").
l "arn:aws:s3:::<bucketName>/*" or "arn:aws:s3:::<bucketName>/<objectName>" -- For object
actions (such as "s3:PutObject").
l Condition -- See details below.

12.2.64.1.1. Supported "Action" Values

Within bucket policy statements, HyperStore supports only the following Action values (also known as per-
mission keywords).

1055
Chapter 12. S3 API

Note For information about how to use Action values in a bucket policy, see the AWS documentation
on Specifying Permissions in a Policy.

Object Actions

l s3:AbortMultipartUpload
l s3:BypassGovernanceRetention
l s3:DeleteObject
l s3:DeleteObjectTagging
l s3:DeleteObjectVersion
l s3:DeleteObjectVersionTagging
l s3:GetObject
l s3:GetObjectAcl
l s3:GetObjectLegalHold
l s3:GetObjectRetention
l s3:GetObjectTagging
l x3:GetObjectTorrent
l s3:GetObjectVersion
l s3:GetObjectVersionAcl
l s3:GetObjectVersionTagging
l s3:ListMultipartUploadParts
l s3:PutObject
l s3:PutObjectAcl
l s3:PutObjectLegalHold
l s3:PutObjectRetention
l s3:PutObjectTagging
l s3:PutObjectVersionAcl
l s3:PutObjectVersionTagging
l s3:RestoreObject

Bucket Actions

l s3:CreateBucket
l s3:DeleteBucket
l s3:ListBucket
l s3:ListBucketMultipartUploads
l s3:ListBucketVersions

Bucket Subresource Actions

l s3:DeleteBucketPolicy
l s3:DeleteBucketWebsite

1056
 12.2. Supported S3 Operations

l s3:GetBucketAcl
l s3:GetBucketCORS
l s3:GetBucketLocation
l s3:GetBucketLogging
l s3:GetBucketNotification
l s3:GetBucketObjectLockConfiguration
l s3:GetBucketPolicy
l s3:GetBucketRequestPayment
l s3:GetBucketTagging
l s3:GetBucketVersioning
l s3:GetBucketWebsite
l s3:GetInventoryConfiguration
l s3:GetLifecycleConfiguration
l s3:GetReplicationConfiguration
l s3:PutBucketAcl
l s3:PutBucketCORS
l s3:PutBucketLogging
l s3:PutBucketNotification
l s3:PutBucketObjectLockConfiguration
l s3:PutBucketPolicy
l s3:PutBucketRequestPayment
l s3:PutBucketTagging
l s3:PutBucketVersioning
l s3:PutBucketWebsite
l s3:PutInventoryConfiguration
l s3:PutLifecycleConfiguration
l s3:PutReplicationConfiguration

Note Like Amazon, the HyperStore system supports the use of a wildcard in your Action configuration
("Action":["s3:*"]). When an Action wildcard is used together with an object-level Resource element
("arn:aws:s3:::<bucketName>/*" or "arn:aws:s3:::<bucketName>/<objectName>"), the wildcard denotes
all the Object actions that HyperStore supports. When an Action wildcard is used together with
bucket-level Resource element ("arn:aws:s3:::<bucketName>"), the wildcard denotes all the Bucket
actions and Bucket Subresource actions that HyperStore supports.

12.2.64.1.2. Supported "Condition" Values

Within bucket policy statements, HyperStore supports only the following Condition operators and keys.

Note For information about how to use condition operators and keys in a bucket policy, see the AWS
documentation on Specifying Conditions in a Policy.

1057
Chapter 12. S3 API

Condition Operators

l ForAllValues:StringLike
l ForAnyValue:StringLike
l IpAddress

Note If you are using load balancers in front of the HyperStore S3 Service, then IP address
based bucket policies will only work if you use PROXY Protocol between the load balancers
and the S3 Service. This protocol allows the load balancers to pass the IP addresses of ori-
ginating clients to the S3 Service along with the S3 requests. For more information about
enabling PROXY Protocol support on the S3 Service side, see "s3_proxy_protocol_enabled"
(page 582) in "common.csv" (page 562). For guidance on configuring the load balancers con-
sult with Cloudian Sales Engineering or Support.

Note that using the "X-Forwarded-For" HTTP header is not sufficient to support IP address
based bucket policies. You must use PROXY Protocol if you have load balancers in front of the
S3 Service and want to use IP address based bucket policies .

l NotIpAddress
l NumericEquals
l NumericNotEquals
l NumericLessThan
l NumericLessThanEquals
l NumericGreaterThan
l NumericGreaterThanEquals
l StringEquals
l StringNotEquals
l StringEqualsIgnoreCase
l StringNotEqualsIgnoreCase
l StringLike
l StringNotLike

Condition Keys

l aws:Referer
l aws:SourceIp

Note If you create a bucket policy that restricts access based on source IP address, these restric-
tions will not apply to IP addresses within your HyperStore cluster. IP addresses from within your
cluster are automatically "whitelisted".

l s3:delimiter
l s3:ExistingObjectTag/<tag-key>
l s3:max-keys
l s3:object-lock-legal-hold

1058
 12.2. Supported S3 Operations

l s3:object-lock-mode
l s3:object-lock-remaining-retention-days
l s3:object-lock-retain-until-date
l s3:prefix
l s3:RequestObjectTag/<tag-keys>
l s3:RequestObjectTagKeys
l s3:VersionId
l s3:x-amz-acl
l s3:x-amz-copy-source
l s3:x-amz-grant-full-control
l s3:x-amz-grant-read
l s3:x-amz-grant-read-acp
l s3:x-amz-grant-write
l s3:x-amz-grant-write-acp
l s3:x-amz-metadata-directive
l s3:x-amz-server-side-encryption

For examples of the kinds of things you can do with bucket policies, see the AWS documentation on Bucket
Policy Examples.

12.2.65. PutBucketReplication
Creates a replication configuration or replaces an existing one.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: PutBucketReplication

Former operation name: PUT Bucket replication

Note
* Unlike Amazon S3, HyperStore does not require that you set up an IAM Role (or anything analogous)
in order to use bucket replication. Also, HyperStore does not require that the destination bucket be in a
different region than the source bucket. With HyperStore you can replicate to a destination bucket that’s
in the same region as the source bucket, if you want to.
* Like Amazon S3, HyperStore bucket replication requires that versioning must be enabled (using the
"PutBucketVersioning" (page 1061) operation) on both the source bucket and the destination bucket.
* Do not set a cross-region replication configuration and a bucket lifecycle rule for auto-tiering on the
same source bucket.

12.2.65.1. Request Headers

l Content-MD5

HyperStore Extension to the S3 API

The HyperStore system supports the following Request Headers as extensions to the "PUT Bucket replication"

1059
Chapter 12. S3 API

operation. Typically these headers are not needed for bucket replication. These headers are required only in a
scenario where you want data to be replicated to a destination bucket in an external S3-compatible system
(rather than in a service region within the same HyperStore system as the source bucket). Before using these
extensions you should review "Cross-System Replication" (page 220) including the limitations and caveats
noted in that section.

Name Description Required

x-gmt-crr-end- Service endpoint of the destination S3 service, in format <pro- See
point tocol>://<endpoint>:<port>. For example https://s3.amazonaws:443. Since description
security credentials will be transmitted in this request (see "x-gmt-crr-cre-
dentials" below), HTTPS is the recommended protocol rather than regular
HTTP. HyperStore does not force HTTPS use here, but for security HTTPS is
advisable.

This header is required only if the destination bucket is not in the same Hyper-
Store system as the source bucket. Do not use this header if the destination
bucket is in the same HyperStore system as the source bucket.

x-gmt-crr-cre- Access key and secret key for the user account that HyperStore should use to See
dentials write to the destination bucket in the destination S3 system, in format <access- description
key>:<secret-key>. For example, 00caf3940d-
c923c59406:Ku0bMR0H5nSA7t8N+ngP6uPPTINSxJ/Q2olCMexx. This user
account must have write permissions on the destination bucket. For example,
if the destination bucket is in the Amazon S3 system, this header is used to
specify the Amazon S3 access key and secret key for an account that has
write permissions on the destination bucket.

This header is required only if the destination bucket is not in the same Hyper-
Store system as the source bucket. Do not use this header if the destination
bucket is in the same HyperStore system as the source bucket.

12.2.65.2. Request Elements

l ReplicationConfiguration
o Role
o Rule
n Destination
l Bucket
l StorageClass
n ID
n Prefix
n Status

Note
* Use the same "Bucket" value formatting as in the Amazon S3 API spec, i.e. arn:aws:s3:::<buck-
etname>.
* As with the Amazon S3 API specification, for HyperStore the "Role" element must be included in the
PUT Bucket replication request. However, HyperStore ignores the "Role" element’s value (so, you can

1060
 12.2. Supported S3 Operations

use any random string as its value). HyperStore does not use an IAM role or anything analogous when
implementing cross-region replication.
* If you include the "StorageClass element" in the request, HyperStore ignores its value.

12.2.66. PutBucketTagging
Sets the tags for a bucket.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: PutBucketTagging

Former operation name: PUT Bucket tagging

Note The HyperStore Admin API supports a method for retrieving all the bucket tags for all users in a
specified group. Because it is implemented through the Admin API, that method does not require the
users' S3 access credentials. For more information see GET /bucketops/gettags.

12.2.66.1. Request Headers

l Content-MD5

12.2.66.2. Request Elements

l Tagging
o TagSet
n Tag
l Key
l Value

12.2.67. PutBucketVersioning
Sets the versioning state of an existing bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutBucketVersioning

Former operation name: PUT Bucket versioning

Note Do not enable versioning on a bucket that is configured for auto-tiering to Azure, Google Cloud,
or Spectra BlackPearl. Auto-tiering to these destinations will not work properly for buckets that have ver-
sioning enabled.

1061
Chapter 12. S3 API

12.2.67.1. Request Elements

l VersioningConfiguration
o Status

12.2.68. PutBucketWebsite
Sets the configuration of the website that is specified in the website subresource.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutBucketWebsite

Former operation name: PUT Bucket website

12.2.68.1. Request Elements

l WebsiteConfiguration
o ErrorDocument
o IndexDocument
o RedirectAllRequestsTo
n HostName
n Protocol

12.2.69. PutObject
Adds an object to a bucket.

Along with the common headers, HyperStore supports the operation-specific headers listed below.

For operation details and examples see the AWS documentation: PutObject

Former operation name: PUT Object

12.2.69.1. Request Headers

l Cache-Control
l Content-Disposition
l Content-Encoding
l Expires
l x-amz-acl
l x-amz-grant-full-control
l x-amz-grant-read
l x-amz-grant-read-acp
l x-amz-grant-write
l x-amz-grant-write-acp
l x-amz-meta-*

1062
 12.2. Supported S3 Operations

Note The metadata values must be UTF-8 and must not contain control characters less than
0x20 except for \r, \n, and \t. Also, normal XML escaping is required where appropriate.

l x-amz-object-lock-legal-hold
l x-amz-object-lock-mode

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see
"Object Lock" (page 128).

l x-amz-object-lock-retain-until-date
l x-amz-server-side-encryption

Note For information about HyperStore's support of the x-amz-server-side-encryption and x-

amz-server-side-encryption-customer-* request headers, and set-up steps that you must per-
form in order to use HyperStore's server-side encryption features, see "Server-Side Encryp-
tion" (page 137).

l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key
l x-amz-server-side-encryption-customer-key-MD5
l x-amz-storage-class

Note HyperStore ignores the value of the x-amz-storage-class header and treats all requests as
being for storage class STANDARD.

l x-amz-tagging
l x-amz-website-redirect-location

HyperStore Restrictions on Object Names

The following control characters cannot be used anywhere in an object name and will result in a 400 Bad
Request response: 0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A ("\n"), 0x0B, 0x0C, 0x0D
("\r"), 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E,
0x1F

Also unsupported are:

l 0x09 ("\t") at the beginning of an object name

l 0xBF (inverted question mark) at the end of an object name
l Object names consisting only of "." or only of ".."
l Object names containing a combination of "." and "/", or a combination of ".." and "/"

12.2.69.2. Response Headers

l x-amz-expiration
l x-amz-server-side-encryption

1063
Chapter 12. S3 API

l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key-MD5
l x-amz-version-id

12.2.70. PutObjectAcl
Uses the acl subresource to set the access control list (ACL) permissions for an object that already exists in a
bucket.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: PutObjectAcl

Former operation name: PUT Object acl

12.2.70.1. Request Headers

l x-amz-acl
l x-amz-grant-full-control
l x-amz-grant-read
l x-amz-grant-read-acp
l x-amz-grant-write
l x-amz-grant-write-acp

12.2.70.2. Request Elements

l AccessControlPolicy
o AccessControlList
n Grant
l Grantee
o DisplayName
o ID
l Permission
o Owner
n DisplayName
n ID

12.2.70.3. Response Headers

l x-amz-version-id

12.2.71. PutObjectLegalHold
Applies a Legal Hold configuration to the specified object.

1064
 12.2. Supported S3 Operations

Along with the common headers, HyperStore supports the operation-specific parameters and elements listed
below.

For operation details and examples see the AWS documentation: PutObjectLegalHold

Former operation name: PUT Object legal hold

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see "Object
Lock" (page 128).

12.2.71.1. Query Parameters

l versionId

12.2.71.2. Request Elements

l LegalHold
o Status

12.2.72. PutObjectLockConfiguration
Places an Object Lock configuration on the specified bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutObjectLockConfiguration

Former operation name: PUT Bucket object lock configuration

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see "Object
Lock" (page 128).

12.2.72.1. Request Elements

l ObjectLockConfiguration
o ObjectLockEnabled
o Rule
n DefaultRetention
l Days
l Mode
l Years

12.2.73. PutObjectRetention
Places an Object Retention configuration on an object.

Along with the common headers, HyperStore supports the operation-specific parameters, headers, and ele-
ments listed below.

1065
Chapter 12. S3 API

For operation details and examples see the AWS documentation: PutObjectRetention

Former operation name: PUT Object retention

Note For more information on HyperStore's support for the S3 "Object Lock" feature, see "Object
Lock" (page 128).

12.2.73.1. Query Parameters

l versionId

12.2.73.2. Request Headers

l x-amz-bypass-governance-retention

12.2.73.3. Request Elements

l Retention
o Mode
o RetainUntilDate

12.2.74. PutObjectTagging
Sets the supplied tag-set to an object that already exists in a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutObjectTagging

Former operation name: PUT Object tagging

12.2.74.1. Request Elements

l Tagging
o TagSet
n Tag
l Key
l Value

12.2.75. PutPublicAccessBlock
Creates or modifies the PublicAccessBlock configuration for a bucket.

Along with the common headers, HyperStore supports the operation-specific elements listed below.

For operation details and examples see the AWS documentation: PutPublicAccessBlock

1066
 12.2. Supported S3 Operations

12.2.75.1. Request Elements

l PublicAccessBlockConfiguration
o BlockPublicAcls
o IgnorePublicAcls
o BlockPublicPolicy
o RestrictPublicBuckets

12.2.75.1.1. HyperStore Implementation Notes

l If BlockPublicAcls is set to true, then:

o Put object/bucket ACL calls will fail with access denied if any grant in the ACL is public (grantee
is AllUsers or AuthenticatedUsers)
o Put object calls will fail with access denied if any grant in the ACL is public (or if canned ACL is
public)
l If IgnorePublicAcls is set to true, then when performing ACL permission checks during operations on
the bucket or its objects, public grants are ignored (and therefore public accounts will be denied
access).
l If BlockPublicPolicy is set to true, then a PutBucketPolicy call will fail if any statement in the policy is pub-
lic. A statement is considered public if the Effect is Allow and the Principal has a wildcard -- unless there
is an IpAddress:{aws:SourceIp condition associated with the statement that restricts the requesting
source IP to one or more specified IP addresses.
l If RestrictPublicBuckets is set to true, then the bucket's bucket policy will be replaced with a policy that
allows access to the bucket and its objects only to the bucket owner. Note that RestrictPublicBuckets
does not restrict the use of ACLs on the bucket or its objects. To restrict ACLs use the BlockPublicAcls
and/or IgnorePublicAcls settings.

12.2.76. RestoreObject
Restores a tiered object back into HyperStore.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: RestoreObject

Former operation name: POST Object restore

Note In the context of the HyperStore system, this standard S3 operation is for temporarily restoring a
copy of an object that has been auto-tiered to a tiering destination, such as Amazon S3 or Amazon Gla-
cier. For information about the HyperStore auto-tiering feature, see "Auto-Tiering Feature Overview"
(page 206).

12.2.76.1. Request Headers

l Content-MD5

1067
Chapter 12. S3 API

12.2.76.2. Request Elements

l RestoreRequest
o Days
o GlacierJobParameters
n Tier

Note For the sake of S3 API compatibility, HyperStore's S3 Service allows the request elements Gla-
cierJobParameters and Tierto be included in a "POST Object restore" request -- but in the current
HyperStore release these elements will have no effect on how the restore request is implemented.

12.2.77. UploadPart
Uploads a part in a multipart upload.

Along with the common headers, HyperStore supports the operation-specific headers listed below.

For operation details and examples see the AWS documentation: UploadPart

Former operation name: Upload Part

12.2.77.1. Request Headers

l x-amz-server-side-encryption

l x-amz-server-side-encryption-customer-algorithm

Note For information about HyperStore's support of the x-amz-server-side-encryption-customer-

* request headers, and set-up steps that you must perform in order to use HyperStore's server-
side encryption features, see "Server-Side Encryption" (page 137).

l x-amz-server-side-encryption-customer-key
l x-amz-server-side-encryption-customer-key-MD5

12.2.77.2. Response Headers

l x-amz-server-side-encryption
l x-amz-server-side-encryption-customer-algorithm
l x-amz-server-side-encryption-customer-key-MD5

12.2.78. UploadPartCopy
Uploads a part by copying data from an existing object as data source.

Along with the common headers, HyperStore supports the operation-specific headers and elements listed
below.

For operation details and examples see the AWS documentation: UploadPartCopy

Former operation name: Upload Part - Copy

1068
 12.2. Supported S3 Operations

12.2.78.1. Request Headers

l x-amz-copy-source
l x-amz-copy-source-if-match
l x-amz-copy-source-if-modified-since
l x-amz-copy-source-if-none-match
l x-amz-copy-source-if-unmodified-since
l x-amz-copy-source-range
l x-amz-source-expected-bucket-owner

12.2.78.2. Response Headers

l x-amz-copy-source-version-id
l x-amz-server-side-encryption

12.2.78.3. Response Elements

l CopyPartResult
o ETag
o LastModified

1069
This page left intentionally blank
Chapter 13. IAM API

13.1. Introduction

13.1.1. HyperStore Support for the AWS IAM API

Subjects covered in this section:

l Introduction (immediately below)

l "Restrictions and Limitations in HyperStore's IAM Support" (page 1071)
l "S3 Access Credentials Are Needed to Access the IAM Service" (page 1071)
l "HyperStore IAM Extensions to Support RBAC for Admin Functions" (page 1072)
l "Deleting or Suspending HyperStore Users Who Have Created IAM Users" (page 1072)
l "Disabling the HyperStore IAM Service" (page 1072)
l "The IAM Service in Multi-Region Systems" (page 1073)

HyperStore provides limited support for the Amazon Web Services Identity and Access Management (IAM)
API. This support enables each HyperStore user, under his or her HyperStore user account, to create IAM
groups and IAM users and IAM roles. The HyperStore user -- also known as the "account root user" -- can then
grant those IAM groups, users, and roles permissions to perform certain actions (such as reading or writing
objects in a particular bucket or buckets). As with Amazon, the means by which a HyperStore account root user
grants such permissions to IAM groups, users, and roles is by creating and attaching "managed" IAM policies
to IAM groups, users, and roles, and/or by creating and embedding "inline" IAM policies for IAM groups, users,
and roles. By default newly created IAM entities have no permissions; they gain permissions only by their
association with managed or inline IAM policies.

In the HyperStore system all S3 object data created by IAM users belongs to the parent HyperStore user
account. Consequently, if an IAM user is deleted by their HyperStore parent user, the IAM user's data is not
deleted from the system.

13.1.1.1. Restrictions and Limitations in HyperStore's IAM Support

l HyperStore supports most but not all of the Amazon IAM API "Actions". For the list of supported actions
see the "Supported IAM Actions" section.
l HyperStore supports most but not all of the Amazon IAM API policy elements, actions, resources, and
condition keys. For more information see "Supported IAM Policy Elements" (page 1105).
l Only HyperStore users -- account root users -- can log into the CMC. The IAM users that HyperStore
users create cannot login to the CMC, and cannot use the CMC as their S3 client application. To access
the HyperStore S3 Service, IAM users must use a third party S3 client application.

13.1.1.2. S3 Access Credentials Are Needed to Access the IAM Service

To access the IAM Service, HyperStore users need S3 access credentials. Whenever you create users in the
CMC or with the Admin API -- whether the users are regular users, group administrators, or additional system
administrators -- S3 access credentials are automatically created for the users.

1071
Chapter 13. IAM API

If users are using a third party IAM client to access the HyperStore IAM Service, the users can obtain their S3
access credentials by logging in to the CMC and going to the Security Credentials page (via the drop-down
menu under the user login name). They can then supply those credentials to the third party IAM client applic-
ation.

If users are using the CMC's built-in IAM client, the CMC automatically uses the user's S3 credentials to access
the IAM service. Through the CMC's IAM section, HyperStore users can create IAM groups and users and so
on.

The exception is the pre-configured default system administrative user -- the user named "admin". The
"admin" user does not have S3 access credentials by default. Consequently, if you are logged into the
CMC as the "admin" user and you go to the CMC's IAM section you will see the following error displayed:

"No valid Access Key detected. Cannot connect to the IAM Service."

If you want to use the functionality in the CMC's IAM section as the "admin" user, you must create S3 cre-
dentials for this user. While logged into the CMC as the "admin" user, go to the Security Credentials page (via
the drop-down menu under the login name). Then in the S3 Access Credentials section of the page, click
Create New Key. This creates S3 access credentials for the "admin" user. Now you can use the CMC's
IAM section without getting an access key error.

13.1.1.3. HyperStore IAM Extensions to Support RBAC for Admin Functions

The HyperStore implementation of the IAM API includes extensions that:

l Allow HyperStore system admins, group admins, or regular users to execute certain read-only Hyper-
Store administrative functions by submitting a request to the IAM Service.
l Allow HyperStore system admins, group admins, or regular users to grant their IAM users permission to
execute those same read-only HyperStore administrative functions.

For more information, including information about the client tool that HyperStore provides to help you use this
feature, see "Role-Based Access to Admin API Operations" (page 794).

13.1.1.4. Deleting or Suspending HyperStore Users Who Have Created IAM Users
If a HyperStore user creates IAM groups, users, and/or roles, and then subsequently you delete that Hyper-
Store user from the system, all IAM resources associated with that HyperStore user will also be deleted from
the system. That includes IAM groups, users, roles, and policies that the HyperStore user created, the security
credentials of those IAM users, and any object data that those IAM users have stored in the system.

If rather than deleting the HyperStore user you suspend the HyperStore user (make the user inactive), then
any IAM groups, users, and/or roles that the HyperStore user created will be unable to access any HyperStore
services (just like the suspended HyperStore user will be unable to access HyperStore services). If you sub-
sequently make the HyperStore user active again, then IAM groups, users, and roles under that HyperStore
user will again be able to access HyperStore services.

13.1.1.5. Disabling the HyperStore IAM Service

HyperStore's IAM Service is enabled by default, and IAM Service functionality can be accessed either through
the CMC or by your third party or custom client applications. If you want to disable the IAM Service, do the fol-
lowing:

1. On your Configuration Master node open this configuration file in a text editor:
/etc/cloudian-<version>-puppet/manifests/extdata/common.csv

1072
 13.1. Introduction

2. Change the setting iam_service_enabled,true to iam_service_enabled,false and save your change.

3. Push your change to the cluster and restart the S3 Service. If you need instructions see "Pushing Con-
figuration File Edits to the Cluster and Restarting Services" (page 556).

If you disable the HyperStore IAM Service, then IAM functions will no longer display in the CMC and the
IAM Service will no longer accept requests from IAM client applications.

13.1.1.6. The IAM Service in Multi-Region Systems

In a multi-region HyperStore system, the IAM Service runs only on nodes in the default service region. In your
DNS set-up, the IAM service endpoint should resolve to a load balancer that distributes traffic to HyperStore
nodes in the default service region.

13.1.2. IAM Client Application Options

Subjects covered in this section:

l Introduction (immediately below)

l "CMC Support for IAM Functions" (page 1073)
l "Accessing the IAM Service with a Third Party Client Application" (page 1073)
l "Creating S3 Access Credentials for the Default System Admin User" (page 1074)

Users can access and use the HyperStore IAM Service either through the CMC or a third party client applic-
ation that supports IAM calls. Whether using the CMC or a third party client application, application users must
have S3 access credentials (access key ID and secret key) in order to use the HyperStore IAM Service.

13.1.2.1. CMC Support for IAM Functions

Through the CMC, HyperStore account root users can:

l Add, Manage, and Delete IAM Users

l Add, Manage, and Delete IAM Groups
l Add, Manage, and Delete IAM Policies
l Add, Manage, and Delete IAM Roles
l Add, Manage, and Delete SAML Identity Providers

13.1.2.2. Accessing the IAM Service with a Third Party Client Application
Third party or custom client applications can access the HyperStore IAM Service at these service endpoints:

http://iam.<organization-domain>:16080

https://iam.<organization-domain>:16443

HyperStore supports the standard IAM request line formatting, for example:

http://iam.enterprise.com:16080/?Action=<action-name>&<Parameter-name>=<value>

Note that:

l These are the default service endpoints for the HyperStore IAM Service. You can customize the end-
points as described in "Changing S3, Admin, CMC, or IAM Service Endpoints" (page 655).

1073
Chapter 13. IAM API

l The HyperStore IAM Service by default uses a self-signed certificate for its HTTPS listener, so if you are
using HTTPS to access the service your client application must be configured to allow self-signed cer-
tificates. For information about managing SSL certificates in HyperStore -- including the option to import
a CA-signed certificate for the IAM Service to use -- see "HTTPS" (page 145).
l You must configure your DNS environment to resolve the IAM Service endpoint as described in
"DNS Set-Up" in the HyperStore Installation Guide.

13.1.2.3. Creating S3 Access Credentials for the Default System Admin User
If you want the default HyperStore system admin user -- the user whose user ID is "admin" in the CMC -- to be
able to use the IAM Service, do the following:

1. Log into the CMC as the "admin" user. (You will see that an IAM tab now displays in the CMC interface,
but clicking that tab will return an authorization error until after you've completed Steps 2 and 3 below.)
2. On the right side of the CMC's top navigation bar, hold your cursor over your login name ("admin") and
then in the drop-down menu select Security Credentials.
3. In the security credentials page's S3 Access Credentials section, click Create New Key.

This creates S3 access credentials (access key ID and secret key) for the "admin" user. S3 access credentials
are required in order to access HyperStore's IAM Service. The CMC will use these credentials automatically
when the "admin" user uses the CMC to access IAM functions (on the IAM tab). Or if you are using a third party
application to access the HyperStore IAM Service, you will need to provide the credentials to that application.

Note If you created any additional system admin users prior to the HyperStore 7.1 release, and if you
want those system admin users to be able to use the IAM Service, those system admin users will need
to complete the steps described above to create S3 access credentials for themselves.

Regular users and group admins created in the CMC are given S3 credentials automatically as part of
the user creation process, so such users already have the credentials that they need to access the IAM
Service. Also, additional system admins that you create in HyperStore 7.1 or later are automatically
given S3 credentials.

13.1.3. IAM Common Request Parameters

From the "Common Parameters" section of the AWS IAM API specification, HyperStore supports the para-
meters listed below. If a common parameter from that specification section is not listed below, HyperStore does
not support it.

l Action
l Version

Note Unlike Amazon's IAM implementation, in HyperStore's IAM implementation the "Version"
request parameter is not required.

l X-Amz-Algorithm
l X-Amz-Credential
l X-Amz-Date

1074
 13.2. Supported IAM Actions

l X-Amz-Signature
l X-Amz-SignedHeaders

Note Like Amazon's IAM implementation, in HyperStore's IAM implementation you can either use
query parameters or the HTTP header Authorization to submit the authentication data required by the
Signature Version 2 or Signature Version 4 protocol. For more information on this topic see the
Amazon documentation topic Task 4: Add the Signature to the HTTP Request.

13.1.4. IAM Common Errors

From the "Common Errors" section of the AWS IAM API specification, HyperStore supports the errors listed
below. If a common error from that specification section is not listed below, HyperStore does not support it.

l AccessDenied
l IncompleteSignature
l InternalFailure
l InvalidAction
l InvalidClientTokenId
l InvalidParameterCombination
l InvalidParameterValue
l InvalidQueryParameter
l MalformedQueryString
l MissingAction
l MissingAuthenticationToken
l MissingParameter
l OptInRequired
l RequestExpired
l ServiceUnavailable
l ThrottlingException
l ValidationError

13.2. Supported IAM Actions

The HyperStore implementation of the AWS IAM API supports the Actions listed in this section. If an IAM Action
is not listed in this section, HyperStore does not support it. For each Action, the documentation here lists the
request parameters and request or response elements that HyperStore supports. For detailed descriptions of
each Action and its associated parameters and elements, see the AWS documentation links.

Note For all "List" actions (such as "ListAccessKeys", "ListGroups" and so on): The HyperStore
IAM Service does not support truncation. If the client request includes the "MaxItems" and "Marker"
request parameters, the HyperStore IAM Service ignores those parameters. Accordingly, in the
response bodies the "IsTruncated" response element will always be "false".

1075
Chapter 13. IAM API

13.2.1. AddUserToGroup
Adds the specified user to the specified group.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: AddUserToGroup

13.2.1.1. Request Parameters

l GroupName
l UserName

13.2.1.2. Errors
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.2. AttachGroupPolicy
Attaches the specified managed policy to the specified IAM group.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: AttachGroupPolicy

13.2.2.1. Request Parameters

l GroupName
l PolicyArn

13.2.2.2. Errors
l InvalidInput
l LimitExceeded
l NoSuchEntity
l PolicyNotAttachable
l ServiceFailure

13.2.3. AttachRolePolicy
Attaches the specified managed policy to the specified IAM role.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: AttachRolePolicy

1076
 13.2. Supported IAM Actions

13.2.3.1. Request Parameters

l PolicyArn
l RoleName

13.2.3.2. Errors
l InvalidInput
l LimitExceeded
l NoSuchEntity
l PolicyNotAttachable
l ServiceFailure
l UnmodifiableEntity

13.2.4. AttachUserPolicy
Attaches the specified managed policy to the specified user.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: AttachUserPolicy

13.2.4.1. Request Parameters

l PolicyArn
l UserName

13.2.4.2. Errors
l InvalidInput
l LimitExceeded
l NoSuchEntity
l PolicyNotAttachable
l ServiceFailure

13.2.5. CreateAccessKey
Creates a new secret access key and corresponding access key ID for the specified user.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: CreateAccessKey

13.2.5.1. Request Parameters

l UserName

1077
Chapter 13. IAM API

13.2.5.2. Response Elements

l AccessKey

13.2.5.3. Errors
l LimitExceeded
l NoSuchEntity
l ServiceFailure

Note By default the HyperStore system allows only two key pairs per IAM user. This restriction is con-
figurable by the "credentials.iamuser.max" (page 626) setting in mts.properties.erb. Note that an IAM
user's inactive credentials (if any) count toward this limit, as well as active credentials.

13.2.6. CreateGroup
Creates a new group.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: CreateGroup

13.2.6.1. Request Parameters

l GroupName
l Path

13.2.6.2. Response Elements

l Group

Note For HyperStore, within the "Group" object the system-generated "GroupId" attribute value
will be in this format: <Canonical-UID-of-HyperStore-User>|<IAM-groupname>

For example: e97eb4557aea18781f53eb2b8f7e282e|iamgroup2

The canonical user ID is that of the HyperStore user account under which the IAM group is cre-
ated. The IAM group name will be preceded by the path if any is specified when the group is cre-
ated.

Similarly, the "Arn" attribute value will be in this format:

arn:aws:iam::<Canonical-UID-of-HyperStore-User>:group/<IAM-groupname>

13.2.6.3. Errors
l EntityAlreadyExists
l LimitExceeded

1078
 13.2. Supported IAM Actions

l NoSuchEntity
l ServiceFailure

13.2.7. CreatePolicy
Creates a new managed policy under your HyperStore account.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: CreatePolicy

13.2.7.1. Request Parameters

l Description
l Path
l PolicyDocument

Note For information about HyperStore's IAM policy document support see "Supported IAM
Policy Elements" (page 1105).

l PolicyName

13.2.7.2. Response Elements

l Policy

13.2.7.3. Errors
l EntityAlreadyExists
l InvalidInput
l LimitExceeded
l MalformedPolicyDocument
l ServiceFailure

13.2.8. CreatePolicyVersion
Creates a new version of the specified managed policy.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: CreatePolicyVersion

13.2.8.1. Request Parameters

l PolicyArn
l PolicyDocument

1079
Chapter 13. IAM API

Note For information about HyperStore's IAM policy document support see "Supported IAM
Policy Elements" (page 1105).

l SetAsDefault

13.2.8.2. Response Elements

l PolicyVersion
o CreateDate
o Document
o IsDefaultVersion
o VersionId

13.2.8.3. Errors
l InvalidInput
l LimitExceeded
l MalformedPolicyDocument
l NoSuchEntity
l ServiceFailure

13.2.9. CreateRole
Creates a new role under your account.

HyperStore supports the parameters, parameters, and errors listed below.

For action details and examples see the AWS documentation: CreateRole

See also the AWS documentation: IAM Roles

13.2.9.1. Request Parameters

l AssumeRolePolicyDocument (also known as the "trust policy")
l Description
l MaxSessionDuration
l Path
l PermissionsBoundary
l RoleName

Example AssumeRolePolicyDocument (trust policy) for a federated/SAML principal:

{
"Version": "2012-10-17",
"Statement": {
"Effect": "Allow",

1080
 13.2. Supported IAM Actions

"Action": "sts:AssumeRoleWithSAML",
"Principal": {"Federated": "arn:aws:iam::123456789012:saml-provider/adfs"},
"Condition": {"StringEquals": {"saml:aud": "https://cmc.mycloudianhyperstore.com/saml"}}
}
}

The example policy above says to trust SAML assertions from the "adfs" SAML Provider to use STS:As-
sumeRoleWithSAML for this role but only if the SAML assertion contains the recipient string matching https://cm-
c.mycloudianhyperstore.com/saml.

For Condition in a trust policy, HyperStore supports only the StringEquals condition operator and only the fol-
lowing condition keys:

l aws:TokenIssueTime
l sts:ExternalId
l saml:aud
l saml:doc
l saml:iss
l saml:namequalifier
l saml:sub
l saml:sub_type

13.2.9.2. Response Elements

l Role

13.2.9.3. Errors
l ConcurrentModification
l EntityAlreadyExists
l InvalidInput
l LimitExceeded
l MalformedPolicyDocument
l ServiceFailure

13.2.10. CreateSAMLProvider
Creates an IAM resource that describes an identity provider (IdP) that supports SAML 2.0.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: CreateSAMLProvider

13.2.10.1. Request Parameters

l Name
l SAMLMetadataDocument

1081
Chapter 13. IAM API

13.2.10.2. Response Elements

l SAMLProviderArn

13.2.10.3. Errors
l EntityAlreadyExists
l LimitExceeded
l ServiceFailure

13.2.11. CreateUser
Creates a new IAM user under your account.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: CreateUser

13.2.11.1. Request Parameters

l Path
l UserName

13.2.11.2. Response Elements

l User

Note For HyperStore, within the "User" object the system-generated "UserId" attribute value will
be in this format: <Canonical-UID-of-HyperStore-User>|<IAM-username>

For example: e97eb4557aea18781f53eb2b8f7e282e|iamuser2

The canonical user ID is that of the HyperStore user account under which the IAM user is cre-
ated. The IAM user name will be preceded by the path if any is specified when the user is cre-
ated.

Similarly, the "Arn" attribute value will be in this format:

arn:aws:iam::<Canonical-UID-of-HyperStore-User>:user/<IAM-username>

13.2.11.3. Errors
l EntityAlreadyExists
l LimitExceeded
l NoSuchEntity
l ServiceFailure

1082
 13.2. Supported IAM Actions

Note IAM users that you create under your HyperStore user account will not be allowed to log into the
CMC or to use the CMC as their S3 client application. IAM users will need to use an S3 client applic-
ation other than the CMC to access the HyperStore S3 Service.

13.2.12. DeleteAccessKey
Deletes the access key pair associated with the specified IAM user.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteAccessKey

13.2.12.1. Request Parameters

l AccessKeyId
l UserName

13.2.12.2. Errors
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.13. DeleteGroup
Deletes the specified IAM group.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteGroup

13.2.13.1. Request Parameters

l GroupName

13.2.13.2. Errors
l DeleteConflict
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.14. DeleteGroupPolicy
Deletes the specified inline policy that is embedded in the specified IAM group.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteGroupPolicy

1083
Chapter 13. IAM API

13.2.14.1. Request Parameters

l GroupName
l PolicyName

13.2.14.2. Errors
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.15. DeletePolicy
Deletes the specified managed policy.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeletePolicy

13.2.15.1. Request Parameters

l PolicyArn

13.2.15.2. Errors
l DeleteConflict
l InvalidInput
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.16. DeletePolicyVersion
Deletes the specified version from the specified managed policy.

Note You cannot delete the default version of a policy using this operation. To delete the default ver-
sion a policy, use "DeletePolicy" (page 1084).

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeletePolicyVersion

13.2.16.1. Request Parameters

l PolicyArn
l VersionId

1084
 13.2. Supported IAM Actions

13.2.16.2. Errors
l DeleteConflict
l InvalidInput
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.17. DeleteRole
Deletes the specified role.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteRole

13.2.17.1. Request Parameters

l RoleName

13.2.17.2. Errors
l ConcurrentModification
l DeleteConflict
l LimitExceeded
l NoSuchEntity
l ServiceFailure
l UnmodifiableEntity

13.2.18. DeleteRolePolicy
Deletes the specified inline policy that is embedded in the specified IAM role.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteRolePolicy

13.2.18.1. Request Parameters

l PolicyName
l RoleName

13.2.18.2. Errors
l LimitExceeded
l NoSuchEntity

1085
Chapter 13. IAM API

l ServiceFailure
l UnmodifiableEntity

13.2.19. DeleteSAMLProvider
Deletes a SAML provider resource in IAM.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteSAMLProvider

13.2.19.1. Request Parameters

l SAMLProviderArn

13.2.19.2. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.20. DeleteUser
Deletes the specified IAM user.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteUser

13.2.20.1. Request Parameters

l UserName

13.2.20.2. Errors
l DeleteConflict
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.21. DeleteUserPolicy
Deletes the specified inline policy that is embedded in the specified IAM user.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteUserPolicy

1086
 13.2. Supported IAM Actions

13.2.21.1. Request Parameters

l PolicyName
l UserName

13.2.21.2. Errors
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.22. DetachGroupPolicy
Removes the specified managed policy from the specified IAM group.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DetachGroupPolicy

13.2.22.1. Request Parameters

l GroupName
l PolicyArn

13.2.22.2. Errors
l InvalidInput
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.23. DetachRolePolicy
Removes the specified managed policy from the specified role.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DetachRolePolicy

13.2.23.1. Request Parameters

l PolicyArn
l RoleName

13.2.23.2. Errors
l InvalidInput
l LimitExceeded

1087
Chapter 13. IAM API

l NoSuchEntity
l ServiceFailure
l UnmodifiableEntity

13.2.24. DetachUserPolicy
Removes the specified managed policy from the specified user.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DetachUserPolicy

13.2.24.1. Request Parameters

l PolicyArn
l UserName

13.2.24.2. Errors
l InvalidInput
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.25. GetGroup
Returns a list of IAM users that are in the specified IAM group.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetGroup

13.2.25.1. Request Parameters

l GroupName

Note The "Marker" and "MaxItems" request parameters, if submitted, are ignored.

13.2.25.2. Response Elements

l Group

Note For HyperStore, within the "Group" object the system-generated "GroupId" attribute value
will be in this format: <Canonical-UID-of-HyperStore-User>|<IAM-groupname>

For example: e97eb4557aea18781f53eb2b8f7e282e|iamgroup2

The canonical user ID is that of the HyperStore user account under which the IAM group was

1088
 13.2. Supported IAM Actions

created. The IAM group name will be preceded by the path if any was specified when the group
was created.

Similarly, the "Arn" attribute value will be in this format:

arn:aws:iam::<Canonical-UID-of-HyperStore-User>:group/<IAM-groupname>

l IsTruncated

Note "IsTruncated" will always be "false".

l Users.member.N

13.2.25.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.26. GetGroupPolicy
Retrieves the specified inline policy document that is embedded in the specified IAM group.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetGroupPolicy

13.2.26.1. Request Parameters

l GroupName
l PolicyName

13.2.26.2. Response Elements

l GroupName
l PolicyDocument
l PolicyName

13.2.26.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.27. GetPolicy
Retrieves information about the specified managed policy, including the policy's default version and the total
number of IAM users, groups, and roles to which the policy is attached.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: GetPolicy

1089
Chapter 13. IAM API

13.2.27.1. Request Parameters

l PolicyArn

13.2.27.2. Response Elements

l Policy

13.2.27.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.28. GetPolicyVersion
Retrieves information about the specified version of the specified managed policy, including the policy doc-
ument.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetPolicyVersion

13.2.28.1. Request Parameters

l PolicyArn
l VersionId

13.2.28.2. Response Elements

l PolicyVersion

13.2.28.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.29. GetRole
Retrieves information about the specified role, including the role's path, GUID, ARN, and the role's trust policy
that grants permission to assume the role.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetRole

1090
 13.2. Supported IAM Actions

13.2.29.1. Request Parameters

l RoleName

13.2.29.2. Response Elements

l Role

13.2.29.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.30. GetRolePolicy
Retrieves the specified inline policy document that is embedded with the specified IAM role.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetRolePolicy

13.2.30.1. Request Parameters

l PolicyName
l RoleName

13.2.30.2. Response Elements

l PolicyDocument
l PolicyName
l RoleName

13.2.30.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.31. GetSAMLProvider
Returns the SAML provider metadata document that was uploaded when the IAM SAML provider resource
object was created or updated.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: GetSAMLProvider

13.2.31.1. Request Parameters

l SAMLProviderArn

1091
Chapter 13. IAM API

13.2.31.2. Response Elements

l CreateDate
l SAMLMetadataDocument
l ValidUntil

13.2.31.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.32. GetUser
Retrieves information about the specified IAM user, including the user's creation date, path, unique ID, and
ARN.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetUser

13.2.32.1. Request Parameters

l UserName

13.2.32.2. Response Elements

l User

Note For HyperStore, within the "User" object the system-generated "UserId" attribute value will
be in this format: <Canonical-UID-of-HyperStore-User>|<IAM-username>

For example: e97eb4557aea18781f53eb2b8f7e282e|iamuser2

The canonical user ID is that of the HyperStore user account under which the IAM user was cre-
ated.The IAM user name will be preceded by the path if any was specified when the user was
created.

Similarly, the "Arn" attribute value will be in this format:

arn:aws:iam::<Canonical-UID-of-HyperStore-User>:user/<IAM-username>

13.2.32.3. Errors
l NoSuchEntity
l ServiceFailure

1092
 13.2. Supported IAM Actions

13.2.33. GetUserPolicy
Retrieves the specified inline policy document that is embedded in the specified IAM user.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetUserPolicy

13.2.33.1. Request Parameters

l PolicyName
l UserName

13.2.33.2. Response Elements

l PolicyDocument
l PolicyName
l UserName

13.2.33.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.34. ListAccessKeys
Returns information about the access key IDs associated with the specified IAM user.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListAccessKeys

13.2.34.1. Request Parameters

l UserName

13.2.34.2. Response Elements

l AccessKeyMetadata.member.N
l IsTruncated

13.2.34.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.35. ListAttachedGroupPolicies
Lists all managed policies that are attached to the specified IAM group.

1093
Chapter 13. IAM API

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListAttachedGroupPolicies

13.2.35.1. Request Parameters

l GroupName
l PathPrefix

13.2.35.2. Response Elements

l AttachedPolicies.member.N
l IsTruncated

13.2.35.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.36. ListAttachedRolePolicies
Lists all managed policies that are attached to the specified IAM role.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListAttachedRolePolicies

13.2.36.1. Request Parameters

l PathPrefix
l UserName

13.2.36.2. Response Elements

l AttachedPolicies.member.N
l IsTruncated

13.2.36.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.37. ListAttachedUserPolicies
Lists all managed policies that are attached to the specified IAM user.

HyperStore supports the parameters, elements, and errors listed below.

1094
 13.2. Supported IAM Actions

For action details and examples see the AWS documentation: ListAttachedUserPolicies

13.2.37.1. Request Parameters

l PathPrefix
l UserName

13.2.37.2. Response Elements

l AttachedPolicies.member.N
l IsTruncated

13.2.37.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.38. ListEntitiesForPolicy
Lists all IAM users, groups, and roles that the specified managed policy is attached to.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListEntitiesForPolicy

13.2.38.1. Request Parameters

l EntityFilter
l PathPrefix
l PolicyArn

13.2.38.2. Response Elements

l IsTruncated

l PolicyGroups.member.N
l PolicyUsers.member.N

13.2.38.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.39. ListGroupPolicies
Lists the names of the inline policies that are embedded in the specified IAM group.

1095
Chapter 13. IAM API

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListGroupPolicies

13.2.39.1. Request Parameters

l GroupName

13.2.39.2. Response Elements

l IsTruncated

l PolicyNames.member.N

13.2.39.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.40. ListGroups
Lists the IAM groups that have the specified path prefix.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListGroups

13.2.40.1. Request Parameters

l PathPrefix

13.2.40.2. Response Elements

l Groups.member.N
l IsTruncated

13.2.40.3. Errors
l ServiceFailure

13.2.41. ListGroupsForUser
Lists the IAM groups that the specified IAM user belongs to.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListGroupsForUser

Request Parameters

l UserName

Response Elements

1096
 13.2. Supported IAM Actions

l Groups.member.N
l IsTruncated

Errors

l NoSuchEntity
l ServiceFailure

13.2.42. ListPolicies
Lists all the managed policies that are available under your account.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListPolicies

13.2.42.1. Request Parameters

l OnlyAttached
l PathPrefix

Note The "Scope" request parameter, if submitted, is ignored and defaults to All. Note however that
only Local policies are currently supported in HyperStore, so the policies returned by this command will
all be Local policies.

13.2.42.2. Response Elements

l IsTruncated

l Policies.member.N

13.2.42.3. Errors
l ServiceFailure

13.2.43. ListPolicyVersions
Lists information about the versions of the specified managed policy, including the version that is currently set
as the policy's default version.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListPolicyVersions

13.2.43.1. Request Parameters

l PolicyArn

13.2.43.2. Response Elements

l IsTruncated

1097
Chapter 13. IAM API

l Versions.member.N

13.2.43.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.44. ListRolePolicies
Lists the names of the inline policies that are embedded in the specified IAM role.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListRolePolicies

13.2.44.1. Request Parameters

l RoleName

13.2.44.2. Response Elements

l IsTruncated

l PolicyNames.member.N

13.2.44.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.45. ListRoles
Lists the IAM roles that have the specified path prefix.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListRoles

13.2.45.1. Request Parameters

l PathPrefix

13.2.45.2. Response Elements

l IsTruncated

l Roles.member.N

1098
 13.2. Supported IAM Actions

13.2.45.3. Errors
l ServiceFailure

13.2.46. ListSAMLProviders
Lists the SAML provider resource objects defined in IAM in the account.

HyperStore supports the elements and errors listed below.

For action details and examples see the AWS documentation: ListSAMLProviders

13.2.46.1. Response Elements

l SAMLProviderList.member.N

13.2.46.2. Errors
l ServiceFailure

13.2.47. ListUserPolicies
Lists the names of the inline policies embedded in the specified IAM user.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListUserPolicies

13.2.47.1. Request Parameters

l UserName

13.2.47.2. Response Elements

l IsTruncated

l PolicyNames.member.N

13.2.47.3. Errors
l NoSuchEntity
l ServiceFailure

13.2.48. ListUsers
Lists the IAM users that have the specified path prefix.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ListUsers

1099
Chapter 13. IAM API

13.2.48.1. Request Parameters

l PathPrefix

13.2.48.2. Response Elements

l IsTruncated

l Users.member.N

13.2.48.3. Errors
l ServiceFailure

13.2.49. PutGroupPolicy
Adds or updates an inline policy document that is embedded in the specified IAM group.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: PutGroupPolicy

13.2.49.1. Request Parameters

l GroupName
l PolicyDocument

Note For information about HyperStore's IAM policy document support see "Supported IAM
Policy Elements" (page 1105).

l PolicyName

13.2.49.2. Errors
l LimitExceeded
l MalformedPolicyDocument
l NoSuchEntity
l ServiceFailure

13.2.50. PutRolePolicy
Adds or updates an inline policy document that is embedded in the specified IAM role.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: PutRolePolicy

1100
 13.2. Supported IAM Actions

13.2.50.1. Request Parameters

l PolicyDocument
l PolicyName
l RoleName

13.2.50.2. Errors
l LimitExceeded
l MalformedPolicyDocument
l NoSuchEntity
l ServiceFailure
l UnmodifiableEntity

13.2.51. PutUserPolicy
Adds or updates an inline policy document that is embedded in the specified IAM user.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: PutUserPolicy

13.2.51.1. Request Parameters

l PolicyDocument

Note For information about HyperStore's IAM policy document support see "Supported IAM
Policy Elements" (page 1105).

l PolicyName
l UserName

13.2.51.2. Errors
l LimitExceeded
l MalformedPolicyDocument
l NoSuchEntity
l ServiceFailure

13.2.52. RemoveUserFromGroup
Removes the specified user from the specified group.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: RemoveUserFromGroup

1101
Chapter 13. IAM API

13.2.52.1. Request Parameters

l GroupName
l UserName

13.2.52.2. Errors
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.53. SetPolicyDefaultVersion
Sets the specified version of the specified policy as the policy's default (operative) version.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: SetPolicyDefaultVersion

13.2.53.1. Request Parameters

l PolicyArn
l VersionId

13.2.53.2. Errors
l InvalidInput
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.54. UpdateAccessKey
Changes the status of the specified access key from Active to Inactive, or vice versa.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: UpdateAccessKey

13.2.54.1. Request Parameters

l AccessKeyId
l Status
l UserName

1102
 13.2. Supported IAM Actions

13.2.54.2. Errors
l LimitExceeded
l NoSuchEntity
l ServiceFailure

Note By default the HyperStore system allows only two key pairs per IAM user. This restriction is con-
figurable by the "credentials.iamuser.max" (page 626) setting in mts.properties.erb. Note that an IAM
user's inactive credentials (if any) count toward this limit, as well as active credentials.

13.2.55. UpdateAssumeRolePolicy
Updates the policy that grants an IAM entity permission to assume a role.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: UpdateAssumeRolePolicy

13.2.55.1. Request Parameters

l PolicyDocument
l RoleName

Note For Conditions in a trust policy, HyperStore supports only the StringEquals condition operator
and only the following condition keys:
aws:TokenIssueTime
sts:ExternalId
saml:aud
saml:doc
saml:iss
saml:namequalifier
saml:sub
saml:sub_type

13.2.55.2. Errors
l LimitExceeded
l MalformedPolicyDocument
l NoSuchEntity
l ServiceFailure
l UnmodifiableEntity

13.2.56. UpdateGroup
Updates the name and/or the path of the specified IAM group.

HyperStore supports the parameters and errors listed below.

1103
Chapter 13. IAM API

For action details and examples see the AWS documentation: UpdateGroup

13.2.56.1. Request Parameters

l GroupName
l NewGroupName
l NewPath

13.2.56.2. Errors
l EntityAlreadyExists
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.2.57. UpdateRole
Updates the description or maximum session duration setting of a role.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: UpdateRole

13.2.57.1. Request Parameters

l Description
l MaxSessionDuration
l RoleName

13.2.57.2. Errors
l NoSuchEntity
l ServiceFailure
l UnmodifiableEntity

13.2.58. UpdateRoleDescription
Although HyperStore supports this Action, it is recommended to use UpdateRole instead.

AWS documentation: UpdateRoleDescription

13.2.59. UpdateSAMLProvider
Updates the metadata document for an existing SAML provider resource object.

HyperStore supports the parameters, response elements, and errors listed below.

For action details and examples see the AWS documentation: UpdateSAMLProvider

1104
 13.3. Supported IAM Policy Elements

13.2.59.1. Request Parameters

l SAMLMetadataDocument
l SAMLProviderArn

13.2.59.2. Response Elements

l SAMLProviderArn

13.2.59.3. Errors
l InvalidInput
l NoSuchEntity
l ServiceFailure

13.2.60. UpdateUser
Updates the name and/or the path of the specified IAM user.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: UpdateUser

13.2.60.1. Request Parameters

l NewPath
l NewUserName
l UserName

13.2.60.2. Errors
l EntityAlreadyExists
l EntityTemporarilyUnmodifiable
l LimitExceeded
l NoSuchEntity
l ServiceFailure

13.3. Supported IAM Policy Elements

IAM policies grant permissions to IAM groups and users. You can create IAM policy documents through the
CMC or by using a third party or custom IAM client. If you use the CMC, you have the choice of using a visual
policy editor or using a JSON editor.

HyperStore supports AWS standard IAM policy formatting and most policy elements for granting S3 or IAM ser-
vice permissions.

1105
Chapter 13. IAM API

For guidance on how to construct IAM policies for S3 service permissions or IAM service permissions, see the
AWS documentation on this topic. For example:

l Policies and Permissions

http://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html

l IAM JSON Policy Elements Reference

https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements.html

l Actions, Resources, and Condition Keys for Amazon S3

https://docs.aws.amazon.com/IAM/latest/UserGuide/list_amazons3.html

l Actions, Resources, and Condition Keys for Identity And Access Management

https://docs.aws.amazon.com/IAM/latest/UserGuide/list_identityandaccessmanagement.html

Below is an example of a simple IAM policy document granting permission to list the contents of a bucket
named "bucket1":

{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::bucket1"
}]
}

Note HyperStore supports most but not all of the S3 Actions and IAM Actions cited in the AWS doc-
umentation for IAM policy formation. In general, when constructing IAM policies you can use all the
Actions that correspond to operations supported by the HyperStore S3 Service and the HyperStore IAM
Service. You can check the HyperStore S3 API documentation and HyperStore IAM API documentation
if you are unsure whether a particular operation is supported. Alternatively you can check the CMC
interface for creating an IAM policy -- the interface lists all the supported S3 actions and IAM actions.

Note Actions in IAM policies are case sensitive, so be sure to exactly match the desired Action name
as it appears in the AWS documentation.

13.4. SAML Support

HyperStore supports Security Assertion Markup Language (SAML 2.0) based access to S3 storage resources.
It does so by supporting the standard AWS IAM Service calls and Security Token Service calls that are needed
to set up and execute SAML based access. With SAML, federated users -- users who have been authenticated
by a trusted identity provider system (IdP) external to HyperStore -- can be granted temporary access to S3
resources, subject to policy-based permission restrictions.

At a high level, the process of setting up and using SAML access for HyperStore works as described below.

l "Downloading the HyperStore SAML Metadata Document for IdP Setup" (page 1107)
l "Using the IAM Service to Create SAML Provider Resources" (page 1107)

1106
 13.4. SAML Support

l "Using the IAM Service to Create Roles" (page 1108)

l "Using the STS Service to Assume a Role" (page 1109)

13.4.1. Downloading the HyperStore SAML Metadata Document for IdP

Setup
For each external identity provider system (IdP) that you expect to be a source of SAML assertions submitted to
HyperStore , you must load the HyperStore SAML Service Provider Metadata document into the IdP. This
metadata document is specific to your HyperStore system, and provides the IdP with information about how to
submit SAML assertions to HyperStore . The procedure for applying the SAML Service Provider metadata doc-
ument into the IdP will depend on the IdP that you are working with (refer to your IdP's documentation), but
regardless of those particulars you can download the document from the CMC at this URL:

https://<cmc FQDN>:<cmc port>/static/saml-metadata.xml

For example:

https://cmc.enterprise.com:8443/static/saml-metadata.xml

If you have configured your load balancers so that external access to the CMC is through a different port
number than the CMC is listening on internally (which is 8443 by default), then before downloading the Hyper-
Store SAML Provider Metadata document run the following commands on your Configuration Master node:

hsctl config set cmc.ports.loadBalancer.https=<load balancer port for CMC>

hsctl config apply saml

For example:

hsctl config set cmc.ports.loadBalancer.https=443

hsctl config apply saml

This will result in the correct CMC external access port being specified within the Service Provider Metadata
document.

13.4.2. Using the IAM Service to Create SAML Provider Resources

HyperStore's IAM Service supports all the calls that you need to create and manage SAML provider resources
within the IAM Service. A SAML provider resource describes an IdP that will be a source of SAML assertions
submitted to HyperStore on behalf of federated users who have been authenticated by the IdP.

You can create SAML provider resources either by using the CMC's Manage Identity Providers page, or by
using a third party IAM client to access the HyperStore IAM Service. If you are using a third party IAM client, the
relevant IAM calls are:

l CreateSAMLProvider
l ListSAMLProviders
l GetSAMLProvider
l UpdateSAMLProvider
l DeleteSAMLProvider

You should create a SAML provider resource for each IdP that will be a trusted source of incoming SAML asser-
tions.

1107
Chapter 13. IAM API

Note S3 credentials are needed to access the HyperStore IAM Service (and the CMC's IAM functions),
and the CMC's system administrative user named "admin" does not have these credentials by default.
For information on creating S3 credentials for the "admin" user so that this user can access the
IAM Service, see "S3 Access Credentials Are Needed to Access the IAM Service" (page 1071).

13.4.3. Using the IAM Service to Create Roles

HyperStore's IAM Service supports all the calls that you need to create and manage IAM roles. As part of cre-
ating an IAM role you define a "trust policy" that specifies who will be allowed to assume that role. To facilitate
SAML based access to HyperStore , you specify one or more SAML providers as the principal within an IAM
role's trust policy, as you create the role. Once an IAM role is created, you then specify the S3 permissions gran-
ted to that role, by either attaching a managed permissions policy to the role or creating an inline permission
policy specific to that role.

You can create and manage IAM roles either by using the CMC's Manage Roles page, or by using a third
party IAM client. If you are using a third party IAM client to access the HyperStore IAM Service, the relevant
IAM calls are:

l CreateRole
l ListRoles
l GetRole
l UpdateRole
l UpdateAssumRolePolicy
l DeleteRole
l AttachRolePolicy
l ListAttachedRolePolicies
l DetachRolePolicy
l PutRolePolicy
l ListRolePolicies
l GetRolePolicy
l DeleteRolePolicy

Note S3 credentials are needed to access the HyperStore IAM Service (and the CMC's IAM functions),
and the CMC's system administrative user named "admin" does not have these credentials by default.
For information on creating S3 credentials for the "admin" user so that this user can access the
IAM Service, see "S3 Access Credentials Are Needed to Access the IAM Service" (page 1071).

13.4.3.1. Limitations
l Role session policies and role tags are not supported.
l Support for Conditions in trust policies is limited; see CreateRole.

1108
 13.4. SAML Support

13.4.4. Using the STS Service to Assume a Role

Once an IdP has been configured with HyperStore's SAML metadata XML document, and you have created a
SAML provider IAM resource for that IdP and specified that provider as part of an IAM role's trust policy, SAML
assertions from that provider can be used to allow federated users to assume that role by calling the Hyper-
Store Security Token Service's AssumeRoleWithSAML API call. This API call initiates a role session during
which properly authenticated and authorized users are provided with temporary S3 security credentials.

There are two options for using this API call to assume a role:

l A third party STS client application can be used to submit an AssumeRoleWithSAML API call to Hyper-
Store's STS Service. For more information on accessing this service see "HyperStore Support for the
AWS STS API" (page 1111).
l The CMC hosts a single-sign-on (SSO) page to which an IdP can submit a SAML assertion on behalf of
a user who has successfully logged into the IdP. The IdP can submit an HTTP POST to the Location
URL identified within the AssertionConsumerService attribute of the HyperStore SAML Metadata doc-
ument. Based on the submitted SAML assertion contents, the CMC will display a list of Roles for which
the user identified in the assertion is eligible. The user can select a Role from that list, and the CMC
will then submit an AssumeRoleWithSAML request for that Role to the HyperStore STS Service. The
CMC then makes the returned temporary security credentials available for the user to copy to their clip-
board, and the user can then paste the temporary credentials into an S3 application and perform the S3
operations permitted by the Role.

1109
Chapter 13. IAM API

13.4.4.1. Limitations
Users with temporary security credentials obtained from the HyperStore STS Service:

l Cannot can log into the CMC

l Cannot access the IAM Service (even with a third party client application)

What users with temporary security credentials can do is access the HyperStore S3 Service by using a third
party S3 client application. Their S3 permissions will be limited to those permissions ascribed to the role that
they have assumed.

1110
Chapter 14. STS API

14.1. Introduction

14.1.1. HyperStore Support for the AWS STS API

To facilitate Security Assertion Markup Language (SAML) based access to HyperStore S3 services, Hyper-
Store provides limited support for the Amazon Web Services Security Token Service (STS) API:

l Only a few STS Actions are supported -- for details see "Supported STS Actions".
l HyperStore does not allow users with temporary security credentials (obtained through the STS Ser-
vice) to perform IAM operations. The HyperStore IAM Service will reject requests that contain tem-
porary credentials. Users with temporary credentials can only access the S3 Service (within the
permission restrictions of the roles that such users assume).
l Users with temporary security credentials are not allowed to log into the CMC.

The default STS Service endpoint URLs for HTTP and HTTPS are:

l http://sts.<organization-domain>:16080
l https://sts.<organization-domain>:16443

Note Be sure to configure the STS endpoint domain in your DNS environment.

Note The STS Service uses the same listening ports as the IAM Service.

14.1.2. STS Common Request Parameters

From the "Common Parameters" section of the AWS STS API specification, HyperStore supports the para-
meters listed below. If a common parameter from that specification section is not listed below, HyperStore does
not support it.

l Action
l Version
l X-Amz-Algorithm
l X-Amz-Credential
l X-Amz-Date
l X-Amz-Security-Token (only supported for GetCallerIdentity requests)
l X-Amz-Signature
l X-Amz-SignedHeaders

14.1.3. STS Common Errors

From the "Common Errors" section of the AWS STS API specification, HyperStore supports the parameters
listed below. If a common parameter from that specification section is not listed below, HyperStore does not

1111
Chapter 14. STS API

support it.

l AccessDeniedException
l InternalFailure
l InvalidAction
l InvalidClientTokenId
l InvalidParameterCombination
l InvalidParameterValue
l MissingAuthenticationToken
l MissingParameter
l ServiceUnavailable
l ValidationError

14.2. Supported STS Actions

The HyperStore implementation of the AWS STS API supports the Actions listed in this section. If an STS Action
is not listed in this section, HyperStore does not support it. For each Action, the documentation here lists the
request parameters and request or response elements that HyperStore supports. For detailed descriptions of
each Action and its associated parameters and elements, see the AWS documentation links.

14.2.1. AssumeRole
Returns a set of temporary security credentials that you can use to access S3 resources that you might not nor-
mally have access to.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: AssumeRole

Note HyperStore does not allow users with temporary security credentials to perform IAM operations.

14.2.1.1. Request Parameters

l DurationSeconds
l ExternalId
l RoleArn
l RoleSessionName

14.2.1.2. Response Elements

l AssumedRoleUser
l Credentials

1112
 14.2. Supported STS Actions

14.2.1.3. Errors
l InvalidParameterValue
l NoSuchEntity

14.2.2. AssumeRoleWithSAML
Returns a set of temporary security credentials for users who have been authenticated via a SAML authen-
tication response.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: AssumeRoleWithSAML

Note For an overview of HyperStore support for SAML see "SAML Support" (page 1106). For more
information about using the STS AssumeRoleWithSAML call with HyperStore see "Using the STS Ser-
vice to Assume a Role" (page 1109).

14.2.2.1. Request Parameters

l DurationSeconds
l PrincipalArn
l RoleArn
l SAMLAssertion

14.2.2.2. Response Elements

l AssumedRoleUser
l Audience
l Credentials
l Issuer
l NameQualifier
l Subject
l SubjectType

14.2.2.3. Errors
l ExpiredToken
l InvalidClientTokenId
l InvalidParameterValue

14.2.3. GetCallerIdentity
Returns details about the IAM user or role whose credentials are used to call the operation.

HyperStore supports the elements listed below.

1113
Chapter 14. STS API

For action details and examples see the AWS documentation: GetCallerIdentity

14.2.3.1. Response Elements

l Account
l Arn
l UserId

14.2.3.2. Errors
l ExpiredToken
l InvalidClientTokenId

1114
Chapter 15. SQS API

15.1. Introduction

15.1.1. HyperStore Support for the AWS SQS API

In support of the bucket notification feature, HyperStore provides limited support for the Amazon Web Services
Simple Queue Service (SQS) API. The queueing and processing of messages is implemented within the Hyper-
Store system. Bucket owners can use the S3 API operation PutBucketNotificationConfiguration to configure
bucket notification so that when specified S3 operations occur within the bucket -- such as objects being
uploaded to the bucket or deleted from the bucket -- HyperStore publishes a notification message to a spe-
cified SQS queue.

In the current HyperStore release, there are these limitations to the bucket notification feature and the SQS Ser-
vice:

l A third party SQS client application must be used to interface with the HyperStore SQS Service to per-
form operations such as creating and configuring queues and receiving and deleting queued mes-
sages. The CMC does not yet support SQS operations.
l A third party S3 client application must be used to execute the PutBucketNotificationConfiguraiton
operation. The CMC does not yet support this S3 operation.
l For bucket notifications to an SQS queue to work, the bucket owner must also be the owner of the
SQS queue.
l HTTPS access to the SQS Service is not supported. Only regular HTTP access is supported.
l The HyperStore SQS Service supports many of the Actions from the Amazon SQS API, but not all of
them. For more detail see "SQS Supported Actions" (page 1116).

15.1.1.1. Enabling and Using the SQS Service and Bucket Notification
HyperStore's bucket notification feature and its SQS Service are disabled by default. To enable the notification
feature and the SQS Service:

1. In common.csv:

l Set sqs_enabled to true

l Set sqs_endpoint to an SQS service endpoint for your domain (the recommended endpoint
format is s3-sqs.<organization-domain>)
l Optionally set sqs_port, if you want an SQS listening port other than the default which is 18090

2. In mts.properties.erb:
o Edit the cloudian.s3.unsupported property to remove notification from the list of unsupported S3
request types. Be sure to delete the preceding comma as well.

Before your edit:

cloudian.s3.unsupported=accelerate,requestPayment,analytics,inventory,metrics,select,no
tification

After your edit:

1115
Chapter 15. SQS API

cloudian.s3.unsupported=accelerate,requestPayment,analytics,inventory,metrics,select

o Below the cloudian.s3.unsupported property, add this new property to the file (it is not in the file
by default):
cloudian.s3.bucketnotification=true

3. Use the installer to push the configuration changes to the cluster and restart the S3 Service and the
SQS Service.

Once you have enabled the bucket notification feature and the SQS Service, then:

l A third party SQS client application can be used to submit requests to the HyperStore SQS Service,
such as for creating and configuring a queue. For HyperStore support of SQS Actions see "SQS Sup-
ported Actions" (page 1116). The default SQS Service endpoint URL including the port number is
http://s3-sqs.<organization-domain>:18090
l A third party S3 client application can be used to submit a PutBucketNotificationConfiguration request
to the HyperStore S3 Service, to configure notifications for an existing bucket. For HyperStore support of
this S3 API method see PutBucketNotificationConfiguration. As noted previously, the bucket owner
must also be the SQS queue owner.

Note Information about requests processed by the SQS Service are logged to cloudian-sqs-request.-
log, which exists on each node. For more information see "S3 Service Logs (including Auto-Tiering,
CRR, and WORM)" (page 676).

15.2. SQS Supported Actions

The HyperStore implementation of the AWS SQS API supports the Actions listed in this section. If an SQS
Action is not listed in this section, HyperStore does not support it. For each Action, the documentation here lists
the request parameters and request or response elements that HyperStore supports. For detailed descriptions
of each Action and its associated parameters and elements, see the AWS documentation links.

Note The HyperStore SQS Service is disabled by default. For information about enabling the service
see "Enabling and Using the SQS Service and Bucket Notification" (page 1115).

15.2.1. ChangeMessageVisibility
Changes the visibility timeout of a specified message in a queue to a new value.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: ChangeMessageVisibility

15.2.1.1. Request Parameters

l QueueUrl
l ReceiptHandle

1116
 15.2. SQS Supported Actions

l VisibilityTimeout

15.2.1.2. Errors
l AWS.SimpleQueueService.MessageNotInflight
l ReceiptHandleIsInvalid

15.2.2. CreateQueue
Creates a new standard queue.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: CreateQueue

Note HyperStore currently only supports Standard queues. HyperStore does not support FIFO queues.

15.2.2.1. Request Parameters

l Attribute

HyperStore currently only supports these queue attributes:

o DelaySeconds
o MaximumMessageSize
o MessageRetentionPeriod
o ReceiveMessageWaitTimeSeconds
o VisibilityTimeout

Any other attributes included in the CreateQueue request will be ignored.

l QueueName
l Tag

15.2.2.2. Response Elements

l QueueUrl

15.2.2.3. Errors
l AWS.SimpleQueueService.QueueDeletedRecently
l QueueAlreadyExists

15.2.3. DeleteMessage
Deletes the specified message from the specified queue.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: DeleteMessage

1117
Chapter 15. SQS API

15.2.3.1. Request Parameters

l QueueUrl
l ReceiptHandle

15.2.3.2. Errors
l InvalidIdFormat
l ReceiptHandleIsInvalid

15.2.4. DeleteQueue
Deletes the queue specified by the QueueUrl, regardless of the queue's contents.

HyperStore supports the parameters listed below.

For action details and examples see the AWS documentation: DeleteQueue

15.2.4.1. Request Parameters

l QueueUrl

15.2.5. GetQueueAttributes
Gets attributes for the specified queue.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetQueueAttributes

15.2.5.1. Request Parameters

l AttributeName.N

Note For the list of queue attributes that HyperStore supports, see "CreateQueue" (page 1117).

l QueueUrl

15.2.5.2. Response Elements

l Attribute

15.2.5.3. Errors
l InvalidAttributeName

15.2.6. GetQueueUrl
Returns the URL of an existing Amazon SQS queue.

1118
 15.2. SQS Supported Actions

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: GetQueueUrl

15.2.6.1. Request Parameters

l QueueName
l QueueOwnerAWSAccountId

15.2.6.2. Response Elements

l QueueUrl

15.2.6.3. Errors
l AWS.SimpleQueueService.NonExistentQueue

15.2.7. ListQueues
Returns a list of your queues in the current region.

HyperStore supports the parameters and elements listed below.

For action details and examples see the AWS documentation: ListQueues

15.2.7.1. Request Parameters

l MaxResults
l NextToken
l QueueNamePrefix

15.2.7.2. Response Elements

l NextToken
l QueueUrl.N

15.2.8. PurgeQueue
Deletes the messages in a queue specified by the QueueURL parameter.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: PurgeQueue

15.2.8.1. Request Parameters

l QueueUrl

1119
Chapter 15. SQS API

15.2.8.2. Errors
l AWS.SimpleQueueService.NonExistentQueue
l AWS.SimpleQueueService.PurgeQueueInProgress

15.2.9. ReceiveMessage
Retrieves one or more messages (up to 10), from the specified queue.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: ReceiveMessage

15.2.9.1. Request Parameters

l MaxNumberOfMessages
l QueueUrl
l ReceiveRequestAttemptId
l VisibilityTimeout
l WaitTimeSeconds

Note HyperStore does not currently support message attributes.

15.2.9.2. Response Elements

l Message.N

15.2.9.3. Errors
l OverLimit

15.2.10. SendMessage
Delivers a message to the specified queue.

HyperStore supports the parameters, elements, and errors listed below.

For action details and examples see the AWS documentation: SendMessage

Note The SendMessage action is not intended to be used by external SQS clients. The HyperStore S3
Service internally uses the SendMessage action to publish notification messages to a queue.

15.2.10.1. Request Parameters

l DelaySeconds
l MessageBody
l QueueUrl

1120
 15.2. SQS Supported Actions

Note HyperStore does not currently support message attributes.

15.2.10.2. Response Elements

l MD5OfMessageBody
l MessageId

15.2.10.3. Errors
l AWS.SimpleQueueService.UnsupportedOperation
l InvalidMessageContents

15.2.11. SetQueueAttributes
Sets the value of one or more queue attributes.

HyperStore supports the parameters and errors listed below.

For action details and examples see the AWS documentation: SetQueueAttributes

15.2.11.1. Request Parameters

l Attribute

Note For the list of queue attributes that HyperStore supports see "CreateQueue" (page 1117).

l QueueUrl

15.2.11.2. Errors
l InvalidAttributeName

1121
This page left intentionally blank
Chapter 16. Open Source License Agree-
ments
Cloudian, Inc. acknowledges the redistribution of open source components under the licenses shown below.

Component or
License License URL Copyright
Library
Copyright 2011 Dain Sundstrom
Apache http://www.apache.org/licenses/LICENSE-
Airlift [email protected] Copyright 2010
2.0 2.0
Cedric Beust [email protected]

Apache http://www.apache.org/licenses/LICENSE- Copyright 2010-2014 Amazon.-

Amazon S3 SDK
2.0 2.0 com, Inc. or its affiliates.

Copyright (c) 2012 Terence Parr

Antlr BSD http://www.antlr.org/license.html
and Sam Harwell

Apache Com- Apache http://www.apache.org/licenses/LICENSE- Copyright © 2018 The Apache

mons 2.0 2.0 Software Foundation.

Apache
Apache http://www.apache.org/licenses/LICENSE- Copyright © 2005-2018 The
HTTPCom-
2.0 2.0 Apache Software Foundation
ponents

Apache http://www.apache.org/licenses/LICENSE- Copyright © 1999-2018, The

Apache Tomcat
2.0 2.0 Apache Software Foundation

Apache http://www.apache.org/licenses/LICENSE- Copyright © 2005-2018 The

Apache Velocity
2.0 2.0 Apache Software Foundation

Apache http://www.apache.org/licenses/LICENSE- Copyright © 2012 The Apache

Avro
2.0 2.0 Software Foundation."

Copyright © 2010 Sebastian

Blueimp MIT https://opensource.org/licenses/MIT
Tschan, https://blueimp.net

Copyright (c) 2011-2018 Twitter,

Bootstrap MIT https://opensource.org/licenses/MIT Inc. Copyright (c) 2011-2018 The
Bootstrap Authors

Apache http://www.apache.org/licenses/LICENSE- Copyright 2009-2014 The Apache

Cassandra
2.0 2.0 Software Foundation

GPL
http://mir- Copyright © 2017 The CentOS Pro-
CentOS and vari-
ror.centos.org/centos/6/os/i386/EULA ject
ous

https://opensource.org/licenses/BSD-3- Copyright 2010-2017 Mike

D3 BSD
Clause Bostock

DataStax Java Apache http://www.apache.org/licenses/LICENSE-

Copyright 2012-2018, DataStax
Driver 2.0 2.0

Copyright (C) 2008-2018, SpryMe-

DataTables MIT https://opensource.org/licenses/MIT
dia Ltd.

Disruptor Apache http://www.apache.org/licenses/LICENSE- None

1123
Chapter 16. Open Source License Agreements

Component or
License License URL Copyright
Library
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE-
DropWizard None
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE-
Gson Copyright 2008 Google Inc.
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE-
Guava None
2.0 2.0

Hector MIT https://opensource.org/licenses/MIT Copyright (c) 2010 Ran Tavory

Public https://github.com/stephenc/high-scale-
High-scale-lib None
Domain lib/blob/master/LICENSE

Apache http://www.apache.org/licenses/LICENSE-
Jackson None
2.0 2.0

Oracle
http://www.or-
binary Copyright © 1995, 2018, Oracle
Java acle.com/technetwork/java/javase/
code and/or its affiliates.
terms/license/index.html
license

Copyright (c) 2002-2015 Atsuhiko

JCraft BSD http://www.jcraft.com/jsch
Yamanaka, JCraft,Inc.

Custom:
No lim-
itation
https://github.com/xetorthio/jedis/blob/ Copyright (c) 2010 Jonathan Leibi-
Jedis if copy-
master/LICENSE.txt usky
right
include-
d

CDDL Copyright ©2010-2017 Oracle Cor-

Jersey https://jersey.java.net/license.html
v1.1 poration.

Apache http://www.apache.org/licenses/LICENSE- Copyright © 2016 The Eclipse

Jetty
2.0 2.0 Foundation.

Apache http://www.apache.org/licenses/LICENSE-
JNA None
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE-
Joda-Time Copyright ©2002-2017 Joda.org.
2.0 2.0

Copyright JS Foundation and

Jquery MIT https://opensource.org/licenses/MIT other contributors, https://js.-
foundation/

Copyright (c) 2015 Boris Moore,

jsviews MIT https://opensource.org/licenses/MIT https://-
github.com/BorisMoore/jsviews

JYaml BSD http://jyaml.sourceforge.net/license.html None

Apache http://www.apache.org/licenses/LICENSE- Copyright © 1999-2018 The

log4j
2.0 2.0 Apache Software Foundation.

1124
 Component or
License License URL Copyright
Library
Apache http://www.apache.org/licenses/LICENSE-
LZ4 None
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE-
Netty None
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE-
OpenCSV None
2.0 2.0

https://github.com/paul-ham-
Copyright (c) 2006 Paul Hammant
Paranamer BSD mant/paranamer/blob/
& ThoughtWorks Inc
master/LICENSE.txt

Apache http://www.apache.org/licenses/LICENSE- Copyright (C) 2005-2016 Puppet,

Puppet
2.0 2.0 Inc.

3-
Copyright (c) 2006-2015, Sal-
Redis clause http://redis.io/topics/license
vatore Sanfilippo
BSD

Apache http://www.apache.org/licenses/LICENSE- Copyright (c) 2011 The LevelDB

RocksDB
2.0 2.0 Authors.

SLF4J MIT https://opensource.org/licenses/MIT Copyright (c) 2004-2017 QOS.ch

Apache http://www.apache.org/licenses/LICENSE-
SnakeYaml None
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE-
Snappy None
2.0 2.0

Apache http://www.apache.org/licenses/LICENSE- Copyright © 2003-2018,

SNMP4J
2.0 2.0 SNMP4J.org

Copyright (c) 2013 GoPivotal, Inc.

Copyright (c) 2000-2011 INRIA,
Apache http://www.apache.org/licenses/LICENSE-
Spring France Telecom Copyright (c)
2.0 2.0
1999-2009, OW2 Consortium
<http://www.ow2.org/>

Apache http://www.apache.org/licenses/LICENSE- Copyright © 2005-2018 The

Thrift
2.0 2.0 Apache Software Foundation

Copyright © 2003-2013 Johann

UUID MIT https://opensource.org/licenses/MIT
Burkard

1125