AWS: Update S3V4RestSignerClient to send body for DeleteObjects requests #8365
Conversation
7cf5447 to
459427e
Compare
aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java
Show resolved
Hide resolved
aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java
Outdated
Show resolved
Hide resolved
aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java
Outdated
Show resolved
Hide resolved
aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java
Show resolved
Hide resolved
b8b2419 to
222142b
Compare
aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java
Outdated
Show resolved
Hide resolved
|
|
||
| private boolean isDeleteObjectsRequest(SdkHttpFullRequest request) { | ||
| return request.method() == SdkHttpMethod.POST | ||
| && request.rawQueryParameters().containsKey("delete"); |
There was a problem hiding this comment.
S3 supports a lot of requests for other operations. Are there any other combinations of requests that might meet this criteria? Just wondering because we may be exposing the client for other operations and I want to make sure this uniquely identifies the bulk delete operation.
There was a problem hiding this comment.
I glanced over a few other common request types and couldn't find anything that would match this, but would be good to do a thorough check
There was a problem hiding this comment.
I wasn't able to find anything either (referencing https://docs.aws.amazon.com/AmazonS3/latest/API/API_Operations_Amazon_Simple_Storage_Service.html) , I'll do a double pass and also see if there's even more specific criteria for bulk deletes beyond the method type and query parameter.
There was a problem hiding this comment.
I did a double pass to make sure there wasn't any other request that could possibly match this criteria.
As far as getting even more unique, we could actually perform an extraction of the DeleteObjectsRequest contents from the body (because we know no other request at the moment would send that). But this is probably too complex for what it brings considering what we have now looks to be unique already (we would need to add the jackson XML dependency to the signer client, etc).
Lmk if you have any concerns @danielcweeks @nastra
aws/src/test/java/org/apache/iceberg/aws/s3/signer/S3SignerServlet.java
Outdated
Show resolved
Hide resolved
aws/src/test/java/org/apache/iceberg/aws/s3/signer/S3SignerServlet.java
Outdated
Show resolved
Hide resolved
aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java
Outdated
Show resolved
Hide resolved
1fd773c to
8e38f9a
Compare
aws/src/main/java/org/apache/iceberg/aws/s3/signer/S3V4RestSignerClient.java
Outdated
Show resolved
Hide resolved
8e38f9a to
10d909a
Compare
|
Merging, thanks @nastra @danielcweeks for the reviews! |
Update S3V4RestSignerClient to send body for DeleteObjects (bulk delete) requests. Added a test which tests against a MinIO container. Follow up to #8361