AWS: bump aws sdk version to 2.20.18 #7003
Conversation
|
fix failures due to adobe/S3Mock#880, seems like sdk v2 since 2.18 updated.
Will run AWS integration tests as well shortly if UNIT tests passes. |
a4f82ed to
0f4f4cc
Compare
| Arrays.asList("application/json"), | ||
| "User-Agent", | ||
| Arrays.asList("aws-sdk-java/2.17.257", "Linux/5.4.0-126"))) | ||
| Arrays.asList("aws-sdk-java/2.20.16", "Linux/5.4.0-126"))) |
There was a problem hiding this comment.
updating this isn't really necessary. When I wrote the test I just wanted to have some real headers :)
We can also just update this to have some random number if that's confusing people.
There was a problem hiding this comment.
I think it's perfectly fine :), In my case I just greped the existing version of sdk and replaced it !
|
The code changes look good to me but it looks like CI is failing, it looks like we're missing passing the part number here https://github.com/apache/iceberg/blame/50863d7c028ea36c4cfb5d857f1324f7f978e579/aws/src/test/java/org/apache/iceberg/aws/s3/signer/TestS3RestSigner.java#L180 |
|
it looks like 2.20.18 is now available, should we upgrade to that version? |
0f4f4cc to
c0d391b
Compare
|
Thanks @amogh-jahagirdar, the S3Signer is failing for the same have updated the partNumber, The S3FileIO / S3InputStream / S3OutputStream were also failing, i have added a hacky fix, basically earlier even if we wanted to create a bucket when there was already an existing bucket with same name it use to pass some how (talking of @before annotation), now it fails which is kind of expected, the tricky part is I tried @BeforeClass as well but seems like it has port finding issue and was consistently failing in my local. For now I have explicitly catched the bucket exists exception (will think of a more cleaner fix, all suggestions are welcomed :)) ! |
Definitely @nastra ! Thanks for calling this out, have updated the pr with the same. |
| } | ||
|
|
||
| @Test | ||
| @Ignore |
There was a problem hiding this comment.
ignored the test for the meanwhile it's failing with, will get back to it
software.amazon.awssdk.services.s3.model.NoSuchUploadException: The specified multipart upload does not exist. The upload ID may be invalid, or the upload may have been aborted or completed. (Service: S3, Status Code: 404, Request ID: 174A3A53489A348F)
There was a problem hiding this comment.
needed to modify this test as this test was using a random upload id rather than the one which we got from creating a multipart upload.
0023281 to
884ed33
Compare
| testImplementation "org.apache.httpcomponents.client5:httpclient5" | ||
| testImplementation 'org.mock-server:mockserver-netty' | ||
| testImplementation 'org.mock-server:mockserver-client-java' | ||
| testImplementation 'javax.xml.bind:jaxb-api' |
There was a problem hiding this comment.
required by s3 mock now
jackye1995
left a comment
There was a problem hiding this comment.
Thanks for fixing all the test failures!!
|
Given this is just a version bump and all tests passed, I will go ahead to merge it without waiting for more approvals. Thanks everyone for the review! @amogh-jahagirdar @nastra , and thanks @singhpk234 for fixing all the breaking tests! |
Bumps AWS java sdk to latest version i.e 2.20.18
cc @jackye1995 @amogh-jahagirdar