[Bug]: Optimizing process failed for Iceberg format table on MinIO

[zhoujinsong]

What happened?

Optimizing process failed in situation:

• Iceberg Format table
• Table without partition spec
• Use MinIO

Affects Versions

master

What engines are you seeing the problem on?

Optimizer

How to reproduce

• Create an Iceberg format catalog on MiniIO
• Create a table without partition spec
• Write into the table and wait for a optimizing job

Relevant log output

2023-10-31 19:48:44,456 ERROR [Optimizer-executor-0] [com.netease.arctic.optimizer.common.OptimizerExecutor] [] - Optimizer executor[0] executed task[OptimizingTaskId(processId:1698752922559, taskId:1)] f
ailed
java.lang.RuntimeException: software.amazon.awssdk.services.s3.model.S3Exception: Object name contains unsupported characters. (Service: S3, Status Code: 400, Request ID: 17932EC87A522F80, Extended Reques
t ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8)
        at com.netease.arctic.io.ArcticFileIOAdapter.doAs(ArcticFileIOAdapter.java:47) ~[amoro-core-0.6.0-SNAPSHOT.jar:?]
        at com.netease.arctic.io.ArcticFileIOAdapter.doAs(ArcticFileIOAdapter.java:42) ~[amoro-core-0.6.0-SNAPSHOT.jar:?]
        at com.netease.arctic.optimizing.AbstractRewriteFilesExecutor.execute(AbstractRewriteFilesExecutor.java:105) ~[amoro-core-0.6.0-SNAPSHOT.jar:?]
        at com.netease.arctic.optimizing.AbstractRewriteFilesExecutor.execute(AbstractRewriteFilesExecutor.java:61) ~[amoro-core-0.6.0-SNAPSHOT.jar:?]
        at com.netease.arctic.optimizer.common.OptimizerExecutor.executeTask(OptimizerExecutor.java:105) ~[optimizer-common-0.6.0-SNAPSHOT.jar:?]
        at com.netease.arctic.optimizer.common.OptimizerExecutor.start(OptimizerExecutor.java:34) ~[optimizer-common-0.6.0-SNAPSHOT.jar:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_291]
Caused by: software.amazon.awssdk.services.s3.model.S3Exception: Object name contains unsupported characters. (Service: S3, Status Code: 400, Request ID: 17932EC87A522F80, Extended Request ID: dd9025bab4a
d464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8)
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleErrorResponse(AwsXmlPredicatedResponseHandler.java:156) ~[aws-xml-protocol-2.20.5.jar:?]
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handleResponse(AwsXmlPredicatedResponseHandler.java:108) ~[aws-xml-protocol-2.20.5.jar:?]
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:85) ~[aws-xml-protocol-2.20.5.jar:?]
        at software.amazon.awssdk.protocols.xml.internal.unmarshall.AwsXmlPredicatedResponseHandler.handle(AwsXmlPredicatedResponseHandler.java:43) ~[aws-xml-protocol-2.20.5.jar:?]
        at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler$Crc32ValidationResponseHandler.handle(AwsSyncClientHandler.java:95) ~[aws-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.handler.BaseClientHandler.lambda$successTransformationResponseHandler$7(BaseClientHandler.java:270) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:40) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.HandleResponseStage.execute(HandleResponseStage.java:30) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:73) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptTimeoutTrackingStage.execute(ApiCallAttemptTimeoutTrackingStage.java:42) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:78) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.TimeoutExceptionHandlingStage.execute(TimeoutExceptionHandlingStage.java:40) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:50) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallAttemptMetricCollectionStage.execute(ApiCallAttemptMetricCollectionStage.java:36) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:81) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.RetryableStage.execute(RetryableStage.java:36) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:56) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.StreamManagingStage.execute(StreamManagingStage.java:36) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.executeWithTimer(ApiCallTimeoutTrackingStage.java:80) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:60) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallTimeoutTrackingStage.execute(ApiCallTimeoutTrackingStage.java:42) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:48) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ApiCallMetricCollectionStage.execute(ApiCallMetricCollectionStage.java:31) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.RequestPipelineBuilder$ComposingRequestPipelineStage.execute(RequestPipelineBuilder.java:206) ~[sdk-core-2.20.5.jar:?]
        at software.amazon.awssdk.core.internal.http.pipeline.stages.ExecutionFailureExceptionReportingStage.execute(ExecutionFailureExceptionReportingStage.java:37) ~[sdk-core-2.20.5.jar:?]

Anything else

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[zhoujinsong]

After enabling debug logs, it was observed that the Optimizer attempted to submit a path: /local-iceberg/db/user/data//00000-0-c7cb83b4-04df-4ae1-bd5e-dd6c7bc6ef1e-00001.parquet but the MinIO service considered this path to be illegal.

But I noticed Iceberg had fixed this issue in the main branch: apache/iceberg#7685, we can remove this change after upgrading Iceberg dependency to 1.5+.