SupportsBulkOperations for ResolvingFileIO

[dramaticlly]

Feature Request / Improvement

Since #6682 provide major deletion performance improvement on S3 and hadoop fileIO, I think we can try to add SupportsBulkOperations for https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java.

Today the ResolvingFileIO do dynamic fileIO loading based on location scheme, which are essentially fall to

• S3FileIO
• HadoopFileIO
  and both implement SupportsBulkOperations interface.

However I realized the its method is given a iterable of location instead of singular,

void deleteFiles(Iterable<String> pathsToDelete) throws BulkDeletionFailureException;

so it does not fit out of box with current hashmap dynamic loading from scheme to its fileIO impl.

Questions

does it make sense to group prefix by scheme and delegate bulk deletion to dynamic IO?

@rdblue and @nastra do you have any suggestions on this?

Query engine

None

[zinking]

would delete too many paths concurrently trigger rate limit etc ?