API: Fix default FileIO#newInputFile ManifestFile, DataFile and DeleteFile implementation to pass lengths

[amogh-jahagirdar]

As part of adding encryption support, in #9592 we added some new FileIO APIs, namely

newInputFile(ManifestFile)
newInputFile(DataFile)
newInputFile(DeleteFile)

The overriden implementaiton in EncryptedFileIO is correct but the default implementation in FileIO for these new APIs should pass in a length since it's always known from the Iceberg metadata.

Without this, FileIO implementations which end up calling these default implementations (specifically referring to these new APIs)will make extra requests to the object store/file system to determine the length which we can avoid.

[ajantha-bhat]

Minor comment about the test validation.

Thanks for fixing it.

I think we do have to conclude whether we need 1.5.1 release because of this. Because one extra IO for each manifest and DataFile reading seems like a problem for me.

[ajantha-bhat]

ping @danielcweeks, @rdblue, @nastra

[amogh-jahagirdar]

@ajantha-bhat Replied my thoughts on the Trino PR. In short, don't think a patch release is required. I'll summarize why I think that here:

To be clear, there is no extra I/O being done in practice for newInputfile(DataFile) since that code path does not look to be exercised yet in the latest release. If I'm wrong there though, then I would definitely change my position.

What's remaining is the newInputFile(ManifestFile) where there would be an extra I/O during engine planning. Since the I/O is an extra head request to object stores which typically is low double digit milliseconds + the fact that only a subset of manifess are read during planning I think that it's not significant enough to warrant a patch release, but I can be convinced otherwise if folks hit a latency regression this in their workloads.

[Fokko]

This is a great catch @amogh-jahagirdar thanks for fixing this 👍

[amogh-jahagirdar]

Thanks for the reviews @ajantha-bhat and @Fokko ! Merging