Fix S3InputStream.readFully connection leak #13899
Conversation
| String range = String.format("bytes=%s-%s", position, position + length - 1); | ||
|
|
||
| IOUtil.readFully(readRange(range), buffer, offset, length); | ||
| try (InputStream stream = readRange(range)) { |
There was a problem hiding this comment.
readTail method has the same problem no?
There was a problem hiding this comment.
Also, it seems that the same pattern is present in ADLS and GCS input streams!
There was a problem hiding this comment.
Yes. I was starting small to see if that's the correct approach. I will do the follow up PRs for other places.
There was a problem hiding this comment.
Nice find @nandorKollar (bad assumption that readFully would close the stream). I would agree there's the same issue for readTail and the other implementations.
danielcweeks
left a comment
There was a problem hiding this comment.
+1 (pending checks)
When S3InputStreamReadFully is called it does not expose the underlying InputStream to the caller and does not close it on its own. This leads to a connection leak in S3Client since the underlying connection is not returned to connection pool in Apace HTTP client.
c682668 to
894e108
Compare
|
@danielcweeks could you please re-approve the GH actions run? It failed on spotless check on the first attempt. Sorry, my bad. |
nandorKollar
left a comment
There was a problem hiding this comment.
LGTM, though the other cases (readTail and ADLS/GCS) mentioned before also might need attention, maybe in a separate PR?
That's exactly what I mentioned above. I'll do another PR for those. Thanks! |
|
Thanks @AnatolyPopov! |
A follow-up to apache#13899 with more fixes of the same issue.
A follow-up to apache#13899 with more fixes of the same issue.
A follow-up to apache#13899 with more fixes of the same issue.
When S3InputStreamReadFully is called it does not expose the underlying InputStream to the caller and does not close it on its own. This leads to connection leak in S3Client since the underlying connection is not returned to connection pool in Apace HTTP client.