`ChunkedToXContentBuilder` should be removed

[original-brownbear]

The current idea behind ChunkedToXContentBuilder is broken. We create the ChunkedToXContentBuilder which is essentially a flat stream of the chunks we're building up-front at the beginning of serialization.
This runs counter to the whole concept of ChunkedToXContent which tries to ideally be O(1) in terms of up-front heap and runtime cost by pushing all iterations down to when we actually serialize.
I get how the API looks nicer than manually building iterators but this kind of API when combined with iteration (both internally or at the start of the serialization) breaks chunking.
We see this practically and very visibly in search response (and to a lesser degree also for bulk responses) serialization slowness today and almost more importantly the approach causes stability issues on the transport layer in an unpredictable way.

I don't think the API can be fixed or that it is worthwhile to fix it. The slight improvement in readability of the code comes at the cost of unpredictable and hard to reason about performance here as would probably any solution not built on statically known iteration counts.
Marking this as >bug since I am observing an uptick in transport thread slowness from it and it's easy to construct search use cases that destabilize a cluster since its introduction into the search response handling.

[elasticsearchmachine]

Pinging @elastic/es-core-infra (Team:Core/Infra)

[elasticsearchmachine]

Pinging @elastic/es-distributed-coordination (Team:Distributed Coordination)

[original-brownbear]

Just to clarify the impact here, we're seeing a 50%+ regression from the use of this in search responses in one of our nightlies. We shouldn't have an API that introduces the risk of accidentally breaking things to this degree.

[thecoop]

The previous API was very non-obvious and confusing to use, and it was unclear what each method in ChunkedXContentHelper was actually doing. I think there's a middle ground to find here, developing a good API whilst keeping the performance hit to a minimum. The changes in #118649 should ease the immediate performance hit in the meantime.

[original-brownbear]

@thecoop those changes will help a little but not much I would expect, partly due to https://bugs.openjdk.org/browse/JDK-8267359. https://bugs.openjdk.org/browse/JDK-8267758 I'd say makes the stream approach bound to fail here.

Fundamentally there is no nice solution here, of that I'm quite convinced. The problem we have is the same you have in e.g. Netty serializers. If you want accurate chunk sizes, you need a low overhead next method. The nested custom iterators we have are (short of using code generation like tricks like manually building callsites) probably as good as it gets.
The last comment in the JDK issue I linked (https://bugs.openjdk.org/browse/JDK-8267359?focusedId=14426679&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-14426679) explains why there is no neat solution here I believe.

This API also doesn't have to be nicely usable. It is designed for the very narrow use case of very large REST responses. We should, going forward, not add new APIs that require this logic (I believe this was consensus when we added chunked encoding). This is to support existing debug style APIs like the cluster state where we have large responses by design and BwC makes it impossibly costly to change the design. Building the APIs from scratch I'd go as far as saying the search APIs are the only use case where chunked encoding just makes sense. So if we're not changing anything about this code and it's so extremely performance critical, why introduce the risk of stuff like this? We've seen how easy it is to accidentally regress massively once you introduce indirection to this logic haven't we? :)

[mhl-b]

Makes sense to me. As I understand from the thread and code, we do double buffering, ChunkedToXContentBuilder accumulates unserialized search results and serialized data will be buffered in Netty channel. The unserialized data produces faster than we flush to the network.

Since ChunkedToXContent is driven by outbound buffer capacity we should have an API that reflects the availability of that buffer, a callback that we provide to network layer that will be invoked every time we have enough of buffer. Propagating callback to the very bottom of the result production might be easier than stream/iterators.

channel.sendResponse(header, (sizeHint) -> produceToXContentChunk(sizeHint));

Netty provides both, a callback on channel writability and buffer space. channelWritabilityChanged and bytesBeforeUnwritable.

[original-brownbear]

@mhl-b we currently do what you are describing and use at least the writeability change hook. The bytesBeforeUnwriteable is a little low for my taste and doesn't reflect actual channel writeability (that's probably unpredictable).
I get your point though that the iterator isn't the best match for the callback.
That's mostly a result of how all of this got started. We had the normal toXContent implementations and added chunking on top. So the iterator thing was the simplest abstraction around that but not very efficient or elegant for sure :)
That's what I was getting at above, ideally you'd have something like what Netty has for many serialization implementations where you have logic that keeps some custom state for the serialization. Like "boolean part1Written" or whatever, you get the gist :) That would probably be quite a bit faster. We could look into that sometime for search and bulk responses where performance really matters. We also need this in search eventually anyway I think, we still lack a clean solution for large binary fields there that I'd like to chunk eventually, the x-content iterators won't cut it there :)

For the time being though, our light and low level iterators seem to be far faster than the stream logic in this class.

[mhl-b]

Thank you for reply, +1 on that. Dont want to derail this thread from the original point of removing stream logic. PS bytesBeforeUnwriteable is predictable to some degree, but we can talk about it separately :)