Add cluster level reduction #117731
Add cluster level reduction #117731
Conversation
821c026 to
20f02a9
Compare
20f02a9 to
50112f1
Compare
|
Pinging @elastic/es-analytical-engine (Team:Analytics) |
|
Hi @dnhatn, I've created a changelog YAML for you. |
| } | ||
| return null; | ||
| return EstimatesRowSize.estimateRowSize(fragment.estimatedRowSize(), reducePlan); | ||
| } |
There was a problem hiding this comment.
I'm not sure about the difference between the instanceof tree and what you've got but I'm going to assume you worked it out and its the same enough.
There was a problem hiding this comment.
I have a question about this, actually. You seem to have only kept AggregateExec here; I have vague memories that there were some tests that tested at least two such scenarios with reducing plans (limit, topn, orderby...). Wondering why no tests failed and why you only kept AggregateExec here.
There was a problem hiding this comment.
Yes, they are the same. The change was made to remove the switch statement.
Wondering why no tests failed and why you only kept AggregateExec here.
We need to change the mode of AggregateExec to emit intermediate outputs.
💚 Backport successful
|
This change introduces cluster-level reduction. Unlike data-node-level reduction, it does not require pragmas because the network latency and throughput across clusters differ significantly from those within a cluster. As a result, the benefits of this reduction should outweigh the risks.
* Add cluster level reduction (#117731) This change introduces cluster-level reduction. Unlike data-node-level reduction, it does not require pragmas because the network latency and throughput across clusters differ significantly from those within a cluster. As a result, the benefits of this reduction should outweigh the risks. * compile
This change introduces cluster-level reduction. Unlike data-node-level reduction, it does not require pragmas because the network latency and throughput across clusters differ significantly from those within a cluster. As a result, the benefits of this reduction should outweigh the risks.