Skip to content

ESQL: Fix Driver status iterations and cpuTime #123290

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 28, 2025

Conversation

ivancea
Copy link
Contributor

@ivancea ivancea commented Feb 24, 2025

Fixes #122967

When the Driver reported status, if the "report at least every X time" report was triggered, it was re-adding the same iterations and cpuTime again, as it wasn't clearing it before the next iteration.
Now, there are separated variables for the last updated iterations and reported time.

@ivancea ivancea added >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) :Analytics/ES|QL AKA ESQL v9.1.0 auto-backport Automatically create backport pull requests when merged v9.0.1 v8.19.0 labels Feb 24, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @ivancea, I've created a changelog YAML for you.

@ivancea ivancea marked this pull request as ready for review February 24, 2025 17:05
@ivancea ivancea requested review from nik9000 and bpintea February 24, 2025 17:05
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine
Copy link
Collaborator

Hi @ivancea, I've updated the changelog YAML for you.

Copy link
Contributor

@bpintea bpintea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

currentIteration++;
unreportedIterations++;

long now = nowSupplier.getAsLong();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: you could extract an elapsed time, since all the checks use that as 1st updateStatus() param.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One branch doesn't use that calculation, and it may be unused if no if is executed. So I would keep it that way, probably

Comment on lines 206 to 208
int currentIteration = 0;
// The iterations to be reported on the next status update.
int unreportedIterations = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional: you could call them totalIterations (and currentIterations, respectively).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just made the variable names more explicit 👀

# Conflicts:
#	x-pack/plugin/esql/compute/src/test/java/org/elasticsearch/compute/operator/DriverTests.java
@ivancea ivancea merged commit e8b01ff into elastic:main Feb 28, 2025
17 checks passed
@ivancea ivancea deleted the driver-iterations branch February 28, 2025 11:05
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.x Commit could not be cherrypicked due to conflicts
9.0 Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 123290

ivancea added a commit to ivancea/elasticsearch that referenced this pull request Feb 28, 2025
Fixes elastic#122967

When the Driver reported status, if the "report at least every X time" report was triggered, it was re-adding the same iterations and cpuTime again, as it wasn't clearing it before the next iteration.
Now, there are separated variables for the last updated iterations and reported time.
ivancea added a commit to ivancea/elasticsearch that referenced this pull request Feb 28, 2025
Fixes elastic#122967

When the Driver reported status, if the "report at least every X time" report was triggered, it was re-adding the same iterations and cpuTime again, as it wasn't clearing it before the next iteration.
Now, there are separated variables for the last updated iterations and reported time.
elasticsearchmachine pushed a commit that referenced this pull request Feb 28, 2025
Fixes #122967

When the Driver reported status, if the "report at least every X time" report was triggered, it was re-adding the same iterations and cpuTime again, as it wasn't clearing it before the next iteration.
Now, there are separated variables for the last updated iterations and reported time.
elasticsearchmachine pushed a commit that referenced this pull request Feb 28, 2025
Fixes #122967

When the Driver reported status, if the "report at least every X time" report was triggered, it was re-adding the same iterations and cpuTime again, as it wasn't clearing it before the next iteration.
Now, there are separated variables for the last updated iterations and reported time.
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request Apr 15, 2025
Without elastic#123290 we're not going to get accurate status reports. They are
indicative, but not accurate.

Closes elastic#126808
nik9000 added a commit that referenced this pull request Apr 15, 2025
Without #123290 we're not going to get accurate status reports. They are
indicative, but not accurate.

Closes #126808
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.19.0 v9.0.1 v9.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ESQL: iterations in profile/task status lies
5 participants