Bug 1580893 - Support filtering tasks by test path #5794
Conversation
cea1700 to
01593ab
Compare
|
Hi @camd I will appreciate an initial quick pass before I spend a bunch of time on writing the tests for this. This works nicely. I will look into fixing the Travis test failures. I made a decision of not storing the data on the backend since retrieving the artifact directly from Taskcluster is very fast and I wrote the code in such a way to not impact the loading of jobs. The commit message has more information about how the feature is implemented. I have started a Heroku Review app and it should be usable within the hour when it has ingested pushes and tasks for The screenshot shows jobs filtered by the
|
01593ab to
3ea72b0
Compare
3ea72b0 to
12fb352
Compare
823d777 to
2a4273f
Compare
camd
left a comment
There was a problem hiding this comment.
This looks pretty good to me. As I say, I'm concerned about performance. But if it can handle 50 pushes as well as we currently can, then I think we're ok. We should check the memory footprint as well. And check for new memory leaks, too.
|
|
||
| const fetchTestManifestsArtifact = async (project, revision) => { | ||
| let taskNameToManifests = {}; | ||
| const rootUrl = 'https://firefox-ci-tc.services.mozilla.com'; |
There was a problem hiding this comment.
I'm sure this is just here for expediency during prototyping, but you would get this value from the repository object tc_root_url field.
There was a problem hiding this comment.
I can look into that but this only makes sense for Firefox test jobs since the artifact is only created for Hg repos.
I might want to prevent the unnecessary call for non-Firefox repos.
| // when they don't care about the specific jobs and results. | ||
| if (!getAllUrlParams().has('nojobs')) { | ||
| this.fetchJobs(); | ||
| await Promise.all([this.fetchJobs(), this.fetchTestManifests()]); |
There was a problem hiding this comment.
I'm very curious how this will perform. I'll admit it gives me pause. It seems like something that will only be used sometimes, but we will pay the performance cost every time. This seems like something that would cause a noticeable performance hit. But perhaps not.
Alternatively, we could only do this fetch when the use has chosen to filter by the test path.
There was a problem hiding this comment.
I suppose we should test before and after this change with a load of 30-50 pushes in memory.
There was a problem hiding this comment.
I just fixed the app! You can now test for yourself.
There's not a noticeable difference.
30 pushes (I've used devtools to with searchStr to do some sort of filtering):
https://treeherder.mozilla.org/#/jobs?repo=autoland&group_state=expanded&fromchange=a225046c4122e79e7bfe8f7f5917bc0bd80493a3&searchStr=devtools
f1af2bb to
814b0b3
Compare
814b0b3 to
920b785
Compare
Codecov Report
@@ Coverage Diff @@
## master #5794 +/- ##
==========================================
+ Coverage 41.23% 41.44% +0.21%
==========================================
Files 209 209
Lines 6873 6944 +71
Branches 1473 1491 +18
==========================================
+ Hits 2834 2878 +44
- Misses 3717 3737 +20
- Partials 322 329 +7
Continue to review full report at Codecov.
|
920b785 to
e772889
Compare
a3a6de3 to
d49a931
Compare
|
There's one known bug to me. I've filed this: |
During [review time](#5794 (comment)) we introduced a small bug in the test filtering feature. Reading `jobList` at the begining of the function instead of just before calling `mapPushJobs` almost guaranteed to start with an empty list which makes the call useless since it needs a non-empty list to work. I also changed the code to call `this.setState` with a callback to `mapPushJobs` to guarantee that `jobTypeNameToManifests` is set in the state and not get an empty object.
During review time[1] we introduced a small bug in the test filtering feature. Reading `jobList` at the begining of the function instead of just before calling `mapPushJobs` almost guaranteed to start with an empty list which makes the call useless since it needs a non-empty list to work. I also changed the code to call `this.setState` with a callback to `mapPushJobs` to guarantee that `jobTypeNameToManifests` is set in the state and not get an empty object. [1] #5794 (comment)
During review time[1] we introduced a small bug in the test filtering feature. Reading `jobList` at the begining of the function instead of just before calling `mapPushJobs` almost guaranteed to start with an empty list which makes the call useless since it needs a non-empty list to work. I also changed the code to call `this.setState` with a callback to `mapPushJobs` to guarantee that `jobTypeNameToManifests` is set in the state and not get an empty object. [1] #5794 (comment)
During review time[1] we introduced a small bug in the test filtering feature. Reading `jobList` at the begining of the function instead of just before calling `mapPushJobs` almost guaranteed to start with an empty list which makes the call useless since it needs a non-empty list to work. I also changed the code to call `this.setState` with a callback to `mapPushJobs` to guarantee that `jobTypeNameToManifests` is set in the state and not get an empty object. [1] #5794 (comment)
* Support filtering tasks by test path For every push, it fetches the artifact `manifests-by-task.json` produced by the Gecko decision task. For every job it adds the `test_paths` property which allows the filtering. Click on the "Filter by a job field" (the funnel icon), select "test path" from the dropdown and you can insert a path like `devtools/client/inspector/changes/test/browser.ini` (You can use substrings). * Use Django's json() * Skip test that only times out on Travis
During review time[1] we introduced a small bug in the test filtering feature. Reading `jobList` at the begining of the function instead of just before calling `mapPushJobs` almost guaranteed to start with an empty list which makes the call useless since it needs a non-empty list to work. I also changed the code to call `this.setState` with a callback to `mapPushJobs` to guarantee that `jobTypeNameToManifests` is set in the state and not get an empty object. [1] mozilla#5794 (comment)
Using filter by "job field" instead of the
searchStr(the latter would require either a hack or a small refactor):