Collect usage about used js modules and outputs #3525
Conversation
eab8e60 to
8c742d3
Compare
We need to collect data about what is the usage of the builtin JS modules and outputs. We only collect the names of the used modules and outputs that are builtin in k6. We don't collect any data about extensions or other private modules.
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## master #3525 +/- ##
==========================================
- Coverage 72.55% 72.53% -0.02%
==========================================
Files 276 278 +2
Lines 20840 20891 +51
==========================================
+ Hits 15120 15154 +34
- Misses 4758 4774 +16
- Partials 962 963 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
| for _, module := range importedModules { | ||
| // Exclude JS modules extensions to prevent to leak | ||
| // any user's custom extensions | ||
| if strings.HasPrefix(module, "k6/x") { |
There was a problem hiding this comment.
Would make sense to keep a list, even if manually, here to also get usage stats for the extensions that we do own? At least until we find a better way to do so.
From the product perspective, I guess it could make lot of sense to know what extensions worth some attention and which ones not.
There was a problem hiding this comment.
xk6-client-tracing
xk6-notification
xk6-disruptor
xk6-sql
xk6-exec
xk6-kubernetes
xk6-ssh
xk6-loki
xk6-dashboard
xk6-output-influxdb
xk6-output-kafka
xk6-output-timescaledb
This is the list of extensions marked as official here: https://k6.io/docs/extensions/get-started/explore. None of them sounds so relevant at the moment to justify the risk of having this logic. Let me know if you have a different opinion.
There was a problem hiding this comment.
Okay! Sounds fair! Thanks!
|
@mstoykov as you are working on refactoring some closer areas, it would be good to get a review from you as well. |
cmd/report.go
Outdated
| } | ||
|
|
||
| builtinOutputs := map[string]bool{ | ||
| "cloud": true, "csv": true, "experimental-prometheus-rw": true, |
There was a problem hiding this comment.
Nit: These output keys (cloud, csv, etc.) come from the map https://github.com/grafana/k6/blob/master/cmd/outputs.go#L26-L47. It's better to extract this as the variable/method and use it here. That way, we protect the future maintainers/contributors from forgetting to update this place.
Some alternatives (e.g., using const) could also be considered, but the main idea is to couple the places through the code, not the string values.
There was a problem hiding this comment.
Added a new proposal for addressing this in the last commits
There was a problem hiding this comment.
LGTM! I did check commit by commit and did some 👍 on some comments that I agreed more strongly with 😅
My only concern around pushing every imported module has been adressed so I am happy.
I guess (as future improvement) we can also send whether and how many k6/x/ modules were imported? Maybe even how many source modules were imported 🤷 . Just as a number.
Also, I guess at some point we might want to have a "whitelist" of `k6/x/' modules that are still reported, but that definitely should be done later and with more discussion.
What?
It collects usage stats about the js modules and outputs used in the test. It collects only the names of the used modules or outputs that are built-in in k6 binary.
Why?
It will help to refine our prioritization method based on more objective and data-driven decisions regarding a feature or a breaking change to introduce.
Updates #2952