feat: Add support for percent of totals in MetricsViewAggregation #5348
Conversation
proto/rill/runtime/v1/queries.proto
Outdated
| @@ -389,6 +389,7 @@ message MetricsViewAggregationMeasure { | |||
| MetricsViewAggregationMeasureComputeComparisonValue comparison_value = 7; | |||
| MetricsViewAggregationMeasureComputeComparisonDelta comparison_delta = 8; | |||
| MetricsViewAggregationMeasureComputeComparisonRatio comparison_ratio = 9; | |||
| MetricsViewAggregationMeasurePercentOfTotal percent_of_total = 10; | |||
There was a problem hiding this comment.
nit: put "compute" in the name like for the other computed measure types: MetricsViewAggregationMeasureComputePercentOfTotal
|
On reflection, I think this might be simpler to solve with an additional query. I think there might actually be a simple way to achieve that. The new For inspiration, you can see this rewriter which turns relative time ranges into fixed time ranges, sometimes making a separate query to get the min/max timestamps of the metrics view: rill/runtime/metricsview/executor_rewrite_time.go Lines 14 to 16 in b67474d |
d72a32b
to
87d22a5
Compare
runtime/metricsview/ast.go
Outdated
| Expression: fmt.Sprintf("%s*100/%f", a.dialect.EscapeIdentifier(m.Name), qm.Compute.PercentOfTotal.Total), | ||
| Type: runtimev1.MetricsViewSpec_MEASURE_TYPE_DERIVED, |
There was a problem hiding this comment.
- I wonder if we could make it simple and inject the expression directly? I.e. use something like
fmt.Sprintf("(%s)/%f", m.Expression, qm.Compute.PercentOfTotal.Total)? I think it would work even with subquery measures, at leastselect (select 100)/10works in both DuckDB and Druid. - Not sure, but maybe the
%fshould be%#f, which always adds a decimal point? Otherwise, if both operands are whole numbers, could it end up doing integer division? (Not sure about the SQL semantics here.)
There was a problem hiding this comment.
- We are calculating the measure value itself in a subquery since it might be used for comparison columns as well. So adding the expression here will recompute it wont it?
- Update to be safer.
There was a problem hiding this comment.
Okay, if you think it will mostly be queried in requests that also query the plain measure value, then I agree the derived approach is better
There was a problem hiding this comment.
Ya. I would assume all instances of totals in dashboard right now can be replaced with this.
| totalsQry := &Query{ | ||
| MetricsView: qry.MetricsView, | ||
| Measures: measures, | ||
| TimeRange: qry.TimeRange, | ||
| Where: qry.Where, | ||
| TimeZone: qry.TimeZone, | ||
| } | ||
|
|
||
| e, err := NewExecutor(ctx, e.rt, e.instanceID, mv, sec, e.priority) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| defer e.Close() | ||
|
|
||
| res, err := e.Query(ctx, totalsQry, nil) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| defer res.Close() |
There was a problem hiding this comment.
The exactify rewriter does a couple things differently that I think we should also do here:
- Explicitly set all members of the
Query{}struct to reduce chance of forgetting to handle one if it's changed in the future - Explicitly call
NewASTande.olap.Executeto a) prevent any chance infinite recursion, b) only apply the rewrites that make sense in this context (I think that would be onlyrewriteApproxComparisonsandrewriteDruidGroups)
There was a problem hiding this comment.
Will this not add some complexity? Any new rewrite need to be added in multiple places and the author needs to be aware of it. Not sure if it can be solved as is right now.
There was a problem hiding this comment.
Yeah it's a little annoying, but I think it's best to explicitly add the rewrites each time at this point. To avoid recursion dangers and because the rewrites are a very mixed bag (some require a specific order of other rewrites, some are expensive to run, some don't make sense in the context of certain calls).
If we end up adding more rewrites, it may make sense to refactor how they're configured to avoid the repetition.
70f31ac
to
5dd3ae8
Compare
To add support for percent of totals based filter criteria in alerts, we need to add percent of totals as a variant of a measure.
This PR adds that support to MetricsViewAggregation by calculating totals and rewriting percent of totals with
measure/<calculated_total>