remove usage reporting from MT #666
Conversation
usage/usage.go
Outdated
| metrics = m | ||
| metricIndex = i | ||
| Clock = cl | ||
| partition := partitions[0] |
There was a problem hiding this comment.
That's an edge case, but I can't see anything that guarantees that there is >=1 partitions and not 0
usage/usage.go
Outdated
| period: period, | ||
| now: make(map[int]orgstat), | ||
| stop: make(chan struct{}), | ||
| shardID: shardID, |
There was a problem hiding this comment.
There is also nothing which sorts the partition IDs after they've been parsed out of the file: https://github.com/raintank/metrictank/blob/master/input/kafkamdm/kafkamdm.go#L147-L164
But considering that the list of partition IDs is part of the shardID i think it might make sense to sort them in order to guarantee that the same combination of partitions results in the same shardID
|
see #665 (comment) |
|
ok, i have come to the realisation that this just can't be done nicely from within MT. So let's just get rid of these usage stats and extend tsdb-gw to collect them. |
- metrictank is designed as an at-scale TSDB. Therefore it should always be run as a cluster. The usage reporting doesnt work in a clustered environment. Usage reporting should be done from the ingestion tool (carbon-relay-ng or tsdb-gw)
can you elaborate on this? |
see #665 (comment) |
|
@Dieter can this be merged now? |
include a "shardID" in the series name used for usage reporting.
this will allow usage of each shard to be aggregated at query time.
the shardID is just each partition concatenated together in the form
"shard-<partition>[-<partition>...]"when writing usage metrics set the "partition" of the metric to one of
the partition numbers the node is handling
issue make per org usage stats work on multinode clusters #665