Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add response histograms to apache_metrics example. #46

Merged
merged 3 commits into from
Oct 23, 2016
Merged

Add response histograms to apache_metrics example. #46

merged 3 commits into from
Oct 23, 2016

Conversation

SuperQ
Copy link
Contributor

@SuperQ SuperQ commented Oct 12, 2016

Sorry, microseconds is the only reasonably compatible time until mtail supports floats metrics.

CC @beorn7

@coveralls
Copy link

coveralls commented Oct 12, 2016

Coverage Status

Coverage remained the same at 71.366% when pulling 2e3dcf6 on SuperQ:master into 987040f on google:master.

counter apache_http_received_bytes_total by server_port, handler, request_method, request_status, request_protocol
counter apache_http_request_time_microseconds by quantile, server_port, handler, request_method, request_status, request_protocol
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"quantile" doesn't seem like the right term here. It's the boundary of the latency bucket we are talking about here. So "bucket" or "upper_bound" or something?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was just using the labeling used by other similar metrics. This should result in normal Prometheus-style cumulative histogram labeling.

@beorn7
Copy link
Contributor

beorn7 commented Oct 13, 2016

10x buckets are really coarse. Check out our default buckets for a "network service": https://github.com/prometheus/client_golang/blob/master/prometheus/histogram.go#L54-L59

Even those are pretty coarse, and only meant as a starting point before you manually tailor the buckets and zoom into the area you are interested in.

@SuperQ
Copy link
Contributor Author

SuperQ commented Oct 13, 2016

This is an example file, I didn't want to get overly granular, but just provided a starting point example for "normal" apache servers. But I can adjust them. Part of the problem is apache servers sometimes also handle very long-lived requests.

@SuperQ
Copy link
Contributor Author

SuperQ commented Oct 13, 2016

@beorn7 quantile in this case is the label used in the output of the text metrics. I have updated this to le.

@coveralls
Copy link

coveralls commented Oct 13, 2016

Coverage Status

Coverage remained the same at 71.366% when pulling dc9d29c on SuperQ:master into 987040f on google:master.

@beorn7
Copy link
Contributor

beorn7 commented Oct 13, 2016

OK, so the labels should be fine now, but it would be good to see an example output.

WRT bucket scheme: Especially because it is an example and because Apache may server very long running queries, I'd use the same bucketing as in https://github.com/prometheus/client_golang/blob/master/prometheus/histogram.go#L54-L59 . The buckets there were laid out under the exact same premises.

Some monitoring systems use 4x buckets, and that's already mostly useless. 10x is really not a good role model, so it shouldn't be used in an example.

@coveralls
Copy link

coveralls commented Oct 13, 2016

Coverage Status

Coverage remained the same at 71.366% when pulling 6953fe1 on SuperQ:master into 987040f on google:master.

1 similar comment
@coveralls
Copy link

coveralls commented Oct 13, 2016

Coverage Status

Coverage remained the same at 71.366% when pulling 6953fe1 on SuperQ:master into 987040f on google:master.

@SuperQ
Copy link
Contributor Author

SuperQ commented Oct 14, 2016

@beorn7 Ok, I've updated the buckets to follow the recommended defaults.

@coveralls
Copy link

coveralls commented Oct 14, 2016

Coverage Status

Coverage remained the same at 71.366% when pulling 52b6770 on SuperQ:master into 987040f on google:master.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage remained the same at 71.366% when pulling 52b6770 on SuperQ:master into 987040f on google:master.

@beorn7
Copy link
Contributor

beorn7 commented Oct 14, 2016

Thanks.

Could you attach a sample output to this PR so that we can check the final outcome?

@jaqx0r
Copy link
Contributor

jaqx0r commented Oct 23, 2016

Thanks!

@jaqx0r jaqx0r merged commit 5588b4a into google:master Oct 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants