tfp.stats.brier_score
Stay organized with collections
Save and categorize content based on your preferences.
Compute Brier score for a probabilistic prediction.
tfp.stats.brier_score(
labels, logits, name=None
)
The [Brier score][1] is a loss function for probabilistic predictions over a
number of discrete outcomes. For a probability vector p and a realized
outcome k the Brier score is sum_i p[i]*p[i] - 2*p[k]. Smaller values are
better in terms of prediction quality. The Brier score can be negative.
Compared to the cross entropy (aka logarithmic scoring rule) the Brier score
does not strongly penalize events which are deemed unlikely but do occur,
see [2]. The Brier score is a strictly proper scoring rule and therefore
yields consistent probabilistic predictions.
References
[1]: G.W. Brier.
Verification of forecasts expressed in terms of probability.
Monthley Weather Review, 1950.
[2]: Tilmann Gneiting, Adrian E. Raftery.
Strictly Proper Scoring Rules, Prediction, and Estimation.
Journal of the American Statistical Association, Vol. 102, 2007.
https://www.stat.washington.edu/raftery/Research/PDF/Gneiting2007jasa.pdf
Args |
|---|
labels
|
Tensor, (N1, ..., Nk), with tf.int32 or tf.int64 elements containing
ground truth class labels in the range [0, num_classes].
|
logits
|
Tensor, (N1, ..., Nk, num_classes), with logits for each example.
|
name
|
Python str name prefixed to Ops created by this function.
|
Returns |
|---|
brier_score
|
Tensor, (N1, ..., Nk), containint elementwise Brier scores;
caller should reduce_mean() over examples in a dataset.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2023-11-21 UTC.
[null,null,["Last updated 2023-11-21 UTC."],[],[],null,["# tfp.stats.brier_score\n\n\u003cbr /\u003e\n\n|----------------------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/probability/blob/v0.23.0/tensorflow_probability/python/stats/calibration.py#L159-L200) |\n\nCompute Brier score for a probabilistic prediction. \n\n tfp.stats.brier_score(\n labels, logits, name=None\n )\n\nThe \\[Brier score\\]\\[1\\] is a loss function for probabilistic predictions over a\nnumber of discrete outcomes. For a probability vector `p` and a realized\noutcome `k` the Brier score is `sum_i p[i]*p[i] - 2*p[k]`. Smaller values are\nbetter in terms of prediction quality. The Brier score can be negative.\n\nCompared to the cross entropy (aka logarithmic scoring rule) the Brier score\ndoes not strongly penalize events which are deemed unlikely but do occur,\nsee \\[2\\]. The Brier score is a strictly proper scoring rule and therefore\nyields consistent probabilistic predictions.\n\n#### References\n\n\\[1\\]: G.W. Brier.\nVerification of forecasts expressed in terms of probability.\nMonthley Weather Review, 1950.\n\\[2\\]: Tilmann Gneiting, Adrian E. Raftery.\nStrictly Proper Scoring Rules, Prediction, and Estimation.\nJournal of the American Statistical Association, Vol. 102, 2007.\n\u003chttps://www.stat.washington.edu/raftery/Research/PDF/Gneiting2007jasa.pdf\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|----------|---------------------------------------------------------------------------------------------------------------------------------|\n| `labels` | Tensor, (N1, ..., Nk), with tf.int32 or tf.int64 elements containing ground truth class labels in the range \\[0, num_classes\\]. |\n| `logits` | Tensor, (N1, ..., Nk, num_classes), with logits for each example. |\n| `name` | Python `str` name prefixed to Ops created by this function. |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ------- ||\n|---------------|-----------------------------------------------------------------------------------------------------------------------|\n| `brier_score` | Tensor, (N1, ..., Nk), containint elementwise Brier scores; caller should `reduce_mean()` over examples in a dataset. |\n\n\u003cbr /\u003e"]]
View source on GitHub