Adaptive sampling for coarse ranking

S Katariya, L Jain, N Sengupta… - International …, 2018 - proceedings.mlr.press
International Conference on Artificial Intelligence and Statistics, 2018proceedings.mlr.press
We consider the problem of active coarse ranking, where the goal is to sort items according
to their means into clusters of pre-specified sizes, by adaptively sampling from their reward
distributions. This setting is useful in many social science applications involving human
raters and the approximate rank of every item is desired. Approximate or coarse ranking can
significantly reduce the number of ratings required in comparison to the number needed to
find an exact ranking. We propose a computationally efficient PAC algorithm LUCBRank for …
Abstract
We consider the problem of active coarse ranking, where the goal is to sort items according to their means into clusters of pre-specified sizes, by adaptively sampling from their reward distributions. This setting is useful in many social science applications involving human raters and the approximate rank of every item is desired. Approximate or coarse ranking can significantly reduce the number of ratings required in comparison to the number needed to find an exact ranking. We propose a computationally efficient PAC algorithm LUCBRank for coarse ranking, and derive an upper bound on its sample complexity. We also derive a nearly matching distribution-dependent lower bound. Experiments on synthetic as well as real-world data show that LUCBRank performs better than state-of-the-art baseline methods, even when these methods have the advantage of knowing the underlying parametric model.
proceedings.mlr.press
Showing the best result for this search. See all results