In this letter, we propose a novel search approach to blur kernel estimation for defocused image restoration. An adaptive binary search on consensus is the main contribution of our research. It is based on binary search and random sample consensus set (RANSAC). Moreover an evaluating function which uses a histogram of gradient distribution is proposed for assessing restored images. Simulations on an image benchmark dataset shows that the proposed algorithm can estimate, on average, the blur kernels 15.14% more accurately than other defocused image restoration algorithms.