Otsu's method

An example image thresholded using Otsu's algorithm

Original image

In computer vision and image processing, Otsu's method, named after Nobuyuki Otsu (大津展之, Ōtsu Nobuyuki), is used to automatically perform clustering-based image thresholding, or, the reduction of a graylevel image to a binary image. The algorithm assumes that the image contains two classes of pixels following bi-modal histogram (foreground pixels and background pixels), it then calculates the optimum threshold separating the two classes so that their combined spread (intra-class variance) is minimal, or equivalently (because the sum of pairwise squared distances is constant), so that their inter-class variance is maximal. Consequently, Otsu's method is roughly a one-dimensional, discrete analog of Fisher's Discriminant Analysis. Otsu's method is also directly related to the Jenks optimization method.

The extension of the original method to multi-level thresholding is referred to as the multi Otsu method.

Method

In Otsu's method we exhaustively search for the threshold that minimizes the intra-class variance (the variance within the class), defined as a weighted sum of variances of the two classes:

$\sigma _{w}^{2}(t)=\omega _{0}(t)\sigma _{0}^{2}(t)+\omega _{1}(t)\sigma _{1}^{2}(t)$

Weights $\omega _{0}$ and $\omega _{1}$ are the probabilities of the two classes separated by a threshold $t$ and $\sigma _{0}^{2}$ and $\sigma _{1}^{2}$ are variances of these two classes.

The class probability $\omega _{0,1}(t)$ is computed from the $L$ histograms:

${\begin{aligned}\omega _{0}(t)&=\sum _{i=0}^{t-1}p(i)\\[4pt]\omega _{1}(t)&=\sum _{i=t}^{L-1}p(i)\end{aligned}}$

Otsu shows that minimizing the intra-class variance is the same as maximizing inter-class variance:

${\begin{aligned}\sigma _{b}^{2}(t)&=\sigma ^{2}-\sigma _{w}^{2}(t)=\omega _{0}(\mu _{0}-\mu _{T})^{2}+\omega _{1}(\mu _{1}-\mu _{T})^{2}\\&=\omega _{0}(t)\omega _{1}(t)\left[\mu _{0}(t)-\mu _{1}(t)\right]^{2}\end{aligned}}$

which is expressed in terms of class probabilities $\omega$ and class means $\mu$.

while the class mean $\mu _{0,1,T}(t)$ is:

${\begin{aligned}\mu _{0}(t)&=\sum _{i=0}^{t-1}i{\frac {p(i)}{\omega _{0}}}\\[4pt]\mu _{1}(t)&=\sum _{i=t}^{L-1}i{\frac {p(i)}{\omega _{1}}}\\\mu _{T}&=\sum _{i=0}^{L-1}ip(i)\end{aligned}}$

The following relations can be easily verified:

${\begin{aligned}\omega _{0}\mu _{0}+\omega _{1}\mu _{1}&=\mu _{T}\\\omega _{0}+\omega _{1}&=1\end{aligned}}$

The class probabilities and class means can be computed iteratively. This idea yields an effective algorithm.

Algorithm

1. Compute histogram and probabilities of each intensity level
2. Set up initial $\omega _{i}(0)$ and $\mu _{i}(0)$
3. Step through all possible thresholds $t=1,\ldots$ maximum intensity
   1. Update $\omega _{i}$ and $\mu _{i}$
   2. Compute $\sigma _{b}^{2}(t)$
4. Desired threshold corresponds to the maximum $\sigma _{b}^{2}(t)$

MATLAB implementation

histogramCounts is a 256-element histogram of a grayscale image different gray-levels (typical for 8-bit images). level is the threshold for the image (double).

function level = otsu(histogramCounts)
total = sum(histogramCounts); % '''total''' is the number of pixels in the given image. 
%% OTSU automatic thresholding method
sumB = 0;
wB = 0;
maximum = 0.0;
sum1 = dot( (0:255), histogramCounts); 
for ii=1:256
    wB = wB + histogramCounts(ii);
    if (wB == 0)
        continue;
    end
    wF = total - wB;
    if (wF == 0)
        break;
    end
    sumB = sumB +  (ii-1) * histogramCounts(ii);
    mB = sumB / wB;
    mF = (sum1 - sumB) / wF;
    between = wB * wF * (mB - mF) * (mB - mF);
    if ( between >= maximum )
        level = ii;
        maximum = between;
    end
end
end

Matlab have built-in functions graythresh() and multithresh() in Image Processing Toolbox which are implemented with Otsu's method and Multi Otsu's method, respectively.

Another approach with vectorized method (could be easily converted into python matrix-array version for GPU processing)

function  [threshold_otsu] = Thresholding_Otsu( Image)
%Intuition:
%(1)pixels are divided into two groups
%(2)pixels within each group are very similar to each other 
%   Parameters:
%   t : threshold 
%   r : pixel value ranging from 1 to 255
%   q_L, q_H : the number of lower and higher group respectively
%   sigma : group variance
%   miu : group mean
%   Author: Lei Wang
%   Date  : 22/09/2013
%   References : Wikepedia, 
%   for multi children Otsu method, please visit : https://drive.google.com/file/d/0BxbR2jt9XyxteF9fZ0NDQ0dKQkU/view?usp=sharing
%   This is my original work

nbins = 256;
counts = imhist(Image,nbins);
p = counts / sum(counts);

for t = 1 : nbins
   q_L = sum(p(1 : t));
   q_H = sum(p(t + 1 : end));
   miu_L = sum(p(1 : t) .* (1 : t)') / q_L;
   miu_H = sum(p(t + 1 : end) .* (t + 1 : nbins)') / q_H;
   sigma_b(t) = q_L * q_H * (miu_L - miu_H)^2;
end

[~,threshold_otsu] = max(sigma_b(:));
end

The implementation has a little redundancy of computation. But since Otsu method is fast, the implementation is acceptable and easy to understand. While in some environment, since we employ vectorisation form, loop computation might be faster. This method can be easily converted to multi-threshold method, with architecture minimum heap—children labels.

JavaScript implementation

Variant 1

NB: The input argument total is the number of pixels in the given image. The input argument histogram is a 256-element histogram of a grayscale image different gray-levels (typical for 8-bit images). This function outputs the threshold for the image.

function otsu(histogram, total) {
    var sum = 0;
    for (var i = 1; i < histogram.length + 1; ++i) //normally it will be 255 but sometimes we want to change step
        sum += i * histogram[i];
    var sumB = 0;
    var wB = 0;
    var wF = 0;
    var mB;
    var mF;
    var max = 0.0;
    var between = 0.0;
    var threshold1 = 0.0;
    var threshold2 = 0.0;
    for (var i = 0; i < histogram.length; ++i) {
        wB += histogram[i];
        if (wB == 0)
            continue;
        wF = total - wB;
        if (wF == 0)
            break;
        sumB += i * histogram[i];
        mB = sumB / wB;
        mF = (sum - sumB) / wF;
        between = wB * wF * (mB - mF) * (mB - mF);
        if ( between >= max ) {
            threshold1 = i;
            if ( between > max ) {
                threshold2 = i;
            }
            max = between;            
        }
    }
    return ( threshold1 + threshold2 ) / 2.0;
}

Variant 2

function otsu(histogram, pixelsNumber) {
	var sum = 0
	  , sumB = 0
	  , wB = 0
	  , wF = 0
      , mB
	  , mF
	  , max = 0
	  , between
	  , threshold = 0;
	for (var i = 0; i < 256; ++i) {
      wB += histogram[i];
      if (wB == 0)
        continue;
      wF = pixelsNumber - wB;
      if (wF == 0)
        break;
      sumB += i * histogram[i];
      mB = sumB / wB;
      mF = (sum - sumB) / wF;
      between = wB * wF * Math.pow(mB - mF, 2);
      if (between > max) {
        max = between;
        threshold = i;
      }
    }
    return threshold;
}

// To test: open any image in browser and run code in console
var im = document.getElementsByTagName('img')[0]
  , cnv = document.createElement('canvas')
  , ctx = cnv.getContext('2d');
cnv.width = im.width;
cnv.height = im.height;
ctx.drawImage(im, 0, 0);
var imData = ctx.getImageData(0, 0, cnv.width, cnv.height)
  , histogram = Array(256)
  , i
  , red
  , green
  , blue
  , gray;
for (i = 0; i < 256; ++i)
	histogram[i] = 0;
for (i = 0; i < imData.data.length; i += 4) {
  red = imData.data[i];
  blue = imData.data[i + 1];
  green = imData.data[i + 2];
  // alpha = imData.data[i + 3];
  // https://en.wikipedia.org/wiki/Grayscale
  gray = red * .2126 + green * .07152 + blue * .0722;
  histogram[Math.round(gray)] += 1;
}
var threshold = otsu(histogram, imData.data.length / 4);
console.log("threshold = %s", threshold);
for (i = 0; i < imData.data.length; i += 4) {
  imData.data[i] = imData.data[i + 1] = imData.data[i + 2] =
    imData.data[i] >= threshold ? 255 : 0;
  // opacity 255 = 100%
  imData.data[i + 3] = 255;
}
ctx.putImageData(imData, 0, 0);
document.body.appendChild(cnv);
console.log("finished");

Result.

Limitations

Otsu’s method exhibits the relatively good performance if the histogram can be assumed to have bimodal distribution and assumed to possess a deep and sharp valley between two peaks. But if the object area is small compared with the background area, the histogram no longer exhibits bimodality. And if the variances of the object and the background intensities are large compared to the mean difference, or the image is severely corrupted by additive noise, the sharp valley of the gray level histogram is degraded. Then the possibly incorrect threshold determined by Otsu’s method results in the segmentation error. (Here we define the object size to be the ratio of the object area to the entire image area and the mean difference to be the difference of the average intensities of the object and the background)

From the experimental results, the performance of global thresholding techniques including Otsu’s method is shown to be limited by the small object size, the small mean difference, the large variances of the object and the background intensities, the large amount of noise added, and so on.

Improvements

There are many improvements focusing on different limitations for Otsu's method. One famous and effective way is known as two-dimensional Otsu's method. In this approach, the gray-level value of each pixel as well as the average value of its immediate neighborhood is studied so that the binarization results are greatly improved, especially for those images corrupted by noise.

At each pixel, the average gray-level value of the neighborhood is calculated. Let the gray level of a given picture be divided into $L$ values and the average gray level is also divided into the same $L$ values. Then a pair is formed: the pixel gray level and the average of the neighborhood. Each pair belongs to a 2-dimensional bin. The total number of bins is obviously $L\times L$. The total number of occurrence(frequency), $f_{ij}$, of a pair$(i,j)$ divided by the total number of pixels in the image $N$, defines the joint probability mass function in 2-dimensional histogram:

$P_{ij}={\frac {f_{ij}}{N}},\qquad \sum _{i=0}^{L-1}\sum _{j=0}^{L-1}P_{ij}=1$

And the 2-dimensional Otsu's method will be developed based on the 2-dimensional histogram as follows.

The probabilities of two classes can be denoted as:

${\begin{aligned}\omega _{0}&=\sum _{i=0}^{s-1}\sum _{j=0}^{t-1}P_{ij}\\\omega _{1}&=\sum _{i=s}^{L-1}\sum _{j=t}^{L-1}P_{ij}\end{aligned}}$

The intensity means value vectors of two classes and total mean vector can be expressed as follows:

${\begin{aligned}\mu _{0}&=[\mu _{0i},\mu _{0j}]^{T}=\left[\sum _{i=0}^{s-1}\sum _{j=0}^{t-1}i{\frac {P_{ij}}{\omega _{0}}},\sum _{i=0}^{s-1}\sum _{j=0}^{t-1}j{\frac {P_{ij}}{\omega _{0}}}\right]^{T}\\\mu _{1}&=[\mu _{1i},\mu _{1j}]^{T}=\left[\sum _{i=s}^{L-1}\sum _{j=t}^{L-1}i{\frac {P_{ij}}{\omega _{1}}},\sum _{i=s}^{L-1}\sum _{j=t}^{L-1}j{\frac {P_{ij}}{\omega _{1}}}\right]^{T}\\\mu _{T}&=[\mu _{Ti},\mu _{Tj}]^{T}=\left[\sum _{i=0}^{L-1}\sum _{j=0}^{L-1}iP_{ij},\sum _{i=0}^{L-1}\sum _{j=0}^{L-1}jP_{ij}\right]^{T}\end{aligned}}$

In most cases, the probability off-diagonal will be negligible so it's easy to verify:

$\omega _{0}+\omega _{1}\cong 1$

$\omega _{0}\mu _{0}+\omega _{1}\mu _{1}\cong \mu _{T}$

The inter-class discrete matrix is defined as

$S_{b}=\sum _{k=0}^{1}\omega _{k}[(\mu _{k}-\mu _{T})(\mu _{k}-\mu _{T})^{T}]$

The trace of discrete matrix can be expressed as

${\begin{aligned}&\operatorname {tr} (S_{b})\\[4pt]={}&\omega _{0}[(\mu _{0i}-\mu _{Ti})^{2}+(\mu _{0j}-\mu _{Tj})^{2}]+\omega _{1}[(\mu _{1i}-\mu _{Ti})^{2}+(\mu _{1j}-\mu _{Tj})^{2}]\\[4pt]={}&{\frac {(\mu _{Ti}\omega _{0}-\mu _{i})^{2}+(\mu _{Tj}\omega _{0}-\mu _{j})^{2}}{\omega _{0}(1-\omega _{0})}}\end{aligned}}$

where

$\mu _{i}=\sum _{i=0}^{s}\sum _{j=0}^{t}iP_{ij}$

$\mu _{j}=\sum _{i=0}^{s}\sum _{j=0}^{t}jP_{ij}$

Similar to one-dimensional Otsu's method, the optimal threshold $(s,t)$ is obtained by maximizing $\operatorname {tr} (S_{b})$.

Algorithm

The $s$ and $t$ is obtained iteratively which is similar with one-dimensional Otsu's method. The values of $s$ and $t$ are changed till we obtain the maximum of $\operatorname {tr} (S_{b})$, that is

max,s,t = 0;
for ss: 0 to L-1 do
    for tt: 0 to L-1 do
        evaluate tr(S_b);
        if tr(S_b) > max
            max = tr(S,b);
            s = ss;
            t = tt;
        end if
    end for
end for
return s,t;

Notice that for evaluating $\operatorname {tr} (S_{b})$, we can use a fast recursive dynamic programming algorithm to improve time performance. However, even with the dynamic programming approach, 2d Otsu's method still has large time complexity. Therefore, many researches have been done to reduce the computation cost.

If summed area tables are used to build the 3 tables, sum over P_i_j, sum over i * P_i_j, and sum over j * P_i_j, then the runtime complexity is the maximum of (O(N_pixels), O(N_bins*N_bins)). Note that if only coarse resolution is needed in terms of threshold, N_bins can be reduced.

See also: Summed_area_table

Matlab implementation

function inputs and output:

hists is a $256\times 256$ 2D-histogram of grayscale value and neighborhood average grayscale value pair.

total is the number of pairs in the given image.

threshold is the threshold obtained.

function threshold = 2D_otsu(hists, total)
maximum = 0.0;
threshold = 0;
helperVec = 0:255;
mu_t0 = sum(sum(repmat(helperVec',1,256).*hists));
mu_t1 = sum(sum(repmat(helperVec,256,1).*hists));
p_0 = zeros(256);
mu_i = p_0;
mu_j = p_0;
for ii = 1:256
    for jj = 1:256
        if jj == 1
            if ii == 1
                p_0(1,1) = hists(1,1);
            else
                p_0(ii,1) = p_0(ii-1,1) + hists(ii,1);
                mu_i(ii,1) = mu_i(ii-1,1)+(ii-1)*hists(ii,1);
                mu_j(ii,1) = mu_j(ii-1,1);
            end
        else
            p_0(ii,jj) = p_0(ii,jj-1)+p_0(ii-1,jj)-p_0(ii-1,jj-1)+hists(ii,jj);
            mu_i(ii,jj) = mu_i(ii,jj-1)+mu_i(ii-1,jj)-mu_i(ii-1,jj-1)+(ii-1)*hists(ii,jj);
            mu_j(ii,jj) = mu_j(ii,jj-1)+mu_j(ii-1,jj)-mu_j(ii-1,jj-1)+(jj-1)*hists(ii,jj);
        end

        if (p_0(ii,jj) == 0)
            continue;
        end
        if (p_0(ii,jj) == total)
            break;
        end
        tr = ((mu_i(ii,jj)-p_0(ii,jj)*mu_t0)^2 + (mu_j(ii,jj)-p_0(ii,jj)*mu_t0)^2)/(p_0(ii,jj)*(1-p_0(ii,jj)));

        if ( tr >= maximum )
            threshold = ii;
            maximum = tr;
        end
    end
end
end

External links

• Implementation of Otsu's thresholding method as GIMP-plugin using Script-Fu (a Scheme-based language).
• Lecture notes on thresholding – covers the Otsu method.
• A plugin for ImageJ using Otsu's method to do the threshold.
• A full explanation of Otsu's method with a working example and Java implementation.
• Implementation of Otsu's method in ITK
• Otsu Thresholding in C# A straightforward C# implementation with explanation.
• Otsu's method using MATLAB