UNIT II FEATURE EXTRACTION AND FEATURE SEGMENTATION

Feature Extraction -Edges - Canny, LOG, DOG; Line detectors (Hough Transform), Corners - Harris
and Hessian Affine, Orientation Histogram, SIFT, SURF, HOG, GLOH, Scale-Space Analysis- Image
Pyramids and Gaussian derivative filters, Gabor Filters and DWT. Image Segmentation -Region Growing,
Edge Based approaches to segmentation, Graph-Cut, Mean-Shift, MRFs, Texture Segmentation.

What Is Feature Extraction?

Feature extraction refers to the process of transforming raw data into numerical features that can be
processed while preserving the information in the original data set. It yields better results than applying
machine learning directly to the raw data.

Feature extraction can be accomplished manually or automatically:

Manual feature extraction requires identifying and describing the features that are relevant for a
given problem and implementing a way to extract those features. In many situations, having a good
understanding of the background or domain can help make informed decisions as to which features could
be useful. Over decades of research, engineers and scientists have developed feature extraction methods
for images, signals, and text. An example of a simple feature is the mean of a window in a signal.
Automated feature extraction uses specialized algorithms or deep networks to extract features
automatically from signals or images without the need for human intervention. This technique can be very
useful when you want to move quickly from raw data to developing machine learning algorithms.
Wavelet scattering is an example of automated feature extraction.
With the ascent of deep learning, feature extraction has been largely replaced by the first layers of deep
networks – but mostly for image data. For signal and time-series applications, feature extraction remains
the first challenge that requires significant expertise before one can build effective predictive models.

Feature Extraction for Signals and Time Series Data

Feature extraction identifies the most discriminating characteristics in signals, which a machine learning
or a deep learning algorithm can more easily consume. Training machine learning or deep learning
directly with raw signals often yields poor results because of the high data rate and information
redundancy.

Schematic process for applying feature extraction to signals and time series data for a machine learning
classifier.

Signal features and time-frequency transformations

When analyzing signals and sensor data, Signal Processing Toolbox™ and Wavelet
Toolbox™ provide functions that let you measure common distinctive features of a signal in the time,
frequency, and time-frequency domains. You can apply pulse and transition metrics, measure signal-to-
noise ratio (SNR), estimate spectral entropy and kurtosis, and compute power spectra.

Time-frequency transformations, such as the short-time Fourier transform (STFT) can be used as
signal representations for training data in machine learning and deep learning models. For example,
convolutional neural networks (CNNs) are commonly used on image data and can successfully learn from
the 2D signal representations returned by time-frequency transformations.
Spectrogram of a signal using short-time Fourier transform. Spectrogram shows variation of frequency
content over time.

Other time-frequency transformations can be used, depending on the specific application or the
characteristics. For example, the constant-Q transform (CQT) provides a logarithmically spaced
frequency distribution; the continuous wavelet transform (CWT) is usually effective at identifying short
transients in non-stationary signals.

Features for audio applications and predictive maintenance

Audio Toolbox™ provides a collection of time-frequency transformations including Mel
spectrograms, octave and gammatone filter banks, and discrete cosine transform (DCT), that are often
used for audio, speech, and acoustics. Other popular feature extraction methods for these types of signals
include Mel frequency cepstral coefficients (MFCC), gammatone cepstral coefficients (GTCC), pitch,
harmonicity, and different types of audio spectral descriptors. The Audio Feature Extractor tool can help
select and extract different audio features from the same source signal while reusing any intermediate
computations for efficiency.

For engineers developing applications for condition monitoring and predictive maintenance,
the Diagnostic Feature Designer app in Predictive Maintenance Toolbox™ lets you extract, visualize, and
rank features to design condition indicators for monitoring machine health.
Diagnostic Feature Designer App lets you design and compare features to discriminate between nominal
and faulty systems.

Automated feature extraction methods

Automated feature extraction is a part of the complete AutoML workflow that delivers optimized
models. The workflow involves three simple steps that automate feature selection, model selection, and
hyperparameter tuning.

New high-level methods have emerged to automatically extract features from signals.
Autoencoders, wavelet scattering, and deep neural networks are commonly used to extract features and
reduce dimensionality of the data.

Wavelet scattering networks automate the extraction of low-variance features from real-valued
time series and image data. This approach produces data representations that minimize differences within
a class while preserving discriminability across classes. Wavelet scattering works well when you do not
have a lot of data to begin with.

Feature Extraction for Image Data

Feature extraction for image data represents the interesting parts of an image as a compact feature
vector. In the past, this was accomplished with specialized feature detection, feature extraction, and
feature matching algorithms. Today, deep learning is prevalent in image and video analysis, and has
become known for its ability to take raw image data as input, skipping the feature extraction step.
Regardless of which approach you take, computer vision applications such as image registration, object
detection and classification, and content-based image retrieval, all require effective representation of
image features – either implicitly by the first layers of a deep network, or explicitly applying some of the
longstanding image feature extraction techniques.

Detecting an object (left) in a cluttered scene (right) using a combination of feature detection, feature
extraction, and matching.
Feature extraction techniques provided by Computer Vision Toolbox™ and Image Processing
Toolbox™ include:

 Histogram of oriented gradients (HOG)

 Speeded-up robust features (SURF)
 Local binary pattern (LBP) features

Histogram of oriented gradients (HOG) feature extraction of image (top). Feature vectors of different
sizes are created to represent the image by varying cell size (bottom)

Edges :
Edgesare significant local changes of intensity in a digital image. An edge can be defined as a set of
connected pixels that forms a boundary between two disjoint regions. There are three types of edges:
 Horizontal edges
 Vertical edges
 Diagonal edges
Edge Detection is a method of segmenting an image into regions of discontinuity. It is a widely used
technique in digital image processing like
 pattern recognition
 image morphology
 feature extraction
Edge detection allows users to observe the features of an image for a significant change in the gray level.
This texture indicating the end of one region in the image and the beginning of another. It reduces the
amount of data in an image and preserves the structural properties of an image.
Edge Detection Operators are of two types:
 Gradient – based operator which computes first-order derivations in a digital image like, Sobel
operator, Prewitt operator, Robert operator
 Gaussian – based operator which computes second-order derivations in a digital image like,
Canny edge detector, Laplacian of Gaussian
 Sobel Operator: It is a discrete differentiation operator. It computes the gradient approximation of
image intensity function for image edge detection. At the pixels of an image, the Sobel operator produces
either the normal to a vector or the corresponding gradient vector. It uses two 3 x 3 kernels or masks
which are convolved with the input image to calculate the vertical and horizontal derivative
approximations respectively –

Advantages:

1. Simple and time efficient computation

2. Very easy at searching for smooth edges
Limitations:

1. Diagonal direction points are not preserved always

2. Highly sensitive to noise
3. Not very accurate in edge detection
4. Detect with thick and rough edges does not give appropriate results
Prewitt Operator: This operator is almost similar to the sobel operator. It also detects vertical and
horizontal edges of an image. It is one of the best ways to detect the orientation and magnitude of an
image. It uses the kernels or masks –

Advantages:

1. Good performance on detecting vertical and horizontal edges

2. Best operator to detect the orientation of an image

Limitations:

1. The magnitude of coefficient is fixed and cannot be changed

2. Diagonal direction points are not preserved always
 Robert Operator: This gradient-based operator computes the sum of squares of the differences
between diagonally adjacent pixels in an image through discrete differentiation. Then the gradient
approximation is made. It uses the following 2 x 2 kernels or masks –

Advantages:

1. Detection of edges and orientation are very easy

2. Diagonal direction points are preserved
Limitations:
1. Very sensitive to noise
2. Not very accurate in edge detection
Marr-Hildreth Operator or Laplacian of Gaussian (LoG): It is a gaussian-based operator which
uses the Laplacian to take the second derivative of an image. This really works well when the transition of
the grey level seems to be abrupt. It works on the zero-crossing method i.e when the second-order
derivative crosses zero, then that particular location corresponds to a maximum level. It is called an edge
location. Here the Gaussian operator reduces the noise and the Laplacian operator detects the sharp
edges.
The Gaussian function is defined by the formula:

Where

is the standard deviation.

And the LoG operator is computed from

Advantages:

1. Easy to detect edges and their various orientations

2. There is fixed characteristics in all directions
Limitations:

1. Very sensitive to noise

2. The localization error may be severe at curved edges
3. It generates noisy responses that do not correspond to edges, so-called “false edges”
Canny Operator: It is a gaussian-based operator in detecting edges. This operator is not susceptible
to noise. It extracts image features without affecting or altering the feature. Canny edge detector have
advanced algorithm derived from the previous work of Laplacian of Gaussian operator. It is widely used
an optimal edge detection technique. It detects edges based on three criteria:
1. Low error rate
2. Edge points must be accurately localized
3. There should be just one single edge response
Advantages:
1. It has good localization
2. It extract image features without altering the features
3. Less Sensitive to noise

Limitations:
1. There is false zero crossing
2. Complex computation and time consuming
Some Real-world Applications of Image Edge Detection:
 medical imaging, study of anatomical structure
 locate an object in satellite images
 automatic traffic controlling systems
 face recognition, and fingerprint recognition

CANNY:
A Canny edge detector is a multi-step algorithm to detect the edges for any input image. It
involves the below-mentioned steps to be followed while detecting edges of an image.

1. Removal of noise in input image using a Gaussian filter.

2. Computing the derivative of Gaussian filter to calculate the gradient of image pixels to obtain
magnitude along x and y dimension.

3. Considering a group of neighbors for any curve in a direction perpendicular to the given edge, suppress
the non-max edge contributor pixel points.

4. Lastly, use the Hysteresis Thresholding method to preserve the pixels higher than the gradient
magnitude and neglect the ones lower than the low threshold value.

Before deep-diving into the steps below are the three conclusions that J.K Canny who derived
the algorithm :

– Good Detection: The optimal detector must eliminate the possibility of getting false positives and false
negatives.
– Good Localisation: The detected edges must be as close to true edges.

– Single Response Constraint: The detector must return one point only for each edge point.

Steps to followed during Canny Algorithm:

Noise Removal or Image Smoothing:

During the noise present, the pixel may not be close to being similar to its neighboring pixels. This
might result in obtaining improper or inappropriate detection of edges. To avoid the same, we use the
Gausian filter, which is convolved with the image and removes the noise, preventing the desired edges in
output images.

In the below example, we are convolving gausian filter or kernel g(x,y) with an image I. Here we
wish to make sure that any given pixel must be similar to its neighboring pixels in output and so we use
the matrix [1 1 1] to maintain the similarity between pixels and remove the noise.

g(x,y)= Gausian Distribution

I = input image

Derivative :

Calculate the derivative of filter w.r.t X and Y dimensions and convolve it with I to give the
gradient magnitude along the dimensions. Also, the direction of the image can be calculated using the
tangent of the angle between the two dimensions.

The above convolution results in a gradient vector that has magnitude and direction.
Below is an example of Gausian Derivatives, which finally contribute to edges in output images.

Non-Max Suppression

Along an edge, it is generally observed that few points make the visibility of the edge clearer. So
we can neglect those edge points which don’t contribute more towards feature visibility. To achieve the
same, we use the Non Maximum Suppression method. Here we mark the points on the curve of the edge
where the magnitude is largest. This can be obtained by looking for a maximum along with a slice normal
to the curve.

Consider the edge in the below figure, which has three edge points. Assume point (x,y) as the
point having the largest gradient of edge. Check for the edge points in the direction perpendicular to the
edge and verify if their gradient is less than (x,y). If the values are less than (x,y) gradient, then we can
suppress those non-maxima points along the curve.
Hysteresis Thresholding :

If the gradient at a pixel is :

– Above “High” declare it as an ‘edge pixel.’

– Below, “Low” declares it as a ‘non-edge pixel.’
– Between “low” and “high.”

 Consider its neighbors iteratively then declare it an “edge pixel” if it’s connected to an “edge
pixel” or via pixels between “low” and “high.”

Advantages:

1. The presence of Gaussian filter allows removing of any noise in an image.

2. The signal can be enhanced with respect to the noise ratio by non-maxima suppression method which
results in one pixel wide ridges as the output.
3.Detects the edges in a noisy state by applying the thresholding method.
4.The effectiveness can be adjusted by using parameters.
5. It gives a good localization, response and is immune to a noisy environment.

Disadvantages:

1. The primary disadvantage of using Canny edge detector is that it consumes a lot of time due to its
complex computation.
2. It is difficult to implement to reach the real-time response.

Laplacian of Gaussian(LOG):
The Laplacian of Gaussian (LoG) is a common variation of the Laplacian filter. Laplacian of
Gaussian is intended to counter the noise sensitivity of the regular Laplacian filter.
Laplacian of Gaussian attempts to remove image noise by implementing image smoothing by means of
a Gaussian blur. In order to optimize performance we can calculate a single matrix representing
a Gaussian blur and Laplacian matrix.
Which was invented by Marr and Hildreth (1980) who combined Gaussian filtering with the Laplacian.
This algorithm is not used frequently inmachine vision.
The second derivative of smoothed step edge is a function that crosses zero at the location of edge.
public static Bitmap
LaplacianOfGaussian(this Bitmap sourceBitmap)
{
Bitmap resultBitmap =
ExtBitmap.ConvolutionFilter(sourceBitmap,
Matrix.LaplacianOfGaussian,
1.0, 0, true);

return resultBitmap;
}
public static double[,] LaplacianOfGaussian
{
get
{
return new double[,]
{ { 0, 0, -1, 0, 0 },
{ 0, -1, -2, -1, 0 },
{ -1, -2, 16, -2, -1 },
{ 0, -1, -2, -1, 0 },
{ 0, 0, -1, 0, 0 } };
}
}
Laplacian of Gaussian

Laplacian (3×3) of Gaussian (3×3)

Different matrix variations can be combined in an attempt to produce results best suited to the
input image. In this case we first apply a 3×3 Gaussian blur followed by a 3×3 Laplacian filter.
public static Bitmap
Laplacian3x3OfGaussian3x3Filter(this Bitmap sourceBitmap)
{
Bitmap resultBitmap =
ExtBitmap.ConvolutionFilter(sourceBitmap,
Matrix.Gaussian3x3,
1.0 / 16.0, 0, true);

resultBitmap = ExtBitmap.ConvolutionFilter(resultBitmap,
Matrix.Laplacian3x3, 1.0, 0, false);
 return resultBitmap;
}
public static double[,] Laplacian3x3
{
get
{
return new double[,]
{ { -1, -1, -1, },
{ -1, 8, -1, },
{ -1, -1, -1, }, };
}
}
public static double[,] Gaussian3x3
{
get
{
return new double[,]
{ { 1, 2, 1, },
{ 2, 4, 2, },
{ 1, 2, 1, } };
}
}
Laplacian 3×3 Of Gaussian 3×3

Laplacian (3×3) of Gaussian (5×5 – Type 1)

In this scenario we apply a variation of a 5×5 Gaussian blur followed by a 3×3 Laplacian filter.
public static Bitmap
Laplacian3x3OfGaussian5x5Filter1(this Bitmap sourceBitmap)
{
Bitmap resultBitmap =
ExtBitmap.ConvolutionFilter(sourceBitmap,
Matrix.Gaussian5x5Type1,
1.0 / 159.0, 0, true);

resultBitmap = ExtBitmap.ConvolutionFilter(resultBitmap,
Matrix.Laplacian3x3, 1.0, 0, false);

return resultBitmap;
}
public static double[,] Laplacian3x3
{
get
{
return new double[,]
{ { -1, -1, -1, },
{ -1, 8, -1, },
{ -1, -1, -1, }, };
}
}
public static double[,] Gaussian5x5Type1
{
get
{
return new double[,]
{ { 2, 04, 05, 04, 2 },
{ 4, 09, 12, 09, 4 },
{ 5, 12, 15, 12, 5 },
{ 4, 09, 12, 09, 4 },
{ 2, 04, 05, 04, 2 }, };
}
}
Laplacian 3×3 Of Gaussian 5×5 – Type 1

Laplacian (3×3) of Gaussian (5×5 – Type 2)

The following implementation is very similar to the previous implementation. Applying a
variation of a 5×5 Gaussian blur results in slight differences.
public static Bitmap
Laplacian3x3OfGaussian5x5Filter2(this Bitmap sourceBitmap)
{
Bitmap resultBitmap =
ExtBitmap.ConvolutionFilter(sourceBitmap,
Matrix.Gaussian5x5Type2,
1.0 / 256.0, 0, true);

resultBitmap = ExtBitmap.ConvolutionFilter(resultBitmap,
Matrix.Laplacian3x3, 1.0, 0, false);

return resultBitmap;
}
public static double[,] Laplacian3x3
{
get
{
return new double[,]
{ { -1, -1, -1, },
{ -1, 8, -1, },
{ -1, -1, -1, }, };
}
}
public static double[,] Gaussian5x5Type2
{
get
{
return new double[,]
{ { 1, 4, 6, 4, 1 },
{ 4, 16, 24, 16, 4 },
{ 6, 24, 36, 24, 6 },
{ 4, 16, 24, 16, 4 },
{ 1, 4, 6, 4, 1 }, };
}
}
Laplacian 3×3 Of Gaussian 5×5 – Type 2

Laplacian (5×5) of Gaussian (3×3)

This variation of the Laplacian of Gaussian filter implements a 3×3 Gaussian blur, followed by
a 5×5 Laplacian matrix. The resulting image appears significantly brighter when compared to
a 3×3 Laplacian matrix.
public static Bitmap
Laplacian5x5OfGaussian3x3Filter(this Bitmap sourceBitmap)
{
Bitmap resultBitmap =
ExtBitmap.ConvolutionFilter(sourceBitmap,
Matrix.Gaussian3x3,
1.0 / 16.0, 0, true);

resultBitmap = ExtBitmap.ConvolutionFilter(resultBitmap,
Matrix.Laplacian5x5, 1.0, 0, false);

return resultBitmap;
}
public static double[,] Laplacian5x5
{
get
{
return new double[,]
{ { -1, -1, -1, -1, -1, },
{ -1, -1, -1, -1, -1, },
{ -1, -1, 24, -1, -1, },
{ -1, -1, -1, -1, -1, },
{ -1, -1, -1, -1, -1 } };
}
}
public static double[,] Gaussian3x3
{
get
{
return new double[,]
{ { 1, 2, 1, },
{ 2, 4, 2, },
{ 1, 2, 1, } };
}
}
Laplacian 5×5 Of Gaussian 3×3

Laplacian (5×5) of Gaussian (5×5 – Type 1)

Implementing a larger Gaussian blur matrix results in a higher degree of image smoothing,
equating to less image noise.
public static Bitmap
Laplacian5x5OfGaussian5x5Filter1(this Bitmap sourceBitmap)
{
Bitmap resultBitmap =
ExtBitmap.ConvolutionFilter(sourceBitmap,
Matrix.Gaussian5x5Type1,
1.0 / 159.0, 0, true);

resultBitmap = ExtBitmap.ConvolutionFilter(resultBitmap,
Matrix.Laplacian5x5, 1.0, 0, false);

return resultBitmap;
}
public static double[,] Laplacian5x5
{
get
{
return new double[,]
{ { -1, -1, -1, -1, -1, },
{ -1, -1, -1, -1, -1, },
{ -1, -1, 24, -1, -1, },
{ -1, -1, -1, -1, -1, },
{ -1, -1, -1, -1, -1 } };
}
}
public static double[,] Gaussian5x5Type1
{
get
{
return new double[,]
{ { 2, 04, 05, 04, 2 },
{ 4, 09, 12, 09, 4 },
{ 5, 12, 15, 12, 5 },
{ 4, 09, 12, 09, 4 },
{ 2, 04, 05, 04, 2 }, };
}
}
Laplacian 5×5 Of Gaussian 5×5 – Type 1

Laplacian (5×5) of Gaussian (5×5 – Type 2)

The variation of Gaussian blur most applicable when implementing a Laplacian of Gaussian filter
depends on image noise expressed by a source image. In this scenario the first variations (Type 1) appears
to result in less image noise.
public static Bitmap
Laplacian5x5OfGaussian5x5Filter2(this Bitmap sourceBitmap)
{
Bitmap resultBitmap =
ExtBitmap.ConvolutionFilter(sourceBitmap,
Matrix.Gaussian5x5Type2,
1.0 / 256.0, 0, true);

resultBitmap =
ExtBitmap.ConvolutionFilter(resultBitmap,
Matrix.Laplacian5x5,
 1.0, 0, false);

return resultBitmap;
}
public static double[,] Laplacian5x5
{
get
{
return new double[,]
{ { -1, -1, -1, -1, -1, },
{ -1, -1, -1, -1, -1, },
{ -1, -1, 24, -1, -1, },
{ -1, -1, -1, -1, -1, },
{ -1, -1, -1, -1, -1 } }; } }
public static double[,] Gaussian5x5Type2
{ get {
return new double[,]
{ { 1, 4, 6, 4, 1 },
{ 4, 16, 24, 16, 4 },
{ 6, 24, 36, 24, 6 },
{ 4, 16, 24, 16, 4 },
{ 1, 4, 6, 4, 1 }, }; }
}
Laplacian 5×5 Of Gaussian 5×5 – Type 2

Advantages:

Easy to detect edges and their orientations. In this cross operator detection of edges and their
orientations is said- to be simple due to the approximation of the gradient magnitude is simple.
The second advantage is the fixed characteristics in all directions.

Disadvantages:

It is very sensitive to the noise. In detecting the edges and their orientations are increased in the
noise to the image this will eventually degrade the magnitude of the edges.
The second disadvantage is that, the operation gets diffractedby some of the existing edges in the noisy
image.
APPLICATIONS:
The behavior of the LoG zero crossing edge detector is largely governed by the standard deviation
of the Gaussian used in the LoG filter.
The higher this value is set,the more smaller features will be smoothed out of existence, and hence
fewer zero crossings will be produced.
Hence, this parameter can be set to remove unwanted detail or noise as desired.
The idea that at different smoothing levels different sized features become prominent is referred to
as ‘scale’.

DIFFERENCE OF GAUSSIAN(DOG):

Introduction
Difference of Gaussians (DoG) filtering is a very useful technique for enhancing the appearance
of small spots and edges in an image. It’s quite straightforward, but time consuming to apply manually
very often – and you might need to experiment with different filter sizes to get good results. This makes it
an excellent candidate for a macro.

Recording a macro
Rather than diving into writing the code, the fastest way to get started is to have ImageJ do most
of the hard work itself. Then you only need to fix up the result. The procedure is as follows:

 Open up an example (2D, non-color) image to use, ideally one including small spot-like or otherwise
round objects. I have used File ‣ Open samples ‣ HeLa Cells, after extracting the red channel only.
 Start the Macro Recorder by choosing Plugins ‣ Macros ‣ Record. Make sure that Record:
Macro appears at the top of this window (see the drop-down list). Every subsequent click you make that
has a corresponding macro command will result in the command being added to the window.
 Convert your image to 32-bit. This will reduce inaccuracies due to rounding whenever the filtering is
applied.
 Duplicate the image.
 Apply Process ‣ Filters ‣ Gaussian Blur… to one of the images (it doesn’t matter if it’s the original
or the duplicate), using a small sigma (e.g. 1) for noise suppression.
 Apply Process ‣ Filters ‣ Gaussian Blur… to the other image, using a larger sigma (e.g. 2).
 Run Process ‣ Image Calculator… and subtract the second filtered image from the first. This
produces the ‘difference of Gaussians’ filtered image, in which small features should appear prominently
and the background is removed.
o Be careful to choose the correct image titles and subtraction operation in the Image Calculator!
 Press the Create button on the macro recorder. This should cause a text file containing the recorded
macro to be opened in Fiji’s Script Editor (which you can find under File ‣ New ‣ Script…).
 Save the text file. The file name should end with the extension .ijm (for ‘ImageJ Macro’), and
include an underscore character somewhere within it.
Now you have a macro!

`You could try it out by pressing the tempting Run button, but this isn’t guaranteed to work (yet). Our
macro remains quite brittle: it depends upon exact image names and might become easily confused. We
will fix this soon.

As an alternative,
 Close Fiji completely (to ensure nothing remains from this session)
 Reopen Fiji
 Open the original image you used
 Open the macro (you can just drag it onto the ImageJ toolbar)
Now the Run button will hopefully give you the same result as when you applied the commands
manually. If not, keep reading anyway and the following steps should fix it.

Cleaning up the code

Now reopen your macro in the Script Editor. It should look something like mine:

Input image
 Output image

For easier copy-and-paste, the content is below:

run("32-bit");
//run("Brightness/Contrast...");
run("Enhance Contrast", "saturated=0.35");
run("Duplicate...", " ");
run("Gaussian Blur...", "sigma=1");
selectWindow("C1-hela-cells.tif");
run("Gaussian Blur...", "sigma=2");
imageCalculator("Subtract create", "C1-hela-cells-1.tif","C1-hela-cells.tif");
selectWindow("Result of C1-hela-cells-1.tif");
Your code is probably not identical, and may well be better. One problem with automatically generated
macros is that they contain (almost) everything – often including a lot of errant clicking, or other non-
essential steps. For example, I changed the contrast of an image, but this was only to look at it – and it
does not need to be included in the macro.

After deleting the unnecessary lines, I get:

run("32-bit");
run("Duplicate...", "title=C1-hela-cells-1.tif");
run("Gaussian Blur...", "sigma=1");
selectWindow("C1-hela-cells.tif");
run("Gaussian Blur...", "sigma=2");
imageCalculator("Subtract create", "C1-hela-cells-1.tif","C1-hela-cells.tif");

Understanding the code

You can most likely work out what the macro is doing, if not necessarily the terminology, just by
looking at it. Taking the first line, run is a **function_ that tells ImageJ to execute a command,
while 32-bit is a piece of text (called a string) that tells it which command. Functions always tell
ImageJ to do something or give you information, and can be recognized because they are normally
followed by parentheses. Strings are recognizable both because they are inside double inverted commas
and the script editor shows them in a different color. Notice also that each line needs to end with a
semicolon so that the macro interpreter knows the line is over.

Functions can require different numbers of pieces of information to do their work. At a

minimum, run needs to know the name of the command and the image to which it should be applied –
which here is taken to be whichever image is currently active, i.e. the one that was selected most recently.
But if the command being used by run requires extra information of its own, then this is included as an
extra string. Therefore
run("Duplicate...", "title=C1-hela-cells-1.tif");
informs the Duplicate command that the image it creates should be called C1-hela-cells-1.tif, and

run("Gaussian Blur...", "sigma=1");

ensures that Process ‣ Filters ‣ Gaussian Blur… is executed with a sigma value of 1.

selectWindow is another function, added to the macro whenever you click on a particular
window to activate it, and which requires the name of the image window to make active. From this you
can see that my example file name was C1-hela-cells.tif. Without this line, the duplicated image would be
filtered twice – and the original not at all.

Finally, the Image Calculator command is special enough to get its own function in the macro
language, imageCalculator. The first string it is given tells it both what sort of calculation to do, and
that it should create a new image for the result – rather than replacing one of the existing images. The
next two strings give it the titles of the images needed for the calculation.

Removing title dependancies

The fact that the original image title appears in the above macro is a problem: if you try to run it
on another image, you are likely to find that it does not work because selectWindow cannot find what
it’s looking for. So the next step is to remove this title dependency so that the macro can be applied to any
(2D) image.

There are two ways to go about this. One is to insert a line that tells the macro the title of the
image being processed at the start, e.g.

titleOrig = getTitle();
where getTitle() is an example of a function that asks for information. The result is then stored
as a variable, so that any time we type titleOrig later this will be replaced by the string corresponding
to the original title 1. Then we just find anywhere the title appears and replace the text with our new
variable name, i.e. in this case by writing

selectWindow(titleOrig);
If we do this, the window we want will probably be activated as required. However, there is a
subtle potential problem. It’s possible that we have two images open at the same time with identical titles
– in which case it’s not clear which window should be selected, and so the results could be unpredictable.
A safer approach is to get a reference to the image ID rather than its title. The ID is a number that should
be unique for each image, which is useful for ImageJ internally but which we do not normally care about
unless we are programming. Using IDs, the updated macro code then becomes:

idOrig = getImageID();
run("32-bit");
run("Duplicate...", "title=[My duplicated image]");
idDuplicate = getImageID();
run("Gaussian Blur...", "sigma=1");
selectImage(idOrig);
run("Gaussian Blur...", "sigma=2");
imageCalculator("Subtract create", idDuplicate, idOrig);
We had to change selectWindow to selectImage for the IDs to work. I also changed the title of
the duplicated image to something more meaninglessly general – which required square brackets, because
it includes spaces that would otherwise mess things up 2. Also, because the duplicated image will be
active immediately after it was created, I ask ImageJ for its ID at that point. This lets me then pass the two
IDs (rather than titles) to the imageCalculator command when necessary.
Adding comments

Whenever macros become more complicated, it can be hard to remember exactly what all the
parts do and why. It’s then a very good idea to add in some extra notes and explanations. This is done by
prefixing a line with //, after which we can write whatever we like because the macro interpreter will
ignore it. These extra notes are called comments, and I will add them from now on.

Customizing sigma values

By changing the size of the Gaussian filters, the macro can be tailored to detecting structures of
different sizes. It would be relatively easy to find the Gaussian Blur lines and change the sigma values
accordingly here, but adjusting settings like this in longer, more complex macros can be awkward. In such
cases, it’s helpful to extract the settings you might wish to change and include them at the start of the
macro.

To do this here, insert the following lines at the very beginning:

// Store the Gaussian sigma values -

// sigma1 should be less than sigma2
sigma1 = 1.5;
sigma2 = 2;
Then, update the later commands to:

run("Gaussian Blur...", "sigma="+sigma1);

selectImage(idOrig);
run("Gaussian Blur...", "sigma="+sigma2);
This creates two new variables, which represent the sigma values to use. Now any time you want
to change sigma1 or sigma2 you do not need to hunt through the macro for the correct lines: you can just
update the lines at the top 3.

Adding interactivity

Usually I would stop at this point. Still, you might wish to share your macro with someone lacking
your macro modification skills, in which case it would be useful to give this person a dialog box into
which they could type the Gaussian sigma values that they wanted. An easy way to do this is to remove
the sigma value information from the run command lines, giving

run("Gaussian Blur...");

Since Process ‣ Filters ‣ Gaussian Blur will not then know what size of filters to use, it will ask.
The disadvantage of this is that the user is prompted to enter sigma values at two different times as the
macro runs, which is slightly more annoying than necessary.

The alternative is to create a dialog box that asks for all the required settings in one go. To do this,
update the beginning of your macro to include something like the following:

Dialog.create("Choose DoG filters");

Dialog.addNumber("Gaussian sigma 1", 1);
Dialog.addNumber("Gaussian sigma 2", 2);
Dialog.show();
sigma1 = Dialog.getNumber();
sigma2 = Dialog.getNumber();

The first line generates a dialog box with the title you specify. Each of the next two lines state that
the required user input should be a number with the specified prompts and default values. The other lines
simply show the dialog box and then read out whatever the user typed and puts it into variables. This is
documented in ImageJ’s list of built-in macro functions.

You can download the complete example macro here.

Installing the macro

If you’d like the macro to appear as an entry in ImageJ’s menus, you have a couple of options.

One is the tempting Plugins ‣ Macros ‣ Install…. This works, but whenever I tested it I found
that it only retains the macro until ImageJ is restarted.

More usefully, Plugins ‣ Install… will prompt you to first select the file containing your macro,
and then to save it within the Plugins directory of ImageJ itself. As long as the macro file you choose has
an underscore in its name, it should appear as its own entry towards the bottom of the Plugins menu –
and be retained even when ImageJ is relaunched.

Suggested improvements
You should now have a macro that does something vaguely useful, and which will work on most 2D
images. It could nevertheless still be enhanced in many ways. For example,

 You could close any unwanted images (e.g. the original and its duplicate) by selecting their IDs, and
then inserting close(); commands afterwards.
 You could make the macro work on entire image stacks. If you want it to process each plane
separately, this involves only inserting the words stack and duplicate in several places – by recording
a new macro in the same way, but using a stack as your example image, you can see where to do this. If
you want the filtering to be applied in 3D, you can use the Process ‣ Filters ‣ Gaussian Blur
3D… command instead of Process ‣ Filters ‣ Gaussian Blur…
 You could create a log of which images you have processed, possibly including the settings used. The
log is output by including a log(text); line, where text is some string you have created,
e.g. text = Image name: + getTitle().
 More impressively, you could turn the macro into a full spot-detector by thresholding the DoG filtered
image, and then running the Analyze ‣ Analyze Particles… command. If you want to measure original
spot intensities, you should remember to go to Analyze ‣ Set Measurements… to make sure the
measurements are redirected to the original image – which you should possibly have duplicated at the
beginning. Without the duplication, the original image will have been Gaussian filtered by the time your
macro reaches the measurement stage.
In any case, the process of developing a macro is usually the same:
1. Record a macro that does basically the right thing
2. Remove all the superfluous lines (contrast adjustment, errant clicking etc.)
3. Replace the image titles with image ID references
4. Add comments to describe what the macro is doing
5. Track down bugs and make improvements

Line detectors (Hough Transform)

The Hough Transform is a method that is used in image processing to detect any shape, if that
shape can be represented in mathematical form. It can detect the shape even if it is broken or distorted a
little bit.
We will see how Hough transform works for line detection using the HoughLine transform method. To
apply the Houghline method, first an edge detection of the specific image is desirable. For the edge
detection technique go through the article Edge detection

Basics of Houghline Method

A line can be represented as y = mx + c or in parametric form, as r = xcos? + ysin? where r is the
perpendicular distance from origin to the line, and ? is the angle formed by this perpendicular line and
horizontal axis measured in counter-clockwise ( That direction varies on how you represent the
coordinate system. This representation is used in OpenCV).

So Any line can be represented in these two terms, (r, ?).

Working of Houghline method:

 First it creates a 2D array or accumulator (to hold values of two parameters) and it is set to zero
initially.
 Let rows denote the r and columns denote the (?)theta.
 Size of array depends on the accuracy you need. Suppose you want the accuracy of angles to be 1
degree, you need 180 columns(Maximum degree for a straight line is 180).
 For r, the maximum distance possible is the diagonal length of the image. So taking one pixel
accuracy, number of rows can be diagonal length of the image.
Example:
Consider a 100×100 image with a horizontal line at the middle. Take the first point of the line. You
know its (x,y) values. Now in the line equation, put the values ?(theta) = 0,1,2,….,180 and check the r
you get. For every (r, ?) pair, you increment value by one in the accumulator in its corresponding (r, ?)
cells. So now in accumulator, the cell (50,90) = 1 along with some other cells.
Now take the second point on the line. Do the same as above. Increment the values in the cells
corresponding to (r, ?) you got. This time, the cell (50,90) = 2. We are actually voting the (r, ?) values.
You continue this process for every point on the line. At each point, the cell (50,90) will be incremented
or voted up, while other cells may or may not be voted up. This way, at the end, the cell (50,90) will have
maximum votes. So if you search the accumulator for maximum votes, you get the value (50,90) which
says, there is a line in this image at distance 50 from origin and at angle 90 degrees.

Everything explained above is encapsulated in the OpenCV function, cv2.HoughLines(). It simply returns
an array of (r, 0) values. r is measured in pixels and 0 is measured in radians.

 Python

# Python program to illustrate HoughLine

# method for line detection

import cv2

import numpy as np

# Reading the required image in

# which operations are to be done.

# Make sure that the image is in the same

# directory in which this python program is

img = cv2.imread('image.jpg')

# Convert the img to grayscale

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

# Apply edge detection method on the image

edges = cv2.Canny(gray, 50, 150, apertureSize=3)

# This returns an array of r and theta values

lines = cv2.HoughLines(edges, 1, np.pi/180, 200)

# The below for loop runs till r and theta values

# are in the range of the 2d array

for r_theta in lines:

arr = np.array(r_theta[0], dtype=np.float64)

r, theta = arr

# Stores the value of cos(theta) in a

a = np.cos(theta)

# Stores the value of sin(theta) in b

b = np.sin(theta)

# x0 stores the value rcos(theta)

x0 = a*r

# y0 stores the value rsin(theta)

y0 = b*r

# x1 stores the rounded off value of (rcos(theta)-1000sin(theta))

x1 = int(x0 + 1000*(-b))

# y1 stores the rounded off value of (rsin(theta)+1000cos(theta))

y1 = int(y0 + 1000*(a))

# x2 stores the rounded off value of (rcos(theta)+1000sin(theta))

x2 = int(x0 - 1000*(-b))

# y2 stores the rounded off value of (rsin(theta)-1000cos(theta))

y2 = int(y0 - 1000*(a))

# cv2.line draws a line in img from the point(x1,y1) to (x2,y2).

 # (0,0,255) denotes the colour of the line to be

# drawn. In this case, it is red.

cv2.line(img, (x1, y1), (x2, y2), (0, 0, 255), 2)

# All the changes made in the input image are finally

# written on a new image houghlines.jpg

cv2.imwrite('linesDetected.jpg', img)

Elaboration of function(cv2.HoughLines (edges,1,np.pi/180, 200)):

1. First parameter, Input image should be a binary image, so apply threshold edge detection before
finding applying hough transform.
2. Second and third parameters are r and ?(theta) accuracies respectively.
3. Fourth argument is the threshold, which means minimum vote it should get for it to be considered
as a line.

4. Remember, number of votes depend upon number of points on the line. So it represents the
minimum length of line that should be detected.

Alternate simpler method for directly extracting points:

 Python3
import cv2

import numpy as np

# Read image

image = cv2.imread('path/to/image.png')

# Convert image to grayscale

gray = cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)

# Use canny edge detection

edges = cv2.Canny(gray,50,150,apertureSize=3)

# Apply HoughLinesP method to

# to directly obtain line end points

lines_list =[]

lines = cv2.HoughLinesP(

edges, # Input edge image

1, # Distance resolution in pixels

np.pi/180, # Angle resolution in radians

threshold=100, # Min number of votes for valid line

minLineLength=5, # Min allowed length of line

maxLineGap=10 # Max allowed gap between line for joining them

# Iterate over points

for points in lines:

# Extracted points nested in the list

x1,y1,x2,y2=points[0]

# Draw the lines joing the points

 # On the original image

cv2.line(image,(x1,y1),(x2,y2),(0,255,0),2)

# Maintain a simples lookup list for points

lines_list.append([(x1,y1),(x2,y2)]

# Save the result image

cv2.imwrite('detectedLines.png',image)

Summarizing the process

 In an image analysis context, the coordinates of the point(s) of edge segments (i.e. X,Y ) in the image
are known and therefore serve as constants in the parametric line equation, while R(rho) and Theta(?) are
the unknown variables we seek.
 If we plot the possible (r) values defined by each (theta), points in cartesian image space map to curves
(i.e. sinusoids) in the polar Hough parameter space. This point-to-curve transformation is the Hough
transformation for straight lines.
 The transform is implemented by quantizing the Hough parameter space into finite intervals or
accumulator cells. As the algorithm runs, each (X,Y) is transformed into a discretized (r,0) curve and the
accumulator(2D array) cells which lie along this curve are incremented.
 Resulting peaks in the accumulator array represent strong evidence that a corresponding straight line
exists in the image.
Applications of Hough transform:

1. It is used to isolate features of a particular shape within an image.

2. Tolerant of gaps in feature boundary descriptions and is relatively unaffected by image noise.
3. Used extensively in barcode scanning, verification and recognition.

Corners – Harris:
Motivation: Matchng Problem
Vision tasks such as stereo and motion estimation require finding corresponding features across two
or more views.