Computer Vision

Module 3: Morphological Image Processing

Lecture

Morphological
Operations and
algorithms
Introduction

Module 1:Lecture 1 –
Introduction
Basic concepts in set theory
Logic operations involving binary pixel
and images
Structuring Elements (SE)
EROSION
EROSION:EXAMPLE 1
EROSION:EXAMPLE 2
DILATION
DILATION:EXAMPLE 1
DILATION:EXAMPLE 2
DILATION: A more interesting example
(bridging gaps)
Duality between dilation and
erosion
Erosion and dilation summary
Erosion vs Dilation
Opening and closing
The hit or miss transformation
The hit or miss transformation
Hit or miss exp:
The hit or miss transformation
The hit or miss transformation
The hit or miss transformation
The hit or miss transformation
The hit or miss transformation
The hit or miss transformation
Basic morphological algorithms
(applications)
Boundary extraction
Boundary extraction
Region filling
Region filling
Extraction of connected
components
Convex Hull
Thinning
Thinning
Thinning example
Thickening
Thickening
Thickening example preview
Thickening example
Skeleton
Skeleton example
Skeleton
Pruning
Feature Descriptors :

• Feature Descriptor is a simplified image representation that contains

only the most important information about the image.

• There are several feature descriptors out there. Here are a few of the
most popular ones
• HOG: Histogram of Oriented Gradients
• SIFT: Scale Invariant Feature Transform
• SURF: Speeded-Up Robust Feature
SIFT :

• This algorithm is a computer vision technique used for feature detection and
description. It detects distinctive key points or features in an image that are
robust to changes in scale, rotation, and affine transformations. SIFT(scale
invariant feature transform) works by identifying key points based on their
local intensity extrema and computing descriptors that capture the local image
information around those key points. These descriptors can then be used for
tasks like image matching, object recognition, and image retrieval.

• SIFT algorithm helps locate the local features in an image, commonly known as
the ‘keypoints‘ of the image. These keypoints are scale & rotation invariants
that can be used for various computer vision applications, like image
matching, object detection, scene detection, etc.
SIFT :
SIFT Process:

The entire process can be divided into 4 parts:

•Constructing a Scale Space: To make sure that features are scale-independent

•Keypoint Localization: Identifying the suitable features or keypoints

•Orientation Assignment: Ensure the keypoints are rotation invariant

•Keypoint Descriptor: Assign a unique fingerprint to each keypoint

SIFT Process: Constructing DOG
scale space
1. To simulate the multi-scale features of image data, capture the
general features at a large scale, and focus on detailed features at a
small scale.

2. By constructing a Gaussian pyramid (each layer uses different

parameters σ for Gaussian blur (weighting)), which is to guarantee
that the image can have corresponding feature points at any scale,
i.e., scale invariance.
SIFT Process: Constructing DOG
scale space
We need to identify the most distinct features in a given input image while ignoring
any noise. Additionally, we need to ensure that the features are not scale-
dependent. These are critical concepts, so let’s talk about them one by one.
We use the Gaussian Blurring technique to reduce the noise in an image. As you
can see, the texture and minor details are removed from the image, and only the
relevant information, like the shape and edges, remain
SIFT Process: Constructing DOG
scale space
The ideal
number of
octaves
should be
four, and for
each octave,
the number of
blur images
should be
five.
SIFT Process: Constructing DOG
scale space
SIFT Process: Constructing DOG
scale space
SIFT Process: Constructing DOG
scale space
SIFT Process: Keypoint
Localization
• Compare the point with adjacent points in the image with different σ values in
the same scale space. If the point is maxima or minima, then it is a feature
point.

• Remove the points with low contrast and unstable edge effects, leaving
representative key points after all the feature points have been found, to
enhance the noise immunity and stability of the match.

• Finally, curve fitting on discrete points is performed to obtain accurate position

and scale information of the key point.
SIFT Process: Keypoint
Localization

To locate the local maxima and minima, we

go through every pixel in the image and
compare it with its neighboring pixels., this
includes not only the surrounding pixels of
that image (in which the pixel lies) but also
the nine pixels for the previous and next
image in the octave.
We will eliminate the keypoints that have
low contrast or lie very close to the edge.
SIFT Process: Keypoint
Localization
SIFT Process: Orientation
Assignment
• Calculate the magnitude and orientation and create a histogram for
magnitude and orientation. The histogram would peak at some point.
The bin at which we see the peak will be the orientation for the key
point.
• To calculate the histogram, each sampling point is weighted using a
Gaussian pyramid, ie., Gaussian smoothing. Through Gaussian
smoothing, the gradient magnitude near the key point can be given a
larger weight, thereby partially compensating for the instability of the
feature points caused by not considering the affine deformation.
• Note that one key point may have multiple key directions, which is
beneficial to enhance the robustness of image matching.
SIFT Process: Orientation
Assignment
SIFT Process: Generation of key point
descriptors

• The key point descriptor includes the key point and the pixels around the key
point that contribute to it to make the key points have stronger invariant
characteristics and improve the efficiency of target matching.

• Consider bilinear interpolation after rotation for the sampling area of the
descriptor to prevent white spots from appearing in the rotated image.

• Rotate θ angle in the nearby area with the feature point as the center, and then
calculate the gradient histogram of the sampling area to form an n-dimensional
SIFT feature vector (such as 128-SIFT) to ensure the invariance of rotation.

• Finally, normalize the feature vectors to remove the influence of illumination

changes.
SIFT Process: Generation of key point
descriptors
import numpy as np
import cv2 as cv
img = cv.imread(path for image')
grey = cv.cvtColor(img, cv.COLOR_BGR2GRAY)

# Create a SIFT object

sift = cv.xfeatures2d_SIFT.create(nfeatures=0, nOctaveLayers=3,
contrastThreshold=0.09, edgeThreshold=10, sigma=1.6)

# Detect and draw keypoints

kp = sift.detect(grey, None)
img = cv.drawKeypoints(grey, kp, img, flags=
cv.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)
cv.imwrite('images/sift-example.jpg', img) # Save images
cv.imshow('image', img) # Show image with keypoints
Histogram of Oriented Gradients
• The Histogram of Oriented Gradients (HOG) is a popular feature descriptor
technique in computer vision and image processing.

• It analyzes the distribution of edge orientations within an object to describe

its shape and appearance.

• The HOG method involves computing the gradient magnitude and orientation
for each pixel in an image and then dividing the image into small cells.

• The HOG feature descriptor counts the occurrences of gradient

orientation in localized portions of an image.
Why is HoG Different?
• The HOG descriptor focuses on the structure or the shape of an object. In the case
of edge features, we only identify if the pixel is an edge or not. HOG can provide
the edge direction as well. This is done by extracting the gradient and
orientation (or you can say magnitude and direction) of the edges

• Additionally, these orientations are calculated in ‘localized’ portions. This means

that the complete image is broken down into smaller regions and for each region,
the gradients and orientation are calculated. We will discuss this in much more
detail in the upcoming sections

• Finally the HOG would generate a Histogram for each of these regions separately.
The histograms are created using the gradients and orientations of the pixel
values, hence the name ‘Histogram of Oriented Gradients’
Steps involved :

Step 1: Preprocess the Data (64 x 128)

Step 2: Calculating Gradients (direction x and y)

Step 3: Calculate the Magnitude and Orientation

Step 4: Calculate Histogram of Gradients in 8×8 cells (9×1)

Step 5: Normalize gradients in 16×16 cell (36×1)

Step 6: Features for the complete image

Step 1: Preprocess the Data (64 x
128)
We need to preprocess the image and bring down the width to
height ratio to 1:2. The image size should preferably be 64 x 128.
This is because we will be dividing the image into 8*8 and 16*16
patches to extract the features. Having the specified size (64 x 128)
will make all our calculations pretty simple.
Step 2 : Calculate the Gradient
Images
To calculate a HOG descriptor, we need to first calculate the horizontal and
vertical gradients; after all, we want to calculate the histogram of gradients.
This is easily achieved by filtering the image with the following kernels.

The gradient of the image is calculated. The gradient is obtained by

combining magnitude and angle from the image. Consider a block of 3x3
pixels, first Gx and Gy is calculated for each pixel using the formula below
for each pixel value.

Where r, c refer to rows and columns respectively.

After calculating Gx and Gy, the magnitude and angle of each pixel is calculated
using the formula mentioned below.
Step 3: Calculate the Magnitude and
Orientation
The gradients are the base and perpendicular here. So, for the
previous example, we had Gx and Gy as 11 and 8. Let’s apply the
Pythagoras theorem to calculate the total gradient magnitude:

Total Gradient Magnitude = √[(Gx)2+(Gy)2]

Next, calculate the orientation (or direction) for the same pixel. We
know that we can write the tan for the angles:
tan(Φ) = Gy / Gx

Hence, the value of the angle would be:

Φ = atan(Gy / Gx)
Step 3: Calculate the Magnitude and
Orientation
Step 4: Calculate Histogram of
• To move on to the next step of the HOG algorithm, make sure that
Gradients in 8×8
the image is divided cells
into cells (9×1)
so that the histogram of gradients
can be calculated for each cell. For example, if you have a 64x128
image, divide your image into 8x8 cells (this will involve a bit of
tweaking and guessing!).
• Feature descriptors will allow for a concise and succinct
representation of particular patches of the images; taking our
example from above, an 8x8 cell can simply be explained using
128 numbers (8x8x2 where the last 2 are from the gradient
magnitude and directional values).
• By further converting these numbers to calculate histograms, we
allow for an image patch that is much more robust to noise and
more compact.
• Make sure to split the histogram into nine separate bins, each
corresponding to angles from 0–160 in increments of 20. Here’s an
example of how an image with the respective gradient magnitudes
and directions can look like (notice the arrows get larger depending
on the magnitude).
• gram of Oriented Gradients
Step 4: Calculate Histogram of
Gradients in 8×8 cells (9×1)
O
T
O
T
O
T
O
T
O
T
Lecture

Boundary descriptors
Lecture

Region descriptors
Lecture

SIFT: Scale-Invariant
Feature Transform
Thank You