BIOMETRICS

Sections Revision
 BIOMETRICS

BIOMETRIC SYSTEM
• A pattern recognition system that recognizes a person by determining the
authenticity of a specific biological and/or behavioral characteristic (biometric).

Types of Biometrics:
− Physiological
− Behavioral

Physiological
• are related to the shape of the body.

− Fingerprint.
− Face recognition.
− DNA.
− Hand and palm geometry.
− Iris recognition.
− Retina.

Behavioral
• are related to the behavior of a person.

− Gait.
− Voice.
− Signature.
− Keystroke .

BIOMETRIC SYSTEM STEPS

• Biometric Capture.
• Transmission and processing .
• Extract enhanced features.

MADE BY π P a g e |2
 BIOMETRICS

• Storage of biometric information.

• Biometric comparison.
• Decision Subsystem.

IMAGE PROCESSING.
• Image processing is a method to perform some operations on an image.
• In order to get an enhanced image or to extract some useful information from
it.
• It is a type of signal processing in which input is an image and output may be
image or characteristics/features associated with that image.

MADE BY π P a g e |3
 BIOMETRICS

COLOR MODELS.

Grayscale :
• which pixel in a grayscale image is represented by a single 8-bit value, ranging
from 0 for black to 255 for white.

BGR :
• which each pixel has a triplet of values representing the blue, green, and red
components.
• each pixel is represented by a triplet of 8-bit values, such as [0, 0, 0] for black,
[255, 0, 0] for blue, [0, 255, 0] for green, [0, 0, 255] for red, and [255, 255, 255]
for white.

HSV
• uses a different triplet of channels.
• Hue is the color's tone.
• saturation is its intensity.
• value represents its brightness.

PYTHONto infinity
• With python you can do everything from GUI development, Web application,
System administration tasks, financial calculation, Data Analysis, Visualization,
and list goes on.

Python applications:
− Data science
− Developing applications with graphical UIS
− Writing network-based software
− Interacting with database
− Scrape data from website

MADE BY π P a g e |4
 BIOMETRICS

− Game development

Setting Environment
• Install Python
• Install Pycharm
• NumPy
• OPENCV LIBRARY

NUMPY LIBRARY
• Numpy is a very popular library for easily creating single, multidimensional
array and matrices.

How to Install Numpy in PyCharm ?

− Step1: Go to the File and click on Settings.
− Step 2: You will see > Project: your_project_name. Click on it. You will see
two option one is Project Interpreter and other Project Structure.
− Step 3: Click on the Project Interpreter. You will see all the packages
installed.
− Step 4:You will see the + button.
Click on it and search for the numpy in the search field.
You will see the numpy package as the left side click on the install package
and its description, version on the right side.

OPENCV LIBRARY
• OpenCV is a huge open-source library for computer vision, machine
learning, and image processing.
• In this course, OpenCV version was 4.0.
• Packages for standard desktop environments (Windows, macOS, almost any
GNU/Linux distribution)

MADE BY π P a g e |5
 BIOMETRICS

− Main modules package: pip install opencv-python

− Full package (contains both main modules and contrib/extra modules):
pip install opencv-contrib-python

OPENCV LIBRARY
IMAGE READING

cv2.imread(path)
• loads an image from the specified file.

Parameters
− cv2.IMREAD_COLOR reads the image with RGB colors
− cv2.IMREAD_GRAYSCALE reads the image as grey image.
− cv2.IMREAD_UNCHANGED reads the image as is from the source

IMAGE WRITING

cv2.imwrite(path)
• Saves an image to a specified file

Show an Image

imshow()
• Displays an image in the window

CONVERT an image from color space to another.

cv2.cvtColor()
• Converts an image from one color space to another

MADE BY π P a g e |6
 BIOMETRICS

BGR TO GRAYSCALE
− grayImage = cv2.cvtColor(originalImage, cv2.COLOR_BGR2GRAY)

BGR TO RGB
− RGBImage = cv2.cvtColor(originalImage, cv2.COLOR_BGR2RGB)

BGR TO HSV
− hsvImage=cv2.cvtColor(originalImage,
cv2.COLOR_BGR2HSV)
− H=hsvImage[:,:,0]
− S=hsvImage[:,:,1]
− V=hsvImage[:,:,2]

waitkey()
• function of Python OpenCV allows users to display a window for given
milliseconds or until any key is pressed.

destroyAllWindow()
• function allows users to destroy all windows at any time

CONVERT FROM GRAY TO BINARY

cv2.threshold()
• takes a grayscale image and converts it to binary
image

Example:
− (thresh, blackAndWhiteImage) = cv2.threshold(grayImage, 127, 255,
cv2.THRESH_BINARY)

MADE BY π P a g e |7
 BIOMETRICS

SPLIT & MERGE Image Channels

split()
− B, G, R = cv2.split(originalImage)

merge()
− m=cv2.merge((B, G, R))

RESIZING IMAGE
half the image
− cv2.resize(image, (0, 0), fx = 0.5, fy = 0.5)

bigger the image

− cv2.resize(image, (1050, 1610))

stretch image
− cv2.resize(image, (780, 540), interpolation = cv2.INTER_NEAREST)

Image Inversion

bitwise_not()
• invert the light pixels to dark and dark
to light

− img_not = cv2.bitwise_not(img)

MADE BY π P a g e |8
 BIOMETRICS

Image Transformation
 A function or operator that takes an image as its input and produces an
image as its output. Depending on the transform chosen, the input and output
images may appear entirely different and have different interpretations.

 Change the image from one form to another

 A set of image transformations where the geometry of image is changed

without altering its actual pixel values

 Examples:

1. Scaling

2. Translation

3. Rotation

Translation
• Translation is the shifting of an object's location.

warpAffine()

Define the translation matrix

Apply the translation

• translated_image = cv2.warpAffine(src=image,
M=translation_matrix, dsize=(width, height))

MADE BY π P a g e |9
 BIOMETRICS

Rotation
• As evident by its name, this technique rotates an image
by a specified angle and by the given axis or point.
• maps the position of a point in current image to the output
image by rotating it by the user-defined angle through the
specified axis.

Define the rotation matrix: getRotationMatrix2D()

• rotate_matrix = cv2.getRotationMatrix2D(center=center,
angle=45, scale=1)

Apply rotation: cv2.warpAffine()

• rotated_image = cv2.warpAffine(src=image,
M=rotate_matrix, dsize=(width, height))

Morphological Transformations of images

 Morphological transformations are some simple operations based on
the image shape. It is normally performed on binary images. It needs
two inputs, one is our original image, second one is called
structuring element or kernel which decides the nature of operation.
Two basic morphological operators are Erosion and Dilation. Then
its variant forms like Opening, Closing, Gradient etc.

Erosion
• A pixel in the original image (either 1 or 0) will be considered 1 if all
the pixels under the kernel is 1, otherwise it is eroded (made to zero)

Apply Erosion: erode()

− kernel = np.ones((5,5),np.uint8)
− erosion = cv.erode(img,kernel,iterations = 1)

MADE BY π P a g e | 10
 BIOMETRICS

Dilation
• It is just opposite of erosion. Here, a pixel element is '1' if at least one
pixel under the kernel is '1'.
• So, it increases the white region in the image or size of foreground
object increases.
• Normally, in cases like noise removal, erosion is followed by dilation.
• Because, erosion removes white noises, but it also shrinks our
object. So, we dilate it. Since noise is gone, they won't come back, but
our object area increases. It is also useful in joining broken parts of an
object.

Apply Dilation: dilate

− dilation = cv.dilate(img,kernel,iterations = 1)

opening
• Opening is just another name of erosion followed by
dilation. It is useful in removing noise

To Apply opening: morphologyEx()

− cv2.morphologyEx(img, cv2.MORPH_OPEN, kernel)

closing
 Closing is reverse of Opening, Dilation followed by Erosion. It is useful in
closing small holes inside the foreground objects, or small black points on
the object.

To Apply closing: morphologyEx()

− cv2.morphologyEx(img, cv2.MORPH_CLOSE, kernel)

MADE BY π P a g e | 11
 BIOMETRICS

Morphological Gradient
• It is the difference between dilation and erosion of an image.
• The result will look like the outline of the object.

To Apply Gradient: morphologyEx()

− Cv2.morphologyEx(img, cv2.MORPH_GRADIENT, kernel)

Top Hat
• It is the difference between input image and Opening of the
image. Below example is done for a 9x9 kernel.

To Apply Top Hat: morphologyEx()

− Cv2.morphologyEx(img, cv2. MORPH_TOPHAT, kernel)

Black Hat
• It is the difference between the closing of the input image and
input image.

To Apply Black Hat: morphologyEx()

− Cv2.morphologyEx(img, cv2. MORPH_BLACKHAT, kernel)

IMAGE FILTERING
• Image filtering involves the application of window operations that perform
useful functions, such as noise removal and image enhancement.

SMOOTHING IMAGES ( IMAGE

FILTERING )
• Images can be filtered with various low-pass filters (LPF)
and high-pass filters (HPF).

MADE BY π P a g e | 12
 BIOMETRICS

• A LPF helps in removing noise or blurring the image.

• A HPF filters helps in finding edges in an image.
• OpenCV provides a function, cv2.filter2D(), to convolve a kernel with an
image.

Advantages of LPF:
• It helps in Noise removal. As noise is considered as high pass signal so by the
application of low pass filter kernel, we restrict noise.
• It helps in smoothing the image.
• Low intensity edges are removed.
• It helps in hiding the details when necessary.

Types :
− Gaussian Blurring.
− Median Blur.
− Bilateral Blur .
− Custom kernel.

AVERAGING FILTER
• For each pixel, a 5x5 window is centered on this pixel, all pixels falling within
this window are summed up, and the result is then divided by 25.
• This equates to computing the average of the pixel values inside that window.
This operation is performed for all the pixels in the image to produce the output
filtered image.
• This is done by convolving the image with a
normalized box filter. It simply takes the average
of all the pixels under kernel area and replaces the
central element with this average

blur() OR cv2.boxFilter()
− blur = cv2.blur(img,(5,5))

MADE BY π P a g e | 13
 BIOMETRICS

GAUSSIAN FILTER

• instead of a box filter consisting of equal filter

coefficients, a Gaussian kernel is used

cv2.GaussianBlur()
− blur = cv2.Gaussianblur(img,(5,5),0)

MEDIAN FILTERING
• computes the median of all the pixels under the
kernel window and the central pixel is replaced
with this median value.

cv2.medianBlur()
− blur = cv2.medianblur(img,5)

BILATERAL FILTERING
• the filters presented earlier tend to blur edges.
• This is not the case for the bilateral filter, and also it is highly effective at noise
removal while preserving edges. operation is slower
compared to other filters.

cv2.bilateralFilter()
− blur = cv2. bilateralFilter(img,9,75,75)

Parameters

d
− A variable of the type integer representing the diameter of the pixel
neighborhood.

sigmaColor
− A variable of the type integer representing the filter sigma in the color space.

MADE BY π P a g e | 14
 BIOMETRICS

sigmaSpace
− A variable of the type integer representing the filter sigma in the coordinate
space

FEATURE DETECTION
• Is an individual measurable property or characteristic of a phenomenon being
observed.
• Features are usually numeric.
• A feature is defined as an "interesting"
part of an image, and features are used as a
starting point for many computer vision
algorithms.

Edges
• are points where there is a boundary (or an edge) between two image regions.

Corners / interest points

• The terms corners and interest points are used
somewhat interchangeably and refer to point-
like features in an image, which have a local
two-dimensional structure.

FEATURE EXTRACTION
• Starts from an initial set of measured data and builds derived values features
intended to be informative and non-redundant, facilitating the subsequent
learning and generalization steps, and in some cases leading to better human
interpretations

MADE BY π P a g e | 15
 BIOMETRICS

EDGE DETECTION
• Edge detection is a technique of image processing used to identify points in a
digital image with discontinuities, simply to say, sharp changes in the image
brightness.
• These points where the image brightness varies sharply are called the edges (or
boundaries) of the image.

WHY DO WE CARE ABOUT EDGES?

• Extract information.
• Recognize objects .
• Edges are significant local changes of intensity in an image.
• Edges typically occur on the boundary between two different regions in an
image.

MODELING INTENSITY CHANGES

• Edges can be modeled according to their intensity profiles.

Step edge
− the image intensity abruptly changes from
one value to one side of the discontinuity to a
different value on the opposite side.

MADE BY π P a g e | 16
 BIOMETRICS

Ramp edge
− a step edge where the intensity change is not
instantaneous but occur over finite distance.

Ridge edge
− the image intensity abruptly changes value
but then returns to the starting value
within some short distance (generated
usually by lines).

Roof edge
− a ridge edge where the intensity change is
not instantaneous but occur over finite
distance (generated usually by the
intersection of surfaces).

STEPS OF EDGE DETECTION

1. Smoothing: suppress as much noise as possible, without destroying the true
edges.
2. Enhancement: apply a filter to enhance the quality of the edges in the image
(sharpening).
3. Detection: determine which edge pixels should be discarded as noise and
which should be retained (usually, thresholding provides the criterion used
for detection).
4. Localization: determine the exact location of an edge (sub-pixel resolution
might be required for some applications, that is, estimate the location of an
edge to better than the spacing between pixels). Edge thinning and linking
are usually required in this step.

MADE BY π P a g e | 17
 BIOMETRICS

EDGE DETECTION
• Laplacian Derivatives .
• Sobel Derivatives.
• Canny.

LAPLACIAN DERIVATIVES.

cv2.Laplacian()
− laplacian = cv2.Laplacian(img, cv2.CV_64F)

SOBEL DERIVATIVES

cv2.Sobel()
− sobelx = cv2.Sobel(img, cv2.CV_64F, 1, 0, ksize=5)
sobely = cv2.Sobel(img, cv2.CV_64F, 0, 1, ksize=5)
sobelImg= cv2.add(sobelx, sobely)

CANNY

cv2.Canny()
− edges = cv2.Canny(img,100,200)

MADE BY π P a g e | 18
 BIOMETRICS

Corner Detection
• The regions in images which have maximum variation when moved (by a
small amount) in all regions around it.
• So, finding these image features is called Feature Detection.
• Computer also should describe the region around the feature so that it can find
it in other images.
• So called description is called Feature Description.

Harris Corner Detection

1. Determine which windows produce very large variations in intensity

when moved in x and y direction.

2. With each such window found, a score R is computed.

3. After applying a threshold to this score, important corners are

selected& marked.

MADE BY π P a g e | 19
 BIOMETRICS

cv2.cornerHarris() Parameters
− img - Input image, it should be grayscale and float32 type.
− blockSize - It is the size of neighborhood considered for corner detection
− ksize - Aperture parameter of Sobel derivative used.
− k - Harris detector free parameter in the equation

Example:
− dst = cv2.cornerHarris(gray, 2, 3, 0.04)
// Threshold for an optimal value, it may vary depending on the image.
− img[dst>0.01*dst.max()]=[0, 0, 255]

Shi-Tomasi Corner Detection

cv2.goodFeaturesToTrack(img, N, X, Y) Parameter
− Img :image should be a grayscale image.
− N: number of max corners.
− X: specify number of corners you want to find (0-1).
− Y: provide the minimum Euclidean distance
between corners detected.

Apply Shi-Tomasi Corner Detection

− corners =
cv2.goodFeaturesToTrack(gray,25,0.01,10)

MADE BY π P a g e | 20
 BIOMETRICS

Scale-Invariant Features
• Harris corner detection is rotation-invariant, which means, even if the image
is rotated, we can find the same corners. It is obvious
because corners remain corners in rotated image also.
But what about scaling? A corner may not be a corner
if the image is scaled.
• Harris corner is not scale invariant.

SIFT (SCALE-INVARIANT FEATURE TRANSFORM)

• SIFT, or Scale Invariant Feature Transform, is a feature detection algorithm
helps locate the local features in an image, commonly known as the
‘keypoints‘ of the image.
• These key-points are scale & rotation invariant that can be used for various
computer vision applications, like image matching, object detection, scene
detection, etc

Algorithm Steps:

Constructing a Scale Space: To make sure that features are scale-

independent
− Identify the most distinct features in an image while ignoring any noise.
− Use the Gaussian Blurring technique to reduce the noise in an image.
− Additionally, we need to ensure that the features are
not scale-dependent.
− This means we will be searching for these features
on multiple scales, by creating a ‘scale space’.
− Scale space is a collection of images having
different scales, generated from a single image.

MADE BY π P a g e | 21
 BIOMETRICS

− To create a new set of images of different scales, we will take the original
image and reduce the scale by half.
− For each new image, we will create blur versions.

• Keypoint Localization: Identifying the suitable features or keypoints

− Once the images have been created, the next step is to find the important
key-points from the image that can be used for feature matching.
▪ Find the local maxima and minima (the pixel x is selected as a
key-point if it is the highest or lowest among the neighbors).
▪ Remove low contrast key-points (key-point selection)

Orientation Assignment: Ensure the keypoints are rotation invariant

− At this stage, we have a set of stable key-points
for the images.
− We will now assign an orientation to each of these
key-points so that they are invariant to rotation.

Key-point Descriptor: Assign a unique fingerprint to each key-point

− In this section, we will use the neighboring pixels, their orientations, and
magnitude, to generate a unique fingerprint for this key-point called a
‘descriptor’.
− Additionally, since we use the surrounding pixels, the descriptors will be
partially invariant to illumination or brightness of the images.

MADE BY π P a g e | 22
 BIOMETRICS

SIFT CODE

Creating Sift Object

− sift = cv.SIFT_create()

Detecting Features on grayscale image

− kp = sift.detect(gray,None)

Drawing circle around the detected features

− img=cv.drawKeypoints(gray,kp,img)

FAST ALGORITHM FOR CORNER DETECTION

1. Select a pixel p in the image which is to be identified as an interest point or
not. Let its intensity be Ip.
2. Select appropriate threshold value t.
3. Consider a circle of 16 pixels around the pixel under test.

Initiate FAST object with default values

− fast = cv.FastFeatureDetector_create()

find and draw the keypoints

− kp = fast.detect(img,None )
− img2 = cv.drawKeypoints(img, kp, None, color=(255,0,0))

Disable nonmaxSuppression
− fast.setNonmaxSuppression(0)

Another Feature detectors & descriptors

− SURF Algorithm.
− BREIF Algorithm.
− ORB Algorithm.

MADE BY π P a g e | 23
 BIOMETRICS

SURF: Speeded Up Robust Features

• It is a speeded-up version of SIFT.
• SURF goes a little further and approximates LoG with Box Filter.
• Convolution with box filter can be easily calculated with the help of integral
images. And it can be done in parallel for different scales.

Create surf object :

− surf= cv2.xfeatures2d.SURF_create(threshold)

Find keypoints using surf:

− kp= surf.detect(img)

Compute the descriptors with surf :

− des= surf.compute(img, kp)

Draw only keypoints :

− cv2.drawKeypoints(img, kp, None, color)

get number of keypoints

− len(kp)

get the size of descriptor

− surf.descriptorSize()

BRIEF (Binary Robust Independent Elementary Features)

• BRIEF is a faster method feature descriptor calculation and matching. It
also provides high recognition rate unless there is large in-plane rotation.
• One important point is that BRIEF is a feature descriptor, it doesn’t provide
any method to find the features.

MADE BY π P a g e | 24
 BIOMETRICS

• So, you will have to use any other feature detectors like
SIFT, SURF etc.

Create STAR detector:

− star = cv2.xfeatures2d.StarDetector_create()

find the keypoints:

− kp = star.detect(img,None)

Initiate BRIEF extractor:

− brief = cv2.xfeatures2d.BriefDescriptorExtractor_create()

compute the descriptors with BRIEF:

− kp, des = brief.compute(img, kp)

Draw only keypoints:

− cv2.drawKeypoints(img,kp,None)

ORB (Oriented FAST and Rotated BRIEF)

• ORB: An efficient alternative to SIFT or SURF.
• it is a good alternative to SIFT and SURF in computation
cost, matching performance and mainly the patents.
• ORB is basically a fusion of FAST keypoint detector and
BRIEF descriptor with many modifications to enhance the
performance.

Create ORB object:

− orb= cv2. ORB_create(max_num_feature)

Find keypoints using orb:

− kp= surf.detect(img)

MADE BY π P a g e | 25
 BIOMETRICS

Compute the descriptors with orb:

− des= orb.compute(img, kp)

Draw only keypoints:

− cv2.drawKeypoints(img, kp, None, color)

Matching
− Brute-Force Matcher
− FLANN Based Matcher

BRUTE-FORCE MATCHER
• Brute-Force matcher is simple. It takes the descriptor of one feature in first set
and is matched with all other features in second set using some distance
calculation.
• It takes a lot of computation power (slower)
• create BFMatcher object

− bf = cv.BFMatcher(cv.NORM_HAMMING, crossCheck=True)

• Match descriptors

− matches = bf.match(des1,des2)

• Sort them in the order of their distance.

− matches = sorted(matches, key = lambda x:x.distance)

• Draw first 10 matches.

− img3=cv.drawMatches(img1,kp1,img2,kp2,matches[:10],None,flags=cv.Dr
awMatchesFlags_NOT_DRAW_SINGLE_POINTS)

MADE BY π P a g e | 26
 BIOMETRICS

FLANN Based Matcher

• Fast Library for Approximate Nearest Neighbors is a library for performing
fast approximate nearest neighbor searches in high dimensional spaces.
• It contains a collection of algorithms that work best for nearest neighbor
search and a system for automatically choosing the best algorithm and
optimum parameters depending on the dataset.
• FLANN_INDEX_LINEAR, FLANN_INDEX_KDTREE,
FLANN_INDEX_KMEANS, FLANN_INDEX_COMPOSITE,
FLANN_INDEX_KDTREE_SINGLE, FLANN_INDEX_HIERARCHICAL,
FLANN_INDEX_LSH
• It is faster than Brute-Force Matcher

Initiate SIFT detector

− sift = cv2.SIFT()

find the keypoints and descriptors with

SIFT
− kp1, des1 =
sift.detectAndCompute(img1,None)
− kp2, des2 = sift.detectAndCompute(img2,None)

FLANN parameters
− FLANN_INDEX_KDTREE = 0
− index_params = dict(algorithm = FLANN_INDEX_KDTREE, trees = 5)
− search_params = dict(checks=50)
− flann = cv2.FlannBasedMatcher(index_params,search_params)
− matches = flann.knnMatch(des1,des2,k=2)

To draw good matches

− cv2.drawMatchesKnn(img1,kp1,img2,kp2,matches,None,**draw_params)

MADE BY π P a g e | 27