CSET344/CMCA544

Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. From IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 6th Jan to 10th Jan 2025
[email protected]
1
 CSET344
Image and Video Processing
Ass_Lec. Ass_Tut. Ass_Lab Ass_Total Faculty Name Faculty mail

4 0 8 12 Mr. Prashant Kapil [email protected]

4 0 0 4 Dr. Gaurav Kumar Dashondhi [email protected]

0 0 10 10 Dr. Kimmi Gupta [email protected]

0 0 10 10 Dr. Shallu Sharma [email protected]

Lect. Week
6th Jan to 10th Jan 2025 2
Motivation
• Three Major Motivation for Processing of Images.

1. Improvement of Pictorial information for human interpretation.

Like using Instagram and Snapchat filters.

2. Processing of Image data for storage, transmission and representation of autonomous

machine perception.

3. Processing is essential to analyze and extract the information.

3
Application of Image Processing

1. Remote sensing.

2. Medical Domain.

3. Security and Surveillance. Object detection and Tracking.

4. Industrial automation.

5. Film and entertainment industry. To add the special effects and create artificial
environment.

4
Application of Image Processing

Automate the Status of Bottle Filling Inspection

5
Application of Image Processing

Automate the IC Connection Inspection : Some connections are broken.

6
Application of Image Processing

Medical Domain

Original Image Processed Image1 Processed Image 2

7
Application of Image Processing
Remote Sensing

8
CSET344 - Syllabus

9
Course Overview : Module 1
1. Analog-to-Digital Image Conversion:
Sampling and Quantization

2. Spatial Domain Image Enhancement:

Histogram Processing: Techniques for modifying the image histogram to improve contrast and visual
appearance.

Histogram Equalization: A specific histogram processing method that aims to redistribute pixel
intensities to achieve a uniform distribution.

3. Convolution:
A fundamental operation in image processing where a kernel (filter) is slide over the image, and an
output pixel is computed as a weighted sum of the input pixels within the kernel's region.

4. Image Smoothing:
Mean Filter, Median Filter, Gaussian Filter

5. Edge Detection:
Prewitt Operator, Sobel Operator, Laplacian Operator, Laplacian of Gaussian (LoG) Operator
And Canny Edge Detector: 10
Course Overview: Module 2
1.Line and Circle Detection using the Hough Transform:

2. Harris Corner Detector:

Corner Detection: A technique for identifying image locations where two edges intersect,
forming a sharp corner.

3. Color Models and Color Transforms:

Color Models: Mathematical representations of color, defining how colors are represented
numerically. Common examples include RGB, HSV, and CIELAB.

Color Transforms: Algorithms for converting colors between different color models, enabling
tasks like color correction, image segmentation, and color analysis.

4. Morphological Operations

5. Texture analysis using GLCM

11
Course Overview: Module 3
1. Concept of Optical Flow:

A technique for estimating the apparent motion of objects between two consecutive image
frames. It involves calculating the pixel-wise displacement vectors that represent the motion
of objects in the scene.

2. Image Enhancement in the Frequency Domain

3. Image Compression using Lossless and Lossy Techniques

4. Discrete Cosine Transform (DCT):

A mathematical transformation that decomposes a signal into a sum of cosine functions at
different frequencies. It is widely used in image and video compression, particularly in the
JPEG standard, as it concentrates most of the signal energy into a few low-frequency
coefficients.

12
Course Overview: Module 4
1. Different Methods of Face Detection

Viola-Jones: A cascade classifier based on Haar-like features and boosting.

Histogram of Oriented Gradients (HOG)
Scale-Invariant Feature Transform (SIFT)

2. PCA for Dimensionality Reduction and other Feature Extractors like HOG, SIFT

3. Techniques related to Video Processing, Formation, Compression and Coding:

4. Silent Object and Human Action Reorganization

5. Detail of the Depth Cameras

Sensors that can measure the distance to objects in a scene, providing depth information.

13
CSET344 – Course Evaluation (Tentative)

1. Mid-Semester: 20 marks
2. End-Semester: 40 marks
3. Project Work: 20 marks
1. Presentation and Q&A (Individual Student) : 10 marks
2. Functionality and Working Condition: 10 marks
4. Laboratory Continuous Assessment: 20 marks
5. Programming Environment: All experiments will be conducted using the Python programming language
with OpenCV on the Google Colab platform or Visual Studio Code.
6. Module Coverage:
1. Before the Mid-Semester : Modules 1 and 2 will be completed.
2. After the Mid-Semester: Modules 3 and 4 will be covered.
7. Question Design: All questions will emphasize logical reasoning and problem-solving.

14
EM Spectrum

Refer: https://www.lumitex.com/blog/visible-light-spectrum

15
Image, Intensity or grey level and Pixel.

It’s a two dimensional function f(x , y) where x, y are the spatial coordinate and the amplitude at that particular
coordinate will be the intensity or grey level.
720 1080

x,y
HD 1920
FULL HD
1320
1320 x 720 1920 x 1080

2100

Image

3800
ULTRA 4k
3800 x 2100

16
Type of Images

0 0 0 0 0 0 0 0 0 0
0 1 1 1 0 0 255 255 255 0
0 0 0 1 0 0 0 0 255 0
0 1 1 1 0 0 255 255 255 0
R,G,B
0 0 0 1 0 0 0 0 255 0
0 1 1 1 0 0 255 255 255 0
0 0 0 0 0 0 0 0 0 0

Black and White Image Gray Scale Image Colour Image

Range 0 to 1 Range 0 to 255 Red Channel Range 0 to 255
Green Channel Range 0 to 255
Blue Channel Range 0 to 255

17
 CSET344/CMCA544
Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 13th Jan to 17th Jan 2025
[email protected]
18
Elements of Digital Image Processing Systems

Scanners
Image Acquisition Cameras

Hard Drives
Image Storage Solid State Drives (SSD)

Image Enhancement
Image Restoration
Image Processing Image Segmentation
Image Analysis

Image Image Transmission using

Communication different networks like LAN, WAN

Image Display Monitors

Printers
19
EM Spectrum

Refer: https://www.lumitex.com/blog/visible-light-spectrum

20
Image Sampling and Quantization

The output of the sensor is

continuous voltage waveform
whose amplitude and
coordinates are continuous.

21
 Image Sampling and Quantization
Z
Sampling : Discretizing the time axis or coordinate.

Quantization : Discretizing the amplitude axis.

Digital image : Sampling and Quantization

A B

Y A- Image Projected on sensor array.

B – Result of sampling and quantization.
X,Y are Coordinates.
Z is the amplitude.

22
Representation of Digital images

0 0 0 0 0 0 0 0 0 0
0 1 1 1 0 0 255 255 255 0
0 0 0 1 0 0 0 0 255 0
0 1 1 1 0 0 255 255 255 0
R,G,B
0 0 0 1 0 0 0 0 255 0
0 1 1 1 0 0 255 255 255 0
0 0 0 0 0 0 0 0 0 0

Black and White Image Gray Scale Image Colour Image

Range 0 to 1 Range 0 to 255 Red Channel Range 0 to 255
Green Channel Range 0 to 255
Blue Channel Range 0 to 255

23
Representation of digital image

It’s a two dimensional function f(x , y) where x, y are the spatial coordinate and the amplitude at that particular
coordinate will be the intensity or grey level.

Center of an Image with dimension M x N is obtained

by Dividing the M and N by 2 and rounding it off to
the nearest integer.

24
Basic Terminologies

Pixel (dots) - It’s a smallest unit of a digital image.

0 0 0 0 0 720 1080
0 255 255 255 0
0 0 0 255 0
F(4,4)
0 255 255 255 0 HD 1920
FULL HD
1320
1320 x 720 1920 x 1080
0 0 0 255 0
0 255 255 255 0
0 0 0 0 0 2100

3800
ULTRA 4k
3800 x 2100

25
Basic Terminologies
Spatial Domain

Spatial means Location.

An image made by pixels. Each pixel has some intensity values.
A domain, where intensity or magnitude values are reflected based on the location.

Image Storage and Intensity levels

Lets say size of the image is M x N and its a 8-bit image. Total bits required = M x N x k.

Example: M = 2048, N = 2048 and k = 16 bit image

Total no of bits = 2048 x 2048 x 16 = 67108864 bits

67108864/8 = 8388608 bytes (1 byte = 8 bits)
8388608/1024 = 8192 Kbytes (1kb = 1024 bytes)

No. of intensity levels = 2^k. if 8 bit image then 2^8 = 256 intensity levels or Grey levels (0 to 255).

26
Basic Terminologies
Dynamic Range, Contrast and Contrast Ratio

Dynamic Range: Dynamic range of any image processing system is the ratio of maximum measurable intensity
to minimum detectable intensity.

Example : [255, 1] = 20log(255/1) = approx. 48dB

Dynamic range in terms of images or image contrast or contrast ratio: Difference between highest and lowest
intensity levels in an image.

High dynamic range = Bright or Clear image.

Low dynamic range = Image dull

Dots per inch (Dpi), Resolution

Resolution means how many Dots per inch (DPI) or Pixel per inch (PPI). It directly refer to the clarity of the
image. If resolution is high, it means more information or details can be identified in better way.

27
Basic Terminologies

Spatial Resolution

Spatial Resolution: Capability of sensor to distinguish between two closely spaced objects.

Higher Spatial Resolution: Pixel size is small and one can see more details.

Lower Spatial Resolution: Pixel size is big and one can not distinguish between two closely spaced objects.
28
Basic Terminologies
Intensity Resolution

Capability to resolve different intensity or brightness levels or color in color image.

High Intensity Resolution:

Ability to capture wide range of brightness or intensity levels.

For 32 bit image 2^32 = 65536 intensity or grey levels.

Low Intensity Resolution:

Ability to capture small range of brightness or intensity levels.

For 8 bit image 2^8 = 256 intensity or grey levels.

29
Basic Relationship Between Pixels

Four Neighbors of P, N4(P)

(x-1,y)
(x,y-1) p(x,y) (x,y+1) 8 Neighbors of P N8(p) = N4(P) + ND(P)
(x+1,y)
(x-1,y-1) (x-1,y) (x-1,y+1)
(x,y-1) p(x,y) (x+1,y+1)
Four Diagonal Neighbors of P,ND(P)
(x+1,y-1) (x+1,y) (x+1,y+1)
(x-1,y-1) (x-1,y+1)
p(x,y)
(x+1,y-1) (x+1,y+1)

30
Distance Measures

1.Euclidean Distance:

[(x-s)^2 + (y-t)^2]1/2

2.City Block Distance:

P(3,3) Q1(3,4) Q2(3,5)
P(x,y)
|x-s|+|y-t| Q3(4,4)
P(2,2)
Q4(5,5)
Q(s,t)
Q(4,4) 3.Chess Board Distance:
Find the distance between
Max{|x-s|,|y-t|} (P,Q1)
(P,Q2)
(P,Q3)
(P,Q4)

31
Basic Relationship Between Pixels

City Block Distance Chess Board Distance

2 2 2 2 2 2
2 1 2 2 1 1 1 2
2 1 0 1 2 2 1 0 1 2
2 1 2 2 1 1 1 2
2 2 2 2 2 2

32
Image Enhancement in the Spatial Domain

Spatial Domain

Intensity Transformation Spatial Filtering

(x-1,y+1) (x,y+1) (x+1,y+1)

P(x,y) (x-1,y) p(x,y) (x+1,y)
(x-1,y-1) (x,y-1) (x+1,y-1)

It is applied on a single pixel. It is applied on group or Neighborhood of pixels.

For Example – For Example –
Contrast Manipulation, Image Smoothening,
Image Thresholding, etc. Image Sharpening, etc.
33
Intensity Transformation

Different approaches for intensity transformation:

F(X0,Y0)
1.Identity Transform, (S = T(r)), Linear Transformation

2. Image Negative (Linear Transformation)

Intensity transformation are such kind of
approaches where results are depend on the 3. Log Transform and Exponential Transform
intensity at a point.
4. Power law Transformation (Gamma Correction)
S = T(r), where S = output intensity.
T = Transformation function. 5.Piecewise Linear Transformation Function
r = input intensity. 5.1 Contrast Stretching
5.2 Grey scale to binary image using thresholding.

Smallest possible neighbourhood is 1x1, where

output of transformation function depends only on
a single point or single pixel.

34
Intensity Transformation

35
Intensity Transformation: Image
Negative
Motivation : These kind of transformation is used to enhance grey level information embedded in the dark region of
an image or it is required when black area is dominant in size as compared to white region.
S=L–1–r
where L = Maximum Intensity Level
r = Input Intensity Level
S = Output Intensity Level
Consider 8-bit (0 - 255). Where L = 255.

r S = L-r S = L-r-1 S = L-r-2 S = L-r-3 S = L-r-4

10
20
30
40
36
Intensity Transformation: Log Transformations

Motivation : These transformation used to expand the dark pixel in an image while compressing the higher-level
values.
S = clog(1+r), where L = Maximum Intensity Level
r = Input Intensity Level
S = Output Intensity Level, C = constant. C = L-1/log(1+rmax)

r S = C log(1+r)
Consider 8-bit image. Where L = 255 and rmax = 255
0
Calculate S value ?
1
5
200
220
240
37
Intensity Transformation: Power Law Transformations (Gamma Correction)
Motivation : Visual quality of image may be hampered by illumination condition or wrong setting of camera sensor.
To rectify the same, one can utilized power law transformation or Gamma Corrections.
Basic idea is to raise the pixel value with certain power to improve the overall brightness and contrast of
the image.

S = c r^y,
where

r = Input Intensity Level

S = Output Intensity Level, C = constant. C = 255 if 8-bit image.
If y< 1
Contrast Increasing
else:
Contrast Decreasing.
38
Intensity Transformation: Power Law Transformations (Gamma Correction)

3 x 3 Input Image
10 200 150
20 100 90 S = c r^y
70 50 220
C = 255
r = image(x,y)/ 255

Note: Considering 8-bit image.

Y=0.5 Y=1 Y=2

39
Piecewise Linear Transformation Function: Contrast Stretching
Motivation : Low Contrast image can result from poor illumination, lack of dynamic range in the imaging sensor
or even thought the wrong setting of a lens aperture during image acquisition.
Contrast stretching expands the intensity range to utilize the full dynamic range of the sensor.

Min-Max Contrast Stretching

S = (r – Imin) x ((Omax-Omin)/(Imax - Imin)) + Omin

S = Output Intensity Level

r = Input Intensity Level
Omax = Maximum Output
Omin = Minimum Output
Imax = Maximum input
Imin = Minimum input

40
Piecewise Linear Transformation Function: Contrast Stretching
S = (r – Imin) x ((Omax-Omin)/(Imax - Imin)) + Omin S = Output Intensity Level
r = Input Intensity Level
Omax = Maximum Output
Omin = Minimum Output
Before Transformation Imax = Maximum input
Imin = Minimum input
10 5 150
20 100 90
Apply Contrast Stretching for r =10
70 50 30
Omax = 255, Omin = 0
After Transformation Imax = 150, Imin = 5

New pixel value S = (5-5)x((255-0)/(150-5)) + 0

New pixel value S = ((150-5)x((255-0)/(150-5))) + 0

41
Piecewise Linear Transformation Function: Thresholding
output Intensity s

If r1 = r2
S1 = 0 and S2 = L-1

Input Intensity r

42
 CSET344/CMCA544
Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 20th Jan to 24th Jan 2025
[email protected]
43
Histogram
A histogram is a graphical representation that shows the relationship between gray levels (pixel intensities)
and their corresponding frequencies in an image.
h(rk) = nk for k = 0,1,2,3……L-1
Frequency

Where, nk is the number of pixels with intensity rk.

P(rk) = h(rk) / MN

P(rk) = Probabilities of Intensity levels occurring in an image.

Grey Levels
M,N are row and column of an image.

Sum of P(rk) for all values of k equal to 1.

Application
1. Image Enhancement
2. Image Thresholding
3. Image Segmentation
4. Image Analysis
44
Histogram Examples

45
Histogram Example
Histogram ? Intensity or Grey Level Frequency Normalized
Histogram
For a 3-bit image and size is 3x 3
1 3 3/9
1 2 6
2 1 1/9
6 1 3
3 1 1/9
1 6 6
6 4 4/9

For a 8-bit image and size is 3x3

150 200 115 Intensity or Grey Level Frequency Normalized
Histogram
200 125 150
250 100 250

46
Histogram Equalization
Its a technique used in image processing to improve the contrast of an image. It works by redistributing the intensity values
of the pixels in an image so that the histogram becomes more uniform.

Sk = T(rk)

Sk = L-1

Where –
L = number of possible intensity levels
Sk = output intensity levels
rk = input intensity levels
Why it is required ?

To enhance the contrast of an image, especially when the pixel intensity values are concentrated in a narrow range
(e.g., very dark or very bright images).

It makes details more visible in poorly contrasted images

47
Histogram Equalization

48
Histogram Equalization

49
Histogram Equalization Example
Consider a 3 bit image of size 64x64 (4096) with intensity distribution shown in the
below table. Calculate the Equalized histogram.

rk nk P(rk) Sk Approximate Value Updated nk Updated

of Sk P(rk)
0 790 790/4096
1 1023
2 850
3 656
4 329
5 245
6 122
7 81

Sk = T(rk)
S = L-1x(Pk) = 7x0.19 = 1.33
Sk = L-1 50
Histogram Equalization Example
Consider a 3 bit image of size 64x64 with intensity distribution shown in the below
table. Calculate the Equalized histogram.
rk nk P(rk) Sk Approximate Value Updated nk Updated
of Sk P(rk)
0 790 0.19 1.33 1 790 790/4096
1 1023 0.25 3.08 3 1023
2 850 0.21 4.55 5 850
3 656 0.16 5.67 6 656+329 = 985
4 329 0.08 6.23 6
5 245 0.06 6.65 7 245+122+81= 448
6 122 0.03 6.86 7
7 81 0.02 7.00 7

Sk = T(rk)

Sk = L-1
51
Concept of Kernal or Filter or Convolution Mask

X1
W1
W1X1 + W2X2

W2
X2

Weights Inputs Weighted Summation

W1 W2 W3 X1 X2 X3 W1X1 W2X2 W3X3
W4 W5 W6 X X4 X5 X6 = W4X4 W5X5 W6X6
W7 W8 W9 X7 X8 X9 W7X7 W8X8 W9X9

52
Spatial Correlation and Convolution: Padding size (M-1)/2 or (N-1)/2
Input Image 0 0 0 0 0 0 0
0 0 0 0 0 Kernel
0 0 0 0 0 0 0
0 0 0 0 0 1 2 3 0 0 0 0 0 0 0
0 0 1 0 0 4 5 6 0 0 0 1 0 0 0
0 0 0 0 0 7 8 9 0 0 0 0 0 0 0
0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0

Correlation Convolution
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 9 8 7 0 0 0 0 1 2 3 0 0
0 0 6 5 4 0 0 0 0 4 5 6 0 0
0 0 3 2 1 0 0 0 0 7 8 9 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 53
Smoothing Spatial Filters
Smoothing Spatial Filter

Linear filter Non-Linear Filter or order static filters

Median Filters
Box Car or Mean
Filter
Max Filter
Weighted Avg.
Filter
Min Filter

Gaussian Filters

54
Image Smoothening: Box Car Filter

A filter that computes the average of pixels in the neighborhood blurs an image. Computing an average is analogous to
integration.
Or

A filter that reduces the sharp transition in the intensity called as smoothening or low pass filtering.

Convolving a smoothening kernel with image result in image blurring and the amount of blurring is always depends on
the size of the kernel.
Kernal Normalized Kernal
1 1 1 0.11 0.11 0.11
1/9 x 1 1 1 = 0.11 0.11 0.11
1 1 1 0.11 0.11 0.11

1 1 1 1
1 1 1 1
1/16 x
1 1 1 1
1 1 1 1 55
Image Smoothening: Box Car Filter
Comparison of outputs between Normalized and Non-Normalized Kernel
Image Normalized Kernel
1 2 3 0.11 0.11 0.11
4 5 6 0.11 0.11 0.11 0.11+0.22+0.33+0.44+0.55+0.66+0.77+0.88+0.99 = 4.95
7 8 9 0.11 0.11 0.11

Not Normalized Kernal

1 2 3 1 1 1
4 5 6 1+2+3+4+5+6+7+8+9 = 45
1 1 1
7 8 9 1 1 1

56
Working Example: Padding size (M-1)/2 or (N-1)/2
Input Image
1 2 5 3 4 Kernel
5 6 7 8 9 0.11 0.11 0.11
2 3 4 5 6 0.11 0.11 0.11
3 6 8 4 2 0.11 0.11 0.11
1 5 6 8 7

Limitation of Box Car Filter

Output is Blurred.
Solution
Circularly Symmetric or isotropic kernel.

Weighted Average Filter

1 2 1
1/16 x 2 4 2
1 2 1
57
Image Smoothening: Gaussian
Filter

(x-1,y-1) (x-1,y) (x-1,y+1)

(x,y-1) p(x,y) (x+1,y+1)
(x+1,y-1) (x+1,y) (x+1,y+1)

58
Image Smoothening: Gaussian
Filter
Calculate the gaussian Kernel for 3x3 Kernel

(x-1,y-1) (x-1,y) (x-1,y+1) -1,-1 -1,0 -1,1

(x,y-1) p(x,y) (x,y+1) 0,-1 0,0 0,1
(x+1,y-1) (x+1,y) (x+1,y+1) 1,-1 1,0 1,1

Consider SD = 1

X=0,y=0 : SD =1 : 1/2pi =

59
 CSET344/CMCA544
Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 27th Jan to 31st Jan 2025
[email protected]
60
Image Smoothening: Gaussian
Filter

(x-1,y-1) (x-1,y) (x-1,y+1)

(x,y-1) p(x,y) (x,y+1)
(x+1,y-1) (x+1,y) (x+1,y+1)

61
Image Smoothening: Gaussian
Filter
Calculate the Gaussian Kernel for size

(x-1,y-1) (x-1,y) (x-1,y+1) -1,-1 -1,0 -1,1

(x,y-1) p(x,y) (x,y+1) 0,-1 0,0 0,1
(x+1,y-1) (x+1,y) (x+1,y+1) 1,-1 1,0 1,1

Sigma =1 Sigma =2 Sigma =3

0.3678 0.6065 0.3678 0.7788 0.8825 0.7788 0.8949 0.9464 0.8949

0.6065 1 0.6065 0.8825 1 0.8825 0.9464 1 0.9464
0.3678 0.6065 0.3678 0.7788 0.8825 0.7788 0.8949 0.9464 0.8949

62
Image Smoothening: Gaussian
Filter
Sigma =1 Sigma =2 Sigma =3

0.3678 0.6065 0.3678 0.7788 0.8825 0.7788 0.8949 0.9464 0.8949

0.6065 1 0.6065 0.8825 1 0.8825 0.9464 1 0.9464
0.3678 0.6065 0.3678 0.7788 0.8825 0.7788 0.8949 0.9464 0.8949

Sum = 4.7138 Sum = 7.6452 Sum = 8.3652

After Normalization After Normalization After Normalization

0.0780 0.1286 0.0780 0.1018 0.1154 0.1018 0.1069 0.1131 0.1069
0.1286 0.2121 0.1286 0.1154 0.1308 0.1154 0.1131 0.1195 0.1131
0.0780 0.1286 0.0780 0.1018 0.1154 0.1018 0.1069 0.1131 0.1069

63
Smoothing Spatial Filters
Smoothing Spatial Filter

Linear filter Non-Linear Filter or order static filters

Box Car or Mean Max Filter

Filter

Weighted Avg. Min Filter

Filter

Gaussian Filters Median Filter

64
Max Filter, Min Filter, Median Filter

65
Examples : Max Filter, Min Filter. Assume Kernel size is 3 x 3

Input Image First Step

3 7 17 18 13 0 0 0 0 0 0 0
10 5 2 20 5 0 3 7 17 18 13 0
9 8 13 1 7 0 10 5 2 20 5 0
16 8 7 20 19 0 9 8 13 1 7 0
14 19 3 30 10 0 16 8 7 20 19 0
0 14 19 3 30 10 0
0 0 0 0 0 0 0

Input Image After Padding Zeros

Second Step
0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 3 7 17 18 13 0 0 3 7 17 18 13 0
0 10 5 2 20 5 0
0 10 5 2 20 5 0
0 9 8 13 1 7 0
0 9 8 13 1 7 0
0 16 8 7 20 19 0
0 14 19 3 30 10 0 0 16 8 7 20 19 0
0 0 0 0 0 0 0 0 14 19 3 30 10 0
0 0 0 0 0 0 0 66
Examples : Max Filter, Min Filter. Assume Kernel size is 3 x 3

Max. Filter Output

10 17 20 20 20
10 17 20 20 20
Input Image After Padding Zeros

0 0 0 0 0 0 0
0 3 7 17 18 13 0
0 10 5 2 20 5 0
Min. Filter Output
0 9 8 13 1 7 0
0 16 8 7 20 19 0 0 0 0 0 0
0 14 19 3 30 10 0 0 2 1 1 0
0 0 0 0 0 0 0 0 2 1 1 0
0 3 1 1 0
0 0 0 0 0

Mid Point Filter = ½(Min filter + Max Filter)

67
Examples : Mean Filter, Median Filter. Assume Kernel size is 3 x 3

Mean Filter Output

Input Image After Padding Zeros

0 0 0 0 0 0 0
0 3 7 17 18 13 0
0 10 5 2 20 5 0
0 9 8 13 1 7 0
0 16 8 7 20 19 0
Median Filter Output
0 14 19 3 30 10 0
0 0 0 0 0 0 0 0 3 5 5 0
5 8 8 13 5

68
Comparison Between Max., Min. and Median Filters output

69
Image Denoising

Original Image Median Filter Output Image

70
Image Denoising

Original Image Median Filter Output Image

71
Edge Detection

Prewitt

Sobel
Edge Detector

Laplacian

Laplacian of Gaussian (LoG)

• High Pass Filters

• Unsharp Masking and High Boost Filtering

72
What is an Edge, Line and Point

Edge: Edge pixels are the pixels at which intensity of an image Changes abruptly.

Line: it may be viewed as a thin edge segment where intensity of background on either side of line is either
much higher or much lower.

Point: It may be viewed as foreground pixel surrounded by background and Vice-Versa. 73

Types of Edge

Averaging or smoothening is analogous to Integration.

Derivatives could able to detect the abrupt change in intensity.

74
Edge

1. Averaging or smoothening is analogous to Integration.

2. Derivatives could able to detect the abrupt change in the intensity.

3. In case of digital Images: Derivative is defined as finite difference

4. Approximation used for the first derivatives:

4.1 It must be zero in the area of constant intensity.

4.2 It must be non zero at the onset and end of an intensity step or ramp.
4.3 It must be non zero at points along an intensity ramp.

5. Approximation used for the Second derivatives:

5.1 It must be zero in the area of constant intensity.

5.2 It must be non zero at the onset and end of an intensity step or ramp.
5.3 It must be zero at points along an intensity ramp.
75
Edge

Black White Black

Sample Image

0 0 0 7 7 7 0 0 0

Step Edge

First Derivative

Second Derivative
Zero Crossing

76
Edge

77
Edge

Edge First Order Second Order

Ramp Non Zero Non zero at onset
(Thick Edge) and end of the ramp
(Thin Edge)
Point Magnitude Less Magnitude High
Line Magnitude Less Magnitude High

1. In case of ramp and step edge both the second

derivative has opposite sign.
2. If second derivative negative : Transition from
light to dark.
3. If second derivative positive : Transition from
dark to light.

78
Edge

Observations

1. First order derivative produces thicker edges.

2. Second order derivative have the stronger response for point, thin lines and noise.

3. Second order derivative produces double edge response at ramp and step transition in

intensity.

4. The sign of second derivative used to determine whether a transition into an edge from light

to dark or dark to light.

79
Edge
Original Image First Derivative Second Derivative

Noise - zero mean and SD = 0.0

Noise - zero mean and SD = 0.1

Noise - zero mean and SD = 1.0

80
 CSET344/CMCA544
Image and Video Processing

Dr. Gaurav Kumar Dashondhi

Ph.D. IIT Bombay

Overall Course Coordinator- Lect. Week

Dr. Gaurav Kumar Dashondhi 3rd Feb to 07th Feb2025
[email protected]
81
Types of Edge

Averaging or smoothening is analogous to Integration.

Derivatives could able to detect the abrupt change in intensity.

82
Edge Detection Image Gradient and its properties

Edge : Magnitude and Direction.

Image Gradient is a tool to identify edge Magnitude and Direction.

Gradient Vector

Magnitude of Gradient Vector

Direction of Gradient Vector

83
Edge Detection Image Gradient and its properties

Edge : An Abrupt or Sudden change in intensity which will help to identify the edges.

Derivatives could able to detect the abrupt change in the intensity.

Horizontal Edge Vertical Edge Right or Principal Diagonal Left Diagonal

10 10 10 10 0 255 0 10 10 255 255 0

0 0 0 10 0 255 255 0 10 255 0 10
255 255 255 10 0 255 255 255 0 0 10 10

84
Edge Detection Image Gradient and its properties

0 1 1 Z1 Z2 Z3
0 0 1 Z4 Z5 Z6
0 0 0 Z7 Z8 Z9

85
Edge Detectors

Prewitt

Sobel
Edge Detector

Laplacian

Laplacian of Gaussian (LoG)

• High Pass Filters

• Unsharp Masking and High Boost Filtering

86
Prewitt and Sobel Edge Detectors
Prewitt Edge Detector

-1 -1 -1 -1 0 1

0 0 0 -1 0 1

1 1 1 -1 0 1

Sobel Edge Detector

-1 -2 -1 -1 0 1

0 0 0 -2 0 2

1 2 1 -1 0 1

Note: Sum of all kernel coefficient is equal to zero. 87

Prewitt and Sobel Edge Detectors example
Find the edge
Prewitt Edge Detector 50 50 50 150 50 50 50
-1 -1 -1 -1 0 1 50 50 50 150 50 50 50
50 50 50 150 50 50 50
0 0 0 -1 0 1 50 50 50 150 50 50 50
50 50 50 150 50 50 50
1 1 1 -1 0 1
50 50 50 150 50 50 50
50 50 50 150 50 50 50

Sobel Edge Detector

50 50 150

-1 -2 -1 -1 0 1 50 50 150

0 0 0 -2 0 2 50 50 150

1 2 1 -1 0 1
Note: Sobel operator has better noise
suppression capability as compared to
Prewitt. 88
Sobel Edge Detectors
Original Image Gradient in x direction

Gradient Angle Image

Note: It generally not contain too much
information but it complement the
information extracted from image
gradient.
Gradient in y direction Gradient Image
89
Smooth image by 5 by 5 kernel and then apply Sobel Edge Detectors
Original Image Gradient in x direction

Gradient in y direction Gradient Image 90

Combining Gradient with Thresholding
Thresholded image
if pixel value > = 33% of the maximum value of gradient image

Output : White pixel

else:
Output: Black Pixel

Thresholding
helps to avoid
the Minor Edges.

Threshold applied on Gradient image Threshold applied on Smoothed Gradient image

Note: When interest lies in high lightening the principal edge and maintaining as much connectivity
as possible, it is common practice to use both smoothening and thresholding.

91
Laplacian of Gaussian (LoG)
Limitation of Prewitt and Sobel Operators

1. They only able to detect the edges in horizontal and vertical direction.
2. For different type of images kernel size will be different.

Silent Features required for any Edge Detectors

1.Second derivative could able to localize edges in the better way as compared to the first derivative.

2.Preprocessing is required before applying edge detector.

3. Edge detector should be isotropic i.e. it could detect edges in all the directions.

4.Larger Kernel should able to detect blurry edges and small kernel could able to detect the sharply focused fine
details.

LoG : Gaussian function first smooth the image and Laplacian will find the second derivative of gaussian
function which directly helps to locate the edge.

92
Laplacian of Gaussian (LoG)
LoG : Gaussian function first smooth the image and Laplacian
will find the second derivative of gaussian function which
directly helps to locate the edge.

Laplacian Function

Laplacian of Gaussian

93
Laplacian of Gaussian (LoG)
10 10 10 10 10
10 10 10 10 10 -1 -1 -1
100 100 100 100 100 -1 8 -1
100 100 100 100 100 -1 -1 -1
10 10 10 10 10
3 by 3 Kernel
Image

Output Image

94
Laplacian of Gaussian (LoG)
10 10 10 10 10
10 10 10 10 10 -1 -1 -1
100 100 100 100 100 -1 8 -1
100 100 100 100 100 -1 -1 -1
10 10 10 10 10
3 by 3 Kernel
Image

Transition from –ve to +ve, it

-220 -220 -220 directly shows the zero crossing
500 500 500 i.e. Edge Location.
500 500 500

95