Basics of Digital Images

Digital Images
Digital images are pictures that are created, stored, and manipulated in a digital format.

Digital images use pixels—tiny, square units of colour—to represent the image.

Digital images can be created using various devices such as digital cameras, smartphones, and
scanners.

They are typically stored in formats like JPEG, PNG, GIF, and TIFF, each with its own method of
compressing and encoding the image data.

These images can be easily edited, shared, and displayed on digital devices such as computers,
tablets, and smartphones.

Image Representation

Image representation refers to how visual information is encoded and stored in a digital format.
Here’s a breakdown of some key concepts in image representation:
1. Pixels
Definition: The smallest unit of a digital image. Each pixel represents a single point in the image.
Color and Brightness: Pixels have values for color and brightness. In color images, this often includes
red, green, and blue (RGB) values, while grayscale images have varying shades of grey.
2. Color Models
RGB (Red, Green, Blue): Commonly used for digital screens. Colors are created by combining red,
green, and blue light in varying intensities.
CMYK (Cyan, Magenta, Yellow, Key/Black): Used in color printing. It represents colors by subtracting
light using cyan, magenta, yellow, and black inks.
HSV (Hue, Saturation, Value): Represents colors in terms of their hue (color type), saturation
(intensity), and value (brightness).
3. Image Formats
JPEG (Joint Photographic Experts Group): Uses lossy compression to reduce file size while
maintaining reasonable image quality. Ideal for photographs and complex images.
PNG (Portable Network Graphics): Uses lossless compression and supports transparency. Good for
images with sharp edges, such as logos or graphics.
GIF (Graphics Interchange Format): Supports a limited color palette (256 colors) and is often used for
simple graphics and animations.
TIFF (Tagged Image File Format): Supports high-quality images and various color depths. Often used
in professional photography and imaging.
4. Resolution
Definition: The amount of detail an image holds, usually measured in pixels per inch (PPI) or dots per
inch (DPI). Higher resolution means more detail and clarity.
5. Compression
Lossy Compression: Reduces file size by removing some data from the image, which can reduce
quality (e.g., JPEG).
Lossless Compression: Reduces file size without losing any data or quality (e.g., PNG).

Image Acquisition Techniques

 Digital cameras use a lens to focus light onto an image sensor, which converts the light into
electrical signals that are then processed and stored as a digital image
o The lens's focal length, aperture, and focus distance affect the image's field of view,
depth of field, and sharpness
o The sensor's size, resolution, and sensitivity influence the image's detail, noise, and
low-light performance
 Scanners capture images by illuminating the subject with a light source and measuring the
reflected or transmitted light using a linear sensor array

o Flatbed scanners are used for digitizing documents, photographs, and other flat objects
o Film scanners are designed specifically for scanning negative or positive film strips or
slides

 Microscopes and telescopes use specialized optics to magnify small or distant objects, enabling
the acquisition of images at scales beyond the capabilities of standard cameras
 Satellite and aerial imaging involve capturing images of the Earth's surface from high altitudes
using sensors mounted on satellites, aircraft, or drones

o Multispectral and hyperspectral sensors are often used to gather information about
vegetation, water, and mineral resources

 Medical imaging techniques, such as X-ray, computed tomography (CT), magnetic resonance
imaging (MRI), and ultrasound, use various forms of energy to visualize internal structures of the
human body for diagnostic and research purposes

Image Sampling

Image sampling is the process of converting a continuous image (analog) into a discrete image
(digital) by selecting specific points from the continuous image. This involves measuring the image
at regular intervals and recording the intensity (brightness) values at those points.

Image Sampling Works

  Grid Overlay: A grid is placed over the continuous image, dividing
it into small, regular sections.

 Pixel Selection: At each intersection of the grid lines, a sample

point (pixel) is chosen.

Examples of Sampling
High Sampling Rate: A digital camera with a high megapixel count captures more details because it
samples the image at more points.

 Low Sampling Rate: An old VGA camera with a lower resolution captures less detail because
it samples the image at fewer points.

Image Quantization

Image quantization is the process of converting the continuous range of pixel values (intensities)
into a limited set of discrete values. This step follows sampling and reduces the precision of the
sampled values to a manageable level for digital representation.

Image Quantization Works

 Value Range Definition: The continuous range of pixel values is divided into a finite number
of intervals or levels.

 Mapping Intensities: Each sampled pixel intensity is mapped to the nearest interval value.

 Assigning Discrete Values: The original continuous intensity values are replaced by the
discrete values corresponding to the intervals.

Examples of Quantization

 High Quantization Levels: An image with 256 levels (8 bits per pixel) can represent shades of
gray more accurately.

 Low Quantization Levels: An image with only 4 levels (2 bits per pixel) has much less detail
and appears more posterized.

Key Differences Between Image Sampling and Quantization

Feature Image Sampling Image Quantization

Conversion of a continuous image into a Conversion of continuous pixel

discrete set of points by selecting specific intensity values into discrete
Definition pixel positions levels

Intensity values (brightness or

Spatial information (locations of pixels)
Process Focus color levels)
Feature Image Sampling Image Quantization

A grid of pixel values representing spatial A set of discrete intensity values

Outcome resolution for each pixel

Resolution Affects spatial resolution (detail and clarity Affects color/gray level resolution
Aspect of the image) (number of shades or colors)

Quantization levels (number of

Sampling rate (number of pixels sampled)
Determined By intensity levels)

More quantization levels

Higher sampling rate captures more spatial
Higher Value represent finer intensity
detail
Effects variations

High megapixel count in cameras for 8-bit color depth for more color
Example detailed images variations

Crucial for applications needing

Crucial for applications needing high spatial
Application high color fidelity like graphic
detail like medical imaging
Impact design

Storage Increases with more quantization

Increases with higher sampling rates
Requirement levels

Measured in pixels per inch (PPI) or dots per

Measured in bits per pixel (bpp)
Typical Values inch (DPI)

Advantages and Disadvantages of Image Sampling and Quantization

Image Sampling:

Advantages:

1. Data Reduction: Converts a continuous signal into a finite set of points, making storage and
processing more manageable.

2. Compatibility: Sampled images are easily processed by digital systems and algorithms.

3. Resolution Control: Allows for control over image resolution by adjusting the sampling rate.

Disadvantages:
 1. Information Loss: Inevitably loses some information by approximating a continuous signal.

2. Aliasing: Can cause distortions and artifacts if the sampling rate is too low.

3. Computationally Intensive: High-resolution sampling demands significant computational

resources and storage space.

Image Quantization:

Advantages:

1. Data Compression: Reduces the amount of data by limiting the number of possible values
for each pixel.

2. Simplified Processing: Makes image processing operations simpler and faster with fewer
distinct values.

3. Noise Reduction: Helps reduce the impact of noise by mapping small variations in intensity
to the same value.

Disadvantages:

1. Loss of Detail: Reduces the range of colors or intensity levels, leading to a loss of fine detail
and potential color banding.

2. Quantization Error: Introduces differences between the original and quantized values, which
can become noticeable.

3. Reduced Image Quality: Overly aggressive quantization can significantly degrade image
quality, making the image appear blocky or posterized

Histogram equalization

is a method in image processing of contrast adjustment using the image’s histogram. This method
usually increases the global contrast of many images, especially when the usable data of the image is
represented by close contrast values. Through this adjustment, the intensities can be better
distributed on the histogram. This allows for areas of lower local contrast to gain a higher contrast.
Histogram equalization accomplishes this by effectively spreading out the most frequent intensity
values. The method is useful in images with backgrounds and foregrounds that are both bright or
both dark. OpenCV has a function to do this,

cv2.equalizeHist()

. Its input is just grayscale image and output is our histogram equalized image.

Input Image :
Below is Python3 code implementing Histogram Equalization :

Python3 1==

# import Opencv

import cv2

# import Numpy

import numpy as np

# read a image using imread

img = cv2.imread(\'F:\\do_nawab.png\', 0)

# creating a Histograms Equalization

# of a image using cv2.equalizeHist()

equ = cv2.equalizeHist(img)

# stacking images side-by-side

res = np.hstack((img, equ))

# show image input vs output

cv2.imshow(\'image\', res)

cv2.waitKey(0)

cv2.destroyAllWindows()

Output :
Spatial domain methods

Spatial domain processing is a fundamental approach in image analysis, working directly with pixel
values to enhance and modify visual information. This technique forms the basis for many image
processing methods, allowing for intuitive and efficient operations on digital images.
Understanding spatial domain concepts is crucial for effective image manipulation and analysis. From
basic pixel operations to complex filtering techniques, these methods enable a wide range of
applications in computer vision, medical imaging, remote sensing, and other fields where image data
plays a vital role.

Fundamentals of spatial domain

 Spatial domain processing operates directly on pixel values in an image, manipulating intensity
levels to enhance or modify visual information
 This approach forms the foundation for many image processing techniques, allowing for intuitive
and computationally efficient operations on digital images
 Understanding spatial domain concepts enables effective analysis and manipulation of images
as data, crucial for various applications in computer vision and image processing

Definition and characteristics

Top images from around the web for Definition and characteristics



scikit-image: image processing in Python [PeerJ] View original

Is this image relevant?


 
Frontiers | Grand Challenges in Satellite Remote Sensing View original

Is this image relevant?




Introduction to Analysis – Introduction to Geomatics View original

Is this image relevant?


 
scikit-image: image processing in Python [PeerJ] View original

Is this image relevant?




Frontiers | Grand Challenges in Satellite Remote Sensing View original

Is this image relevant?

1 of 3

 Refers to the image plane itself, where processing occurs directly on pixel values
 Operates on the spatial coordinates (x, y) of pixels in an image
  Allows for direct manipulation of pixel intensities without transforming the image to another
domain
 Enables localized modifications based on pixel neighborhoods or regions of interest

Pixel-based operations
 Involve manipulating individual pixel values without considering neighboring pixels
 Include point processing techniques such as thresholding, brightness adjustment, and contrast
enhancement
 Utilize pixel-wise mathematical operations (addition, subtraction, multiplication) to modify image
intensities
 Apply lookup tables (LUTs) for efficient implementation of complex pixel transformations

Neighborhood operations
 Process pixels based on the values of surrounding pixels in a defined neighborhood
 Utilize spatial filters or masks to perform local operations on pixel groups
 Include smoothing filters for noise reduction and sharpening filters for edge enhancement
 Implement operations like median filtering, which replaces pixel values with the median of
neighboring pixels

Image enhancement techniques

 Image enhancement aims to improve the visual quality or interpretability of images for human
viewers or subsequent processing tasks
 These techniques play a crucial role in preparing images as data for analysis, highlighting
important features, and reducing unwanted artifacts
 Understanding various enhancement methods allows for effective preprocessing of images in
computer vision and image analysis applications

Histogram equalization
 Redistributes pixel intensities to enhance overall image contrast
 Spreads out the most frequent intensity values across the available range
 Calculates cumulative distribution function (CDF) of pixel intensities
 Applies a transformation function to map original intensities to new, equalized values
 Particularly effective for images with poor contrast or limited dynamic range

Contrast stretching
 Expands the range of intensity values in an image to utilize the full dynamic range
 Identifies minimum and maximum intensity values in the original image
 Applies a linear scaling function to map original intensities to the desired range
 Can be performed globally or locally to enhance specific image regions
 Useful for improving visibility of details in low-contrast images (satellite imagery)

Noise reduction filters

 Reduce unwanted variations in pixel intensities caused by various sources of noise
  Implement spatial filtering techniques to smooth out noise while preserving important image
features
 Include methods such as:

o Gaussian filtering: applies a weighted average based on a Gaussian distribution

o Median filtering: replaces pixel values with the median of neighboring pixels
o Bilateral filtering: combines spatial and intensity information to preserve edges while
smoothing

Spatial filtering operations

 Spatial filtering involves applying a filter or kernel to an image to modify pixel values based on
their local neighborhood
 These operations form the basis for many image processing tasks, including edge detection,
noise reduction, and feature extraction
 Understanding spatial filtering is crucial for manipulating images as data and extracting
meaningful information for further analysis

Linear vs nonlinear filters

 Linear filters apply a weighted sum operation to pixel neighborhoods

o Output is a linear combination of input pixel values

o Include averaging filters, Gaussian filters, and Sobel operators

 Nonlinear filters use more complex operations that cannot be expressed as a linear combination

o Include median filters, morphological operations, and adaptive filters

o Often better at preserving edges and fine details while reducing noise

Convolution process
 Fundamental operation in spatial filtering, involving sliding a kernel over an image
 Multiplies kernel values with corresponding pixel intensities in the neighborhood
 Sums the products to produce the output pixel value
 Can be expressed mathematically
as:g(x,y)=∑s=−aa∑t=−bbh(s,t)f(x−s,y−t)g(x,y)=∑s=−aa∑t=−bbh(s,t)f(x−s,y−t)
 Where g(x,y)g(x,y) is the output image, f(x,y)f(x,y) is the input image,
and h(s,t)h(s,t) is the kernel

Kernel design principles

 Kernels (also called masks or filters) are small matrices used in convolution
 Size and values of the kernel determine the specific filtering effect
 Symmetric kernels ensure consistent behavior in all directions
 Normalization (sum of kernel values equals 1) maintains overall image brightness
 Different kernel designs achieve various effects:

o Box filter for simple smoothing

o Gaussian kernel for edge-preserving smoothing
 o Sobel or Prewitt kernels for edge detection

Edge detection methods

 Edge detection identifies significant local changes in image intensity, revealing object
boundaries and structural information
 These techniques are fundamental in image analysis, segmentation, and feature extraction for
computer vision tasks
 Understanding various edge detection methods allows for effective extraction of important image
features in data analysis applications

Gradient-based approaches
 Detect edges by calculating the first-order derivatives of image intensity
 Compute gradient magnitude and direction to identify edge strength and orientation
 Common gradient operators include:

o Sobel operator: uses 3x3 kernels to approximate horizontal and vertical gradients
o Prewitt operator: similar to Sobel but with different kernel weights
o Roberts cross operator: uses 2x2 kernels for diagonal edge detection

 Threshold gradient magnitudes to obtain binary edge maps

Laplacian operators
 Utilize second-order derivatives to detect edges in all directions simultaneously
 Identify edges as zero-crossings in the second derivative of image intensity
 Laplacian operator is defined as: ∇2f=∂2f∂x2+∂2f∂y2∇2f=∂x2∂2f+∂y2∂2f
 Often combined with Gaussian smoothing (Laplacian of Gaussian) to reduce noise sensitivity
 Produce thin edges but are more sensitive to noise compared to gradient-based methods

Canny edge detector

 Multi-stage algorithm considered optimal for edge detection in many applications
 Steps include:

1. Gaussian smoothing to reduce noise

2. Gradient computation using Sobel operators
3. Non-maximum suppression to thin edges
4. Double thresholding to identify strong and weak edges
5. Edge tracking by hysteresis to connect weak edges to strong ones

 Produces clean, thin edges with good localization and low false positive rate
 Widely used in computer vision for its robust performance across various image types

Image sharpening techniques

  Image sharpening enhances the visibility of edges and fine details in an image, improving overall
perceived sharpness
 These techniques are crucial for enhancing image quality in various applications, from medical
imaging to digital photography
 Understanding sharpening methods allows for effective enhancement of important features in
image data analysis

Unsharp masking
 Creates a sharpened image by subtracting a blurred version from the original
 Process involves:

1. Blurring the original image (typically using Gaussian blur)

2. Subtracting the blurred image from the original to create an "unsharp mask"
3. Adding a scaled version of the mask back to the original image

 Enhances edges and fine details while preserving overall image structure
 Amount of sharpening controlled by scaling factor applied to the unsharp mask

High-boost filtering
 Extends unsharp masking by amplifying high-frequency components more aggressively
 g(x,y)=A⋅f(x,y)−fsmooth(x,y) g(x,y)=A⋅f(x,y)−fsmooth(x,y)
General form:
 Where A>1A>1 is a boost factor, f(x,y)f(x,y) is the original image,
and fsmooth(x,y)fsmooth(x,y) is the smoothed version
 Allows for greater control over the degree of sharpening and overall image brightness
 Useful for enhancing subtle details in low-contrast images (astronomical imagery)

Laplacian sharpening
 Utilizes the Laplacian operator to enhance edges in all directions
 Process involves:

1. Calculating the Laplacian of the image

2. Subtracting the scaled Laplacian from the original image

 Can be expressed as: g(x,y)=f(x,y)−c⋅∇2f(x,y) g(x,y)=f(x,y)−c⋅∇2f(x,y)

 Where cc is a scaling factor controlling the amount of sharpening
 Effectively enhances edges while maintaining overall image brightness
 Often combined with smoothing to reduce noise sensitivity (Laplacian of Gaussian)

Morphological operations
 Morphological operations are nonlinear filters based on set theory and mathematical morphology
 These techniques are powerful tools for analyzing and processing the shape and structure of
objects in binary and grayscale images
 Understanding morphological operations is crucial for tasks such as noise removal, object
segmentation, and feature extraction in image data analysis
Dilation and erosion
 Fundamental morphological operations that form the basis for more complex transformations
 Dilation:

o Expands objects by adding pixels to boundaries

o Uses a structuring element to determine the extent of expansion
o Fills small holes and connects nearby objects
o Useful for closing gaps in broken lines or objects

 Erosion:

o Shrinks objects by removing pixels from boundaries

o Also uses a structuring element to define the erosion pattern
o Removes small protrusions and separates connected objects
o Effective for eliminating small noise particles or thinning objects

Opening and closing

 Compound operations formed by combining dilation and erosion
 Opening:

o Erosion followed by dilation

o Smooths object contours, breaks narrow isthmuses, and eliminates small islands
o Useful for removing small noise particles while preserving larger structures

 Closing:

o Dilation followed by erosion

o Fills small holes and gaps, connects nearby objects
o Effective for closing small breaks in object boundaries or filling small holes

Hit-or-miss transform
 Advanced morphological operation used for shape detection and feature extraction
 Utilizes two structuring elements: one for foreground and one for background
 Identifies locations where the foreground structuring element fits the object and the background
element fits the complement
 Applications include:

o Detecting specific shapes or patterns in binary images

o Locating corners or junction points in object boundaries
o Template matching for simple geometric shapes

Spatial domain vs frequency domain

 Image processing can be performed in either the spatial domain (directly on pixel values) or the
frequency domain (after applying a transform like Fourier)
 Understanding the differences between these approaches is crucial for selecting appropriate
techniques for various image analysis tasks
  Both domains offer unique advantages and are often used complementarily in comprehensive
image processing pipelines

Advantages and limitations

 Spatial domain advantages:

o Intuitive and direct manipulation of pixel values

o Easier to visualize and interpret results
o Suitable for local operations and edge detection

 Spatial domain limitations:

o Some operations can be computationally intensive for large images

o Difficulty in separating frequency components

 Frequency domain advantages:

o Efficient for certain global operations (filtering, compression)

o Allows easy separation and manipulation of frequency components
o Useful for analyzing periodic patterns and textures

 Frequency domain limitations:

o Less intuitive interpretation of operations

o Requires additional transforms (forward and inverse)

Computational efficiency
 Spatial domain:

o Efficiency depends on the size of the image and the complexity of the operation
o Simple point operations are very fast
o Neighborhood operations can become slow for large kernels or images

 Frequency domain:

o Initial transform (FFT) has complexity of O(N log N) for an NxN image
o Many operations become simple multiplications in frequency domain
o Efficient for global operations and large-scale filtering

 Hybrid approaches:

o Combine spatial and frequency domain techniques for optimal performance

o Use frequency domain for global operations, spatial for local refinements

Application scenarios
 Spatial domain applications:

o Edge detection and local feature extraction

 o Noise reduction using median or bilateral filtering
o Morphological operations for shape analysis

 Frequency domain applications:

o Image compression (JPEG)

o Global filtering (low-pass, high-pass, band-pass)
o Periodic noise removal
o Texture analysis and synthesis

 Choice of domain depends on:

o Nature of the processing task

o Computational resources available
o Required accuracy and speed of the operation

Implementation considerations
 Implementing spatial domain processing techniques requires careful consideration of various
factors to ensure efficient and effective image analysis
 Understanding these considerations is crucial for developing robust image processing systems
and optimizing performance in real-world applications
 Balancing algorithm complexity, hardware capabilities, and processing requirements is essential
for successful implementation of image analysis tasks

Algorithm complexity
 Impacts processing time and resource requirements for spatial domain operations
 Considerations include:

o Time complexity: how processing time scales with image size

o Space complexity: memory requirements for storing intermediate results
o Trade-offs between accuracy and speed for different algorithms

 Optimization techniques:

o Efficient data structures for representing images and intermediate results

o Algorithmic improvements (fast box filters, integral images)
o Parallelization of operations where possible

Hardware acceleration
 Utilizes specialized hardware to speed up spatial domain processing tasks
 Graphics Processing Units (GPUs):

o Highly parallel architecture suitable for many image processing operations

o CUDA and OpenCL frameworks for GPU-accelerated image processing
o Significant speedups for operations like convolution and morphological processing

 Field-Programmable Gate Arrays (FPGAs):

 o Custom hardware designs for specific image processing tasks
o Low latency and high throughput for real-time applications
o Suitable for embedded systems with specific processing requirements

 Digital Signal Processors (DSPs):

o Optimized for signal processing tasks, including image processing

o Efficient for operations involving complex mathematical computations

Real-time processing challenges

 Achieving real-time performance requires balancing processing speed and accuracy
 Challenges include:

o Meeting strict timing constraints for video or streaming applications

o Handling varying input data rates and image sizes
o Minimizing latency between input acquisition and output generation

 Strategies for real-time processing:

o Algorithm simplification or approximation where appropriate

o Multi-threading and parallel processing techniques
o Pipelining of operations to maximize throughput
o Adaptive processing based on available computational resources
o Efficient memory management to minimize data transfer bottlenecks

Applications in image analysis

 Spatial domain processing techniques find wide-ranging applications across various fields of
image analysis and computer vision
 These methods form the foundation for extracting meaningful information from images as data in
diverse domains
 Understanding the applications of spatial domain processing is crucial for leveraging these
techniques in real-world problem-solving and data analysis tasks

Medical imaging
 Enhances diagnostic capabilities and supports medical research through image processing
 Applications include:

o Noise reduction in X-ray and MRI images to improve clarity

o Edge detection for identifying organ boundaries and tumors
o Contrast enhancement to highlight subtle tissue differences
o Segmentation of anatomical structures for 3D visualization
o Texture analysis for characterizing tissue properties

 Supports computer-aided diagnosis (CAD) systems for early detection of diseases (cancer)

Remote sensing
  Processes satellite and aerial imagery for environmental monitoring and mapping
 Key applications:

o Image enhancement to improve visibility of land features

o Change detection for monitoring deforestation or urban growth
o Object detection and classification (buildings, roads, vegetation)
o Atmospheric correction to remove haze and improve image quality
o Pansharpening to combine high-resolution panchromatic with lower-resolution
multispectral data

 Supports applications in agriculture, urban planning, and natural resource management

Computer vision tasks

 Utilizes spatial domain processing as a foundation for higher-level vision tasks
 Applications in various domains:

o Object detection and recognition in autonomous vehicles

o Facial recognition for security and authentication systems
o Optical character recognition (OCR) for document digitization
o Quality control in manufacturing (defect detection)
o Augmented reality for overlaying digital information on real-world images

 Combines spatial domain techniques with machine learning for advanced image understanding
and analysis

Frequency domain processing

Frequency domain processing transforms images from spatial coordinates to frequency components,
revealing patterns and periodicities. This approach enables efficient analysis of global image
characteristics, complementing spatial domain techniques and offering insights into image structure and
content distribution.

The Fourier transform is key to frequency domain analysis, decomposing images into sinusoidal
components. This allows for specialized processing techniques, including filtering, noise reduction, and
edge sharpening, which can be more efficient in the frequency domain than in the spatial domain.

Fundamentals of frequency domain

 Frequency domain analysis transforms image data from spatial coordinates to frequency
components, revealing underlying patterns and periodicities
 Enables efficient processing of global image characteristics, crucial for various image
enhancement and analysis tasks in the field of Images as Data
 Provides a complementary perspective to spatial domain techniques, offering insights into image
structure and content distribution

Spatial vs frequency domain

Top images from around the web for Spatial vs frequency domain

 

Frontiers | The Role of Low-Spatial Frequency Components in the

Processing of Deceptive Faces: A ... View original


Is this image relevant?




Frontiers | Differential Effects of Orientation and Spatial-Frequency

Spectra on Visual ... View original


Is this image relevant?



 

bastibe.de View original



Is this image relevant?




Frontiers | The Role of Low-Spatial Frequency Components in the

Processing of Deceptive Faces: A ... View original


Is this image relevant?



 

Frontiers | Differential Effects of Orientation and Spatial-Frequency

Spectra on Visual ... View original


Is this image relevant?


3 of 3

 Spatial domain represents images as intensity values at specific pixel coordinates

 Frequency domain decomposes images into sinusoidal components of varying frequencies and
amplitudes
 Spatial domain operations focus on local pixel neighborhoods, while frequency domain
manipulates global image characteristics
 Frequency domain facilitates analysis of repetitive patterns and texture information in images
 Transformations between domains allow for specialized processing techniques tailored to
specific image analysis tasks

Fourier transform basics

 Mathematical tool that decomposes signals into constituent sinusoidal components
 Represents images as a sum of complex exponentials with different frequencies and amplitudes
 F(u,v)=∫−∞∞∫−∞∞f(x,y)e−j2π(ux+vy)dxdyF(u,v)=∫−∞∞∫−∞∞
Utilizes the formula
f(x,y)e−j2π(ux+vy)dxdy for continuous 2D signals
 Enables analysis of image content in terms of frequency components rather than spatial
coordinates
 Inverse Fourier transform reconstructs the original image from its frequency representation

Discrete Fourier transform (DFT)

 Adapts Fourier transform for digital images with finite, discrete pixel values
 Computes frequency components for a discrete set of frequencies
  Uses the
formulaF(u,v)=1MN∑x=0M−1∑y=0N−1f(x,y)e−j2π(uxM+vyN)F(u,v)=MN1
∑x=0M−1∑y=0N−1f(x,y)e−j2π(Mux+Nvy) for M x N images
 Produces a complex-valued output representing frequency content of the input image
 Enables efficient computation through algorithms like the Fast Fourier Transform (FFT)

Image representation in frequency

 Frequency domain representation provides insights into image structure and content distribution
 Facilitates analysis of global image characteristics and periodic patterns
 Enables efficient manipulation of specific frequency components for various image processing
tasks

Magnitude and phase spectra

 Magnitude spectrum represents the strength of different frequency components in the image
 Phase spectrum encodes the relative positions of frequency components
 Computed using ∣F(u,v)∣=R2(u,v)+I2(u,v)∣F(u,v)∣=R2(u,v)+I2(u,v) for magnitude
and ϕ(u,v)=tan⁡−1(I(u,v)R(u,v))ϕ(u,v)=tan−1(R(u,v)I(u,v)) for phase
 Magnitude spectrum often visualized as a grayscale image with bright spots indicating strong
frequency components
 Phase spectrum crucial for preserving spatial relationships and edge information in images

Low vs high frequency components

 Low frequencies represent slowly varying intensity changes and overall image structure
 High frequencies correspond to rapid intensity variations, edges, and fine details
 Low frequencies concentrated near the center of the frequency domain representation
 High frequencies located towards the periphery of the frequency domain image
 Manipulating specific frequency ranges allows for targeted image enhancement and analysis

2D Fourier transform
 Extends 1D Fourier transform to two-dimensional image data
 Computes frequency components along both horizontal and vertical directions
 Produces a 2D frequency domain representation with u and v frequency coordinates
 Enables analysis of directional patterns and textures in images
 Facilitates operations like filtering and compression in the frequency domain

Frequency domain filters

 Frequency domain filters modify specific frequency components to achieve desired image
processing effects
 Enable global image manipulation by altering the Fourier transform of the image
 Provide efficient alternatives to spatial domain filtering for certain image enhancement tasks

Low-pass vs high-pass filters

  Low-pass filters attenuate high-frequency components while preserving low frequencies
 High-pass filters suppress low-frequency components and enhance high frequencies
 Low-pass filters used for image smoothing and noise reduction (Gaussian blur)
 High-pass filters applied for edge detection and image sharpening (Laplacian filter)
 Combination of low-pass and high-pass filters creates band-pass filters for selective frequency
range manipulation

Ideal vs Gaussian filters

 Ideal filters have a sharp cutoff frequency, abruptly transitioning between passed and blocked
frequencies
 Gaussian filters use a smooth, bell-shaped frequency response for gradual attenuation
 H(u,v)={1if D(u,v)≤D00if D(u,v)>D0H(u,v)={10
Ideal filters defined by
if D(u,v)≤D0if D(u,v)>D0 where D(u,v) is the distance from the origin
 Gaussian filters use H(u,v)=e−D2(u,v)/(2σ2)H(u,v)=e−D2(u,v)/(2σ2) where σ controls the
filter's spread
 Gaussian filters often preferred due to reduced ringing artifacts compared to ideal filters

Butterworth filter characteristics

 Butterworth filters offer a compromise between ideal and Gaussian filter characteristics
 Provide a smooth transition between passed and blocked frequencies with adjustable rolloff
 Defined by the transfer function H(u,v)=11+[D(u,v)/D0]2nH(u,v)=1+[D(u,v)/D0]2n1 for low-
pass filtering
 Order n controls the steepness of the frequency response curve
 Higher-order Butterworth filters approach the behavior of ideal filters while maintaining smoother
transitions

Image enhancement techniques

 Frequency domain techniques offer powerful tools for improving image quality and extracting
useful information
 Enable global image manipulation by modifying specific frequency components
 Provide efficient alternatives to spatial domain methods for certain enhancement tasks

Noise reduction in frequency domain

 Exploits the fact that noise often manifests as high-frequency components in images
 Applies low-pass filtering to attenuate high-frequency noise while preserving image structure
 Wiener filtering adapts to local image statistics for optimal noise reduction
 Notch filters remove periodic noise patterns by targeting specific frequency components
 Homomorphic filtering separates illumination and reflectance components for improved noise
reduction

Edge sharpening methods

 Utilize high-pass filtering to enhance high-frequency components associated with edges
 Unsharp masking boosts high frequencies by subtracting a blurred version from the original
image
  Laplacian filtering in the frequency domain enhances edges by amplifying high-frequency
components
 Emphasizes fine details and improves image contrast by modifying the magnitude spectrum
 Can be combined with noise reduction techniques for optimal image enhancement

Homomorphic filtering
 Addresses non-uniform illumination issues in images by separating illumination and reflectance
components
 Applies the logarithm to convert multiplicative illumination effects to additive components
 Utilizes high-pass filtering in the frequency domain to reduce low-frequency illumination
variations
 Enhances image contrast and normalizes brightness across the image
 Inverse operation reconstructs the enhanced image with improved illumination characteristics

Frequency domain operations

 Frequency domain enables efficient implementation of various image processing operations
 Exploits properties of the Fourier transform to simplify complex spatial domain computations
 Facilitates analysis and manipulation of global image characteristics

Convolution theorem
 States that convolution in the spatial domain equals multiplication in the frequency domain
 Expressed mathematically
as F{f(x,y)∗h(x,y)}=F(u,v)H(u,v) F{f(x,y)∗h(x,y)}=F(u,v)H(u,v)
 Simplifies filtering operations by replacing spatial convolution with frequency domain
multiplication
 Enables efficient implementation of large convolution kernels
 Particularly useful for operations involving large filters or repeated convolutions

Correlation in frequency domain

 Correlation between two images computed efficiently using frequency domain techniques
 Utilizes the
∗
relationship F{f(x,y)g(x,y)}=F∗(u,v)G(u,v) F{f(x,y)⋆g(x,y)}=F∗(u,v)G(u,v) wher
e * denotes complex conjugate
 Facilitates template matching and pattern recognition tasks in image processing
 Enables efficient computation of autocorrelation for texture analysis
 Cross-correlation in frequency domain used for image registration and motion estimation

Sampling and aliasing effects

 Sampling in spatial domain corresponds to periodicity in the frequency domain
 Nyquist-Shannon sampling theorem defines the minimum sampling rate to avoid aliasing
 Aliasing occurs when high-frequency components are undersampled, causing distortion
 Manifests as spurious low-frequency components in the frequency domain representation
 Prevented by ensuring the sampling frequency is at least twice the highest frequency in the
image
Applications in image processing
 Frequency domain techniques find widespread use in various image processing applications
 Enable efficient implementation of complex operations and analysis tasks
 Provide unique insights into image structure and content distribution

Compression using DCT

 Discrete Cosine Transform (DCT) used in JPEG and other image compression standards
 Transforms image blocks into frequency domain representations
 Concentrates image energy in low-frequency coefficients for efficient coding
 Quantization of DCT coefficients allows for lossy compression with controllable quality
 Inverse DCT reconstructs approximated image blocks from compressed data

Pattern recognition techniques

 Frequency domain analysis reveals characteristic patterns in image spectra
 Rotation-invariant features extracted from magnitude spectra for object recognition
 Mel-frequency cepstral coefficients (MFCCs) derived from frequency domain for texture
classification
 Fourier descriptors used for shape analysis and recognition tasks
 Frequency domain correlation techniques enable efficient template matching and object
detection

Texture analysis methods

 Frequency domain representations capture periodic patterns and structural information in
textures
 Power spectrum analysis reveals dominant frequencies and orientations in textured regions
 Ring and wedge filters extract specific frequency bands for texture feature computation
 Gabor filters in the frequency domain provide multi-scale and multi-orientation texture analysis
 Wavelet transforms offer localized frequency analysis for texture segmentation and classification

Implementation considerations
 Practical implementation of frequency domain techniques requires careful consideration of
computational aspects
 Efficient algorithms and software tools enable real-time processing of large image datasets
 Understanding implementation details crucial for optimizing performance in image processing
applications

Fast Fourier transform (FFT)

 Efficient algorithm for computing the Discrete Fourier Transform (DFT)
 Reduces computational complexity from O(N^2) to O(N log N) for N-point DFT
 Radix-2 FFT algorithm widely used for power-of-two sized inputs
 Utilizes divide-and-conquer approach to recursively compute smaller DFTs
 Enables real-time frequency domain processing of large images and video streams
Computational complexity
 Frequency domain operations often more efficient for large filter sizes compared to spatial
domain
 Trade-off between computational cost of forward/inverse transforms and efficiency of frequency
domain processing
 Memory requirements increase for storing complex-valued frequency domain representations
 Parallel processing techniques can significantly accelerate frequency domain computations
 GPU acceleration commonly used for real-time frequency domain image processing tasks

Software tools for frequency processing

 Numerous libraries and frameworks available for implementing frequency domain techniques
 NumPy and SciPy in Python provide efficient FFT implementations and related functions
 OpenCV offers optimized frequency domain processing routines for computer vision applications
 MATLAB's Image Processing Toolbox includes comprehensive frequency domain analysis tools
 Custom CUDA or OpenCL implementations enable GPU-accelerated frequency domain
processing

Limitations and challenges

 Frequency domain techniques, while powerful, come with certain limitations and challenges
 Understanding these issues crucial for proper interpretation and application of frequency domain
methods
 Careful consideration required to mitigate artifacts and ensure accurate results

Ringing artifacts
 Gibbs phenomenon causes oscillations near sharp discontinuities in frequency domain filtering
 Results from truncation of high-frequency components in the Fourier series representation
 Manifests as ripple-like patterns around edges in the processed image
 Mitigated by using smooth transition filters (Gaussian) instead of ideal filters
 Windowing techniques applied to reduce ringing artifacts in certain applications

Boundary effects
 Periodic nature of DFT assumes image content repeats infinitely in all directions
 Leads to artifacts at image boundaries when applying frequency domain operations
 Discontinuities at image edges introduce high-frequency components in the spectrum
 Mitigated by techniques like image padding, symmetric extension, or windowing
 Careful handling of boundary conditions required for accurate frequency domain analysis

Interpretation of frequency results

 Frequency domain representations can be unintuitive and challenging to interpret directly
 Magnitude spectra often easier to visualize and understand compared to phase spectra
 Log-scaling of magnitude spectra often used to enhance visibility of low-amplitude components
 Proper normalization and scaling required for meaningful comparison of frequency domain
results
  Understanding of Fourier transform properties crucial for correct interpretation of processed
images

2.image restoration
Image degradration model :
Image restoration is the process of recovering an image that has been degraded by some
knowledge of degradation function H and the additive noise term . Thus in
restoration, degradation is modelled and its inverse process is applied to recover the original
image.

Fig: Image Restoration and Image Degradation Model

Objective of image restoration:

Image restoration certifications

The objective of image restoration is to obtain an estimate of the original image .

Here, by some knowledge of H and , we find the appropriate restoration filters, so

that output image is as close as original image as possible since it is

practically not possible (or very difficult) to completely (or exactly) restore the original image.

Terminology:

 = degraded image
 = input or original image

 = recovered or restored image

 = additive noise term
In spatial domain:

where, represents convolution

In frequency domain:
After taking fourier transform of the above equation:

If the restoration filter applied is , then

(for restoration)

as restoration filter is the reverse of degration function and neglecting the

noise term. Here, is linear and position invariant.

Noise Reduction algorithms

Now that we have understood convolution, let us look at image filtering and some of
the most commonly used image filtering methods.

A digital image often contains noise. Noise introduces erroneous pixel values. Image
filtering is the process of removing these errors. Convolving a noisy image with an
appropriate kernel practically nullifies the noise.

Let us have a look at the different image filtering methods in the subsequent
paragraphs. For all the below filters, let the kernel size be K height*Kwidth.

Normalized Box Filter:

If all the elements of a kernel are given unit values, convolving it with image would
mean replacing the pixel values with the sum of its neighbor in K height*Kwidth window.
If each element in kernel is now divided by the kernel size, then the sum of all
elements will be 1, the normalized form. Such a convolution would result in pixel
values being replaced by mean or arithmetic average of neighboring pixels in the
Kheight*Kwidth window. This effect can be used to achieve smoothness in images.
Taking average would mean reduction in sudden changes in intensity values between
neighboring pixels.

Normalized box filter of size 3*3 =

Gaussian Filter:
A Gaussian equation is of the following form:

————————————— eq(1)

In the above diagram, we can see that the values near the reference point are more
significant. This same Gaussian distribution is achieved in a 2-dimensional kernel
with the reference point being matrix center. As opposed to Normalized Box filter
which gives equal weight to all neighboring pixels, a Gaussian kernel gives more
weight to pixels near the current pixel and much lesser weight to distant pixels when
calculating sum.

Bilateral Filter:
While Gaussian filter gives us more control and accurate results than box filter, both
the filters err when taking weighted sum of edge pixels. Both of the filters smoothen
the edge pixels, thereby diminishing the intensity variant. Bilateral filter overcomes
this shortcoming.

While Gaussian filter assigns weight to neighbors based only on their distance from
the current pixel, bilateral filter brings intensity to the picture. Now the shape of the
Gaussian function is determined solely based on or the standard deviation. Let us
assume the following symbols:

 Çd = standard deviation of Gaussian used for spatial filtering

 ci = standard deviation of Gaussian used for intensity filtering
 α = a pixel in the input image
 I(α) = intensity at pixel α
 If (α) = value at pixel α after taking weighted sum of neighboring pixels as per
the convolving kernel
 || αr – αs || = Euclidian distance between the two pixels α r and αs
 ϴ = window centered at pixel α, the size of window is the size of the kernel
 N = normalization factor for the kernel
If we were taking only Gaussian filter, then the calculated value at pixel α will be:
 ——————————eq(2)

We can again observe from the above equation that the weight given to I(α n) –
intensity at a neighbor pixel αn simply depends on its Euclidian distance from the
current pixel α in the sum. The closer a neighbor pixel is to reference (current) pixel,
the more weight it gets.

Let’s see what happens when we change the above equation to one as below:

————————————–eq(3)

Now the weight given to I(αn) – intensity at a neighbor pixel αn also depends on its
intensity difference when compared with the intensity at pixel α – I(α). The closer a
neighbor pixel is to reference (current) pixel, the more weight it gets. But
additionally, it also needs to be close in terms of intensity to reference pixel. Even if
a neighbor pixel is close but differs a lot when intensities are compared to reference
pixel, it will be given much less weight.

Now let’s see what happens when during convolution, we are at an edge pixel as
highlighted below:

Eq(2) gives the same weight to pixel value at the immediate left of edge pixel as the
weight given to immediate right. This weight distribution will blur the difference
made apparent by this edge pixel.

Eq(3) however, observes that there is a considerable difference in intensities for the
pixel at immediate left of edge pixel. As we know the behavior of Gaussian, the
larger the difference from reference point, the smaller the weight it generates. Here
the reference point is the intensity at edge pixel. Hence it gives more weight to pixel
at right and much lesser weight to pixel at left in comparison. Hence the edge is
preserved.

Analysis
Let us analyze each of the above three algorithms as image filtering operations via a
test case.

Problem Statement:

When we think of filtering in images, we always define image features that we want
to modify and those that we want to preserve. Let us observe the following original
image with no noise and surface plot of one of its patch.
Following are the observations:

1. The image consists of uniform whitish background embedded with blackish

bars. In fact for the above system, it is constrained that intensity of any pixel
within a bar and that of any background pixel exceeds a minimum
difference δ.
2. Another thing of note portrayed by the surface plot is that each black bar has
a vertical line where the intensity peaks within.
3. The width of the bars and peak pixels within them seem to be varying
discretely. This is the encoded information that the bars represent.
We can now define the features and strict constraints on them that we are
interested in:

1. edge pixels that form the boundary between bars and background can easily
be labeled by detecting pixels whose intensity difference with immediate
neighboring pixel exceeds δ.
2. peak pixels are ensured if we strictly detect that consecutive two adjacent
pixels on either side have a decreasing trend in intensity. Namely if p1, p2, p3,
p4, p5 are consecutive pixels horizontally, ensuring I(p2) < I(p3) & I(p1) <
I(p2) & I(p4) < I(p3) & I(p5) < I(p4) should strictly ensure p3 is a peak pixel.
Applying the above two constraints on the given image, the following feature image
is generated.

Illustration 2: features of original image

An edge is always normal to intensity variation. Thereby,

 Red pixels are edge pixels with horizontal intensity difference > δ
 Green pixels are edge pixel with vertical intensity difference > δ
 Blue pixels are peak pixels
Entrance of the Enemy:

Let us introduce some reality into our ideal world till now – noise. Let us model the
real world noise as normally distributed in accordance with the Central Limit
Theorem. We are using the python opencv function randn to generate this noise.
When added to the above original image, it has the following effect:

 The surface is not smooth anymore and has numerous distorted spikes
 Peaks seem to be the most affected by noise
Performance Measure:

We want to measure how closely we have retrieved the original image features after
applying a noise removal algorithm. If after applying the same strict constraints on a
filtered image, red, blue or green pixels are detected, we want to measure how many
of them have preserved their original position respectively. True positives and false
positives are indicators of that. Equally, we also want to measure how much of our
expectations are met. Below are the four performance metrics for peak pixels (blue):

Positive indicators:
 true positives: in percentage of all the blue pixels detected, how many were
at their original (correct) location
 expectations met: in percentage from all the blue pixels from original
image, how many were detected in the current image

Negative indicators:
 false positives: complementary of true positives, in percentage of all the
blue pixels detected, how many were at different (false) location
 expectations failed: complementary of expectations met, in percentage
from all the blue pixels from original image, how many were not detected in
the current image
Similar metrics apply for red and green pixels.

State of noisy image:

The result for edge pixels is perfect. Though noise has distorted the image, it is not
strong enough to alter edge pixel intensities so much that neighbor difference falls
below δ, as apparent from figure 2. But the peaks have taken the worst hit. Of all the
pixels that were detected, just 31.86% of them were true peaks. Moreover only
34.06% of our expectations were met. 65.94% of true peaks were never detected.
Let us address this serious concern by applying noise reduction algorithms one by
one and choose the best one.

Apply Normalized Box Filter:

The box filter algorithm only exposes one parameter that we can control – the kernel
dimension which can only be odd numbered. Keeping the kernel size as 3*3 has very
little effect on the image. We try with kernel size of 5, which most literature
recommends. We will find out the reason soon.

Peak pixels are now truly positive. 99.57% of all peaks detected are indeed true
peaks. But the reality is only 29.80% of expected true peaks are detected. More
drastic is the effect on edge pixels. Not a single red or green pixels are detected
which means an expectation failure of 100%.

Figure 4 clearly explains the reason. Though the surface has smoothed and noise has
been virtually eliminated, taking the average of neighbor pixels has resulted in peaks
becoming flatter and sharp edges demonstrating slanting behavior.

Increasing the kernel size increases the computational cost while further flattening
out peaks and diminishing edges.

Apply Gaussian Filter:

Overcoming the shortcoming of box filter, Gaussian filter distributes weight among
its neighbor pixels depending on parameter –c d, the standard deviation in space.
Keeping the kernel size same as 5*5 and varying c d, we achieve the best result with
standard deviation as 1.

Apply Bilateral Filter:

Besides kernel size and cd, bilateral filter exposes an additional parameter to control
performance c- I, the standard deviation in intensity domain. Keeping the same
kernel size as 5*5, we obtain the best result for cd as 2 and c i as 10.9.

Gaussian Filtering
Gaussian filtering is a linear smoothing technique based on applying a Gaussian function to
the image. This technique is widely used in image processing to blur an image and reduce
the effect of noise. In OpenCV, we can apply Gaussian filtering using
the cv2.GaussianBlur() function.

import cv2

image = cv2.imread('your_image.jpg') # Read image

blur = cv2.GaussianBlur(image, (5, 5), 0) # Apply Gaussian filter

In the code above, we read an image using the cv2.imread() function. Then we apply
Gaussian filtering with a kernel size of (5, 5), which determines the amount of smoothing.
The larger kernel size results in more blurring. The last argument is the standard deviation,
which controls the spread of the Gaussian distribution. A value of 0 indicates that OpenCV
should automatically calculate it based on the kernel size.

Median Filtering
Median filtering is a non-linear noise reduction technique that replaces each pixel's value
with the median value of its neighborhood. This technique is particularly useful for removing
salt-and-pepper type of noise. OpenCV provides the cv2.medianBlur() function for applying
median filtering.

import cv2

image = cv2.imread('your_image.jpg') # Read image

median = cv2.medianBlur(image, 5) # Apply median filter

In the code snippet above, we read an image using the cv2.imread() function and then apply
median filtering with a kernel size of 5. Similar to Gaussian filtering, higher kernel sizes
result in stronger noise reduction but may also blur the image's details.