Computer Vision Clarify& Codes

֎ Computer vision
• Science that deals with how computers can be made to gain high-level
understanding from digital images or videos to simulate human vision system
֎ Computer vision applications
• Image enhancement, Image Classification
• Object Detection
• Medical imaging
• Human Pose Estimation
• Object Tracking
• Self-Driving Car
֎ Computer vision system
• Is a system that includes methods for acquiring, processing, analyzing, and
understanding digital images and in general deal with the extraction of high
dimensional data from the real world
֎ Image processing-based system/ computer vison systems structure and
stages

֎ Basic Elements of the system

• Acquisition
• Enhancement
• Restoration
• Segmentation
• Description
• Feature extraction
• Detection
• Recognition
• Encoding/Decoding

Omnia Khaled
֎ Image types
• RGB image: Represent as 3D matrix (height x width x 3 channels)
• Gray image: Represent as 2D matrix (height x width) Each pixel represents
intensity information in range 0 (for black) up to 255 (for white)
• Binary Image: has only 2 values for black and white (e.g. 0 and 255)
֎ Image acquisition
• Convert a scene to a digital image such as digital camera and scanner
Image Digital image
Acquisition
(3D worlds scene)

֎ Sampling
• Divide the image into samples (N in x) and (M in y) as result will have M*N
picture elements which is called pixels
֎ Quantization
• Process of converting analog image to digital image grouping a set of
grays/colors into gray value or a specific color as result will have matrix at
dimension M*N
• Number of possible colors/grays are 2q
֎ False contouring/ coarse sampling
• Insufficient number of intensity levels in smooth areas of digital image

֎ Image resolution
• Refers to the number of pixels in an image resolution is identified by the width
and height of the image and number of colors in an image
• Resolution= M (samples x) * N(samples y) * Q(number of color or gray)
• Example: image resolution= 1024*1024*256
֎ Digital image
• Is a digital form of a scene or given analog image taken by acquisition
֎ Image restoration
• Is the operation of taking a noisy image and estimating the clean original image
• This operation is done before enhancement
• Noise or corruption may come in many forms such as motion blur and camera
mis-focus
Omnia Khaled
֎ Neighborhoods
• To identify objects in a digital pattern, we need to locate groups of black pixels
that are "connected" to each other. In other words, the objects in each digital
pattern are the connected components of that pattern.
• Relation between pixels in some way.

֎ Connectivity
• Pixels are considered connected if and only if:
• They’re neighbors “according to some neighboring”
• f (X1, Y1), f (X2, Y2)∈ ξ → Where ξ is the gray levels connectivity set {.....}.
֎ Image path
• The Image path is a sequence of connected pixels starts at the first pixel
f(x1,y1) and ends at f (xn, yn).
• This Path sometimes doesn’t exist as there might be a disconnection in the Gray
scale with respect to the connectivity set, could be unique, and could be
multiple paths
֎ Image path length
• The number of steps when you move from pixel to another on path
• Image path length is (n-1)
֎ Region / connected pixels
• The group of connected pixels within the image which is represent object
֎ Connected regions
• Set of pixels that are connected such that exist number of f(xi , yi) that are
connected to any pixel within the region and not include in the region
֎ Image matrix
• Defines the number of rows and columns that make up the pixels in the matrix
֎ Labeling of connected regions
• Labeling is a mean to find out the regions that exists within an image matrix
• Condition = neighboring + connectivity set

Omnia Khaled
֎ Histogram
• Histogram: count the number of occurrences per gray/color component value
within the image matrix/matrices.
• So, histogram is a table of two columns: gray/color, counts.
• The total number of counts should be equal to M*N pixel counts
• Importance/ applications of histogram
▪ Analysis of the image, Image brightness
▪ Using histogram to equalize an image
֎ Brightness
• The amount of energy output by a source of light relative to the source we are
comparing it to
• Brightness refers to the overall lightness or darkness of the image
֎ Contrast
• The difference between maximum and minimum pixel intensity in an image

֎ Normalized histogram
• It gives the probability of each number in the dataset, or you can say that it
basically gives the count of frequency of each element
֎ Equalized histogram
• Histogram is used for enhancing the contrast of the image
▪ In range and Full range
֎ Distance Metrics

֎ Image enhancement
• The aim of image enhancement to better appearance and noise removal and
facilitates processing steps
• Image enhancement techniques can be divided into two categories
▪ Spatial domain methods: which operate directly on pixels
▪ Frequency domain methods: which operate on the Fourier transform of an
image

Omnia Khaled
֎ Image enhancement classification/application
• Pixel based/ point based
▪ Only deals with the point or pixel
• Region based
▪ Deals with the neighbors of the pixel in the region
• Global entire image
▪ Deals with the whole image
֎ Image negative
• Reproduces the bright parts of image as dark and the dark part as light areas
• f′ (x, y) negative Image = (L − 1) – f (x, y) original image
• L (gray level) = 2q = 28 = 256
• (L-1) = 255
֎ Contrast stretching / histogram stretching
• There are two methods of enhancing contrast
▪ The first is called histogram stretching that increase contrast
▪ The second one is called histogram equalization
• Stretching means improve contrast or increase contrast
֎ Contrast contraction
• Algorithm that can solve the problem of objects are seen as multi-agents due to
difference in gray value
֎ Gray level scaling
• Is a way to high light gray range of interest to a viewer of further processing

֎ Bit plane scaling

• An image could be seen as a set of planes where a plane is the set of pixels bit
with the same weight -i.e.: for an image with 256 grays will mean each pixel
gray assigned 8-bit gray value, this means that such an image could have 8
planes a plane per each bit
֎ Image averaging
• If we take more than one shot of the same scene, any white noise will not be
repeated on the same pixels, from that comes the idea of averaging and as the
number of copies increase the more the removal of white noise from the image
1
that is: f′(x, y)= ∑𝑁𝑖=1 𝐹𝑖 (𝑥, 𝑌)
𝑁

Omnia Khaled
֎ Image subtraction and motion detection
• Sometimes the interest of an application will be on motion of elements or the
changes in the seen the subtraction of a seen from the previous shots will
highlight the changes
• A number of change or still scene will out come near zero or zero image

֎ Spark noise
• A pixel or very small group of pixels with gray value on the opposite position
with its region

֎ Sharp edge
• Edge that can separate object from another

֎ Blur to edge/ edges blurring

• Reducing the edge content and makes the transition from one color to the other
very smooth

Omnia Khaled
֎ Linear filters
• Advantages
▪ Reduce the spark noise
• Disadvantages
▪ Blurring of edges
• Types (low pass filters)
▪ Average filter/traditional low pass/ simple traditional filter

▪ Circular filter

▪ Pyramid filter

▪ Cone filter

▪ Gaussian filter

Omnia Khaled
֎ Non-linear filters
• Advantages
▪ Fix the problem of luring of edges and keep it sharp
• Disadvantages
▪ Need more processing time
• Types (low pass filters)
▪ Median filter
✓ The median filter can perform the smoothing without blurring the
edges
✓ Steps
1) Sort the (gray Values-(pixel)) udder the filter
2) Select the middle value after sorting
3) Get the mid value then replace it with current value
▪ Kuwahara filter
1) Divide the region under the filter into 4 regions
𝑋1+𝑋2+⋯+𝑋𝑛
2) find the mean for R1, R2, R3, R4 mean=
𝑛
3) find the standard deviation for R1, R2, R3, R4
4) We select the mean of region that has smallest standard deviation
֎ Border treatment methods in filter application
• Passing the filters center over the border pixels will filter grid values out of the
image matrix.
• This problem could be treated by any of the following methods.
▪ Ignore
✓ That is to copy the borders pixels as to the enhanced matrix
▪ Repeat
✓ Consider as if the images continue the last row /column without
changes
▪ Reflect
✓ Mirror The row/column across boarder

Omnia Khaled
֎ Derivative/Differential (High pass filters)
• Derivative filters could be used in enhancement to improve edges and corners to
have better vision as well as more accurate segmentations
• Derivative filters used mainly in: segmentation, feature extraction, and in
Convolutional neural networks
• Types (high pass filters)
▪ Laplace/ traditional/ regular/ normal filter

▪ Prewitt filter

▪ Sobel filter

֎ Image edges Sharpening

• Idea: compute intensity differences in local image regions.
• Useful for emphasizing transitions in intensity (e.g., in edge detection).
֎ Image transformation/ Image transformation relation to enhancement
• It will be useful to transfer the image to another domain for better processing or
applying specific operator
• The transformation domain of interest is the frequency domain
• The transformations mapping the spatial domain f(x,y) original image to the
frequency domain f(u,v) enhanced image

Omnia Khaled
֎ Frequency domain
• We first transform the image to its frequency distribution then the system
performs processing and the output in this case not an image but a
transformation, after performing inverse transformation, it is converted into an
image which is then viewed in spatial domain

֎ Fourier transform
• Type of transformation under frequency domain, it used for enhancement and
analysis and better processing

֎ Discrete cosines transform (DCT)

• Used to decompose an image into a finite sequence of data points in terms of
sum of cosine functions
• Transform of the discrete cosine is from Real Domain ➔ Real Domain

֎ Enhancement in frequency domain

• Enhancement the original image to get the enhanced image in frequency domain
• Types (Ideal filters, Butter filters)
֎ Ideal filters
• Filters that zeroed out frequency above the given frequency and use radial mask
to set which frequency are in or out
• Types: Ideal pass filter (Low pass filter, high pass filter)
✓ Low pass filter used for image smoothing and high used for image
sharping

Omnia Khaled
֎ Butter worth filter
• Types (Low worth filter , high worth filter)
▪ Low pass filter used for image smoothing and high used for image
sharping

֎ Image segmentation

• Image segmentation process: Separates the objects from the background and, it
separates between objects.
• Segmentation Methods/ techniques
▪ Threshold Based: we choose a specific value and make it the thresholding
and what is before it is considered object
✓ Ex: histogram analysis, Otsu’s method
▪ Edge Based: use derivative filter, makes a high pass filter to detect the
edges to get the object off the image
✓ Ex of derivative filters: Prewitt filter, Sobel filter, Laplace filter
▪ Region Based: use labeling to line the outlines of the object but this will
be margining not splitting

Omnia Khaled
֎ Threshold segmentation techniques
• Histogram analysis using symmetric property
▪ Dark symmetrical (dark object, white background)
✓ T= (M-start)+M

▪ White symmetrical (white object, dark background)

✓ T= M-(start+M)

▪ Both symmetrical (get the dark symmetrical, white symmetrical)

• Histogram analysis using normal distribution

Omnia Khaled
 • Otsu’s method
▪ Otsu's method searches for the threshold that minimizes the intra-class
variance
▪ Which minimizes (the variance within the class),
▪ Defined as a weighted sum of variances of the two classes: -

֎ Edge Descriptors
• Edge direction: perpendicular to the direction of maximum intensity change
(i.e., edge normal)
• Edge strength: related to the local image contrast along the normal.
• Edge position: the image position at which the edge is located.
֎ Types of edges
• Step edge: the image intensity abruptly changes from one value on one side of
the discontinuity to a different value on the opposite side.
• Ramp edge: a step edge where the intensity change is not instantaneous but
occur over a finite distance.
• Ridge edge
• Roof edge

Omnia Khaled
֎ Image Morphology
• Method to enhance the binary matrix or the result of image segmentation
• It fixes noise in an image using a small shape or template called a structuring
element, the structuring element is positioned at all possible location in the
image, and it is compared with the corresponding neighborhood of pixels
• There are 5 types of morphology
▪ Dilation morphology
✓ D(AB)= 𝐴 ⊕ 𝐵= ⋃(𝐴 + 𝐵)
✓ The basic effect of the operator on a binary image is to enlarge the
boundaries of regions of foreground pixels
▪ Erosion morphology
✓ E (A, B) =𝐴 ⊖ (−𝐵) = ⋂(𝐴 − 𝐵)
✓ The basic effect at the operator on a binary image is to erode away
the boundaries of regions of foreground pixels
▪ Opening morphology
✓ O(A, B)= 𝐴0 𝐵 = 𝐷(𝐸(𝐴, 𝐵), 𝐵)
✓ An erosion followed by a dilation using the same structuring
element for both operations
▪ Closing morphology
✓ C(A, B)= 𝐴0 𝐵 = 𝐷(𝐸(𝐴, −𝐵), −𝐵)
✓ A dilation followed by an erosion using the same structuring
element for both operations
▪ Boundary morphology
✓ boundary =𝐴 − (𝐴 ⊖ 𝐵)
✓ perform erosion then subtract the result from original object then
the boundary will remain

Omnia Khaled
֎ Object description/ object contour finding/ object contour
representation techniques/ object boundary detection/ shape detection
• Once an object detected then it will be needed to describe that object by
describing its contour
• Contour representation techniques
▪ chain codes
✓ Uses 8 numbers to represent the relative position of the next border
pixel to the current pixels.
✓ The process starts at a point and move clockwise till it comes to the
start point.
✓ So, it moves from one a pixel point to another and register the
relation.

▪ crack codes
✓ Uses 4 numbers to represent the relative position of the next border
pixel to the current pixels.
✓ The process starts at a point and move clockwise till it comes to the
start point.
✓ So, it moves from one a pixel point to another and register the
relation.

▪ Run length codes

✓ Uses different concepts of representation which is describing the
pixels occupied by the object per row / column
✓ Row ( (x,y), number of count in row repeated respectively)
✓ Column ( (x,y), number of count in column repeated respectively)

Omnia Khaled
֎ Feature Extraction
• The process of defining a set of features or image characteristics, which will
most efficiently or meaningfully represent the information that is important for
analysis and classification
Object Features
Algorithm

• Some of features
✓ Hue moments (7 features don’t depend on scale)
✓ DCT (Discrete Cosine Transform)
✓ PCA (Principle Component Analysis)
1) Get the mean
2) Subtract mean (center at zero)
3) Vector from(2) put in matrix to get covariance matrix
4) Compute eigenvalues
5) Select eigen vector in descending order from eigen values
✓ Legendre moments
• Good features
✓ Within the same class small variance
✓ Out of class large variance
✓ With scaling small variance
✓ With rotation small variance
✓ With pose small variance
✓ With lighting change small variance
• Redundant features
▪ More than one feature measures the same thing
• Feature re-engineering
▪ Ensure that the features are good features (feature selection)
▪ To make a feature extraction we should make feature engineering

Omnia Khaled
֎ Classification Problem

• Build up classifier require training dataset

• Classifier types
▪ Nearest Neighbor Based Classifiers
▪ Bayesian/Probability based Classifiers
▪ Decision Tree Classifiers
▪ Clustering algorithm-based classifiers
▪ Fully connected Neural Classifiers
▪ Convolution network-based classifiers
֎ Nearest neighbor (cluster mean)
• Compute mean of each set m1, m2, .... . mq
• Types
▪ 1-nearset neighbor – first neighbor (save all dataset)
▪ N-nearest neighbor
✓ Ex: 5-nearest neighbor (find the nearest 5 (
֎ Face detection Vs Recognition
• Detection (detect / find the face)
• Recognition (recognize who is? )

֎ The Mahalanobis distance (MD)

• Is the distance between two points in multivariate space. ... The Mahalanobis
distance measures distance relative to the centroid which is a base or central
point which can be thought of as an overall mean for multivariate data.

Omnia Khaled
֎ Load and convert image to gray
imgGray = cv2.imread('tree.jpg',0)
arr = np.array(imgGray)
plt.imshow(arr, cmap='gray', vmin=0, vmax=255)

֎ Get mean, min, max, variance

arr = np.array(imgGray)
mean = np.mean(arr)
mx = np.max(arr)
mn = np.min(arr)
std = np.std(arr)
print("the mean = " + str(mean))
print("the max. = " + str(mx))
print("the min. = " + str(mn))
print("the std. = "+ str(std))

֎ Histogram
plt.hist(image.ravel(),255,[0,255])
plt.show()

֎ Equalized Histogram
equalized = cv2.equalizeHist(image)
plt.show(equalized)

֎ Image negative
neg_img=[[255-pixel for pixel in row]for row in image]
plt.imshow(neg_img);

֎ Gray level scaling

r1=180
r2=255
gray_scale_img=[[255 if pixel<r2 and pixel>r1 else pixel for pixel in row]for row in image]
plt.imshow(gray_scale_img);

Omnia Khaled
֎ Contraction and stretching

֎ Outline a sub-program based on a threshold to map the image matrix

into binary matrix (object, Background).
image=cv2.imread("image.jpg",0)
arr=np.array(image)
T=35
for i in range(arr.Shape[0]):
for j in range(arr.Shape[1]):
if (arr[i][j]>=T):
arr[i][j]=1
else:
arr[i][j]=0

Omnia Khaled
֎ Method to apply filters

Omnia Khaled
֎ Classifiers

֎ Labeling

Omnia Khaled