CV pipeline Preprocessing Stage

Dr. Hussien Karam

Team Members:

1. Shaza Hossam
2. Ne`am Mohamed
3. Mariam Abd EL-Twab
4. Dina Mamdoh
Def preprocessing:

Image preprocessing are the steps taken to format images before they are used
by model training and inference.

A preliminary processing of data in order to prepare it for the primary processing

or for further analysis. The term can be applied to any first or preparatory
processing stage when there are several steps required to prepare data for the
user. For example, extracting data from a larger set, filtering it for various reasons
and combining sets of data could be preprocessing steps.

Why preprocessing?

• The acquired data are usually messy and come from different sources. To
feed them to the ML model (or neural network), they need to be
standardized and cleaned up. More often than not, preprocessing is used
to conduct steps that reduce the complexity and increase the accuracy of
the applied algorithm. We can’t write a unique algorithm for each of the
condition in which an image is taken, thus, when we acquire an image, we
tend to convert it into a form that allows a general algorithm to solve it.
• When it comes to creating a Machine Learning model, data preprocessing
is the first step marking the initiation of the process. Typically, real-world
data is incomplete, inconsistent, inaccurate (contains errors or outliers), and
often lacks specific attribute values/trends. This is where data
preprocessing enters the scenario – it helps to clean, format, and organize
the raw data, thereby making it ready-to-go for Machine Learning models.
• Before raw data could be sent through a machine learning model it has to
undergo preprocessing. And it’s simply because data in the real world are
generally Incomplete, Noisy and Inconsistent. And if this is fed into the
machine learning model, results can come unexpectedly!
 And that’s not really what we want. Data preprocessing is a proven method
for resolving such issues.

• Data preprocessing is a way of converting data from the raw form to a much
more usable or desired form, i.e., making data more meaningful by
rescaling, standardizing, binarizing, one hot encoding, and label encoding.

• Preprocessing is required to clean image data for model input. For example,
fully connected layers in convolutional neural networks required that all
images are the same sized arrays.

Goals of Image processing

• Assuring the image satisfies certain level of quality in order to pass it to a

Computer Vision function.

• Confirming the image coordinate system is correct (Resizing the Image).

• Reduce noise of the image.

• Contrast enhancement.

Preprocessing techniques

Convert color images to grayscale to reduce computation

complexity: in certain problems you’ll find it useful to lose unnecessary
information from your images to reduce space or computational complexity.
For example, converting your colored images to grayscale images. This is because
in many objects, color isn’t necessary to recognize and interpret an image.
Grayscale can be good enough for recognizing certain objects. Because color
images contain more information than black and white images, they can add
unnecessary complexity and take up more space in memory (Remember how color
images are represented in three channels, which means that converting it to
grayscale reduces the number of pixels that need to be processed).

Figure 1

In the example above, you can see how patterns in brightness and darkness of an
object (intensity) can be used to define the shape and characteristics of many
objects. In other applications, color is important to define certain objects. Like skin
cancer detection which relies heavily on the skin colors (red rashes).
The rule of thumb to identify the importance of colors in your problem is to look at
the image with the human eye, if you are able to identify the object that you are
looking for in a gray image then you probably have enough information to feed to
your model. If not, then you definitely need more information (colors) in your
images. The same rule can be applied for most other preprocessing techniques
that will be discussed next.

Standardize images: One important constraint that exists in some machine

learning algorithms, such as CNN, is the need to resize the images in your dataset
to a unified dimension. This implies that our images must be preprocessed and
scaled to have identical widths and heights before fed to the learning algorithm.

Data augmentation: Another common pre-processing technique involves

augmenting the existing dataset with perturbed versions of the existing images like
scaling and rotations.
This is done to enlarge your dataset and expose the neural network to a wide
variety of variations of your images. This makes it more likely that your model
recognizes objects when they appear in any form and shape. Here’s an example
of image augmentation applied to a butterfly image.

Resize: Changing the size of an image sounds trivial, but there are
considerations to take into account.

Many model architectures call for square input images, but few devices capture
perfectly square images. Altering an image to be a square call for either stretching
its dimensions to fit to be a square or keeping its aspect ratio constant and filling
in newly created “dead space” with new pixels. Moreover, input images may be
various sizes, and some may be smaller than the desired input size.

Random Flips: Randomly mirroring an image about its x- or y-axis forces our
model to recognize that an object need not always be read from left to right or up
to down. Flipping may be illogical for order-dependent contexts, like interpreting
text.

Best tips: for most real-world objects, flipping is a strong way to improve
performance.

Random Rotations: Rotating an image is particularly important when a model

may be used in non-fixed position, like a mobile app. Rotating can be tricky as it,
too, generates “dead pixels” on the edges of our images and, for bounding
boxes, requires trigonometry to update any bounding boxes.

Best tips: if an object may be a variety of different orientations relative to the

captured images, rotation is a good option. This would not be true for, say,
screenshots, where the image content is always in a fixed position.

Random Noise

Adding noise to images can take a variety of forms. A common technique is “salt
and pepper noise,” wherein image pixels are randomly converted to be completely
black or completely white. While deliberately adding noise to an image may reduce
training performance, this can be the goal if a model is overfitting on the wrong
elements.

Best tips: if a model is severely overfitting on image artifacts, salt and pepper noise
can effectively reduce this.
Contrast: Contrast is the difference in luminance or color that makes an object
distinguishable from other objects within the same field of view.

Clearly, the left image has a low contrast because it is difficult to identify the details
present in the image as compared to the right image.

Low contrast images can result from Poor illumination, lack of dynamic range in
the imaging sensor or even wrong setting of lens aperture during image acquisition
etc.

When performing Contrast enhancement, you must first decide whether you want
to do global or local contrast enhancement. Global means increasing the contrast
of the whole image, While in local we divide the image into small regions and
perform contrast enhancement on these regions independently. Don’t Worry, we
will discuss these in detail in the next blogs.
References:

1. CV_eBook_HK, Hussien Karam Hussien, 2020.

2. https://medium.com/@boelsmaxence/introduction-to-image-processing-filters-
179607f9824a

3. https://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm

4. Image preprocessing in the spatial domain, local neighborhood operations, Czech

Technical University in Prague, Czech Institute of Informatics, Robotics and Cybernetics,
Václav Hlavác.

5. https://towardsdatascience.com/image-pre-processing-c1aec0be3edf

6. https://freecontent.manning.com/the-computer-vision-pipeline-part-3-image-
preprocessing/

7. https://www.yourdictionary.com/preprocessing