DIP lab

Segmentation lab Material

Image Segmentation is essentially the process of partitioning a digital image into

multiple segments to simplify and/or change the representation of an image into
something that is more meaningful and easier to analyse.

The Segmentation process as a combination of Supervised and Unsupervised

algorithms.

Supervised segmentation: Some prior knowledge, possibly from human input, is

used to guide the algorithm.
Unsupervised segmentation: No prior knowledge is required. These algorithms
attempt to subdivide images into meaningful regions automatically. The user may
still be able to tweak certain settings to obtain desired outputs.

Thresoulding: It is the simplest way to segment objects from background by

choosing pixels above or below a certain threshold. This is generally helpful when
we intend to segment objects from their background.

Basic Imports
import numpy as np
import matplotlib.pyplot as pltimport skimage.data as data
import skimage.segmentation as seg
import skimage.filters as filters
import skimage.draw as draw
import skimage.color as color

Image

text = data.page()
image_show(text)

Supervised thresholding

Since we will be choosing the thresholding value ourselves, we call it supervised

thresholding.

text_segmented = text > (value concluded from histogram i.e 50,70,120 )

image_show(text_segmented);

Unsupervised thresholding
Scikit-image has a number of automatic thresholding methods, which require no
input in choosing an optimal threshold. Some of the methods are:

otsu, li, local.

text_threshold = filters.threshold_ # Hit tab with the cursor after the underscore to
get all the methods.

image_show(text < text_threshold);

In the case of local, we also need to specify the block_size . Offset helps to tune the
image for better results.
text_threshold = filters.threshold_local(text,block_size=51, offset=10)
image_show(text > text_threshold);

Before doing any segmentation on an image, it is a good idea to de-noise it using

some filters.

However, in our case, the image is not very noisy, so we will take it as it is. Next
step would be to convert the image to grayscale with rgb2gray.

scikit-image can be installed as follows:

pip install -U scikit-image(Linux and OSX)
pip install scikit-image(Windows)# For Conda-based distributions
conda install scikit-image

 Importing a GrayScale Image from the skimage library

The skimage data module contains some inbuilt example data sets which are
generally stored in jpeg or png format.
from skimage import data
import numpy as np
import matplotlib.pyplot as pltimage = data.binary_blobs()
plt.imshow(image, cmap='gray')

 Importing a Colored Image from the skimage library

from skimage import data
import numpy as np
import matplotlib.pyplot as pltimage = data.astronaut()plt.imshow(image)

 Importing an image from an external source

# The I/O module is used for importing the image
from skimage import data
import numpy as np
import matplotlib.pyplot as plt
from skimage import ioimage = io.imread('skimage_logo.png')plt.imshow(image);

 Loading multiple images

images = io.ImageCollection('../images/*.png:../images/*.jpg')print('Type:',
type(images))
images.files
Out[]: Type: <class ‘skimage.io.collection.ImageCollection’>

 Saving images
#Saving file as ‘logo.png’
io.imsave('logo.png', logo)

Active Contour segmentation also called snakes and is initialized using a user-
defined contour or line, around the area of interest, and this contour then slowly
contracts and is attracted or repelled from light and edges.

For our example image, let’s draw a circle around the person’s head to initialize the
snake.
def circle_points(resolution, center, radius): """
Generate points which define a circle on an image.Centre refers to the centre of
the circle
"""
radians = np.linspace(0, 2*np.pi, resolution) c = center[1] +
radius*np.cos(radians)#polar co-ordinates
r = center[0] + radius*np.sin(radians)

return np.array([c, r]).T# Exclude last point because a closed path should not
have duplicate points
points = circle_points(200, [80, 250], 80)[:-1]

The above calculations calculate x and y co-ordinates of the points on the periphery
of the circle. Since we have given the resolution to be 200, it will calculate 200
such points.

Random walker segmentation

In this method, a user interactively labels a small number of pixels which are known
as labels. Each unlabeled pixel is then imagined to release a random walker and
one can then determine the probability of a random walker starting at each
unlabeled pixel and reaching one of the prelabeled pixels. By assigning each pixel
to the label for which the greatest probability is calculated, high-quality image
segmentation may be obtained.

Unsupervised segmentation requires no prior knowledge. Consider an image that

is so large that it is not feasible to consider all pixels simultaneously. So in such
cases, Unsupervised segmentation can breakdown the image into several sub-
regions, so instead of millions of pixels, you have tens to hundreds of regions.

SLIC( Simple Linear Iterative Clustering)

SLIC algorithm actually uses a machine-learning algorithm called K-Means under
the hood. It takes in all the pixel values of the image and tries to separate them out
into the given number of sub-regions.

SLIC works in color so we will use the original image.

image_slic = seg.slic(image,n_segments=155)

All we’re doing is just setting each sub-image or sub-region that we have found, to
the average of that region which makes it look less like a patchwork of randomly
assigned colors and more like an image that has been decomposed into areas that
are kind of similar.
# label2rgb replaces each discrete label with the average interior color
image_show(color.label2rgb(image_slic, image, kind='avg'));

Felzenszwalb

This algorithm also uses a machine-learning algorithm called minimum-spanning

tree clustering under the hood. Felzenszwaib doesn’t tell us the exact number of
clusters that the image will be partitioned into. It’s going to run and generate as
many clusters as it thinks is appropriate for that
given scale or zoom factor on the image

image_felzenszwalb=seg.felzenszwalb(image)
image_show(image_felzenszwalb);

Region-based Segmentation

One simple way to segment different objects could be to use their pixel values. An
important point to note – the pixel values will be different for the objects and the
image’s background if there’s a sharp contrast between them.
In this case, we can set a threshold value. The pixel values falling below or above
that threshold can be classified accordingly (as an object or the background). This
technique is known as Threshold Segmentation.

If we want to divide the image into two regions (object and background), we define
a single threshold value. This is known as the global threshold.

If we have multiple objects along with the background, we must define multiple
thresholds. These thresholds are collectively known as the local threshold.

import the required libraries.

Fromskimage.color
import rgb2gray
import numpy as np
import cv2
import matplotlib.pyplot as plt
%matplotlib inline
from scipy import ndimage

Note:
We will take the mean of the pixel values and use that as a threshold. If the
pixel value is more than our threshold, we can say that it belongs to an object. If
the pixel value is less than the threshold, it will be treated as the background.

Edge Detection Segmentation

There is always an edge between two adjacent regions with different grayscale
values (pixel values). The edges can be considered as the discontinuous local
features of an image.

We can make use of this discontinuity to detect edges and hence define a boundary
of the object. This helps us in detecting the shapes of multiple objects present in a
given image.

the step-by-step process of how this works:

 Take the weight matrix

 Put it on top of the image
 Perform element-wise multiplication and get the output
  Move the weight matrix as per the stride chosen
 Convolve until all the pixels of the input are used

The values of the weight matrix define the output of the convolution.

Steps to be followed:

 extract features from the input.

 choosing some specific values for these weight matrices helps us to detect
horizontal or vertical edges (or even the combination of horizontal and
vertical edges).
 One such weight matrix is the sobel operator. It is typically used to detect
edges. The sobel operator has two weight matrices – one for detecting
horizontal edges and the other for detecting vertical edges.

Image Segmentation based on Clustering

Clustering is the task of dividing the population (data points) into a number of
groups, such that data points in the same groups are more similar to other data
points in that same group than those in other groups. These groups are known as
clusters.

One of the most commonly used clustering algorithms is k-means. Here, the k
represents the number of clusters (not to be confused with k-nearest neighbor).
Let’s understand how k-means works:

1. First, randomly select k initial clusters

2. Randomly assign each data point to any one of the k clusters
3. Calculate the centers of these clusters
4. Calculate the distance of all the points from the center of each cluster
5. Depending on this distance, the points are reassigned to the nearest cluster
6. Calculate the center of the newly formed clusters
7. Finally, repeat steps (4), (5) and (6) until either the center of the clusters does
not change or we reach the set number of iterations

The key advantage of using k-means algorithm is that it is simple and easy to
understand. We are assigning the points to the clusters which are closest to them.

Fromsklearn.cluster
importKMeans
kmeans = KMeans(n_clusters=5, random_state=0).fit(pic_n)
pic2show = kmeans.cluster_centers_[kmeans.labels_]
cluster_pic = pic2show.reshape(pic.shape[0], pic.shape[1], pic.shape[2])
plt.imshow(cluster_pic)

k-means works really well when we have a small dataset. It can segment the objects
in the image and give impressive results. But the algorithm hits a roadblock when
applied on a large dataset (more number of images).

It looks at all the samples at every iteration, so the time taken is too high. Hence,
it’s also too expensive to implement. And since k-means is a distance-based
algorithm, it is only applicable to convex datasets and is not suitable for
clustering non-convex clusters.

Mask R-CNN

Data scientists and researchers at Facebook AI Research (FAIR) pioneered a deep

learning architecture, called Mask R-CNN, that can create a pixel-wise mask for
each object in an image

Mask R-CNN is an extension of the popular Faster R-CNN object detection

architecture. Mask R-CNN adds a branch to the already existing Faster R-CNN
outputs. The Faster R-CNN method generates two things for each object in the
image:

 Its class
 The bounding box coordinates

Mask R-CNN adds a third branch to this which outputs the object mask as well.

Steps to be followed:

1. We take an image as input and pass it to the ConvNet, which returns the
feature map for that image
2. Region proposal network (RPN) is applied on these feature maps. This
returns the object proposals along with their objectness score
3. A RoI pooling layer is applied on these proposals to bring down all the
proposals to the same size
4. Finally, the proposals are passed to a fully connected layer to classify and
output the bounding boxes for objects. It also returns the mask for each
proposal

Mask R-CNN is the current state-of-the-art for image segmentation and runs at 5
fps.