Lecture 5: BLOB analysis

Robotic Perception 2017

Rikke Gade [email protected]
Agenda
BLOB extraction
Features
Feature matching

OpenCV example: Segmentation of apple and BLOB

extraction

2
Overview

Image Pre- Representa-

Camera Segmentation Classification
acquisition processing tion

Object #1: [x,y] Object #3

Object #2: [x,y]
is a dog
Object #10: [x,y]

Methods presented
today belong here

3
What is a BLOB?
BLOB = Binary Large Object
Group of connected white (object) pixels

.
.
.

4
Why BLOB analysis?

5
Why BLOB analysis?

6
What are BLOBs used for?
Isolate the different objects (BLOBs)
Represented as a list of pixels
Representing each object (BLOB) by a set of features
(characteristics)
Calculated from the list of pixels
For example the position and size of an object

Feature vector = [2,1,,3]

. .
. .
. .
Feature vector = [4,7,,0]

7
How are BLOBs extracted?
What we want:
For each object in the image, a list with its pixels
How do we get that?
Connected component analysis
Recursive
Sequential
(Region growing)
Define: Connectivity
Who are my neighbours?
4-connected
8-connected

8
Connected component analysis
Binary image [0,1]
Seed point: Where do we start?
Grassfire concept
Delete (burn) the pixels we visit
Visit all connected (4 or 8) neighbours

4
3 1
2

9
Connected component analysis

4 4 4
3 x 1 3x x 1 x 3x x 1
2 2 2

10
Connected component analysis

4
x3 x 1x x 4x x x x x
2 3 x 1 4x
2 3 x 1
2

11
Connected component analysis

4 4
x 4x x 3x x 1x 3 x 1x x
3 x 1 2x 2 x
2x x x

12
Connected component analysis

Implementation: Recursive or sequential

13
Mini exercise 1
A BLOB analysis is performed on the binary image in the figure below. How
many BLOBs are present in the image when 4-connectivity and 8-connectivity
is used, respectively?

14
Region growing
Recursive connected component analysis, but:
Operates on a greyscale (or color) image
Seed point: Where do we start?
Inside an object!
Only finds one object at a time

15
Region growing
Region growing = Thresholding + connected component
analysis
A pixel belongs to an object if its value is:
< <
and are set:
Beforehand
Relative to the value of the seed pixel(s)
For example: = value(seed) - 10 and = value(seed) + 10

Application example: Red-eye removal

16
Extracting features

? ? ?

17
Typical image processing
1. Isolate objects
2. Find BLOBs connected component analysis
3. Analyse each object
Find features:
Center
Size
Shape
Many others

18
Why calculate features
When more objects are present after segmentation, how
will you figure out which is the one of interest?

? ? ?

19
Why calculate features
When more objects are present after segmentation, how
will you figure out which is the one of interest?
Solution:
1. Make a model of the object you are looking for
2. Calculate its characteristics (features)
3. Calculate the characteristics (features) of each BLOB in the
image
4. Compare the features of each BLOB with the features of the
model
5. The best match defines the object

20
Feature extraction
Generate some entities (numbers) that can be used in
pattern recognition
For example size and shape
From image operations to mathematical operations
Input: a list of pixel positions
Output: Feature vector
First step: Remove too small, too big, and border BLOBs

21
Features
Examples of useful BLOB characteristics:
Area (number of pixels)
Also used for noise removal (small/big objects)
Number of holes in the object One BLOB
Holes area
Total area = area + holes area
Perimeter = length of contour
Bounding box
Upper left corner
Height and width of bounding box

22
Features
Bounding box

Bounding circle

Convex hull

23
Features

Center of mass (xm,ym):

1 1
xm
N
x
iobject
i ym
N
y
iobject
i

Center by median:

Center of bounding box:

24
Features
Compactness
Area
Width Height

Circularity (Heywood)
Perimeter
2 area

Longest distance (Ferets diameter)

Orientation of the object
Orientation of Ferets diameter

25
Features
Shapes
Bounding box ratio: Height/Width
Says something about the elongation
How well does the object fit a rectangle
Can be derived from: Area and Perimeter
How well does the object fit an ellipse
Can be derived from: Area and Perimeter

Many other features: look at the images of interest and be

creative!

26
Mini exercise 2
What is the bounding box height and width of the BLOB with
second biggest area (using 4-connectivity) in the figure below?

27
Classification
How to compare features?

Short introduction today more advanced classification

models next lecture

28
Classification
Remember the recipe:
1. Make a model of the object you are looking for
2. Calculate its characteristics (features)
3. Calculate the characteristics (features) of each BLOB in the
image
4. Compare the features of each BLOB with the features of the
model
5. The best match defines the object

Simplest solution: Measure the distance from each BLOB to

the model and pick the BLOB with the shortest distance, i.e.
the BLOB most similar to the model
How do we define a distance?
29
Feature matching
Distance in feature-space
Feature 1: Area
Feature 2: Circularity
2 dimensional feature space
Feature 2: Circularity
Model:
MATCH!
BLOBs:

Feature 1: Area
Feature matching
Find large circles

31
Feature matching
Distance from ith BLOB to the model
# features
D(i ) ( M f 1 Bi , f 1 ) 2 ( M f 2 Bi , f 2 ) 2 fj i, fj
( M
j 1
B ) 2

Euclidean distance Feature 2

Feature 1
How do we choose the feature values for the model?
We learn them!

32
Feature matching
What if we trust some features more than others?
For example, with small objects the perimeter might be uncertain

Solution: Weighting
# features
D(i ) ( M f 1 Bi , f 1 ) ( M f 2 Bi , f 2 )
2 2
fj i, fj
( M
j 1
B ) 2

# features
D(i ) W1 ( M f 1 Bi , f 1 ) W2 ( M f 2 Bi , f 2 )
2 2
j fj i, fj
W (
j 1
M B ) 2

33
Feature matching
Problem:
Area is measured in1000s and circularity from 0-1
That means that the area will dominate the distance measure
completely!
Solution
Use ratios: W/H is a better feature than W and H
Normalize all features to the same interval, e.g. [0,1]

34
What to remember
BLOB = Binary Large Object
Find a list of pixels for each object
Connectivity: 4- vs. 8-connected

Connected component analysis

Grassfire algorithm: Recursive and sequential
Region growing
Features (characteristics) of an object
Many exists
Many more can be derived
Feature matching
Make a model of each object to be classified
To be continued

35
OpenCV example
Code...

36
Exercises
Extract the four objects in the image
In a new image, draw the contours in different colours
Choose and extract some features which can be used for
classifying the objects
Make a simple classification system so that you can search for
and show one specific object at a time
Experiment with different features

37