Photogrammetry & Robotics Lab

Binary Images
and Commonly Used Operations
(connected components; distance transform; morphological operators)

Cyrill Stachniss

The slides have been created by Cyrill Stachniss.

1
Binary Image
§  So far, we considered grayscale
images where each pixel typically
takes values between 0 and 255
§  A binary image is
an image with a 1 bit
color depth
§  Each pixel is either
black or white
§  Either 0 or 1 (0 or 255)

2
Binary Image Examples

Handwritten digits

Scanned documents Image Courtesy: Leibe

Background Subtraction

- =

Image Courtesy: Jäggli

3
Connected Components

4
Connected Components
§  For several applications, we need to
identify which components are
connected
§  Example: Separation of characters

5
Connected Components
Two points A, B are connected if there
exists a path from A to B that goes only
through the same component

Image courtesy: Förstner 6

Neighborhoods on Grids
§  We need to define a neighborhood
relationship for pixels
§  Two popular neighborhood definitions

N4 neighborhood N8 neighborhood

7
N4 Neighborhood
§  The left, right, up, down pixels are
connected

§  Also called “city-block” neighborhood

or Manhattan neighborhood

8
N8 Neighborhood
§  The left, right, up, down and the
diagonal pixels
are connected

9
How to Determine Connected
Components?

Image courtesy: Förstner 10

Determining Connected
Components via a Graph
§  Build graphs of the foreground pixels
with edges according to N8 neighbors
§  Idea: Label the nodes in the graphs

Image courtesy: Förstner 11

Labeling Approach Informally
1.  Select an unlabeled node and
assign a new label to it
2.  For each unlabeled neighbor of a labeled
node, assign the same label
3.  Repeat step 2 until all neighbors are
labeled
4.  Repeat step 1 until all nodes are labeled

Also called: a “brushfire” approach

12
Connected Components

13
Example

2
1
1 1 1 1
1

14
Steps (outer loop, 1st iteration)
outer
loop 1
inner
loop 1

inner
loop 2

inner
loop 3

1
1 1 1 1
1
15
Steps (outer loop, 2nd iteration)
outer
loop 2

inner
loop 1

2
1
1 1 1 1
1
16
Properties of the Algorithm
§  Provides the connected components
§  Work in general graphs
§  Does not exploit the systematic
neighborhood of images

Is there any property we can exploit

to make the algorithm more efficient?

17
Labeling by Exploiting the
Grid Structure of the Image
Idea
§  Process the image in one pass
§  Generate a temporary label for a
foreground pixel based on the already
processed neighbors
§  In case of multiple labels for the same
component, use an equivalence table

18
 Connected Components on Grids
§  Process the image from left to NOTE: Example uses
right, top to bottom: max to select the label
If the next pixel to process is 1
A) If none of its (top or left) neighbor
N4 is 1, assign new label.

B) If all neighboring (top or left) labels

are identical, copy the label

C) If the labels differ, copy the one

label (e.g., min or max) and update
the equivalence table.

Slide courtesy: J. Neira 19

 Connected Components on Grids
§  Process the image from left to NOTE: Example uses
right, top to bottom: max to select the label
If the next pixel to process is 1
A) If none of its (top or left) neighbor
N4 is 1, assign new label.

B) If all neighboring (top or left) labels

are identical, copy the label

C) If the labels differ, copy the one

label (e.g., min or max) and update
the equivalence table.

§  Re-label with the smallest of equivalent

labels
§  Extends trivially for N8 neighborhoods

Slide courtesy: J. Neira 20

 Connected Components for
Grids/Binary Images

21
Example (N8 Neighborhood)

Image courtesy: Förstner 22

Example (N8 Neighborhood)

Equivalence table: {1=3, 2=4=5}

Image courtesy: Förstner 23
Connected Components for
Grids/Binary Images
§  Exploits the grid neighborhood
§  Requires only one pass through the
image for labeling
§  Second pass to eliminate duplicate
labels
§  Linear complexity in the number of
foreground pixels

24
Distance Transform

25
Distance Transformation
Several problems require to compute
the distance from any pixel to the
border of the components

Examples
§  Nearest neighbor problems
§  Sensor models for range sensor
§  Map visualization
§  User interfaces
26
Distance Transformation
§  Distance transform on the foreground
pixels is defined as

border

§  with different distance functions

27
Examples
N4 N8

1 1 1 1 1 1
1 2 2 2 1 1 1 2 1 1
1 2 3 2 1 1 2 2 2 1
1 2 2 2 1 1 2 2 2 1
1 2 1 1 1 1 1 1 1 1

1 1 1 1

1 1

28
Distance Transformation
§  Distance transform can be computed
similar to the connected components
§  Two passes over the image
§  1st: top-down, left-right
§  2nd: down-up, right-left
§  Always store the minimum distance

29
First and Second Pass (N4)
First pass

1st pass dist. 2nd pass dist. 30

First and Second Pass (N4)
First pass Second pass

1 1 1
1 2 2 2 1
1 2 3 3 2
1 2 3 4 3
1 2 3 4 4
1 2
1

1st pass dist. 2nd pass dist. 31

First and Second Pass (N4)
First pass Second pass

1 1 1 1 1 1

1 2 2 2 1 1 2 2 2 1

1 2 3 3 2 1 2 3 2 1

1 2 3 4 3 1 2 2 2 1

1 2 3 4 4 1 2 1 1 1

1 2 1 1

1 1

1st pass dist. 2nd pass dist. 32

Analogous for N8 Neighborhood
First pass Second pass

1 1 1 1 1 1

1 1 2 1 1 1 1 2 1 1

1 2 2 2 1 1 2 2 2 1

1 2 3 2 1 1 2 2 2 1

1 2 3 2 1 1 1 1 1 1

1 2 1 1

1 1

1st pass dist. 2nd pass dist. 33

Examples for N4 and N8
Distances to the Center
N4 N8

Image courtesy: Förstner 34

N4 Distance Transformation
1.  Initialization

2.  1st pass (top-left to bottom-right)

3.  2nd pass (bottom-right to top-left)

35
N8 Distance Transformation
1.  Initialization

2.  1st pass (top-left to bottom-right)

3.  2nd pass (bottom-right to top-left)

36
N4 vs. N8 Neighborhood 1 1 1
1 2 2 2 1
§  The N4 neighborhood 1 2 3 2 1
overestimates the 1 2 2 2 1
Euclidean distance 1 2 1 1 1

§  The N8 neighborhood 1 1
N4
1
underestimates the
1 1 1
Euclidean distance 1 1 2 1 1
1 2 2 2 1
1 2 2 2 1
1 1 1 1 1
1 1
N8
1
37
N4 vs. N8 Neighborhood 1 1 1
1 2 2 2 1
§  The N4 neighborhood 1 2 3 2 1
overestimates the 1 2 2 2 1
Euclidean distance 1 2 1 1 1

§  The N8 neighborhood 1 1
N4
1
underestimates the
1 1 1
Euclidean distance 1 1 2 1 1
§  Can we efficiently 1 2 2 2 1

combine both? 1 2 2 2 1
1 1 1 1 1
1 1
N8
1
38
Combined Distance
§  The real cost of the diagonal is
§  If we approximate
§  The sum provides a better
approximation for twice the distance
§  Thus we can use the average distance

§  By using we can exploit

integer computations
39
Euclidian Distance Transform
§  Computing the real Euclidian distance
for every cell to the closest border is
more difficult
§  EDT in Python (scipy) available as
ndimage.morphology.distance_transform_edt
§  EDT in Matlab available as bwdist()
§  See, for example,: Breu et al., 1995

40
Morphological Operators:
Erosion and Dilation

41
Filtering
§  Binary images are often obtained
through thresholding (point operator)
§  Operator:

Image Courtesy: Whitaker 42

What We Have…

How to
Fix This?

Image Courtesy: Whitaker 43

What We Want…

Image Courtesy: Whitaker 44

Morphological Operators
§  Shrinking the foreground (“erosion”)
§  ︎Expanding the foreground (“dilation”)
§  ︎Removing holes in the foreground
(“closing”)
§  ︎Removing stray foreground pixels in
background (“opening”)
§  ︎…

45
Erosion
Change each foreground (here black)
pixel to a background (here white) pixel
if it has a background pixel as its N4
neighbor

46
Erosion Example

47
Erosion Example

48
Erosion Example

49
Dilation
Change each background (here while)
pixel to a foreground (here black) pixel
if it has a foreground pixel as its N4
neighbor

50
Dilation Example

51
Dilation Example

52
Dilation Example

53
Erosion and Dilation
§  Erosion shrinks the foreground
§  Erosion removes foreground outliers
§  Dilation expands the foreground
§  Dilation fills holes

Can we combine both to improve

segmentation and masking?

54
Opening and Closing
Opening
§  Removing stray foreground pixels in
background
§  Step 1: erosion; Step 2: dilation

Closing
§  Remove holes in the foreground
§  Step 1: dilation; Step 2: erosion

55
Original Image

Image Courtesy: Whitaker 56

Opening

Image Courtesy: Whitaker 57

Closing (After Opening)

Image Courtesy: Whitaker 58

Opening + Closing Result

Image Courtesy: Whitaker 59

Morphological Operators
§  Morphological Operators can be used
to eliminate noisy masks
§  Combination of Opening and Closing
removes stray pixels and fills holes
§  Erosion and dilation are local operators

60
Summary
§  Binary images are relevant tools in
several image processing applications
§  Connected components
§  Distance transforms
§  Erosion, dilation, opening, and closing

61
Literature
§  Szeliski, Computer Vision: Algorithms
and Applications, Chapter 3.3
§  Förstner, Scriptum Photogrammetrie I,
Chapter 6

62
Slide Information
§  The slides have been created by Cyrill Stachniss as part of the
photogrammetry and robotics courses.
§  I tried to acknowledge all people from whom I used
images or videos. In case I made a mistake or missed
someone, please let me know.
§  The photogrammetry material heavily relies on the very well
written lecture notes by Wolfgang Förstner and the
Photogrammetric Computer Vision book by Förstner & Wrobel.
§  Parts of the robotics material stems from the great
Probabilistic Robotics book by Thrun, Burgard and Fox.
§  If you are a university lecturer, feel free to use the course
material. If you adapt the course material, please make sure
that you keep the acknowledgements to others and please
acknowledge me as well. To satisfy my own curiosity, please
send me email notice if you use my slides.

Cyrill Stachniss, [email protected] 63