Technische Universität München

Image Processing
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

0.1 Information
 Lecture ( Wednesday, 08:00 – 09:30):
 Participation necessary
 Basis: Printed slides
 Completion and notes necessary
 Questions welcome
Prof. Stilla
 Exercises:
 Partially during lecture
 Some additional exercise courses for programming

 Exams:
 Answering questions L. Hoegner
 Calculations, working on images
 Question block – true / false statements

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_00] 2

0.2 Literature (1)
 Literature (German):  Wahl F (1984)
 Azad P, Gockel T, Dillmann R Digitale Bildverarbeitung.
(2007) Computer Vision – Das Berlin: Springer
Praxisbuch. Aachen: Elektor  Voss K, Süße H (1991)
 Bässmann H, Besslich PW Praktische Bildverarbeitung.
(1989) Konturorientierte München: Hanser
Verfahren der digitalen
Bildverarbeitung.  Liedtke CE, Ender M (1989)
Berlin: Springer Wissensbasierte
 Burger W, Burge M J (2005) Bildverarbeitung.
Digitale Bildverarbeitung, Berlin: Springer
eXamen.press: Springer  Bunke H (1985)
 Haberäcker P (1987) Modellbasierte Bildanalyse.
Digitale Bildverarbeitung: Stuttgart: Teubner
Grundlagen und  Jähne B (1993)
Anwendungen. Digitale Bildverarbeitung.
München: Hanser Berlin: Springer
 Kopp H (1997) Bildverarbeitung
interaktiv: Teubner
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_00] 3

0.2 Literature (2)

 Literature English:
 Castleman KR (1995)
Digital Image Processing.
New Jersey, Englewood Cliff: Prentice Hall
 Gonzalez RC, Woods RE (2002)
Digital image processing.
Prentice Hall
 Haralick R, Shapiro L (1992)
Computer and Robot Vision.
Vol. 1, New York: Addison-Wesley
 Jain AK (1988)
Fundamentals of digital image processing.
Prentice Hall
 Rosenfeld A, Kak A ()
Digital picture processing.
Academic Press

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_00] 4

0.3 Software
 Viewer: IrfanView  free
 http://www.irfanview.com/
 MATLAB Mathworks & Image Processing Toolbox  kommerziell
 Scilab & SIVP Scilab Image and Video Processing Toolbox  frei
 http://www.scilab.org/
 http://sivp.sourceforge.net/
 HALCON MVTec  commercial
 GNU Octave  free
 http://www.octave.com/

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_00] 5

0.4 Image Processing - Contents

Image processing

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_00] 6

Technische Universität München

Image Processing
1 Introduction
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

Image Processing - Contents

Image Processing

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 2

1 Introduction
 Introduction
 Motivation
 Applications
 Application: Tomography
 Application: Character recognition
 Application: Interpretation of Earth surface
 Imaging
 Separation of image processing
 Image analysis
 Image- and scene description
 From knowledge to model
 Hierarchical object modeling
 Processing layers

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 3

1.1 Motivation
 Digital Image processing
 operates upon pictorial information of a digit form
 uses digital circuit, computer processors and software to carry out an
operation
 Aim
 better or more illustrative representation
 information extraction (as automatic as possible)
 Information extraction requires
 knowledge about imaging process (real world → image) and
 interpretation process of humans (image → objects)
therefore, image processing involves methods of
 signal processing
 pattern recognition and object recognition
 Shining example
 Human / eye, visual perception
 2 images → 3 D description of the visible scene in object space
 Dream of a “visual equipped" robot (very old idea)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 4

 1.2 Applications
 Archeology: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Astronomy: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Biology: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Remote sensing: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Radio and television: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Geodesy: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Criminology: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Medicine: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Metallurgy: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Metrology: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Physics: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Photogrammetry: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Production: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Quality control: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Robotics: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
 Military: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 5

1.2.1 Application: Tomography, NMR - 3D-Plot

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 7

1.2.1 Application: Tomography, NMR

 spin echo image (34ms)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 8

1.2.1 Application: Tomography, NMR

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 9

1.2.2 Application: character recognition, metal

Milled and punched characters in metal workpieces

illuminated in different ways

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 10

1.2.2 Application: character recognition, metal

Number five (5) illuminated in different ways

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 11

1.2.2 Application: character recognition - 3D Plot

(Inverted
brightness)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 12

1.2.2 Application: character recognition, gray value profile

255 255

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 13

1.2.3 Image interpretation (aerial image)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 14

1.2.3 Image interpreation (map TK 1:25 000)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 15

1.3 Imaging: Image brightness
 Geometric and radiometric (physical radiation) projection:
Scene (3D) → Image (2D)
 geometry: position of the 3D point in the image e.g. central
perspective
 radiometry: quantitative measure of the brightness
 optics are included
 Information content: electromagnetic radiation (EMR) for a certain
wavelength
 reflected radiation (e.g. sunlight, radar pulse)
 particular radiation (e.g. thermal radiation)
 Recording:
Energy of the EMR or another additional proportional measurement
 Device (sensor):
 photographic Film (is simultaneously storing the images)
 electrical sensors e.g. CCD sensor (additional image storage is
required)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 16

1.3 Imaging: general images-radar image (SAR)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 17

1.3 Imaging: general images
 Beside brightness, an image can represent other values which are
distributed in space, e.g.
 elevations related to a reference plane
 distances to a sensor
 concentrations of chemical substances
 distributions of temperature
 distributions of pressure
 Etc.
 In principle any mathematical function having the following form
z = z(x,y)
can be seen as an image.
 Interpretation of those images requires in general information about:
 way of projection
 type of sensor
 transmitted information
 information transmitter

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 18

1.3 Imaging: general images–Digital Elevation Model

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 19

1.4 Separation: Digital image processing

in Image Description
out
generative
Image
Image Computer Image analysis (also):
Processing
Graphics  Image understanding
 Machine vision
 Image interpretation
Descript.
Image Everything  Computer Vision

Analysis else
General aim of image
analysis:
Generation of an
image or scene
description

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 20

1.5.1 Image and scene description

 Steps of generation  Items or topics to describe
 Objects of the image or scene
objecs  Features of the objects
 Relations between objects
Image interpretation
 Interpretation of objects
primitives
Feature extraction

Segmented image
Preprocessing

sampled image
Mapping

scene

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 21

1.5.2 From knowledge to model

Knowledge Model
 Object modeling
 attributive description
 model–based image analysis specification of the object
 model-driven image analysis through its global
properties
 ...
(e.g. building: attributes
list such as height, surface
 Description of image formation area, building volume, ...)
through:
 Illumination modeling  structural description
 Sensor modeling specification of the general
 Scene or object modeling object layout through its
components (e.g. building
= walls + roofs+…)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 22

1.5.3 Hierarchical object modeling

 Hierarchical object modeling
Models that include an hierarchical ordering of the knowledge
 Scalar hierarchy
 Ordering of the knowledge according to a desired level of details
 The levels could be nested through statements such as „is_composed_of“
or „is_part_from“
 Example:
A settlement is_composed_of buildings and roads. A building
is_composed_of walls and a roof. A wall is_composed_of masonry, doors
and windows.
Or: A building is_part_from a settlement. A road is_part_from…

 Specification hierarchy
 Ordering of the knowledge according to a certain level of specification
 The specification levels could be linked through statements such as
„is_a_kind_of“
 Example:
A family house is_a_kind_of residential houses. A residential house
is_a_kind_of buildings. A building is_a_kind_of constructions.
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 23
1.5.4 Processing layers

processing processed
operations
layer data

symbolic
high level objects
recognition

feature
medium level features
extraction

image
low level pixel
manipulation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_01] 24

Technische Universität München

Image Processing
2 Digital image characterization
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

Image Processing - Topics

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 2

 2 Digital image characterization
 Image model
 Sampling
 Pixels
 Geometries and neighborhoods
 Coordinate systems
 Quantization
 Examples
 Storage of digital images
 Memory space
 Coding
 Statistical image characterization
 Mean, variance and standard deviation
 Histogram
 Entropy
 Co-occurrence matrix
 Covariance
 Correlation coefficient

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 3

2.1.1.1 Raster - different geometries

geometries neighborhoods connections

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 4

 2.1.1.2 Sampling – pixels
column Pixel

row

Grey
black value

gray

Gray value (Intensity value )

in the position (r,c) white

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 6

2.1.1.2 Sampling - coordinate systems

Image matrix B Pixel b: picture element
c x

B = b(r,c) B = b(x,y)

y
r
y
x

B = b(x,y) B = b(x,y)

x
y

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 7

 2.1.1.2 Sampling - coordinate systems - Exercise
From a given image B=(b(x,y)) with nx=768
columns and ny=512 rows a subimage S=(s(u,v))
with nu=384 columns and nv=256 rows is
extracted. The upper left image point of B is
blu=b(1,1) and the y-axis points down. The upper
left image point of S is slu = s(1,1) and is identical
with b(37,25).
a) Which x,y -coordinates correspond to the
lower right image point srl of S ?

b) The same image content of B is available in a

different image processing system given by
B’=(b’(x’,y’)). However, in this system the y-axis
points up and the lower left image point of B’ is
b’(0,0). You want to cut out a subimage
S’=(s’(u’,v’)) showing the same content like S.
Which x’,y’-coordinates correspond to the
lower left image point s’ll=s’(0,0) and the upper
right image point s’ru=s’(383,255) ?

for a) sru: b ( , )
for b) s‘lu: b‘ ( , ); s‘ro: b‘ ( , )

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 9

2.1.2 Quantization (1)

 The continuous gray- or multi-  The 8-bit converter is the mostly
channel spectral function is used one:
quantized by an A/D converter
into a number of possible gray
values.
black

gray value
gray

b(r,c)  G = {g0, g1, g2, ..., gk-1} white

black white
 After sampling and quantization,
k=2L the digital image is generated.
L=number of the converter bits  In order to save memory, the data
is usually compressed.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 11

2.1.2 Quantization (2)
binary image
G={0,1} b scalar for storage and display of the digital image 1 bit is
required for each pixel

b scalar 2-value image

G={0,255}
e.g. g0=0, g1=255

gray value image (monochrome)

for storing often
G={0,1, … 255} b scalar 8 bit (=1 byte) are used

multi spectral image

b vector for storing often
G={0,1, … 255} 8xn bits are used
(n elements)

(true-) color image

b vector
G={0,1, … 255} 3 Bands: RGB
(3 elements)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 12

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 13

2.1.3 Example: Different samplings
Resolution:
1x1, 2x2, 4x4
8x8, 16x16, 32x32
64x64, 128x128, 256x256

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 14

2.1.3 Example – image sequences (in space and time)

z t

y y
Bi+2(x,y,zi+2) Bi+2(x,y,ti+2)

Bi+1(x,y,zi+1) Bi+1(x,y,ti+1)

Bi(x,y,zi) Bi(x,y,ti)
x x

Spatially extended objects Dynamic 2D-process

A spatially extended object is split into If a 2-dimensional object shows
a sequence of sub-images Bi (x, y, zi). changes over time, a sequence Bi (x,
This sequence could be used to get a y, ti) could be created to describe the
snapshot of the 3D object. temporal changing of the object.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 15

 2.2 Storage of digital images
 Memory space  Data compression
 Image matrix nx=512; ny=512  Lossy/lossless compression
nxy=262144  Binary images
 Binary image 1 bit = KB  Elimination of gray levels
 Gray value image 8 bit = KB  Storage of the differences
 RGB- image 3x8 bit = KB between the gray values
 Storage of multi-band images
 Band-Interleaved-by-Line (BIL)  Coding
or Row-Interleaved  Run-length encoding
 Quad-trees structures
 Band-Interleaved-by-Pixel or
 Chain codes
Pixel-Interleaved (BIP)

 File formats
 Band-Sequential or Band-
Interleaved (BSQ)

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 16

2.2.1 Storage of digital images - memory space

 Physical nomenclature
 1 Megabyte (MByte) = 1000 Kilobyte (kByte)
 1 Kilobyte (kByte) = 1000 Byte
 1 Megabyte (MByte) = 1.000.000 Byte
 Computer science
 1 Megabyte (MB) = 1024 Kilobyte (KB)
 1 Kilobyte (KB) = 1024 Byte
 1 Megabyte (MB) = 1.048.576 Byte
 Example
 Grey value image 8 bit per pixel
 Image size 23x23 cm²
 Pixel size 15x15 μm²
 23 cm / 15 μm = 15300 pixel / row
 15300 pixel² = 234.090.000 pixel
 234.090.000 Byte = 234 MByte = 223 MB

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 18

 2.2.1 Storage of digital images - memory space
 Assumption:1 bit is used per pixel and band (channel)

image type pixel size approx.

image size given in

Grey value image

monochrome

color image

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 19

2.2.2.1 Run-length encoding:RLE (1)

 Example of binary row: 00001111111 000011111000
 Coding 1:
 Coding 2:
 Background value g=0
 Origin and length of g=1
 Efficiency of the RLE?
 Error-free reconstruction?
 Extension to 2D binary images: row neighborhood graph (4th neighbor)
0 1 2 3 4 5 6 7
0:
1:
2:
3:
4:
5:
6:
7:

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 20

 2.2.2.1 Run-length encoding:RLE (2)
 Extension to gray-value images
 Description: (Origin, gray value, Length)
 meaningful: ?

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 22

2.2.2.2 Quad-Trees (1)

 Partition into quadrants  Types of nodes
0 1
 Homogeneity analysis 2 3 homogeneous black areas
 Tree structure homogeneous white areas
 nodes inhomogeneous areas
 edges step
 Example: Binary image, nx=ny =2k, k=3 root 3

2
Q0(k-1) Q1 (k-1)

1
Q2 (k-1) Q3 (k-1)

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 24

 2.2.2.2 Quad-Trees (2)
 Node indexing step

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 26

2.2.2.2 Quad-Trees (3)

 Compact storage  Compression versus analysis
 Without node indexing and edges flexibility
 Definition, for example:
 0: black
 1: white
 2: hybrid nodes
For example level 2: 2 0 2 1
 Recursive storage of sub-trees

2021
1220
1000
1010
2211
1010
0001

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 28

 2.2.2.3 Chain codes
Description of objects through the boundary  Example:
 Binary image: Object (g=0) on a
background (g=1) x
 Definition of neighborhood codes
0 1 2 3 4 5 6 7
0
(e.g. 8th)
code vector y 1
x αi (aix, aiy) 2
3 2 1 0 1, 0 3
4 * 0 1 1,-1 4
5 6 7 2 0,-1 5
y 3 -1,-1 6
4 -1, 0
7
5 -1, 1
6 0, 1
7 1, 1  Origin:
 Boundary description through  Chain code:
 Origin
 Chain direction (code direction)

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 30

2.2.2.3 Chain code- in practice

 Integration of additional information through special codes (04++)
 End of the chain (e.g. 0400)
 Generalization for gray value images through storage of the gray
values g (e.g. 0422 g)
 Identification of areas or one-pixel-thick lines (contour lines)
 Design of the chain through masks comparison
 Line-by-line digital scanning
 Raster directed chains (directions 0,5,6,7)
-> general directed chains

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 32

 2.2.2.3 Chain codes
 Lenght of direction chain  Further processing steps

ng: number of even k 
nu: number of uneven k 
 Translation  Convex elementary musters
(splitting any area into convex elementary
musters)
 Roation of k x 45°, k=1,2,3 directed codes have special structure
-> monotone directed codes
 Object dimensions
chain code ai -> vector (aix,aiy)
p.e. ai=5 (aix,aiy) = (-1, 1)
chain code: a0 a1 … an-1

i
x i   a jx  x 0
i
yi   a jy  y0
j1 j1

Width and height: Chain code:

Coding:
(8 parameters)

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 33

2.2.3 Data formats

 RAW
 Raw data with 8, 10, 12 or 14 bpp (bit per pixel), lossless, camera specific
 TIF or TIFF (Tagged Image File Format)
 Several images in one file possible (Multipage-TIFF)
 Many color represenations and compressions possible, common 8,16,24 bpp,
uncompressed or run length coding, lossless
 „tags“ for additional information like GeoTIFF: Tag 33922 (ModelTiepointTag)
tie point on rasterized coordinated with pixelvalue (i,j,k) and model coordinates
(x,y,z)
 Not applicable for web due to complexity and interal pointers
 Often used for archiving
 GIF (Graphics Interchange Format)
 max 8 bpp, colors with LUT (Look Up Table)
 Color index in LUT can be marked as transparent
 lossless LZW compression (Lempel-Ziv-Welch)
 Introduced by CompuServe, algorithm under patent
 Often used for websites

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 35

2.2.3 Data formats
 PNG
 max 48 bpp, prefiltering (difference to left and /or upper neighboring pixel)
 lossless LZ77 compression (Lempl-Ziv) plus Huffmann coding
 Uses an „alpha chanal" (transparency mask)
 PNG is licence free in contrast to GIF
 JPG or JPEG
 Color space transform RGB -> YCbCr,
 The YCbCr model seperates color information in luminance Y and two color
components Cb (/Blue-Yellow Chrominance/) and Cr (/Red-Green
Chrominance/).
 Structured in 8×8 blocks with discrete cosine transform of these blocks,
theoretically lossless or with defined specific loss
 BMP
 Standard format for Microsoft Windows
 No compression
 Often as 24bpp 3 channel true color (3x8bpp) or as 8 bpp color with look up
table

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 36

2.3 Statistical image characterization

 Mean grey value
 Variance and standard deviation
 Relative histogram
 Cumulative histogram
 Histogram of several channels
 Entropy
 Co-occurrence matrix
 Covariance
 Correlation coefficient
 Spatial frequency

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 37

 2.3.1 Mean, variance and standard deviation
 Mean value μ
The mean is the average gray values of an image B or a certain
image region

 Variance δB2
The variance is the gray values‘ mean-squared deviation from the mean

 Standard deviation δB

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 38

2.3.2 Mean square deviation - variance - calculation

nr nc

  b  r, c    
1 2
 2B 
n rc r 1 c 1
nr nc

  b  r, c   2b(r, c)   
1
 2 2

n rc r 1 c 1

1  n r nc 2 n r nc n r nc
2
   b  r, c   2 b(r, c)    
n rc  r 1 c 1 r 1 c 1 r 1 c 1 
1  n r nc 2 
   b  r, c   2n rc   n rc 
2

n rc  r 1 c 1 
1  n r nc 2 
   b  r, c    n rc 
2

n rc  r 1 c 1 Note: μ and δB2 can be calculated


in one step and thus less
nr nc
1
 b  r, c   
calculations are necessary.
 2B  2 2

n rc r 1 c 1

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 40

2.3.3 Histogram
 The histogram PB(g) is a distribution function, which provides for
each gray value the number of pixels having this same gray value.
g max


g i  g min
PB  g i   n rc

 Often the histogram is scaled by the number of pixels and called:

Histogram of the relative frequencies p(g).
g max

pB  g  
1
PB  g   pB  gi   1
n rc gi  g min

0  pB  gi   1
 The histogram of an image gives reliable information about
brightness and contrast, although the entire spatial information get
lost.
 Graphical representations of histograms= bar charts

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 43

2.3.3.1 Computation of mean and variance from histogram

A simplified calculation is given by using the histogram

 mean
g max
1

n rc

gi  g min
g i  PB  g i 

g max
 
gi  g min
gi  pB  gi 

 variance
g max
1
 g     PB  g i 
2
 
2
B i
n rc gi  g min
g max

 g     pB  gi 
2
 2B  i
gi  g min

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 44

 2.3.4 Cumulative histogram

 Cumulative histogram S(g)

g
SB  g   
gi  g min
PB  g i 

SB  g max   n rc
 Cumulative realtive histogram s(g)

1
sB  g   SB  g 
n rc
s B  g max   1

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 46

2.3.4 Example histogram and cumulative histogram

1

0.9

0.8

0.7

0.6

0.5

0.4

0.3
0.05

0.2
0.045

0.1
0.04

0
0.035 50 100 150 200 250

0.03

0.025

0.02

0.015

0.01

0.005

0
50 100 150 200 250

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 47

 2.3.5 Entropy
Entropy: average information content of the image
 Computation using the histogram PB(g)

g max
1
H 
gi  g min
p B  g i   log 2
pB  gi 
g max
H 
gi  g min
p B  gi   log 2 p B  gi 

 Example
 homogeneous image B=(b(x,y))=g
 Dual image (pB (g1)=0,5; pB (g2)=0,5)
 PB (g) uniformly distributed

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 49

2.3.6 Co-occurrence matrix

 The co-occurrence matrix is useful for the description of certain relations
in the image

φ: relation between pairs of image point positions (r1,c1) and (r2,c2)

Example:
 right neighbor
 neighbor at Δr, Δc

ag1,g2 : frequency of gray value combinations (g1,g2)=(b(r1,c1), b(r2,c2))

WB, (g1 , g 2 )   a g1,g 2 

 The co-occurrence matrix for the relation φ is

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 51

 2.3.6 Co-occurrence matrix - example
 Example
 Relation φ=φ1 (right neighbor)
 Grey values G= {0, 1, 2, 3}
 image nr=6 , nc=6

WB,φ(g1,g2) =
0 0 1 1 2 3
0 0 0 1 2 3
0 0 1 2 3 3
 Low contrast
0 1 1 2 3 3 
1 2 2 3 3 3  High contrast

2 2 3 3 3 3

 Meaning of the elements in the main diagonal :

 Meaning of the side elements g1,g2

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 52

2.3.7 Spatial frequency (1)

 Analogue to temporal periodic changes I(t), images can be described as

spatial periodic changes I(x) by frequencies. To distinguish temporal
and spatial changes, you then use the term spatial frequencies.
 A spatial frequency is defined as the inverse of the length of a spatial
period. The spatial frequency gives the number of periods within a
spatial interval und thus has the dimension 1/length (e.g. line pairs per
millimeter).
x in [mm] or [pixel]
Sinus grid low
spatial frequency
(long periods)
Period length (line pair bright/dark)

Sinus grid high

spatial frequency
(short periods)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 54

 2.3.7 Spatial frequency (2)

 Draw the grey value changes of a line and determine the spatial frequency
in x direction!

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 56

2.4.1 Covariance

 The covariance of two images B1 and B2 is written as

1  nr nc 
 B1B2     b1  r, c   1   b 2  r, c    2  
n rc  r 1 c 1 
n r nc
1
 B1B2   b1  r, c   b2  r, c   1 2
n rc r 1 c 1

If B1=B2  covariance = variance

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 58

2.4.2 Correlation coefficient

 The correlation coefficient ρ of two images B1 und B2 is described by

nr nc

B1B2   b  r, c       b  r, c    
1 1 2 2
B1B2   r 1 c 1

2B1  2B2  nr nc 2 r

n nc
2
   b1  r, c   1      b 2  r, c    2  
 r 1 c 1   r 1 c 1 

δB1B2 and ρB1B2 decribe the similarity of two images

 ρB1B2 = 0 B1 and B2 are uncorrelated
 ρB1B2 = 1 B1 and B2 are correlated to 100% positive
b2=b1+ ;  > 0
 ρB1B2 = -1 B1 and B2 are correlated to 100% negative
b2= b1+;  < 0
 The correlation coefficient ρ is invariant to linear tranformations!
B1B2 (b1  , b 2  )  sgn(,  ) B1B2 (b1 , b 2 )

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 59

2.4.2 Correlation coefficient - examples

not defined

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 60

 2.4.2 Correlation coefficient – example character recognition

 Example of use of the correlation coefficient for character recognition

 Correlation coefficient = measure of similarity between a reference or
„template“ and a sliding window in the test image
 Template B2 is moved all over the test image so that the
corresponding window B1 is found
 The correlation coefficient is computed between the template B2 and
the window B1
 The correlation coefficients are then stored in the correlation image
(correlation image: black = -1 ; white = +1)
 Identification of the maxima (high positive correlation) and minima
(high negative correlation) of the correlation image

B2 B1

Template

Test image Correlation image

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 61

2.4.2 Correlation coefficient – example character recognition

search image correlation image

arrow: high positive correlation

search image correlation image

arrow: high negative correlation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 62

2.4.2 Correlation coefficient – masked correlation

 Special case: the reference (or template) has only 2 gray values. The
computation of the corresponding correlation coefficient could get easier.
 Not all the pixel pairs b1(r,c) (from the sliding window B1 in the test image),
b2(r,c) (from the template B2) are used in the computation of the
correlation. Only certain areas that remain after applying a mask to the
template or after zeroing some values from the template image, are
investigated. This is called the masked correlation.
 Example of a masked template B2
 Black area: gray value g1
 Gray area : gray value g2
 white area: the corresponding pixels are
not used in the computation of the correlation.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 63

2.4.2 Correlation coefficient – masked correlation - example

 Masked correlation

reference

Test image Correlattion

(white: 0 black: 1)

Test image

Correlation image
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 64
2.4.2 Correlation coefficient – example character recognition

 Mask-based correlation
 Query with one template in order to find the position in the test image
where the template and the character in the test image fit at best

Test image reference

Correlation image (white: 0, black: 1)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 65

2.4.2 Correlation coefficient – example character recognition

Mask-based correlation
 Query with different templates for the template that matches the best
with the test image

Test image references

Correlation image (white: 0, black: 1)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ip2_14_02] 66

Technische Universität München

Image Processing
3 Image transformation
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

Image Processing - Contents

Image Processing

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 2

3 Image transformation
 Geometrical transformation
 A geometrical image transformation converts an input digital image
into an output digital image where the geometrical position of the
pixels have changed.
 Spatial transformation
 Resampling

 Modification of the gray value distribution

 Point processing
 Local image operators
 Global image operators

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 3

3.1 Geometrical transformation

 Geometrical transformation are often used for
 Co-registration between two images
 Image geocoding
 Image rectification

 The geometrical transformation is a 2-step process

 Spatial transformation
 Resampling
Different methods:
 Nearest neighbor
 g gets the gray value of its closest pixel
 Bilinear interpolation
 g is a linear weighted average of its 4 neighbors gray values
 Higher order interpolation (e.g. cubic spline)
 g is a weighted average of the 16 closest neighbors gray
values
Quality versus computational effort
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 4
 3.1.1.1 Spatial image transformation (Examples)
 Translation

 Mirroring (according to the mirror line (x=c))

 Rotation (rotation angle α)

 Zooming

 Affine transformation

 Projective transformation

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 5

3.1.1.2 Direct spatial transformation

 Direct method

Input image Output image

 Procedure
Each pixel in the input image is projected in the output one, where the gray
values are interpolated.

 Disadvantage
When few pixels are available in the input image, the corresponding pixels in
the output image will be assigned with “NO VALUE” (holes in the output image).

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 7

 3.1.1.3 Indirect spatial transformation
 Indirect method

Input image Output image

 Procedure
Each pixel is replaced in its position in the input image and then
assigned with the interpolated gray values from the output image
 Advantage
No uncovered pixels
 Often used in the digital photogrammety

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 8

3.1.2 Resampling - Bilinear interpolation

 Computation of the gray values from the 4 closest neighboring pixels
g(0,0), g(0,1), g(1,0) und g(1,1)

g(0,1)
g(1,1)

(0,1)
g(0,0)

y‘‘ (1,1)

(0,0) g(1,0)
x‘‘

(1,0)
x

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 9

 3.1.2 Resampling: bilinear interpolation - Exercise
An image B is subjected to a geometrical
transformation.
A point (x‘1,y‘1) in the transformed image
B‘ has the following coordinates in the
input image B
(x1,y1)= ( 36.75 , 24.25 ).

The gray values of its 4 neighbors are:

b ( 36, 24 ) = 64
b ( 36, 25 ) = 192
b ( 37, 24 ) = 192
b ( 37, 25 ) = 64

Draw a 3D sketch and determine the new

gray value b‘ ( x‘1, y‘1 ) by a bilinear
interpolation!

b‘ ( x‘1, y‘1 ) =

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 11

3.2 Modification of the gray value distribution

 The semantics of an image are stored in its gray values, as well as the
local and global arrangements of its pixels. When an image transformation
is based on the modification of its gray value distribution, different
procedures are possible.
 Point processing
A pixel is assigned with a new value according to a certain rule that
involves only its original value and possibly its position (in the input
image)
 Local processing
A pixel, located in a specific area (e.g. central pixel) will be assigned
with a new gray value, according to certain rule that involves the gray
values of the surrounding pixels
 Global processing
A pixel is assigned with a new gray value according to a certain rule
that involves the entire image

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 13

 3.2.1 Point processing
 Point processing
 Homogeneous point processing
The point operation is independent of the position of the pixel. It
involves only the original gray value of the pixel.
 Inhomogeneous point processing
This point operation is dependent of the position and gray value of the
pixel.

 Examples of homogeneous point processing

 Characteristic curves
 Histogram scaling and linearization
 Color transformation

The different transformations based on the above described point

processing will be illustrated and discussed in terms of histograms.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 14

3.2.1.1 Characteristic curves: inverting

b' ( x, y)  [b( x, y)  255]

b‘
255

0 b
0 255

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 15

 3.2.1.2 Characteristic curves: linear scaling

b' ( x, y)  [b( x, y)  k2 )] k1 k2  0

k2  0
k1  1
b‘ k1  1
255

0 b
0 255

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 17

3.2.1.2 Characteristic curves: linear scaling

 Areas with steep curves (slope>1) are characterized by a wide range of gray
values (higher contrast).
 Areas with smooth curves (slope<1) are characterized by few gray values
(lower contrast).
 The general layout of the histogram does not change after a linear scaling
 Mean and variance of b‘ are:

(Proof by inserting the equations of μ und δ2 in the linear equation)

 The images could thus be normalized for a given mean and standard
deviation.

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 19

 3.2.1.3 Characteristic curves: piecewise linear scaling

 0 if [b( x, y)  k2 ]  k1  0 255
 k1 
b '( x, y)  [b( x, y)  k2 ]  k1 bmax  bmin
 [b( x, y)  k2 ]  k1  255
 255 if
k2  bmin
b‘
255

0 b
0 bmin bmax 255

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 21

3.2.1.4 Characteristic curves: general scaling


[ f b( x, y)  f min ]  0
255
 0 falls
 f max  f min
 255
b' ( x, y)   [ f b( x, y)  f min ]
 f max  f min

[ f b( x, y)  f min ]  255
255
 255 falls
 f max  f min

f(b): general function

f max  f bmax 
f min  f bmin 

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 23

 3.2.1.5 Characteristic curves: cut off

 0 if b  bmin suppress non-relevant


b '( x, y)  b( x, y) if bmin  b  bmax information (e.g. background),
or emphasize certain
 0 b  bmax
 if Information
b‘
255

0 b
0 bmin bmax 255

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 24

3.2.1.6 Non-linear characteristic curve transformation

 Contrast enhancement through non-linear characteristic curve

transformations, for e.g.
 Elimination of the non-linearities caused by the sensors
 Compensation of image display non-linearities

 Through, e.g.
 Exponential transformation
 Logarithmic transformation
 Sigmoid transformation
 Tangent transformation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 26

 3.2.1.6 Non-linear characteristic curves: Exponential

b' ( x, y)  [eb( x, y ) k1 1] stretching a small

bright gray-value
k1  45,985 interval

b‘
255

0 b
0 255

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 27

3.2.1.6 Non-linear characteristic curves: Logarithmic

b' ( x, y)  k2  log[b( x, y)  1] stretching a small

dark gray-value
k2  106,09 interval

b‘
255

0 b
0 255

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 29

 3.2.1.6 Non-linear characteristic curves: Sigmoid
 
bmax  1   b(x, y) 1  
b'(x, y)  1  sin      0    1
2       bmax 2  
 sin    
  2  
b‘
255  Stretch the mid-luminance range:
increase contrast at the central part of
the histogram (slope>1) and
 Flatten the very dark and light regions of
the image: decrease contrast at the
histogram borders (slope <1)

0 b
0 255
α = 0.9

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 31

3.2.1.6 Non-linear characteristic curves: Tangent

 
bmax  1   b(x, y) 1  
b'(x, y)  1  tan      0    1
2      bmax
 2  
 tan    
  2 
b‘
255  Flatten the mid-luminance areas:
decrease contrast at the central part of
the histogram (slope<1) and
 Stretch the very dark and very light
areas: increase contrast at the
histogram borders (slope >1)

0 b
0 255

α = 0.9

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 33

 3.2.1.7 Characteristic curves: static threshold

255 if b( x, y)  bs
 Segmentation by a static
threshold
b '( x, y)    Gray value image: 2-level image
 0 else  (e.g. images of scanned
drawings or text)
b‘
255

0 b
0 bs 255 bs  128

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 35

3.2.1.8 Characteristic curves: two static thresholds

 0 if b( x, y)  bmin  Segmentation by 2
 static thresholds
b '( x, y)  255 if bmin  b( x, y)  bmax  Gray value image:
 0 if b( x, y)  bmax 2-level image

b‘
255

0 b
0 bmin bmax 255 bmin  128 bmax  160

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 37

 3.2.1.9 Characteristic curves: iso densities
 Piecewise constant
scaling function
b' ( x, y)  gk für bk  b( x, y)  bk 1  Gray value image:
k  0,1, 2, ...n 1 multi-level image

b‘
255

0 b
0 b2 b3 255
b1 n4

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 39

3.2.1.10 Modification of grey value distribution

 Histogram scaling  Linearization function using
relative cumulative frequencies
pB(g) pB‘(g)
g
sB  g   
gi gmin
pB  gi  0  sB  g   1

g g
input histogram output histogram
transformation with fn
Histogram linearization or equalization
b` r,c   f n (b(r,c))


pB(g)
pB‘(g)
1
so that sB'  g   g
gmax
Advantage: simple implementation,
g since the scaling function can be
g directly derived from the gray-value
input histogram output histogram distribution

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 41

3.2.1.10 Example of histogram scaling
1
0.1
0.9
0.09

0.8
0.08

0.7
0.07

0.6
0.06

0.5
0.05

0.4
0.04

0.03 0.3

0.02 0.2

0.01 0.1

0 0
50 100 150 200 250 50 100 150 200 250

0.05
0.9

0.045
0.8

0.04
0.7
0.035
0.6
0.03
0.5
0.025

0.4
0.02

0.3
0.015

0.01 0.2

0.005 0.1

0 0
50 100 150 200 250 50 100 150 200 250

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 45

3.2.1.10 Example of histogram linearization

1

0.05
0.9

0.045
0.8

0.04
0.7
0.035
0.6
0.03
0.5
0.025

0.4
0.02

0.3
0.015

0.01 0.2

0.005 0.1

0 0
50 100 150 200 250 50 100 150 200 250

0.05
0.9

0.045
0.8

0.04
0.7
0.035
0.6
0.03
0.5
0.025

0.4
0.02

0.3
0.015

0.01 0.2

0.005 0.1

0 0
50 100 150 200 250 50 100 150 200 250

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 46

3.2.1.11 Color transformation
 From „Look-Up-Table“
(LUT) to rgb (red-green-
blue)

LUT r g b

0 r0 g0 b0
1 r1 g1 b1
. . . .
5 000 101 189 .
. . . .
255 r255 g255 b255

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 47

3.2.2 Local image operators

 Digital filter
 Convolution
 Mean operator
 Gradient operator
 Laplace operator
 LOG and DOG
 Effects of linear convolution operators
 Computation effort for convolution
 Morphological operators

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 48

3.2.2.1 Digitale Filter
Input image Output image Characterization of filters
Discrete
 effect
system
b(r,c) b‘(r,c)  image point
 local
Applications  global
 reduction of noise  operation
 elimination of distortions  linear
 balancing of brightness  non-linear
 restoration of burred edges  processing
 emphasizing or suppressing  parallel
details  sequential
 detection of edges and  recursive
objects
 analysis
 segmentation of ROIs
 Space invariant (homogeneous)
(ROI: Region Of Interest)
 Space variant (inhomogeneous)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 49

3.2.2.2 Convolution
 Input image B with b(r,c)
 Filter matrix h(k,l) with operators K,L: (uneven)
(Filter mask, Filter kernel, Impule response)
 K  1 K  1  L  1 L  1  Often the sum of weights (=
k   ;  ; l   ;
 2 2   2 2  elements of the filter matrix)
K l L l
2 2
 
K l L l
h(k, l)  1
k  l 
 Convolution 2 2

K l L l
2 2
b '  r, c    K l L l
b(r  k, c  l)  h(k, l) r, c
k  l 
2 2

b'  bh * is the convolution operator

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 50

 3.2.2.2 Convolution - properties
 b‘ is calculated as weighted arithmetic average of the corresponding
section of the input image and the filter matrix
 The convolution is a linear operation: a gray value b‘(r,c) is a linear
function of gray values b(r-k,c-l)
 Since all gray values b‘(r,c) of the output image are calculated in the same
way  convolution is shift invariant

Properties of convolution (results from linearity)

 commutative

 associative

 distributive

 scalar multiplication

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 51

3.2.2.2 Convolution - graphical representation

 In graphical representation a
convolution can by displayed by
input successively moving the mask
image b(r,c) over the image and calculating
the output value by muliplying
superposed values and summing
operator up the results
mask h(k,l)
 Note:
this will only be valid, if the filter
mask is mirrored at the center in
advance
(due to the definition of
indices k und l)
output
image b‘(r,c)

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 53

 3.2.2.2 Convolution - borders
 Problem: convolution is not defined  Solutions:
for pixels of the border. 




? 

 Border effects can be neglected in case of a single convolution. In case of

multiple convolutions the propagation of border effects can not be
neglected.
 In complex image processing systems, an appropriate handling of borders
is important.
 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 55

3.2.2.3 Mean operator (smoothing, low-pass)

 Application: Reduction of noise by smoothing (low-pass filtering) (pre-
processing)
 Procedure: Computation of the mean for a given region nk x nl
surrounding a pixel at the position (r,c)
(region often square: e.g. 3x3, 5x5, 7x7, …)
i i
1
b '  r, c     b  r  k, c  l 
with nk= nl ; nk= 3, 5, 7, …
n kl nkl = nknl
k  i l  i
i = (nk-1) / 2
with nk= 3 considered region
1 1 1 b  r  1, c  1 b  r  1, c  b  r  1, c  1
b '  r, c     b  r  k, c  l  b  r, c  1 b  r, c  b  r, c  1
9 k 1 l1 b  r  1, c  1 b  r  1, c  b  r  1, c  1

1 1 1
h  k, l   1 1 1
1
b'  bh
9
1 1 1
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 57
 3.2.2.3 Effect of mean operators - edge

BR 20 20 20 20 20 20 20 20 20 80 80 80 80 80 80 80 80 80 80
1 1 1 1
HR3 3

BR*HR3

HR5 1 1 1 1 1 1
5

BR*HR5

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 58

3.2.2.3 Effect of mean operators - distortions

BR 20 20 20 20 26 20 20 20 20 80 80 80 80 50 80 80 50 80 80

HR 1 1 1 1
3

BR*HR

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 60

 3.2.2.3 Gaussian smoothing operator (1)
 Smoothing filter with a Gaussian distribution

 k  k m 2  l lm 2
1
h( k , l )  e 2 2 where km,lm= coordinate
2 2 of the central pixel

with   1  0.0001 0.0015 0.0067 0.0111 0.0067 0.0015 0.0001

 0.0015 0.0183 0.0821 0.1353 0.0821 0.0183 0.0015 

0.0067 0.0821 0.3679 0.6065 0.3679 0.0821 0.0067 
1   -3

h g (k, l)  0.0111 0.1353 0.6065 1.0000 0.6065 0.1353 0.0111

2  -2

0.0067 0.0821 0.3679 0.6065 0.3679 0.0821 0.0067 

 
-1

 0.0015 0.0183 0.0821 0.1353 0.0821 0.0183 0.0015  0

 0.0001 0.0015 0.0067 0.0111 0.0067 0.0015 0.0001


1

1 3

h(k, l)   h (k,l)
 h g (k, l) g
-3 -2 -1 0 1 2 3

k,l

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 62

3.2.2.3 Gaussian smoothing operator (2)

 Properties in comparison to the mean operator
 A higher weighting is given to the central pixel
 The weighting of the neighboring pixels is dependent of the distance
 Better and more adaptative filter properties

 Size of the filter mask

 Gauss function is infinite
 Filter mask finite
 Typical window/kernel size: 2x3+1
 Only elements with high weights are kept
 Elements with insignificant weighting are eliminated  no
needless multiplications
 3-area covers >99%

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 63

3.2.2.3 Gaussian smoothing operator (3)
 Example of Gaussian filters with different sigma and a same filter size
(31x31)

 
-15 -15

-10 -10

-5 -5

0 0

5 5

10 10

15 15
-15 -10 -5 0 5 10 15 -15 -10 -5 0 5 10 15

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 64

3.2.2.4 Gradient operator (1)

 For sake of simplicity edge
 1D case. bright
dark
 Continuous smooth distribution.
 1D function g(x) shows an edge x
(dark/light transition).
 First derivative g'(x)
 The maximum of g'(x) indicates g
where the strongest jump of g(x) g(x)
(highest variation of the gray
values) occurs.
 The location x0 of the maximum
corresponds to the position of the g´(x)
edge.
 Second derivative g''(x)
 The zero-crossing indicates the
inflexion point of g(x). x
 The location x0 of the zero-
crossing represents the position x0 g´´(x)
of the edge.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 65

 3.2.2.4 Gradient operator (2)
 Application: emphasis on local differences of gray values
 The gradient of a continuous function b(x,y) is defined as a vector

b(x, y)  b(x, y) 

grad{b(x, y)}  ex  ey
x y
 Using partial derivations, the strength and orientation can be calculated
2
 b(x, y)   b(x, y) 
2
 magnitude: grad{b(x, y)}     
 x   y 
 b(x, y) b(x, y) 
grad{b(x, y)}  arctan 
x 
 direction:
 y

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 66

3.2.2.4 Gradient operator - effect along a row

Bx 20 20 20 20 20 20 20 21 32 50 68 79 80 80 80 80 80 80 80

Hx -1 0 1

B‘x

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 68

3.2.2.4 PREWITT Operator
 Only few pixels are taken into consideration when the discrete differential
gradient operator is used. Therefore, this operator is very sensitive to
noise. To cope with this problem, low pass filters are generally applied
before the derivation is computed.

 Alternatively, a larger neighborhood can be taken into consideration for

the computation of the derivates, which then includes an average
operation. An example is shown by the PREWITT operator, where the
original image is convolved with the following two 3x3 kernels to calculate
approximations of the derivatives - one for horizontal changes, and one
for vertical ones:

 1 0 1  1  1  1
H PREWITT _ x   1 0 1 H PREWITT _ y   0 0 0
 1 0 1  1 1 1

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 70

3.2.2.4 SOBEL Operator

 A better effect can be achieved by weighting the elements in a different
way as done in the SOBEL operator whose kernels are:

 1 0 1  1  2  1
H SOBEL _ x   2 0 2 H SOBEL _ y   0 0 0
  1 0 1  1 2 1

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 71

3.2.2.4 Magnitude & direction of gradient by SOBEL

bSOBEL _ MAG 

_ x  bSOBEL _ y
2 2
bSOBEL

BSOBEL _ x BSOBEL _ MAG

bSOBEL _ DIR 
 bSOBEL _ y 
arctan  
 bSOBEL _ x
 

BSOBEL _ y BSOBEL _ DIR

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 72

3.2.2.4 Gradient operator - exercise

 Draw the steps of the calculation of the gradient using the sobel operator!

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 73

3.2.2.4 Approximations for magnitude of the gradient
 Often approximations for the gradient magnitude are calculated from the
direction dependent differences of intensity by a sum e.g.:

b(x  1,y  1 )  b(x  1,y  1 ) b(x  1,y  1 )  b(x  1,y  1 )

b'SOBEL   2b(x  1,y)  2b(x  1,y)   2b(x,y  1 )  2b(x,y  1 )
 (x  1,y  1 )  b(x  1,y  1 )  b(x  1,y  1 )  b(x  1,y  1 )
 b'SOBEL _ x  b'SOBEL _ y

b'ROBERTS  Max  b( x  1, y  1)  b( x, y ) , b( x, y  1)  b( x  1, y ) 

b'KOMPAß  Max  b' N , b'O , b'S , b'W , b' NO , b'SO , b' NW , b'SW 

With convolution with compass gradient masks

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 75

3.2.2.4 Kompaßgradienten, Masken

 Direction dependent difference operators

 1 1 1  1 1 1  1 1 1
 1  1
H NW 
  1  2  1 HN   1  2 H NO   1  2
 1  1  1   1  1  1   1  1 1
Northwest North Northeast

 1 1  1  1 1 1
 1
H W   1  2  1
 HO   1  2
 1 1  1   1 1 1
West East
 1  1  1  1 1  1  1 1 1
H SW   1  2  1 H S   1  2 1 H SO    1  2 1
 1 1 1  1 1 1  1 1 1
Southwest South Southeast

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 76

3.2.2.5 Laplace operator
 The Laplace operator  calculates the sum of the second partial
derivations of a continuous function b(x,y) for the variables x and y.

 2 b ( x, y )  2 b ( x, y )
b ( x , y )  
x 2 y 2

The second derivation can be implemented by the difference of the

difference of neighboring pixels along the row and column
 2 b(x, y)  b(x  1, y)  b(x, y)    (b(x, y)  b(x  1, y) 

x 2  (x  1)  x 
2

 b(x  1, y)  2b(x, y)  b(x  1, y)

 b(x, y)
2
 b(x, y  1)  2b(x, y)  b(x, y  1)
y 2

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 77

3.2.2.5 Laplace operator, masks

 The differences can be calculated by vectors or a matrix H.

 0 0 0  0 1 0  0 1 0
H   H x  H y   1  2 1   0  2 0   1  4 1
 0 0 0  0 1 0  0 1 0

Often different masks are used, which use algebraic sign vice versa or
other binomial representations.

 0  1 0  1  1  1  1 2 1   1  2  1
H1   1 4  1 H 2   1 8  1 H 3   2 4  2 H 4   2 12  2
 0  1 0  1  1  1  1  2 1   1  2  1

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 78

 3.2.2.5 Laplace operator, effect along a row

Bx 20 20 20 20 20 20 20 21 32 50 68 79 80 80 80 80 80 80 80

Hx 1 -2 1

B‘x

 The values of the resulting image can not immediately displayed, because
of positive and negative values -> scaling between 0 and 255
 The Laplace-Operator ist very sensitive to noise.

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 79

3.2.2.5 Laplace operator, examples

 0  1 0  1  1  1
H1   1 4  1 H 2   1 8  1
 0  1 0  1  1  1

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 81

3.2.2.6 LoG Operator (Laplace of Gaussian)
 To reduce the noise, the image can be first smoothed by a Gaussian filter
(G*Image). Then, the result is convoluted by the Laplace operator
(D(G*Image)). Alternatively, ((DG)*Image) can by computed.

 2 G ( x , y )  2 G ( x, y ) 1 
x2  y2
LoG : G   G ( x, y )  e 2 2
(x) 2 (y ) 2 2 2

x y
2 2
x y
2 2
G(x, y) 1  x 1 
 e 2 2
  e 2 2
 ( x)
x 22  2
24
x 2  y2 x 2  y2
 2 G(x, y) 1  x 1 
 e 2 2
 2  ( x)  e 2 2
 (1)
(x) 2
24  2 4
1 
x 2  y2
 x  2 
 e 2 2
   1
24    
 2 G(x, y) 1 
x 2  y2
 y  2 
 e 2 2
   1
(y) 2
24    
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 82

3.2.2.6 LoG Operator (Laplace of Gaussian)

x 2  y2
1   x 2  y2  x2  y2
G  e 2 2
  2  k2
24  
2

2
2
1  k2 2
LoG : G  e  k  2 
2 4

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 83

3.2.2.6 LoG Operator (Laplace of Gaussian)
 Due to the similarity between the shape of LoG and a Sombrero, the
operator is called Mexican-Hat-Operator.
 The effect of the LoG-Operators for edge detection depends on the
standard deviation  .

=4 =5

An edge is detected by finding the zero-crossings

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 84

3.2.2.6 DoG-Operator (Difference of Gaussians)

 The LoG operator can be approximated by the difference of two Gaussian
(DoG).
 The relation of the standard deviation should be 1/2=1.6

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 85

 3.2.2.7 Effect of linear operators (convolution)

space domain frequency domain

 smoothing edges
 reduction of distortions by weighted decreasing of high
small objects summation of space frequencies
 reducing noise neighboring (lowpass filter)
pixel values
 deleting small details

 emphasizing edges
 detecting contours weighted decreasing of low
 suppression of big differences of space frequencies
homogeneous areas neighboring (highpass filter)
pixel values
 emphasizing details

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 86

3.2.2.8 Computational effort for convolution

 Direct computation
 For a mask size nhx x nhx, for each pixel
 nhx2 multiplications
 nhx 2-1 additions
 Example: mask with the size 5x5, for an 512x512 pixels image
 Computational effort:
 Decomposition of the filters
 Objective: reduction of the computational effort
 Decomposition of the filter mask H into two vectors H1,H2, so that
H=H1*H2
B * H = B * (H1*H2) = (B * H1) * H2
 Procedure: first line-by-line then column-by-column convolution
 Example: smoothing filter

1 1 1 0 0 0  0 1 0  1  before decomposition:

h(k, l)  1 1 1  1 1 1   0 1 0   1 1 1  1
1  1    1
9 9 9
1 1 1 0 0 0  0 1 0  1  after decomposition:

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 87

3.2.2.8 Filter decomposition, Example: smoothing filter

 Example

 Line-by-line convolution b2 = b1 * h1 1 1 1 1

(2 additions each) 1   1 1 1 1  1
h(k, l)  1 1 1  
b2(14.93) = 113+116+104 = 333 9  9 
1 1 1 1
b2(15.93) = 099+101+125 = 325
b2(16.93) = 000+107+105 = 212

 Convolution of the result column-by column b3 = b2 * h2

(2 additions + 1 division)
b3(15.93) =1/9 · (333+325+212) = 96.7 ≈ 97
 Note: Perform the division as last step to minimize errors from rounding
and quantization

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 89

3.2.2.8 Computational effort: Binomial filter decomposition (1)

 Binomial filter properties

 Discrete approximation of Gaussian filters
 Used as low pass filter
 Deducible from Pascal´schem Dreieck
with the binomial coefficients
 All binomial masks are separable:
n n!
(h(k,l))=(h‘(k))(h‘(k))  
 m  m! n  m  !
 1 4 6 4 1 1 
 4 16 24 16 4 4
1  1 1  
h(k, l)   6 24 36 24 6   1 4 6 4 1   6 
256   16 16  
 4 16 24 16 4 4
 1 4 6 4 1 1 

Further
decomposition:
 Linear increase of the computation effort with nhx [1 1]*[1 1] =[1 2 1]
 2nhx multiplications [1 2 1] *[1 1]=[1 3 3 1]
 nhx-2 additions
usw.
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 90
3.2.2.8 Computational effort: Binomial filter decomposition (2)
1 2 1 
 Binomial filter Example
h(k, l)   2 4 2
1 
16
1 2 1 
 Computation of the convolution without decomposition
1
b (15,93)  (1 113  2 116  1 104  2  99  4 101  2 125  1  0  2 107  1 105)  101.3  101
16
 Computation after decomposition
113  116 116  104   229 220   229  220   449 
b1  1 1   99  101 101  125    200 226   b  1 1  1 1  200  226  426
1

 0  107 107  105 107 212 107  212  319 

449  426
  b  1 1  1 1  1   426  319  745
1 875
1
    
1 1  1 1
b2   
  b1  1 1  1 1          875  745   101.25  101
16  1  1 16

 Before the separation: 5 multiplications + 8 additions + 1 division

 After the Separation: 4 additions + 1 division

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 91

3.2.2.9 Morphology - Rank operators

 The set of pixels within an operator window (structuring element, mask M)
will be sorted according to their gray values. The value of a defined pixel
position (often central pixel) will be replace by the value of the pixel with
the rank which is characteristic for the operator.

98 91 19 18 19 Rank sequence : {13, 16, 22, 31, 36, 39, 57, 69, 96}
92 96 57 16 10
88 69 31 22 11 niedrigster mittler höchster
42 39 36 13 15 Rang Rang Rang
35 33 32 17 12
Erosion: Median: Dilatation:

13 36 96

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 94

3.2.2.9 Median Filter
 Median value med{f(k)} of a sequence f(k)
 Example: f1(k)= 11, 12, 11, 9, 13, 15, 14
med{f1(k)} = 1212
 Example: f2(k)= 13, 12, 13, 14, 13, 13, 13
med{f2(k)} = 1313
 Effect
 smoothing
 elimination of small
sporadic image distortions
 Rules
 med{c+f(k)}=c+med{f(k)}
 med{c f(k)}=c med{f(k)}

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 95

3.3 Algebraic transformations

 An algebraic transformation combines two input images to an output
image:
b3(r,c) = f (b1(r,c), b2(r,c))

 Addition:

b1(r,c) b2(r,c) b3(r,c)=b1(r,c)+b2(r,c)

 Subtraction:

b1(r,c) b2(r,c) b3(r,c)=b1(r,c)-b2(r,c)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 96

3.3 Algebraic transformations - applications
 Applications
 Noise reduction by multiple image capturing and averaging

1 n
b  r, c    bk  r, c   r, c
n k 1
 Capturing object and background (b1) and background alone (b2) and
subtraction
 Movement detection by subtraction of two images captured at
different times
 Color composite and color transformation

 In special cases algebraic transformations can be replaced by point

transformations, e.g.
 b1(r,c) any image and b2(r,c) = const.  r,c
 b1(r,c) = b2(r,c)  r,c

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 97

3.3 Algebraic transformations - Example

red green blue

Generation of a monochromatic image from a true

color image
ggray = 0,299 gred + 0,587 ggreen + 0,144 gblue

gray

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 98

3.3 Algebraic transformations - Example
 Generation of a monochromatic image from a true color image

ggray = f (gred, ggreen, gblue)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 100

3.3 Algebraic transformations - Example NDVI

 Normalized Difference Vegetations Index (NDVI)
bNDVI = (binfrared - bred) / (binfrared + bred); [-1;+1]

Binfrared „rule of thumb“:

as higher NDVI,
the more vital is
vegetation

NDVI BNDVI

Bred Pseudo color

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 101

3.3 Algebraic transformations – Example: motion tracking

B1 B2

B = B1 - B2 +128

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 102

3.4 Image pyramid representation (1)

 Storage of the image information in different
resolution levels
Multi-scale sub-sampling of the image (often
performed with a factor of 2)

 Problem: Aliasing

 Procedure for the first level E1:

 Definition of the scale factor
 Smoothing the image E0 with a low-pass
filter, so that frequencies over a certain
threshold are eliminated
 Sub-sampling the smoothed image to
generate E1

 Further levels are obtained by replacing E0 by E1,

smoothing and then sub-sampling E0 …

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 103

3.4 Image pyramid representation (2)
 Applications
 Memory saving: the higher resolution levels are often not needed.
 Hierarchical algorithms:
 Approximations in lower resolution levels
 More accurate computation/results in higher resolution!
 Efficiency increase

 Remarks
 For its interesting frequency
properties, the Gaussian filter is often
used!
 Gaussian pyramid
 Instead of discrete levels, a
continuous scale factor could be used
 scale-space representation
 The total memory needed to store an
image pyramid, generated with a sub-
sampling factor of 2, is increased only
with the 1/3 of the memory required
for the original image.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_03] 104

Technische Universität München

Image Processing
4 Segmentation
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

Image Processing - Contents

Image Processing

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 2

4 Segmentation
 Pixel based operators
 Simple threshold
 Histogram based threshold
 Dynamic threshold
 Hysteretic threshold
 Region based operators
 Difference to start point
 Difference to mean value
 Difference to neighborhood
 Low-level model based operators
 Local operators
 1d case
 2d case
 Global operators (Hough-Transformation)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 3

4 Segmentation (1)
Segmentation
 Definition (Haralick und Shapiro, 1992):
”An image segmentation is the partition of an image into a set of
nonoverlapping regions whose union is the entire image. The purpose of
segmentation is to decompose the image into parts that are meaningful
with respect to a particular application“
 In image analysis often certain objects or
relevant areas has to be quantitatively
R2
described. For this purpose the objects have
to be extracted from the hole image. Formally
spoken: pixels of region R1 have to be R1/p
composed using a uniformity condition p and
to distinguished from region R2.
 Goal of segmentation:
Splitting of images into meaningful image R1: Object
parts R2: Background
p: Uniformity condition

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 4

4 Segmentation (2)
 Example
 Input: image, if necessary preprocessed, e.g.
by image point operation, filtering
 Output: different image areas (”regions“,
”equivalence classes“) which fulfill certain
conditions (e.g. foreground = dark image areas,
background= bright image areas)

 Segmentation is a decision process about

(important and unimportant) image contents

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 5

4 Segmentation (3)
 Approaches:
 Point based approach: Search all points or point sets that apply to
criterion p.
 Check each pixel, if it satisfies a specified (homogeneity) criterion and
assign the pixel to the according region.
 For the check, the attribute of the pixel is necessary. Often the grey value
or color of the pixel is used, but others are possible too like: local
neighborhood or global mathematical models.
 Region based approach: Determine the crossover – border of the area
of validity of p.
 Segmentation operations can combine several criterions to check the
pixel’s attributes (i.e. gradient strenght and gradient direction). Dependent
on the rules for combining criterions pixel can be assigned to different
regions. Resulting overlapping regions should be seperated afterwards.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 6

4 Segmentation (4)
 Coding of the results of the segmentation
 binary image for two equivalence classes (foreground and background)
 ”Label“-image for more equivalence classes (every grey value
corresponds to a class), or several binary images (allows to describe
overlapping equivalence classes)
 Targets and applications
 Data reduction(less grey values)
 Simple description of image content
 pre-stage of vectorization and feature extraction (points, lines,
regions)
 Focus on interesting image parts
 pre-stage to object recognition / image interpretation

In this chapter mainly grey values are used as examples. The methods are
usable for other features without any limitations.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 7

4.1 Pixel based procedures

 considering a pixel as a isolated unity (almost) without taking into account
the local neighborhood of a pixels
 Segmentation is based on featured related to the pixels: grayvalue,
colorvalue, edge magnitude, edge direction, etc.
 Simple Threshold
 Histogram based Threshold
 Dynamic Threshold
 Hysteretic Threshold

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 8

4.1.1 Simple threshold
 Assumption: a searched object has in the (pre-processed) input image a
homogenous grayvalue, which do not occur in the hole background
 exact knowledge about brightness distribution of the image required
 Segmentation of image B(r, c) by using one or more thresholds. Often a
lower (su) and upper threshold (so) is used
 absolute, global thresholds
 segmented region R is defined as: R = {(r,c) | su ≤ b(r,c) ≤ so}

Bild Segmentierung: su = 130, so = 180

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 9

4.1.1 Simple threshold - example

so=148 so=162 so=176 so=190 so=197

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 10

4.1.2 Histogram based threshold
 same as simple threshold, but the value is automatically taken from the
histogram of the image
 assumption:
objects and background have homogeneous, but different grayvalues,
which are dominant (= great occurrence, local maxima in the histogram)
 method:
 calculation of the histogram
 determination of local minima in the histogram ( = thresholds)
 segmentation by thresholds
 advantage compared to simple threshold:
 automatic determination of thresholds
 relations in brightness or contrast can change as long as grayvalues
of objects and background have different areas in the histogram.
 disadvantage: possibly many minima are not significant
 over segmentation by many thresholds
(remedy: smoothing of histograms)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 11

4.1.2 Histogram based threshold - example

 Histogram bases segmentation:
Coding of equivalence classes in grey scale

thresholds from thresholds from

original histogram smoothed histogram
55 equivalence classes 6 equivalence classes

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 12

4.1.3 Dynamic threshold
 assumption: searched object has a homogeneous grayvalue which
differers locally between object and background
 relativ, global threshold (difference between object and background)
 brightness of background must be locally estimated in the image
 method:
 estimating the background by intense smoothing
(rule of thumb: filter mask twice as large as the searched object)
 Segmentation based on the difference per pixel between original and
smoothed image
 let:
B (r,c) original imge
G (r,c) smoothed image
s relative threshold (minimal difference between object and background)R
segmented region
then:
 R = {(r,c) | B (r,c) − G (r,c)  s} for segmentation of bright objects
 R = {(r,c) | G (r, c) − B (r,c)  s} for segmentation of dark objects
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 13

4.1.3 Dynamic threshold - example profile

b(r,c)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 14

4.1.3 Dynamic threshold – example image

original image

smoothed image

segmentation
with global
threshold

segmentation
with dynamic
threshold

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 15

4.1.3 Dynamic threshold – pros and cons

 Advantages:
 Compensation of brightness differences
 Knowledge of relative brightness differences of object an background
sufficient
 Usage of local neighborhood for background estimation
 Disadvantages:
 More computations, critical only for real-time applications
 Object size has to be known
 Objects have to be positioned isolated. Otherwise the background
estimation fails
 Possible wrong segmentation on edges
 In general
 Better solution compared to global threshold
 Universal usability

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 16

4.1.4 Hysteretic threshold
 A generel problem of threshold operations: seldom a correct threshold
can be adjusted ”ad hoc“
 oversegementation, if threshold to generous,
 undersegmentation, if threshold to conservative

 Example: thresholds for road marking

s = 175 (oversegmented) s = 220 (undersegmented)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 17

4.1.4 Hysteretic threshold (2)

 Often grayvalue intervals are known, which cover in any case the
grayvalues of the searched object or never cover the grayvalues of the
searched object
 pre-segmentation into ”reliable foreground“ (s1) and ” reliable
background“ (s2)
 afterwards the remaining ”unreliable“ grayvalues will be assigned to one
of both pre-segmented regions according to a pre-defined condition.
 Most commonly used condition for assignment to the foreground:
 unreliable grayvalues must be neighbored to at least one reliable
grayvalue
 starting from a reliable grayvalue all unreliable grayvalues will be
iterativly tracked. Possible additional condition: shortest path from a
unreliable to a reliable grayvalue not have to exceed maximal length
(lmax).
 All remaining unreliable grayvalues will be assigned to the background.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 18

4.1.4 Hysteretic threshold - pros and cons
 Advantage:
 fixed threshold will be avoided
 minor variations of brightness in the image can be compensated
 local neighbourhood will be considered
 Disadvantage:
 depending on (hysteresis) condition -> longer calculation time
 global thresholds, major variations of brightness in the image cannot
be compensated
 Common applications:
 segmentaion of homogeneous regions
 determination of prominent edges from image of the gradient
magnitudes

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 19

4.1.4 Hysteretic threshold - example

Reliable foreground
g≥s1=220 (black) Unreliable greyvalules Foreground: assigned
s1≥g≥s2 (black) unreliable greyvalues

Results for foreground with

Reliable background hysteresis and Background: assigned
g≥s2=175 (black) lmax=30 (black) unrealiable greyvalues

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 20

4.2 Region based procedures
 basic idea: most objects in the image are represented by larger connected
regions ( = connected pixels)
 considering of pixel neighborhood during segmentation
 used approach : region growing
 Initialization (t0): selection of start point
 first pixel of the region
 examination of neighbored pixels concerning a pre-defined criterion
of homogeneity related to the features of the start point  adding of
”positiv“ tested pixels to region
 iterativ growing of region (ti i=1,2, . . .): examination of currently
neighboring pixels
 abort iteration, if no neighbouring pixels fulfill the criterion or all pixels
of the image are processed
 Difference to start point
 Difference to average value
 Difference to neighboring values

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 21

4.2 Region based procedures (2)

initialization segmentation growing

examined region
current region

 Remark: Only region growing algorithms are introduced here. There are many
other – more complex – methods of region based segmentation.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 22

4.2.1 Difference to start point
 Difference to start point
 Criterion for region growing: magnitude of grayvalue difference ΔgS,(r,c)
between start point S and checked pixel (r, c)
 segmentation of homogeneous regions
 Advantage:
 fast calculation
 stops at prominent edges
 Disadvantage:
 sensitive to noise
 grayvalue of start point possibly not representative for whole region
 ”holey“ or ”frayed“ regions as segmentation result
 For time-critical applications using simple images

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 23

4.2.1 Difference to start point - example

Initialisation
– maximum difference
ΔgS,(r,c)max=18

Segmented region
(black)

Initialisation
– maximum difference
ΔgS,(r,c)max=18

Segmented region
(black)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 24

4.2.2 Difference to mean value
 difference to mean value
 Criterion: magnitude of grayvalue difference Δgμi,(r,c) between checked
pixel (r,c) and average grayvalue μi of segmented region at iteration i.
Update average value, if all examined pixels of one iteration are
processed.
 segmentation of to a large extend homogeneous regions
 Advantage:
 less sensitive to noise compared to difference to start point
 Disdvantage:
 possibly cross the edges
 calculation time is longer

segmented region (black)

for lane border
maximal difference ΔgS,(r,c)max=18

segmented region (black)

for road
maximal difference ΔgS,(r,c)max=18

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 25

4.2.3 Difference to neighboring points

 Difference to neighboring points
 Criterion: magnitude of smallest grayvalue difference Δg min,(r,c)
between checked pixel (r c) and neighbored (maximal 8) pixels, which
belongs already to the region.
 segmentation of smooth areas, which must not be homogeneous,
e.g. grayvalue wedge
 Advantage:
 fast calculation
 stops at prominent edges
 result is independent on position of start point in area which has to be
segmented, because of considering only differences of neighbored pixels.
 Disadvantage:
 tends to cross weak edge areas (see example)
 sensitive to noise (but less than difference to start point)
 for more application useful, more flexible

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 26

4.2.3 Difference to neighbored points (2)
 Problem with difference to next neighbour
(maximal difference Δg min,(r,c)max =1):
 two grayvalue wedges in opposite direction and a black disconected bar

Original resulting label image with 3 regions

segmented region (black) segmented region (black)

for lane border für Straße (rechts)
maximal difference Δg min,(r,c)max = 3 maximal difference Δg min,(r,c)max = 3

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 27

4.3 Low-level model-based approaches

 Goal: Description of discrete image information with simple mathematical
functions derived from curves and areas
 Approaches:
 Local: Parameters of the function are calculated for every pixel from ist direct
neighborhood
 Global: Parameters are calculated taking into account the complete image
 Calculation of features from the functions (extrema, inlection, etc.)
 Segmentation = Determination of areas, where the image functions fits the
calcualted features

 Local approaches
Determination of local function properties:
Example with local maximum, but transferable to other properties like
minimum, inflecioni point, etc.
 Global approaches
Determination of global function properties
Esample with Hough transformation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 28

4.3.1.1 1d case
 Feature from image function
 Test, if investigated pixel is local extremum in its neighborhood
 Segmented region R defined as:
R = {x | g(x) = max(g(x + n)), n = −N, . . . ,N} (meist N = 1 oder N = 2)
g(x)

 Main application:
Postprocessing of edge filtered images
 Edge strenght and gradient direction per pixel given
 Test, if investigated pixel x0 is local maximum in gradient direction (where
necessary interpolation of neighboring grey values for x−1 und x1, graphic).
 Pixel that are no maxima are supprest
 Thninnung pf blurred edges =
Non-Maximum-Suppression

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 29

4.3.1.1 1d case – non-maximum-suppression

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 30

4.3.1.1 1d case (2)
 Attributes from approximated function (here: parable)
 Adaptation of second order polynom (parable) in each pixel. Local
maximum, if apex of parable (=maximum) falls into pixel
 Determination of maximum with sub-pixel accuracy

y=g(x)

Formula parable:
y = a + bx + cx2 y‘ = b + 2cx y‘‘ = 2c

Determine coefficients a, b, c with grey values

y−1, y0, y1 at x−1, x0, x1 (mit x−1 = −1, x0 = 0, x1 = 1)

y 0  a  bx 0  cx 02  a a  y0
1
y 1  a  bx 1  cx 21  a  b  c  b   y1  y1   y0'
2
1 1
y1  a  bx1  cx12  a  b  c c   y 1  2y0  y1   y0''
2 2
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 31

4.3.1.1 1d case (3)

 coeffizients b and c can be determined with the 1. and 2. derivative at
position y0
 Determination of apex xS of the parable (from coefficients at position x0):

 b  y '0
y '  b  2cx S  0  x S   ''
2c y0
 xS is local maximum, if y‘‘ = 2c < 0 and xS within the borders of the
investigated pixel.
 Applicattions: like before, but with higher accuracy
 Camera calibation, object measurements in industrial IP, extraction of
roads from airborne images.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 32

4.3.1.1 1d case (4)

original

Edge strength subpixel accurate edge position

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 33

4.3.1.2 2d case
 Features from image functions
 Test, if investigated pixel is local maximum (minimum) for a predifined
neighborhood (2D Non-Maximum-Suppression):
 For the segmentation of linear structures: Grey value mit describe
a local maximum in horizontal and/or vertical direction
  Test of 4 pixels
 For the segmentation of local maxima: Grey value must be local
maximum within a 3 × 3 or bigger neighborhhood
  Test of 8 pixels
 Applications: Non-Maximum-Suppression, if no information about the
direction is available

4 neighborhood 8 neighborhood

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 34

4.3.1.2 2d case (2)

 Non-Maximum-Suppression without information about the direction

 Grey value must be maximum in vertical and/or horizontal direction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 35

4.3.1.2 2d case (3)

 Features from approximated functions
 Image function is seen as „grey value mountains“: Adaptation of
planes of second order or higher in3×3 or bigger neighborhood of
each pixel. Normally formalized as adjustment:
 observations: grey values
 unknown: coefficients of the plane function
 Determination of differential geometric values from calculated planes
(Extrema, like Maxima, Minima, flat points, saddle points):
”Topographic Primal Sketch“
 Determination of extrema with sub-pixel accuracy

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 36

4.3.1.2 2d case (4)
 Main applications:
 Like before but with higher accuracy (subpixel)
 Determination of subpixel center from prominent points determined
with point operations
 Detection of conspicious image structures like gauge marks,
nucleus…)
 Topographic Primal Sketch: Struktural complete description of
complete image function
 Input for different applications in image interpretation: Analysis of
digital terrain models (DTM), analysis of cellular structures …

 Example:
valley line and ridge line from DTM with differential geometry, i.e.
watersheds of glaciers and creeks.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 37

4.3.1.2 2d case (5)

 DTM-Analysis Vernagtferner: DTM-Analysis Vernagtferner:

valley and ridge lines: white and black 3D view of valley and ridge lines

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 38

4.3.2 Global approaches – Hough Transformation (1)
 Determination of global attributes of the image function
 normally for linear structures (after edge filtering)
 Example: Segementation of staight lines (Hough-Transformation).
 Useable for other structures like curves and circles (generalised
Hough-Transformation)

 Hough-Transformation:
 Parameter description of a line with
d = r · sin(α) + c · cos(α)
 Determination of a and d per pixel:
 a from edges detection,
 d from line parameterisation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 39

4.3.2 Global approaches – Hough Transformation (2)

 Calcualtion of 2d histogram p(a,d) with
pixel coordinates (r,c) saved. Sometimes
weighting of histogram entries with the
edge strenght
→ Strong maxima in the hinstogram
correspond to strong straight lines in the
image

 Determination of strong histogram maxima;

the pixel coordinates saved at the found
maxima are the segmented edges

 If only points on edges are given with no

edge direction, the (distance-) curve of the
point is added to the histogram

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 40

4.3.2 Global approaches – Hough Transformation (3)

 Determination of noticable histogram maxima. The pixel coordinates

stored there (and in their direct neighborhood) are the segemented
noticable straight image edges.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 41

4.3.2 Global approaches – Hough Transformation (4)

 Advantages
 Fast calculation. Usable for real-
time applications
 Lines may be discontinuous
(important for road mark
extraction and detection of
partial occluded objects)

 Disadvantage
 For functions with many
parameters the maxima search
can be very complex (ellipsoid: 5
parameters  5D histogram).

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 42

4.3.2 Example – segmentation of straight edges

Original Edge direction Hysteresis thresold

Edge strenght (Sobel filter) Non-Maximum-Suppression Hough-Transformation

 to edge direction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_04] 43

Technische Universität München

Image Processing
5 Binary image processing
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

Image Processing - Contents

Image Processing

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 2

5 Binary image processing
 Neighborhood
 Euler number and holes
 Morphologic operations
 Basic operations
 Dilatation
 Erosion
 Opening
 Closing
 Mean axis
 Distance transformation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 3

5 Binary image processing (2)

 Intensity values of pixels are limited to 0 and 1 for binary images
(quatization).
 Example:
 Pixel with value 1
 relevant information: set pixels
 Total quantity of these pixels: image foreground
 Pixel with value 0
 Total quantity of these pixels: image background
 Binary image result from:
 Segmentation (without subpixel accuracy)
 manual digitizing
 Scanning of documents / maps
 Reading of a GIS layer
 Binary image often result in very efficient algorithms

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 4

5.1 Neighborhood and connected component
 In image defferent types of neighborhood can be defined, i.e:
 4 neighborhood
 8 neighborhood
 Definition connected component
 A path from position (i0, j0) to position (in, jn) is called a pixel sequence based on
the k neighborhood, if (ih−1, jh−1) is connected to (ih, jh) by such a k
neighborhood. (h = 1, . . . , n)
 A pixel set S is called a pixel set based on the k neighborhood, if
 Two random pixels of S can be connected by a pixel sequence based on the k
neighborhood, which lies completetly in S, or if
 S has only one pixel.
 A pixel set S is called connected component based on the k neighborhood, if:
 S is a pixel set based on the k neighborhood,
 The value of the image elements of pixel set is equal (often: image foreground),
and
 No other pixel set S0 exists, containing S
 A connected component contains the maximum pixel set based on the k
neighborhood (of the image foreground).

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 5

5.1 Connected component - example

Binary image with set (1)

and unset (0) pixels

Two connected components based on Four connected components based on

the 8 neighborhood. the 4 neighborhood

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 6

 5.2 Euler number and holes
 Euler number e
is the number of connected components of an image B minus the number
of holes in B.
 Hole
is a conncted component of hte background which is not conneted to the
image border.
 The calculation of the Euler number uses either the 4 neighborhood for the
foreground and the 8 neighborhood for the background or vice versa.
e4(B1) =

e8(B1) =

e4(B2) =

e8(B2) =
B1 B2

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 7

5.3 Morphologic operations

 Mathematic morphology is based on
 Image processing based on shape information
 Operations with set theory
 Simplify images with
 Isolation or amplification of relevant shape information
 Elimination of disturbances
 Structuring element is the base element of mathematic morphology
 It consists of a set of pixels
 Rerefence point of a structuring element is the point of reference of
the morphologic operation
 Structuring element is in general much smaller than the binary image
 Examples:

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 9

 5.3.1 Erosion and dilatation on binary images
 Erosion
 Bright objects (g=1) on dark background (g=0) will decrease
 For binary images bb‘(x,y) small objects (g=1) disappear, e.g. 3x3

bb ' ( x, y )  bb ( x  1, y  1)  bb ( x  1, y )  bb ( x  1, y  1)
 bb ( x, y  1)  bb ( x, y )  bb ( x, y  1)
 bb ( x  1, y  1)  bb ( x  1, y )  bb ( x  1, y  1)

 Dilatation
 Bright objects (g=1) on dark background (g=0) will increase
 For binary images bb‘(x,y) small objects (g=0) disappear, e.g. 3x3

bb ' ( x, y )  bb ( x  1, y  1)  bb ( x  1, y )  bb ( x  1, y  1)
 bb ( x, y  1)  bb ( x, y )  bb ( x, y  1)
 bb ( x  1, y  1)  bb ( x  1, y )  bb ( x  1, y  1)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 10

5.3.1 Erosion, Dilatation and Median

:0
:1

M
a) section of image B c) Dilatation: B  M

duality:

BM=BM

BM=BM
b) Erosion: B  M d) Median

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 11

 5.3.2 Opening and Closing
:0 :1

B B

Erosion Dilatation
BM BM

Opening Closing
BM BM

B  M = (B  M)  M B  M = (B  M)  M

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 13

5.3.2 n times Opening or Closing

:0 :1

B B

BM BM

(BM)M (BM)M

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 15

 5.3.3 Morphology - binary images, contour
:0 :1 / set difference

B = B / B  M
=BBM
M =BBM

Erosion: B  M contour: B = B / B  M

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 17

5.3.4 Morphology - binary images, skeleton

nmax
Simple skeleton: S =  [(B  nM) / ((B  nM)  M )]
n=0

nmax: maximal size after

B is eroded to an empty
set

M B  nM

disadvantage:

(B  nM)  M

(B  nM) / ((B  nM)  M ) S

n=0 n=1 n=2

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 19

5.4 Centerline (I)
 Elongated regions in binary images often are reduced to topological
(connected) lines with one pixel width (also called skeleton)
 This is also done for binary edges extracted from grey value images using
thresholds.
But for edges non-maximum-suppression is preferred because the center
of an edge segmented using a threshold is not necessarily the maximum
of the gradient.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 26

5.4 Centerline (II)

 Centerline: Set of all points within an object satisfying the condition that at
least two points exist on the object‘s border with the same shortest
distance to the centerline.

 Calculation of the centerline:

 Raster based (here without subpixel accuracy)
 Vector based (here not discussed)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 27

5.4.1 Centerline calculation – raster based
 „steppe-file - algorithm“
 Fire is sparked at the border of an “ homogeneous“ object, which is
burning inwards with constant speed. The centerline corresponds to
the extinction points of the flame fronts.

 Rasterized image is removed in layers around the object until only the
topological connected line remains ( = centerline).
 Iterative execution of the erosion with different structuring elements.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 28

5.4.1 Centerline calculation - vector based

 Computational geometry:
Calculation of the centerline directly from the surrounding polygon
(more precise, computational effort)
 Remark:
If the border of the objects consists of straight line pieces (polygon), then
the centerline only constists of straight line and parable pieces.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 29

 5.4.2 Centerline - problems (1)
 Attention: Results of centerline transformation do mostly not correspond
to the optimal geometric model:
 Node shift: Line intersections are shifted
 Node bridges: Line intersections are falsified – two ore more nodes
instead of one
 This is not an error of the method but inherent to the model of the
centerline.

Node bridge Node shift

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 30

5.4.2 Centerline - example

b) Segentation with
histogram threshold
c) Closing and Opening
a) Original image

d) Selection of big connected components e) Dilatation f) Centerline

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 31

5.4.2 Centerline - problems (2)

a b c d e f
 The segmented image (b) is processed with thinning algorithms of Hilditch
(c), Tsuruoka (d), Deutsch (e) and Tamura (f).
 The results show the different influence of disturbances of the contour on
the algorithms and the similar influence of inclusions.
 One possible solution to reduce the disturbances of the segmented image
are morphologic filling operations like dilatation and erosion

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 32

5.4.2 Centerline - problems (3)

Segementation
with simple
threshold so

Erosion and
Dilatation

Centerline

so=148 so=162 so=176 so=190 so=197

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 33

5.5 Distance transformation (1)
 Goal:
Image containing the shortest distance of every pixel to a line segment.
The line segment can be the border of the given object respectively the
border of the connected component.
 Centerline is generated by chaining of the realtive maxima.

 Attention: All pixels of the centerline are maxima of the distance

transformation, but not vice versa! That‘s why the centerline is often
calculated using thinning and only the distance values are calculated from
the distance transformation.
 Distance metric determines how accurate a circle is approximated (how
isotrop the metric is: Metric non isotrop  different distances in different
directions).
 In general, filters are the faster and less isotrop, the more local they
operate and the less they weight.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 34

5.5 Distance transformation (2)

 Generation of distance map
 Pixel are given the distance to the next edge pixel as greyvalue in the
distance map
 Edge pixels are given the grey value 0
 Efficient calculation with non-iterative algorithm

 Distance map is generated for model image and search image

 Object recognition by matching of distance maps (z.B. correlation)

=> Advantage to grey value matching

homogeneous regions (without edge pixels) get additional information
(distance to nearest edge)!

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 35

5.5 Distance transformation (3)
4 neighborhood 8 neighborhood
(Cityblock distance) (Chessboard distance)

654333333456 333333333333
543222222345 322222222223
432111111234 1 321111111123 111
321 ** ** ** 123 1*1 321 ** ** ** 123 1 *1
432111111234 1 321111111123 111
543222222345 322222222223
654333333456 333333333333

Kampfer(3,4)-distance: 4/3≈√2,
i.e. Euklidean distance over diagonal is
approximated

2 11 10 9 9 9 9 9 9 10 11 12
11 8 7 6 6 6 6 6 6 7 8 11
10 7 4 3 3 3 3 3 3 4 7 10 434
9 6 3 * * * * * * 3 6 3 3*3
10 7 4 3 3 3 3 3 3 4 7 10 434
11 8 7 6 6 6 6 6 6 7 8 11
12 11 10 9 9 9 9 9 9 10 11 12

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 37

5.5 Distance transformation - example

 Distance matrices of the line picture of the number five. The distance values
are coded as grey values (Euklidian distance).
The initial point set of the lines is set to black; the higher the distance value,
the brighter the grey value.

Image of reference line segements Image of line segments with noise

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_05] 38
Technische Universität München

Image Processing
6 Vectorization and geometric primitives
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

Image Processing - Contents

Image Processing

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 2

6 Vectorization and geometric primitives (1)
Input:
 Plane or line structures as binary image, for example region as
segmentation result.
 Eventually additional information like edge direction or edge
magnitude of a pixel

Output:
 Vector data like polygons, straight lines, circles and ellipses
-> so called geometric „primitives“

Goal:
 Data reduction with focussing on relevant features
 Extraction of more significant structural and topological information
like start point and end point of a polygon, neighborhood relation of
regions, et al.

-> Symbolic description of important image content with simple geoemtric

elements.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 3

6 Vectorization and geometric primitives (2)

 Applications:
 Precise measurements in industrial image processing
 calibration
 Prestep for methods in automatic image interpretation like grouping
and graphs

Region as binary Approximation with Skeleton as Approximation with

image circle binary image parable line

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 4

6 Vectorization and geometric primitives (3)
 Often two-step processing for geoemtric primitives:
 Determination of topological explicit pixel chains, so called „contours“
 „Vectorization“, „Raster-vector-conversation“
 Simplification of pixel chains with fitting of polygons, lines, circle and
ellipsoid elements Determination of geometric primitives

Kontur 1 (r,c):
0 0
1 1
1 2
2 3
3 3
Kontur 2 (r,c):
3 3
4 4
5 5
Kontur 3 (r,c):
3 3
3 2
4 1
Approximation
Skeleton as 5 1 with polygons
binary image
Vectorization:
contours
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 5

6.1 Vectorization of linear structures

 Goal
 Linking of (centers of) pixels of skeletons
 Storage with topological explicit data structur

 Results
 Contours with end nodes, inner nodes and cross nodes

 Approach
 Linking
 Simple Linking
 Direction dependent Linking
 Chain-Code (Freeman-Code)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 6

6.1.1 Linking - simple Linking
 Background: Skeleton as binary image

 Approach:
 Determination of start points: Search image for endpoints and cross
points of the skeleton (e.g. with masks)
 Linking: Follow the skeleton from the start point until the next start
point is reached
 Break, if all start point are processed

 Advantage:
 Explicite description of the skeleton, no loss of information

 Disadvantage:
 Needs much memory

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 7

6.1.1 Linking - simple Linking

Skeleton as binary Masks for cross Found start points

image points Contour 1 (r,c):
0 0
1 1
1 2
2 3
3 3
Contour 2 (r,c):
3 3
4 4
5 5
Contour 3 (r,c):
3 3
3 2
Result: 4 1
Start of Linking Linking (1st step) Contours 5 1

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 8

6.1.1 Linking – direction dependent Linking
 Background:
 Skeleton as binary image
 Direction information per pixel, p.e. from edge detection

 Approach:
 Determination of start points: like simple linking
 Linking: Check if the difference of the directions of neighboring pixels is smaller
than a given threshold.
 smaller (equal): Linking
 greater: End of contour and definition of a new star point (p.e. in the break
between the two pixels)  Continuation with new contour
 Break, if all start point are processed

 Advantage:
 explicite desription of the skeleton, no loss of information
 Better adaptation to image information

 Disadvantage:
 Needs much memory
 Possible fragmentation at contours with uncertain definition of th direction, p.e.
in the direct neighborhood of cross points

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 9

6.1.1 Linking – direction dependent Linking

Contour 1 (r,c):
0 0
1 1
1 2
1.5 2.5
Contour 2 (r,c):
1.5 2.5
2 3
3 3
Contour 3 (r,c):
3 3
4 4
5 5
Contour 4 (r,c):
Skeleton with directional Addintional start point for 3 3
information and start point difference of direction 3 2
> 60° 4 1
5 1

Result: more contours

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 10

6.1.2 Chain-Code (Freeman-Code)
 Method related to simple linking but with modified representation of
contours
 Determination of start points (like simple linking)
 Linking: Store coordinates of start point and direction zu next pixel
 Coding of direction from 0, ... , 7
Contour 1:
r,c: 0 0
dir: 3
2
3
4
7 0 1 Contour 2:
r,c: 3 3
6 2 dir: 3
3
5 4 3 Contour 3:
r,c: 3 3
dir: 6
5
4
Direction code Start points Result: contours

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 11

6.1.2 Chain-Code (Freeman-Code)

 Advantages:
 Only few memory space necessary (coordinates of start points and 3
bit per additional contour point)
 Simple identification of breaks in a contours directly out of the chain-
code (alike for straight or parallel line segements, if their direction is
0°, 45°, 90°,…)

 Disadvantage:
 less explicite
 Time consuming because the original coordinates have to be
reconstructed for many algorithms and applications (p.e. Display
contour).

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 12

6.2 Vectorization of plane structures
 Goal
 Surrounding polygon with similar area than original region

 Result
 Always closed contours

 In principle similar approach like for linear structures

 Determination of border pixels of a region
 Store pixel coordinates as contour

 Problem
 Surrounding polygons of neighboring regions do not necessarily tangent
(see regions A and B)  Changes in the topology

 Solution
 Store coordinates of outer pixel edges instead of pixel coordinates
 more memory space, but normally, the perpetuation of the topology is
critical.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 13

6.2 Vectorization of plane structures

A
B

Two regions Surrounding polygons Surrounding polygons

with stored pixel with stored pixel edges
coordinates

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 14

6.2 Vectorization of plane structures
 For more than one region  often post processing
 Splitting at cross points
 Elimination of overlapping (redundant) contour segments

 Additionally hybrid data structure

 per contour: Store indices of neighboring regions
 per region: Store indices of (maybe splitted) surrounding polygons

 This is necessary p.e. for geo-information systems

 Example border of estates
 „Which regions are separated by the given contour?“
 Example Landmanagment
 „Which regions are neighbored to a given region?“
 Example borders of a glacier
 „Which borders enclose a given region?“

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 15

6.2 Vectorization of plane structures

Contour 1: Region A:
- coordinates (r/c): - coordinates (r/c):
.. , .. , .. 0/2, 0/3, 1/1, 1/2, ..
3
1 - neighboring regions: - contours:
A 4 A 1,2,3
2 5
B Contour 2:
- coordinates (r/c): Region B:
.. , .. , .. - coordinates (r/c):
- neighboring regions .. , .. , ..
-Split contour at cross : - contours :
points A,B 5,4,2
Contour 3:
- Eliminate redundant ... ...
segements (contour „2“)
Data structure for Data structure for
contours regions

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 16

6.3 Determination of geometric primitives
 Goal
 Simplification from contours to geometric curves like polygons or
circle and ellipsoid segments
 Replacement of many contour points with few curve parameters
opt. With start unf end points (datareduction)
 Accentuation of fundamental geometric structures (focussing)

 Contour approximation
 With polygons
 With line segments
 With curve and ellipsoid segements
 combined

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 17

6.3.1 Polygon approximation

here: „Ramer“ or „Douglas-Peucker“ algorithm
 Conditions
 Approximating polygon must not have a distance bigger than d to the
contour at every contour point.
 Recursive procedure:
 Linking of start and end points of the contour or a contour segment
with a line segment
 initial polygon
 Determination of contour point ps with the biggest distance ds to the
line segment
 Test: ds > dmax
 no: line segment approximates the contour adequate
 yes: P is inserted as new polygon point
 Polygon constisting of two line segments
 Continuation of the procedure for every line segment of the polygon
 Break if no line segment has a distance bigger than dmax to the
contour

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 18

6.3.1 Polygon approximation – splitt-algorithm of Ramer

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 19

6.3.1 Polygon approximation – splitt-algorithm of Ramer

p p
s s

P
s

p p
b e

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 20

6.3.1 Polygon approximation – splitt-algorithm of Ramer

 Advantages
 Fast and efficient
 Resulting polygons are equal to a sub group of contour points
 no shift of the polygon

 Disadvantages:
 The surface area is in general getting smaller for closed contours
 Polygon point are not always in the corners of a contour
( = points with maximum curvature)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 21

6.3.1 Polygon approximation – sequential splitt-algorithm

 Ramer algorithm
 Sometimes oversegmentation
 sequential splitt-algorithm
 Approximation of contour sequentially with line segments
 Conditions
 Minimal lenght nS of the line segements nkl of the contour
 Break if quality criterion dappr is reached

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 22

 6.3.1 Polygon approximation – sequential splitt-algorithm
p
s=25
1
d appr 
nkl
d
pi Pkl
2
( pi )
n step

d appr  d max or nkl  ns min

ps  Pkl ; d ( ps )  max d ( pi )
k  i l
p p P p
b e k=1 l=47
p p p p
s=17 l=25 l=17 k=17
p
s=33

p p p
k=1 k=1 l=47
p
17
p p p
k=17 k=33 33
p
l=33

p p p
l=47 b e

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 23

6.3.2 Approximation of line segments

b) Binary image (after c) Contour image (Contour

a) Grey value image
threshold operation) chains)

d) Approximation with line

segments. Short contours
are not approximated.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 24

6.3.2 Approximation of line segments
 Simple case: Regression line of a contour
 Parametrization: Hesse-Normalform
a xi + b yi + c = 0 mit a2 + b2 = 1

 Adjustment:
 Observations: contour points xi , yi
 Unknown: Parameter of the line a, b, c
 Condition: a 2 + b2 = 1
 Approximation: deduce from contour points

 Determination of start and end points of the line segment:

 Perpendicular from start and end point of the contour to the line

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 25

6.3.3 Approximation of circle and ellipsoid segments

Similar methold like line segments:
 Parameterization:
 Circle: (xi – xm)2 + (yi – ym)2 – r2 = 0
 Ellipsoid: axi2 + bxiyi + cyi2 + dxi + eyi + f = 0
 Adjustment:
 Observations: Contour points xi , yi
 Unknown: center point xm , ym and radius r
respectively params a,b,c,d,e,f
 Conditions for ellipsoids: b2 – 4 a c < 0
in Implementation: b2 – 4 a c = –1
 Approximation: deduce from contour points
→ generally a robust adjustment is essential

 Determination of start and end points of circle and ellipsoid segments like
for lines

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 26

6.3.3 Approximation of circle and ellipsoid segments

Segmented objects of Contour (red) and fitted Contour (red) and

a calibration field circle (green) robust adjusted circle
(green)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 27

6.3.4 Combined Approximation

 Usual problem:
 No a priori knowledge about the best approximating geometric primitives for a
contour.
 combined processing necessary
 Approach:
 Polygon approximation and splitting of contour at polygon points
 many short contour segments in curved sections
 Approximation of a circle or ellipsoid segment to the aggregation of two
neighboring contour segments
 If the error of the approximation gets smaller, replace the contour segtment
with the circle or ellipsoid segment
 Repeat until all contour segment are replaced and no replacement is
successful
 Post processing: Combination of end points of neighboring primitives (p.e.
snapping)
 Final remark:
 Small disturbances can impede the approximation of contours with few
geometric primitives because then the approximation error does not become
smaller for circle and ellipsoid segments. Often grouping algorithms have to be
performed after to handle these problems in a better way.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 28

6.3.4 Combined Approximation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_06] 29

Technische Universität München

Image Processing
7 Feature extraction
2014 WS

Uwe Stilla
[email protected]
Ludwig Hoegner
[email protected]

Photogrammetry & Remote Sensing

Prof. Dr.-Ing. U. Stilla

Image Processing - Contents

Image Processing

 Introduction
 Digital image characterization
 Image transformation
 Segmentation
 Binary image processing
 Vectorization and geometric primitives
 Feature extraction

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 2

7.1 Introduction
 Similar task as segmentation (classification of the image in important and
non-important regions, see chapter 5), exactly: Extraction of geometric
primitives
 Feature points
 Linear structures (lines and edges)
 Regions (homogeneous grey values or texture or color or…)
 Feature extraction focusses less on separation of foreground and
background (like segmentation) but more in specific geometric,
radiometric or topologic properties of primitives
 Main task is the description of important image content with points, linear
structures and regions (in contrast to segmentation: binary or labelled
image)
 necessary:
 „Model“ of a primitive: „Which criterions distinguish a specific primitive like p.e.
a line from other image content?“
 Extraction scheme: point transformation, filtering, transfer from raster to vector
data or the like, mostly combination of such operations.
 Goal and application:
 Focussing on important image content
 Data reduction
 Basic of many algorithms in digital photogrammetry, remote sensing and image
analysis / interpretation

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 3

7.2 Extraction of feature points (1)

 Requirements for the model of feature points:
Description of points, which are
 conspicuous (rare)
 invariant to geometric abd radiometric transformations (noise insensitive)
 Easy to localize / indentify

 Example: Moravec operator

 Belongs to the class of so called statistic interest operators
 Creates an interest value V(r,c) per pixel (r,c)
 Calculation of the mean value of the four directional quadratic grey value
difference V0, V45, V90, V135 in a windows of size K / L.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 4

7.2 Extraction of feature points (2)

 Interest value V is calculated from:

V = min{ V0, V45, V90, V135 }
 A pixel is counted as feature point, if V is greater than an given
threshold t

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 5

7.2 Extraction of feature points (3)

 Determination of threshold t for V:
 P.e. from relative cummulative
histogram s(V) of all values V,
calculated for the complete image window
(p.e. the best 30% of all points: α = 0.3 ⇒ t = s−1 (1−α )
 If the points should be distributed homogeneous over the image:
Kachelung des Bildes und Anwendung des Schwellwertes pro Kachel
 Requirement for:
 Assignment of homologous („equal“) points to determine the relative
orientation of image pairs for 3d reconstruction from stereo images
 Object localization
 Determination of center of gravity of rings, discs, reticles …
 Application example: Identification of homologous points in
photogrammetry to determine the relative orientation of image pairs in
images blocks.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 6

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 7

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 8

 STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 9

7.3 Extraction of lines and edges (1)

 Lines: elongated bright or dark image structures
 Edges: linear image structures with high grey value changes across the
edge direction
 Image edges appear as bright line in the image of the gradient
strenght
 An algorithm for line extraction is also capable for edge extraction, if it
is applied on the gradient image.
 Requirements:
 High completeness and correctness of the extraction
 High geometric accuracy (Subpixel accuracy)
 Few parameters; their influence on the extraction should be easy to
understand.
 Topologic correct data structure (Contours or polygons with unique
identification of end and cross points)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 10

7.3 Extraction of lines and edges (2)
 Example: Line extraction by Busch
 Model: several line points show big curvature across the line direction.
 Method: local, differential-geometric explication of the image function
(see chapter „low-level models “) for describing the direction and
curvature of lines
 Procedure:
 Smoothing of the image function b(r,c) with Gauß- (Binomial-) filter h(k,l) to
suppress noise (see chapter 3.2.2.3)
 Calculation of the second derivate of the smoothed image (see chapter
3.2.2.5 )

b  f h

d2b d2b db db
b rr  b cc  b rc 
dr 2 dc 2 dr dc

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 11

7.3 Extraction of lines and edges (3)

 Determination of direction (r,c) across the line direction in every pixel (see
differential geometry)

 1 b  r, c  
 arctan rc für b cc  r, c   b rr  r, c  
 2 t  r, c  
  r, c    
 1 arctan b rc  r, c    für bcc  r, c   b rr  r, c  
2 t  r, c  2 
 
mit t  r, c  
1
2
 bcc  r, c   brr  r, c  
 Detection of line points with adaptation of a 1d parable in direction  and
determination of the subpixel accurate position (see chapter 4.3.1.1). The
curvature  (2nd derivation) is a degree for the conspicuity of the line at
this position.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 12

7.3 Extraction of lines and edges (4)
 Selection of conspicuous line points with threshold, related to  (better:
Hysteretic threshold, see chapter 4.1.4).
 Generation of contours by connecting neighboring line points (see chapter
6.1)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 13

7.3 Extraction of lines and edges (5)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 14

7.3 Extraction of lines and edges (6)

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 15

7.4 Extraction of regions (1)

 Search for homogeneous image regions:
 quasi identical definition to segmentation: homogeneous grey value,
texture or color or ...
 That means, every segmentation algorithm whose result consists of
regions, us usable (especially chapter „Pixel oriented procedures“
and „Regionbased procedures“)
 Pre-processing steps to make image region homegeneous before
segmentation.

Example: Texture based plane extraction

 Goal: Algorithm should distinguish strong textures areas, that means that
these areas should be homogeneous bright, the other areas dark
 Model: strong textured areas show high standard deviation of the grey
values in ther local neighborhood; weak textured region show a low
standard deviation.

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 16

7.4 Extraction of regions (2)
 Method:
 Calculation of the standard deviation ( see chapter 3) in a pre-defined
neighborhood of a pixel, p.e. within an window of specific size
 Moving the mask over the hole image und calculation of the standard deviation
for all pixels, that means a image of local standard deviations is generated

 Smoothing o the image by median filtering to distinguish homogeneous regions

 Segmentation of the filtered image with histogram based threshold, that means
no additional parameter necessary
 If applicable, post-processing of the segmented region with morphologic
operations

STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 17

7.4 Simultaneous extraction of forest and open areas

a) image b) Image of local standard c) Median filtering (19x19 mask)

deviation ( 9x9 mask)

d) Threshold t from histogram e) Segmented regions (from f) Post-processing: automatic

minima of the median image (here median image) overlayed with the detection of tree crowns in textured
t = 26) original image regions
STILLA & HOEGNER (2014-10-14) Image Processing [tum_ipc_14_07] 18